Lunes, 20 Octubre 2014 17:52
beggarly

GZip compression

Request new features on Xmap component
Forum rules
This

GZip compression

Postby thepisu » Wed Apr 16, 2008 11:57 am

What about adding an option to enable GZIP compression of XML output? It would be useful for very large sitemaps..

And, this seems to be very easy.

File xmap.php, line 67:
Code: Select all

if ( $view=='xml' ) {
   ob_start("ob_gzhandler");
   header('Content-Encoding: gzip');
   Header("Content-type: text/xml; charset=UTF-8");
   Header("Content-encoding: UTF-8");
}

Note the first 2 lines of IF block, this is my addition.


Or.. maybe can be used the DoGzip Joomla function:
Code: Select all

/**
* Perform GZIP
*/
function doGzip() {
   global $do_gzip_compress;
   if ( $do_gzip_compress ) {
      /**
      *Borrowed from php.net!
      */
      $gzip_contents = ob_get_contents();
      ob_end_clean();

      $gzip_size = strlen($gzip_contents);
      $gzip_crc = crc32($gzip_contents);

      $gzip_contents = gzcompress($gzip_contents, 9);
      $gzip_contents = substr($gzip_contents, 0, strlen($gzip_contents) - 4);

      echo "x1fx8bx08x00x00x00x00x00";
      echo $gzip_contents;
      echo pack('V', $gzip_crc);
      echo pack('V', $gzip_size);
   } else {
      ob_end_flush();
   }
}


What about this? Actually, there is no free Joomla component that is able to output gzip-compressed sitemap!
Last edited by thepisu on Thu Jan 01, 1970 12:00 am, edited 65535 times in total.
Reason: 1
thepisu
Fresh Boarder
Fresh Boarder
 
Posts: 6
Joined: Tue Apr 08, 2008 8:11 am
Location: Italy

Re:GZip compression

Postby guilleva » Wed Apr 16, 2008 1:38 pm

Hi Pisu, it looks great, but I'm not sure if that's the best for large sitemaps because it increases the memory usage. In a previous version I included an instruction to avoid the output buffering, so each link is sent to the browser as it is generated and not stored in memory. In Joomap, every link is stored on memory and printed at the end of the generation. So at some point it uses memory for a big array plus the output buffering, Xmaps avoids both.

Anyway, I think it would be really great to have that option available and let the people test it and use it if suitable for them!

Thansk again!
User avatar
guilleva
Administrator
Administrator
 
Posts: 1527
Joined: Wed Sep 12, 2007 3:10 am
Location: San José, Costa Rica

Re:GZip compression

Postby thepisu » Fri May 23, 2008 6:52 am

My code had problems, specially does not work if Joomla's GZIP is activated.. :(

Now I have found the correct solution, using Joomla's GZIP functions.

Leave file xmap.php as original (no lines ob_start("ob_gzhandler"); header('Content-Encoding: gzip')), and modify xmap.xml.php instead; in function "startOutput", add initGzip();, just after the @ob_end_clean(); line:

Code: Select all

   function startOutput( &$menus, &$config ) {
      global  $mosConfig_live_site;
      @ob_end_clean();
      initGzip();
      header('Content-type: text/xml; charset=utf-8');
      echo '<?xml version="1.0" encoding="UTF-8"?>'."
";
      if (!$config->exclude_xsl) {
         echo '<?xml-stylesheet type="text/xsl" href="'. $mosConfig_live_site.'/index2.php?option=com_xmap&view=xslfile"?>'."
";
      }
      echo '<urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">'."
";
   }


In function endOutput, just add doGzip():

Code: Select all

   function endOutput( &$menus ) {
      echo "</urlset>
";
      doGzip();
   }


And, in this way everything seems to work, with Joomla's GZIP activated. And, it's also nicer.

Probably you're right for the memory usage... But, I'm managing a shop site with thousands of products, and the sitemap (without GZIP) was 3 MB: what a bandwith usage! With GZIP compression, the size was drastically reduced: to about 65 kb!
thepisu
Fresh Boarder
Fresh Boarder
 
Posts: 6
Joined: Tue Apr 08, 2008 8:11 am
Location: Italy


Return to New Features



Who is online

Users browsing this forum: No registered users and 2 guests

cron