GZip compression

[thepisu]

What about adding an option to enable GZIP compression of XML output? It would be useful for very large sitemaps..

And, this seems to be very easy.

File xmap.php, line 67:

Code: Select all


if ( $view=='xml' ) {
   ob_start("ob_gzhandler");
   header('Content-Encoding: gzip');
   Header("Content-type: text/xml; charset=UTF-8");
   Header("Content-encoding: UTF-8");
}


Note the first 2 lines of IF block, this is my addition.


Or.. maybe can be used the DoGzip Joomla function:

Code: Select all


/**
* Perform GZIP
*/
function doGzip() {
   global $do_gzip_compress;
   if ( $do_gzip_compress ) {
      /**
      *Borrowed from php.net!
      */
      $gzip_contents = ob_get_contents();
      ob_end_clean();

      $gzip_size = strlen($gzip_contents);
      $gzip_crc = crc32($gzip_contents);

      $gzip_contents = gzcompress($gzip_contents, 9);
      $gzip_contents = substr($gzip_contents, 0, strlen($gzip_contents) - 4);

      echo "x1fx8bx08x00x00x00x00x00";
      echo $gzip_contents;
      echo pack('V', $gzip_crc);
      echo pack('V', $gzip_size);
   } else {
      ob_end_flush();
   }
}


What about this? Actually, there is no free Joomla component that is able to output gzip-compressed sitemap!

[guilleva]

Hi Pisu, it looks great, but I'm not sure if that's the best for large sitemaps because it increases the memory usage. In a previous version I included an instruction to avoid the output buffering, so each link is sent to the browser as it is generated and not stored in memory. In Joomap, every link is stored on memory and printed at the end of the generation. So at some point it uses memory for a big array plus the output buffering, Xmaps avoids both.

Anyway, I think it would be really great to have that option available and let the people test it and use it if suitable for them!

Thansk again!

[thepisu]

My code had problems, specially does not work if Joomla's GZIP is activated.. :(

Now I have found the correct solution, using Joomla's GZIP functions.

Leave file xmap.php as original (no lines ob_start("ob_gzhandler"); header('Content-Encoding: gzip')), and modify xmap.xml.php instead; in function "startOutput", add initGzip();, just after the @ob_end_clean(); line:

Code: Select all


   function startOutput( &$menus, &$config ) {
      global  $mosConfig_live_site;
      @ob_end_clean();
      initGzip();
      header('Content-type: text/xml; charset=utf-8');
      echo '<?xml version="1.0" encoding="UTF-8"?>'."
";
      if (!$config->exclude_xsl) {
         echo '<?xml-stylesheet type="text/xsl" href="'. $mosConfig_live_site.'/index2.php?option=com_xmap&view=xslfile"?>'."
";
      }
      echo '<urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">'."
";
   }


In function endOutput, just add doGzip():

Code: Select all


   function endOutput( &$menus ) {
      echo "</urlset>
";
      doGzip();
   }


And, in this way everything seems to work, with Joomla's GZIP activated. And, it's also nicer.

Probably you're right for the memory usage... But, I'm managing a shop site with thousands of products, and the sitemap (without GZIP) was 3 MB: what a bandwith usage! With GZIP compression, the size was drastically reduced: to about 65 kb!