doc/filter-canonicalize-js.html - incubator-pagespeed-website - Git at Google

 <!--
 Licensed to the Apache Software Foundation (ASF) under one
 or more contributor license agreements.  See the NOTICE file
 distributed with this work for additional information
 regarding copyright ownership.  The ASF licenses this file
 to you under the Apache License, Version 2.0 (the
 "License"); you may not use this file except in compliance
 with the License.  You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing,
 software distributed under the License is distributed on an
 "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 KIND, either express or implied.  See the License for the
 specific language governing permissions and limitations
 under the License.
 -->

 <html>
   <head>
     <meta name="viewport" content="width=device-width, initial-scale=1">
     <title>Canonicalize JavaScript Libraries</title>
     <link rel="stylesheet" href="doc.css">
   </head>
   <body>
 <!--#include virtual="_header.html" -->


   <div id=content>
 <h1>Canonicalize JavaScript Libraries</h1>
 <h2>Configuration</h2>
 <p>
 The 'Canonicalize JavaScript Libraries' filter is enabled by specifying:

 <dl>
   <dt>Apache:<dd><pre class="prettyprint"
      >ModPagespeedEnableFilters canonicalize_javascript_libraries</pre>
   <dt>Nginx:<dd><pre class="prettyprint">
 pagespeed EnableFilters canonicalize_javascript_libraries;

 </pre>
 </dl>
 <p>
 in the configuration file.
 </p>

 <h2>Description</h2>
 <p>
 This filter identifies popular JavaScript libraries that can be replaced with
 ones hosted for free by a JavaScript library hosting service &mdash; by default
 the <a href="/speed/libraries/">Google Hosted Libraries</a>. This has several
 benefits:
 <ul>
   <li>Most important, first-time site visitors can benefit from browser caching,
 since they may have visited other sites making use of the same service to obtain
 the libraries.</li>
   <li>The JavaScript hosting service acts as a content delivery network for the
 hosted files, reducing load on the server and improving browser load times.</li>
   <li>There are no charges for the resulting use of bandwidth by site
 visitors.</li>
   <li>The hosted versions of library code are generally optimized with
 third-party minification tools. These optimizations can make use of
 library-specific annotations or minification settings that aren't portable to
 arbitrary JavaScript code, so the libraries benefit from more aggressive
 optimization than can be provided by PageSpeed.</li>
 </ul>
 <p>
 In Apache the default set of libraries can be found in
 the <code><a href="https://github.com/apache/incubator-pagespeed-mod/blob/master/net/instaweb/genfiles/conf/pagespeed_libraries.conf">pagespeed_libraries.conf</a></code>
 file, which is loaded along with <code>pagespeed.conf</code> when Apache starts
 up. It contains signatures for all the <a href="/speed/libraries/">Google Hosted
 Libraries</a>.  In Nginx you need to
 convert <code>pagespeed_libraries.conf</code> from Apache-format to Nginx
 format:
 </p>
 <pre class="prettyprint lang-sh">
 $ scripts/pagespeed_libraries_generator.sh > ~/pagespeed_libraries.conf
 $ sudo mv ~/pagespeed_libraries.conf /path/to/nginx/configuration_files/
 </pre>
 <p>
 You also need to include it in your Nginx configuration by reference:
 </p>
 <pre class="prettyprint lang-sh">
 include pagespeed_libraries.conf;
 </pre>
 <p>
 Don't edit <code>pagespeed_libraries.conf</code>.  Local edits will keep you
 from being able to update it when you update PageSpeed.  Rather than editing it
 you should add additional libraries to your main configuration file:
 <dl>
   <dt>Apache:<dd><pre class="prettyprint">
 ModPagespeedLibrary 43 1o978_K0_LNE5_ystNklf \
    //www.modpagespeed.com/rewrite_javascript.js</pre>
   <dt>Nginx:<dd><pre class="prettyprint">
 pagespeed Library 43 1o978_K0_LNE5_ystNklf
    //www.modpagespeed.com/rewrite_javascript.js;</pre>
 </dl>
 <p>
 The general format of these entries is:
 <dl>
   <dt>Apache:<dd><pre class="prettyprint"
      >ModPagespeedLibrary bytes MD5 canonical_url</pre>
   <dt>Nginx:<dd><pre class="prettyprint"
      >pagespeed Library bytes MD5 canonical_url;</pre>
 </dl>
 <p>
 Here <code>bytes</code> is the size in bytes of the library <em>after</em>
 minification by PageSpeed, and <code>MD5</code> is the MD5 hash of the
 library after minification. Minification controls for differences in whitespace
 that may occur when the same script is obtained from different
 sources. The <code>canonical_url</code> is the hosting service URL used to
 replace occurrences of the script. Note that the canonical URL in the above
 example is protocol-relative; this means the data will be fetched using the same
 protocol (<code>http</code> or <code>https</code>) as the containing
 page. Because older browsers don't handle protocol-relative URLs reliably,
 PageSpeed resolves a protocol-relative library URL to an absolute URL based
 on the protocol of the containing page. Do not use <code>http</code> canonical
 URLs in configurations that may serve content over <code>https</code>, or the
 rewritten pages will expose your site to attack and trigger a mixed-content
 warning in the browser. Similarly, avoid using <code>https</code> URLs unless
 you know that the resulting library will eventually be fetched from a secure
 page, as SSL negotiation adds overhead to the initial request.
 </p>
 <p>
 Additional library configuration metadata can be generated with the help of
 the <code>pagespeed_js_minify</code> utility installed along with PageSpeed.
 To use this utility, you will need a local copy of the JavaScript code that you
 wish to replace.  If this is stored in <code>library.js</code>, you can generate
 <code>bytes</code> and <code>MD5</code> as follows:
 <dl>
   <dt>Apache:<dd><pre class="prettyprint"
      >$ pagespeed_js_minify --print_size_and_hash library.js</pre>
   <dt>Nginx:<dd><pre class="prettyprint"
      >$ cd /path/to/psol/lib/Release/linux/ia32/
       $ pagespeed_js_minify --print_size_and_hash library.js</pre>
 </dl>
 <p>
 If you're using the <a href="filter-js-minify#new-minifier">new
 javascript minifier</a>, add the <code>--use_experimental_minifier</code>
 argument to <code>pagespeed_js_minify</code>. If you're using the old minifier,
 add <code>--nouse_experimental_minifier</code>. (As of 1.10.33.0,
 <code>--use_experimental_minifier</code> is default. Previously,
 <code>--nouse_experimental_minifier</code> was.)
 The default <code>pagespeed_libraries.conf</code> includes hashes for both
 the old and new minifiers.
 </p>
 <p>
 This filter is based on the best practices of
 <a target="_blank" href="https://developers.google.com/speed/docs/best-practices/caching#LeverageBrowserCaching">
 optimizing browser caching</a>
 and <a target="_blank" href="https://developers.google.com/speed/docs/best-practices/payload#MinifyJS">reducing payload
 size</a>.
 </p>

 <h2>Operation</h2>
 <p>
 In order to identify a library and canonicalize its URL, PageSpeed must of
 course be able to fetch the JavaScript code from the URL on the original page.
 Because library canonicalization identifies libraries solely by their size and
 hash signature, it is not necessary to authorize PageSpeed to fetch content
 from the domain hosting the canonical content itself.  This means that it is
 safe to use this filter behind a reverse proxy or in other situations where
 network access by PageSpeed is deliberately restricted.  Browsers visiting
 the site will fetch the content from the canonical URL, but PageSpeed itself
 does not need to do so.
 </p>

 <h3>Examples</h3>
 <p>
 You can see the filter in action at <code>www.modpagespeed.com</code> on this
 <a href="https://www.modpagespeed.com/examples/canonicalize_javascript_libraries.html?ModPagespeed=on&amp;ModPagespeedFilters=canonicalize_javascript_libraries">example</a>.
 </p>
 <p>
 If the HTML document looks like this:
 </p>
 <pre class="prettyprint">
 &lt;html&gt;
   &lt;head&gt;
     &lt;script src="jquery_1_8.js"&gt;
     &lt;/script&gt;
     &lt;script src="a.js"&gt;
     &lt;/script&gt;
     &lt;script src="b.js"&gt;
     &lt;/script&gt;
   &lt;/head&gt;
   &lt;body&gt;
   ...
   &lt;/body&gt;
 &lt;/html&gt;
 </pre>
 <p>
 Then, assuming <code>jquery_1_8.js</code> was an unminified copy of the jquery
 library and <code>a.js</code> and <code>b.js</code> contained site-specific code
 that made use of jquery, the page would be rewritten as follows:
 </p>
 <pre class="prettyprint">
 &lt;html&gt;
   &lt;head&gt;
     &lt;script src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js"&gt;
     &lt;/script&gt;
     &lt;script src="a.js"&gt;
     &lt;/script&gt;
     &lt;script src="b.js"&gt;
     &lt;/script&gt;
   &lt;/head&gt;
   &lt;body&gt;
   ...
   &lt;/body&gt;
 &lt;/html&gt;
 </pre>
 <p>
 The library URL has been replaced by a reference to the canonical minified
 version hosted on <code>ajax.googleapis.com</code>.  Note that canonical
 libraries do not participate in most other JavaScript optimizations.  For
 example, if <a href="filter-js-combine">Combine JavaScript</a> is also enabled,
 the above page will be rewritten as follows:
 </p>
 <pre class="prettyprint">
 &lt;html&gt;
   &lt;head&gt;
     &lt;script src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js"&gt;
     &lt;/script&gt;
     &lt;script src="http://www.example.com/a.js+b.js.pagespeed.jc.zYiUaxFS8I.js"&gt;
     &lt;/script&gt;
   &lt;/head&gt;
   &lt;body&gt;
   ...
   &lt;/body&gt;
 &lt;/html&gt;
 </pre>
 <p>
 The canonical library is <em>not</em> combined with the other two JavaScript
 files, since this would lose the bandwidth and caching benefits of fetching it
 from the canonical URL.
 </p>
 <p>
 If <a href="filter-js-defer">defer_javascript</a> is enabled, <em>and</em> library
 is <em>not</em> tagged with <code>data-pagespeed-no-defer</code>,
 the canonicalized library is deferred.
 </p>

 <h2>Requirements</h2>
 <p>
 Only complete, unmodified libraries referenced by <code>&lt;script&gt;</code>
 tags in the HTML will be rewritten. Libraries that are loaded by other means
 (for example by injecting a loader script) or that have been modified will not
 be canonicalized.
 </p>

 <h2>Risks</h2>
 <p>
 You must ensure that you abide by the terms of service of the providers of the
 canonical content before enabling canonicalization. The terms of service for the
 default configuration can be found
 at <a href="/speed/libraries/terms">https://developers.google.com/speed/libraries/terms</a>.
 </p>
 <p>
 The canonical URL refers to a third-party domain; this can cause additional DNS
 lookup latency the first time a library is loaded.  This is mitigated by the
 fact that the canonical copy of the data is shared among multiple sites.
 </p>
 <p>
 The initial request for a canonical URL will contain a <code>Referer:</code>
 header with the URL of the referring page.  This permits the host of the
 canonical content to see a subset of traffic to your site (the first load of a
 page on your site that contains an identified library by a browser that does not
 already have that library in its cache). The provider should describe how this
 data is used in its terms of service. The terms of service for the default
 configuration can be found at
 <a href="/speed/libraries/terms">https://developers.google.com/speed/libraries/terms</a>.
 Again, this risk is mitigated by the fact that canonical libraries are shared
 among multiple sites; a popular library is likely to already reside in the
 browser cache.
 </p>
 <p>
 Sites serving content on both <code>http</code> and <code>https</code> URLs must
 use protocol-relative canonical URLs as shown <a href="#sample">above</a>.
 Fetching a library insecurely from a secure page exposes a site to attack.
 Fetching a library securely from an ordinary page can increase load time due to
 SSL overheads.
 </p>
 <p>
 It is theoretically possible to craft a JavaScript file whose minified size and
 hash exactly match that of a canonical library, but whose code behaves
 differently.  In such a case the library will be replaced with the canonical
 (widely-used) library.  This will break the page that contains the reference to
 the crafted JavaScript.
 </p>
   </div>
   <!--#include virtual="_footer.html" -->
   </body>
 </html>
	<!--
	Licensed to the Apache Software Foundation (ASF) under one
	or more contributor license agreements. See the NOTICE file
	distributed with this work for additional information
	regarding copyright ownership. The ASF licenses this file
	to you under the Apache License, Version 2.0 (the
	"License"); you may not use this file except in compliance
	with the License. You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

	Unless required by applicable law or agreed to in writing,
	software distributed under the License is distributed on an
	"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	KIND, either express or implied. See the License for the
	specific language governing permissions and limitations
	under the License.
	-->

	<html>
	<head>
	<meta name="viewport" content="width=device-width, initial-scale=1">
	<title>Canonicalize JavaScript Libraries</title>
	<link rel="stylesheet" href="doc.css">
	</head>
	<body>
	<!--#include virtual="_header.html" -->


	<div id=content>
	<h1>Canonicalize JavaScript Libraries</h1>
	<h2>Configuration</h2>
	<p>
	The 'Canonicalize JavaScript Libraries' filter is enabled by specifying:

	<dl>
	<dt>Apache:<dd><pre class="prettyprint"
	>ModPagespeedEnableFilters canonicalize_javascript_libraries</pre>
	<dt>Nginx:<dd><pre class="prettyprint">
	pagespeed EnableFilters canonicalize_javascript_libraries;

	</pre>
	</dl>
	<p>
	in the configuration file.
	</p>

	<h2>Description</h2>
	<p>
	This filter identifies popular JavaScript libraries that can be replaced with
	ones hosted for free by a JavaScript library hosting service — by default
	the <a href="/speed/libraries/">Google Hosted Libraries</a>. This has several
	benefits:
	<ul>
	<li>Most important, first-time site visitors can benefit from browser caching,
	since they may have visited other sites making use of the same service to obtain
	the libraries.</li>
	<li>The JavaScript hosting service acts as a content delivery network for the
	hosted files, reducing load on the server and improving browser load times.</li>
	<li>There are no charges for the resulting use of bandwidth by site
	visitors.</li>
	<li>The hosted versions of library code are generally optimized with
	third-party minification tools. These optimizations can make use of
	library-specific annotations or minification settings that aren't portable to
	arbitrary JavaScript code, so the libraries benefit from more aggressive
	optimization than can be provided by PageSpeed.</li>
	</ul>
	<p>
	In Apache the default set of libraries can be found in
	the <code><a href="https://github.com/apache/incubator-pagespeed-mod/blob/master/net/instaweb/genfiles/conf/pagespeed_libraries.conf">pagespeed_libraries.conf</a></code>
	file, which is loaded along with <code>pagespeed.conf</code> when Apache starts
	up. It contains signatures for all the <a href="/speed/libraries/">Google Hosted
	Libraries</a>. In Nginx you need to
	convert <code>pagespeed_libraries.conf</code> from Apache-format to Nginx
	format:
	</p>
	<pre class="prettyprint lang-sh">
	$ scripts/pagespeed_libraries_generator.sh > ~/pagespeed_libraries.conf
	$ sudo mv ~/pagespeed_libraries.conf /path/to/nginx/configuration_files/
	</pre>
	<p>
	You also need to include it in your Nginx configuration by reference:
	</p>
	<pre class="prettyprint lang-sh">
	include pagespeed_libraries.conf;
	</pre>
	<p>
	Don't edit <code>pagespeed_libraries.conf</code>. Local edits will keep you
	from being able to update it when you update PageSpeed. Rather than editing it
	you should add additional libraries to your main configuration file:
	<dl>
	<dt>Apache:<dd><pre class="prettyprint">
	ModPagespeedLibrary 43 1o978_K0_LNE5_ystNklf \
	//www.modpagespeed.com/rewrite_javascript.js</pre>
	<dt>Nginx:<dd><pre class="prettyprint">
	pagespeed Library 43 1o978_K0_LNE5_ystNklf
	//www.modpagespeed.com/rewrite_javascript.js;</pre>
	</dl>
	<p>
	The general format of these entries is:
	<dl>
	<dt>Apache:<dd><pre class="prettyprint"
	>ModPagespeedLibrary bytes MD5 canonical_url</pre>
	<dt>Nginx:<dd><pre class="prettyprint"
	>pagespeed Library bytes MD5 canonical_url;</pre>
	</dl>
	<p>
	Here <code>bytes</code> is the size in bytes of the library <em>after</em>
	minification by PageSpeed, and <code>MD5</code> is the MD5 hash of the
	library after minification. Minification controls for differences in whitespace
	that may occur when the same script is obtained from different
	sources. The <code>canonical_url</code> is the hosting service URL used to
	replace occurrences of the script. Note that the canonical URL in the above
	example is protocol-relative; this means the data will be fetched using the same
	protocol (<code>http</code> or <code>https</code>) as the containing
	page. Because older browsers don't handle protocol-relative URLs reliably,
	PageSpeed resolves a protocol-relative library URL to an absolute URL based
	on the protocol of the containing page. Do not use <code>http</code> canonical
	URLs in configurations that may serve content over <code>https</code>, or the
	rewritten pages will expose your site to attack and trigger a mixed-content
	warning in the browser. Similarly, avoid using <code>https</code> URLs unless
	you know that the resulting library will eventually be fetched from a secure
	page, as SSL negotiation adds overhead to the initial request.
	</p>
	<p>
	Additional library configuration metadata can be generated with the help of
	the <code>pagespeed_js_minify</code> utility installed along with PageSpeed.
	To use this utility, you will need a local copy of the JavaScript code that you
	wish to replace. If this is stored in <code>library.js</code>, you can generate
	<code>bytes</code> and <code>MD5</code> as follows:
	<dl>
	<dt>Apache:<dd><pre class="prettyprint"
	>$ pagespeed_js_minify --print_size_and_hash library.js</pre>
	<dt>Nginx:<dd><pre class="prettyprint"
	>$ cd /path/to/psol/lib/Release/linux/ia32/
	$ pagespeed_js_minify --print_size_and_hash library.js</pre>
	</dl>
	<p>
	If you're using the <a href="filter-js-minify#new-minifier">new
	javascript minifier</a>, add the <code>--use_experimental_minifier</code>
	argument to <code>pagespeed_js_minify</code>. If you're using the old minifier,
	add <code>--nouse_experimental_minifier</code>. (As of 1.10.33.0,
	<code>--use_experimental_minifier</code> is default. Previously,
	<code>--nouse_experimental_minifier</code> was.)
	The default <code>pagespeed_libraries.conf</code> includes hashes for both
	the old and new minifiers.
	</p>
	<p>
	This filter is based on the best practices of
	<a target="_blank" href="https://developers.google.com/speed/docs/best-practices/caching#LeverageBrowserCaching">
	optimizing browser caching</a>
	and <a target="_blank" href="https://developers.google.com/speed/docs/best-practices/payload#MinifyJS">reducing payload
	size</a>.
	</p>

	<h2>Operation</h2>
	<p>
	In order to identify a library and canonicalize its URL, PageSpeed must of
	course be able to fetch the JavaScript code from the URL on the original page.
	Because library canonicalization identifies libraries solely by their size and
	hash signature, it is not necessary to authorize PageSpeed to fetch content
	from the domain hosting the canonical content itself. This means that it is
	safe to use this filter behind a reverse proxy or in other situations where
	network access by PageSpeed is deliberately restricted. Browsers visiting
	the site will fetch the content from the canonical URL, but PageSpeed itself
	does not need to do so.
	</p>

	<h3>Examples</h3>
	<p>
	You can see the filter in action at <code>www.modpagespeed.com</code> on this
	<a href="https://www.modpagespeed.com/examples/canonicalize_javascript_libraries.html?ModPagespeed=on&ModPagespeedFilters=canonicalize_javascript_libraries">example</a>.
	</p>
	<p>
	If the HTML document looks like this:
	</p>
	<pre class="prettyprint">
	<html>
	<head>
	<script src="jquery_1_8.js">
	</script>
	<script src="a.js">
	</script>
	<script src="b.js">
	</script>
	</head>
	<body>
	...
	</body>
	</html>
	</pre>
	<p>
	Then, assuming <code>jquery_1_8.js</code> was an unminified copy of the jquery
	library and <code>a.js</code> and <code>b.js</code> contained site-specific code
	that made use of jquery, the page would be rewritten as follows:
	</p>
	<pre class="prettyprint">
	<html>
	<head>
	<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js">
	</script>
	<script src="a.js">
	</script>
	<script src="b.js">
	</script>
	</head>
	<body>
	...
	</body>
	</html>
	</pre>
	<p>
	The library URL has been replaced by a reference to the canonical minified
	version hosted on <code>ajax.googleapis.com</code>. Note that canonical
	libraries do not participate in most other JavaScript optimizations. For
	example, if <a href="filter-js-combine">Combine JavaScript</a> is also enabled,
	the above page will be rewritten as follows:
	</p>
	<pre class="prettyprint">
	<html>
	<head>
	<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js">
	</script>
	<script src="http://www.example.com/a.js+b.js.pagespeed.jc.zYiUaxFS8I.js">
	</script>
	</head>
	<body>
	...
	</body>
	</html>
	</pre>
	<p>
	The canonical library is <em>not</em> combined with the other two JavaScript
	files, since this would lose the bandwidth and caching benefits of fetching it
	from the canonical URL.
	</p>
	<p>
	If <a href="filter-js-defer">defer_javascript</a> is enabled, <em>and</em> library
	is <em>not</em> tagged with <code>data-pagespeed-no-defer</code>,
	the canonicalized library is deferred.
	</p>

	<h2>Requirements</h2>
	<p>
	Only complete, unmodified libraries referenced by <code><script></code>
	tags in the HTML will be rewritten. Libraries that are loaded by other means
	(for example by injecting a loader script) or that have been modified will not
	be canonicalized.
	</p>

	<h2>Risks</h2>
	<p>
	You must ensure that you abide by the terms of service of the providers of the
	canonical content before enabling canonicalization. The terms of service for the
	default configuration can be found
	at <a href="/speed/libraries/terms">https://developers.google.com/speed/libraries/terms</a>.
	</p>
	<p>
	The canonical URL refers to a third-party domain; this can cause additional DNS
	lookup latency the first time a library is loaded. This is mitigated by the
	fact that the canonical copy of the data is shared among multiple sites.
	</p>
	<p>
	The initial request for a canonical URL will contain a <code>Referer:</code>
	header with the URL of the referring page. This permits the host of the
	canonical content to see a subset of traffic to your site (the first load of a
	page on your site that contains an identified library by a browser that does not
	already have that library in its cache). The provider should describe how this
	data is used in its terms of service. The terms of service for the default
	configuration can be found at
	<a href="/speed/libraries/terms">https://developers.google.com/speed/libraries/terms</a>.
	Again, this risk is mitigated by the fact that canonical libraries are shared
	among multiple sites; a popular library is likely to already reside in the
	browser cache.
	</p>
	<p>
	Sites serving content on both <code>http</code> and <code>https</code> URLs must
	use protocol-relative canonical URLs as shown <a href="#sample">above</a>.
	Fetching a library insecurely from a secure page exposes a site to attack.
	Fetching a library securely from an ordinary page can increase load time due to
	SSL overheads.
	</p>
	<p>
	It is theoretically possible to craft a JavaScript file whose minified size and
	hash exactly match that of a canonical library, but whose code behaves
	differently. In such a case the library will be replaced with the canonical
	(widely-used) library. This will break the page that contains the reference to
	the crafted JavaScript.
	</p>
	</div>
	<!--#include virtual="_footer.html" -->
	</body>
	</html>