| commit | 9788be32dda3deff06be746c15869ae23b859a79 | [log] [tgz] |
|---|---|---|
| author | Jeff Kaufman <jefftk@google.com> | Fri Apr 12 17:47:39 2013 -0400 |
| committer | Jeff Kaufman <jefftk@google.com> | Fri Apr 12 17:47:39 2013 -0400 |
| tree | 1987eabba94f82f76c720df9f74d68cd10a0ee99 | |
| parent | 572e6aaac0d5c85999f7f3edb8b5a11f21189d39 [diff] |
missing comma
This is the nginx port of mod_pagespeed.
ngx_pagespeed is alpha. If you are interested in test-driving the module, or contributing to the project, see below. For feedback, questions, and to follow the progress of the project:
The goal of ngx_pagespeed is to speed up your site and reduce page load time by automatically applying web performance best practices to pages and associated assets (CSS, JavaScript, images) without requiring that you modify your existing content or workflow. Features include:
To see ngx_pagespeed in action, with example pages for each of the optimizations, see our demonstration site.
Because nginx does not support dynamic loading of modules, you need to add ngx_pagespeed as a build-time dependency.
# These are for RedHat, CentOS, and Fedora. $ sudo yum install git gcc-c++ pcre-dev pcre-devel zlib-devel make # These are for Debian. Ubuntu will be similar. $ sudo apt-get install git-core build-essential zlib1g-dev libpcre3 libpcre3-dev
$ cd ~ $ git clone https://github.com/pagespeed/ngx_pagespeed.git
$ # check http://nginx.org/en/download.html for the latest version $ wget http://nginx.org/download/nginx-1.3.15.tar.gz $ tar -xvzf nginx-1.3.15.tar.gz $ cd nginx-1.3.15/ $ ./configure --add-module=$HOME/ngx_pagespeed $ make install
If make fails with unknown type name ‘off64_t’, add --with-cc-opt='-DLINUX=2 -D_REENTRANT -D_LARGEFILE64_SOURCE -march=i686 -pthread' to ./configure and try to make again.
If configure fails with checking for psol ... not found then open objs/autoconf.err and search for psol.
If it‘s not clear what’s wrong from the error message, then send it to the mailing list and we'll have a look at it.
First build mod_pagespeed against the current revision we work at:
$ mkdir -p ~/bin $ cd ~/bin $ svn co http://src.chromium.org/svn/trunk/tools/depot_tools $ export PATH=$PATH:~/bin/depot_tools $ mkdir ~/mod_pagespeed $ cd ~/mod_pagespeed $ gclient config http://modpagespeed.googlecode.com/svn/trunk/src $ gclient sync --revision src@2748 --force --jobs=1 $ make AR.host="$PWD/build/wrappers/ar.sh" \ AR.target="$PWD/build/wrappers/ar.sh" \ BUILDTYPE=Release \ mod_pagespeed_test pagespeed_automatic_test
(See mod_pagespeed: build from source if you run into trouble, or ask for help on the mailing list.)
Then build the pagespeed optimization library:
$ cd ~/mod_pagespeed/src/net/instaweb/automatic $ make AR.host="$PWD/../../../build/wrappers/ar.sh" \ AR.target="$PWD/../../../build/wrappers/ar.sh" \ all
While make all will always report an error, as long as it creates pagespeed_automatic.a you have what you need.
Check out ngx_pagespeed:
$ cd ~ $ git clone https://github.com/pagespeed/ngx_pagespeed.git
Download and build nginx:
$ # check http://nginx.org/en/download.html for the latest version $ wget http://nginx.org/download/nginx-1.2.6.tar.gz $ tar -xvzf nginx-1.2.6.tar.gz $ cd nginx-1.2.6/ $ MOD_PAGESPEED_DIR="$HOME/mod_pagespeed/src" ./configure \ --add-module=$HOME/ngx_pagespeed $ make install
This assumes you put everything in your home directory; if not, change paths appropriately. All paths need to be absolute.
For a debug build, remove the BUILDTYPE=Release option when running make mod_pagespeed_test pagespeed_automatic_test and add the flag --with-debug to ./configure --add-module=....
If you‘re testing this and don’t want to install this as root, which is a good idea, you can use --prefix, as in ./configure --add-module=... --prefix=$HOME/nginx and then nginx will install to a single directory inside your home directory.
Tengine is an Nginx distribution that supports dynamically loaded modules. You can add ngx_pagespeed to an existing Tengine install without recompiling Tengine. First follow one of the two installation methods above until you get to the “Download and build nginx” section. Then run:
# This might be /usr/local/tengine, depending on your configuration. $ cd /path/to/tengine/sbin/ $ ./dso_tool --add-module=/path/to/ngx_pagespeed
This will prepare a dynamically loadable module out of ngx_pagespeed. To check that it worked you can verify that /path/to/tengine/modules/ contains an ngx_pagespeed.so.
You need to tell tengine to load this module. Before continuing with “How to use” below, add this to the top of your configuration:
dso {
load ngx_pagespeed.so;
}
In your nginx.conf, add to the main or server block:
pagespeed on; # needs to exist and be writable by nginx pagespeed FileCachePath /var/ngx_pagespeed_cache;
In every server block where pagespeed is enabled add:
# This is a temporary workaround that ensures requests for pagespeed
# optimized resources go to the pagespeed handler.
location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" { }
location ~ "^/ngx_pagespeed_static/" { }
location ~ "^/ngx_pagespeed_beacon$" { }
location /ngx_pagespeed_statistics {
allow 127.0.0.1;
deny all;
}
location /ngx_pagespeed_message {
allow 127.0.0.1;
deny all;
}
If you're proxying, you may want to strip off the Accept-Encoding header. This makes the upstream send html uncompressed to pagespeed. Because pagespeed needs the html uncompressed in order to rewrite it, if you leave the Accept-Encoding header on, the upstream will probably send a gzipped response that pagespeed has to ungzip, process, and then gzip. If your proxy is out over the open internet this may be a good idea to save bandwidth, but in the common case where your proxy is on a nearby server or even the same server gzipping will just slow things down:
proxy_set_header Accept-Encoding "";
To confirm that the module is loaded, fetch a page and check that you see the X-Page-Speed header:
$ curl -s -D- 'http://localhost:8050/some_page/' | grep X-Page-Speed X-Page-Speed: 1.4.0.0-2729
Looking at the source of a few pages you should see various changes, such as urls being replaced with new ones like yellow.css.pagespeed.ce.lzJ8VcVi1l.css.
Often pagespeed needs to request urls referenced from other files in order to optimize them. To do this it uses a fetcher. By default it uses the same fetcher mod_pagespeed does, serf, but it also has an experimental fetcher that avoids the need for a separate thread by using native nginx events. In initial testing this fetcher is about 10% faster. To use it, put in your http block:
pagespeed UseNativeFetcher on; resolver 8.8.8.8;
Your DNS resolver doesn‘t have to be 8.8.8.8; any domain name server your server has access to will work. If you don’t specify a DNS resolver, pagespeed will still work but will be limited to fetching only from IP addresses.
PageSpeed can use a beacon to track load times. By default PageSpeed sends beacons to /ngx_pagespeed_beacon on your site, but you can change this:
pagespeed BeaconUrl /path/to/beacon;
If you do, you also need to change the regexp above from location ~ "^/ngx_pagespeed_beacon$" { } to location ~ "^/path/to/beacon$" { }.
As with ModPagespeedBeaconUrl you can set your beacons to go to another site by specifying a full path:
pagespeed BeaconUrl http://thirdpartyanalytics.example.com/my/beacon;
The generic Pagespeed system test is ported, and all but three tests pass. To run it you need to first build nginx. You also need to check out mod_pagespeed, but we can take a shortcut and do this the easy way, without gyp, because we don't need any dependencies:
$ svn checkout https://modpagespeed.googlecode.com/svn/trunk/ mod_pagespeed
Then run:
test/run_tests.sh \ primary_port \ secondary_port \ mod_pagespeed_dir \ nginx_executable_path
For example:
$ test/run_tests.sh 8050 8051 /path/to/mod_pagespeed \ /path/to/sbin/nginx
All of these paths need to be absolute.
This should print out a lot of lines like:
TEST: Make sure 404s aren't rewritten
check_not fgrep /mod_pagespeed_beacon /dev/fd/63
and then eventually:
Failing Tests: In-place resource optimization In-place resource optimization In-place resource optimization convert_meta_tags insert_dns_prefetch insert_dns_prefetch FAIL. With serf fetcher setup.
Each of these failed tests is a known issue:
If it fails with:
TEST: PHP is enabled. ... in 'PHP is enabled.' FAIL.
the problem is that the test expects a php server to be running on port 9000. One way to do that is:
$ sudo apt-get install php5-fpm $ sudo service php5-fpm restart
Alternatively you can do:
$ sudo apt-get install php5-cgi $ php-cgi -b 127.0.0.1:9000 &
If it fails with some other error, that's a problem, and it would be helpful for you to submit a bug.
Log files are in test/tmp/error.log and test/tmp/access.log.
Start an memcached server:
$ memcached -p 11211
In ngx_pagespeed/test/pagespeed_test.conf.template uncomment:
pagespeed MemcachedServers "localhost:11211"; pagespeed MemcachedThreads 1;
Then run the system test as above.
If you set the environment variable USE_VALGRIND=true then the tests will run with valgrind:
USE_VALGRIND=true test/nginx_system_test.sh ...
Most mod_pagespeed configuration directives work in ngx_pagespeed after a small adjustment: replace ‘“ModPagespeed”’ with ‘"pagespeed "’:
mod_pagespeed.conf: ModPagespeedEnableFilters collapse_whitespace,add_instrumentation ModPagespeedRunExperiment on ModPagespeedExperimentSpec id=3;percent=50;default ModPagespeedExperimentSpec id=4;percent=50 ngx_pagespeed.conf: pagespeed EnableFilters collapse_whitespace,add_instrumentation; pagespeed RunExperiment on; pagespeed ExperimentSpec "id=3;percent=50;default"; pagespeed ExperimentSpec "id=4;percent=50";
See: https://github.com/pagespeed/ngx_pagespeed/wiki/Building-Release-Binaries
If you just want to run ngx_pagespeed you don't need this. This is documentation on how the psol/ directory was created and is maintained.