Branch for 0.9.13.x releases
diff --git a/trunk/LICENSE b/trunk/LICENSE
new file mode 100644
index 0000000..9d18d4c
--- /dev/null
+++ b/trunk/LICENSE
@@ -0,0 +1,202 @@
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright 2010 Google Inc.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
diff --git a/trunk/src/DEPS b/trunk/src/DEPS
new file mode 100644
index 0000000..5adbc13
--- /dev/null
+++ b/trunk/src/DEPS
@@ -0,0 +1,212 @@
+# Copyright 2009 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+vars = {
+ "chromium_trunk": "http://src.chromium.org/svn/trunk",
+ "chromium_revision": "@55218",
+ "chromium_deps_root": "src/third_party/chromium_deps",
+
+ # We should change this back to a tag once Pagespeed cuts a new branch.
+ #"libpagespeed_trunk": "http://page-speed.googlecode.com/svn/lib/tags/lib-1.9.3.0",
+ #"libpagespeed_revision": "@head",
+ "libpagespeed_trunk": "http://page-speed.googlecode.com/svn/lib/trunk",
+ "libpagespeed_revision": "@1300",
+ "libpagespeed_deps_root": "src/third_party/libpagespeed_deps",
+
+ "mod_spdy_trunk": "http://mod-spdy.googlecode.com/svn/trunk",
+ "mod_spdy_revision": "@143",
+ "mod_spdy_deps_root": "src/third_party/mod_spdy_deps",
+
+ "serf_src": "http://serf.googlecode.com/svn/tags/0.3.1",
+ "serf_revision": "@head",
+
+ "apr_src": "http://svn.apache.org/repos/asf/apr/apr/tags/1.4.2",
+ "apr_revision": "@head",
+
+ "aprutil_src": "http://svn.apache.org/repos/asf/apr/apr-util/tags/1.3.9",
+ "aprutil_revision": "@head",
+
+ "apache_httpd_src": "http://svn.apache.org/repos/asf/httpd/httpd/tags/2.2.15",
+ "apache_httpd_revision": "@head",
+
+ "opencv_src": "https://code.ros.org/svn/opencv/tags/2.1",
+ "opencv_revision": "@head",
+
+ "gflags_root": "http://google-gflags.googlecode.com/svn/tags/gflags-1.3/src",
+ "gflags_revision": "@head",
+
+ "google_sparsehash_root": "http://google-sparsehash.googlecode.com/svn/tags/sparsehash-1.8.1/src",
+ "google_sparsehash_revision": "@head",
+}
+
+deps = {
+ # Fetch dependent DEPS so we can sync our other dependencies relative
+ # to them.
+ Var("chromium_deps_root"):
+ File(Var("chromium_trunk") + "/src/DEPS" + Var("chromium_revision")),
+
+ Var("libpagespeed_deps_root"):
+ File(Var("libpagespeed_trunk") + "/src/DEPS" +
+ Var("libpagespeed_revision")),
+
+ # Other dependencies.
+ "src/base":
+ Var("libpagespeed_trunk") + "/src/base" + Var("libpagespeed_revision"),
+
+ "src/build/temp_gyp":
+ Var("libpagespeed_trunk") + "/src/build/temp_gyp" +
+ Var("libpagespeed_revision"),
+
+ "src/build/internal": From(Var("libpagespeed_deps_root")),
+ "src/build/linux": From(Var("libpagespeed_deps_root")),
+ "src/build/mac": From(Var("libpagespeed_deps_root")),
+ "src/build/util": From(Var("libpagespeed_deps_root")),
+ "src/build/win": From(Var("libpagespeed_deps_root")),
+
+ "src/googleurl": From(Var("chromium_deps_root")),
+
+ "src/googleurl_noicu":
+ (Var("libpagespeed_trunk") + "/src/googleurl_noicu" +
+ Var("libpagespeed_revision")),
+
+ "src/testing": From(Var("libpagespeed_deps_root")),
+ "src/testing/gtest": From(Var("chromium_deps_root")),
+
+ "src/third_party/apache":
+ Var("mod_spdy_trunk") + "/src/third_party/apache" +
+ Var("mod_spdy_revision"),
+
+ "src/third_party/apache/apr/src":
+ Var("apr_src") + Var("apr_revision"),
+
+ "src/third_party/apache/aprutil/src":
+ Var("aprutil_src") + Var("aprutil_revision"),
+
+ "src/third_party/apache/httpd/src/include":
+ Var("apache_httpd_src") + "/include" + Var("apache_httpd_revision"),
+
+ "src/third_party/apache/httpd/src/os":
+ Var("apache_httpd_src") + "/os" + Var("apache_httpd_revision"),
+
+ "src/third_party/chromium/src/base": From(Var("libpagespeed_deps_root")),
+ "src/third_party/chromium/src/build": From(Var("libpagespeed_deps_root")),
+ "src/third_party/chromium/src/chrome/tools/build":
+ File(Var("chromium_trunk") + "/src/chrome/tools/build/version.py" +
+ Var("chromium_revision")),
+ "src/third_party/chromium/src/net/base": From(Var("libpagespeed_deps_root")),
+
+ "src/third_party/chromium/src/net/tools/dump_cache":
+ Var("chromium_trunk") + "/src/net/tools/dump_cache" +
+ Var("chromium_revision"),
+
+ "src/third_party/gflags/src": Var("gflags_root") + Var("gflags_revision"),
+ "src/third_party/google-sparsehash/src":
+ Var("google_sparsehash_root") + Var("google_sparsehash_revision"),
+ "src/third_party/libjpeg": From(Var("libpagespeed_deps_root")),
+
+ "src/third_party/icu":
+ (Var("libpagespeed_trunk") + "/src/third_party/icu" +
+ Var("libpagespeed_revision")),
+
+ "src/third_party/libpagespeed/src/build":
+ (Var("libpagespeed_trunk") + "/src/build" +
+ Var("libpagespeed_revision")),
+
+ "src/third_party/libpagespeed/src/build/internal":
+ Var("chromium_trunk") + "/src/build/internal" + Var("chromium_revision"),
+
+ "src/third_party/libpagespeed/src/pagespeed/core":
+ (Var("libpagespeed_trunk") + "/src/pagespeed/core" +
+ Var("libpagespeed_revision")),
+
+ "src/third_party/libpagespeed/src/pagespeed/image_compression":
+ (Var("libpagespeed_trunk") + "/src/pagespeed/image_compression" +
+ Var("libpagespeed_revision")),
+
+ "src/third_party/libpagespeed/src/pagespeed/jsminify":
+ (Var("libpagespeed_trunk") + "/src/pagespeed/jsminify" +
+ Var("libpagespeed_revision")),
+
+ "src/third_party/libpagespeed/src/pagespeed/l10n":
+ (Var("libpagespeed_trunk") + "/src/pagespeed/l10n" +
+ Var("libpagespeed_revision")),
+
+ "src/third_party/libpagespeed/src/pagespeed/proto":
+ (Var("libpagespeed_trunk") + "/src/pagespeed/proto" +
+ Var("libpagespeed_revision")),
+
+ "src/third_party/libpagespeed/src/third_party":
+ (Var("libpagespeed_trunk") + "/src/third_party" +
+ Var("libpagespeed_revision")),
+
+ "src/third_party/protobuf2": From(Var("libpagespeed_deps_root")),
+ "src/third_party/protobuf2/src": From(Var("chromium_deps_root")),
+
+ "src/third_party/opencv/src/opencv/src":
+ Var("opencv_src") + "/opencv/src" + Var("opencv_revision"),
+
+ "src/third_party/opencv/src/opencv/include":
+ Var("opencv_src") + "/opencv/include" + Var("opencv_revision"),
+
+ "src/third_party/opencv/src/opencv/3rdparty/flann":
+ Var("opencv_src") + "/opencv/3rdparty/flann" + Var("opencv_revision"),
+
+ "src/third_party/opencv/src/opencv/3rdparty/lapack":
+ Var("opencv_src") + "/opencv/3rdparty/lapack" + Var("opencv_revision"),
+
+ "src/third_party/opencv/src/opencv/3rdparty/include":
+ Var("opencv_src") + "/opencv/3rdparty/include" + Var("opencv_revision"),
+
+ "src/third_party/serf/src": Var("serf_src") + Var("serf_revision"),
+
+ "src/third_party/zlib": From(Var("libpagespeed_deps_root")),
+
+ "src/tools/data_pack": From(Var("libpagespeed_deps_root")),
+ "src/tools/grit": From(Var("libpagespeed_deps_root")),
+ "src/tools/gyp": From(Var("chromium_deps_root")),
+}
+
+
+deps_os = {
+ "win": {
+ "src/third_party/cygwin": From(Var("chromium_deps_root")),
+ "src/third_party/python_24": From(Var("chromium_deps_root")),
+ },
+ "mac": {
+ },
+ "unix": {
+ },
+}
+
+
+include_rules = [
+ # Everybody can use some things.
+ "+base",
+ "+build",
+]
+
+
+# checkdeps.py shouldn't check include paths for files in these dirs:
+skip_child_includes = [
+ "testing",
+]
+
+
+hooks = [
+ {
+ # A change to a .gyp, .gypi, or to GYP itself should run the generator.
+ "pattern": ".",
+ "action": ["python", "src/build/gyp_chromium"],
+ },
+]
diff --git a/trunk/src/build/all.gyp b/trunk/src/build/all.gyp
new file mode 100644
index 0000000..618f901
--- /dev/null
+++ b/trunk/src/build/all.gyp
@@ -0,0 +1,28 @@
+# Copyright 2009 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'targets': [
+ {
+ 'target_name': 'All',
+ 'type': 'none',
+ 'xcode_create_dependents_test_runner': 1,
+ 'dependencies': [
+ '../net/instaweb/instaweb.gyp:*',
+ '../net/instaweb/instaweb_core.gyp:*',
+ '../net/instaweb/instaweb_html_rewriter.gyp:*',
+ '../net/instaweb/mod_pagespeed.gyp:*',
+ 'install.gyp:*',
+ ],} ]
+}
diff --git a/trunk/src/build/build_util.gyp b/trunk/src/build/build_util.gyp
new file mode 100644
index 0000000..e18276c
--- /dev/null
+++ b/trunk/src/build/build_util.gyp
@@ -0,0 +1,85 @@
+# Copyright (c) 2009 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+{
+ 'variables': {
+ 'version_py_path': 'version.py',
+ 'instaweb_path': '<(DEPTH)/net/instaweb',
+ 'version_path': '<(instaweb_path)/public/VERSION',
+ 'version_h_in_path': '<(instaweb_path)/public/version.h.in',
+ 'public_path' : 'net/instaweb/public',
+ 'version_h_path': '<(SHARED_INTERMEDIATE_DIR)/<(public_path)/version.h',
+ 'lastchange_out_path': '<(SHARED_INTERMEDIATE_DIR)/build/LASTCHANGE',
+ },
+ 'targets': [
+ {
+ 'target_name': 'lastchange',
+ 'type': 'none',
+ 'variables': {
+ 'default_lastchange_path': '../LASTCHANGE.in',
+ },
+ 'actions': [
+ {
+ 'action_name': 'lastchange',
+ 'inputs': [
+ # Note: <(default_lastchange_path) is optional,
+ # so it doesn't show up in inputs.
+ 'util/lastchange.py',
+ ],
+ 'outputs': [
+ '<(lastchange_out_path).always',
+ '<(lastchange_out_path)',
+ ],
+ 'action': [
+ 'python', '<@(_inputs)',
+ '-o', '<(lastchange_out_path)',
+ '-d', '<(default_lastchange_path)',
+ ],
+ 'message': 'Extracting last change to <(lastchange_out_path)',
+ 'process_outputs_as_sources': '1',
+ },
+ ],
+ },
+ {
+ 'target_name': 'mod_pagespeed_version_header',
+ 'type': 'none',
+ 'dependencies': [
+ 'lastchange',
+ ],
+ 'actions': [
+ {
+ 'action_name': 'version_header',
+ 'inputs': [
+ '<(version_path)',
+ '<(lastchange_out_path)',
+ '<(version_h_in_path)',
+ ],
+ 'outputs': [
+ '<(version_h_path)',
+ ],
+ 'action': [
+ 'python',
+ '<(version_py_path)',
+ '-f', '<(version_path)',
+ '-f', '<(lastchange_out_path)',
+ '<(version_h_in_path)',
+ '<@(_outputs)',
+ ],
+ 'message': 'Generating version header file: <@(_outputs)',
+ },
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '<(SHARED_INTERMEDIATE_DIR)',
+ ],
+ },
+ },
+ ]
+}
+
+# Local Variables:
+# tab-width:2
+# indent-tabs-mode:nil
+# End:
+# vim: set expandtab tabstop=2 shiftwidth=2:
diff --git a/trunk/src/build/common.gypi b/trunk/src/build/common.gypi
new file mode 100644
index 0000000..188a9ce
--- /dev/null
+++ b/trunk/src/build/common.gypi
@@ -0,0 +1,84 @@
+# Copyright 2009 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'variables': {
+ # Make sure we link statically so everything gets linked into a
+ # single shared object.
+ 'library': 'static_library',
+
+ # We're building a shared library, so everything needs to be built
+ # with Position-Independent Code.
+ 'linux_fpic': 1,
+
+ # Define the overridable use_system_libs variable in its own
+ # nested block, so it's available for use in the conditions block
+ # below.
+ 'variables': {
+ 'use_system_libs%': 0,
+ },
+
+ 'conditions': [
+ ['use_system_libs==1', {
+ 'use_system_apache_dev': 1,
+ 'use_system_libjpeg': 1,
+ 'use_system_libpng': 1,
+ 'use_system_opencv': 1,
+ 'use_system_zlib': 1,
+ },{
+ 'use_system_apache_dev%': 0,
+ }],
+ ],
+ },
+ 'includes': [
+ '../third_party/libpagespeed/src/build/common.gypi',
+ ],
+ 'target_defaults': {
+ 'conditions': [
+ ['OS == "linux"', {
+ 'cflags': [
+ # Our dependency on OpenCV need us to turn on exceptions.
+ '-fexceptions',
+ # Now we are using exceptions. -fno-asynchronous-unwind-tables is
+ # set in libpagespeed's common.gypi. Now enable it.
+ '-fasynchronous-unwind-tables',
+ ],
+ 'cflags_cc': [
+ '-frtti', # Hardy's g++ 4.2 <trl/function> uses typeid
+ ],
+ }],
+ ['OS == "mac"', {
+ 'xcode_settings':{
+ 'GCC_ENABLE_CPP_EXCEPTIONS': 'YES', # -fexceptions
+ 'GCC_ENABLE_CPP_RTTI': 'YES', # -frtti
+ },
+ }],
+ ],
+
+ # Permit building us with coverage information
+ 'configurations': {
+ 'Debug_Coverage': {
+ 'inherit_from': ['Debug'],
+ 'cflags': [
+ '-ftest-coverage',
+ '-fprofile-arcs',
+ ],
+ 'ldflags': [
+ # takes care of -lgcov for us, but can be in a build configuration
+ '-ftest-coverage -fprofile-arcs',
+ ],
+ },
+ },
+ },
+}
diff --git a/trunk/src/build/compiler_version.py b/trunk/src/build/compiler_version.py
new file mode 100755
index 0000000..be71db2
--- /dev/null
+++ b/trunk/src/build/compiler_version.py
@@ -0,0 +1,24 @@
+#!/usr/bin/python
+
+# Copyright 2010 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This script is wrapper for the Chromium version of compiler_version.py.
+
+import os
+
+script_dir = os.path.dirname(__file__)
+chrome_src = os.path.normpath(os.path.join(script_dir, os.pardir, 'third_party', 'chromium', 'src'))
+
+execfile(os.path.join(chrome_src, 'build', 'compiler_version.py'))
diff --git a/trunk/src/build/features_override.gypi b/trunk/src/build/features_override.gypi
new file mode 100644
index 0000000..a655733
--- /dev/null
+++ b/trunk/src/build/features_override.gypi
@@ -0,0 +1,18 @@
+# Copyright 2009 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Chromium expects this file to be here, but for our (Page Speed) purposes, it
+# doesn't need to actually do anything.
+
+{}
diff --git a/trunk/src/build/gyp_chromium b/trunk/src/build/gyp_chromium
new file mode 100755
index 0000000..de60768
--- /dev/null
+++ b/trunk/src/build/gyp_chromium
@@ -0,0 +1,24 @@
+#!/usr/bin/python
+
+# Copyright 2009 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This script is wrapper for the Chromium version of gyp_chromium.
+
+import os
+
+script_dir = os.path.dirname(__file__)
+chrome_src = os.path.normpath(os.path.join(script_dir, os.pardir, 'third_party', 'chromium', 'src'))
+
+execfile(os.path.join(chrome_src, 'build', 'gyp_chromium'))
diff --git a/trunk/src/build/install.gyp b/trunk/src/build/install.gyp
new file mode 100644
index 0000000..0221db5
--- /dev/null
+++ b/trunk/src/build/install.gyp
@@ -0,0 +1,191 @@
+# Copyright (c) 2010 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+{
+ 'variables': {
+ 'install_path': '<(DEPTH)/install',
+ 'version_py_path': '<(DEPTH)/build/version.py',
+ 'version_path': '<(DEPTH)/net/instaweb/public/VERSION',
+ 'lastchange_path': '<(SHARED_INTERMEDIATE_DIR)/build/LASTCHANGE',
+ 'branding_dir': '<(install_path)/common',
+ },
+ 'conditions': [
+ ['OS=="linux"', {
+ 'variables': {
+ 'version' : '<!(python <(version_py_path) -f <(version_path) -t "@MAJOR@.@MINOR@.@BUILD@.@PATCH@")',
+ 'revision' : '<!(python <(DEPTH)/build/util/lastchange.py | cut -d "=" -f 2)',
+ 'packaging_files_common': [
+ '<(install_path)/common/apt.include',
+ '<(install_path)/common/mod-pagespeed/mod-pagespeed.info',
+ '<(install_path)/common/installer.include',
+ '<(install_path)/common/repo.cron',
+ '<(install_path)/common/rpm.include',
+ '<(install_path)/common/rpmrepo.cron',
+ '<(install_path)/common/updater',
+ '<(install_path)/common/variables.include',
+ '<(install_path)/common/BRANDING',
+ '<(install_path)/common/pagespeed.load.template',
+ '<(install_path)/common/pagespeed.conf.template',
+ ],
+ 'packaging_files_deb': [
+ '<(install_path)/debian/build.sh',
+ '<(install_path)/debian/changelog.template',
+ '<(install_path)/debian/conffiles',
+ '<(install_path)/debian/control.template',
+ '<(install_path)/debian/postinst',
+ '<(install_path)/debian/postrm',
+ '<(install_path)/debian/prerm',
+ ],
+ 'packaging_files_rpm': [
+ '<(install_path)/rpm/build.sh',
+ '<(install_path)/rpm/mod-pagespeed.spec.template',
+ ],
+ 'packaging_files_binaries': [
+ '<(PRODUCT_DIR)/libmod_pagespeed.so',
+ ],
+ 'flock_bash': ['flock', '--', '/tmp/linux_package_lock', 'bash'],
+ 'deb_build': '<(PRODUCT_DIR)/install/debian/build.sh',
+ 'rpm_build': '<(PRODUCT_DIR)/install/rpm/build.sh',
+ 'deb_cmd': ['<@(flock_bash)', '<(deb_build)', '-o' '<(PRODUCT_DIR)',
+ '-b', '<(PRODUCT_DIR)', '-a', '<(target_arch)'],
+ 'rpm_cmd': ['<@(flock_bash)', '<(rpm_build)', '-o' '<(PRODUCT_DIR)',
+ '-b', '<(PRODUCT_DIR)', '-a', '<(target_arch)'],
+ 'conditions': [
+ ['target_arch=="ia32"', {
+ 'deb_arch': 'i386',
+ 'rpm_arch': 'i386',
+ }],
+ ['target_arch=="x64"', {
+ 'deb_arch': 'amd64',
+ 'rpm_arch': 'x86_64',
+ }],
+ ],
+ },
+ 'targets': [
+ {
+ 'target_name': 'linux_installer_configs',
+ 'type': 'none',
+ # Add these files to the build output so the build archives will be
+ # "hermetic" for packaging.
+ 'copies': [
+ {
+ 'destination': '<(PRODUCT_DIR)/install/debian/',
+ 'files': [
+ '<@(packaging_files_deb)',
+ ]
+ },
+ {
+ 'destination': '<(PRODUCT_DIR)/install/rpm/',
+ 'files': [
+ '<@(packaging_files_rpm)',
+ ]
+ },
+ {
+ 'destination': '<(PRODUCT_DIR)/install/common/',
+ 'files': [
+ '<@(packaging_files_common)',
+ ]
+ },
+ ],
+ 'actions': [
+ {
+ 'action_name': 'save_build_info',
+ 'inputs': [
+ '<(branding_dir)/BRANDING',
+ '<(version_path)',
+ '<(lastchange_path)',
+ ],
+ 'outputs': [
+ '<(PRODUCT_DIR)/installer/version.txt',
+ ],
+ # Just output the default version info variables.
+ 'action': [
+ 'python', '<(version_py_path)',
+ '-f', '<(branding_dir)/BRANDING',
+ '-f', '<(version_path)',
+ '-f', '<(lastchange_path)',
+ '-o', '<@(_outputs)'
+ ],
+ },
+ ],
+ },
+ {
+ 'target_name': 'linux_packages',
+ 'suppress_wildcard': 1,
+ 'type': 'none',
+ 'dependencies': [
+ 'linux_package_deb',
+ 'linux_package_rpm',
+ ],
+ },
+ {
+ 'target_name': 'linux_package_deb',
+ 'suppress_wildcard': 1,
+ 'type': 'none',
+ 'dependencies': [
+ '<(DEPTH)/net/instaweb/mod_pagespeed.gyp:mod_pagespeed',
+ 'linux_installer_configs',
+ ],
+ 'actions': [
+ {
+ 'variables': {
+ 'channel': 'beta',
+ },
+ 'action_name': 'deb_package_<(channel)',
+ 'process_outputs_as_sources': 1,
+ 'inputs': [
+ '<(deb_build)',
+ '<@(packaging_files_binaries)',
+ '<@(packaging_files_common)',
+ '<@(packaging_files_deb)',
+ ],
+ 'outputs': [
+ '<(PRODUCT_DIR)/mod-pagespeed-<(channel)-<(version)-r<(revision)_<(deb_arch).deb',
+ ],
+ 'action': [ '<@(deb_cmd)', '-c', '<(channel)', ],
+ },
+ ],
+ },
+ {
+ 'target_name': 'linux_package_rpm',
+ 'suppress_wildcard': 1,
+ 'type': 'none',
+ 'dependencies': [
+ '<(DEPTH)/net/instaweb/mod_pagespeed.gyp:mod_pagespeed',
+ 'linux_installer_configs',
+ ],
+ 'actions': [
+ {
+ 'variables': {
+ 'channel': 'beta',
+ },
+ 'action_name': 'rpm_package_<(channel)',
+ 'process_outputs_as_sources': 1,
+ 'inputs': [
+ '<(rpm_build)',
+ '<(PRODUCT_DIR)/install/rpm/mod-pagespeed.spec.template',
+ '<@(packaging_files_binaries)',
+ '<@(packaging_files_common)',
+ '<@(packaging_files_rpm)',
+ ],
+ 'outputs': [
+ '<(PRODUCT_DIR)/mod-pagespeed-<(channel)-<(version)-r<(revision).<(rpm_arch).rpm',
+ ],
+ 'action': [ '<@(rpm_cmd)', '-c', '<(channel)', ],
+ },
+ ],
+ },
+ ],
+ },{
+ 'targets': [
+ ],
+ }],
+ ],
+}
+
+# Local Variables:
+# tab-width:2
+# indent-tabs-mode:nil
+# End:
+# vim: set expandtab tabstop=2 shiftwidth=2:
diff --git a/trunk/src/build/output_dll_copy.rules b/trunk/src/build/output_dll_copy.rules
new file mode 100644
index 0000000..c6e9051
--- /dev/null
+++ b/trunk/src/build/output_dll_copy.rules
@@ -0,0 +1,17 @@
+<?xml version="1.0" encoding="utf-8"?>
+<VisualStudioToolFile
+ Name="Output DLL copy"
+ Version="8.00"
+ >
+ <Rules>
+ <CustomBuildRule
+ Name="Output DLL copy"
+ CommandLine="xcopy /R /C /Y $(InputPath) $(OutDir)"
+ Outputs="$(OutDir)\$(InputFileName)"
+ FileExtensions="*.dll"
+ >
+ <Properties>
+ </Properties>
+ </CustomBuildRule>
+ </Rules>
+</VisualStudioToolFile>
diff --git a/trunk/src/build/release.gypi b/trunk/src/build/release.gypi
new file mode 100644
index 0000000..c12526b
--- /dev/null
+++ b/trunk/src/build/release.gypi
@@ -0,0 +1,19 @@
+{
+ 'conditions': [
+ # Handle build types.
+ ['buildtype=="Dev"', {
+ 'includes': ['internal/release_impl.gypi'],
+ }],
+ ['buildtype=="Official"', {
+ 'includes': ['internal/release_impl_official.gypi'],
+ }],
+ # TODO(bradnelson): may also need:
+ # checksenabled
+ # coverage
+ # dom_stats
+ # pgo_instrument
+ # pgo_optimize
+ # purify
+ ],
+}
+
diff --git a/trunk/src/build/version.py b/trunk/src/build/version.py
new file mode 100755
index 0000000..428f761
--- /dev/null
+++ b/trunk/src/build/version.py
@@ -0,0 +1,25 @@
+#!/usr/bin/python
+
+# Copyright 2010 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This script is wrapper for the Chromium version of compiler_version.py.
+
+import os
+
+script_dir = os.path.dirname(__file__)
+chrome_src = os.path.normpath(os.path.join(script_dir, os.pardir, 'third_party',
+ 'chromium', 'src'))
+
+execfile(os.path.join(chrome_src, 'chrome', 'tools', 'build', 'version.py'))
diff --git a/trunk/src/install/Makefile b/trunk/src/install/Makefile
new file mode 100644
index 0000000..810a560
--- /dev/null
+++ b/trunk/src/install/Makefile
@@ -0,0 +1,236 @@
+# This Makefile is used to help drive the installation of mod_pagespeed into
+# an Apache installation.
+#
+# Note that the location of the Apache configuration files may vary by
+# Linux distribution. For example, we have seen the following installation
+# directories for the default Apache install.
+#
+# Ubuntu /etc/apache2/mods-enabled/*.conf
+# CentOS /etc/httpd/conf.d/*.conf
+# Custom Apache build from source /usr/local/apache2/conf/extra/
+#
+# In the case of the custom Apache build, you must also
+# edit /usr/local/apache2/conf # to add "Include conf/extra/pagespeed.conf"
+#
+# The goal of this Makefile is to help generate basic default
+# configuration files that can then be edited to tune the HTML
+# performance based on the Apache installation, internet-visible
+# hostnames, and the specific needs of the site.
+#
+# The usage model of this Makefile is that, as an unpriviledged user, you
+# create the desired configuration files in /tmp, where you can examine
+# them before installing them. You can then do either of these:
+#
+# (a) Run "make -n install" to see the recommended installation commands,
+# and execute them by hand
+# (b) Run "sudo make install" to install them automatically.
+#
+#
+# To install mod_pagespeed properly, we need to know the locations of
+# Apache configuration scripts and binaries. These can are specified
+# as Makefile variables which can be overridden on the command line.
+# They have defaults, which will often need to be changed.
+
+
+# The location of the Apache root installation directory. This helps form
+# defaults for other variables, but each of those can be overridden.
+APACHE_ROOT = /etc/httpd
+
+# The installation directory for modules (mod*.so)
+APACHE_MODULES = $(APACHE_ROOT)/modules
+
+# The root directory Apache uses for serving files.
+APACHE_DOC_ROOT = /var/www
+
+APACHE_CONTROL_PROGRAM = /etc/init.d/apache2
+APACHE_START = $(APACHE_CONTROL_PROGRAM) start
+APACHE_STOP = $(APACHE_CONTROL_PROGRAM) stop
+
+# A temp directory to stage generated configuration files. This must be
+# writable by the user, and readable by root.
+STAGING_DIR = /tmp/mod_pagespeed.install
+
+# The mod_pagespeed module is specified relative to the install directory,
+# which is src/install.
+MOD_PAGESPEED_ROOT = ..
+PAGESPEED_MODULE = $(MOD_PAGESPEED_ROOT)/out/Release/libmod_pagespeed.so
+
+# On systems dervied from the NCSA configuration files by Rob McCool,
+# you enable a module by writing its .conf file into
+# $(APACHE_ROOT)/mods-enabled/pagespeed.conf, and a single Load command into
+# $(APACHE_ROOT)/mods-enabled/pagespeed.conf. So if that exists, then we'll
+# try to automate that.
+MODS_ENABLED_DIR = $(shell if [ -d $(APACHE_ROOT)/mods-enabled ]; then \
+ echo $(APACHE_ROOT)/mods-enabled; fi)
+
+# Determines should mod_pagespeed should write files
+MOD_PAGESPEED_FILE_ROOT = /var/mod_pagespeed
+
+# The username used to run apache. This is needed to create the directory
+# used to store mod_pagespeed files and cache data.
+APACHE_USER = www-data
+
+# Set this to 1 to enable mod_proxy and mod_rewrite
+ENABLE_PROXY = 0
+
+.PHONY: config_file echo_vars
+
+echo_vars:
+ @echo Run "restart" to add default instaweb config to apache
+ @echo Or run "stop", "staging", "install", and "start".
+ @echo These configuration variables can be reset on the make command line,
+ @echo e.g. \"make config_file\"
+ @echo ""
+ @echo " APACHE_CONF=$(APACHE_CONF)"
+ @echo " APACHE_MODULES=$(APACHE_MODULES)"
+ @echo " APACHE_ROOT=$(APACHE_ROOT)"
+ @echo " APACHE_START=$(APACHE_START)"
+ @echo " APACHE_STOP=$(APACHE_STOP)"
+ @echo " MOD_PAGESPEED_FILE_ROOT=$(MOD_PAGESPEED_FILE_ROOT)"
+ @echo " MODS_ENABLED_DIR=$(MODS_ENABLED_DIR)"
+ @echo " STAGING_DIR=$(STAGING_DIR)"
+ @echo " ENABLE_PROXY=${ENABLE_PROXY}"
+ @echo " SLURP_DIR=${SLURP_DIR}"
+
+
+# In some Linux distributions, such as Ubuntu, there are two commands
+# in the default root config file:
+# Include /etc/apache2/modes-enabled/*.load
+# Include /etc/apache2/modes-enabled/*.conf
+# we need to write a one-line '.load' file and put that and our '.conf' file
+# into .../mods-enabled.
+#
+# In other distributions, such as CentOS, there is an 'Include DIR/*.conf',
+# but there is no implicit loading of modules, so we write our Load line
+# directly into our config file
+
+ifeq ($(MODS_ENABLED_DIR),)
+
+# This is a CentOS-like installation, where there is no explicit .load
+# file, and we instead pre-pend the LoadModule command to the .conf file.
+APACHE_CONF = $(APACHE_ROOT)/conf.d
+CONF_SOURCES = $(STAGING_DIR)/pagespeed.load $(STAGING_DIR)/pagespeed.conf
+
+else
+
+# This is an Ubuntu-like installation, where the .load files are placed
+# separately into a mods-enabled directory, and the .conf file is loaded
+# independently.
+MODS_ENABLED_INSTALL_COMMANDS = \
+ cp $(STAGING_DIR)/pagespeed.load $(MODS_ENABLED_DIR) ; \
+ rm -f $(MODS_ENABLED_DIR)/headers.load ; \
+ cd $(MODS_ENABLED_DIR) && ln -s ../mods-available/headers.load ; \
+ rm -f $(MODS_ENABLED_DIR)/deflate.load ; \
+ cd $(MODS_ENABLED_DIR) && ln -s ../mods-available/deflate.load
+
+APACHE_CONF = $(MODS_ENABLED_DIR)
+CONF_SOURCES = $(STAGING_DIR)/pagespeed.conf
+
+endif
+
+
+# We will generate 'proxy.conf' in the staging area
+# unconditiontionally, but we will load it into the
+# Apache server only if the user installs with ENABLE_PROXY=1
+ifeq ($(ENABLE_PROXY),1)
+CONF_SOURCES += $(STAGING_DIR)/proxy.conf
+endif
+
+APACHE_SLURP_READ_ONLY_COMMAND=\#ModPagespeedSlurpReadOnly on
+
+ifeq ($(SLURP_DIR),)
+ APACHE_SLURP_DIR_COMMAND = \#ModPagespeedSlurpDirectory ...
+else
+ APACHE_SLURP_DIR_COMMAND = ModPagespeedSlurpDirectory $(SLURP_DIR)
+ ifeq ($(SLURP_WRITE),1)
+ APACHE_SLURP_READ_ONLY_COMMAND=ModPagespeedSlurpReadOnly off
+ else
+ APACHE_SLURP_READ_ONLY_COMMAND=ModPagespeedSlurpReadOnly on
+ endif
+endif
+
+ifeq ($(STRESS_TEST),1)
+ # remove prefix
+ STRESS_TEST_SED_PATTERN=^\#STRESS
+else
+ # remove whole line
+ STRESS_TEST_SED_PATTERN=^\#STRESS.*
+endif
+
+# Note that the quoted sed replacement for APACHE_SLURP_DIR_COMMAND is because
+# that might have embedded spaces, and 'sed' is interpreted first by bash.
+
+$(STAGING_DIR)/pagespeed.conf : common/pagespeed.conf.template debug.conf.template
+ sed -e s@APACHE_DOC_ROOT@$(APACHE_DOC_ROOT)@g \
+ -e s!@@MODPAGESPEED_CACHE_ROOT@@!$(MOD_PAGESPEED_FILE_ROOT)!g \
+ -e "s@# ModPagespeedSlurpDirectory ...@$(APACHE_SLURP_DIR_COMMAND)@g" \
+ -e "s@# ModPagespeedSlurpReadOnly on@$(APACHE_SLURP_READ_ONLY_COMMAND)@g" \
+ -e "s@$(STRESS_TEST_SED_PATTERN)@@" \
+ $^ > $@
+
+$(STAGING_DIR)/proxy.conf : proxy.conf.template
+ sed -e s@APACHE_MODULES@$(APACHE_MODULES)@g \
+ $< > $@
+
+CONF_TEMPLATES = $(STAGING_DIR)/pagespeed.conf \
+ $(STAGING_DIR)/proxy.conf
+
+setup_staging_dir :
+ rm -rf $(STAGING_DIR)
+ mkdir -p $(STAGING_DIR)
+
+# Generate a configuration file and copy it to the staging area.
+# Also copy the example tree, and the built Apache module
+staging: setup_staging_dir $(CONF_TEMPLATES)
+ echo "LoadModule pagespeed_module $(APACHE_MODULES)/mod_pagespeed.so" \
+ > $(STAGING_DIR)/pagespeed.load
+ echo "LoadModule deflate_module $(APACHE_MODULES)/mod_deflate.so" \
+ >> $(STAGING_DIR)/pagespeed.load
+ $(MODS_ENABLED_STAGING_COMMANDS)
+ cp -rp mod_pagespeed_example $(STAGING_DIR)
+ cp $(PAGESPEED_MODULE) $(STAGING_DIR)/mod_pagespeed.so
+
+# To install the mod_pagespeed configuration into the system, you must
+# run this as root, or under sudo.
+install : mod_pagespeed_file_root
+ $(MODS_ENABLED_INSTALL_COMMANDS)
+ cat $(CONF_SOURCES) > $(APACHE_CONF)/pagespeed.conf
+ rm -rf $(APACHE_DOC_ROOT)/mod_pagespeed_example
+ cp -r $(STAGING_DIR)/mod_pagespeed_example $(APACHE_DOC_ROOT)
+ chown -R $(APACHE_USER) $(APACHE_DOC_ROOT)/mod_pagespeed_example
+ cp $(STAGING_DIR)/mod_pagespeed.so $(APACHE_MODULES)
+
+mod_pagespeed_file_root :
+ mkdir -p $(MOD_PAGESPEED_FILE_ROOT)/cache
+ mkdir -p $(MOD_PAGESPEED_FILE_ROOT)/files
+ chown -R $(APACHE_USER) $(MOD_PAGESPEED_FILE_ROOT)
+
+flush_disk_cache :
+ rm -rf $(MOD_PAGESPEED_FILE_ROOT)
+ $(MAKE) MOD_PAGESPEED_FILE_ROOT=$(MOD_PAGESPEED_FILE_ROOT) \
+ APACHE_USER=$(APACHE_USER) mod_pagespeed_file_root
+
+# Starts Apache server
+start :
+ sudo $(APACHE_START)
+stop :
+ sudo $(APACHE_STOP)
+
+# To run a complete iteration, stopping Apache, reconfiguring
+# it, and and restarting it, you can run 'make restart [args...]
+restart : staging
+ sudo $(APACHE_STOP)
+ $(MAKE) staging
+ sudo $(MAKE) install \
+ APACHE_DOC_ROOT=$(APACHE_DOC_ROOT) \
+ STAGING_DIR=$(STAGING_DIR) \
+ APACHE_CONF=$(APACHE_CONF) \
+ APACHE_MODULES=$(APACHE_MODULES) \
+ MODS_ENABLED_DIR=$(MODS_ENABLED_DIR) \
+ APACHE_USER=$(APACHE_USER) \
+ ENABLE_PROXY=$(ENABLE_PROXY)
+ sudo $(APACHE_START)
+
+# Tests that the installed mod_pagespeed server is working.
+test :
+ ./system_test.sh localhost
diff --git a/trunk/src/install/centos.sh b/trunk/src/install/centos.sh
new file mode 100755
index 0000000..77ee2d9
--- /dev/null
+++ b/trunk/src/install/centos.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+
+set -x
+exec make \
+ APACHE_CONTROL_PROGRAM=/etc/init.d/httpd \
+ APACHE_USER=apache \
+ APACHE_DOC_ROOT=/var/www/html \
+ $*
diff --git a/trunk/src/install/common/BRANDING b/trunk/src/install/common/BRANDING
new file mode 100644
index 0000000..e0a7de9
--- /dev/null
+++ b/trunk/src/install/common/BRANDING
@@ -0,0 +1,5 @@
+COMPANY_FULLNAME=Google Inc.
+COMPANY_SHORTNAME=Google Inc.
+PRODUCT_FULLNAME=mod_pagespeed
+PRODUCT_SHORTNAME=mod_pagespeed
+COPYRIGHT=Copyright (C) 2010.
diff --git a/trunk/src/install/common/apt.include b/trunk/src/install/common/apt.include
new file mode 100644
index 0000000..6c26417
--- /dev/null
+++ b/trunk/src/install/common/apt.include
@@ -0,0 +1,188 @@
+@@include@@variables.include
+
+APT_GET="`which apt-get 2> /dev/null`"
+APT_CONFIG="`which apt-config 2> /dev/null`"
+
+SOURCES_PREAMBLE="### THIS FILE IS AUTOMATICALLY CONFIGURED ###
+# You may comment out this entry, but any other modifications may be lost.\n"
+
+# Parse apt configuration and return requested variable value.
+apt_config_val() {
+ APTVAR="$1"
+ if [ -x "$APT_CONFIG" ]; then
+ "$APT_CONFIG" dump | sed -e "/^$APTVAR /"'!d' -e "s/^$APTVAR \"\(.*\)\".*/\1/"
+ fi
+}
+
+# Install the repository signing key (see also:
+# http://www.google.com/linuxrepositories/aboutkey.html)
+install_key() {
+ APT_KEY="`which apt-key 2> /dev/null`"
+ if [ -x "$APT_KEY" ]; then
+ "$APT_KEY" add - >/dev/null 2>&1 <<KEYDATA
+-----BEGIN PGP PUBLIC KEY BLOCK-----
+Version: GnuPG v1.4.2.2 (GNU/Linux)
+
+mQGiBEXwb0YRBADQva2NLpYXxgjNkbuP0LnPoEXruGmvi3XMIxjEUFuGNCP4Rj/a
+kv2E5VixBP1vcQFDRJ+p1puh8NU0XERlhpyZrVMzzS/RdWdyXf7E5S8oqNXsoD1z
+fvmI+i9b2EhHAA19Kgw7ifV8vMa4tkwslEmcTiwiw8lyUl28Wh4Et8SxzwCggDcA
+feGqtn3PP5YAdD0km4S4XeMEAJjlrqPoPv2Gf//tfznY2UyS9PUqFCPLHgFLe80u
+QhI2U5jt6jUKN4fHauvR6z3seSAsh1YyzyZCKxJFEKXCCqnrFSoh4WSJsbFNc4PN
+b0V0SqiTCkWADZyLT5wll8sWuQ5ylTf3z1ENoHf+G3um3/wk/+xmEHvj9HCTBEXP
+78X0A/0Tqlhc2RBnEf+AqxWvM8sk8LzJI/XGjwBvKfXe+l3rnSR2kEAvGzj5Sg0X
+4XmfTg4Jl8BNjWyvm2Wmjfet41LPmYJKsux3g0b8yzQxeOA4pQKKAU3Z4+rgzGmf
+HdwCG5MNT2A5XxD/eDd+L4fRx0HbFkIQoAi1J3YWQSiTk15fw7RMR29vZ2xlLCBJ
+bmMuIExpbnV4IFBhY2thZ2UgU2lnbmluZyBLZXkgPGxpbnV4LXBhY2thZ2VzLWtl
+eW1hc3RlckBnb29nbGUuY29tPohjBBMRAgAjAhsDBgsJCAcDAgQVAggDBBYCAwEC
+HgECF4AFAkYVdn8CGQEACgkQoECDD3+sWZHKSgCfdq3HtNYJLv+XZleb6HN4zOcF
+AJEAniSFbuv8V5FSHxeRimHx25671az+uQINBEXwb0sQCACuA8HT2nr+FM5y/kzI
+A51ZcC46KFtIDgjQJ31Q3OrkYP8LbxOpKMRIzvOZrsjOlFmDVqitiVc7qj3lYp6U
+rgNVaFv6Qu4bo2/ctjNHDDBdv6nufmusJUWq/9TwieepM/cwnXd+HMxu1XBKRVk9
+XyAZ9SvfcW4EtxVgysI+XlptKFa5JCqFM3qJllVohMmr7lMwO8+sxTWTXqxsptJo
+pZeKz+UBEEqPyw7CUIVYGC9ENEtIMFvAvPqnhj1GS96REMpry+5s9WKuLEaclWpd
+K3krttbDlY1NaeQUCRvBYZ8iAG9YSLHUHMTuI2oea07Rh4dtIAqPwAX8xn36JAYG
+2vgLAAMFB/wKqaycjWAZwIe98Yt0qHsdkpmIbarD9fGiA6kfkK/UxjL/k7tmS4Vm
+CljrrDZkPSQ/19mpdRcGXtb0NI9+nyM5trweTvtPw+HPkDiJlTaiCcx+izg79Fj9
+KcofuNb3lPdXZb9tzf5oDnmm/B+4vkeTuEZJ//IFty8cmvCpzvY+DAz1Vo9rA+Zn
+cpWY1n6z6oSS9AsyT/IFlWWBZZ17SpMHu+h4Bxy62+AbPHKGSujEGQhWq8ZRoJAT
+G0KSObnmZ7FwFWu1e9XFoUCt0bSjiJWTIyaObMrWu/LvJ3e9I87HseSJStfw6fki
+5og9qFEkMrIrBCp3QGuQWBq/rTdMuwNFiEkEGBECAAkFAkXwb0sCGwwACgkQoECD
+D3+sWZF/WACfeNAu1/1hwZtUo1bR+MWiCjpvHtwAnA1R3IHqFLQ2X3xJ40XPuAyY
+/FJG
+=Quqp
+-----END PGP PUBLIC KEY BLOCK-----
+KEYDATA
+ fi
+}
+
+# Set variables for the locations of the apt sources lists.
+find_apt_sources() {
+ APTDIR=$(apt_config_val Dir)
+ APTETC=$(apt_config_val 'Dir::Etc')
+ APT_SOURCES="$APTDIR$APTETC$(apt_config_val 'Dir::Etc::sourcelist')"
+ APT_SOURCESDIR="$APTDIR$APTETC$(apt_config_val 'Dir::Etc::sourceparts')"
+}
+
+# Update the Google repository if it's not set correctly.
+# Note: this doesn't necessarily enable the repository, it just makes sure the
+# correct settings are available in the sources list.
+# Returns:
+# 0 - no update necessary
+# 2 - error
+update_bad_sources() {
+ if [ ! "$REPOCONFIG" ]; then
+ return 0
+ fi
+
+ find_apt_sources
+
+ SOURCELIST="$APT_SOURCESDIR/@@PACKAGE@@.list"
+ # Don't do anything if the file isn't there, since that probably means the
+ # user disabled it.
+ if [ ! -r "$SOURCELIST" ]; then
+ return 0
+ fi
+
+ # Basic check for active configurations (non-blank, non-comment lines).
+ ACTIVECONFIGS=$(grep -v "^[[:space:]]*\(#.*\)\?$" "$SOURCELIST" 2>/dev/null)
+
+ # Check if the correct repository configuration is in there.
+ REPOMATCH=$(grep "^[[:space:]#]*\b$REPOCONFIG\b" "$SOURCELIST" \
+ 2>/dev/null)
+
+ # Check if the correct repository is disabled.
+ MATCH_DISABLED=$(echo "$REPOMATCH" | grep "^[[:space:]]*#" 2>/dev/null)
+
+ # Now figure out if we need to fix things.
+ BADCONFIG=1
+ if [ "$REPOMATCH" ]; then
+ # If it's there and active, that's ideal, so nothing to do.
+ if [ ! "$MATCH_DISABLED" ]; then
+ BADCONFIG=0
+ else
+ # If it's not active, but neither is anything else, that's fine too.
+ if [ ! "$ACTIVECONFIGS" ]; then
+ BADCONFIG=0
+ fi
+ fi
+ fi
+
+ if [ $BADCONFIG -eq 0 ]; then
+ return 0
+ fi
+
+ # At this point, either the correct configuration is completely missing, or
+ # the wrong configuration is active. In that case, just abandon the mess and
+ # recreate the file with the correct configuration. If there were no active
+ # configurations before, create the new configuration disabled.
+ DISABLE=""
+ if [ ! "$ACTIVECONFIGS" ]; then
+ DISABLE="#"
+ fi
+ printf "$SOURCES_PREAMBLE" > "$SOURCELIST"
+ printf "$DISABLE$REPOCONFIG\n" >> "$SOURCELIST"
+ if [ $? -eq 0 ]; then
+ return 0
+ fi
+ return 2
+}
+
+# Add the Google repository to the apt sources.
+# Returns:
+# 0 - sources list was created
+# 2 - error
+create_sources_lists() {
+ if [ ! "$REPOCONFIG" ]; then
+ return 0
+ fi
+
+ find_apt_sources
+
+ SOURCELIST="$APT_SOURCESDIR/@@PACKAGE@@.list"
+ if [ -d "$APT_SOURCESDIR" ]; then
+ printf "$SOURCES_PREAMBLE" > "$SOURCELIST"
+ printf "$REPOCONFIG\n" >> "$SOURCELIST"
+ if [ $? -eq 0 ]; then
+ return 0
+ fi
+ fi
+ return 2
+}
+
+# Remove our custom sources list file.
+# Returns:
+# 0 - successfully removed, or not configured
+# !0 - failed to remove
+clean_sources_lists() {
+ if [ ! "$REPOCONFIG" ]; then
+ return 0
+ fi
+
+ find_apt_sources
+
+ rm -f "$APT_SOURCESDIR/@@PACKAGE@@.list" \
+ "$APT_SOURCESDIR/@@PACKAGE@@-@@CHANNEL@@.list"
+}
+
+# Detect if the repo config was disabled by distro upgrade and enable if
+# necessary.
+handle_distro_upgrade() {
+ if [ ! "$REPOCONFIG" ]; then
+ return 0
+ fi
+
+ find_apt_sources
+ SOURCELIST="$APT_SOURCESDIR/@@PACKAGE@@.list"
+ if [ -r "$SOURCELIST" ]; then
+ REPOLINE=$(grep -E "^[[:space:]]*#[[:space:]]*$REPOCONFIG[[:space:]]*# disabled on upgrade to .*" "$SOURCELIST")
+ if [ $? -eq 0 ]; then
+ sed -i -e "s,^[[:space:]]*#[[:space:]]*\($REPOCONFIG\)[[:space:]]*# disabled on upgrade to .*,\1," \
+ "$SOURCELIST"
+ LOGGER=$(which logger 2> /dev/null)
+ if [ "$LOGGER" ]; then
+ "$LOGGER" -t "$0" "Reverted repository modification: $REPOLINE."
+ fi
+ fi
+ fi
+}
+
diff --git a/trunk/src/install/common/installer.include b/trunk/src/install/common/installer.include
new file mode 100644
index 0000000..33de58e
--- /dev/null
+++ b/trunk/src/install/common/installer.include
@@ -0,0 +1,118 @@
+# Recursively replace @@include@@ template variables with the referenced file,
+# and write the resulting text to stdout.
+process_template_includes() {
+ INCSTACK+="$1->"
+ # Includes are relative to the file that does the include.
+ INCDIR=$(dirname $1)
+ # Clear IFS so 'read' doesn't trim whitespace
+ local OLDIFS="$IFS"
+ IFS=''
+ while read -r LINE
+ do
+ INCLINE=$(sed -e '/^[[:space:]]*@@include@@/!d' <<<$LINE)
+ if [ -n "$INCLINE" ]; then
+ INCFILE=$(echo $INCLINE | sed -e "s#@@include@@\(.*\)#\1#")
+ # Simple filename match to detect cyclic includes.
+ CYCLE=$(sed -e "\#$INCFILE#"'!d' <<<$INCSTACK)
+ if [ "$CYCLE" ]; then
+ echo "ERROR: Possible cyclic include detected." 1>&2
+ echo "$INCSTACK$INCFILE" 1>&2
+ exit 1
+ fi
+ if [ ! -r "$INCDIR/$INCFILE" ]; then
+ echo "ERROR: Couldn't read include file: $INCDIR/$INCFILE" 1>&2
+ exit 1
+ fi
+ process_template_includes "$INCDIR/$INCFILE"
+ else
+ echo "$LINE"
+ fi
+ done < "$1"
+ IFS="$OLDIFS"
+ INCSTACK=${INCSTACK%"$1->"}
+}
+
+# Replace template variables (@@VARNAME@@) in the given template file. If a
+# second argument is given, save the processed text to that filename, otherwise
+# modify the template file in place.
+process_template() (
+ # Don't worry if some of these substitution variables aren't set.
+ # Note that this function is run in a sub-shell so we don't leak this
+ # setting, since we still want unbound variables to be an error elsewhere.
+ set +u
+
+ local TMPLIN="$1"
+ if [ -z "$2" ]; then
+ local TMPLOUT="$TMPLIN"
+ else
+ local TMPLOUT="$2"
+ fi
+ # Process includes first so included text also gets substitutions.
+ TMPLINCL="$(process_template_includes "$TMPLIN")"
+ sed \
+ -e "s#@@PACKAGE@@#${PACKAGE}#g" \
+ -e "s#@@CHANNEL@@#${CHANNEL}#g" \
+ -e "s#@@COMPANY_FULLNAME@@#${COMPANY_FULLNAME}#g" \
+ -e "s#@@VERSION@@#${VERSION}#g" \
+ -e "s#@@REVISION@@#${REVISION}#g" \
+ -e "s#@@VERSIONFULL@@#${VERSIONFULL}#g" \
+ -e "s#@@BUILDDIR@@#${BUILDDIR}#g" \
+ -e "s#@@STAGEDIR@@#${STAGEDIR}#g" \
+ -e "s#@@SCRIPTDIR@@#${SCRIPTDIR}#g" \
+ -e "s#@@PRODUCTURL@@#${PRODUCTURL}#g" \
+ -e "s#@@PREDEPENDS@@#${PREDEPENDS}#g" \
+ -e "s#@@DEPENDS@@#${DEPENDS}#g" \
+ -e "s#@@PROVIDES@@#${PROVIDES}#g" \
+ -e "s#@@REPLACES@@#${REPLACES}#g" \
+ -e "s#@@CONFLICTS@@#${CONFLICTS}#g" \
+ -e "s#@@ARCHITECTURE@@#${HOST_ARCH}#g" \
+ -e "s#@@MAINTNAME@@#${MAINTNAME}#g" \
+ -e "s#@@MAINTMAIL@@#${MAINTMAIL}#g" \
+ -e "s#@@REPOCONFIG@@#${REPOCONFIG}#g" \
+ -e "s#@@SHORTDESC@@#${SHORTDESC}#g" \
+ -e "s#@@FULLDESC@@#${FULLDESC}#g" \
+ -e "s#@@APACHE_CONFDIR@@#${APACHE_CONFDIR}#g" \
+ -e "s#@@APACHE_MODULEDIR@@#${APACHE_MODULEDIR}#g" \
+ -e "s#@@APACHE_USER@@#${APACHE_USER}#g" \
+ -e "s#@@MODPAGESPEED_CACHE_ROOT@@#${MODPAGESPEED_CACHE_ROOT}#g" \
+ -e "s#@@MODPAGESPEED_ENABLE_UPDATES@@#${MODPAGESPEED_ENABLE_UPDATES}#g" \
+ -e "s#@@COMMENT_OUT_DEFLATE@@#${COMMENT_OUT_DEFLATE}#g" \
+ > "$TMPLOUT" <<< "$TMPLINCL"
+)
+
+# Setup the installation directory hierachy in the package staging area.
+prep_staging_common() {
+ install -m 755 -d \
+ "${STAGEDIR}${APACHE_CONFDIR}" \
+ "${STAGEDIR}${APACHE_MODULEDIR}" \
+ "${STAGEDIR}${MODPAGESPEED_CACHE_ROOT}/files" \
+ "${STAGEDIR}${MODPAGESPEED_CACHE_ROOT}/cache"
+}
+
+get_version_info() {
+ # Default to a bogus low version, so if somebody creates and installs
+ # a package with no version info, it won't prevent upgrading when
+ # trying to install a properly versioned package (i.e. a proper
+ # package will always be "newer").
+ VERSION="0.0.0.0"
+ # Use epoch timestamp so packages with bogus versions still increment
+ # and will upgrade older bogus-versioned packages.
+ REVISION=$(date +"%s")
+ # Default to non-official build since official builds set this
+ # properly.
+ OFFICIAL_BUILD=0
+
+ VERSIONFILE="${BUILDDIR}/installer/version.txt"
+ if [ -f "${VERSIONFILE}" ]; then
+ source "${VERSIONFILE}"
+ VERSION="${MAJOR}.${MINOR}.${BUILD}.${PATCH}"
+ REVISION="${LASTCHANGE}"
+ fi
+}
+
+stage_install_common() {
+ echo "Staging common install files in '${STAGEDIR}'..."
+
+ # app and resources
+ install -m 644 -s "${BUILDDIR}/libmod_pagespeed.so" "${STAGEDIR}${APACHE_MODULEDIR}/mod_pagespeed.so"
+}
diff --git a/trunk/src/install/common/mod-pagespeed/mod-pagespeed.info b/trunk/src/install/common/mod-pagespeed/mod-pagespeed.info
new file mode 100644
index 0000000..9c1e2fd
--- /dev/null
+++ b/trunk/src/install/common/mod-pagespeed/mod-pagespeed.info
@@ -0,0 +1,20 @@
+# Copyright (c) 2009 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+#
+# This file provides common configuration information for building
+# mod-pagespeed packages for various platforms.
+
+# Base name of the package.
+PACKAGE="mod-pagespeed"
+
+# Brief package description.
+SHORTDESC="Apache 2 module to optimize web content."
+
+# Detailed package description.
+FULLDESC="mod_pagespeed is an Apache module that aims to speed up load time of pages by applying web performance best practices automatically."
+
+# Package maintainer information.
+MAINTNAME="mod_pagespeed developers"
+MAINTMAIL="mod-pagespeed-dev@googlegroups.com"
+PRODUCTURL="http://code.google.com/p/modpagespeed/"
diff --git a/trunk/src/install/common/pagespeed.conf.template b/trunk/src/install/common/pagespeed.conf.template
new file mode 100644
index 0000000..505a7f0
--- /dev/null
+++ b/trunk/src/install/common/pagespeed.conf.template
@@ -0,0 +1,119 @@
+<IfModule pagespeed_module>
+ # Turn on mod_pagespeed. To completely disable mod_pagespeed, you
+ # can set this to "off".
+ ModPagespeed on
+
+ # Direct Apache to send all HTML output to the mod_pagespeed
+ # output handler.
+ AddOutputFilterByType MOD_PAGESPEED_OUTPUT_FILTER text/html
+
+ # The ModPagespeedFileCachePath and
+ # ModPagespeedGeneratedFilePrefix directories must exist and be
+ # writable by the apache user (as specified by the User
+ # directive).
+ ModPagespeedFileCachePath "@@MODPAGESPEED_CACHE_ROOT@@/cache/"
+ ModPagespeedGeneratedFilePrefix "@@MODPAGESPEED_CACHE_ROOT@@/files/"
+
+ # Override the mod_pagespeed 'rewrite level'. The default level
+ # "CoreFilters" uses a set of rewrite filters that are generally
+ # safe for most web pages. Most sites should not need to change
+ # this value and can instead fine-tune the configuration using the
+ # ModPagespeedDisableFilters and ModPagespeedEnableFilters
+ # directives, below. Valid values for ModPagespeedRewriteLevel are
+ # PassThrough, CoreFilters, TestingCoreFilters and AllFilters.
+ #
+ # ModPagespeedRewriteLevel PassThrough
+
+ # Explicitly disables specific filters. This is useful in
+ # conjuction with ModPagespeedRewriteLevel. For instance, if one
+ # of the filters in the CoreFilters needs to be disabled for a
+ # site, that filter can be added to
+ # ModPagespeedDisableFilters. This directive contains a
+ # comma-separated list of filter names, and can be repeated.
+ #
+ # ModPagespeedDisableFilters rewrite_images
+
+ # Explicitly enables specific filters. This is useful in
+ # conjuction with ModPagespeedRewriteLevel. For instance, filters
+ # not included in the CoreFilters may be enabled using this
+ # directive. This directive contains a comma-separated list of
+ # filter names, and can be repeated.
+ #
+ # ModPagespeedEnableFilters rewrite_javascript,rewrite_css
+ # ModPagespeedEnableFilters collapse_whitespace,elide_attributes
+
+ # ModPagespeedDomain
+ # authorizes rewriting of JS, CSS, and Image files found in this
+ # domain. By default only resources with the same origin as the
+ # HTML file are rewritten. For example:
+ #
+ # ModPagespeedDomain cdn.myhost.com
+ #
+ # This will allow resources found on http://cdn.myhost.com to be
+ # rewritten in addition to those in the same domain as the HTML.
+ #
+ # Wildcards (* and ?) are allowed in the domain specification. Be
+ # careful when using them as if you rewrite domains that do not
+ # send you traffic, then the site receiving the traffic will not
+ # know how to serve the rewritten content.
+
+ # Other defaults (cache sizes and thresholds):
+ # ModPagespeedFileCacheSizeKb 102400
+ # ModPagespeedFileCacheCleanIntervalMs 3600000
+ # ModPagespeedLRUCacheKbPerProcess 1024
+ # ModPagespeedLRUCacheByteLimit 16384
+ # ModPagespeedCssInlineMaxBytes 2048
+ # ModPagespeedImgInlineMaxBytes 2048
+ # ModPagespeedJsInlineMaxBytes 2048
+ # ModPagespeedCssOutlineMinBytes 3000
+ # ModPagespeedJsOutlineMinBytes 3000
+
+ # Bound the number of images that can be rewritten at any one time; this
+ # avoids overloading the CPU. Set this to 0 to remove the bound.
+ # ModPagespeedImgMaxRewritesAtOnce 8
+
+ # When Apache is set up as a browser proxy, mod_pagespeed can record
+ # web-sites as they are requested, so that an image of the web is built up
+ # in the directory of the proxy administrator's choosing. When ReadOnly is
+ # on, only files already present in the SlurpDirectory are served by the
+ # proxy.
+ # ModPagespeedSlurpDirectory ...
+ # ModPagespeedSlurpReadOnly on
+
+
+ # Enables server-side instrumentation and statistics. If this rewriter is
+ # enabled, then each rewritten HTML page will have instrumentation javacript
+ # added that sends latency beacons to /mod_pagespeed_beacon. These
+ # statistics can be accessed at /mod_pagespeed_statistics. You must also
+ # enable the mod_pagespeed_statistics and mod_pagespeed_beacon handlers
+ # below.
+ #
+ # ModPagespeedEnableFilters add_instrumentation
+
+
+ # This handles the client-side instrumentation callbacks which are injected
+ # by the add_instrumentation filter.
+ # You can use a different location by adding the ModPagespeedBeaconUrl
+ # directive; see the documentation on add_instrumentation.
+ #
+ # <Location /mod_pagespeed_beacon>
+ # SetHandler mod_pagespeed_beacon
+ # </Location>
+
+ # Uncomment the following line if you want to disable statistics entirely.
+ # ModPagespeedStatistics off
+
+ # This page lets you view statistics about the mod_pagespeed module.
+ <Location /mod_pagespeed_statistics>
+ Order allow,deny
+ # You may insert other "Allow from" lines to add hosts you want to
+ # allow to look at generated statistics. Another possibility is
+ # to comment out the "Order" and "Allow" options from the config
+ # file, to allow any client that can reach your server to examine
+ # statistics. This might be appropriate in an experimental setup or
+ # if the Apache server is protected by a reverse proxy that will
+ # filter URLs in some fashion.
+ Allow from localhost
+ SetHandler mod_pagespeed_statistics
+ </Location>
+</IfModule>
diff --git a/trunk/src/install/common/pagespeed.load.template b/trunk/src/install/common/pagespeed.load.template
new file mode 100644
index 0000000..9644d1a
--- /dev/null
+++ b/trunk/src/install/common/pagespeed.load.template
@@ -0,0 +1,6 @@
+LoadModule pagespeed_module @@APACHE_MODULEDIR@@/mod_pagespeed.so
+
+# Only attempt to load mod_deflate if it hasn't been loaded already.
+<IfModule !mod_deflate.c>
+@@COMMENT_OUT_DEFLATE@@ LoadModule deflate_module @@APACHE_MODULEDIR@@/mod_deflate.so
+</IfModule>
diff --git a/trunk/src/install/common/repo.cron b/trunk/src/install/common/repo.cron
new file mode 100644
index 0000000..fbb3e74
--- /dev/null
+++ b/trunk/src/install/common/repo.cron
@@ -0,0 +1,42 @@
+#!/bin/sh
+#
+# Copyright (c) 2009 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+#
+# This script is part of the @@PACKAGE@@ package.
+#
+# It creates the repository configuration file for package updates, and it
+# monitors that config to see if it has been disabled by the overly aggressive
+# distro upgrade process (e.g. intrepid -> jaunty). When this situation is
+# detected, the respository will be re-enabled. If the respository is disabled
+# for any other reason, this won't re-enable it.
+#
+# This functionality can be controlled by creating the $DEFAULTS_FILE and
+# setting "repo_add_once" and/or "repo_reenable_on_distupgrade" to "true" or
+# "false" as desired. An empty $DEFAULTS_FILE is the same as setting both values
+# to "false".
+
+@@include@@apt.include
+
+## MAIN ##
+DEFAULTS_FILE="/etc/default/@@PACKAGE@@"
+if [ -r "$DEFAULTS_FILE" ]; then
+ . "$DEFAULTS_FILE"
+fi
+
+if [ "$repo_add_once" = "true" ]; then
+ install_key
+ create_sources_lists
+ RES=$?
+ # Sources creation succeeded, so stop trying.
+ if [ $RES -ne 2 ]; then
+ sed -i -e 's/[[:space:]]*repo_add_once=.*/repo_add_once="false"/' "$DEFAULTS_FILE"
+ fi
+else
+ update_bad_sources
+fi
+
+if [ "$repo_reenable_on_distupgrade" = "true" ]; then
+ handle_distro_upgrade
+fi
diff --git a/trunk/src/install/common/rpm.include b/trunk/src/install/common/rpm.include
new file mode 100644
index 0000000..33d33a5
--- /dev/null
+++ b/trunk/src/install/common/rpm.include
@@ -0,0 +1,305 @@
+@@include@@variables.include
+
+# Install the repository signing key (see also:
+# http://www.google.com/linuxrepositories/aboutkey.html)
+install_rpm_key() {
+ # Check to see if key already exists.
+ rpm -q gpg-pubkey-7fac5991-4615767f > /dev/null 2>&1
+ if [ "$?" -eq "0" ]; then
+ # Key already exists
+ return 0
+ fi
+ # This is to work around a bug in RPM 4.7.0. (see http://crbug.com/22312)
+ rpm -q gpg-pubkey-7fac5991-45f06f46 > /dev/null 2>&1
+ if [ "$?" -eq "0" ]; then
+ # Key already exists
+ return 0
+ fi
+
+ # RPM on Mandriva 2009 is dumb and does not understand "rpm --import -"
+ TMPKEY=$(mktemp /tmp/google.sig.XXXXXX)
+ if [ -n "$TMPKEY" ]; then
+ cat > "$TMPKEY" <<KEYDATA
+-----BEGIN PGP PUBLIC KEY BLOCK-----
+Version: GnuPG v1.4.2.2 (GNU/Linux)
+
+mQGiBEXwb0YRBADQva2NLpYXxgjNkbuP0LnPoEXruGmvi3XMIxjEUFuGNCP4Rj/a
+kv2E5VixBP1vcQFDRJ+p1puh8NU0XERlhpyZrVMzzS/RdWdyXf7E5S8oqNXsoD1z
+fvmI+i9b2EhHAA19Kgw7ifV8vMa4tkwslEmcTiwiw8lyUl28Wh4Et8SxzwCggDcA
+feGqtn3PP5YAdD0km4S4XeMEAJjlrqPoPv2Gf//tfznY2UyS9PUqFCPLHgFLe80u
+QhI2U5jt6jUKN4fHauvR6z3seSAsh1YyzyZCKxJFEKXCCqnrFSoh4WSJsbFNc4PN
+b0V0SqiTCkWADZyLT5wll8sWuQ5ylTf3z1ENoHf+G3um3/wk/+xmEHvj9HCTBEXP
+78X0A/0Tqlhc2RBnEf+AqxWvM8sk8LzJI/XGjwBvKfXe+l3rnSR2kEAvGzj5Sg0X
+4XmfTg4Jl8BNjWyvm2Wmjfet41LPmYJKsux3g0b8yzQxeOA4pQKKAU3Z4+rgzGmf
+HdwCG5MNT2A5XxD/eDd+L4fRx0HbFkIQoAi1J3YWQSiTk15fw7RMR29vZ2xlLCBJ
+bmMuIExpbnV4IFBhY2thZ2UgU2lnbmluZyBLZXkgPGxpbnV4LXBhY2thZ2VzLWtl
+eW1hc3RlckBnb29nbGUuY29tPohjBBMRAgAjAhsDBgsJCAcDAgQVAggDBBYCAwEC
+HgECF4AFAkYVdn8CGQEACgkQoECDD3+sWZHKSgCfdq3HtNYJLv+XZleb6HN4zOcF
+AJEAniSFbuv8V5FSHxeRimHx25671az+uQINBEXwb0sQCACuA8HT2nr+FM5y/kzI
+A51ZcC46KFtIDgjQJ31Q3OrkYP8LbxOpKMRIzvOZrsjOlFmDVqitiVc7qj3lYp6U
+rgNVaFv6Qu4bo2/ctjNHDDBdv6nufmusJUWq/9TwieepM/cwnXd+HMxu1XBKRVk9
+XyAZ9SvfcW4EtxVgysI+XlptKFa5JCqFM3qJllVohMmr7lMwO8+sxTWTXqxsptJo
+pZeKz+UBEEqPyw7CUIVYGC9ENEtIMFvAvPqnhj1GS96REMpry+5s9WKuLEaclWpd
+K3krttbDlY1NaeQUCRvBYZ8iAG9YSLHUHMTuI2oea07Rh4dtIAqPwAX8xn36JAYG
+2vgLAAMFB/wKqaycjWAZwIe98Yt0qHsdkpmIbarD9fGiA6kfkK/UxjL/k7tmS4Vm
+CljrrDZkPSQ/19mpdRcGXtb0NI9+nyM5trweTvtPw+HPkDiJlTaiCcx+izg79Fj9
+KcofuNb3lPdXZb9tzf5oDnmm/B+4vkeTuEZJ//IFty8cmvCpzvY+DAz1Vo9rA+Zn
+cpWY1n6z6oSS9AsyT/IFlWWBZZ17SpMHu+h4Bxy62+AbPHKGSujEGQhWq8ZRoJAT
+G0KSObnmZ7FwFWu1e9XFoUCt0bSjiJWTIyaObMrWu/LvJ3e9I87HseSJStfw6fki
+5og9qFEkMrIrBCp3QGuQWBq/rTdMuwNFiEkEGBECAAkFAkXwb0sCGwwACgkQoECD
+D3+sWZF/WACfeNAu1/1hwZtUo1bR+MWiCjpvHtwAnA1R3IHqFLQ2X3xJ40XPuAyY
+/FJG
+=Quqp
+-----END PGP PUBLIC KEY BLOCK-----
+KEYDATA
+ rpm --import "$TMPKEY"
+ rc=$?
+ rm -f "$TMPKEY"
+ if [ "$rc" -eq "0" ]; then
+ return 0
+ fi
+ fi
+ return 1
+}
+
+determine_rpm_package_manager() {
+ local RELEASE
+ LSB_RELEASE="$(which lsb_release 2> /dev/null)"
+ if [ -x "$LSB_RELEASE" ]; then
+ RELEASE=$(lsb_release -i 2> /dev/null)
+ case $DISTRIB_ID in
+ "Fedora")
+ PACKAGEMANAGER=yum
+ ;;
+ "MandrivaLinux")
+ PACKAGEMANAGER=urpmi
+ ;;
+ "SUSE LINUX")
+ PACKAGEMANAGER=yast
+ ;;
+ esac
+ fi
+
+ if [ "$PACKAGEMANAGER" ]; then
+ return
+ fi
+
+ # Fallback methods that are probably unnecessary on modern systems.
+ if [ -f "/etc/lsb-release" ]; then
+ # file missing on Fedora, does not contain DISTRIB_ID on OpenSUSE.
+ eval $(sed -e '/DISTRIB_ID/!d' /etc/lsb-release)
+ case $DISTRIB_ID in
+ MandrivaLinux)
+ PACKAGEMANAGER=urpmi
+ ;;
+ esac
+ fi
+
+ if [ "$PACKAGEMANAGER" ]; then
+ return
+ fi
+
+ if [ -f "/etc/fedora-release" ] || [ -f "/etc/redhat-release" ]; then
+ PACKAGEMANAGER=yum
+ elif [ -f "/etc/SuSE-release" ]; then
+ PACKAGEMANAGER=yast
+ elif [ -f "/etc/mandriva-release" ]; then
+ PACKAGEMANAGER=urpmi
+ fi
+}
+
+DEFAULT_ARCH="@@ARCHITECTURE@@"
+YUM_REPO_FILE="/etc/yum.repos.d/@@PACKAGE@@.repo"
+ZYPPER_REPO_FILE="/etc/zypp/repos.d/@@PACKAGE@@.repo"
+URPMI_REPO_FILE="/etc/urpmi/urpmi.cfg"
+
+install_yum() {
+ install_rpm_key
+
+ if [ ! "$REPOCONFIG" ]; then
+ return 0
+ fi
+
+ if [ -d "/etc/yum.repos.d" ]; then
+cat > "$YUM_REPO_FILE" << REPOCONTENT
+[@@PACKAGE@@]
+name=@@PACKAGE@@
+baseurl=$REPOCONFIG/$DEFAULT_ARCH
+enabled=1
+gpgcheck=1
+REPOCONTENT
+ fi
+}
+
+# This is called by the cron job, rather than in the RPM postinstall.
+# We cannot do this during the install when urpmi is running due to
+# database locking. We also need to enable the repository, and we can
+# only do that while we are online.
+# see: https://qa.mandriva.com/show_bug.cgi?id=31893
+configure_urpmi() {
+ if [ ! "$REPOCONFIG" ]; then
+ return 0
+ fi
+
+ urpmq --list-media | grep -q -s "^@@PACKAGE@@$"
+ if [ "$?" -eq "0" ]; then
+ # Repository already configured
+ return 0
+ fi
+ urpmi.addmedia --update \
+ "@@PACKAGE@@" "$REPOCONFIG/$DEFAULT_ARCH"
+}
+
+install_urpmi() {
+ # urpmi not smart enough to pull media_info/pubkey from the repository?
+ install_rpm_key
+
+ # Defer urpmi.addmedia to configure_urpmi() in the cron job.
+ # See comment there.
+ #
+ # urpmi.addmedia --update \
+ # "@@PACKAGE@@" "$REPOCONFIG/$DEFAULT_ARCH"
+}
+
+install_yast() {
+ if [ ! "$REPOCONFIG" ]; then
+ return 0
+ fi
+
+ # We defer adding the key to later. See comment in the cron job.
+
+ # Ideally, we would run: zypper addrepo -t YUM -f \
+ # "$REPOCONFIG/$DEFAULT_ARCH" "@@PACKAGE@@"
+ # but that does not work when zypper is running.
+ if [ -d "/etc/zypp/repos.d" ]; then
+cat > "$ZYPPER_REPO_FILE" << REPOCONTENT
+[@@PACKAGE@@]
+name=@@PACKAGE@@
+enabled=1
+autorefresh=1
+baseurl=$REPOCONFIG/$DEFAULT_ARCH
+type=rpm-md
+keeppackages=0
+REPOCONTENT
+ fi
+}
+
+# Check if the automatic repository configuration is done, so we know when to
+# stop trying.
+verify_install() {
+ # It's probably enough to see that the repo configs have been created. If they
+ # aren't configured properly, update_bad_repo should catch that when it's run.
+ case $1 in
+ "yum")
+ [ -f "$YUM_REPO_FILE" ]
+ ;;
+ "yast")
+ [ -f "$ZYPPER_REPO_FILE" ]
+ ;;
+ "urpmi")
+ urpmq --list-url | grep -q -s "\b@@PACKAGE@@\b"
+ ;;
+ esac
+}
+
+# Update the Google repository if it's not set correctly.
+update_bad_repo() {
+ if [ ! "$REPOCONFIG" ]; then
+ return 0
+ fi
+
+ determine_rpm_package_manager
+
+ case $PACKAGEMANAGER in
+ "yum")
+ update_repo_file "$YUM_REPO_FILE"
+ ;;
+ "yast")
+ update_repo_file "$ZYPPER_REPO_FILE"
+ ;;
+ "urpmi")
+ update_urpmi_cfg
+ ;;
+ esac
+}
+
+update_repo_file() {
+ REPO_FILE="$1"
+
+ # Don't do anything if the file isn't there, since that probably means the
+ # user disabled it.
+ if [ ! -r "$REPO_FILE" ]; then
+ return 0
+ fi
+
+ # Check if the correct repository configuration is in there.
+ REPOMATCH=$(grep "^baseurl=$REPOCONFIG/$DEFAULT_ARCH" "$REPO_FILE" \
+ 2>/dev/null)
+ # If it's there, nothing to do
+ if [ "$REPOMATCH" ]; then
+ return 0
+ fi
+
+ # Check if it's there but disabled by commenting out (as opposed to using the
+ # 'enabled' setting).
+ MATCH_DISABLED=$(grep "^[[:space:]]*#.*baseurl=$REPOCONFIG/$DEFAULT_ARCH" \
+ "$REPO_FILE" 2>/dev/null)
+ if [ "$MATCH_DISABLED" ]; then
+ # It's OK for it to be disabled, as long as nothing bogus is enabled in its
+ # place.
+ ACTIVECONFIGS=$(grep "^baseurl=.*" "$REPO_FILE" 2>/dev/null)
+ if [ ! "$ACTIVECONFIGS" ]; then
+ return 0
+ fi
+ fi
+
+ # If we get here, the correct repository wasn't found, or something else is
+ # active, so fix it. This assumes there is a 'baseurl' setting, but if not,
+ # then that's just another way of disabling, so we won't try to add it.
+ sed -i -e "s,^baseurl=.*,baseurl=$REPOCONFIG/$DEFAULT_ARCH," "$REPO_FILE"
+}
+
+update_urpmi_cfg() {
+ REPOCFG=$(urpmq --list-url | grep "\b@@PACKAGE@@\b")
+ if [ ! "$REPOCFG" ]; then
+ # Don't do anything if the repo isn't there, since that probably means the
+ # user deleted it.
+ return 0
+ fi
+
+ # See if it's the right repo URL
+ REPOMATCH=$(echo "$REPOCFG" | grep "\b$REPOCONFIG/$DEFAULT_ARCH\b")
+ # If so, nothing to do
+ if [ "$REPOMATCH" ]; then
+ return 0
+ fi
+
+ # Looks like it's the wrong URL, so recreate it.
+ urpmi.removemedia "@@PACKAGE@@" && \
+ urpmi.addmedia --update "@@PACKAGE@@" "$REPOCONFIG/$DEFAULT_ARCH"
+}
+
+# We only remove the repository configuration during a purge. Since RPM has
+# no equivalent to dpkg --purge, the code below is actually never used. We
+# keep it only for reference purposes, should we ever need it.
+#
+#remove_yum() {
+# rm -f "$YUM_REPO_FILE"
+#}
+#
+#remove_urpmi() {
+# # Ideally, we would run: urpmi.removemedia "@@PACKAGE@@"
+# # but that does not work when urpmi is running.
+# # Sentinel comment text does not work either because urpmi.update removes
+# # all comments. So we just delete the entry that matches what we originally
+# # inserted. If such an entry was added manually, that's tough luck.
+# if [ -f "$URPMI_REPO_FILE" ]; then
+# sed -i '\_^@@PACKAGE@@ $REPOCONFIG/$DEFAULT_ARCH {$_,/^}$/d' "$URPMI_REPO_FILE"
+# fi
+#}
+#
+#remove_yast() {
+# # Ideally, we would run: zypper removerepo "@@PACKAGE@@"
+# # but that does not work when zypper is running.
+# rm -f /etc/zypp/repos.d/@@PACKAGE@@.repo
+#}
diff --git a/trunk/src/install/common/rpmrepo.cron b/trunk/src/install/common/rpmrepo.cron
new file mode 100644
index 0000000..8b0043a
--- /dev/null
+++ b/trunk/src/install/common/rpmrepo.cron
@@ -0,0 +1,56 @@
+#!/bin/sh
+#
+# Copyright (c) 2009 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+#
+# This script is part of the @@PACKAGE@@ package.
+#
+# It creates the repository configuration file for package updates, since
+# we cannot do this during the @@PACKAGE@@ installation since the repository
+# is locked.
+#
+# This functionality can be controlled by creating the $DEFAULTS_FILE and
+# setting "repo_add_once" to "true" or "false" as desired. An empty
+# $DEFAULTS_FILE is the same as setting the value to "false".
+
+@@include@@rpm.include
+
+## MAIN ##
+DEFAULTS_FILE="/etc/default/@@PACKAGE@@"
+if [ -r "$DEFAULTS_FILE" ]; then
+ . "$DEFAULTS_FILE"
+fi
+
+if [ "$repo_add_once" = "true" ]; then
+ determine_rpm_package_manager
+
+ case $PACKAGEMANAGER in
+ "urpmi")
+ # We need to configure urpmi after the install has finished.
+ # See configure_urpmi() for details.
+ configure_urpmi
+ ;;
+ "yast")
+ # It looks as though yast/zypper has a lock on the RPM DB during
+ # postinstall, so we cannot add the signing key with install_rpm_key().
+ # Instead, we attempt to do this here. If the user attempt to update before
+ # the cron job imports the key, Yast will grab the key from our server and
+ # prompt the user to accept the key.
+ install_rpm_key
+ ;;
+ esac
+
+ if [ $? -eq 0 ]; then
+ # Before we quit auto-configuration, check that everything looks sane, since
+ # part of this happened during package install and we don't have the return
+ # value of that process.
+ verify_install $PACKAGEMANAGER
+ if [ $? -eq 0 ]; then
+ sed -i -e 's/[[:space:]]*repo_add_once=.*/repo_add_once="false"/' \
+ "$DEFAULTS_FILE"
+ fi
+ fi
+else
+ update_bad_repo
+fi
diff --git a/trunk/src/install/common/updater b/trunk/src/install/common/updater
new file mode 100755
index 0000000..53a8a80
--- /dev/null
+++ b/trunk/src/install/common/updater
@@ -0,0 +1,26 @@
+#!/bin/sh
+#
+# Copyright (c) 2009 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+# TODO
+# - handle other distros (e.g. non-apt).
+
+@@include@@apt.include
+
+if [ -x "$APT_GET" ]; then
+ update_sources_lists
+ # If the repo was just added, force a cache update.
+ if [ $? -eq 1 ]; then
+ install_key
+ "$APT_GET" -qq update
+ fi
+
+ # TODO(mmoss) detect if apt cache is stale (> 1 day) and force update?
+
+ # Just try to install the packge. If it's already installed, apt-get won't do
+ # anything.
+ "$APT_GET" install -y -q @@PACKAGE@@
+fi
+
diff --git a/trunk/src/install/common/variables.include b/trunk/src/install/common/variables.include
new file mode 100644
index 0000000..f3a17cd
--- /dev/null
+++ b/trunk/src/install/common/variables.include
@@ -0,0 +1,5 @@
+# System-wide package configuration.
+DEFAULTS_FILE="/etc/default/@@PACKAGE@@"
+
+# sources.list setting for @@PACKAGE@@ updates.
+REPOCONFIG="@@REPOCONFIG@@"
diff --git a/trunk/src/install/debian/build.sh b/trunk/src/install/debian/build.sh
new file mode 100755
index 0000000..e4f2cb3
--- /dev/null
+++ b/trunk/src/install/debian/build.sh
@@ -0,0 +1,247 @@
+#!/bin/bash
+#
+# Copyright (c) 2009 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+set -e
+if [ "$VERBOSE" ]; then
+ set -x
+fi
+set -u
+
+# Create the Debian changelog file needed by dpkg-gencontrol. This just adds a
+# placeholder change, indicating it is the result of an automatic build.
+gen_changelog() {
+ rm -f "${DEB_CHANGELOG}"
+ process_template "${SCRIPTDIR}/changelog.template" "${DEB_CHANGELOG}"
+ debchange -a --nomultimaint -m --changelog "${DEB_CHANGELOG}" \
+ --distribution UNRELEASED "automatic build"
+}
+
+# Create the Debian control file needed by dpkg-deb.
+gen_control() {
+ dpkg-gencontrol -v"${VERSIONFULL}" -c"${DEB_CONTROL}" -l"${DEB_CHANGELOG}" \
+ -f"${DEB_FILES}" -p"${PACKAGE}-${CHANNEL}" -P"${STAGEDIR}" -T"${DEB_SUBST}" \
+ -O > "${STAGEDIR}/DEBIAN/control"
+ rm -f "${DEB_CONTROL}"
+}
+
+# Create the Debian substvars file needed by dpkg-gencontrol.
+gen_substvars() {
+ # dpkg-shlibdeps requires a control file in debian/control, so we're
+ # forced to prepare a fake debian directory.
+ mkdir "${SUBSTFILEDIR}/debian"
+ cp "${DEB_CONTROL}" "${SUBSTFILEDIR}/debian"
+ pushd "${SUBSTFILEDIR}" >/dev/null
+ dpkg-shlibdeps "${STAGEDIR}${APACHE_MODULEDIR}/mod_pagespeed.so" \
+ -O >> "${DEB_SUBST}" 2>/dev/null
+ popd >/dev/null
+}
+
+# Setup the installation directory hierachy in the package staging area.
+prep_staging_debian() {
+ prep_staging_common
+ install -m 755 -d "${STAGEDIR}/DEBIAN" \
+ "${STAGEDIR}/etc/cron.daily"
+}
+
+# Put the package contents in the staging area.
+stage_install_debian() {
+ prep_staging_debian
+ stage_install_common
+ echo "Staging Debian install files in '${STAGEDIR}'..."
+ process_template "${BUILDDIR}/install/common/repo.cron" \
+ "${STAGEDIR}/etc/cron.daily/${PACKAGE}"
+ chmod 755 "${STAGEDIR}/etc/cron.daily/${PACKAGE}"
+ process_template "${BUILDDIR}/install/debian/postinst" \
+ "${STAGEDIR}/DEBIAN/postinst"
+ chmod 755 "${STAGEDIR}/DEBIAN/postinst"
+ process_template "${BUILDDIR}/install/debian/prerm" \
+ "${STAGEDIR}/DEBIAN/prerm"
+ chmod 755 "${STAGEDIR}/DEBIAN/prerm"
+ process_template "${BUILDDIR}/install/debian/postrm" \
+ "${STAGEDIR}/DEBIAN/postrm"
+ chmod 755 "${STAGEDIR}/DEBIAN/postrm"
+ install -m 644 "${BUILDDIR}/install/debian/conffiles" \
+ "${STAGEDIR}/DEBIAN/conffiles"
+ process_template "${BUILDDIR}/install/common/pagespeed.load.template" \
+ "${STAGEDIR}${APACHE_CONFDIR}/pagespeed.load"
+ chmod 644 "${STAGEDIR}${APACHE_CONFDIR}/pagespeed.load"
+ process_template "${BUILDDIR}/install/common/pagespeed.conf.template" \
+ "${STAGEDIR}${APACHE_CONFDIR}/pagespeed.conf"
+ chmod 644 "${STAGEDIR}${APACHE_CONFDIR}/pagespeed.conf"
+}
+
+# Build the deb file within a fakeroot.
+do_package_in_fakeroot() {
+ FAKEROOTFILE=$(mktemp -t fakeroot.tmp.XXXXXX) || exit 1
+ fakeroot -s "${FAKEROOTFILE}" -- \
+ chown -R ${APACHE_USER}:${APACHE_USER} ${STAGEDIR}${MODPAGESPEED_CACHE_ROOT}
+ fakeroot -i "${FAKEROOTFILE}" -- \
+ dpkg-deb -b "${STAGEDIR}" .
+ rm -f "${FAKEROOTFILE}"
+}
+
+# Actually generate the package file.
+do_package() {
+ export HOST_ARCH="$1"
+ echo "Packaging ${HOST_ARCH}..."
+ PREDEPENDS="$COMMON_PREDEPS"
+ DEPENDS="${COMMON_DEPS}"
+ gen_changelog
+ process_template "${SCRIPTDIR}/control.template" "${DEB_CONTROL}"
+ export DEB_HOST_ARCH="${HOST_ARCH}"
+ gen_substvars
+ if [ -f "${DEB_CONTROL}" ]; then
+ gen_control
+ fi
+
+ do_package_in_fakeroot
+}
+
+# Remove temporary files and unwanted packaging output.
+cleanup() {
+ echo "Cleaning..."
+ rm -rf "${STAGEDIR}"
+ rm -rf "${TMPFILEDIR}"
+ rm -rf "${SUBSTFILEDIR}"
+}
+
+usage() {
+ echo "usage: $(basename $0) [-c channel] [-a target_arch] [-o 'dir'] [-b 'dir']"
+ echo "-c channel the package channel (unstable, beta, stable)"
+ echo "-a arch package architecture (ia32 or x64)"
+ echo "-o dir package output directory [${OUTPUTDIR}]"
+ echo "-b dir build input directory [${BUILDDIR}]"
+ echo "-h this help message"
+}
+
+# Check that the channel name is one of the allowable ones.
+verify_channel() {
+ case $CHANNEL in
+ stable )
+ CHANNEL=stable
+ ;;
+ unstable|dev|alpha )
+ CHANNEL=unstable
+ ;;
+ testing|beta )
+ CHANNEL=beta
+ ;;
+ * )
+ echo
+ echo "ERROR: '$CHANNEL' is not a valid channel type."
+ echo
+ exit 1
+ ;;
+ esac
+}
+
+process_opts() {
+ while getopts ":o:b:c:a:h" OPTNAME
+ do
+ case $OPTNAME in
+ o )
+ OUTPUTDIR="$OPTARG"
+ mkdir -p "${OUTPUTDIR}"
+ ;;
+ b )
+ BUILDDIR=$(readlink -f "${OPTARG}")
+ ;;
+ c )
+ CHANNEL="$OPTARG"
+ ;;
+ a )
+ TARGETARCH="$OPTARG"
+ ;;
+ h )
+ usage
+ exit 0
+ ;;
+ \: )
+ echo "'-$OPTARG' needs an argument."
+ usage
+ exit 1
+ ;;
+ * )
+ echo "invalid command-line option: $OPTARG"
+ usage
+ exit 1
+ ;;
+ esac
+ done
+}
+
+#=========
+# MAIN
+#=========
+
+SCRIPTDIR=$(readlink -f "$(dirname "$0")")
+OUTPUTDIR="${PWD}"
+STAGEDIR=$(mktemp -d -t deb.build.XXXXXX) || exit 1
+TMPFILEDIR=$(mktemp -d -t deb.tmp.XXXXXX) || exit 1
+SUBSTFILEDIR=$(mktemp -d -t deb.subst.XXXXXX) || exit 1
+DEB_CHANGELOG="${TMPFILEDIR}/changelog"
+DEB_FILES="${TMPFILEDIR}/files"
+DEB_CONTROL="${TMPFILEDIR}/control"
+DEB_SUBST="${SUBSTFILEDIR}/debian/substvars"
+CHANNEL="beta"
+# Default target architecture to same as build host.
+if [ "$(uname -m)" = "x86_64" ]; then
+ TARGETARCH="x64"
+else
+ TARGETARCH="ia32"
+fi
+
+# call cleanup() on exit
+trap cleanup 0
+process_opts "$@"
+if [ ! "$BUILDDIR" ]; then
+ BUILDDIR=$(readlink -f "${SCRIPTDIR}/../../out/Release")
+fi
+
+source ${BUILDDIR}/install/common/installer.include
+
+get_version_info
+VERSIONFULL="${VERSION}-r${REVISION}"
+
+source "${BUILDDIR}/install/common/mod-pagespeed.info"
+eval $(sed -e "s/^\([^=]\+\)=\(.*\)$/export \1='\2'/" \
+ "${BUILDDIR}/install/common/BRANDING")
+
+REPOCONFIG="deb http://dl.google.com/linux/${PACKAGE#google-}/deb/ stable main"
+verify_channel
+
+# Some Debian packaging tools want these set.
+export DEBFULLNAME="${MAINTNAME}"
+export DEBEMAIL="${MAINTMAIL}"
+
+# Make everything happen in the OUTPUTDIR.
+cd "${OUTPUTDIR}"
+
+COMMON_DEPS="apache2.2-common"
+COMMON_PREDEPS="dpkg (>= 1.14.0)"
+
+APACHE_MODULEDIR="/usr/lib/apache2/modules"
+APACHE_CONFDIR="/etc/apache2/mods-available"
+MODPAGESPEED_CACHE_ROOT="/var/mod_pagespeed"
+APACHE_USER="www-data"
+COMMENT_OUT_DEFLATE=
+
+case "$TARGETARCH" in
+ ia32 )
+ stage_install_debian
+ do_package "i386"
+ ;;
+ x64 )
+ stage_install_debian
+ do_package "amd64"
+ ;;
+ * )
+ echo
+ echo "ERROR: Don't know how to build DEBs for '$TARGETARCH'."
+ echo
+ exit 1
+ ;;
+esac
diff --git a/trunk/src/install/debian/changelog.template b/trunk/src/install/debian/changelog.template
new file mode 100644
index 0000000..4ed22e5
--- /dev/null
+++ b/trunk/src/install/debian/changelog.template
@@ -0,0 +1,4 @@
+@@PACKAGE@@-@@CHANNEL@@ (@@VERSIONFULL@@) UNRELEASED; urgency=low
+ * No changes
+
+ -- @@MAINTNAME@@ <@@MAINTMAIL@@> Wed, 20 Oct 2010 14:54:35 -0800
diff --git a/trunk/src/install/debian/conffiles b/trunk/src/install/debian/conffiles
new file mode 100644
index 0000000..230698b
--- /dev/null
+++ b/trunk/src/install/debian/conffiles
@@ -0,0 +1,2 @@
+/etc/apache2/mods-available/pagespeed.load
+/etc/apache2/mods-available/pagespeed.conf
diff --git a/trunk/src/install/debian/control.template b/trunk/src/install/debian/control.template
new file mode 100644
index 0000000..cc9b1d0
--- /dev/null
+++ b/trunk/src/install/debian/control.template
@@ -0,0 +1,16 @@
+Source: @@PACKAGE@@-@@CHANNEL@@
+Section: httpd
+Priority: optional
+Maintainer: @@MAINTNAME@@ <@@MAINTMAIL@@>
+Build-Depends: dpkg-dev, devscripts, fakeroot
+Standards-Version: 3.8.0
+
+Package: @@PACKAGE@@-@@CHANNEL@@
+Provides: @@PROVIDES@@
+Replaces: @@REPLACES@@
+Conflicts: @@CONFLICTS@@
+Pre-Depends: @@PREDEPENDS@@
+Depends: ${shlibs:Depends}, @@DEPENDS@@
+Architecture: @@ARCHITECTURE@@
+Description: @@SHORTDESC@@
+ @@FULLDESC@@
diff --git a/trunk/src/install/debian/postinst b/trunk/src/install/debian/postinst
new file mode 100755
index 0000000..370207d
--- /dev/null
+++ b/trunk/src/install/debian/postinst
@@ -0,0 +1,31 @@
+#!/bin/sh
+
+# Based on postinst from Chromium and Google Talk.
+
+@@include@@../common/apt.include
+
+MODPAGESPEED_ENABLE_UPDATES=@@MODPAGESPEED_ENABLE_UPDATES@@
+
+case "$1" in
+ configure)
+ if [ -n "${MODPAGESPEED_ENABLE_UPDATES}" -a ! -e "$DEFAULTS_FILE" ]; then
+ echo 'repo_add_once="true"' > "$DEFAULTS_FILE"
+ echo 'repo_reenable_on_distupgrade="true"' >> "$DEFAULTS_FILE"
+ fi
+
+ # Run the cron job immediately to perform repository
+ # configuration.
+ nohup sh /etc/cron.daily/@@PACKAGE@@ > /dev/null 2>&1 &
+
+ test ! -e /etc/apache2/mods-enabled/pagespeed.load && \
+ a2enmod pagespeed
+ ;;
+ abort-upgrade|abort-remove|abort-deconfigure)
+ ;;
+ *)
+ echo "postinst called with unknown argument \`$1'" >&2
+ exit 1
+ ;;
+esac
+
+exit 0
diff --git a/trunk/src/install/debian/postrm b/trunk/src/install/debian/postrm
new file mode 100755
index 0000000..0dc3058
--- /dev/null
+++ b/trunk/src/install/debian/postrm
@@ -0,0 +1,27 @@
+#!/bin/sh
+#
+# Copyright (c) 2009 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+action="$1"
+
+# Only do complete clean-up on purge.
+if [ "$action" != "purge" ] ; then
+ exit 0
+fi
+
+@@include@@../common/apt.include
+
+# Only remove the defaults file if it is not empty. An empty file was probably
+# put there by the sysadmin to disable automatic repository configuration, as
+# per the instructions on the package download page.
+if [ -s "$DEFAULTS_FILE" ]; then
+ # Make sure the package defaults are removed before the repository config,
+ # otherwise it could result in the repository config being removed, but the
+ # package defaults remain and are set to not recreate the repository config.
+ # In that case, future installs won't recreate it and won't get auto-updated.
+ rm "$DEFAULTS_FILE" || exit 1
+fi
+# Remove any Google repository added by the package.
+clean_sources_lists
diff --git a/trunk/src/install/debian/prerm b/trunk/src/install/debian/prerm
new file mode 100755
index 0000000..d9ab1f9
--- /dev/null
+++ b/trunk/src/install/debian/prerm
@@ -0,0 +1,15 @@
+#!/bin/sh
+
+case "$1" in
+ remove)
+ test -e /etc/apache2/mods-enabled/pagespeed.load && a2dismod pagespeed
+ ;;
+ upgrade|deconfigure|failed-upgrade)
+ ;;
+ *)
+ echo "prerm called with unknown argument \`$1'" >&2
+ exit 1
+ ;;
+esac
+
+exit 0
diff --git a/trunk/src/install/debug.conf.template b/trunk/src/install/debug.conf.template
new file mode 100644
index 0000000..d5722e7
--- /dev/null
+++ b/trunk/src/install/debug.conf.template
@@ -0,0 +1,36 @@
+# These caching headers are set up for the mod_pagespeed example, and
+# also serve as a demonstration of good values to set for the entire
+# site, if it is to be optimized by mod_pagespeed.
+<Directory APACHE_DOC_ROOT/mod_pagespeed_example>
+ <IfModule headers_module>
+ # To enable to show that mod_pagespeed to rewrites web pages, we must
+ # turn off Etags for HTML files and eliminate caching altogether.
+ # mod_pagespeed should rewrite HTML files each time they are served.
+ # The first time mod_pagespeed sees an HTML file, it may not optimize
+ # it fully. It will optimize better after the second view. Caching
+ # defeats this behavior.
+ <FilesMatch "\.(html|htm)$">
+ Header unset Etag
+ Header set Cache-control "max-age=0, no-cache, no-store"
+ </FilesMatch>
+
+ # Images, styles, and javascript are all cache-extended for
+ # a year by rewriting URLs to include a content hash. mod_pagespeed
+ # can only do this if the resources are cacheable in the first place.
+ # The origin caching policy, set here to 10 minutes, dictates how
+ # frequently mod_pagespeed must re-read the content files and recompute
+ # the content-hash. As long as the content doesn't actually change,
+ # the content-hash will remain the same, and the resources stored
+ # in browser caches will stay relevant.
+ <FilesMatch "\.(jpg|jpeg|gif|png|js|css)$">
+ Header unset Etag
+ Header set Cache-control "public, max-age=600"
+ </FilesMatch>
+ </IfModule>
+</Directory>
+
+#STRESS # These lines are only needed for the stress test.
+#STRESS <Directory /usr/local/apache2/htdocs/mod_pagespeed_example/cgi>
+#STRESS Options +ExecCGI
+#STRESS </Directory>
+#STRESS AddHandler cgi-script .cgi
diff --git a/trunk/src/install/install_apxs.sh b/trunk/src/install/install_apxs.sh
new file mode 100644
index 0000000..d60a5c6
--- /dev/null
+++ b/trunk/src/install/install_apxs.sh
@@ -0,0 +1,265 @@
+#!/bin/bash
+#
+# Install Page Speed, using the Apache apxs tool to determine the
+# installation locations.
+#
+# You can specify the path to apxs to install to a non-system-default
+# Apache.
+#
+# APXS_BIN=/path/to/apxs ./install_apxs.sh
+#
+# To install to a location that does not require superuser access:
+#
+# NO_SUDO=1 ./install_apxs.sh
+
+SRC_ROOT="$(dirname $0)/.."
+BUILD_ROOT="${SRC_ROOT}/out/Release"
+MODPAGESPEED_SO_PATH="${BUILD_ROOT}/libmod_pagespeed.so"
+
+MODPAGESPEED_CACHE_ROOT=${MODPAGESPEED_CACHE_ROOT:-"/var/mod_pagespeed"}
+APACHE_CONF_FILENAME=${APACHE_CONF_FILENAME:-"httpd.conf"}
+MODPAGESPEED_SO_NAME=${MODPAGESPEED_SO_NAME:-"mod_pagespeed.so"}
+MODPAGESPEED_CONF_NAME=${MODPAGESPEED_CONF_NAME:-"pagespeed.conf"}
+
+MODPAGESPEED_FILE_USER=${MODPAGESPEED_FILE_USER:-"root"}
+MODPAGESPEED_FILE_GROUP=${MODPAGESPEED_FILE_GROUP:-${MODPAGESPEED_FILE_USER}}
+SUDO_CMD=${SUDO_CMD:-"sudo"}
+
+# If NO_SUDO was specified, then we should use a user and group that
+# matches the current user and group when installing files.
+if [ ! -z "${NO_SUDO}" ]; then
+ MODPAGESPEED_FILE_USER=$USER
+ MODPAGESPEED_FILE_GROUP=$(groups | cut -d\ -f1)
+ SUDO_CMD=""
+fi
+
+# Load the script used to perform template substitutions on our config
+# files.
+source ${SRC_ROOT}/install/common/installer.include
+
+# Args: variable name
+#
+# Takes a variable name, and makes sure that it is set.
+function is_set() {
+ local DOLLAR='$';
+ local TO_EVAL="${DOLLAR}$1"
+ local VALUE=$(eval "echo $TO_EVAL")
+ local RET=1
+ if [ ! -z "${VALUE}" ]; then
+ RET=0
+ fi
+ return $RET
+}
+
+# Args: variable name, expression, debug string
+#
+# Exits if the specified variable name does not have an assigned value
+# or if the expression evaluates to false.
+function check() {
+ if ! is_set "$1" || ! eval "$2"; then
+ echo "Unable to determine $3."
+ echo "Please set the $1 environment variable when invoking $0."
+ exit 1
+ fi
+}
+
+# Args: user, group, misc, src, dst
+#
+# Some hackery to get around the fact that 'install' refuses to take
+# owner/group arguments unless run as root.
+function do_install() {
+ local INST_USER_GROUP=""
+ if [ -z "${NO_SUDO}" ]; then
+ INST_USER_GROUP="-o $1 -g $2"
+ fi
+ eval "${SUDO_CMD} install $INST_USER_GROUP $3 $4 $5"
+}
+
+# Args: setting name
+#
+# Extract an Apache compile-time setting with the given name.
+function extract_compile_setting() {
+ EXTRACT_COMPILE_SETTING=
+ APACHE_CONF_LINE=$(${APACHE_BIN} -V | grep $1)
+ if [ ! -z "${APACHE_CONF_LINE}" ]; then
+ local SED_REGEX="s/^.*${1}=?[\"\'\ ]*//"
+ EXTRACTED_COMPILE_SETTING=$(echo "${APACHE_CONF_LINE}" |
+ sed -r "${SED_REGEX}" |
+ sed "s/[\"\'\ ]*$//")
+ fi
+}
+
+if [ ! -f "${MODPAGESPEED_SO_PATH}" ]; then
+ echo "${MODPAGESPEED_SO_PATH} doesn't exist. Need to build first."
+ exit 1
+fi
+
+# Find the apxs binary, if not specified.
+if [ -z "${APXS_BIN}" ]; then
+ APXS_BIN=$(which apxs 2> /dev/null)
+ if [ -z "${APXS_BIN}" ]; then
+ APXS_BIN=$(which apxs2 2> /dev/null)
+ fi
+ if [ -z "${APXS_BIN}" ]; then
+ # Default location when Apache is installed from source.
+ APXS_BIN="/usr/local/apache2/bin/apxs"
+ fi
+fi
+
+# Find apxs which tells us about the system.
+check APXS_BIN "[ -f ${APXS_BIN} -a -x ${APXS_BIN} ]" "path to Apache apxs"
+
+echo "Using ${APXS_BIN} to determine installation location."
+echo ""
+
+# This is an optional configuration variable. If set, the conf file
+# path is relative to it.
+APACHE_ROOT=$(${APXS_BIN} -q PREFIX)
+
+# Find the Apache shared module dir.
+APACHE_MODULEDIR=$(${APXS_BIN} -q LIBEXECDIR)
+check APACHE_MODULEDIR "[ -d ${APACHE_MODULEDIR} ]" "Apache module dir"
+
+# Find the Apache conf dir.
+APACHE_CONFDIR=$(${APXS_BIN} -q SYSCONFDIR)
+check APACHE_CONFDIR "[ -d ${APACHE_CONFDIR} ]" "Apache conf dir"
+
+APACHE_SBINDIR=$(${APXS_BIN} -q SBINDIR)
+check APACHE_SBINDIR "[ -d ${APACHE_SBINDIR} ]" "Apache bin dir"
+
+APACHE_TARGET=$(${APXS_BIN} -q TARGET)
+APACHE_BIN="${APACHE_SBINDIR}/${APACHE_TARGET}"
+check APACHE_BIN "[ -f ${APACHE_BIN} -a -x ${APACHE_BIN} ]" "Apache binary"
+
+# Find the Apache conf file.
+if [ -z "${APACHE_CONF_FILE}" ]; then
+ extract_compile_setting SERVER_CONFIG_FILE
+ APACHE_CONF_FILE="${EXTRACTED_COMPILE_SETTING}"
+fi
+if [ ! -z "${APACHE_ROOT}" ]; then
+ APACHE_CONF_FILE="${APACHE_ROOT}/${APACHE_CONF_FILE}"
+fi
+if [ -z "${APACHE_CONF_FILE}" ]; then
+ APACHE_CONF_FILE="${APACHE_CONFDIR}/${APACHE_CONF_FILENAME}"
+fi
+check APACHE_CONF_FILE "[ -f ${APACHE_CONF_FILE} ]" "Apache configuration file"
+
+# Try to grep for the Apache user.
+if [ -z "${APACHE_USER}" ]; then
+ APACHE_USER_LINE=$(egrep -i "^[[:blank:]]*User[[:blank:]]+" "${APACHE_CONF_FILE}")
+ if [ ! -z "${APACHE_USER_LINE}" ]; then
+ APACHE_USER=$(echo "${APACHE_USER_LINE}" |
+ sed -r s/^.*User[[:blank:]]+[\"\']*// |
+ sed s/[\"\'[:blank:]]*$//)
+ fi
+fi
+
+# Try to grep for the Apache group.
+if [ -z "${APACHE_GROUP}" ]; then
+ APACHE_GROUP_LINE=$(egrep -i "^[[:blank:]]*Group[[:blank:]]+" "${APACHE_CONF_FILE}")
+ if [ ! -z "${APACHE_GROUP_LINE}" ]; then
+ APACHE_GROUP=$(echo "${APACHE_GROUP_LINE}" |
+ sed -r s/^.*Group[[:blank:]]+[\"\']*// |
+ sed s/[\"\'[:blank:]]*$//)
+ fi
+fi
+
+# Make sure we have an Apache user and group.
+check APACHE_USER "[ ! -z \'${APACHE_USER}\' ]" "Apache user"
+check APACHE_GROUP "[ ! -z \'${APACHE_GROUP}\' ]" "Apache group"
+
+# Make sure the user is valid.
+check APACHE_USER "id '${APACHE_USER}' &> /dev/null" "valid Apache user '${APACHE_USER}'"
+
+# Make sure the group is valid.
+# TODO: is there a way to ask the system if a group exists, similar to
+# the 'id' command?
+check APACHE_GROUP "egrep -q '^${APACHE_GROUP}:' /etc/group" "valid Apache group '${APACHE_GROUP}'"
+
+MODPAGESPEED_CONFDIR=${MODPAGESPEED_CONFDIR:-${APACHE_CONFDIR}}
+
+echo "mod_pagespeed needs to cache optimized resources on the file system."
+echo "The default location for this cache is '${MODPAGESPEED_CACHE_ROOT}'."
+read -p "Would you like to specify a different location? (y/N) " -n1 PROMPT
+if [ "${PROMPT}" = "y" -o "${PROMPT}" = "Y" ]; then
+ echo ""
+ read -p "Location for mod_pagespeed file cache: " MODPAGESPEED_CACHE_ROOT
+fi
+
+if [ -z "${MODPAGESPEED_CACHE_ROOT}" ]; then
+ echo ""
+ echo "Must specify a mod_pagespeed file cache."
+ exit 1
+fi
+
+echo ""
+echo "Preparing to install to the following locations:"
+echo "${APACHE_MODULEDIR}/${MODPAGESPEED_SO_NAME} (${MODPAGESPEED_FILE_USER}:${MODPAGESPEED_FILE_GROUP})"
+echo "${MODPAGESPEED_CONFDIR}/${MODPAGESPEED_CONF_NAME} (${MODPAGESPEED_FILE_USER}:${MODPAGESPEED_FILE_GROUP})"
+echo "${MODPAGESPEED_CACHE_ROOT}/cache (${APACHE_USER}:${APACHE_GROUP})"
+echo "${MODPAGESPEED_CACHE_ROOT}/files (${APACHE_USER}:${APACHE_GROUP})"
+echo ""
+if [ -z "${NO_PROMPT}" ]; then
+ echo -n "Continue? (y/N) "
+ read -n1 PROMPT
+ echo ""
+ if [ "${PROMPT}" != "y" -a "${PROMPT}" != "Y" ]; then
+ echo "Not continuing."
+ exit 1
+ fi
+fi
+
+if [ -d "${MODPAGESPEED_CACHE_ROOT}/cache" ]; then
+ echo "${MODPAGESPEED_CACHE_ROOT}/cache already exists. Not creating."
+fi
+if [ -d "${MODPAGESPEED_CACHE_ROOT}/files" ]; then
+ echo "${MODPAGESPEED_CACHE_ROOT}/files already exists. Not creating."
+fi
+
+# Only attempt to load mod_deflate in our conf file if it's actually
+# present on the system.
+COMMENT_OUT_DEFLATE='\#'
+if [ -f "${APACHE_MODULEDIR}/mod_deflate.so" ]; then
+ COMMENT_OUT_DEFLATE=
+else
+ echo "Unable to find mod_deflate.so. HTTP compression support not enabled!"
+fi
+
+TMP_CONF=$(mktemp -t conf.tmp.XXXXXX) || exit 1
+process_template "${SRC_ROOT}/install/common/pagespeed.conf.template" "${TMP_CONF}"
+TMP_LOAD=$(mktemp -t load.tmp.XXXXXX) || exit 1
+process_template "${SRC_ROOT}/install/common/pagespeed.load.template" "${TMP_LOAD}"
+cat "${TMP_CONF}" >> "${TMP_LOAD}"
+
+INSTALLATION_SUCCEEDED=0
+if (
+do_install "${MODPAGESPEED_FILE_USER}" "${MODPAGESPEED_FILE_GROUP}" "-m 644 -s" \
+ "${MODPAGESPEED_SO_PATH}" \
+ "${APACHE_MODULEDIR}/${MODPAGESPEED_SO_NAME}" &&
+do_install "${MODPAGESPEED_FILE_USER}" "${MODPAGESPEED_FILE_GROUP}" "-m 644" \
+ "${TMP_LOAD}" \
+ "${MODPAGESPEED_CONFDIR}/${MODPAGESPEED_CONF_NAME}" &&
+do_install "${APACHE_USER}" "${APACHE_GROUP}" "-m 755 -d" \
+ "${MODPAGESPEED_CACHE_ROOT}/cache" \
+ "${MODPAGESPEED_CACHE_ROOT}/files"
+); then
+ MODPAGESPEED_LOAD_LINE="Include ${MODPAGESPEED_CONFDIR}/${MODPAGESPEED_CONF_NAME}"
+ if ! grep -q "${MODPAGESPEED_LOAD_LINE}" "${APACHE_CONF_FILE}"; then
+ echo "Adding a load line for mod_pagespeed to ${APACHE_CONF_FILE}."
+ ${SUDO_CMD} sh -c "echo ${MODPAGESPEED_LOAD_LINE} >> ${APACHE_CONF_FILE}"
+ fi
+ if grep -q "${MODPAGESPEED_LOAD_LINE}" "${APACHE_CONF_FILE}"; then
+ INSTALLATION_SUCCEEDED=1
+ fi
+fi
+
+echo ""
+if [ $INSTALLATION_SUCCEEDED -eq 1 ]; then
+ echo "Installation succeeded."
+ echo "Restart apache to enable mod_pagespeed."
+else
+ echo "Installation failed."
+fi
+
+rm -f "${TMP_CONF}" "${TMP_LOAD}"
+
diff --git a/trunk/src/install/mod_pagespeed_example/.htaccess b/trunk/src/install/mod_pagespeed_example/.htaccess
new file mode 100644
index 0000000..9113fe2
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/.htaccess
@@ -0,0 +1,22 @@
+# Whether this file will be read by httpd depends on settings in httpd.conf. For
+# now, the presence of it verifies that these options are allowed.
+#
+# TODO(jmarantz): Add system test that proves these options are functional.
+ModPagespeed on
+ModPagespeedEnableFilters combine_css
+ModPagespeedJsOutlineMinBytes 3000
+ModPagespeedDisallow *
+ModPagespeedAllow *.htm*
+ModPagespeedAllow *.shtm*
+ModPagespeedAllow *.xhtm*
+ModPagespeedAllow */
+ModPagespeedAllow http://*/images/*.png
+ModPagespeedAllow http://*/images/*.jpg
+ModPagespeedAllow http://*/images/*.gif
+ModPagespeedAllow http://*/styles/*.css
+ModPagespeedAllow http://*/*.js
+ModPagespeedDisallow */images/captcha/*
+ModPagespeedMapRewriteDomain rewrite_to_domain rewrite_from_domain
+ModPagespeedMapOriginDomain origin_to_domain origin_from_domain
+ModPagespeedDomain my_domain
+ModPagespeedAllow *.cgi*
diff --git a/trunk/src/install/mod_pagespeed_example/add_instrumentation.html b/trunk/src/install/mod_pagespeed_example/add_instrumentation.html
new file mode 100644
index 0000000..a611823
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/add_instrumentation.html
@@ -0,0 +1 @@
+<html><body>This page will load very quickly.</body></html>
diff --git a/trunk/src/install/mod_pagespeed_example/cgi/slow.cgi b/trunk/src/install/mod_pagespeed_example/cgi/slow.cgi
new file mode 100755
index 0000000..5cd5d54
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/cgi/slow.cgi
@@ -0,0 +1,8 @@
+#!/bin/bash
+# This cgi script just sleeps for a while then returns an image. It's meant to
+# simulate a server environment where some resources are dynamically generated
+# by a process which is subject to delay (e.g. mysql, php).
+sleep 10;
+echo Content-type: image/jpeg
+echo
+cat ../images/Puzzle.jpg
diff --git a/trunk/src/install/mod_pagespeed_example/collapse_whitespace.html b/trunk/src/install/mod_pagespeed_example/collapse_whitespace.html
new file mode 100644
index 0000000..32ae8b7
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/collapse_whitespace.html
@@ -0,0 +1,16 @@
+<html>
+
+ <head>
+ <title>Hello, world! </title>
+ <script var x = 'Hello, world!';</script>
+ </head>
+
+ <body>
+ Hello, World!
+ <pre>
+ Hello,
+ World!
+ </pre>
+ </body>
+
+</html>
diff --git a/trunk/src/install/mod_pagespeed_example/combine_css.html b/trunk/src/install/mod_pagespeed_example/combine_css.html
new file mode 100644
index 0000000..19f7b28
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/combine_css.html
@@ -0,0 +1,11 @@
+<head>
+ <link rel="stylesheet" type="text/css" href="styles/yellow.css">
+ <link rel="stylesheet" type="text/css" href="styles/blue.css">
+ <link rel="stylesheet" type="text/css" href="styles/big.css">
+ <link rel="stylesheet" type="text/css" href="styles/bold.css">
+</head>
+<body>
+ <div class="blue yellow big bold">
+ Hello, mod_pagespeed!
+ </div>
+</body>
diff --git a/trunk/src/install/mod_pagespeed_example/combine_heads.html b/trunk/src/install/mod_pagespeed_example/combine_heads.html
new file mode 100644
index 0000000..d98beae
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/combine_heads.html
@@ -0,0 +1,11 @@
+ <head>
+ <title>Hello, world!</title>
+ </head>
+ <body>
+ <div class="big">
+ Hello, world!
+ </div>
+ </body>
+ <head>
+ <link rel="stylesheet" type="text/css" href="styles/big.css">
+ </head>
diff --git a/trunk/src/install/mod_pagespeed_example/elide_attributes.html b/trunk/src/install/mod_pagespeed_example/elide_attributes.html
new file mode 100644
index 0000000..36df69b
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/elide_attributes.html
@@ -0,0 +1,7 @@
+<html>
+ <body>
+ Here is a disabled button:
+ <button name="ok" disabled="disabled">button</button><br/>
+ Here is a text input: <input name="email" type="text"/>
+ </body>
+</html>
diff --git a/trunk/src/install/mod_pagespeed_example/extend_cache.html b/trunk/src/install/mod_pagespeed_example/extend_cache.html
new file mode 100644
index 0000000..748afe3
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/extend_cache.html
@@ -0,0 +1,10 @@
+<html>
+ <body>
+ <p>mod_pagespeed will rewrite this image's source to include a content-hash.
+ Note that this requires a background asynchronous fetch of the image;
+ therefore the page may not be rewritten the first time. If it isn't, wait
+ a few seconds and reload.</p>
+
+ <img src="images/Puzzle.jpg"/>
+ </body>
+</html>
diff --git a/trunk/src/install/mod_pagespeed_example/images/BikeCrashIcn.png b/trunk/src/install/mod_pagespeed_example/images/BikeCrashIcn.png
new file mode 100644
index 0000000..cdfcb41
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/images/BikeCrashIcn.png
Binary files differ
diff --git a/trunk/src/install/mod_pagespeed_example/images/Cuppa.png b/trunk/src/install/mod_pagespeed_example/images/Cuppa.png
new file mode 100644
index 0000000..6fb9cf3
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/images/Cuppa.png
Binary files differ
diff --git a/trunk/src/install/mod_pagespeed_example/images/Puzzle.jpg b/trunk/src/install/mod_pagespeed_example/images/Puzzle.jpg
new file mode 100644
index 0000000..a119109
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/images/Puzzle.jpg
Binary files differ
diff --git a/trunk/src/install/mod_pagespeed_example/index.html b/trunk/src/install/mod_pagespeed_example/index.html
new file mode 100644
index 0000000..dd019b6
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/index.html
@@ -0,0 +1,361 @@
+<html>
+ <head>
+ <title>mod_pagespeed Examples Directory</title>
+ <style type="text/css">
+ table { width: 100%; border-collapse: collapse; text-color: black;}
+ td { padding: 1em }
+ tr { background-color: #eee; border: 1px solid black; }
+ td.doc { width: 20%;}
+ .code {font-family:monospace;}
+ td.before, td.after { width: 10% }
+ </style>
+ </head>
+ <body>
+ <h2><span class="code">mod_pagespeed</span> Filter Examples</h2>
+ <p>
+ Here are some of the most useful filters provided by mod_pagespeed. Each
+ one has a simple HTML example attached; click "before" to see the original
+ file, and "after" to see what mod_pagespeed produces with that filter (and
+ only that filter) enabled. The two versions should look exactly the same,
+ but the "after" one will be (slightly) speedier. Use "view source" to see
+ the <span class="code">mod_pagespeed</span> difference!
+ </p>
+ <table>
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-instrumentation-add.html">
+ add_instrumentation
+ </a>
+ </td>
+ <td class="desc">
+ Adds client-side latency instrumentation.
+ </td>
+ <td class="before">
+ <a href="add_instrumentation.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="add_instrumentation.html?ModPagespeed=on&ModPagespeedFilters=add_instrumentation">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-cache-extend.html">
+ extend_cache
+ </a>
+ </td>
+ <td class="desc">
+ Improves cacheability.
+ </td>
+ <td class="before">
+ <a href="extend_cache.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="extend_cache.html?ModPagespeed=on&ModPagespeedFilters=extend_cache">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-whitespace-collapse.html">
+collapse_whitespace
+ </a>
+ </td>
+ <td class="desc">
+ Removes unnecessary whitespace in HTML.
+ </td>
+ <td class="before">
+ <a href="collapse_whitespace.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="collapse_whitespace.html?ModPagespeed=on&ModPagespeedFilters=collapse_whitespace">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-css-combine.html">
+combine_css
+ </a>
+ </td>
+ <td class="desc">
+ Combines multiple CSS files into one.
+ </td>
+ <td class="before">
+ <a href="combine_css.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="combine_css.html?ModPagespeed=on&ModPagespeedFilters=combine_css">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-head-combine.html">
+combine_heads
+ </a>
+ </td>
+ <td class="desc">
+ Combines multiple <head> elements into one.
+ </td>
+ <td class="before">
+ <a href="combine_heads.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="combine_heads.html?ModPagespeed=on&ModPagespeedFilters=combine_heads">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-css-to-head.html">
+move_css_to_head
+ </a>
+ </td>
+ <td class="desc">
+ Moves CSS into the <head> element.
+ </td>
+ <td class="before">
+ <a href="move_css_to_head.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="move_css_to_head.html?ModPagespeed=on&ModPagespeedFilters=move_css_to_head">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-attribute-elide.html">
+elide_attributes
+ </a>
+ </td>
+ <td class="desc">
+ Removes unnecessary attributes in HTML tags.
+ </td>
+ <td class="before">
+ <a href="elide_attributes.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="elide_attributes.html?ModPagespeed=on&ModPagespeedFilters=elide_attributes">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-css-inline.html">
+inline_css
+ </a>
+ </td>
+ <td class="desc">
+ Inlines small external CSS files.
+ </td>
+ <td class="before">
+ <a href="inline_css.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="inline_css.html?ModPagespeed=on&ModPagespeedFilters=inline_css">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-js-inline.html">
+inline_javascript
+ </a>
+ </td>
+ <td class="desc">
+ Inlines small external Javascript files.
+ </td>
+ <td class="before">
+ <a href="inline_javascript.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="inline_javascript.html?ModPagespeed=on&ModPagespeedFilters=inline_javascript">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-css-outline.html">
+outline_css
+ </a>
+ </td>
+ <td class="desc">
+ Moves large inline <style> tags into external files for cacheability.
+ </td>
+ <td class="before">
+ <a href="outline_css.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="outline_css.html?ModPagespeed=on&ModPagespeedFilters=outline_css">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-js-outline.html">
+outline_javascript
+ </a>
+ </td>
+ <td class="desc">
+ Moves large inline <script> tags into external files for cacheability.
+ </td>
+ <td class="before">
+ <a href="outline_javascript.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="outline_javascript.html?ModPagespeed=on&ModPagespeedFilters=outline_javascript">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-quote-remove.html">
+remove_quotes
+ </a>
+ </td>
+ <td class="desc">
+ Removes unnecessary quotes in HTML tags.
+ </td>
+ <td class="before">
+ <a href="remove_quotes.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="remove_quotes.html?ModPagespeed=on&ModPagespeedFilters=remove_quotes">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-comment-remove.html">
+remove_comments
+ </a>
+ </td>
+ <td class="desc">
+ Removes HTML comments.
+ </td>
+ <td class="before">
+ <a href="remove_comments.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="remove_comments.html?ModPagespeed=on&ModPagespeedFilters=remove_comments">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-css-minify.html">
+rewrite_css
+ </a>
+ </td>
+ <td class="desc">
+ Minifies CSS.
+ </td>
+ <td class="before">
+ <a href="rewrite_css.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="rewrite_css.html?ModPagespeed=on&ModPagespeedFilters=rewrite_css">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-image-optimize.html">
+rewrite_images
+ </a>
+ </td>
+ <td class="desc">
+ Rescales, and compresses images; inlines small ones.
+ </td>
+ <td class="before">
+ <a href="rewrite_images.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="rewrite_images.html?ModPagespeed=on&ModPagespeedFilters=rewrite_images">
+ after
+ </a>
+ </td>
+</tr>
+
+<tr>
+ <td class="code">
+ <a href="http://code.google.com/speed/page-speed/docs/filter-js-minify.html">
+rewrite_javascript
+ </a>
+ </td>
+ <td class="desc">
+ Minifies Javascript.
+ </td>
+ <td class="before">
+ <a href="rewrite_javascript.html?ModPagespeed=off">
+ before
+ </a>
+ </td>
+ <td class="after">
+ <a href="rewrite_javascript.html?ModPagespeed=on&ModPagespeedFilters=rewrite_javascript">
+ after
+ </a>
+ </td>
+</tr>
+ </table>
+ </body>
+<html>
diff --git a/trunk/src/install/mod_pagespeed_example/inline_css.html b/trunk/src/install/mod_pagespeed_example/inline_css.html
new file mode 100644
index 0000000..e7bb01f
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/inline_css.html
@@ -0,0 +1,10 @@
+<html>
+ <head>
+ <link rel="stylesheet" href="styles/all_styles.css">
+ </head>
+ <body>
+ <div class="blue yellow big bold">
+ Hello, world!
+ </div>
+ </body>
+</html>
diff --git a/trunk/src/install/mod_pagespeed_example/inline_javascript.html b/trunk/src/install/mod_pagespeed_example/inline_javascript.html
new file mode 100644
index 0000000..316fc90
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/inline_javascript.html
@@ -0,0 +1,8 @@
+<html>
+ <head>
+ <script type="text/javascript" src="inline_javascript.js"></script>
+ </head>
+ <body>
+ world!
+ </body>
+</html>
diff --git a/trunk/src/install/mod_pagespeed_example/inline_javascript.js b/trunk/src/install/mod_pagespeed_example/inline_javascript.js
new file mode 100644
index 0000000..fc84b31
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/inline_javascript.js
@@ -0,0 +1 @@
+document.write("Hello, ");
diff --git a/trunk/src/install/mod_pagespeed_example/move_css_to_head.html b/trunk/src/install/mod_pagespeed_example/move_css_to_head.html
new file mode 100644
index 0000000..af201a8
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/move_css_to_head.html
@@ -0,0 +1,9 @@
+<html>
+ <head></head>
+ <body>
+ <div class="blue yellow big bold">
+ Hello, world!
+ </div>
+ <link rel="stylesheet" type="text/css" href="styles/all_styles.css">
+ </body>
+</html>
diff --git a/trunk/src/install/mod_pagespeed_example/outline_css.html b/trunk/src/install/mod_pagespeed_example/outline_css.html
new file mode 100644
index 0000000..8779527
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/outline_css.html
@@ -0,0 +1,64 @@
+<html>
+ <head>
+ <title>Outline Resources example</title>
+ <style type="text/css" id="small">
+ .yellow {background-color: yellow;}
+ .blue {color: blue;}
+ .big {font-size: 8em;}
+ .bold {font-weight: bold;}
+ </style>
+ <style type="text/css" id="large">
+<!-- Just padding this out so it exceeds ModPagespeedCssOutlineMinBytes. -->
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ .yellow {background-color: yellow;} .blue {color: blue;} .big {font-size: 8em;} .bold {font-weight: bold;}
+ </style>
+ </head>
+ <body>
+ <div class="blue yellow big bold">
+ Hello, mod_pagespeed!
+ </div>
+ </body>
+</html>
diff --git a/trunk/src/install/mod_pagespeed_example/outline_javascript.html b/trunk/src/install/mod_pagespeed_example/outline_javascript.html
new file mode 100644
index 0000000..5207bea
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/outline_javascript.html
@@ -0,0 +1,1223 @@
+<html>
+ <head>
+ <title>Outline Resources example</title>
+ <script type="text/javascript" id="small"> var hello = 1; </script>
+ <script type="text/javascript" id="large">
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Copyright 2006 Google Inc. All Rights Reserved.
+
+/**
+ * @fileoverview Bootstrap for the Google JS Library (Closure).
+ *
+ * In uncompiled mode base.js will write out Closure's deps file, unless the
+ * global <code>CLOSURE_NO_DEPS</code> is set to true. This allows projects to
+ * include their own deps file(s) from different locations.
+ *
+ */
+
+/**
+ * @define {boolean} Overridden to true by the compiler when --closure_pass
+ * or --mark_as_compiled is specified.
+ */
+var COMPILED = false;
+
+
+/**
+ * Base namespace for the Closure library. Checks to see goog is
+ * already defined in the current scope before assigning to prevent
+ * clobbering if base.js is loaded more than once.
+ */
+var goog = goog || {}; // Check to see if already defined in current scope
+
+
+/**
+ * Reference to the global context. In most cases this will be 'window'.
+ */
+goog.global = this;
+
+
+/**
+ * @define {boolean} DEBUG is provided as a convenience so that debugging code
+ * that should not be included in a production js_binary can be easily stripped
+ * by specifying --define goog.DEBUG=false to the JSCompiler. For example, most
+ * toString() methods should be declared inside an "if (goog.DEBUG)" conditional
+ * because they are generally used for debugging purposes and it is difficult
+ * for the JSCompiler to statically determine whether they are used.
+ */
+goog.DEBUG = true;
+
+
+/**
+ * @define {string} LOCALE defines the locale being used for compilation. It is
+ * used to select locale specific data to be compiled in js binary. BUILD rule
+ * can specify this value by "--define goog.LOCALE=<locale_name>" as JSCompiler
+ * option.
+ *
+ * Take into account that the locale code format is important. You should use
+ * the canonical Unicode format with hyphen as a delimiter. Language must be
+ * lowercase, Language Script - Capitalized, Region - UPPERCASE.
+ * There are few examples: pt-BR, en, en-US, sr-Latin-BO, zh-Hans-CN.
+ *
+ * See more info about locale codes here:
+ * http://www.unicode.org/reports/tr35/#Unicode_Language_and_Locale_Identifiers
+ *
+ * For language codes you should use values defined by ISO 693-1. See it here
+ * http://www.w3.org/WAI/ER/IG/ert/iso639.htm. There is only one exception from
+ * this rule: the Hebrew language. For legacy reasons the old code (iw) should
+ * be used instead of the new code (he), see http://wiki/Main/IIISynonyms.
+ */
+goog.LOCALE = 'en'; // default to en
+
+
+/**
+ * Indicates whether or not we can call 'eval' directly to eval code in the
+ * global scope. Set to a Boolean by the first call to goog.globalEval (which
+ * empirically tests whether eval works for globals). @see goog.globalEval
+ * @type {boolean?}
+ * @private
+ */
+goog.evalWorksForGlobals_ = null;
+
+
+/**
+ * Creates object stubs for a namespace. When present in a file, goog.provide
+ * also indicates that the file defines the indicated object. Calls to
+ * goog.provide are resolved by the compiler if --closure_pass is set.
+ * @param {string} name name of the object that this file defines.
+ */
+goog.provide = function(name) {
+ if (!COMPILED) {
+ // Ensure that the same namespace isn't provided twice. This is intended
+ // to teach new developers that 'goog.provide' is effectively a variable
+ // declaration. And when JSCompiler transforms goog.provide into a real
+ // variable declaration, the compiled JS should work the same as the raw
+ // JS--even when the raw JS uses goog.provide incorrectly.
+ if (goog.getObjectByName(name) && !goog.implicitNamespaces_[name]) {
+ throw Error('Namespace "' + name + '" already declared.');
+ }
+
+ var namespace = name;
+ while ((namespace = namespace.substring(0, namespace.lastIndexOf('.')))) {
+ goog.implicitNamespaces_[namespace] = true;
+ }
+ }
+
+ goog.exportPath_(name);
+};
+
+
+if (!COMPILED) {
+ /**
+ * Namespaces implicitly defined by goog.provide. For example,
+ * goog.provide('goog.events.Event') implicitly declares
+ * that 'goog' and 'goog.events' must be namespaces.
+ *
+ * @type {Object}
+ * @private
+ */
+ goog.implicitNamespaces_ = {};
+}
+
+
+/**
+ * Builds an object structure for the provided namespace path,
+ * ensuring that names that already exist are not overwritten. For
+ * example:
+ * "a.b.c" -> a = {};a.b={};a.b.c={};
+ * Used by goog.provide and goog.exportSymbol.
+ * @param {string} name name of the object that this file defines.
+ * @param {Object} opt_object the object to expose at the end of the path.
+ * @param {Object} opt_objectToExportTo The object to add the path to; default
+ * is |goog.global|.
+ * @private
+ */
+goog.exportPath_ = function(name, opt_object, opt_objectToExportTo) {
+ var parts = name.split('.');
+ var cur = opt_objectToExportTo || goog.global;
+
+ // Internet Explorer exhibits strange behavior when throwing errors from
+ // methods externed in this manner. See the testExportSymbolExceptions in
+ // base_test.html for an example.
+ if (!(parts[0] in cur) && cur.execScript) {
+ cur.execScript('var ' + parts[0]);
+ }
+
+ // Certain browsers cannot parse code in the form for((a in b); c;);
+ // This pattern is produced by the JSCompiler when it collapses the
+ // statement above into the conditional loop below. To prevent this from
+ // happening, use a for-loop and reserve the init logic as below.
+
+ // Parentheses added to eliminate strict JS warning in Firefox.
+ for (var part; parts.length && (part = parts.shift());) {
+ if (!parts.length && goog.isDef(opt_object)) {
+ // last part and we have an object; use it
+ cur[part] = opt_object;
+ } else if (cur[part]) {
+ cur = cur[part];
+ } else {
+ cur = cur[part] = {};
+ }
+ }
+};
+
+
+/**
+ * Returns an object based on its fully qualified external name. If you are
+ * using a compilation pass that renames property names beware that using this
+ * function will not find renamed properties.
+ *
+ * @param {string} name The fully qualified name.
+ * @param {Object} opt_obj The object within which to look; default is
+ * |goog.global|.
+ * @return {Object?} The object or, if not found, null.
+ */
+goog.getObjectByName = function(name, opt_obj) {
+ var parts = name.split('.');
+ var cur = opt_obj || goog.global;
+ for (var part; part = parts.shift(); ) {
+ if (cur[part]) {
+ cur = cur[part];
+ } else {
+ return null;
+ }
+ }
+ return cur;
+};
+
+
+/**
+ * Globalizes a whole namespace, such as goog or goog.lang.
+ *
+ * @param {Object} obj The namespace to globalize.
+ * @param {Object} opt_global The object to add the properties to.
+ * @deprecated Properties may be explicitly exported to the global scope, but
+ * this should no longer be done in bulk.
+ */
+goog.globalize = function(obj, opt_global) {
+ var global = opt_global || goog.global;
+ for (var x in obj) {
+ global[x] = obj[x];
+ }
+};
+
+
+/**
+ * Adds a dependency from a file to the files it requires.
+ * @param {string} relPath The path to the js file.
+ * @param {Array} provides An array of strings with the names of the objects
+ * this file provides.
+ * @param {Array} requires An array of strings with the names of the objects
+ * this file requires.
+ */
+goog.addDependency = function(relPath, provides, requires) {
+ if (!COMPILED) {
+ var provide, require;
+ var path = relPath.replace(/\\/g, '/');
+ var deps = goog.dependencies_;
+ for (var i = 0; provide = provides[i]; i++) {
+ deps.nameToPath[provide] = path;
+ if (!(path in deps.pathToNames)) {
+ deps.pathToNames[path] = {};
+ }
+ deps.pathToNames[path][provide] = true;
+ }
+ for (var j = 0; require = requires[j]; j++) {
+ if (!(path in deps.requires)) {
+ deps.requires[path] = {};
+ }
+ deps.requires[path][require] = true;
+ }
+ }
+};
+
+
+/**
+ * Implements a system for the dynamic resolution of dependencies
+ * that works in parallel with the BUILD system. Note that all calls
+ * to goog.require will be stripped by the JSCompiler when the
+ * --closure_pass option is used.
+ * @param {string} rule Rule to include, in the form goog.package.part.
+ */
+goog.require = function(rule) {
+
+ // if the object already exists we do not need do do anything
+ // TODO: If we start to support require based on file name this has
+ // to change
+ // TODO: If we allow goog.foo.* this has to change
+ // TODO: If we implement dynamic load after page load we should probably
+ // not remove this code for the compiled output
+ if (!COMPILED) {
+ if (goog.getObjectByName(rule)) {
+ return;
+ }
+ var path = goog.getPathFromDeps_(rule);
+ if (path) {
+ goog.included_[path] = true;
+ goog.writeScripts_();
+ } else {
+ var errorMessage = 'goog.require could not find: ' + rule;
+ if (goog.global.console) {
+ goog.global.console['error'](errorMessage);
+ }
+
+ throw Error(errorMessage);
+ }
+ }
+};
+
+
+/**
+ * Whether goog.require should throw an exception if it fails.
+ * @type {boolean}
+ */
+goog.useStrictRequires = false;
+
+
+/**
+ * Path for included scripts
+ * @type {string}
+ */
+goog.basePath = '';
+
+
+/**
+ * A hook for overriding the base path.
+ * @type {string|undefined}
+ */
+goog.global.CLOSURE_BASE_PATH;
+
+
+/**
+ * Whether to write out Closure's deps file. By default,
+ * the deps are written.
+ * @type {boolean|undefined}
+ */
+goog.global.CLOSURE_NO_DEPS;
+
+
+/**
+ * Null function used for default values of callbacks, etc.
+ * @type {!Function}
+ */
+goog.nullFunction = function() {};
+
+
+/**
+ * The identity function. Returns its first argument.
+ *
+ * @param {*} var_args The arguments of the function.
+ * @return {*} The first argument.
+ * @deprecated Use goog.functions.identity instead.
+ */
+goog.identityFunction = function(var_args) {
+ return arguments[0];
+};
+
+
+/**
+ * When defining a class Foo with an abstract method bar(), you can do:
+ *
+ * Foo.prototype.bar = goog.abstractMethod
+ *
+ * Now if a subclass of Foo fails to override bar(), an error
+ * will be thrown when bar() is invoked.
+ *
+ * Note: This does not take the name of the function to override as
+ * an argument because that would make it more difficult to obfuscate
+ * our JavaScript code.
+ *
+ * @type {!Function}
+ * @throws {Error} when invoked to indicate the method should be
+ * overridden.
+ */
+goog.abstractMethod = function() {
+ throw Error('unimplemented abstract method');
+};
+
+
+/**
+ * Adds a {@code getInstance} static method that always return the same instance
+ * object.
+ * @param {!Function} ctor The constructor for the class to add the static
+ * method to.
+ */
+goog.addSingletonGetter = function(ctor) {
+ ctor.getInstance = function() {
+ return ctor.instance_ || (ctor.instance_ = new ctor());
+ };
+};
+
+
+if (!COMPILED) {
+ /**
+ * Object used to keep track of urls that have already been added. This
+ * record allows the prevention of circular dependencies.
+ * @type {Object}
+ * @private
+ */
+ goog.included_ = {};
+
+
+ /**
+ * This object is used to keep track of dependencies and other data that is
+ * used for loading scripts
+ * @private
+ * @type {Object}
+ */
+ goog.dependencies_ = {
+ pathToNames: {}, // 1 to many
+ nameToPath: {}, // 1 to 1
+ requires: {}, // 1 to many
+ visited: {}, // used when resolving dependencies to prevent us from
+ // visiting the file twice
+ written: {} // used to keep track of script files we have written
+ };
+
+
+ /**
+ * Tries to detect whether is in the context of an HTML document.
+ * @return {boolean} True if it looks like HTML document.
+ * @private
+ */
+ goog.inHtmlDocument_ = function() {
+ var doc = goog.global.document;
+ return typeof doc != 'undefined' &&
+ 'write' in doc; // XULDocument misses write.
+ };
+
+
+ /**
+ * Tries to detect the base path of the base.js script that bootstraps Closure
+ * @private
+ */
+ goog.findBasePath_ = function() {
+ if (!goog.inHtmlDocument_()) {
+ return;
+ }
+ var doc = goog.global.document;
+ if (goog.global.CLOSURE_BASE_PATH) {
+ goog.basePath = goog.global.CLOSURE_BASE_PATH;
+ return;
+ }
+ var scripts = doc.getElementsByTagName('script');
+ for (var script, i = 0; script = scripts[i]; i++) {
+ var src = script.src;
+ var l = src.length;
+ if (src.substr(l - 7) == 'base.js') {
+ goog.basePath = src.substr(0, l - 7);
+ return;
+ }
+ }
+ };
+
+
+ /**
+ * Writes a script tag if, and only if, that script hasn't already been added
+ * to the document. (Must be called at execution time)
+ * @param {string} src Script source.
+ * @private
+ */
+ goog.writeScriptTag_ = function(src) {
+ if (goog.inHtmlDocument_() &&
+ !goog.dependencies_.written[src]) {
+ goog.dependencies_.written[src] = true;
+ var doc = goog.global.document;
+ doc.write('<script type="text/javascript" src="' +
+ src + '"></' + 'script>');
+ }
+ };
+
+
+ /**
+ * Resolves dependencies based on the dependencies added using addDependency
+ * and calls writeScriptTag_ in the correct order.
+ * @private
+ */
+ goog.writeScripts_ = function() {
+ // the scripts we need to write this time
+ var scripts = [];
+ var seenScript = {};
+ var deps = goog.dependencies_;
+
+ function visitNode(path) {
+ if (path in deps.written) {
+ return;
+ }
+
+ // we have already visited this one. We can get here if we have cyclic
+ // dependencies
+ if (path in deps.visited) {
+ if (!(path in seenScript)) {
+ seenScript[path] = true;
+ scripts.push(path);
+ }
+ return;
+ }
+
+ deps.visited[path] = true;
+
+ if (path in deps.requires) {
+ for (var requireName in deps.requires[path]) {
+ if (requireName in deps.nameToPath) {
+ visitNode(deps.nameToPath[requireName]);
+ } else if (!goog.getObjectByName(requireName)) {
+ // If the required name is defined, we assume that this
+ // dependency was bootstapped by other means. Otherwise,
+ // throw an exception.
+ throw Error('Undefined nameToPath for ' + requireName);
+ }
+ }
+ }
+
+ if (!(path in seenScript)) {
+ seenScript[path] = true;
+ scripts.push(path);
+ }
+ }
+
+ for (var path in goog.included_) {
+ if (!deps.written[path]) {
+ visitNode(path);
+ }
+ }
+
+ for (var i = 0; i < scripts.length; i++) {
+ if (scripts[i]) {
+ goog.writeScriptTag_(goog.basePath + scripts[i]);
+ } else {
+ throw Error('Undefined script input');
+ }
+ }
+ };
+
+
+ /**
+ * Looks at the dependency rules and tries to determine the script file that
+ * fulfills a particular rule.
+ * @param {string} rule In the form goog.namespace.Class or project.script.
+ * @return {string?} Url corresponding to the rule, or null.
+ * @private
+ */
+ goog.getPathFromDeps_ = function(rule) {
+ if (rule in goog.dependencies_.nameToPath) {
+ return goog.dependencies_.nameToPath[rule];
+ } else {
+ return null;
+ }
+ };
+
+ goog.findBasePath_();
+
+ // Allow projects to manage the deps files themselves.
+ if (!goog.global.CLOSURE_NO_DEPS) {
+ // [MODIFIED FOR MOD_PAGESPEED_DEMO] goog.writeScriptTag_(goog.basePath + 'deps.js');
+ }
+}
+
+
+
+//==============================================================================
+// Language Enhancements
+//==============================================================================
+
+
+/**
+ * This is a "fixed" version of the typeof operator. It differs from the typeof
+ * operator in such a way that null returns 'null' and arrays return 'array'.
+ * @param {*} value The value to get the type of.
+ * @return {string} The name of the type.
+ */
+goog.typeOf = function(value) {
+ var s = typeof value;
+ if (s == 'object') {
+ if (value) {
+ // We cannot use constructor == Array or instanceof Array because
+ // different frames have different Array objects. In IE6, if the iframe
+ // where the array was created is destroyed, the array loses its
+ // prototype. Then dereferencing val.splice here throws an exception, so
+ // we can't use goog.isFunction. Calling typeof directly returns 'unknown'
+ // so that will work. In this case, this function will return false and
+ // most array functions will still work because the array is still
+ // array-like (supports length and []) even though it has lost its
+ // prototype.
+ // Mark Miller noticed that Object.prototype.toString
+ // allows access to the unforgeable [[Class]] property.
+ // 15.2.4.2 Object.prototype.toString ( )
+ // When the toString method is called, the following steps are taken:
+ // 1. Get the [[Class]] property of this object.
+ // 2. Compute a string value by concatenating the three strings
+ // "[object ", Result(1), and "]".
+ // 3. Return Result(2).
+ // and this behavior survives the destruction of the execution context.
+ if (value instanceof Array || // Works quickly in same execution context.
+ // If value is from a different execution context then
+ // !(value instanceof Object), which lets us early out in the common
+ // case when value is from the same context but not an array.
+ // The {if (value)} check above means we don't have to worry about
+ // undefined behavior of Object.prototype.toString on null/undefined.
+ //
+ // HACK: In order to use an Object prototype method on the arbitrary
+ // value, the compiler requires the value be cast to type Object,
+ // even though the ECMA spec explicitly allows it.
+ (!(value instanceof Object) &&
+ (Object.prototype.toString.call(
+ /** @type {Object} */ (value)) == '[object Array]') ||
+
+ // In IE all non value types are wrapped as objects across window
+ // boundaries (not iframe though) so we have to do object detection
+ // for this edge case
+ typeof value.length == 'number' &&
+ typeof value.splice != 'undefined' &&
+ typeof value.propertyIsEnumerable != 'undefined' &&
+ !value.propertyIsEnumerable('splice')
+
+ )) {
+ return 'array';
+ }
+ // HACK: There is still an array case that fails.
+ // function ArrayImpostor() {}
+ // ArrayImpostor.prototype = [];
+ // var impostor = new ArrayImpostor;
+ // this can be fixed by getting rid of the fast path
+ // (value instanceof Array) and solely relying on
+ // (value && Object.prototype.toString.vall(value) === '[object Array]')
+ // but that would require many more function calls and is not warranted
+ // unless closure code is receiving objects from untrusted sources.
+
+ // IE in cross-window calls does not correctly marshal the function type
+ // (it appears just as an object) so we cannot use just typeof val ==
+ // 'function'. However, if the object has a call property, it is a
+ // function.
+ if (!(value instanceof Object) &&
+ (Object.prototype.toString.call(
+ /** @type {Object} */ (value)) == '[object Function]' ||
+ typeof value.call != 'undefined' &&
+ typeof value.propertyIsEnumerable != 'undefined' &&
+ !value.propertyIsEnumerable('call'))) {
+ return 'function';
+ }
+
+
+ } else {
+ return 'null';
+ }
+
+ // In Safari typeof nodeList returns 'function', and on Firefox
+ // typeof behaves similarly for HTML{Applet,Embed,Object}Elements
+ // and RegExps. We would like to return object for those and we can
+ // detect an invalid function by making sure that the function
+ // object has a call method.
+ } else if (s == 'function' && typeof value.call == 'undefined') {
+ return 'object';
+ }
+ return s;
+};
+
+
+/**
+ * Safe way to test whether a property is enumarable. It allows testing
+ * for enumerable on objects where 'propertyIsEnumerable' is overridden or
+ * does not exist (like DOM nodes in IE). Does not use browser native
+ * Object.propertyIsEnumerable.
+ * @param {Object} object The object to test if the property is enumerable.
+ * @param {string} propName The property name to check for.
+ * @return {boolean} True if the property is enumarable.
+ * @private
+ */
+goog.propertyIsEnumerableCustom_ = function(object, propName) {
+ // KJS in Safari 2 is not ECMAScript compatible and lacks crucial methods
+ // such as propertyIsEnumerable. We therefore use a workaround.
+ // Does anyone know a more efficient work around?
+ if (propName in object) {
+ for (var key in object) {
+ if (key == propName &&
+ Object.prototype.hasOwnProperty.call(object, propName)) {
+ return true;
+ }
+ }
+ }
+ return false;
+};
+
+
+/**
+ * Safe way to test whether a property is enumarable. It allows testing
+ * for enumerable on objects where 'propertyIsEnumerable' is overridden or
+ * does not exist (like DOM nodes in IE).
+ * @param {Object} object The object to test if the property is enumerable.
+ * @param {string} propName The property name to check for.
+ * @return {boolean} True if the property is enumarable.
+ * @private
+ */
+goog.propertyIsEnumerable_ = function(object, propName) {
+ // In IE if object is from another window, cannot use propertyIsEnumerable
+ // from this window's Object. Will raise a 'JScript object expected' error.
+ if (object instanceof Object) {
+ return Object.prototype.propertyIsEnumerable.call(object, propName);
+ } else {
+ return goog.propertyIsEnumerableCustom_(object, propName);
+ }
+};
+
+
+/**
+ * Returns true if the specified value is not |undefined|.
+ * WARNING: Do not use this to test if an object has a property. Use the in
+ * operator instead. Additionally, this function assumes that the global
+ * undefined variable has not been redefined.
+ * @param {*} val Variable to test.
+ * @return {boolean} Whether variable is defined.
+ */
+goog.isDef = function(val) {
+ return val !== undefined;
+};
+
+
+/**
+ * Returns true if the specified value is |null|
+ * @param {*} val Variable to test.
+ * @return {boolean} Whether variable is null.
+ */
+goog.isNull = function(val) {
+ return val === null;
+};
+
+
+/**
+ * Returns true if the specified value is defined and not null
+ * @param {*} val Variable to test.
+ * @return {boolean} Whether variable is defined and not null.
+ */
+goog.isDefAndNotNull = function(val) {
+ // Note that undefined == null.
+ return val != null;
+};
+
+
+/**
+ * Returns true if the specified value is an array
+ * @param {*} val Variable to test.
+ * @return {boolean} Whether variable is an array.
+ */
+goog.isArray = function(val) {
+ return goog.typeOf(val) == 'array';
+};
+
+
+/**
+ * Returns true if the object looks like an array. To qualify as array like
+ * the value needs to be either a NodeList or an object with a Number length
+ * property.
+ * @param {*} val Variable to test.
+ * @return {boolean} Whether variable is an array.
+ */
+goog.isArrayLike = function(val) {
+ var type = goog.typeOf(val);
+ return type == 'array' || type == 'object' && typeof val.length == 'number';
+};
+
+
+/**
+ * Returns true if the object looks like a Date. To qualify as Date-like
+ * the value needs to be an object and have a getFullYear() function.
+ * @param {*} val Variable to test.
+ * @return {boolean} Whether variable is a like a Date.
+ */
+goog.isDateLike = function(val) {
+ return goog.isObject(val) && typeof val.getFullYear == 'function';
+};
+
+
+/**
+ * Returns true if the specified value is a string
+ * @param {*} val Variable to test.
+ * @return {boolean} Whether variable is a string.
+ */
+goog.isString = function(val) {
+ return typeof val == 'string';
+};
+
+
+/**
+ * Returns true if the specified value is a boolean
+ * @param {*} val Variable to test.
+ * @return {boolean} Whether variable is boolean.
+ */
+goog.isBoolean = function(val) {
+ return typeof val == 'boolean';
+};
+
+
+/**
+ * Returns true if the specified value is a number
+ * @param {*} val Variable to test.
+ * @return {boolean} Whether variable is a number.
+ */
+goog.isNumber = function(val) {
+ return typeof val == 'number';
+};
+
+
+/**
+ * Returns true if the specified value is a function
+ * @param {*} val Variable to test.
+ * @return {boolean} Whether variable is a function.
+ */
+goog.isFunction = function(val) {
+ return goog.typeOf(val) == 'function';
+};
+
+
+/**
+ * Returns true if the specified value is an object. This includes arrays
+ * and functions.
+ * @param {*} val Variable to test.
+ * @return {boolean} Whether variable is an object.
+ */
+goog.isObject = function(val) {
+ var type = goog.typeOf(val);
+ return type == 'object' || type == 'array' || type == 'function';
+};
+
+
+/**
+ * Adds a hash code field to an object. The hash code is unique for the
+ * given object.
+ * @param {Object} obj The object to get the hash code for.
+ * @return {number} The hash code for the object.
+ */
+goog.getHashCode = function(obj) {
+ // In IE, DOM nodes do not extend Object so they do not have this method.
+ // we need to check hasOwnProperty because the proto might have this set.
+
+ // TODO: There is a proposal to add hashcode as a global function to JS2
+ // we should keep track of this process so we can use that whenever
+ // it starts to show up in the real world.
+ if (obj.hasOwnProperty && obj.hasOwnProperty(goog.HASH_CODE_PROPERTY_)) {
+ return obj[goog.HASH_CODE_PROPERTY_];
+ }
+ if (!obj[goog.HASH_CODE_PROPERTY_]) {
+ obj[goog.HASH_CODE_PROPERTY_] = ++goog.hashCodeCounter_;
+ }
+ return obj[goog.HASH_CODE_PROPERTY_];
+};
+
+
+/**
+ * Removes the hash code field from an object.
+ * @param {Object} obj The object to remove the field from.
+ */
+goog.removeHashCode = function(obj) {
+ // DOM nodes in IE are not instance of Object and throws exception
+ // for delete. Instead we try to use removeAttribute
+ if ('removeAttribute' in obj) {
+ obj.removeAttribute(goog.HASH_CODE_PROPERTY_);
+ }
+ /** @preserveTry */
+ try {
+ delete obj[goog.HASH_CODE_PROPERTY_];
+ } catch (ex) {
+ }
+};
+
+
+/**
+ * Name for hash code property. Initialized in a way to help avoid collisions
+ * with other closure javascript on the same page.
+ * @type {string}
+ * @private
+ */
+goog.HASH_CODE_PROPERTY_ = 'closure_hashCode_' +
+ Math.floor(Math.random() * 2147483648).toString(36);
+
+
+/**
+ * Counter for hash codes.
+ * @type {number}
+ * @private
+ */
+goog.hashCodeCounter_ = 0;
+
+
+/**
+ * Clone an object/array (recursively)
+ * @param {Object} proto Object to clone.
+ * @return {Object} Clone of x;.
+ */
+goog.cloneObject = function(proto) {
+ var type = goog.typeOf(proto);
+ if (type == 'object' || type == 'array') {
+ if (proto.clone) {
+ // TODO Change to proto.clone() once # args warn is removed
+ return proto.clone.call(proto);
+ }
+ var clone = type == 'array' ? [] : {};
+ for (var key in proto) {
+ clone[key] = goog.cloneObject(proto[key]);
+ }
+ return clone;
+ }
+
+ return proto;
+};
+
+
+/**
+ * Forward declaration for the clone method. This is necessary until the
+ * compiler can better support duck-typing constructs as used in
+ * goog.cloneObject.
+ *
+ * TODO: Remove once the JSCompiler can infer that the check for
+ * proto.clone is safe in goog.cloneObject.
+ *
+ * @type {Function}
+ */
+Object.prototype.clone;
+
+
+/**
+ * Partially applies this function to a particular 'this object' and zero or
+ * more arguments. The result is a new function with some arguments of the first
+ * function pre-filled and the value of |this| 'pre-specified'.<br><br>
+ *
+ * Remaining arguments specified at call-time are appended to the pre-
+ * specified ones.<br><br>
+ *
+ * Also see: {@link #partial}.<br><br>
+ *
+ * Usage:
+ * <pre>var barMethBound = bind(myFunction, myObj, 'arg1', 'arg2');
+ * barMethBound('arg3', 'arg4');</pre>
+ *
+ * @param {Function} fn A function to partially apply.
+ * @param {Object|undefined} selfObj Specifies the object which |this| should
+ * point to when the function is run. If the value is null or undefined, it
+ * will default to the global object.
+ * @param {*} var_args Additional arguments that are partially
+ * applied to the function.
+ *
+ * @return {!Function} A partially-applied form of the function bind() was
+ * invoked as a method of.
+ */
+goog.bind = function(fn, selfObj, var_args) {
+ var context = selfObj || goog.global;
+
+ if (arguments.length > 2) {
+ var boundArgs = Array.prototype.slice.call(arguments, 2);
+ return function() {
+ // Prepend the bound arguments to the current arguments.
+ var newArgs = Array.prototype.slice.call(arguments);
+ Array.prototype.unshift.apply(newArgs, boundArgs);
+ return fn.apply(context, newArgs);
+ };
+
+ } else {
+ return function() {
+ return fn.apply(context, arguments);
+ };
+ }
+};
+
+
+/**
+ * Like bind(), except that a 'this object' is not required. Useful when the
+ * target function is already bound.
+ *
+ * Usage:
+ * var g = partial(f, arg1, arg2);
+ * g(arg3, arg4);
+ *
+ * @param {Function} fn A function to partially apply.
+ * @param {*} var_args Additional arguments that are partially
+ * applied to fn.
+ * @return {!Function} A partially-applied form of the function bind() was
+ * invoked as a method of.
+ */
+goog.partial = function(fn, var_args) {
+ var args = Array.prototype.slice.call(arguments, 1);
+ return function() {
+ // Prepend the bound arguments to the current arguments.
+ var newArgs = Array.prototype.slice.call(arguments);
+ newArgs.unshift.apply(newArgs, args);
+ return fn.apply(this, newArgs);
+ };
+};
+
+
+/**
+ * Copies all the members of a source object to a target object.
+ * @param {Object} target Target.
+ * @param {Object} source Source.
+ * @deprecated Use goog.object.extend instead.
+ */
+goog.mixin = function(target, source) {
+ for (var x in source) {
+ target[x] = source[x];
+ }
+
+ // For IE the for-in-loop does not contain any properties that are not
+ // enumerable on the prototype object (for example, isPrototypeOf from
+ // Object.prototype) but also it will not include 'replace' on objects that
+ // extend String and change 'replace' (not that it is common for anyone to
+ // extend anything except Object).
+};
+
+
+/**
+ * A simple wrapper for new Date().getTime().
+ *
+ * @return {number} An integer value representing the number of milliseconds
+ * between midnight, January 1, 1970 and the current time.
+ */
+goog.now = Date.now || (function() {
+ return new Date().getTime();
+});
+
+
+/**
+ * Evals javascript in the global scope. In IE this uses execScript, other
+ * browsers use goog.global.eval. If goog.global.eval does not evaluate in the
+ * global scope (for example, in Safari), appends a script tag instead.
+ * Throws an exception if neither execScript or eval is defined.
+ * @param {string} script JavaScript string.
+ */
+goog.globalEval = function(script) {
+ if (goog.global.execScript) {
+ goog.global.execScript(script, 'JavaScript');
+ } else if (goog.global.eval) {
+ // Test to see if eval works
+ if (goog.evalWorksForGlobals_ == null) {
+ goog.global.eval('var _et_ = 1;');
+ if (typeof goog.global['_et_'] != 'undefined') {
+ delete goog.global['_et_'];
+ goog.evalWorksForGlobals_ = true;
+ } else {
+ goog.evalWorksForGlobals_ = false;
+ }
+ }
+
+ if (goog.evalWorksForGlobals_) {
+ goog.global.eval(script);
+ } else {
+ var doc = goog.global.document;
+ var scriptElt = doc.createElement('script');
+ scriptElt.type = 'text/javascript';
+ scriptElt.defer = false;
+ // NOTE: can't use .innerHTML since "t('<test>')" will fail and
+ // .text doesn't work in Safari 2. Therefore we append a text node.
+ scriptElt.appendChild(doc.createTextNode(script));
+ doc.body.appendChild(scriptElt);
+ doc.body.removeChild(scriptElt);
+ }
+ } else {
+ throw Error('goog.globalEval not available');
+ }
+};
+
+
+/**
+ * A macro for defining composite types.
+ *
+ * By assigning goog.typedef to a name, this tells JSCompiler that this is not
+ * the name of a class, but rather it's the name of a composite type.
+ *
+ * For example,
+ * /** @type {Array|NodeList} / goog.ArrayLike = goog.typedef;
+ * will tell JSCompiler to replace all appearances of goog.ArrayLike in type
+ * definitions with the union of Array and NodeList.
+ *
+ * Does nothing in uncompiled code.
+ */
+goog.typedef = true;
+
+
+/**
+ * Optional map of CSS class names to obfuscated names used with
+ * goog.getCssName().
+ * @type {Object|undefined}
+ * @private
+ * @see goog.setCssNameMapping
+ */
+goog.cssNameMapping_;
+
+
+/**
+ * Handles strings that are intended to be used as CSS class names.
+ *
+ * Without JS Compiler the arguments are simple joined with a hyphen and passed
+ * through unaltered.
+ *
+ * With the JS Compiler the arguments are inlined, e.g:
+ * var x = goog.getCssName('foo');
+ * var y = goog.getCssName(this.baseClass, 'active');
+ * becomes:
+ * var x= 'foo';
+ * var y = this.baseClass + '-active';
+ *
+ * If a CSS renaming map is passed to the compiler it will replace symbols in
+ * the classname. If one argument is passed it will be processed, if two are
+ * passed only the modifier will be processed, as it is assumed the first
+ * argument was generated as a result of calling goog.getCssName.
+ *
+ * Names are split on 'hyphen' and processed in parts such that the following
+ * are equivalent:
+ * var base = goog.getCssName('baseclass');
+ * goog.getCssName(base, 'modifier');
+ * goog.getCSsName('baseclass-modifier');
+ *
+ * If any part does not appear in the renaming map a warning is logged and the
+ * original, unobfuscated class name is inlined.
+ *
+ * @param {string} className The class name.
+ * @param {string} opt_modifier A modifier to be appended to the class name.
+ * @return {string} The class name or the concatenation of the class name and
+ * the modifier.
+ */
+goog.getCssName = function(className, opt_modifier) {
+ var cssName = className + (opt_modifier ? '-' + opt_modifier : '');
+ return (goog.cssNameMapping_ && (cssName in goog.cssNameMapping_)) ?
+ goog.cssNameMapping_[cssName] : cssName;
+};
+
+
+/**
+ * Sets the map to check when returning a value from goog.getCssName(). Example:
+ * <pre>
+ * goog.setCssNameMapping({
+ * "goog-menu": "a",
+ * "goog-menu-disabled": "a-b",
+ * "CSS_LOGO": "b",
+ * "hidden": "c"
+ * });
+ *
+ * // The following evaluates to: "a a-b".
+ * goog.getCssName('goog-menu') + ' ' + goog.getCssName('goog-menu', 'disabled')
+ * </pre>
+ * When declared as a map of string literals to string literals, the JSCompiler
+ * will replace all calls to goog.getCssName() using the supplied map if the
+ * --closure_pass flag is set.
+ *
+ * @param {!Object} mapping A map of strings to strings where keys are possible
+ * arguments to goog.getCssName() and values are the corresponding values
+ * that should be returned.
+ */
+goog.setCssNameMapping = function(mapping) {
+ goog.cssNameMapping_ = mapping;
+};
+
+
+/**
+ * Abstract implementation of goog.getMsg for use with localized messages.
+ * @param {string} str Translatable string, places holders in the form {$foo}.
+ * @param {Object} opt_values Map of place holder name to value.
+ * @return {string} message with placeholders filled.
+ */
+goog.getMsg = function(str, opt_values) {
+ var values = opt_values || {};
+ for (var key in values) {
+ str = str.replace(new RegExp('\\{\\$' + key + '\\}', 'gi'), values[key]);
+ }
+ return str;
+};
+
+
+/**
+ * Exposes an unobfuscated global namespace path for the given object.
+ * Note that fields of the exported object *will* be obfuscated,
+ * unless they are exported in turn via this function or
+ * goog.exportProperty
+ *
+ * <p>Also handy for making public items that are defined in anonymous
+ * closures.
+ *
+ * ex. goog.exportSymbol('Foo', Foo);
+ *
+ * ex. goog.exportSymbol('public.path.Foo.staticFunction',
+ * Foo.staticFunction);
+ * public.path.Foo.staticFunction();
+ *
+ * ex. goog.exportSymbol('public.path.Foo.prototype.myMethod',
+ * Foo.prototype.myMethod);
+ * new public.path.Foo().myMethod();
+ *
+ * @param {string} publicPath Unobfuscated name to export.
+ * @param {Object} object Object the name should point to.
+ * @param {Object} opt_objectToExportTo The object to add the path to; default
+ * is |goog.global|.
+ */
+goog.exportSymbol = function(publicPath, object, opt_objectToExportTo) {
+ goog.exportPath_(publicPath, object, opt_objectToExportTo);
+};
+
+
+/**
+ * Exports a property unobfuscated into the object's namespace.
+ * ex. goog.exportProperty(Foo, 'staticFunction', Foo.staticFunction);
+ * ex. goog.exportProperty(Foo.prototype, 'myMethod', Foo.prototype.myMethod);
+ * @param {Object} object Object whose static property is being exported.
+ * @param {string} publicName Unobfuscated name to export.
+ * @param {Object} symbol Object the name should point to.
+ */
+goog.exportProperty = function(object, publicName, symbol) {
+ object[publicName] = symbol;
+};
+
+
+/**
+ * Inherit the prototype methods from one constructor into another.
+ *
+ * Usage:
+ * <pre>
+ * function ParentClass(a, b) { }
+ * ParentClass.prototype.foo = function(a) { }
+ *
+ * function ChildClass(a, b, c) {
+ * ParentClass.call(this, a, b);
+ * }
+ *
+ * goog.inherits(ChildClass, ParentClass);
+ *
+ * var child = new ChildClass('a', 'b', 'see');
+ * child.foo(); // works
+ * </pre>
+ *
+ * In addition, a superclass' implementation of a method can be invoked
+ * as follows:
+ *
+ * <pre>
+ * ChildClass.prototype.foo = function(a) {
+ * ChildClass.superClass_.foo.call(this, a);
+ * // other code
+ * };
+ * </pre>
+ *
+ * @param {Function} childCtor Child class.
+ * @param {Function} parentCtor Parent class.
+ */
+goog.inherits = function(childCtor, parentCtor) {
+ /** @constructor */
+ function tempCtor() {};
+ tempCtor.prototype = parentCtor.prototype;
+ childCtor.superClass_ = parentCtor.prototype;
+ childCtor.prototype = new tempCtor();
+ childCtor.prototype.constructor = childCtor;
+};
+ </script>
+ </head>
+ <body>
+ <div>
+ Hello, mod_pagespeed!
+ </div>
+ </body>
+</html>
diff --git a/trunk/src/install/mod_pagespeed_example/remove_comments.html b/trunk/src/install/mod_pagespeed_example/remove_comments.html
new file mode 100644
index 0000000..3adc442
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/remove_comments.html
@@ -0,0 +1,10 @@
+<html>
+<body>
+<!-- This comment will be removed -->
+<div>Hello, world!</div>
+<!-- Apply IE-specific CSS -->
+<!-- [if IE ]>
+<link href="iecss.css" rel="stylesheet" type="text/css" title="This IE directive will be preserved">
+<![endif]-->
+</body>
+</html>
diff --git a/trunk/src/install/mod_pagespeed_example/remove_quotes.html b/trunk/src/install/mod_pagespeed_example/remove_quotes.html
new file mode 100644
index 0000000..ac781c7
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/remove_quotes.html
@@ -0,0 +1,7 @@
+<html>
+ <head>
+ </head>
+ <body>
+ <img src="images/BikeCrashIcn.png" align='right' alt="" border="2" width='70' height='70'>
+ </body>
+</html>
diff --git a/trunk/src/install/mod_pagespeed_example/rewrite_css.html b/trunk/src/install/mod_pagespeed_example/rewrite_css.html
new file mode 100644
index 0000000..815c8c6
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/rewrite_css.html
@@ -0,0 +1,14 @@
+<html><head><style type='text/css'>
+ #iddy, #anotherId {
+ /* This comment will be stripped.*/
+ border: solid 1px #cccccc;
+ padding: 1.2em;
+ float: left;
+ color:WindowText;
+ }
+ div.classy span, div.classy img {
+ display: block;
+ border: none !important;
+ background: none !important;
+ }
+</style></head><body>Yo, <div class="classy" id="iddy">this page is <span>stylin'</span>.</div></body></html>
diff --git a/trunk/src/install/mod_pagespeed_example/rewrite_images.html b/trunk/src/install/mod_pagespeed_example/rewrite_images.html
new file mode 100644
index 0000000..04aa4d6
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/rewrite_images.html
@@ -0,0 +1,10 @@
+<html>
+ <head>
+ <title>Rewrite Images example</title>
+ </head>
+ <body>
+ <img src="images/Puzzle.jpg" width="1023" height="766" title="A large image. mod_pagespeed will recompress it."/><br/>
+ <img src="images/Puzzle.jpg" width="256" height="192" title="The same image, but with smaller width and height attributes. mod_pagespeed will resize it."/><br/>
+ <img src="images/Cuppa.png" title="A small image. mod_pagespeed will inline it."/><br/>
+ </body>
+</html>
diff --git a/trunk/src/install/mod_pagespeed_example/rewrite_javascript.html b/trunk/src/install/mod_pagespeed_example/rewrite_javascript.html
new file mode 100644
index 0000000..4135181
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/rewrite_javascript.html
@@ -0,0 +1,44 @@
+<html>
+ <head>
+ </head>
+ <body>
+ <p>
+ Expected: Internal 0 External 1 External 2<br/>
+ Actual:
+ <script type="text/javascript" id="int1">
+ // This comment will be removed
+ var state = 0;
+ document.write("Internal " + state);
+ state = 1
+ </script>
+ <script src="rewrite_javascript.js" type="text/javascript" id="ext1">
+ </script>
+ <script src="rewrite_javascript.js" id="ext2">
+ /* Note that the contents and comments of a script block are preserved
+ if a src is specified. Since they are not executed, it is assumed the
+ code will be using the script block contents as data. */
+ document.write("Internal script; state = " + state); state = 42
+ </script>
+ </p><p>
+ External script url:
+ <script type="text/javascript">
+ /* This comment will also be removed */
+ document.write(document.getElementById("ext1").src);
+ </script>
+ </p><p>
+ Comment in internal script:
+ <script type="text/javascript">
+ // This comment will be removed too.
+ var source = document.getElementById("int1").innerHTML;
+ document.write(source.match("comment") ? "yes" : "no");
+ </script>
+ </p><p>
+ Comment in external script:
+ <script type="text/javascript">
+ /* Of course, this comment will be removed as well. */
+ var source = document.getElementById("ext2").innerHTML;
+ document.write(source.match("comment") ? "yes" : "no");
+ </script>
+ </p>
+ </body>
+</html>
diff --git a/trunk/src/install/mod_pagespeed_example/rewrite_javascript.js b/trunk/src/install/mod_pagespeed_example/rewrite_javascript.js
new file mode 100644
index 0000000..21e1f97
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/rewrite_javascript.js
@@ -0,0 +1,7 @@
+/*
+ * This external script is part of the rewrite_javascript example.
+ * This comment will be removed.
+ */
+// First inject the state into the document. This comment will also be removed.
+ document.write("External " + state);
+ state += 1; // Then update it.
diff --git a/trunk/src/install/mod_pagespeed_example/stress_test.html b/trunk/src/install/mod_pagespeed_example/stress_test.html
new file mode 100644
index 0000000..43ec720
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/stress_test.html
@@ -0,0 +1,20 @@
+<html>
+ <head>
+ <title>Stress Test</title>
+ </head>
+ <body>
+ This page includes a bunch of slow images for stress-testing mod_pagespeed.
+ <img src="cgi/slow.cgi?id=0"/><br/>
+ <img src="cgi/slow.cgi?id=1"/><br/>
+ <img src="cgi/slow.cgi?id=2"/><br/>
+ <img src="cgi/slow.cgi?id=3"/><br/>
+ <img src="cgi/slow.cgi?id=4"/><br/>
+ <img src="cgi/slow.cgi?id=5"/><br/>
+ <img src="cgi/slow.cgi?id=6"/><br/>
+ <img src="cgi/slow.cgi?id=7"/><br/>
+ <img src="cgi/slow.cgi?id=8"/><br/>
+ <img src="cgi/slow.cgi?id=9"/><br/>
+ <img src="cgi/slow.cgi?id=10"/><br/>
+ <img src="cgi/slow.cgi?id=11"/><br/>
+ </body>
+</html>
diff --git a/trunk/src/install/mod_pagespeed_example/styles/all_styles.css b/trunk/src/install/mod_pagespeed_example/styles/all_styles.css
new file mode 100644
index 0000000..a6214b8
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/styles/all_styles.css
@@ -0,0 +1,4 @@
+.yellow {background-color: yellow;}
+.blue {color: blue;}
+.big { font-size: 8em; }
+.bold { font-weight: bold; }
diff --git a/trunk/src/install/mod_pagespeed_example/styles/big.css b/trunk/src/install/mod_pagespeed_example/styles/big.css
new file mode 100644
index 0000000..d5f4055
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/styles/big.css
@@ -0,0 +1,48 @@
+.big {
+ font-size: 8em;
+}
+
+/*
+ * The class names below are just here to prevent the minifier and inlining from
+ * working on this file, to allow the css combiner to be demonstrated.
+ */
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_a0 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_a1 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_a0 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_a1 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_a0 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_a1 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_a6 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_a7 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_a8 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_a9 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_b0 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_b1 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_b2 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_b3 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_b4 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_b5 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_b6 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_b7 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_b8 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_b9 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_c0 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_c1 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_c2 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_c3 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_c4 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_c5 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_c6 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_c7 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_c8 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_c9 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_d0 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_d1 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_d2 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_d3 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_d4 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_d5 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_d6 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_d7 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_d8 { font-size: 8em; }
+.very_large_class_name_that_will_be_hard_to_minify_to_keep_file_above_threshold_d9 { font-size: 8em; }
diff --git a/trunk/src/install/mod_pagespeed_example/styles/blue.css b/trunk/src/install/mod_pagespeed_example/styles/blue.css
new file mode 100644
index 0000000..84d7d00
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/styles/blue.css
@@ -0,0 +1 @@
+.blue {color: blue;}
diff --git a/trunk/src/install/mod_pagespeed_example/styles/bold.css b/trunk/src/install/mod_pagespeed_example/styles/bold.css
new file mode 100644
index 0000000..81d40c0
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/styles/bold.css
@@ -0,0 +1,3 @@
+.bold {
+ font-weight: bold;
+}
diff --git a/trunk/src/install/mod_pagespeed_example/styles/yellow.css b/trunk/src/install/mod_pagespeed_example/styles/yellow.css
new file mode 100644
index 0000000..f8f5895
--- /dev/null
+++ b/trunk/src/install/mod_pagespeed_example/styles/yellow.css
@@ -0,0 +1 @@
+.yellow {background-color: yellow;}
diff --git a/trunk/src/install/proxy.conf.template b/trunk/src/install/proxy.conf.template
new file mode 100644
index 0000000..08342bd
--- /dev/null
+++ b/trunk/src/install/proxy.conf.template
@@ -0,0 +1,28 @@
+# Proxy configuration file to enable Instaweb to rewrite external
+# content. In this configuration we use assume a browser proxy,
+# pointing to localhost:80.
+
+LoadModule proxy_module APACHE_MODULES/mod_proxy.so
+# Depends: proxy
+LoadModule proxy_http_module APACHE_MODULES/mod_proxy_http.so
+
+<IfModule mod_proxy.c>
+ ProxyRequests On
+ ProxyVia On
+
+ # limit connections to LAN clients
+ <Proxy *>
+ AddDefaultCharset off
+ Order Deny,Allow
+ Allow from all
+ </Proxy>
+ ProxyPreserveHost On
+ ProxyStatus On
+ ProxyBadHeader Ignore
+
+ # Enable/disable the handling of HTTP/1.1 "Via:" headers.
+ # ("Full" adds the server version; "Block" removes all outgoing Via: headers)
+ # Set to one of: Off | On | Full | Block
+ ProxyVia On
+
+</IfModule>
diff --git a/trunk/src/install/rpm/build.sh b/trunk/src/install/rpm/build.sh
new file mode 100755
index 0000000..c98622d
--- /dev/null
+++ b/trunk/src/install/rpm/build.sh
@@ -0,0 +1,230 @@
+#!/bin/bash
+#
+# Copyright (c) 2009 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+set -e
+if [ "$VERBOSE" ]; then
+ set -x
+fi
+set -u
+
+gen_spec() {
+ rm -f "${SPEC}"
+ process_template "${SCRIPTDIR}/mod-pagespeed.spec.template" "${SPEC}"
+}
+
+# Setup the installation directory hierachy in the package staging area.
+prep_staging_rpm() {
+ prep_staging_common
+ install -m 755 -d "${STAGEDIR}/etc/cron.daily"
+}
+
+# Put the package contents in the staging area.
+stage_install_rpm() {
+ prep_staging_rpm
+ stage_install_common
+ echo "Staging RPM install files in '${STAGEDIR}'..."
+ process_template "${BUILDDIR}/install/common/rpmrepo.cron" \
+ "${STAGEDIR}/etc/cron.daily/${PACKAGE}"
+ chmod 755 "${STAGEDIR}/etc/cron.daily/${PACKAGE}"
+
+ # For CentOS, the load and conf files are combined into a single
+ # 'conf' file. So we install the load template as the conf file, and
+ # then concatenate the actual conf file.
+ process_template "${BUILDDIR}/install/common/pagespeed.load.template" \
+ "${STAGEDIR}${APACHE_CONFDIR}/pagespeed.conf"
+ process_template "${BUILDDIR}/install/common/pagespeed.conf.template" \
+ "${BUILDDIR}/install/common/pagespeed.conf"
+ cat "${BUILDDIR}/install/common/pagespeed.conf" >> \
+ "${STAGEDIR}${APACHE_CONFDIR}/pagespeed.conf"
+ chmod 644 "${STAGEDIR}${APACHE_CONFDIR}/pagespeed.conf"
+}
+
+# Actually generate the package file.
+do_package() {
+ echo "Packaging ${HOST_ARCH}..."
+ PROVIDES="${PACKAGE}"
+ local REPS="$REPLACES"
+ REPLACES=""
+ for rep in $REPS; do
+ if [ -z "$REPLACES" ]; then
+ REPLACES="$PACKAGE-$rep"
+ else
+ REPLACES="$REPLACES $PACKAGE-$rep"
+ fi
+ done
+
+ # If we specify a dependecy of foo.so below, we would depend on both the
+ # 32 and 64-bit versions on a 64-bit machine. The current version of RPM
+ # we use is too old and doesn't provide %{_isa}, so we do this manually.
+ if [ "$HOST_ARCH" = "x86_64" ] ; then
+ local EMPTY_VERSION="()"
+ local PKG_ARCH="(64bit)"
+ elif [ "$HOST_ARCH" = "i386" ] ; then
+ local EMPTY_VERSION=""
+ local PKG_ARCH=""
+ fi
+
+ DEPENDS="httpd >= 2.2, \
+ libstdc++ >= 4.1.2, \
+ at"
+ gen_spec
+
+ # Create temporary rpmbuild dirs.
+ RPMBUILD_DIR=$(mktemp -d -t rpmbuild.XXXXXX) || exit 1
+ mkdir -p "$RPMBUILD_DIR/BUILD"
+ mkdir -p "$RPMBUILD_DIR/RPMS"
+
+ rpmbuild --buildroot="$RPMBUILD_DIR/BUILD" -bb \
+ --target="$HOST_ARCH" --rmspec \
+ --define "_topdir $RPMBUILD_DIR" \
+ --define "_binary_payload w9.bzdio" \
+ "${SPEC}"
+ PKGNAME="${PACKAGE}-${CHANNEL}-${VERSION}-${REVISION}"
+ mv "$RPMBUILD_DIR/RPMS/$HOST_ARCH/${PKGNAME}.${HOST_ARCH}.rpm" "${OUTPUTDIR}"
+ # Make sure the package is world-readable, otherwise it causes problems when
+ # copied to share drive.
+ chmod a+r "${OUTPUTDIR}/${PKGNAME}.$HOST_ARCH.rpm"
+ rm -rf "$RPMBUILD_DIR"
+}
+
+# Remove temporary files and unwanted packaging output.
+cleanup() {
+ rm -rf "${STAGEDIR}"
+ rm -rf "${TMPFILEDIR}"
+}
+
+usage() {
+ echo "usage: $(basename $0) [-c channel] [-a target_arch] [-o 'dir'] [-b 'dir']"
+ echo "-c channel the package channel (unstable, beta, stable)"
+ echo "-a arch package architecture (ia32 or x64)"
+ echo "-o dir package output directory [${OUTPUTDIR}]"
+ echo "-b dir build input directory [${BUILDDIR}]"
+ echo "-h this help message"
+}
+
+# Check that the channel name is one of the allowable ones.
+verify_channel() {
+ case $CHANNEL in
+ stable )
+ CHANNEL=stable
+ REPLACES="unstable beta"
+ ;;
+ unstable|dev|alpha )
+ CHANNEL=unstable
+ REPLACES="stable beta"
+ ;;
+ testing|beta )
+ CHANNEL=beta
+ REPLACES="unstable stable"
+ ;;
+ * )
+ echo
+ echo "ERROR: '$CHANNEL' is not a valid channel type."
+ echo
+ exit 1
+ ;;
+ esac
+}
+
+process_opts() {
+ while getopts ":o:b:c:a:h" OPTNAME
+ do
+ case $OPTNAME in
+ o )
+ OUTPUTDIR="$OPTARG"
+ mkdir -p "${OUTPUTDIR}"
+ ;;
+ b )
+ BUILDDIR=$(readlink -f "${OPTARG}")
+ ;;
+ c )
+ CHANNEL="$OPTARG"
+ verify_channel
+ ;;
+ a )
+ TARGETARCH="$OPTARG"
+ ;;
+ h )
+ usage
+ exit 0
+ ;;
+ \: )
+ echo "'-$OPTARG' needs an argument."
+ usage
+ exit 1
+ ;;
+ * )
+ echo "invalid command-line option: $OPTARG"
+ usage
+ exit 1
+ ;;
+ esac
+ done
+}
+
+#=========
+# MAIN
+#=========
+
+SCRIPTDIR=$(readlink -f "$(dirname "$0")")
+OUTPUTDIR="${PWD}"
+STAGEDIR=$(mktemp -d -t rpm.build.XXXXXX) || exit 1
+TMPFILEDIR=$(mktemp -d -t rpm.tmp.XXXXXX) || exit 1
+CHANNEL="beta"
+# Default target architecture to same as build host.
+if [ "$(uname -m)" = "x86_64" ]; then
+ TARGETARCH="x64"
+else
+ TARGETARCH="ia32"
+fi
+SPEC="${TMPFILEDIR}/mod-pagespeed.spec"
+
+# call cleanup() on exit
+trap cleanup 0
+process_opts "$@"
+if [ ! "$BUILDDIR" ]; then
+ BUILDDIR=$(readlink -f "${SCRIPTDIR}/../../out/Release")
+fi
+
+source ${BUILDDIR}/install/common/installer.include
+
+get_version_info
+
+source "${BUILDDIR}/install/common/mod-pagespeed.info"
+eval $(sed -e "s/^\([^=]\+\)=\(.*\)$/export \1='\2'/" \
+ "${BUILDDIR}/install/common/BRANDING")
+
+REPOCONFIG="http://dl.google.com/linux/${PACKAGE#google-}/rpm/stable"
+verify_channel
+
+APACHE_CONFDIR="/etc/httpd/conf.d"
+MODPAGESPEED_CACHE_ROOT="/var/www/mod_pagespeed"
+APACHE_USER="apache"
+COMMENT_OUT_DEFLATE=
+
+# Make everything happen in the OUTPUTDIR.
+cd "${OUTPUTDIR}"
+
+case "$TARGETARCH" in
+ ia32 )
+ export APACHE_MODULEDIR="/usr/lib/httpd/modules"
+ export HOST_ARCH="i386"
+ stage_install_rpm
+ ;;
+ x64 )
+ export APACHE_MODULEDIR="/usr/lib64/httpd/modules"
+ export HOST_ARCH="x86_64"
+ stage_install_rpm
+ ;;
+ * )
+ echo
+ echo "ERROR: Don't know how to build RPMs for '$TARGETARCH'."
+ echo
+ exit 1
+ ;;
+esac
+
+do_package "$HOST_ARCH"
diff --git a/trunk/src/install/rpm/mod-pagespeed.spec.template b/trunk/src/install/rpm/mod-pagespeed.spec.template
new file mode 100644
index 0000000..f399f42
--- /dev/null
+++ b/trunk/src/install/rpm/mod-pagespeed.spec.template
@@ -0,0 +1,172 @@
+#------------------------------------------------------------------------------
+# mod-pagespeed.spec
+#------------------------------------------------------------------------------
+
+#------------------------------------------------------------------------------
+# Prologue information
+#------------------------------------------------------------------------------
+Summary : @@SHORTDESC@@
+License : Apache Software License
+Name : @@PACKAGE@@-@@CHANNEL@@
+Version : @@VERSION@@
+Release : @@REVISION@@
+Group : System Environment/Daemons
+Vendor : @@COMPANY_FULLNAME@@
+Url : @@PRODUCTURL@@
+Packager : @@MAINTNAME@@ <@@MAINTMAIL@@>
+
+#------------------------------------------------------------------------------
+# Tested on:
+# TODO
+#------------------------------------------------------------------------------
+
+Provides : @@PROVIDES@@ = %{version}
+Requires : @@DEPENDS@@
+Conflicts : @@REPLACES@@
+
+#------------------------------------------------------------------------------
+# Description
+#------------------------------------------------------------------------------
+%Description
+@@FULLDESC@@
+
+#------------------------------------------------------------------------------
+# Build rule - How to make the package
+#------------------------------------------------------------------------------
+%build
+
+#------------------------------------------------------------------------------
+# Installation rule - how to install it (note that it
+# gets installed into a temp directory given by $RPM_BUILD_ROOT)
+#------------------------------------------------------------------------------
+%install
+rm -rf "$RPM_BUILD_ROOT"
+
+if [ -z "@@STAGEDIR@@" -o ! -d "@@STAGEDIR@@" ] ; then
+ echo "@@STAGEDIR@@ appears to be incorrectly set - aborting"
+ exit 1
+fi
+
+install -m 755 -d \
+ "$RPM_BUILD_ROOT/etc" \
+ "$RPM_BUILD_ROOT/usr" \
+ "$RPM_BUILD_ROOT/var"
+# This is hard coded for now
+cp -a "@@STAGEDIR@@/etc/" "$RPM_BUILD_ROOT/"
+cp -a "@@STAGEDIR@@/usr/" "$RPM_BUILD_ROOT/"
+cp -a "@@STAGEDIR@@/var/" "$RPM_BUILD_ROOT/"
+
+#------------------------------------------------------------------------------
+# Rule to clean up a build
+#------------------------------------------------------------------------------
+%clean
+rm -rf "$RPM_BUILD_ROOT"
+
+#------------------------------------------------------------------------------
+# Files listing.
+#------------------------------------------------------------------------------
+%files
+%defattr(-,root,root)
+@@APACHE_MODULEDIR@@/mod_pagespeed.so
+%config(noreplace) @@APACHE_CONFDIR@@/pagespeed.conf
+/etc/cron.daily/mod-pagespeed
+%attr(-, @@APACHE_USER@@, @@APACHE_USER@@) @@MODPAGESPEED_CACHE_ROOT@@/cache
+%attr(-, @@APACHE_USER@@, @@APACHE_USER@@) @@MODPAGESPEED_CACHE_ROOT@@/files
+
+#------------------------------------------------------------------------------
+# Pre install script
+#------------------------------------------------------------------------------
+%pre
+
+exit 0
+
+
+
+
+#------------------------------------------------------------------------------
+# Post install script
+#------------------------------------------------------------------------------
+%post
+
+@@include@@../common/rpm.include
+
+MODPAGESPEED_ENABLE_UPDATES=@@MODPAGESPEED_ENABLE_UPDATES@@
+
+DEFAULTS_FILE="/etc/default/@@PACKAGE@@"
+if [ -n "${MODPAGESPEED_ENABLE_UPDATES}" -a ! -e "$DEFAULTS_FILE" ]; then
+ echo 'repo_add_once="true"' > "$DEFAULTS_FILE"
+fi
+
+if [ -e "$DEFAULTS_FILE" ]; then
+. "$DEFAULTS_FILE"
+
+if [ "$repo_add_once" = "true" ]; then
+ determine_rpm_package_manager
+
+ case $PACKAGEMANAGER in
+ "yum")
+ install_yum
+ ;;
+ "urpmi")
+ install_urpmi
+ ;;
+ "yast")
+ install_yast
+ ;;
+ esac
+fi
+
+# Some package managers have locks that prevent everything from being
+# configured at install time, so wait a bit then kick the cron job to do
+# whatever is left. Probably the db will be unlocked by then, but if not, the
+# cron job will keep retrying.
+# Do this with 'at' instead of a backgrounded shell because zypper waits on all
+# sub-shells to finish before it finishes, which is exactly the opposite of
+# what we want here. Also preemptively start atd because for some reason it's
+# not always running, which kind of defeats the purpose of having 'at' as a
+# required LSB command.
+service atd start
+echo "sh /etc/cron.daily/@@PACKAGE@@" | at now + 2 minute
+fi
+
+exit 0
+
+
+#------------------------------------------------------------------------------
+# Pre uninstallation script
+#------------------------------------------------------------------------------
+%preun
+
+if [ "$1" -eq "0" ]; then
+ mode="uninstall"
+elif [ "$1" -eq "1" ]; then
+ mode="upgrade"
+fi
+
+@@include@@../common/rpm.include
+
+# On Debian we only remove when we purge. However, RPM has no equivalent to
+# dpkg --purge, so this is all disabled.
+#
+#determine_rpm_package_manager
+#
+#case $PACKAGEMANAGER in
+#"yum")
+# remove_yum
+# ;;
+#"urpmi")
+# remove_urpmi
+# ;;
+#"yast")
+# remove_yast
+# ;;
+#esac
+
+exit 0
+
+#------------------------------------------------------------------------------
+# Post uninstallation script
+#------------------------------------------------------------------------------
+%postun
+
+exit 0
diff --git a/trunk/src/install/stress_test.sh b/trunk/src/install/stress_test.sh
new file mode 100755
index 0000000..4e11c2a
--- /dev/null
+++ b/trunk/src/install/stress_test.sh
@@ -0,0 +1,64 @@
+#!/bin/bash
+# Copyright 2010 Google Inc. All Rights Reserved.
+# Author: abliss@google.com (Adam Bliss)
+#
+# Usage: ./stress_test.sh HOSTPORT
+# Stress-tests a mod_pagespeed installation. This currently takes about 15sec.
+# Exits with status 0 if all tests pass. Exits 1 immediately if any test fails.
+# You should probably wipe out your cache and restart the server before starting
+# the test.
+
+if [ $# != 1 ]; then
+ echo Usage: ./stress_test.sh HOSTPORT;
+ exit 2;
+fi;
+HOSTPORT=$1
+
+TEST_DIR=/tmp/mod_pagespeed_stress_test.$USER;
+mkdir -p $TEST_DIR;
+cd $TEST_DIR;
+
+echo "Starting 10 simultaneous recursive wgets"
+X=0;
+PIDS="";
+while [ $X -lt 10 ]; do
+ wget -q -P $X -p http://$HOSTPORT/mod_pagespeed_example/stress_test.html &
+ PIDS="$PIDS $!";
+ X=$((X+1));
+done;
+
+# Monitor the number of processes for 10 seconds
+echo;
+echo;
+MAX=0;
+X=22;
+
+if [ $TERM == dumb ]; then
+ OVERWRITE=""
+else
+ OVERWRITE="\033[2A"
+fi
+
+while [ $X -ge 0 ]; do
+ NUM=` ps -efww|egrep 'bin/[h]ttpd|bin/[a]pache'|wc -l`
+ if [ $NUM -gt $MAX ]; then
+ MAX=$NUM;
+ fi;
+ /bin/echo -e "${OVERWRITE}Apache processes: $NUM ";
+ echo "Time remaining: $((X/2)) "
+ sleep 0.5;
+ X=$((X-1))
+done;
+
+echo "Test complete; killing wgets"
+kill $PIDS 2>/dev/null;
+
+if [ $MAX -gt 100 ]; then
+ echo "FAIL: $MAX processes were spawned.";
+ RETURN_VAL=1;
+else
+ echo "PASS."
+ RETURN_VAL=0;
+fi;
+
+exit $RETURN_VAL;
diff --git a/trunk/src/install/system_test.sh b/trunk/src/install/system_test.sh
new file mode 100755
index 0000000..71defea
--- /dev/null
+++ b/trunk/src/install/system_test.sh
@@ -0,0 +1,321 @@
+#!/bin/bash
+# Copyright 2010 Google Inc. All Rights Reserved.
+# Author: abliss@google.com (Adam Bliss)
+#
+# Usage: ./system_test.sh HOSTNAME
+# Tests a mod_pagespeed installation by fetching and verifying all the examples.
+# Exits with status 0 if all tests pass. Exits 1 immediately if any test fails.
+
+if [ $# != 1 ]; then
+ echo Usage: ./system_test.sh HOSTNAME
+ exit 2
+fi;
+
+# If the user has specified an alternate WGET as an environment variable, then
+# use that, otherwise use the one in the path.
+if [ "$WGET" == "" ]; then
+ WGET=wget
+else
+ echo WGET = $WGET
+fi
+
+$WGET --version | head -1 | grep 1.12 >/dev/null
+if [ $? != 0 ]; then
+ echo You have the wrong version of wget. 1.12 is required.
+ exit 1
+fi
+
+HOSTNAME=$1
+PORT=${HOSTNAME/*:/};
+if [ $PORT = $HOSTNAME ]; then
+ PORT=80
+fi;
+EXAMPLE_ROOT=http://$HOSTNAME/mod_pagespeed_example
+STATISTICS_URL=http://localhost:$PORT/mod_pagespeed_statistics
+BAD_RESOURCE_URL=http://$HOSTNAME/mod_pagespeed/bad.pagespeed.cf.hash.css
+
+OUTDIR=/tmp/mod_pagespeed_test.$USER/fetched_directory
+rm -rf $OUTDIR
+
+# Wget is used three different ways. The first way is nonrecursive and dumps a
+# single page (with headers) to standard out. This is useful for grepping for a
+# single expected string that's the result of a first-pass rewrite:
+# wget -q -O --save-headers - $URL | grep -q foo
+# "-q" quells wget's noisy output; "-O -" dumps to stdout; grep's -q quells
+# its output and uses the return value to indicate whether the string was
+# found. Note that exiting with a nonzero value will immediately kill
+# the make run.
+#
+# Sometimes we want to check for a condition that's not true on the first dump
+# of a page, but becomes true after a few seconds as the server's asynchronous
+# fetches complete. For this we use the the fetch_until() function:
+# fetch_until $URL 'grep -c delayed_foo' 1
+# In this case we will continuously fetch $URL and pipe the output to
+# grep -c (which prints the count of matches); we repeat until the number is 1.
+#
+# The final way we use wget is in a recursive mode to download all prerequisites
+# of a page. This fetches all resources associated with the page, and thereby
+# validates the resources generated by mod_pagespeed:
+# wget -H -p -S -o $WGET_OUTPUT -nd -P $OUTDIR $EXAMPLE_ROOT/$FILE
+# Here -H allows wget to cross hosts (e.g. in the case of a sharded domain); -p
+# means to fetch all prerequisites; "-S -o $WGET_OUTPUT" saves wget output
+# (including server headers) for later analysis; -nd puts all results in one
+# directory; -P specifies that directory. We can then run commands on
+# $OUTDIR/$FILE and nuke $OUTDIR when we're done.
+# TODO(abliss): some of these will fail on windows where wget escapes saved
+# filenames differently.
+
+WGET_OUTPUT=$OUTDIR/wget_output.txt
+WGET_DUMP="$WGET -q -O - --save-headers"
+WGET_PREREQ="$WGET -H -p -S -o $WGET_OUTPUT -nd -P $OUTDIR"
+
+# Call with a command and its args. Echos the command, then tries to eval it.
+# If it returns false, fail the tests.
+function check() {
+ echo " " $@
+ if eval "$@"; then
+ return;
+ else
+ echo FAIL.
+ exit 1;
+ fi;
+}
+
+# Continously fetches URL and pipes the output to COMMAND. Loops until
+# COMMAND outputs RESULT, in which case we return 0, or until 10 seconds have
+# passed, in which case we return 1.
+function fetch_until() {
+ URL=$1
+ COMMAND=$2
+ RESULT=$3
+
+ TIMEOUT=10
+ START=`date +%s`
+ STOP=$((START+$TIMEOUT))
+
+ echo " " Fetching $URL until '`'$COMMAND'`' = $RESULT
+ while test -t; do
+ if [ `$WGET -q -O - $URL 2>&1 | $COMMAND` = $RESULT ]; then
+ /bin/echo "."
+ return;
+ fi;
+ if [ `date +%s` -gt $STOP ]; then
+ /bin/echo "FAIL."
+ exit 1;
+ fi;
+ /bin/echo -n "."
+ sleep 0.1
+ done;
+}
+
+# Helper to set up most filter tests
+function test_filter() {
+ rm -rf $OUTDIR
+ mkdir -p $OUTDIR
+ FILTER_NAME=$1;
+ shift;
+ FILTER_DESCRIPTION=$@
+ echo TEST: $FILTER_NAME $FILTER_DESCRIPTION
+ FILE=$FILTER_NAME.html?ModPagespeedFilters=$FILTER_NAME
+ URL=$EXAMPLE_ROOT/$FILE
+ FETCHED=$OUTDIR/$FILE
+}
+
+
+# General system tests
+
+echo TEST: mod_pagespeed is running in Apache and writes the expected header.
+HTML_HEADERS=$($WGET_DUMP $EXAMPLE_ROOT/combine_css.html)
+
+echo Checking for X-Mod-Pagespeed header
+echo $HTML_HEADERS | grep -qi X-Mod-Pagespeed
+check [ $? = 0 ]
+
+echo Checking for lack of E-tag
+echo $HTML_HEADERS | grep -qi Etag
+check [ $? != 0 ]
+
+echo Checking for presence of Vary.
+echo $HTML_HEADERS | grep -qi 'Vary: Accept-Encoding'
+check [ $? = 0 ]
+
+echo Checking for absence of Last-Modified
+echo $HTML_HEADERS | grep -qi 'Last-Modified'
+check [ $? != 0 ]
+
+echo Checking for presence of Cache-control: no-cache
+echo $HTML_HEADERS | grep -qi 'Cache-Control: max-age=0, no-cache, no-store'
+check [ $? = 0 ]
+
+echo Checking for absense of Expires
+echo $HTML_HEADERS | grep -qi 'Expires'
+check [ $? != 0 ]
+
+# Determine whether statistics are enabled or not. If not, don't test them.
+grep "# ModPagespeedStatistics off" /usr/local/apache2/conf/pagespeed.conf \
+ >/dev/null
+if [ $? = 0 ]; then
+ echo TEST: 404s are served and properly recorded.
+ NUM_404=$($WGET_DUMP $STATISTICS_URL | grep resource_404_count | cut -d: -f2)
+ NUM_404=$(($NUM_404+1))
+ check "$WGET -O /dev/null $BAD_RESOURCE_URL 2>&1| grep -q '404 Not Found'"
+ check "$WGET_DUMP $STATISTICS_URL | grep -q 'resource_404_count: $NUM_404'"
+else
+ echo TEST: 404s are served. Statistics are disabled so not checking them.
+ check "$WGET -O /dev/null $BAD_RESOURCE_URL 2>&1| grep -q '404 Not Found'"
+fi
+
+echo TEST: directory is mapped to index.html.
+rm -rf $OUTDIR
+mkdir -p $OUTDIR
+check "$WGET_PREREQ $EXAMPLE_ROOT"
+check "$WGET_PREREQ $EXAMPLE_ROOT/index.html"
+check diff $OUTDIR/index.html $OUTDIR/mod_pagespeed_example
+
+echo TEST: compression is enabled for HTML.
+check "$WGET -O /dev/null -q -S --header='Accept-Encoding: gzip' \
+ $EXAMPLE_ROOT/ 2>&1 | grep -qi 'Content-Encoding: gzip'"
+
+# Individual filter tests, in alphabetical order
+
+test_filter add_instrumentation adds 2 script tags
+check $WGET_PREREQ $URL
+check [ `cat $FETCHED | sed 's/>/>\n/g' | grep -c '<script'` = 2 ]
+
+test_filter collapse_whitespace removes whitespace, but not from pre tags.
+check $WGET_PREREQ $URL
+check [ `egrep -c '^ +<' $FETCHED` = 1 ]
+
+test_filter combine_css combines 4 CSS files into 1.
+fetch_until $URL 'grep -c text/css' 1
+check $WGET_PREREQ $URL
+
+test_filter combine_heads combines 2 heads into 1
+check $WGET_PREREQ $URL
+check [ `grep -ce '<head>' $FETCHED` = 1 ]
+
+test_filter elide_attributes removes boolean and default attributes.
+check $WGET_PREREQ $URL
+grep "disabled=" $FETCHED # boolean, should not find
+check [ $? != 0 ]
+grep "type=" $FETCHED # default, should not find
+check [ $? != 0 ]
+
+test_filter extend_cache rewrites an image tag.
+fetch_until $URL 'grep -c src.*91_WewrLtP' 1
+check $WGET_PREREQ $URL
+
+echo TEST: Cache-extended image should respond 304 to an If-Modified-Since.
+URL=$EXAMPLE_ROOT/images/Puzzle.jpg.pagespeed.ce.91_WewrLtP.jpg
+DATE=`date -R`
+$WGET_PREREQ --header "If-Modified-Since: $DATE" $URL
+check grep '"304 Not Modified"' $WGET_OUTPUT
+
+echo TEST: Legacy format URLs should still work.
+URL=$EXAMPLE_ROOT/images/ce.0123456789abcdef0123456789abcdef.Puzzle,j.jpg
+check "$WGET_DUMP $URL | grep -q 'HTTP/1.1 200 OK'"
+
+test_filter move_css_to_head does what it says on the tin.
+check $WGET_PREREQ $URL
+check grep -q "'<head><link'" $FETCHED # link moved to head
+
+test_filter inline_css converts a link tag to a style tag
+fetch_until $URL 'grep -c style' 2
+
+test_filter inline_javascript inlines a small JS file
+fetch_until $URL 'grep -c document.write' 1
+
+test_filter outline_css outlines large styles, but not small ones.
+check $WGET_PREREQ $URL
+check egrep -q "'<link.*text/css.*large'" $FETCHED # outlined
+check egrep -q "'<style.*small'" $FETCHED # not outlined
+
+test_filter outline_javascript outlines large scripts, but not small ones.
+check $WGET_PREREQ $URL
+check egrep -q "'<script.*src=.*large'" $FETCHED # outlined
+check egrep -q "'<script.*small.*var hello'" $FETCHED # not outlined
+
+echo TEST: compression is enabled for rewritten JS.
+JS_URL=$(egrep -o http://.*.pagespeed.*.js $FETCHED)
+JS_HEADERS=$($WGET -O /dev/null -q -S --header='Accept-Encoding: gzip' \
+ $JS_URL 2>&1)
+echo $JS_HEADERS | grep -qi 'Content-Encoding: gzip'
+check [ $? = 0 ]
+echo $JS_HEADERS | grep -qi 'Vary: Accept-Encoding'
+check [ $? = 0 ]
+echo $JS_HEADERS | grep -qi 'Etag: W/0'
+check [ $? = 0 ]
+echo $JS_HEADERS | grep -qi 'Last-Modified:'
+check [ $? = 0 ]
+
+test_filter remove_comments removes comments but not IE directives.
+check $WGET_PREREQ $URL
+grep "removed" $FETCHED # comment, should not find
+check [ $? != 0 ]
+check grep -q preserved $FETCHED # preserves IE directives
+
+test_filter remove_quotes does what it says on the tin.
+check $WGET_PREREQ $URL
+check [ `sed 's/ /\n/g' $FETCHED | grep -c '"' ` = 2 ] # 2 quoted attrs
+check [ `grep -c "'" $FETCHED` = 0 ] # no apostrophes
+
+test_filter rewrite_css removes comments and saves a bunch of bytes.
+check $WGET_PREREQ $URL
+grep "comment" $FETCHED # comment, should not find
+check [ $? != 0 ]
+check [ `stat -c %s $FETCHED` -lt 315 ] # down from 472
+
+test_filter rewrite_images inlines, compresses, and resizes.
+fetch_until $URL 'grep -c image/png' 1 # inlined
+check $WGET_PREREQ $URL
+check [ `stat -c %s $OUTDIR/*1023x766*Puzzle*` -lt 241260 ] # compressed
+check [ `stat -c %s $OUTDIR/*256x192*Puzzle*` -lt 24126 ] # resized
+
+IMG_URL=$(egrep -o http://.*.pagespeed.*.jpg $FETCHED | head -n1)
+echo TEST: headers for rewrritten image "$IMG_URL"
+IMG_HEADERS=$($WGET -O /dev/null -q -S --header='Accept-Encoding: gzip' \
+ $IMG_URL 2>&1)
+# Make sure we have some valid headers.
+echo \"$IMG_HEADERS\" | grep -qi 'Content-Type: image/jpeg'
+check [ $? = 0 ]
+
+# Make sure the response was not gzipped.
+echo TEST: Images are not gzipped
+echo "$IMG_HEADERS" | grep -qi 'Content-Encoding: gzip'
+check [ $? != 0 ]
+
+# Make sure there is no vary-encoding
+echo TEST: Vary is not set for images
+echo "$IMG_HEADERS" | grep -qi 'Vary: Accept-Encoding'
+check [ $? != 0 ]
+
+# Make sure there is an etag
+echo TEST: Etags is present
+echo "$IMG_HEADERS" | grep -qi 'Etag: W/0'
+check [ $? = 0 ]
+
+# Make sure there is a last-modified tag
+echo TEST: Last-modified is present
+echo "$IMG_HEADERS" | grep -qi 'Last-Modified'
+check [ $? = 0 ]
+
+IMG_URL=${IMG_URL/Puzzle/BadName}
+echo TEST: rewrite_images redirects unknown image $IMG_URL
+$WGET_PREREQ $IMG_URL; # fails
+check grep '"307 Temporary Redirect"' $WGET_OUTPUT
+
+
+test_filter rewrite_javascript removes comments and saves a bunch of bytes.
+fetch_until $URL 'grep -c src.*1o978_K0_L' 2 # external scripts rewritten
+check $WGET_PREREQ $URL
+grep -R "removed" $OUTDIR # comments, should not find any
+check [ $? != 0 ]
+check [ `stat -c %s $FETCHED` -lt 1560 ] # net savings
+check grep -q preserved $FETCHED # preserves certain comments
+# rewritten JS is cache-extended
+check grep -qi "'Cache-control: max-age=31536000'" $WGET_OUTPUT
+check grep -qi "'Expires:'" $WGET_OUTPUT
+
+# Cleanup
+rm -rf $OUTDIR
+echo "PASS."
diff --git a/trunk/src/install/ubuntu.sh b/trunk/src/install/ubuntu.sh
new file mode 100755
index 0000000..747a0c1
--- /dev/null
+++ b/trunk/src/install/ubuntu.sh
@@ -0,0 +1,6 @@
+#!/bin/sh
+
+exec make \
+ APACHE_ROOT=/etc/apache2 \
+ APACHE_MODULES=/usr/lib/apache2/modules \
+ $*
diff --git a/trunk/src/net/instaweb/apache/apache_config.cc b/trunk/src/net/instaweb/apache/apache_config.cc
new file mode 100644
index 0000000..5a29e51
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apache_config.cc
@@ -0,0 +1,25 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/apache/apache_config.h"
+
+namespace net_instaweb {
+
+ApacheConfig::ApacheConfig(const StringPiece& description)
+ : description_(description.data(), description.size()) {
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/apache_config.h b/trunk/src/net/instaweb/apache/apache_config.h
new file mode 100644
index 0000000..8edb9a8
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apache_config.h
@@ -0,0 +1,54 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_APACHE_APACHE_CONFIG_H_
+#define NET_INSTAWEB_APACHE_APACHE_CONFIG_H_
+
+#include "net/instaweb/rewriter/public/rewrite_options.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+// The httpd header must be after the
+// apache_rewrite_driver_factory.h. Otherwise, the compiler will
+// complain "strtoul_is_not_a_portable_function_use_strtol_instead".
+#include "httpd.h"
+
+namespace net_instaweb {
+
+// Establishes a context for directory-scoped options, either via .htaccess or
+// <Directory>...</Directory>.
+class ApacheConfig {
+ public:
+ explicit ApacheConfig(const StringPiece& dir);
+ ~ApacheConfig() {}
+
+ // Human-readable description of what this configuration is for. This
+ // may be a directory, or a string indicating a combination of directives
+ // for multiple directories.
+ StringPiece description() const { return description_; }
+
+ RewriteOptions* options() { return &options_; }
+
+ private:
+ std::string description_;
+ RewriteOptions options_;
+
+ DISALLOW_COPY_AND_ASSIGN(ApacheConfig);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_APACHE_APACHE_CONFIG_H_
diff --git a/trunk/src/net/instaweb/apache/apache_message_handler.cc b/trunk/src/net/instaweb/apache/apache_message_handler.cc
new file mode 100644
index 0000000..a7c81b7
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apache_message_handler.cc
@@ -0,0 +1,98 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/apache/apache_message_handler.h"
+
+#include "net/instaweb/apache/log_message_handler.h"
+
+#include "httpd.h"
+// When HAVE_SYSLOG is defined, apache http_log.h will include syslog.h, which
+// #defined LOG_* as numbers. This conflicts with what we are using those here.
+#undef HAVE_SYSLOG
+#include "http_log.h"
+
+namespace {
+
+// This name will be prefixed to every logged message. This could be made
+// smaller if people think it's too long. In my opinion it's probably OK,
+// and it would be good to let people know where messages are coming from.
+const char kModuleName[] = "mod_pagespeed";
+
+}
+
+namespace net_instaweb {
+
+ApacheMessageHandler::ApacheMessageHandler(const server_rec* server,
+ const StringPiece& version)
+ : server_rec_(server),
+ version_(version.data(), version.size()) {
+ // Tell log_message_handler about this server_rec and version.
+ log_message_handler::AddServerConfig(server_rec_, version);
+
+ // TODO(jmarantz): consider making this a little terser by default.
+ // The string we expect in is something like "0.9.1.1-171" and we will
+ // may be able to pick off some of the 5 fields that prove to be boring.
+}
+
+int ApacheMessageHandler::GetApacheLogLevel(MessageType type) {
+ switch (type) {
+ case kInfo:
+ // TODO(sligocki): Do we want this to be INFO or NOTICE.
+ return APLOG_INFO;
+ case kWarning:
+ return APLOG_WARNING;
+ case kError:
+ return APLOG_ERR;
+ case kFatal:
+ return APLOG_ALERT;
+ }
+
+ // This should never fall through, but some compilers seem to complain if
+ // we don't include this.
+ return APLOG_ALERT;
+}
+
+void ApacheMessageHandler::MessageVImpl(MessageType type, const char* msg,
+ va_list args) {
+ int log_level = GetApacheLogLevel(type);
+ std::string formatted_message = Format(msg, args);
+ ap_log_error(APLOG_MARK, log_level, APR_SUCCESS, server_rec_,
+ "[%s %s] %s",
+ kModuleName, version_.c_str(), formatted_message.c_str());
+}
+
+void ApacheMessageHandler::FileMessageVImpl(MessageType type, const char* file,
+ int line, const char* msg,
+ va_list args) {
+ int log_level = GetApacheLogLevel(type);
+ std::string formatted_message = Format(msg, args);
+ ap_log_error(APLOG_MARK, log_level, APR_SUCCESS, server_rec_,
+ "[%s %s] %s:%d: %s",
+ kModuleName, version_.c_str(), file, line,
+ formatted_message.c_str());
+}
+
+// TODO(sligocki): It'd be nice not to do so much string copying.
+std::string ApacheMessageHandler::Format(const char* msg, va_list args) {
+ std::string buffer;
+
+ // Ignore the name of this routine: it formats with vsnprintf.
+ // See base/stringprintf.cc.
+ StringAppendV(&buffer, msg, args);
+ return buffer;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/apache_message_handler.h b/trunk/src/net/instaweb/apache/apache_message_handler.h
new file mode 100644
index 0000000..669070d
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apache_message_handler.h
@@ -0,0 +1,54 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_APACHE_APACHE_MESSAGE_HANDLER_H_
+#define NET_INSTAWEB_APACHE_APACHE_MESSAGE_HANDLER_H_
+
+#include <string>
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+struct server_rec;
+
+namespace net_instaweb {
+
+// Implementation of an HTML parser message handler that uses Apache
+// logging to emit messsages.
+class ApacheMessageHandler : public MessageHandler {
+ public:
+ ApacheMessageHandler(const server_rec* server, const StringPiece& version);
+
+ protected:
+ virtual void MessageVImpl(MessageType type, const char* msg, va_list args);
+
+ virtual void FileMessageVImpl(MessageType type, const char* filename,
+ int line, const char* msg, va_list args);
+
+ private:
+ int GetApacheLogLevel(MessageType type);
+ std::string Format(const char* msg, va_list args);
+
+ const server_rec* server_rec_;
+ const std::string version_;
+
+ DISALLOW_COPY_AND_ASSIGN(ApacheMessageHandler);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_APACHE_APACHE_MESSAGE_HANDLER_H_
diff --git a/trunk/src/net/instaweb/apache/apache_rewrite_driver_factory.cc b/trunk/src/net/instaweb/apache/apache_rewrite_driver_factory.cc
new file mode 100644
index 0000000..4f7e898
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apache_rewrite_driver_factory.cc
@@ -0,0 +1,162 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "net/instaweb/apache/apache_rewrite_driver_factory.h"
+
+#include "apr_pools.h"
+#include "net/instaweb/apache/apache_message_handler.h"
+#include "net/instaweb/apache/apr_file_system.h"
+#include "net/instaweb/apache/apr_mutex.h"
+#include "net/instaweb/apache/apr_statistics.h"
+#include "net/instaweb/apache/apr_timer.h"
+#include "net/instaweb/apache/serf_url_async_fetcher.h"
+#include "net/instaweb/apache/serf_url_fetcher.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+#include "net/instaweb/rewriter/public/rewrite_options.h"
+#include "net/instaweb/util/public/file_cache.h"
+#include "net/instaweb/util/public/lru_cache.h"
+#include "net/instaweb/util/public/md5_hasher.h"
+#include "net/instaweb/util/public/threadsafe_cache.h"
+#include "net/instaweb/util/public/write_through_cache.h"
+
+namespace net_instaweb {
+
+ApacheRewriteDriverFactory::ApacheRewriteDriverFactory(
+ server_rec* server, const StringPiece& version)
+ : server_rec_(server),
+ serf_url_fetcher_(NULL),
+ serf_url_async_fetcher_(NULL),
+ statistics_(NULL),
+ lru_cache_kb_per_process_(0),
+ lru_cache_byte_limit_(0),
+ file_cache_clean_interval_ms_(Timer::kHourMs),
+ file_cache_clean_size_kb_(100 * 1024), // 100 megabytes
+ fetcher_time_out_ms_(5 * Timer::kSecondMs),
+ slurp_flush_limit_(0),
+ version_(version.data(), version.size()),
+ statistics_enabled_(true) {
+ apr_pool_create(&pool_, NULL);
+ cache_mutex_.reset(NewMutex());
+ rewrite_drivers_mutex_.reset(NewMutex());
+
+ // In Apache, we default to using the "core filters".
+ options()->SetDefaultRewriteLevel(RewriteOptions::kCoreFilters);
+}
+
+ApacheRewriteDriverFactory::~ApacheRewriteDriverFactory() {
+ // We free all the resources before destroying the pool, because some of the
+ // resource uses the sub-pool and will need that pool to be around to
+ // clean up properly.
+ ShutDown();
+
+ apr_pool_destroy(pool_);
+}
+
+FileSystem* ApacheRewriteDriverFactory::DefaultFileSystem() {
+ return new AprFileSystem(pool_);
+}
+
+Hasher* ApacheRewriteDriverFactory::NewHasher() {
+ return new MD5Hasher();
+}
+
+Timer* ApacheRewriteDriverFactory::DefaultTimer() {
+ return new AprTimer();
+}
+
+MessageHandler* ApacheRewriteDriverFactory::DefaultHtmlParseMessageHandler() {
+ return new ApacheMessageHandler(server_rec_, version_);
+}
+
+MessageHandler* ApacheRewriteDriverFactory::DefaultMessageHandler() {
+ return new ApacheMessageHandler(server_rec_, version_);
+}
+
+CacheInterface* ApacheRewriteDriverFactory::DefaultCacheInterface() {
+ FileCache::CachePolicy* policy = new FileCache::CachePolicy(
+ timer(), file_cache_clean_interval_ms_, file_cache_clean_size_kb_);
+ CacheInterface* cache = new FileCache(
+ file_cache_path_, file_system(), filename_encoder(), policy,
+ message_handler());
+ if (lru_cache_kb_per_process_ != 0) {
+ LRUCache* lru_cache = new LRUCache(lru_cache_kb_per_process_ * 1024);
+
+ // We only add the threadsafe-wrapper to the LRUCache. The FileCache
+ // is naturally thread-safe because it's got no writable member variables.
+ // And surrounding that slower-running class with a mutex would likely
+ // cause contention.
+ ThreadsafeCache* ts_cache = new ThreadsafeCache(lru_cache, cache_mutex());
+ WriteThroughCache* write_through_cache =
+ new WriteThroughCache(ts_cache, cache);
+ // By default, WriteThroughCache does not limit the size of entries going
+ // into its front cache.
+ if (lru_cache_byte_limit_ != 0) {
+ write_through_cache->set_cache1_limit(lru_cache_byte_limit_);
+ }
+ cache = write_through_cache;
+ }
+ return cache;
+}
+
+UrlFetcher* ApacheRewriteDriverFactory::DefaultUrlFetcher() {
+ if (serf_url_fetcher_ == NULL) {
+ DefaultAsyncUrlFetcher(); // Create async fetcher if necessary.
+ serf_url_fetcher_ = new SerfUrlFetcher(
+ fetcher_time_out_ms_, serf_url_async_fetcher_);
+ }
+ return serf_url_fetcher_;
+}
+
+UrlAsyncFetcher* ApacheRewriteDriverFactory::DefaultAsyncUrlFetcher() {
+ if (serf_url_async_fetcher_ == NULL) {
+ serf_url_async_fetcher_ = new SerfUrlAsyncFetcher(
+ fetcher_proxy_.c_str(), pool_, statistics_, timer(),
+ fetcher_time_out_ms_);
+ }
+ return serf_url_async_fetcher_;
+}
+
+
+HtmlParse* ApacheRewriteDriverFactory::DefaultHtmlParse() {
+ return new HtmlParse(html_parse_message_handler());
+}
+
+AbstractMutex* ApacheRewriteDriverFactory::NewMutex() {
+ return new AprMutex(pool_);
+}
+
+ResourceManager* ApacheRewriteDriverFactory::ComputeResourceManager() {
+ ResourceManager* resource_manager =
+ RewriteDriverFactory::ComputeResourceManager();
+ resource_manager->set_statistics(statistics_);
+ http_cache()->SetStatistics(statistics_);
+ return resource_manager;
+}
+
+void ApacheRewriteDriverFactory::ShutDown() {
+ if (serf_url_async_fetcher_ != NULL) {
+ serf_url_async_fetcher_->WaitForInProgressFetches(
+ fetcher_time_out_ms_, message_handler(),
+ SerfUrlAsyncFetcher::kThreadedAndMainline);
+ }
+ cache_mutex_.reset(NULL);
+ rewrite_drivers_mutex_.reset(NULL);
+ RewriteDriverFactory::ShutDown();
+}
+
+void ApacheRewriteDriverFactory::Terminate() {
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/apache_rewrite_driver_factory.h b/trunk/src/net/instaweb/apache/apache_rewrite_driver_factory.h
new file mode 100644
index 0000000..fbf8512
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apache_rewrite_driver_factory.h
@@ -0,0 +1,132 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#ifndef NET_INSTAWEB_APACHE_APACHE_REWRITE_DRIVER_FACTORY_H_
+#define NET_INSTAWEB_APACHE_APACHE_REWRITE_DRIVER_FACTORY_H_
+
+#include <stdio.h>
+#include <set>
+#include <string>
+#include <vector>
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/rewrite_driver_factory.h"
+
+struct apr_pool_t;
+struct server_rec;
+
+namespace net_instaweb {
+
+class AprStatistics;
+class SerfUrlAsyncFetcher;
+class SerfUrlFetcher;
+
+// Creates an Apache RewriteDriver.
+class ApacheRewriteDriverFactory : public RewriteDriverFactory {
+ public:
+ explicit ApacheRewriteDriverFactory(server_rec* server,
+ const StringPiece& version);
+ virtual ~ApacheRewriteDriverFactory();
+
+ virtual Hasher* NewHasher();
+ virtual AbstractMutex* NewMutex();
+
+ SerfUrlAsyncFetcher* serf_url_async_fetcher() {
+ return serf_url_async_fetcher_;
+ }
+
+ void set_lru_cache_kb_per_process(int64 x) { lru_cache_kb_per_process_ = x; }
+ void set_lru_cache_byte_limit(int64 x) { lru_cache_byte_limit_ = x; }
+ void set_slurp_flush_limit(int64 x) { slurp_flush_limit_ = x; }
+ int slurp_flush_limit() const { return slurp_flush_limit_; }
+ void set_file_cache_clean_interval_ms(int64 x) {
+ file_cache_clean_interval_ms_ = x;
+ }
+ void set_file_cache_clean_size_kb(int64 x) { file_cache_clean_size_kb_ = x; }
+ void set_fetcher_time_out_ms(int64 x) { fetcher_time_out_ms_ = x; }
+ void set_file_cache_path(const StringPiece& x) {
+ x.CopyToString(&file_cache_path_);
+ }
+ void set_fetcher_proxy(const StringPiece& x) {
+ x.CopyToString(&fetcher_proxy_);
+ }
+
+ StringPiece file_cache_path() { return file_cache_path_; }
+ int64 file_cache_clean_size_kb() { return file_cache_clean_size_kb_; }
+ int64 fetcher_time_out_ms() { return fetcher_time_out_ms_; }
+ AprStatistics* statistics() { return statistics_; }
+ void set_statistics(AprStatistics* x) { statistics_ = x; }
+ void set_statistics_enabled(bool x) { statistics_enabled_ = x; }
+ bool statistics_enabled() const { return statistics_enabled_; }
+
+ // Relinquish all static data
+ static void Terminate();
+
+ protected:
+ virtual UrlFetcher* DefaultUrlFetcher();
+ virtual UrlAsyncFetcher* DefaultAsyncUrlFetcher();
+
+ // Provide defaults.
+ virtual MessageHandler* DefaultHtmlParseMessageHandler();
+ virtual MessageHandler* DefaultMessageHandler();
+ virtual FileSystem* DefaultFileSystem();
+ virtual HtmlParse* DefaultHtmlParse();
+ virtual Timer* DefaultTimer();
+ virtual CacheInterface* DefaultCacheInterface();
+ virtual AbstractMutex* cache_mutex() { return cache_mutex_.get(); }
+ virtual AbstractMutex* rewrite_drivers_mutex() {
+ return rewrite_drivers_mutex_.get(); }
+
+ // Disable the Resource Manager's filesystem since we have a
+ // write-through http_cache.
+ virtual bool ShouldWriteResourcesToFileSystem() { return false; }
+
+ // When computing the resource manager for Apache, be sure to set up
+ // the statistics.
+ virtual ResourceManager* ComputeResourceManager();
+
+ // Release all the resources. It also calls the base class ShutDown to release
+ // the base class resources.
+ void ShutDown();
+
+ private:
+ apr_pool_t* pool_;
+ server_rec* server_rec_;
+ scoped_ptr<AbstractMutex> cache_mutex_;
+ scoped_ptr<AbstractMutex> rewrite_drivers_mutex_;
+ SerfUrlFetcher* serf_url_fetcher_;
+ SerfUrlAsyncFetcher* serf_url_async_fetcher_;
+ AprStatistics* statistics_;
+
+ // TODO(jmarantz): These options could be consolidated in a protobuf or
+ // some other struct, which would keep them distinct from the rest of the
+ // state. Note also that some of the options are in the base class,
+ // RewriteDriverFactory, so we'd have to sort out how that worked.
+ int64 lru_cache_kb_per_process_;
+ int64 lru_cache_byte_limit_;
+ int64 file_cache_clean_interval_ms_;
+ int64 file_cache_clean_size_kb_;
+ int64 fetcher_time_out_ms_;
+ int64 slurp_flush_limit_;
+ std::string file_cache_path_;
+ std::string fetcher_proxy_;
+ std::string version_;
+ bool statistics_enabled_;
+
+ DISALLOW_COPY_AND_ASSIGN(ApacheRewriteDriverFactory);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_APACHE_APACHE_REWRITE_DRIVER_FACTORY_H_
diff --git a/trunk/src/net/instaweb/apache/apache_slurp.cc b/trunk/src/net/instaweb/apache/apache_slurp.cc
new file mode 100644
index 0000000..92c3b62
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apache_slurp.cc
@@ -0,0 +1,268 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Must precede any Apache includes, for now, due a conflict in
+// the use of 'OK" as an Instaweb enum and as an Apache #define.
+#include "base/string_util.h"
+#include "net/instaweb/apache/header_util.h"
+
+// TODO(jmarantz): serf_url_async_fetcher evidently sets
+// 'gzip' unconditionally, and the response includes the 'gzip'
+// encoding header, but serf unzips the response itself.
+//
+// I think the correct behavior is that our async fetcher should
+// transmit the 'gzip' request header if it was specified in the call
+// to StreamingFetch. This would be easy to fix.
+//
+// Unfortunately, serf 0.31 appears to unzip the content for us, which
+// is not what we are asking for. And it leaves the 'gzip' response
+// header in despite having unzipped it itself. I have tried later
+// versions of serf, but the API is not compatible (easy to deal with)
+// but the resulting binary has unresolved symbols. I am wondering
+// whether we will have to go to libcurl.
+//
+// For now use wget when slurping additional files.
+
+#include "net/instaweb/apache/apache_rewrite_driver_factory.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/query_params.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include <string>
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/url_fetcher.h"
+
+// The Apache headers must be after instaweb headers. Otherwise, the
+// compiler will complain
+// "strtoul_is_not_a_portable_function_use_strtol_instead".
+#include "apr_strings.h" // for apr_pstrdup
+#include "http_protocol.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// Default handler when the file is not found
+void SlurpDefaultHandler(request_rec* r) {
+ ap_set_content_type(r, "text/html; charset=utf-8");
+ std::string buf = StringPrintf(
+ "<html><head><title>Slurp Error</title></head>"
+ "<body><h1>Slurp failed</h1>\n"
+ "<p>host=%s\n"
+ "<p>uri=%s\n"
+ "</body></html>",
+ r->hostname, r->unparsed_uri);
+ ap_rputs(buf.c_str(), r);
+ r->status = HttpStatus::kNotFound;
+ r->status_line = "Not Found";
+}
+
+// TODO(jmarantz): The ApacheWriter defined below is much more
+// efficient than the mechanism we are currently using, which is to
+// buffer the entire response in a string and then send it later.
+// For some reason, this did not work when I tried it, but it's
+// worth another look.
+
+class ApacheWriter : public Writer {
+ public:
+ ApacheWriter(request_rec* r, SimpleMetaData* response_headers,
+ int64 flush_limit)
+ : request_(r),
+ response_headers_(response_headers),
+ size_(0),
+ flush_limit_(flush_limit),
+ headers_out_(false) {
+ }
+
+ virtual bool Write(const StringPiece& str, MessageHandler* handler) {
+ if (!headers_out_) {
+ OutputHeaders();
+ }
+ ap_rwrite(str.data(), str.size(), request_);
+ size_ += str.size();
+ if ((flush_limit_ != 0) && (size_ > flush_limit_)) {
+ Flush(handler);
+ }
+ return true;
+ }
+
+ virtual bool Flush(MessageHandler* handler) {
+ ap_rflush(request_);
+ size_ = 0;
+ return true;
+ }
+
+ int64 size() const { return size_; }
+
+ void OutputHeaders() {
+ if (headers_out_) {
+ return;
+ }
+ headers_out_ = true;
+
+ // Apache2 defaults to set the status line as HTTP/1.1. If the
+ // original content was HTTP/1.0, we need to force the server to use
+ // HTTP/1.0. I'm not sure why/whether we need to do this; it was in
+ // mod_static from the sdpy project, which is where I copied this
+ // code from.
+ if ((response_headers_->major_version() == 1) &&
+ (response_headers_->minor_version() == 0)) {
+ apr_table_set(request_->subprocess_env, "force-response-1.0", "1");
+ }
+
+ char* content_type = NULL;
+ CharStarVector v;
+ CHECK(response_headers_->headers_complete());
+ if (response_headers_->Lookup(HttpAttributes::kContentType, &v)) {
+ CHECK(!v.empty());
+ // ap_set_content_type does not make a copy of the string, we need
+ // to duplicate it. Note that we will update the content type below,
+ // after transforming the headers.
+ content_type = apr_pstrdup(request_->pool, v[v.size() - 1]);
+ }
+ response_headers_->RemoveAll(HttpAttributes::kTransferEncoding);
+ response_headers_->RemoveAll(HttpAttributes::kContentLength);
+ MetaDataToApacheHeader(*response_headers_, request_->headers_out,
+ &request_->status, &request_->proto_num);
+ if (content_type != NULL) {
+ ap_set_content_type(request_, content_type);
+ }
+
+ // Recompute the content-length, because the content is decoded.
+ // TODO(lsong): We don't know the content size, do we?
+ // ap_set_content_length(request_, contents.size());
+ }
+
+ private:
+ request_rec* request_;
+ SimpleMetaData* response_headers_;
+ int size_;
+ int flush_limit_;
+ bool headers_out_;
+
+ DISALLOW_COPY_AND_ASSIGN(ApacheWriter);
+};
+
+} // namespace
+
+// Remove any mod-pagespeed-specific modifiers before we go to our slurped
+// fetcher.
+//
+// TODO(jmarantz): share the string constants from mod_instaweb.cc and
+// formalize the prefix-matching assumed here.
+std::string RemoveModPageSpeedQueryParams(
+ const std::string& uri, const char* query_param_string) {
+ QueryParams query_params, stripped_query_params;
+ query_params.Parse(query_param_string);
+ bool rewrite_query_params = false;
+
+ for (int i = 0; i < query_params.size(); ++i) {
+ const char* name = query_params.name(i);
+ static const char kModPagespeed[] = "ModPagespeed";
+ if (strncmp(name, kModPagespeed, sizeof(kModPagespeed) - 1) == 0) {
+ rewrite_query_params = true;
+ } else {
+ stripped_query_params.Add(name, query_params.value(i));
+ }
+ }
+
+ std::string stripped_url;
+ if (rewrite_query_params) {
+ // TODO(jmarantz): It would be nice to use GoogleUrl to do this but
+ // it's not clear how it would help. Instead just hack the string.
+ std::string::size_type question_mark = uri.find('?');
+ CHECK(question_mark != std::string::npos);
+ stripped_url.append(uri.data(), question_mark); // does not include "?" yet
+ if (stripped_query_params.size() != 0) {
+ stripped_url += StrCat("?", stripped_query_params.ToString());
+ }
+ } else {
+ stripped_url = uri;
+ }
+ return stripped_url;
+}
+
+// Some of the sites we are trying to slurp have mod-pagespeed enabled already.
+// We actually want to start with the non-pagespeed-enabled site. But we'd
+// rather not send ModPagespeed=off to servers that are not expecting it.
+class ModPagespeedStrippingFetcher : public UrlFetcher {
+ public:
+ ModPagespeedStrippingFetcher(UrlFetcher* fetcher) : fetcher_(fetcher) {}
+ virtual bool StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* fetched_content_writer,
+ MessageHandler* message_handler) {
+ std::string contents;
+ StringWriter writer(&contents);
+ bool fetched = fetcher_->StreamingFetchUrl(
+ url, request_headers, response_headers, &writer, message_handler);
+ if (fetched) {
+ CharStarVector v;
+ if (response_headers->Lookup("X-Mod-Pagespeed", &v)) {
+ response_headers->Clear();
+ std::string::size_type question = url.find('?');
+ std::string url_pagespeed_off(url);
+ if (question == std::string::npos) {
+ url_pagespeed_off += "?ModPagespeed=off";
+ } else {
+ url_pagespeed_off += "&ModPagespeed=off";
+ }
+ fetched = fetcher_->StreamingFetchUrl(
+ url_pagespeed_off, request_headers, response_headers,
+ fetched_content_writer, message_handler);
+ } else {
+ // It was not mod-pagespeed in the first place; just pass it through
+ // so it can be saved in the slurp directory.
+ fetched = fetched_content_writer->Write(contents, message_handler);
+ }
+ }
+ return fetched;
+ }
+ private:
+ UrlFetcher* fetcher_;
+};
+
+void SlurpUrl(const std::string& uri, ApacheRewriteDriverFactory* factory,
+ request_rec* r) {
+ SimpleMetaData request_headers, response_headers;
+ ApacheHeaderToMetaData(r->headers_in, 0, 0, &request_headers);
+ std::string contents;
+ ApacheWriter writer(r, &response_headers, factory->slurp_flush_limit());
+
+ std::string stripped_url = RemoveModPageSpeedQueryParams(
+ uri, r->parsed_uri.query);
+
+ UrlFetcher* fetcher = factory->ComputeUrlFetcher();
+ ModPagespeedStrippingFetcher stripping_fetcher(fetcher);
+
+ if (stripping_fetcher.StreamingFetchUrl(stripped_url, request_headers,
+ &response_headers, &writer,
+ factory->message_handler())) {
+ // In the event of empty content, the writer's Write method may not be
+ // called, but we should still emit headers.
+ writer.OutputHeaders();
+ } else {
+ MessageHandler* handler = factory->message_handler();
+ handler->Message(kInfo, "mod_pagespeed: slurp of url %s failed.\n"
+ "Request Headers: %s\n\nResponse Headers: %s",
+ stripped_url.c_str(),
+ request_headers.ToString().c_str(),
+ response_headers.ToString().c_str());
+ SlurpDefaultHandler(r);
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/apache_slurp.h b/trunk/src/net/instaweb/apache/apache_slurp.h
new file mode 100644
index 0000000..c0ffab6
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apache_slurp.h
@@ -0,0 +1,35 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_APACHE_APACHE_SLURP_H_
+#define NET_INSTAWEB_APACHE_APACHE_SLURP_H_
+
+#include <string>
+
+struct request_rec;
+
+namespace net_instaweb {
+
+class ApacheRewriteDriverFactory;
+
+// Loads the URL based on the fetchers and other infrastructure in the
+// factory.
+void SlurpUrl(const std::string& uri, ApacheRewriteDriverFactory* factory,
+ request_rec* r);
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_APACHE_APACHE_SLURP_H_
diff --git a/trunk/src/net/instaweb/apache/apr_file_system.cc b/trunk/src/net/instaweb/apache/apr_file_system.cc
new file mode 100644
index 0000000..35849e0
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apr_file_system.cc
@@ -0,0 +1,423 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "net/instaweb/apache/apr_file_system.h"
+
+#include <string>
+
+#include "apr_file_info.h"
+#include "apr_file_io.h"
+#include "apr_pools.h"
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/timer.h"
+#include "net/instaweb/util/stack_buffer.h"
+
+namespace net_instaweb {
+
+void AprReportError(MessageHandler* message_handler, const char* filename,
+ int line, const char* message, int error_code) {
+ char buf[kStackBufferSize];
+ apr_strerror(error_code, buf, sizeof(buf));
+ std::string error_format(message);
+ error_format.append(" (code=%d %s)");
+ message_handler->Error(filename, line, error_format.c_str(), error_code, buf);
+}
+
+// Helper class to factor out common implementation details between Input and
+// Output files, in lieu of multiple inheritance.
+class FileHelper {
+ public:
+ FileHelper(apr_file_t* file, const char* filename)
+ : file_(file),
+ filename_(filename) {
+ }
+
+ void ReportError(MessageHandler* message_handler, const char* format,
+ int error_code) {
+ AprReportError(message_handler, filename_.c_str(), 0, format, error_code);
+ }
+
+ bool Close(MessageHandler* message_handler);
+ apr_file_t* file() { return file_; }
+ const std::string& filename() const { return filename_; }
+
+ private:
+ apr_file_t* const file_;
+ const std::string filename_;
+
+ DISALLOW_COPY_AND_ASSIGN(FileHelper);
+};
+
+bool FileHelper::Close(MessageHandler* message_handler) {
+ apr_status_t ret = apr_file_close(file_);
+ if (ret != APR_SUCCESS) {
+ ReportError(message_handler, "close file", ret);
+ return false;
+ } else {
+ return true;
+ }
+}
+
+class HtmlWriterInputFile : public FileSystem::InputFile {
+ public:
+ HtmlWriterInputFile(apr_file_t* file, const char* filename);
+ virtual int Read(char* buf, int size, MessageHandler* message_handler);
+ virtual bool Close(MessageHandler* message_handler) {
+ return helper_.Close(message_handler);
+ }
+ virtual const char* filename() { return helper_.filename().c_str(); }
+ private:
+ FileHelper helper_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlWriterInputFile);
+};
+
+class HtmlWriterOutputFile : public FileSystem::OutputFile {
+ public:
+ HtmlWriterOutputFile(apr_file_t* file, const char* filename);
+ virtual bool Write(const StringPiece& buf,
+ MessageHandler* message_handler);
+ virtual bool Flush(MessageHandler* message_handler);
+ virtual bool Close(MessageHandler* message_handler) {
+ return helper_.Close(message_handler);
+ }
+ virtual bool SetWorldReadable(MessageHandler* message_handler);
+ virtual const char* filename() { return helper_.filename().c_str(); }
+ private:
+ FileHelper helper_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlWriterOutputFile);
+};
+
+HtmlWriterInputFile::HtmlWriterInputFile(apr_file_t* file, const char* filename)
+ : helper_(file, filename) {
+}
+
+int HtmlWriterInputFile::Read(char* buf,
+ int size,
+ MessageHandler* message_handler) {
+ apr_size_t bytes = size;
+ apr_status_t ret = apr_file_read(helper_.file(), buf, &bytes);
+ if (ret == APR_EOF) {
+ return 0;
+ }
+ if (ret != APR_SUCCESS) {
+ bytes = 0;
+ helper_.ReportError(message_handler, "read file", ret);
+ }
+ return bytes;
+}
+
+HtmlWriterOutputFile::HtmlWriterOutputFile(apr_file_t* file,
+ const char* filename)
+ : helper_(file, filename) {
+}
+
+bool HtmlWriterOutputFile::Write(const StringPiece& buf,
+ MessageHandler* message_handler) {
+ bool success = false;
+ apr_size_t bytes = buf.size();
+ apr_status_t ret = apr_file_write(helper_.file(), buf.data(), &bytes);
+ if (ret != APR_SUCCESS) {
+ helper_.ReportError(message_handler, "write file", ret);
+ } else if (bytes != buf.size()) {
+ helper_.ReportError(message_handler, "write file partial", ret);
+ } else {
+ success = true;
+ }
+ return success;
+}
+
+bool HtmlWriterOutputFile::Flush(MessageHandler* message_handler) {
+ apr_status_t ret = apr_file_flush(helper_.file());
+ if (ret != APR_SUCCESS) {
+ helper_.ReportError(message_handler, "flush file", ret);
+ return false;
+ }
+ return true;
+}
+
+bool HtmlWriterOutputFile::SetWorldReadable(MessageHandler* message_handler) {
+ apr_status_t ret = apr_file_perms_set(helper_.filename().c_str(),
+ APR_FPROT_UREAD | APR_FPROT_UWRITE |
+ APR_FPROT_GREAD | APR_FPROT_WREAD);
+ if (ret != APR_SUCCESS) {
+ helper_.ReportError(message_handler, "set permission", ret);
+ return false;
+ }
+ return true;
+}
+
+AprFileSystem::AprFileSystem(apr_pool_t* pool)
+ : pool_(NULL) {
+ apr_pool_create(&pool_, pool);
+}
+
+AprFileSystem::~AprFileSystem() {
+ apr_pool_destroy(pool_);
+}
+
+FileSystem::InputFile* AprFileSystem::OpenInputFile(
+ const char* filename, MessageHandler* message_handler) {
+ apr_file_t* file;
+ apr_status_t ret = apr_file_open(&file, filename, APR_FOPEN_READ,
+ APR_OS_DEFAULT, pool_);
+ if (ret != APR_SUCCESS) {
+ AprReportError(message_handler, filename, 0, "open input file", ret);
+ return NULL;
+ }
+ return new HtmlWriterInputFile(file, filename);
+}
+
+FileSystem::OutputFile* AprFileSystem::OpenOutputFileHelper(
+ const char* filename, MessageHandler* message_handler) {
+ apr_file_t* file;
+ apr_status_t ret = apr_file_open(&file, filename,
+ APR_WRITE | APR_CREATE | APR_TRUNCATE,
+ APR_OS_DEFAULT, pool_);
+ if (ret != APR_SUCCESS) {
+ AprReportError(message_handler, filename, 0, "open output file", ret);
+ return NULL;
+ }
+ return new HtmlWriterOutputFile(file, filename);
+}
+
+FileSystem::OutputFile* AprFileSystem::OpenTempFileHelper(
+ const StringPiece& prefix_name,
+ MessageHandler* message_handler) {
+ static const char mkstemp_hook[] = "XXXXXX";
+ scoped_array<char> template_name(
+ new char[prefix_name.size() + sizeof(mkstemp_hook)]);
+ memcpy(template_name.get(), prefix_name.data(), prefix_name.size());
+ memcpy(template_name.get() + prefix_name.size(), mkstemp_hook,
+ sizeof(mkstemp_hook));
+
+ apr_file_t* file;
+ // A temp file will be generated with the XXXXXX part of template_name being
+ // replaced.
+ // Do not use flag APR_DELONCLOSE to delete the temp file on close, it will
+ // be renamed for later use.
+ apr_status_t ret = apr_file_mktemp(
+ &file, template_name.get(),
+ APR_CREATE | APR_READ | APR_WRITE | APR_EXCL, pool_);
+ if (ret != APR_SUCCESS) {
+ AprReportError(message_handler, template_name.get(), 0,
+ "open temp file", ret);
+ return NULL;
+ }
+ return new HtmlWriterOutputFile(file, template_name.get());
+}
+
+bool AprFileSystem::RenameFileHelper(
+ const char* old_filename, const char* new_filename,
+ MessageHandler* message_handler) {
+ apr_status_t ret = apr_file_rename(old_filename, new_filename, pool_);
+ if (ret != APR_SUCCESS) {
+ AprReportError(message_handler, new_filename, 0, "renaming temp file", ret);
+ return false;
+ }
+ return true;
+}
+bool AprFileSystem::RemoveFile(const char* filename,
+ MessageHandler* message_handler) {
+ apr_status_t ret = apr_file_remove(filename, pool_);
+ if (ret != APR_SUCCESS) {
+ AprReportError(message_handler, filename, 0, "removing file", ret);
+ return false;
+ }
+ return true;
+}
+
+bool AprFileSystem::MakeDir(const char* directory_path,
+ MessageHandler* handler) {
+ apr_status_t ret = apr_dir_make(directory_path, APR_FPROT_OS_DEFAULT, pool_);
+ if (ret != APR_SUCCESS) {
+ AprReportError(handler, directory_path, 0, "creating dir", ret);
+ return false;
+ }
+ return true;
+}
+
+BoolOrError AprFileSystem::Exists(const char* path, MessageHandler* handler) {
+ BoolOrError exists; // Error is the default state.
+ apr_int32_t wanted = APR_FINFO_TYPE;
+ apr_finfo_t finfo;
+ apr_status_t ret = apr_stat(&finfo, path, wanted, pool_);
+ if (ret != APR_SUCCESS && ret != APR_ENOENT) {
+ AprReportError(handler, path, 0, "failed to stat", ret);
+ exists.set_error();
+ } else {
+ exists.set(ret == APR_SUCCESS);
+ }
+ return exists;
+}
+
+BoolOrError AprFileSystem::IsDir(const char* path, MessageHandler* handler) {
+ BoolOrError is_dir; // Error is the default state.
+ apr_int32_t wanted = APR_FINFO_TYPE;
+ apr_finfo_t finfo;
+ apr_status_t ret = apr_stat(&finfo, path, wanted, pool_);
+ if (ret != APR_SUCCESS && ret != APR_ENOENT) {
+ AprReportError(handler, path, 0, "failed to stat", ret);
+ is_dir.set_error();
+ } else {
+ is_dir.set(ret == APR_SUCCESS && finfo.filetype == APR_DIR);
+ }
+ return is_dir;
+}
+
+bool AprFileSystem::ListContents(const StringPiece& dir,
+ StringVector* files,
+ MessageHandler* handler) {
+ std::string dirString = dir.as_string();
+ EnsureEndsInSlash(&dirString);
+ const char* dirname = dirString.c_str();
+ apr_dir_t* mydir;
+ apr_status_t ret = apr_dir_open(&mydir, dirname, pool_);
+ if (ret != APR_SUCCESS) {
+ AprReportError(handler, dirname, 0, "failed to opendir", ret);
+ return false;
+ } else {
+ apr_finfo_t finfo;
+ apr_int32_t wanted = APR_FINFO_NAME;
+ while (apr_dir_read(&finfo, wanted, mydir) != APR_ENOENT) {
+ if ((strcmp(finfo.name, ".") != 0) &&
+ (strcmp(finfo.name, "..") != 0)) {
+ files->push_back(dirString + finfo.name);
+ }
+ }
+ }
+ ret = apr_dir_close(mydir);
+ if (ret != APR_SUCCESS) {
+ AprReportError(handler, dirname, 0, "failed to closedir", ret);
+ return false;
+ }
+ return true;
+}
+
+bool AprFileSystem::Atime(const StringPiece& path,
+ int64* timestamp_sec, MessageHandler* handler) {
+ // TODO(abliss): there are some situations where this doesn't work
+ // -- e.g. if the filesystem is mounted noatime.
+ const std::string path_string = path.as_string();
+ const char* path_str = path_string.c_str();
+ apr_int32_t wanted = APR_FINFO_ATIME;
+ apr_finfo_t finfo;
+ apr_status_t ret = apr_stat(&finfo, path_str, wanted, pool_);
+ if (ret != APR_SUCCESS) {
+ AprReportError(handler, path_str, 0, "failed to stat", ret);
+ return false;
+ } else {
+ *timestamp_sec = finfo.atime;
+ return true;
+ }
+}
+
+bool AprFileSystem::Ctime(const StringPiece& path,
+ int64* timestamp_sec, MessageHandler* handler) {
+ const std::string path_string = path.as_string();
+ const char* path_str = path_string.c_str();
+ apr_int32_t wanted = APR_FINFO_CTIME;
+ apr_finfo_t finfo;
+ apr_status_t ret = apr_stat(&finfo, path_str, wanted, pool_);
+ if (ret != APR_SUCCESS) {
+ AprReportError(handler, path_str, 0, "failed to stat", ret);
+ return false;
+ } else {
+ *timestamp_sec = finfo.ctime;
+ return true;
+ }
+}
+
+bool AprFileSystem::Size(const StringPiece& path, int64* size,
+ MessageHandler* handler) {
+ const std::string path_string = path.as_string();
+ const char* path_str = path_string.c_str();
+ apr_int32_t wanted = APR_FINFO_SIZE;
+ apr_finfo_t finfo;
+ apr_status_t ret = apr_stat(&finfo, path_str, wanted, pool_);
+ if (ret != APR_SUCCESS) {
+ AprReportError(handler, path_str, 0, "failed to stat", ret);
+ return false;
+ } else {
+ *size = finfo.size;
+ return true;
+ }
+}
+
+BoolOrError AprFileSystem::TryLock(const StringPiece& lock_name,
+ MessageHandler* handler) {
+ const std::string lock_string = lock_name.as_string();
+ const char* lock_str = lock_string.c_str();
+ // TODO(abliss): mkdir is not atomic on all platforms. We should
+ // perhaps use an apr_global_mutex_t here.
+ apr_status_t ret = apr_dir_make(lock_str, APR_FPROT_OS_DEFAULT, pool_);
+ if (ret == APR_SUCCESS) {
+ return BoolOrError(true);
+ } else if (errno == EEXIST) {
+ return BoolOrError(false);
+ } else {
+ AprReportError(handler, lock_str, 0, "creating dir", ret);
+ return BoolOrError();
+ }
+}
+
+BoolOrError AprFileSystem::TryLockWithTimeout(const StringPiece& lock_name,
+ int64 timeout_ms,
+ MessageHandler* handler) {
+ const std::string lock_string = lock_name.as_string();
+ const char* lock_str = lock_string.c_str();
+ BoolOrError result = TryLock(lock_name, handler);
+ if (result.is_true() || result.is_error()) {
+ // We got the lock, or the lock is ungettable.
+ return result;
+ }
+ int64 c_time_us;
+ if (!Ctime(lock_name, &c_time_us, handler)) {
+ // We can't stat the lockfile.
+ return BoolOrError();
+ }
+
+ if (apr_time_now() - c_time_us < timeout_ms * 1000) {
+ // The lock is insufficiently stale.
+ return BoolOrError(false);
+ }
+ if (!Unlock(lock_name, handler)) {
+ // We couldn't break the lock. Maybe someone else beat us to it.
+ return BoolOrError();
+ }
+ handler->Info(lock_str, 0, "Breaking lock! now-ctime=%d-%d > %d (sec)",
+ static_cast<int>(apr_time_now() / Timer::kSecondUs),
+ static_cast<int>(c_time_us / Timer::kSecondUs),
+ static_cast<int>(timeout_ms / Timer::kSecondMs));
+
+ return TryLock(lock_name, handler);
+}
+
+bool AprFileSystem::Unlock(const StringPiece& lock_name,
+ MessageHandler* handler) {
+ const std::string lock_string = lock_name.as_string();
+ const char* lock_str = lock_string.c_str();
+ apr_status_t ret = apr_dir_remove(lock_str, pool_);
+ if (ret != APR_SUCCESS) {
+ AprReportError(handler, lock_str, 0, "removing dir", ret);
+ return false;
+ }
+ return true;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/apr_file_system.h b/trunk/src/net/instaweb/apache/apr_file_system.h
new file mode 100644
index 0000000..de49745
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apr_file_system.h
@@ -0,0 +1,79 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#ifndef NET_INSTAWEB_APACHE_APR_FILE_SYSTEM_H_
+#define NET_INSTAWEB_APACHE_APR_FILE_SYSTEM_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/message_handler.h"
+
+struct apr_pool_t;
+
+namespace net_instaweb {
+
+void AprReportError(MessageHandler* message_handler, const char* filename,
+ int line, const char* message, int error_code);
+
+class AprFileSystem : public FileSystem {
+ public:
+ explicit AprFileSystem(apr_pool_t* pool);
+ ~AprFileSystem();
+ virtual bool Atime(const StringPiece& path,
+ int64* timestamp_sec, MessageHandler* handler);
+ bool Ctime(const StringPiece& path,
+ int64* timestamp_sec, MessageHandler* handler);
+ virtual InputFile* OpenInputFile(
+ const char* file, MessageHandler* message_handler);
+ virtual OutputFile* OpenOutputFileHelper(
+ const char* file, MessageHandler* message_handler);
+ // See FileSystem interface for specifics of OpenTempFile.
+ virtual OutputFile* OpenTempFileHelper(
+ const StringPiece& prefix_name,
+ MessageHandler* message_handler);
+ virtual bool RenameFileHelper(const char* old_filename,
+ const char* new_filename,
+ MessageHandler* message_handler);
+ virtual bool RemoveFile(const char* filename,
+ MessageHandler* message_handler);
+ virtual bool Size(const StringPiece& path, int64* size,
+ MessageHandler* handler);
+ // Like POSIX 'mkdir', makes a directory only if parent directory exists.
+ // Fails if directory_name already exists or parent directory doesn't exist.
+ virtual bool MakeDir(const char* directory_path, MessageHandler* handler);
+
+ // Like POSIX 'test -e', checks if path exists (is a file, directory, etc.).
+ virtual BoolOrError Exists(const char* path, MessageHandler* handler);
+
+ // Like POSIX 'test -d', checks if path exists and refers to a directory.
+ virtual BoolOrError IsDir(const char* path, MessageHandler* handler);
+
+ virtual bool ListContents(const StringPiece& dir, StringVector* files,
+ MessageHandler* handler);
+ virtual BoolOrError TryLock(const StringPiece& lock_name,
+ MessageHandler* handler);
+ virtual BoolOrError TryLockWithTimeout(const StringPiece& lock_name,
+ int64 timeout_ms,
+ MessageHandler* handler);
+ virtual bool Unlock(const StringPiece& lock_name, MessageHandler* handler);
+
+ private:
+ apr_pool_t* pool_;
+
+ DISALLOW_COPY_AND_ASSIGN(AprFileSystem);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_APACHE_APR_FILE_SYSTEM_H_
diff --git a/trunk/src/net/instaweb/apache/apr_file_system_test.cc b/trunk/src/net/instaweb/apache/apr_file_system_test.cc
new file mode 100644
index 0000000..0ec4870
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apr_file_system_test.cc
@@ -0,0 +1,177 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include <string>
+
+#include "apr_file_io.h"
+#include "apr_pools.h"
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/apache/apr_file_system.h"
+#include "net/instaweb/util/public/file_system_test.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "testing/gtest/include/gtest/gtest.h"
+
+namespace net_instaweb {
+
+class AprFileSystemTest : public FileSystemTest {
+ protected:
+ AprFileSystemTest() { }
+
+ virtual void DeleteRecursively(const StringPiece& filename) {
+ MyDeleteFileRecursively(filename.as_string(), NULL, NULL);
+ }
+ virtual FileSystem* file_system() {
+ return file_system_.get();
+ }
+ virtual std::string test_tmpdir() {
+ return test_tmpdir_;
+ }
+ virtual void SetUp() {
+ apr_initialize();
+ atexit(apr_terminate);
+ apr_pool_create(&pool_, NULL);
+ file_system_.reset(new AprFileSystem(pool_));
+ GetAprFileSystemTestDir(&test_tmpdir_);
+ }
+
+ virtual void TearDown() {
+ file_system_.reset();
+ apr_pool_destroy(pool_);
+ }
+
+ void GetAprFileSystemTestDir(std::string* str) {
+ const char* tmpdir;
+ apr_status_t status = apr_temp_dir_get(&tmpdir, pool_);
+ ASSERT_EQ(APR_SUCCESS, status);
+ char* test_temp_dir;
+ status = apr_filepath_merge(&test_temp_dir, tmpdir, "apr_file_sytem_test",
+ APR_FILEPATH_NATIVE, pool_);
+ ASSERT_EQ(APR_SUCCESS, status);
+ if (file_system_->Exists(test_temp_dir, &handler_).is_false()) {
+ ASSERT_TRUE(file_system_->MakeDir(test_temp_dir, &handler_));
+ }
+ ASSERT_TRUE(file_system_->Exists(test_temp_dir, &handler_).is_true());
+ *str = test_temp_dir;
+ }
+
+ void MyDeleteFileRecursively(const std::string& filename,
+ const char* /*a*/,
+ const char* /*b*/) {
+ if (file_system_->IsDir(filename.c_str(), &handler_).is_true()) {
+ // TODO(lsong): Make it recursive.
+ apr_status_t status = apr_dir_remove(filename.c_str(), pool_);
+ if (status != APR_SUCCESS) {
+ AprReportError(&handler_, __FILE__, __LINE__, "dir remove", status);
+ // TODO(lsong): Rename the dir to try.
+ if (APR_STATUS_IS_ENOTEMPTY(status)) {
+ // Need a tempname to rename to.
+ char* template_name;
+ std::string tempname = filename + "-apr-XXXXXX";
+ status = apr_filepath_merge(&template_name, test_tmpdir_.c_str(),
+ tempname.c_str(), APR_FILEPATH_NATIVE,
+ pool_);
+ ASSERT_EQ(APR_SUCCESS, status);
+ apr_file_t* file;
+ status = apr_file_mktemp(&file, template_name, 0, pool_);
+ ASSERT_EQ(APR_SUCCESS, status);
+ const char* the_path_name;
+ status = apr_file_name_get(&the_path_name, file);
+ ASSERT_EQ(APR_SUCCESS, status);
+ status = apr_file_close(file);
+ ASSERT_EQ(APR_SUCCESS, status);
+ // Got the name to rename to.
+ status = apr_file_rename(filename.c_str(), the_path_name, pool_);
+ if (status != APR_SUCCESS) {
+ AprReportError(&handler_, __FILE__, __LINE__, "dir rename", status);
+ }
+ }
+ }
+ ASSERT_EQ(APR_SUCCESS, status);
+ } else {
+ file_system_->RemoveFile(filename.c_str(), &handler_);
+ }
+ }
+
+ protected:
+ GoogleMessageHandler handler_;
+ scoped_ptr<AprFileSystem> file_system_;
+ apr_pool_t* pool_;
+ std::string test_tmpdir_;
+
+ DISALLOW_COPY_AND_ASSIGN(AprFileSystemTest);
+};
+
+TEST_F(AprFileSystemTest, TestWriteRead) {
+ TestWriteRead();
+}
+
+TEST_F(AprFileSystemTest, TestTemp) {
+ TestTemp();
+}
+
+TEST_F(AprFileSystemTest, TestRename) {
+ TestRename();
+}
+
+TEST_F(AprFileSystemTest, TestRemove) {
+ TestRemove();
+}
+
+TEST_F(AprFileSystemTest, TestExists) {
+ TestExists();
+}
+
+TEST_F(AprFileSystemTest, TestCreateFileInDir) {
+ TestCreateFileInDir();
+}
+
+
+TEST_F(AprFileSystemTest, TestMakeDir) {
+ TestMakeDir();
+}
+
+TEST_F(AprFileSystemTest, TestIsDir) {
+ TestIsDir();
+}
+
+TEST_F(AprFileSystemTest, TestRecursivelyMakeDir) {
+ TestRecursivelyMakeDir();
+}
+
+TEST_F(AprFileSystemTest, TestRecursivelyMakeDir_NoPermission) {
+ TestRecursivelyMakeDir_NoPermission();
+}
+
+TEST_F(AprFileSystemTest, TestRecursivelyMakeDir_FileInPath) {
+ TestRecursivelyMakeDir_FileInPath();
+}
+
+TEST_F(AprFileSystemTest, TestListContents) {
+ TestListContents();
+}
+
+TEST_F(AprFileSystemTest, TestAtime) {
+ TestAtime();
+}
+
+TEST_F(AprFileSystemTest, TestSize) {
+ TestSize();
+}
+
+TEST_F(AprFileSystemTest, TestLock) {
+ TestLock();
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/apr_mutex.cc b/trunk/src/net/instaweb/apache/apr_mutex.cc
new file mode 100644
index 0000000..162e5c5
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apr_mutex.cc
@@ -0,0 +1,38 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "net/instaweb/apache/apr_mutex.h"
+
+#include "apr_pools.h"
+#include "apr_thread_mutex.h"
+
+namespace net_instaweb {
+
+AprMutex::AprMutex(apr_pool_t* pool) {
+ apr_thread_mutex_create(&thread_mutex_, APR_THREAD_MUTEX_DEFAULT, pool);
+}
+
+AprMutex::~AprMutex() {
+ apr_thread_mutex_destroy(thread_mutex_);
+}
+
+void AprMutex::Lock() {
+ apr_thread_mutex_lock(thread_mutex_);
+}
+
+void AprMutex::Unlock() {
+ apr_thread_mutex_unlock(thread_mutex_);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/apr_mutex.h b/trunk/src/net/instaweb/apache/apr_mutex.h
new file mode 100644
index 0000000..9a4da21
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apr_mutex.h
@@ -0,0 +1,41 @@
+// Copyright 2010 Google Inc. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#ifndef NET_INSTAWEB_APACHE_APR_MUTEX_H_
+#define NET_INSTAWEB_APACHE_APR_MUTEX_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/abstract_mutex.h"
+
+// Forward declaration.
+struct apr_thread_mutex_t;
+struct apr_pool_t;
+
+namespace net_instaweb {
+
+class AprMutex : public net_instaweb::AbstractMutex {
+ public:
+ explicit AprMutex(apr_pool_t* pool);
+ virtual ~AprMutex();
+ virtual void Lock();
+ virtual void Unlock();
+ private:
+ apr_thread_mutex_t* thread_mutex_;
+
+ DISALLOW_COPY_AND_ASSIGN(AprMutex);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_APACHE_APR_MUTEX_H_
diff --git a/trunk/src/net/instaweb/apache/apr_statistics.cc b/trunk/src/net/instaweb/apache/apr_statistics.cc
new file mode 100644
index 0000000..26eb8ca
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apr_statistics.cc
@@ -0,0 +1,278 @@
+// Copyright 2010 Google Inc. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "net/instaweb/apache/apr_statistics.h"
+
+#include "apr_global_mutex.h"
+#include "apr_shm.h"
+#include "apr_strings.h"
+#include "apr.h"
+#include "apr_errno.h"
+#include "apr_pools.h"
+#include "apr_time.h"
+#include "base/logging.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/writer.h"
+#include "net/instaweb/util/stack_buffer.h"
+
+extern "C" {
+#include "httpd.h"
+#include "http_config.h"
+#ifdef AP_NEED_SET_MUTEX_PERMS
+#include "unixd.h"
+#endif
+}
+
+
+namespace {
+
+const char kStatisticsDir[] = "statistics";
+const char kStatisticsMutexPrefix[] = "statistics/stats_mutex.";
+const char kStatisticsValuePrefix[] = "statistics/stats_value.";
+
+} // namespace
+
+namespace net_instaweb {
+
+#define COUNT_LOCK_WAIT_TIME 0
+#if COUNT_LOCK_WAIT_TIME
+// Unlocked counter, which is reasonably safe on x86 if it's 32-bit, which
+// is good for over hour of of locked time, which is good enough for
+// experiments, particularly on a prefork system where the only extra
+// thread is the one from serf.
+static uint32 accumulated_time_in_global_locks_us = 0;
+static uint32 prev_message_us = 0;
+#endif
+
+// Helper class for lexically scoped mutexing.
+// TODO(jmarantz): consier merging this with ScopedLock.
+class AprScopedGlobalLock {
+ public:
+ explicit AprScopedGlobalLock(const AprVariable* var) : variable_(var) {
+ if (variable_->mutex_ == NULL) {
+ acquired_ = false;
+ } else {
+#if COUNT_LOCK_WAIT_TIME
+ int64 time_us = apr_time_now();
+#endif
+ acquired_ = variable_->CheckResult(
+ apr_global_mutex_lock(variable_->mutex_), "lock mutex");
+#if COUNT_LOCK_WAIT_TIME
+ int64 delta_us = apr_time_now() - time_us;
+ if (delta_us > 0) {
+ accumulated_time_in_global_locks_us += delta_us;
+ if ((accumulated_time_in_global_locks_us - prev_message_us) > 1000000) {
+ // race condition is possible here
+ prev_message_us = accumulated_time_in_global_locks_us;
+ double time_wasted_seconds = accumulated_time_in_global_locks_us
+ / 1000000.0;
+ LOG(ERROR) << "Cumulative locked time spent: " << time_wasted_seconds
+ << "seconds";
+ }
+ }
+#endif
+ }
+ }
+
+ ~AprScopedGlobalLock() {
+ if (acquired_) {
+ variable_->CheckResult(
+ apr_global_mutex_unlock(variable_->mutex_), "unlock mutex");
+ }
+ }
+
+ bool acquired() const { return acquired_; }
+
+ private:
+ const AprVariable* variable_;
+ bool acquired_;
+
+ DISALLOW_COPY_AND_ASSIGN(AprScopedGlobalLock);
+};
+
+
+
+AprVariable::AprVariable(const StringPiece& name)
+ : mutex_(NULL), name_(name.as_string()), shm_(NULL), value_ptr_(NULL) {
+}
+
+int64 AprVariable::Get64() const {
+ AprScopedGlobalLock lock(this);
+ int64 value = lock.acquired() ? *value_ptr_ : -1;
+ return value;
+}
+
+int AprVariable::Get() const {
+ return Get64();
+}
+
+void AprVariable::Set(int newValue) {
+ AprScopedGlobalLock lock(this);
+ if (lock.acquired()) {
+ *value_ptr_ = newValue;
+ }
+}
+
+void AprVariable::Add(int delta) {
+ AprScopedGlobalLock lock(this);
+ if (lock.acquired()) {
+ *value_ptr_ += delta;
+ }
+}
+
+bool AprVariable::CheckResult(
+ const apr_status_t result, const StringPiece& verb,
+ const StringPiece& filename) const {
+ if (result != APR_SUCCESS) {
+ char buf[kStackBufferSize];
+ apr_strerror(result, buf, sizeof(buf));
+ LOG(ERROR) << "Variable " << name_ << " cannot " << verb << ": " << buf
+ << " " << filename;
+ return false;
+ }
+ return true;
+}
+
+bool AprVariable::InitMutex(const StringPiece& filename_prefix,
+ apr_pool_t* pool, bool parent) {
+ // Create or attach to the mutex if we don't have one already.
+ const char* filename = apr_pstrcat(
+ pool, filename_prefix.as_string().c_str(), kStatisticsMutexPrefix,
+ name_.c_str(), NULL);
+ if (parent) {
+ // We're being called from post_config. Must create mutex.
+ // Ensure the directory exists
+ apr_dir_make(StrCat(filename_prefix, kStatisticsDir).c_str(),
+ APR_FPROT_OS_DEFAULT, pool);
+ // TODO(abliss): do we need to destroy this mutex later?
+ if (CheckResult(apr_global_mutex_create(
+ &mutex_, filename, APR_LOCK_DEFAULT, pool),
+ "create mutex", filename)) {
+ // On apache installations which (a) are unix-based, (b) use a
+ // flock-based mutex, and (c) start the parent process as root but child
+ // processes as a less-privileged user, we need this extra code to set
+ // up the permissions of the lock.
+#ifdef AP_NEED_SET_MUTEX_PERMS
+ CheckResult(unixd_set_global_mutex_perms(mutex_), "chown mutex",
+ filename);
+#endif
+ return true;
+ }
+ } else {
+ // We're being called from child_init. Mutex must already exist.
+ if (mutex_) {
+ if (CheckResult(apr_global_mutex_child_init(&mutex_, filename, pool),
+ "attach mutex", filename)) {
+ return true;
+ } else {
+ // Something went wrong; disable this variable by nulling its mutex.
+ mutex_ = NULL;
+ }
+ } else {
+ CheckResult(APR_ENOLOCK, "attach mutex", filename);
+ }
+ }
+ return false;
+}
+
+bool AprVariable::InitShm(const StringPiece& filename_prefix,
+ apr_pool_t* pool, bool parent) {
+ // On some platforms we inherit the existing segment...
+ if (!shm_) {
+ // ... but on others we must reattach to it.
+ const char* filename = apr_pstrcat(
+ pool, filename_prefix.as_string().c_str(), kStatisticsValuePrefix,
+ name_.c_str(), NULL);
+ if (parent) {
+ // Sometimes the shm/file are leftover from a previous unclean exit.
+ apr_shm_remove(filename, pool);
+ apr_file_remove(filename, pool);
+ int64 foo;
+ // This shm is destroyed when apache is shutdown cleanly.
+ CheckResult(apr_shm_create(&shm_, sizeof(foo), filename, pool),
+ "create shared memory", filename);
+ } else {
+ CheckResult(apr_shm_attach(&shm_, filename, pool),
+ "attach to shared memory", filename);
+ }
+ }
+ if (shm_) {
+ // value_ptr always needs to be reset, even if shm was inherited,
+ // since its base address may have changed.
+ value_ptr_ = reinterpret_cast<int64*>(apr_shm_baseaddr_get(shm_));
+ return true;
+ } else {
+ // Something went wrong; disable this variable by nulling its mutex.
+ mutex_ = NULL;
+ return false;
+ }
+}
+
+AprStatistics::AprStatistics(const StringPiece& filename_prefix)
+ : frozen_(false), is_child_(false), filename_prefix_(filename_prefix) {
+ apr_pool_create(&pool_, NULL);
+}
+
+AprStatistics::~AprStatistics() {
+ if (!is_child_) {
+ apr_pool_destroy(pool_);
+ }
+}
+
+AprVariable* AprStatistics::NewVariable(const StringPiece& name, int index) {
+ if (frozen_) {
+ LOG(ERROR) << "Cannot add variable " << name
+ << " after AprStatistics is frozen!";
+ return NULL;
+ } else {
+ return new AprVariable(name);
+ }
+}
+
+void AprStatistics::InitVariables(bool parent) {
+ is_child_ |= !parent;
+ if (frozen_) {
+ return;
+ }
+ frozen_ = true;
+ // Set up a global mutex and a shared-memory segment for each variable.
+ for (int i = 0, n = variables_.size(); i < n; ++i) {
+ AprVariable* var = variables_[i];
+ if (!var->InitMutex(filename_prefix_, pool_, parent) ||
+ !var->InitShm(filename_prefix_, pool_, parent)) {
+ LOG(ERROR) << "Statistics initialization failed in pid " << getpid();
+ return;
+ }
+ }
+}
+
+void AprStatistics::Dump(Writer* writer, MessageHandler* message_handler) {
+ for (int i = 0, n = variables_.size(); i < n; ++i) {
+ AprVariable* var = variables_[i];
+ writer->Write(var->name(), message_handler);
+ writer->Write(": ", message_handler);
+ writer->Write(Integer64ToString(var->Get64()), message_handler);
+ writer->Write("\n", message_handler);
+ }
+}
+
+void AprStatistics::Clear() {
+ for (int i = 0, n = variables_.size(); i < n; ++i) {
+ AprVariable* var = variables_[i];
+ var->Set(0);
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/apr_statistics.h b/trunk/src/net/instaweb/apache/apr_statistics.h
new file mode 100644
index 0000000..7b50969
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apr_statistics.h
@@ -0,0 +1,114 @@
+// Copyright 2010 Google Inc. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#ifndef NET_INSTAWEB_APACHE_APR_STATISTICS_H_
+#define NET_INSTAWEB_APACHE_APR_STATISTICS_H_
+
+#include <string>
+
+#include "net/instaweb/util/public/statistics_template.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "apr_shm.h"
+#include "apr.h"
+#include "apr_errno.h"
+
+struct apr_pool_t;
+struct apr_global_mutex_t;
+namespace net_instaweb {
+
+class Writer;
+class MessageHandler;
+
+// An implementation of Statistics using the APR's Shared Memory module,
+// apr_shm. These statistics will be shared amongst all processes and threads
+// spawned by Apache. Note that we will be obtaining a global mutex for every
+// read and write to these variables. Since this may be expensive, it is
+// recommended that each thread keep a local cache and infrequently write
+// through to this Statistics object. TODO(abliss): actually do this.
+//
+// Because we must allocate shared memory segments before the module forks off
+// its children, all AddVariable calls must be in the post_config hook. Once
+// all variables are added, you must call InitVariables.
+//
+// If a variable fails to initialize (due to either its mutex or its shared
+// memory segment not working), it will not increment in that process (and a
+// warning message will be logged). Other variables will work normally. If the
+// variable fails to initialize in the process that happens to serve the
+// mod_pagespeed_statistics page, then the variable will show up with value -1.
+//
+// Implementation details heavily cribbed from mod_shm_counter by Aaron Bannert.
+
+class AprVariable : public Variable {
+ public:
+ explicit AprVariable(const StringPiece& name);
+ int64 Get64() const;
+ virtual int Get() const;
+ virtual void Set(int newValue);
+ virtual void Add(int delta);
+ private:
+ friend class AprStatistics;
+ friend class AprScopedGlobalLock;
+
+ // Logs an error message and returns false if the result is not SUCCESS.
+ bool CheckResult(
+ const apr_status_t result, const StringPiece& verb,
+ const StringPiece& filename = EmptyString::kEmptyString) const;
+ // Initialize this variable's mutex
+ bool InitMutex(const StringPiece& filename_prefix, apr_pool_t* pool,
+ bool parent);
+ // Initialize this variable's shared memory segment.
+ bool InitShm(const StringPiece& filename_prefix, apr_pool_t* pool,
+ bool parent);
+ const std::string& name() const { return name_; }
+ // The global (cross-thread, cross-process) mutex protecting value_.
+ // This is NULL if the variable has not yet been properly initialized.
+ apr_global_mutex_t* mutex_;
+ // The name of this variable.
+ const std::string name_;
+ // Pointer to the shared-memory segment containing our current value.
+ apr_shm_t* shm_;
+ // Offset within the shared-memory segment for our current value.
+ int64* value_ptr_;
+};
+
+class AprStatistics : public StatisticsTemplate<AprVariable> {
+ public:
+ AprStatistics(const StringPiece& filename_prefix);
+ virtual ~AprStatistics();
+
+ virtual AprVariable* NewVariable(const StringPiece& name, int index);
+
+ // Allocate shared memory segments and mutices for all variables. This must
+ // be called with parent=true from the post_config hook, and with parent=false
+ // from the child_init hook. After this is called, you must no longer call
+ // AddVariable.
+ void InitVariables(bool parent);
+
+ // Dump the statistics to the given writer.
+ void Dump(Writer* writer, MessageHandler* message_handler);
+
+ // Set all statistics to 0.
+ void Clear();
+
+ bool frozen() const { return frozen_; }
+ private:
+ bool frozen_;
+ bool is_child_;
+ const StringPiece& filename_prefix_;
+ apr_pool_t* pool_;
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_APACHE_APR_STATISTICS_H_
diff --git a/trunk/src/net/instaweb/apache/apr_timer.cc b/trunk/src/net/instaweb/apache/apr_timer.cc
new file mode 100644
index 0000000..c085873
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apr_timer.cc
@@ -0,0 +1,32 @@
+// Copyright 2010 Google Inc. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "apr_time.h"
+#include "net/instaweb/apache/apr_timer.h"
+
+namespace net_instaweb {
+
+AprTimer::~AprTimer() {
+}
+
+int64 AprTimer::NowUs() const {
+ // apr_time_now returns microseconds.
+ return apr_time_now();
+}
+
+void AprTimer::SleepUs(int64 us) {
+ apr_sleep(us);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/apr_timer.h b/trunk/src/net/instaweb/apache/apr_timer.h
new file mode 100644
index 0000000..90ba26a
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/apr_timer.h
@@ -0,0 +1,33 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#ifndef HTML_REWRITER_APR_TIMER_H_
+#define HTML_REWRITER_APR_TIMER_H_
+
+#include "net/instaweb/util/public/timer.h"
+
+using net_instaweb::Timer;
+
+namespace net_instaweb {
+
+class AprTimer : public Timer {
+ public:
+ virtual ~AprTimer();
+ virtual int64 NowUs() const;
+ virtual void SleepUs(int64 us);
+};
+
+} // namespace net_instaweb
+
+#endif // HTML_REWRITER_APR_TIMER_H_
diff --git a/trunk/src/net/instaweb/apache/header_util.cc b/trunk/src/net/instaweb/apache/header_util.cc
new file mode 100644
index 0000000..f87912d
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/header_util.cc
@@ -0,0 +1,143 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "net/instaweb/apache/header_util.h"
+#include "net/instaweb/apache/apr_timer.h"
+#include "net/instaweb/apache/instaweb_context.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/time_util.h"
+
+#include "http_core.h"
+
+namespace net_instaweb {
+
+namespace {
+
+const char kRepairCachingHeader[] = "X-Mod-Pagespeed-Repair";
+
+int AddAttributeCallback(void *rec, const char *key, const char *value) {
+ MetaData* meta_data = static_cast<MetaData*>(rec);
+ meta_data->Add(key, value);
+ return 1;
+}
+
+} // namespace
+
+// proto_num is the version number of protocol; 1.1 = 1001
+void ApacheHeaderToMetaData(const apr_table_t* apache_headers,
+ int status_code,
+ int proto_num,
+ MetaData* meta_data) {
+ meta_data->SetStatusAndReason(static_cast<HttpStatus::Code>(status_code));
+ if (proto_num >= 1000) {
+ meta_data->set_major_version(proto_num / 1000);
+ meta_data->set_minor_version(proto_num % 1000);
+ }
+ apr_table_do(AddAttributeCallback, meta_data, apache_headers, NULL);
+}
+
+void MetaDataToApacheHeader(const MetaData& meta_data,
+ apr_table_t* apache_headers,
+ int* status_code,
+ int* proto_num) {
+ *status_code = meta_data.status_code();
+ *proto_num = (meta_data.major_version() * 1000) + meta_data.minor_version();
+ for (int i = 0, n = meta_data.NumAttributes(); i < n; ++i) {
+ apr_table_add(apache_headers, meta_data.Name(i), meta_data.Value(i));
+ }
+}
+
+void UpdateCacheHeaders(const char* cache_control, request_rec* request) {
+ SimpleMetaData response_headers;
+ response_headers.set_status_code(request->status);
+ if (request->proto_num >= 1000) {
+ response_headers.set_major_version(request->proto_num / 1000);
+ response_headers.set_minor_version(request->proto_num % 1000);
+ }
+ AprTimer timer;
+ response_headers.Add(HttpAttributes::kCacheControl, cache_control);
+ response_headers.SetDate(timer.NowMs());
+ response_headers.ComputeCaching();
+ if (response_headers.IsCacheable()) {
+ apr_table_set(request->headers_out, HttpAttributes::kCacheControl,
+ cache_control);
+ apr_table_setn(request->headers_out, HttpAttributes::kEtag,
+ ResourceManager::kResourceEtagValue); // no copy neeeded
+
+ // Convert our own cache-control data into an Expires header.
+ int64 expire_time_ms = response_headers.CacheExpirationTimeMs();
+ bool unset_expires = true;
+ if (expire_time_ms > 0) {
+ std::string time_string;
+ if (ConvertTimeToString(expire_time_ms, &time_string)) {
+ apr_table_set(request->headers_out, HttpAttributes::kExpires,
+ time_string.c_str());
+ unset_expires = false;
+ }
+ }
+ if (unset_expires) {
+ apr_table_unset(request->headers_out, HttpAttributes::kExpires);
+ }
+ } else {
+ apr_table_unset(request->headers_out, HttpAttributes::kExpires);
+ apr_table_unset(request->headers_out, HttpAttributes::kEtag);
+ apr_table_unset(request->headers_out, HttpAttributes::kLastModified);
+ }
+}
+
+void SetupCacheRepair(const char* cache_control, request_rec* request) {
+ // In case Apache configuration directives set up caching headers,
+ // we will need override our cache-extended resources in a late-running
+ // output filter.
+ // TODO(jmarantz): Do not use headers_out as out message passing
+ // mechanism. Switch to configuration vectors or something like that
+ // instead.
+ apr_table_add(request->headers_out, kRepairCachingHeader, cache_control);
+ // Add the repair headers filter to fix the cacheing header.
+ ap_add_output_filter(InstawebContext::kRepairHeadersFilterName,
+ NULL, request, request->connection);
+}
+
+void RepairCachingHeaders(request_rec* request) {
+ const char* cache_control = apr_table_get(request->headers_out,
+ kRepairCachingHeader);
+ if (cache_control != NULL) {
+ SetCacheControl(cache_control, request);
+ apr_table_unset(request->headers_out, kRepairCachingHeader);
+ }
+}
+
+void SetCacheControl(const char* cache_control, request_rec* request) {
+ UpdateCacheHeaders(cache_control, request);
+ apr_table_set(request->headers_out, HttpAttributes::kCacheControl,
+ cache_control);
+}
+
+int PrintAttributeCallback(void *rec, const char *key, const char *value) {
+ fprintf(stdout, " %s: %s\n", key, value);
+ return 1;
+}
+
+// This routine is intended for debugging so fprintf to stdout is the way
+// to get instant feedback.
+void PrintHeaders(request_rec* request) {
+ fprintf(stdout, "Input headers:\n");
+ apr_table_do(PrintAttributeCallback, NULL, request->headers_in, NULL);
+ fprintf(stdout, "Output headers:\n");
+ apr_table_do(PrintAttributeCallback, NULL, request->headers_in, NULL);
+ fflush(stdout);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/header_util.h b/trunk/src/net/instaweb/apache/header_util.h
new file mode 100644
index 0000000..4d86c8e
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/header_util.h
@@ -0,0 +1,69 @@
+// Copyright 2010 Google Inc. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#ifndef NET_INSTAWEB_APACHE_HEADER_UTIL_H_
+#define NET_INSTAWEB_APACHE_HEADER_UTIL_H_
+
+#include "net/instaweb/util/public/meta_data.h"
+// The httpd header must be after the meta_data.h. Otherwise, the
+// compiler will complain
+// "strtoul_is_not_a_portable_function_use_strtol_instead".
+#include "httpd.h"
+
+namespace net_instaweb {
+
+// Converts Apache header structure into an Instaweb MetaData.
+//
+// proto_num is the version number of protocol; 1.1 = 1001
+void ApacheHeaderToMetaData(const apr_table_t* apache_headers,
+ int status_code,
+ int proto_num,
+ MetaData* meta_data);
+
+// Converts MetaData structure into an Apache header.
+//
+// proto_num is the version number of protocol; 1.1 = 1001
+void MetaDataToApacheHeader(const MetaData& meta_data,
+ apr_table_t* apache_headers,
+ int* status_code,
+ int* proto_num);
+
+// Examines a cache-control string and updates the output headers in
+// request to match it. If the response is cacheable, then
+// we assume it's cacheable forever (via cache extension, and
+// so we set an etag and a matching Expires header.
+//
+// If the content is not cacheable, then we instead clear the etag,
+// last-modified, and expires, in addition to setting the cache-control
+// as specified.
+void SetCacheControl(const char* cache_control, request_rec* request);
+
+// Like SetCacheControl but only updates the other headers, does not
+// set the cache-control header itself. Call this if cache-control is
+// already set and you wish to make the other headers consistent.
+void UpdateCacheHeaders(const char* cache_control, request_rec* request);
+
+// mod_headers typically runs after mod_pagespeed and borks its headers.
+// So we must insert a late filter to unbork the headers.
+void SetupCacheRepair(const char* cache_control, request_rec* request);
+
+// This is called from the late-running filter to unbork the headers.
+void RepairCachingHeaders(request_rec* request);
+
+// Debug utility for printing Apache headers to stdout
+void PrintHeaders(request_rec* request);
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_APACHE_HEADER_UTIL_H_
diff --git a/trunk/src/net/instaweb/apache/instaweb_context.cc b/trunk/src/net/instaweb/apache/instaweb_context.cc
new file mode 100644
index 0000000..2ea9e1d
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/instaweb_context.cc
@@ -0,0 +1,213 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Author: jmarantz@google.com (Joshua Marantz)
+// lsong@google.com (Libo Song)
+
+#include "net/instaweb/apache/instaweb_context.h"
+#include "net/instaweb/apache/header_util.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/gzip_inflater.h"
+#include "net/instaweb/util/stack_buffer.h"
+#include "http_config.h"
+
+extern "C" {
+extern module AP_MODULE_DECLARE_DATA pagespeed_module;
+}
+
+namespace net_instaweb {
+
+const char InstawebContext::kRepairHeadersFilterName[] =
+ "MOD_PAGESPEED_REPAIR_HEADERS";
+
+InstawebContext::InstawebContext(request_rec* request,
+ const ContentType& content_type,
+ ApacheRewriteDriverFactory* factory,
+ const std::string& absolute_url,
+ bool use_custom_options,
+ const RewriteOptions& custom_options)
+ : content_encoding_(kNone),
+ content_type_(content_type),
+ factory_(factory),
+ string_writer_(&output_),
+ inflater_(NULL),
+ content_detection_state_(kStart),
+ absolute_url_(absolute_url) {
+ if (use_custom_options) {
+ // TODO(jmarantz): this is a temporary hack until we sort out better
+ // memory management of RewriteOptions. This will drag on performance.
+ // We need to do this because we are changing RewriteDriver to keep
+ // a reference to its options throughout its lifetime to refer to the
+ // domain lawyer and other options.
+ rewrite_options_.CopyFrom(custom_options);
+ custom_rewriter_.reset(factory->NewCustomRewriteDriver(rewrite_options_));
+ rewrite_driver_ = custom_rewriter_.get();
+ } else {
+ rewrite_driver_ = factory->NewRewriteDriver();
+ }
+
+ ComputeContentEncoding(request);
+ apr_pool_cleanup_register(request->pool, this, Cleanup,
+ apr_pool_cleanup_null);
+
+ bucket_brigade_ = apr_brigade_create(request->pool,
+ request->connection->bucket_alloc);
+
+ if (content_encoding_ == kGzip || content_encoding_ == kDeflate) {
+ // TODO(jmarantz): consider keeping a pool of these if they are expensive
+ // to initialize.
+ if (content_encoding_ == kGzip) {
+ inflater_.reset(new GzipInflater(GzipInflater::kGzip));
+ } else {
+ inflater_.reset(new GzipInflater(GzipInflater::kDeflate));
+ }
+ inflater_->Init();
+ }
+
+
+ const char* user_agent = apr_table_get(request->headers_in,
+ HttpAttributes::kUserAgent);
+ rewrite_driver_->SetUserAgent(user_agent);
+ // TODO(lsong): Bypass the string buffer, writer data directly to the next
+ // apache bucket.
+ rewrite_driver_->SetWriter(&string_writer_);
+}
+
+InstawebContext::~InstawebContext() {
+ if (custom_rewriter_ == NULL) {
+ factory_->ReleaseRewriteDriver(rewrite_driver_);
+ }
+}
+
+void InstawebContext::Rewrite(const char* input, int size) {
+ if (inflater_.get() != NULL) {
+ char buf[kStackBufferSize];
+ inflater_->SetInput(input, size);
+ while (inflater_->HasUnconsumedInput()) {
+ int num_inflated_bytes = inflater_->InflateBytes(buf, kStackBufferSize);
+ ProcessBytes(buf, num_inflated_bytes);
+ }
+ } else {
+ ProcessBytes(input, size);
+ }
+}
+
+namespace {
+
+// http://en.wikipedia.org/wiki/Byte_order_mark
+//
+// The byte-order marker sequence will typically appear at the beginning of
+// an HTML or XML file. We probably should be order-sensitive but for now
+// we will just treat all such characters as allowable characters preceding
+// the HTML. Note the use of unsigned char here to avoid sign-extending when
+// comparing to the int constants.
+inline bool IsByteOrderMarkerCharacter(unsigned char c) {
+ return ((c == 0xef) || (c == 0xbb) || (c == 0xbf));
+}
+
+} // namespace
+
+void InstawebContext::ProcessBytes(const char* input, int size) {
+ // Try to figure out whether this looks like HTML or not, if we haven't
+ // figured it out already. We just scan past whitespace for '<'.
+ for (int i = 0; (content_detection_state_ == kStart) && (i < size); ++i) {
+ char c = input[i];
+ if (c == '<') {
+ content_detection_state_ = kHtml;
+ rewrite_driver_->html_parse()->StartParseWithType(absolute_url_,
+ content_type_);
+ } else if (!isspace(c) && !IsByteOrderMarkerCharacter(c)) {
+ // TODO(jmarantz): figure out whether it's possible to remove our
+ // filter from the chain entirely.
+ //
+ // TODO(jmarantz): look for 'gzip' data. We do not expect to see
+ // this if the Content-Encoding header is set upstream of mod_pagespeed,
+ // but we have heard evidence from the field that WordPress plugins and
+ // possibly other modules send compressed data through without that
+ // header.
+ content_detection_state_ = kNotHtml;
+ }
+ }
+
+ switch (content_detection_state_) {
+ case kStart:
+ // Handle the corner where the first buffer of text contains
+ // only whitespace, which we will retain for the next call.
+ buffer_.append(input, size);
+ break;
+
+ case kHtml:
+ // Looks like HTML: send it through the HTML rewriter.
+ if (!buffer_.empty()) {
+ rewrite_driver_->html_parse()->ParseText(
+ buffer_.data(), buffer_.size());
+ buffer_.clear();
+ }
+ rewrite_driver_->html_parse()->ParseText(input, size);
+ break;
+
+ case kNotHtml:
+ // Looks like something that's not HTML. Send it directly to the
+ // output buffer.
+ output_.append(buffer_.data(), buffer_.size());
+ buffer_.clear();
+ output_.append(input, size);
+ break;
+ }
+}
+
+apr_status_t InstawebContext::Cleanup(void* object) {
+ InstawebContext* ic = static_cast<InstawebContext*>(object);
+ delete ic;
+ return APR_SUCCESS;
+}
+
+void InstawebContext::ComputeContentEncoding(request_rec* request) {
+ // Check if the content is gzipped. Steal from mod_deflate.
+ const char* encoding = apr_table_get(
+ request->headers_out, HttpAttributes::kContentEncoding);
+ if (encoding) {
+ const char* err_enc = apr_table_get(request->err_headers_out,
+ HttpAttributes::kContentEncoding);
+ if (err_enc) {
+ // We don't properly handle stacked encodings now.
+ content_encoding_ = kOther;
+ }
+ } else {
+ encoding = apr_table_get(request->err_headers_out,
+ HttpAttributes::kContentEncoding);
+ }
+
+ if (encoding) {
+ if (strcasecmp(encoding, HttpAttributes::kGzip) == 0) {
+ content_encoding_ = kGzip;
+ } else if (strcasecmp(encoding, HttpAttributes::kDeflate) == 0) {
+ content_encoding_ = kDeflate;
+ } else {
+ content_encoding_ = kOther;
+ }
+ }
+
+ // Copy the output headers coming into our own filter into response_headers_.
+ // This is purely for debugging context.
+ ApacheHeaderToMetaData(request->headers_out, request->status,
+ request->proto_num, &response_headers_);
+}
+
+ApacheRewriteDriverFactory* InstawebContext::Factory(server_rec* server) {
+ return static_cast<ApacheRewriteDriverFactory*>
+ ap_get_module_config(server->module_config, &pagespeed_module);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/instaweb_context.h b/trunk/src/net/instaweb/apache/instaweb_context.h
new file mode 100644
index 0000000..7727bd8
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/instaweb_context.h
@@ -0,0 +1,105 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Author: jmarantz@google.com (Joshua Marantz)
+// lsong@google.com (Libo Song)
+
+#ifndef NET_INSTAWEB_APACHE_INSTAWEB_CONTEXT_H_
+#define NET_INSTAWEB_APACHE_INSTAWEB_CONTEXT_H_
+
+#include "base/scoped_ptr.h"
+#include "net/instaweb/apache/apache_rewrite_driver_factory.h"
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+#include "net/instaweb/util/public/content_type.h"
+#include <string>
+#include "net/instaweb/util/public/string_writer.h"
+
+// The httpd header must be after the
+// apache_rewrite_driver_factory.h. Otherwise, the compiler will
+// complain "strtoul_is_not_a_portable_function_use_strtol_instead".
+#include "httpd.h"
+
+namespace net_instaweb {
+
+class GzipInflater;
+class RewriteOptions;
+
+// We use the following structure to keep the instaweb module context. The
+// rewriter will put the rewritten content into the output string when flushed
+// or finished. We call Flush when we see the FLUSH bucket, and call Finish when
+// we see the EOS bucket.
+class InstawebContext {
+ public:
+ enum ContentEncoding {kNone, kGzip, kDeflate, kOther};
+ enum ContentDetectionState {kStart, kHtml, kNotHtml};
+
+ static const char kRepairHeadersFilterName[];
+
+ InstawebContext(request_rec* request,
+ const ContentType& content_type,
+ net_instaweb::ApacheRewriteDriverFactory* factory,
+ const std::string& base_url,
+ bool use_custom_options,
+ const RewriteOptions& custom_options);
+ ~InstawebContext();
+
+ void Rewrite(const char* input, int size);
+ void Flush() {
+ if (content_detection_state_ == kHtml) {
+ rewrite_driver_->html_parse()->Flush();
+ }
+ }
+ void Finish() {
+ if (content_detection_state_ == kHtml) {
+ rewrite_driver_->html_parse()->FinishParse();
+ }
+ }
+ bool empty() const { return output_.empty(); }
+ apr_bucket_brigade* bucket_brigade() const { return bucket_brigade_; }
+ const std::string& output() { return output_; }
+ void clear() { output_.clear(); } // TODO(jmarantz): needed?
+ ContentEncoding content_encoding() const { return content_encoding_; }
+
+ // Looks up the factory from the server rec.
+ // TODO(jmarantz): Is there a better place to put this? It needs to
+ // be used by both mod_instaweb.cc and instaweb_handler.cc.
+ static ApacheRewriteDriverFactory* Factory(server_rec* server);
+
+ private:
+ void ComputeContentEncoding(request_rec* request);
+ void ProcessBytes(const char* input, int size);
+ static apr_status_t Cleanup(void* object);
+
+ std::string output_; // content after instaweb rewritten.
+ apr_bucket_brigade* bucket_brigade_;
+ ContentEncoding content_encoding_;
+ const ContentType content_type_;
+
+ net_instaweb::ApacheRewriteDriverFactory* factory_;
+ net_instaweb::RewriteDriver* rewrite_driver_;
+ net_instaweb::StringWriter string_writer_;
+ scoped_ptr<GzipInflater> inflater_;
+ scoped_ptr<RewriteDriver> custom_rewriter_;
+ std::string buffer_;
+ SimpleMetaData response_headers_;
+ ContentDetectionState content_detection_state_;
+ std::string absolute_url_;
+ RewriteOptions rewrite_options_;
+
+ DISALLOW_COPY_AND_ASSIGN(InstawebContext);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_APACHE_INSTAWEB_CONTEXT_H_
diff --git a/trunk/src/net/instaweb/apache/instaweb_handler.cc b/trunk/src/net/instaweb/apache/instaweb_handler.cc
new file mode 100644
index 0000000..9645e8b
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/instaweb_handler.cc
@@ -0,0 +1,263 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Author: lsong@google.com (Libo Song)
+// jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/apache/instaweb_handler.h"
+
+#include "apr_strings.h"
+#include "base/basictypes.h"
+#include "base/string_util.h"
+#include "net/instaweb/apache/apache_slurp.h"
+#include "net/instaweb/apache/apr_statistics.h"
+#include "net/instaweb/apache/apr_timer.h"
+#include "net/instaweb/apache/header_util.h"
+#include "net/instaweb/apache/instaweb_context.h"
+#include "net/instaweb/apache/serf_async_callback.h"
+#include "net/instaweb/apache/serf_url_async_fetcher.h"
+#include "net/instaweb/apache/mod_instaweb.h"
+#include "net/instaweb/rewriter/public/add_instrumentation_filter.h"
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/string_writer.h"
+#include "http_config.h"
+#include "http_core.h"
+#include "http_log.h"
+#include "http_protocol.h"
+
+namespace net_instaweb {
+
+namespace {
+
+bool IsCompressibleContentType(const char* content_type) {
+ if (content_type == NULL) {
+ return false;
+ }
+ std::string type = content_type;
+ size_t separator_idx = type.find(";");
+ if (separator_idx != std::string::npos) {
+ type.erase(separator_idx);
+ }
+
+ bool res = false;
+ if (type.find("text/") == 0) {
+ res = true;
+ } else if (type.find("application/") == 0) {
+ if (type.find("javascript") != type.npos ||
+ type.find("json") != type.npos ||
+ type.find("ecmascript") != type.npos ||
+ type == "application/livescript" ||
+ type == "application/js" ||
+ type == "application/jscript" ||
+ type == "application/x-js" ||
+ type == "application/xhtml+xml" ||
+ type == "application/xml") {
+ res = true;
+ }
+ }
+
+ return res;
+}
+
+// Default handler when the file is not found
+void instaweb_default_handler(const std::string& url, request_rec* request) {
+ request->status = HTTP_NOT_FOUND;
+ ap_set_content_type(request, "text/html; charset=utf-8");
+ ap_rputs("<html><head><title>Not Found</title></head>", request);
+ ap_rputs("<body><h1>Apache server with mod_pagespeed</h1>OK", request);
+ ap_rputs("<hr>NOT FOUND:", request);
+ ap_rputs(url.c_str(), request);
+ ap_rputs("</body></html>", request);
+}
+
+// predeclare to minimize diffs for now. TODO(jmarantz): reorder
+void send_out_headers_and_body(
+ request_rec* request,
+ const SimpleMetaData& response_headers,
+ const std::string& output);
+
+// Determines whether the url can be handled as a mod_pagespeed resource,
+// and handles it, returning true. A 'true' routine means that this
+// method believed the URL was a mod_pagespeed resource -- it does not
+// imply that it was handled successfully. That information will be
+// in the status code in the response headers.
+bool handle_as_resource(ApacheRewriteDriverFactory* factory,
+ request_rec* request,
+ const std::string& url) {
+ RewriteDriver* rewrite_driver = factory->NewRewriteDriver();
+
+ SimpleMetaData request_headers, response_headers;
+ int n = arraysize(RewriteDriver::kPassThroughRequestAttributes);
+ for (int i = 0; i < n; ++i) {
+ const char* value = apr_table_get(
+ request->headers_in,
+ RewriteDriver::kPassThroughRequestAttributes[i]);
+ if (value != NULL) {
+ request_headers.Add(RewriteDriver::kPassThroughRequestAttributes[i],
+ value);
+ }
+ }
+ std::string output; // TODO(jmarantz): quit buffering resource output
+ StringWriter writer(&output);
+ MessageHandler* message_handler = factory->message_handler();
+ SerfAsyncCallback* callback = new SerfAsyncCallback(
+ &response_headers, &writer);
+ bool handled = rewrite_driver->FetchResource(
+ url, request_headers, callback->response_headers(), callback->writer(),
+ message_handler, callback);
+ if (handled) {
+ message_handler->Message(kInfo, "Fetching resource %s...", url.c_str());
+ if (!callback->done()) {
+ SerfUrlAsyncFetcher* serf_async_fetcher =
+ factory->serf_url_async_fetcher();
+ AprTimer timer;
+ int64 max_ms = factory->fetcher_time_out_ms();
+ for (int64 start_ms = timer.NowMs(), now_ms = start_ms;
+ !callback->done() && now_ms - start_ms < max_ms;
+ now_ms = timer.NowMs()) {
+ int64 remaining_us = max_ms - (now_ms - start_ms);
+ serf_async_fetcher->Poll(remaining_us);
+ }
+
+ if (!callback->done()) {
+ message_handler->Message(kError, "Timeout on url %s", url.c_str());
+ }
+ }
+ if (callback->success()) {
+ message_handler->Message(kInfo, "Fetch succeeded for %s, status=%d",
+ url.c_str(), response_headers.status_code());
+ send_out_headers_and_body(request, response_headers, output);
+ } else {
+ message_handler->Message(kError, "Fetch failed for %s, status=%d",
+ url.c_str(), response_headers.status_code());
+ factory->Increment404Count();
+ instaweb_default_handler(url, request);
+ }
+ } else {
+ callback->Done(false);
+ }
+ callback->Release();
+ factory->ReleaseRewriteDriver(rewrite_driver);
+ return handled;
+}
+
+void send_out_headers_and_body(
+ request_rec* request,
+ const SimpleMetaData& response_headers,
+ const std::string& output) {
+ if (response_headers.status_code() != 0) {
+ request->status = response_headers.status_code();
+ }
+ for (int idx = 0; idx < response_headers.NumAttributes(); ++idx) {
+ const char* name = response_headers.Name(idx);
+ const char* value = response_headers.Value(idx);
+ if (strcasecmp(name, HttpAttributes::kContentType) == 0) {
+ // ap_set_content_type does not make a copy of the string, we need
+ // to duplicate it.
+ char* ptr = apr_pstrdup(request->pool, value);
+ ap_set_content_type(request, ptr);
+ } else {
+ if (strcasecmp(name, HttpAttributes::kCacheControl) == 0) {
+ SetupCacheRepair(value, request);
+ }
+ // apr_table_add makes copies of both head key and value, so we do not
+ // have to duplicate them.
+ apr_table_add(request->headers_out, name, value);
+ }
+ }
+ if (response_headers.status_code() == HttpStatus::kOK &&
+ IsCompressibleContentType(request->content_type)) {
+ // Make sure compression is enabled for this response.
+ ap_add_output_filter("DEFLATE", NULL, request, request->connection);
+ }
+
+ // Recompute the content-length, because the content may have changed.
+ ap_set_content_length(request, output.size());
+ // Send the body
+ ap_rwrite(output.c_str(), output.size(), request);
+}
+
+} // namespace
+
+apr_status_t repair_caching_header(ap_filter_t *filter,
+ apr_bucket_brigade *bb) {
+ request_rec* request = filter->r;
+ RepairCachingHeaders(request);
+ ap_remove_output_filter(filter);
+ return ap_pass_brigade(filter->next, bb);
+}
+
+int instaweb_handler(request_rec* request) {
+ ApacheRewriteDriverFactory* factory =
+ InstawebContext::Factory(request->server);
+ int ret = OK;
+
+ // Only handle GET request
+ if (request->method_number != M_GET) {
+ ap_log_rerror(APLOG_MARK, APLOG_DEBUG, APR_SUCCESS, request,
+ "Not GET request: %d.", request->method_number);
+ ret = DECLINED;
+ } else if (strcmp(request->handler, "mod_pagespeed_statistics") == 0) {
+ std::string output;
+ SimpleMetaData response_headers;
+ StringWriter writer(&output);
+ AprStatistics* statistics = factory->statistics();
+ if (statistics) {
+ statistics->Dump(&writer, factory->message_handler());
+ }
+ send_out_headers_and_body(request, response_headers, output);
+ } else if (strcmp(request->handler, "mod_pagespeed_beacon") == 0) {
+ RewriteDriver* driver = factory->NewRewriteDriver();
+ AddInstrumentationFilter* aif = driver->add_instrumentation_filter();
+ if (aif && aif->HandleBeacon(request->unparsed_uri)) {
+ ret = HTTP_NO_CONTENT;
+ } else {
+ ret = DECLINED;
+ }
+ factory->ReleaseRewriteDriver(driver);
+ } else {
+ /*
+ * In some contexts we are seeing relative URLs passed
+ * into request->unparsed_uri. But when using mod_slurp, the rewritten
+ * HTML contains complete URLs, so this construction yields the host:port
+ * prefix twice.
+ *
+ * TODO(jmarantz): Figure out how to do this correctly at all times.
+ */
+ std::string url;
+ if (strncmp(request->unparsed_uri, "http://", 7) == 0) {
+ url = request->unparsed_uri;
+ } else {
+ url = ap_construct_url(request->pool, request->unparsed_uri, request);
+ }
+
+ if (!handle_as_resource(factory, request, url)) {
+ if (factory->slurping_enabled()) {
+ SlurpUrl(url, factory, request);
+ if (request->status == HTTP_NOT_FOUND) {
+ factory->IncrementSlurpCount();
+ }
+ } else {
+ ret = DECLINED;
+ }
+ }
+ }
+ return ret;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/instaweb_handler.h b/trunk/src/net/instaweb/apache/instaweb_handler.h
new file mode 100644
index 0000000..f364f82
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/instaweb_handler.h
@@ -0,0 +1,41 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#ifndef MOD_INSTAWEB_INSTAWEB_HANDLER_H_
+#define MOD_INSTAWEB_INSTAWEB_HANDLER_H_
+
+// The httpd header must be after the instaweb_context.h. Otherwise,
+// the compiler will complain
+// "strtoul_is_not_a_portable_function_use_strtol_instead".
+#include "net/instaweb/apache/instaweb_context.h"
+#include "httpd.h" // NOLINT
+
+namespace net_instaweb {
+
+// The content generator for instaweb generated content, for example, the
+// combined CSS file. Requests for not-instab generated content will be
+// declined so that other Apache handlers may operate on them.
+int instaweb_handler(request_rec* request);
+
+// output-filter function to repair caching headers, which might have
+// been altered by a directive like:
+//
+// <FilesMatch "\.(jpg|jpeg|gif|png|js|css)$">
+// Header set Cache-control "public, max-age=600"
+// </FilesMatch>
+apr_status_t repair_caching_header(ap_filter_t *filter, apr_bucket_brigade *bb);
+
+} // namespace net_instaweb
+
+#endif // MOD_INSTAWEB_INSTAWEB_HANDLER_H_
diff --git a/trunk/src/net/instaweb/apache/log_message_handler.cc b/trunk/src/net/instaweb/apache/log_message_handler.cc
new file mode 100644
index 0000000..38a9d4a
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/log_message_handler.cc
@@ -0,0 +1,132 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "net/instaweb/apache/log_message_handler.h"
+
+#include <limits>
+#include <string>
+#include "base/debug_util.h"
+#include "base/logging.h"
+#include "httpd.h"
+
+// When HAVE_SYSLOG is defined, apache http_log.h will include syslog.h, which
+// #defined LOG_* as numbers. This conflicts with what we are using those here.
+#undef HAVE_SYSLOG
+#include "http_log.h"
+
+// Make sure we don't attempt to use LOG macros here, since doing so
+// would cause us to go into an infinite log loop.
+#undef LOG
+#define LOG USING_LOG_HERE_WOULD_CAUSE_INFINITE_RECURSION
+
+namespace {
+
+apr_pool_t* log_pool = NULL;
+
+const int kMaxInt = std::numeric_limits<int>::max();
+int log_level_cutoff = kMaxInt;
+std::string* mod_pagespeed_version = NULL;
+
+int GetApacheLogLevel(int severity) {
+ switch (severity) {
+ case logging::LOG_INFO:
+ return APLOG_NOTICE;
+ case logging::LOG_WARNING:
+ return APLOG_WARNING;
+ case logging::LOG_ERROR:
+ return APLOG_ERR;
+ case logging::LOG_ERROR_REPORT:
+ return APLOG_CRIT;
+ case logging::LOG_FATAL:
+ return APLOG_ALERT;
+ default:
+ return APLOG_NOTICE;
+ }
+}
+
+bool LogMessageHandler(int severity, const std::string& str) {
+ const int this_log_level = GetApacheLogLevel(severity);
+
+ std::string message = str;
+ if (severity == logging::LOG_FATAL) {
+ if (DebugUtil::BeingDebugged()) {
+ DebugUtil::BreakDebugger();
+ } else {
+ StackTrace trace;
+ std::ostringstream stream;
+ trace.OutputToStream(&stream);
+ message.append(stream.str());
+ }
+ }
+
+ // Trim the newline off the end of the message string.
+ size_t last_msg_character_index = message.length() - 1;
+ if (message[last_msg_character_index] == '\n') {
+ message.resize(last_msg_character_index);
+ }
+
+ if (this_log_level <= log_level_cutoff || log_level_cutoff == kMaxInt) {
+ ap_log_perror(APLOG_MARK, this_log_level, APR_SUCCESS, log_pool,
+ "[mod_pagespeed %s] %s",
+ (mod_pagespeed_version == NULL)
+ ? ""
+ : mod_pagespeed_version->c_str(),
+ message.c_str());
+ }
+
+ if (severity == logging::LOG_FATAL) {
+ // Crash the process to generate a dump.
+ DebugUtil::BreakDebugger();
+ }
+
+ return true;
+}
+
+} // namespace
+
+
+namespace net_instaweb {
+
+namespace log_message_handler {
+
+void Install(apr_pool_t* pool) {
+ log_pool = pool;
+ logging::SetLogMessageHandler(&LogMessageHandler);
+}
+
+void ShutDown() {
+ if (mod_pagespeed_version != NULL) {
+ delete mod_pagespeed_version;
+ mod_pagespeed_version = NULL;
+ }
+}
+
+// TODO(sligocki): This is not thread-safe, do we care? Error case is when
+// you have multiple server_rec's with different LogLevels which start-up
+// simultaniously. In which case we might get a non-min global LogLevel
+void AddServerConfig(const server_rec* server, const StringPiece& version) {
+ // TODO(sligocki): Maybe use ap_log_error(server) if there is exactly one
+ // server added?
+ log_level_cutoff = std::min(server->loglevel, log_level_cutoff);
+ if (mod_pagespeed_version == NULL) {
+ mod_pagespeed_version = new std::string(version.as_string());
+ } else {
+ *mod_pagespeed_version = version.as_string();
+ }
+}
+
+
+} // namespace log_message_handler
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/log_message_handler.h b/trunk/src/net/instaweb/apache/log_message_handler.h
new file mode 100644
index 0000000..92cb671
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/log_message_handler.h
@@ -0,0 +1,46 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#ifndef NET_INSTAWEB_APACHE_LOG_MESSAGE_HANDLER_H_
+#define NET_INSTAWEB_APACHE_LOG_MESSAGE_HANDLER_H_
+
+#include <algorithm> // for std::min
+#include "apr_pools.h"
+
+#include "net/instaweb/util/public/string_util.h"
+
+struct server_rec;
+
+namespace net_instaweb {
+
+namespace log_message_handler {
+
+// Install a log message handler that routes LOG() messages to the
+// apache error log. Should be called once at startup.
+void Install(apr_pool_t* pool);
+
+// The log_message_handler is not attached to a specific server_rec, so the
+// LogLevel is not automatically set for it. Every server_rec instance
+// should call AddServerConfig and let us decide what level to log at.
+// Currently we set it to the min LogLevel.
+void AddServerConfig(const server_rec* server, const StringPiece& version);
+
+// Free the memory from the log message handler
+void ShutDown();
+
+} // namespace log_message_handler
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_APACHE_LOG_MESSAGE_HANDLER_H_
diff --git a/trunk/src/net/instaweb/apache/mem_clean_up.cc b/trunk/src/net/instaweb/apache/mem_clean_up.cc
new file mode 100644
index 0000000..a3dadcc
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/mem_clean_up.cc
@@ -0,0 +1,39 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "googleurl/src/url_util.h"
+#include "net/instaweb/htmlparse/public/html_escape.h"
+#include "net/instaweb/rewriter/public/css_filter.h"
+#include "third_party/protobuf2/src/src/google/protobuf/stubs/common.h"
+
+// Clean up valgrind-based memory-leak checks by deleting statically allocated
+// data from various libraries. This must be called both from unit-tests
+// and from the Apache module, so that valgrind can be run on both of them.
+
+namespace {
+
+class MemCleanUp {
+ public:
+ ~MemCleanUp() {
+ net_instaweb::CssFilter::Terminate();
+ net_instaweb::HtmlEscape::ShutDown();
+ google::protobuf::ShutdownProtobufLibrary();
+ url_util::Shutdown();
+ }
+};
+MemCleanUp mem_clean_up;
+
+} // namespace
diff --git a/trunk/src/net/instaweb/apache/mem_debug.cc b/trunk/src/net/instaweb/apache/mem_debug.cc
new file mode 100644
index 0000000..a14ccf9
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/mem_debug.cc
@@ -0,0 +1,144 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Very simple memory debugging overrides for operator new/delete, to
+// help us quickly find simple memory violations:
+// 1. Double destruct
+// 2. Read before write (via scribbling)
+// 3. Read after delete (via scribbling)
+//
+// Note that valgrind does all of this much better, but is too slow to
+// run all the time, and it's not obvious how well it works when
+// Apache is forking processes that do all the interesting work. However,
+// if this code helps find a problem then you can run valgrind with Apache
+// in -X mode and that will be a better debug tool.
+//
+// Principle of operation: add 8 bytes to every allocation. The first
+// 4 bytes are a marker (kLiveMarker or kDeadMarker1). The next 4
+// bytes are used to store size of the allocation, which helps us
+// know how many bytes to scribble when we free.
+//
+// TODO(jmarantz): consider integrating a richer memory debug library such
+// as dmalloc, efence, or the heap-checking features of tcmalloc.
+//
+// Note: this memory debugging will interfere with Valgrind's ability to
+// detect read-before-write errors, and hence should be disabled if you
+// want to run with valgrind. This can be detected automatically using
+// macros from valgrind.h, but then that would *require* valgrind be installed
+// before building mod_pagespeed. See
+//
+// http://valgrind.org/docs/manual/manual-core-adv.html#manual-core-adv.clientreq
+
+// TODO(jmarantz): consider controlling inclusion of this module in the gyp
+// files rather than as an ifdef.
+//
+// Note: to be assured that these overrides are included in Debug builds but
+// are not included in Release builds, type:
+// nm out/Debug/libmod_pagespeed.so | /usr/bin/c++filt |grep 'operator new'
+// nm out/Release/libmod_pagespeed.so | /usr/bin/c++filt |grep 'operator new'
+#ifndef NDEBUG
+
+#include <stdlib.h>
+#include "base/basictypes.h"
+#include "base/logging.h"
+
+namespace {
+
+const int32 kLiveMarker = 0xfeedface; // first 4 bytes after alloc
+const int32 kDeadMarker1 = 0xabacabff; // first 4 bytes after free
+const int32 kDeadMarker2 = 0xdeadbeef; // overwrites the 'size' field on free
+const size_t kOverhead = 2 * sizeof(int32); // number of extra bytes to alloc
+
+size_t rounded_size(size_t size) {
+ if (size == 0) {
+ size = kOverhead;
+ } else if ((size % kOverhead) != 0) {
+ size = size + kOverhead - (size % kOverhead);
+ }
+ return size;
+}
+
+void scribble(void* ptr, size_t size, int32 scribble_word) {
+ CHECK_EQ(0U, size % sizeof(int32));
+ int num_ints = size / sizeof(int32);
+ int* p = static_cast<int*>(ptr);
+ for (int i = 0; i < num_ints; ++i, ++p) {
+ *p = scribble_word;
+ }
+}
+
+void* debug_malloc(size_t size) {
+ size_t rounded = rounded_size(size);
+ int32* marker = static_cast<int*>(malloc(rounded + kOverhead));
+ CHECK(marker != NULL);
+ marker[0] = kLiveMarker;
+ marker[1] = size;
+ int32* ret = marker + 2;
+ scribble(ret, rounded, kLiveMarker);
+ return reinterpret_cast<char*>(marker) + kOverhead;
+}
+
+void debug_free(void* ptr) {
+ if (ptr != NULL) {
+ char* alloced_ptr = static_cast<char*>(ptr) - kOverhead;
+ int32* marker = reinterpret_cast<int32*>(alloced_ptr);
+ scribble(ptr, rounded_size(marker[1]), kDeadMarker2);
+ CHECK_EQ(kLiveMarker, marker[0]);
+ marker[0] = kDeadMarker1;
+ marker[1] = kDeadMarker2;
+ free(marker);
+ }
+}
+
+} // namespace
+
+// C++ operator new/delete overrides.
+
+void* operator new(size_t size) throw (std::bad_alloc) {
+ return debug_malloc(size);
+}
+
+void* operator new(size_t size, const std::nothrow_t&) __THROW {
+ return debug_malloc(size);
+}
+
+void operator delete(void* ptr) __THROW {
+ debug_free(ptr);
+}
+
+void operator delete(void* ptr, const std::nothrow_t&) __THROW {
+ debug_free(ptr);
+}
+
+void* operator new[](size_t size) throw (std::bad_alloc) {
+ return debug_malloc(size);
+}
+
+void* operator new[](size_t size, const std::nothrow_t&) __THROW {
+ return debug_malloc(size);
+}
+
+void operator delete[](void* ptr) __THROW {
+ debug_free(ptr);
+}
+
+void operator delete[](void* ptr, const std::nothrow_t&) __THROW {
+ debug_free(ptr);
+}
+
+#endif // !NDEBUG
diff --git a/trunk/src/net/instaweb/apache/mod_instaweb.cc b/trunk/src/net/instaweb/apache/mod_instaweb.cc
new file mode 100644
index 0000000..53ca303
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/mod_instaweb.cc
@@ -0,0 +1,1101 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include <set>
+#include <string>
+
+#include "apr_strings.h"
+#include "apr_timer.h"
+#include "apr_version.h"
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/apache/apache_config.h"
+#include "net/instaweb/apache/header_util.h"
+#include "net/instaweb/apache/log_message_handler.h"
+#include "net/instaweb/apache/serf_url_async_fetcher.h"
+#include "net/instaweb/apache/instaweb_context.h"
+#include "net/instaweb/apache/instaweb_handler.h"
+#include "net/instaweb/apache/mod_instaweb.h"
+#include "net/instaweb/apache/apache_rewrite_driver_factory.h"
+#include "net/instaweb/apache/apr_statistics.h"
+#include "net/instaweb/public/version.h"
+#include "net/instaweb/public/global_constants.h"
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+#include "net/instaweb/rewriter/public/rewrite_options.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/query_params.h"
+#include "net/instaweb/util/public/stl_util.h"
+#include "net/instaweb/util/public/string_util.h"
+
+// Note: a very useful reference is this file, which demos many Apache module
+// options:
+// http://svn.apache.org/repos/asf/httpd/httpd/trunk/modules/examples/mod_example_hooks.c
+
+// The httpd header must be after the pagepseed_server_context.h. Otherwise,
+// the compiler will complain
+// "strtoul_is_not_a_portable_function_use_strtol_instead".
+#include "httpd.h"
+#include "http_config.h"
+#include "http_core.h"
+// When HAVE_SYSLOG, syslog.h is included and #defined LOG_*, which conflicts
+// with log_message_handler.
+#undef HAVE_SYSLOG
+#include "http_log.h"
+#include "http_protocol.h"
+#if USE_FIXUP_HOOK
+#include "http_request.h" // NOLINT
+#endif
+
+extern "C" {
+ extern module AP_MODULE_DECLARE_DATA pagespeed_module;
+}
+
+namespace net_instaweb {
+
+namespace {
+
+// Instaweb directive names -- these must match
+// install/common/pagespeed.conf.template.
+const char* kModPagespeed = "ModPagespeed";
+const char* kModPagespeedUrlPrefix = "ModPagespeedUrlPrefix";
+const char* kModPagespeedFetchProxy = "ModPagespeedFetchProxy";
+const char* kModPagespeedGeneratedFilePrefix =
+ "ModPagespeedGeneratedFilePrefix";
+const char* kModPagespeedFileCachePath = "ModPagespeedFileCachePath";
+const char* kModPagespeedFileCacheSizeKb = "ModPagespeedFileCacheSizeKb";
+const char* kModPagespeedFileCacheCleanIntervalMs
+ = "ModPagespeedFileCacheCleanIntervalMs";
+const char* kModPagespeedLRUCacheKbPerProcess =
+ "ModPagespeedLRUCacheKbPerProcess";
+const char* kModPagespeedLRUCacheByteLimit = "ModPagespeedLRUCacheByteLimit";
+const char* kModPagespeedFetcherTimeoutMs = "ModPagespeedFetcherTimeOutMs";
+const char* kModPagespeedNumShards = "ModPagespeedNumShards";
+const char* kModPagespeedCssOutlineMinBytes = "ModPagespeedCssOutlineMinBytes";
+const char* kModPagespeedJsOutlineMinBytes = "ModPagespeedJsOutlineMinBytes";
+const char* kModPagespeedFilters = "ModPagespeedFilters";
+const char* kModPagespeedRewriteLevel = "ModPagespeedRewriteLevel";
+const char* kModPagespeedEnableFilters = "ModPagespeedEnableFilters";
+const char* kModPagespeedDisableFilters = "ModPagespeedDisableFilters";
+const char* kModPagespeedSlurpDirectory = "ModPagespeedSlurpDirectory";
+const char* kModPagespeedSlurpReadOnly = "ModPagespeedSlurpReadOnly";
+const char* kModPagespeedSlurpFlushLimit = "ModPagespeedSlurpFlushLimit";
+const char* kModPagespeedForceCaching = "ModPagespeedForceCaching";
+const char* kModPagespeedCssInlineMaxBytes = "ModPagespeedCssInlineMaxBytes";
+const char* kModPagespeedImgInlineMaxBytes = "ModPagespeedImgInlineMaxBytes";
+const char* kModPagespeedImgMaxRewritesAtOnce =
+ "ModPagespeedImgMaxRewritesAtOnce";
+const char* kModPagespeedJsInlineMaxBytes = "ModPagespeedJsInlineMaxBytes";
+const char* kModPagespeedDomain = "ModPagespeedDomain";
+const char* kModPagespeedMapRewriteDomain = "ModPagespeedMapRewriteDomain";
+const char* kModPagespeedMapOriginDomain = "ModPagespeedMapOriginDomain";
+const char* kModPagespeedFilterName = "MOD_PAGESPEED_OUTPUT_FILTER";
+const char* kModPagespeedBeaconUrl = "ModPagespeedBeaconUrl";
+const char* kModPagespeedAllow = "ModPagespeedAllow";
+const char* kModPagespeedDisallow = "ModPagespeedDisallow";
+const char* kModPagespeedStatistics = "ModPagespeedStatistics";
+
+// TODO(jmarantz): determine the version-number from SVN at build time.
+const char kModPagespeedVersion[] = MOD_PAGESPEED_VERSION_STRING "-"
+ LASTCHANGE_STRING;
+
+enum RewriteOperation {REWRITE, FLUSH, FINISH};
+enum ConfigSwitch {CONFIG_ON, CONFIG_OFF, CONFIG_ERROR};
+
+// Check if pagespeed optimization rules applicable.
+bool check_pagespeed_applicable(request_rec* request,
+ const ContentType& content_type) {
+ // We can't operate on Content-Ranges.
+ if (apr_table_get(request->headers_out, "Content-Range") != NULL) {
+ ap_log_rerror(APLOG_MARK, APLOG_DEBUG, APR_SUCCESS, request,
+ "Content-Range is not available");
+ return false;
+ }
+
+ // Only rewrite HTML-like content.
+ if (!content_type.IsHtmlLike()) {
+ ap_log_rerror(APLOG_MARK, APLOG_DEBUG, APR_SUCCESS, request,
+ "Content-Type=%s Host=%s Uri=%s",
+ request->content_type, request->hostname,
+ request->unparsed_uri);
+ return false;
+ }
+
+ // mod_pagespeed often creates requests while rewriting an HTML. These
+ // requests are only intended to fetch resources (images, css, javascript) but
+ // in some circumstances they can end up fetching HTML. This HTML, if
+ // rewrittten, could in turn spawn more requests which could cascade into a
+ // bad situation. To mod_pagespeed, any fetched HTML is an error condition,
+ // so there's no reason to rewrite it anyway.
+ const char* user_agent = apr_table_get(request->headers_in,
+ HttpAttributes::kUserAgent);
+ // TODO(abliss): unify this string literal with the one in
+ // serf_url_async_fetcher.cc
+ if ((user_agent != NULL) && strstr(user_agent, "mod_pagespeed")) {
+ ap_log_rerror(APLOG_MARK, APLOG_INFO, APR_SUCCESS, request,
+ "Not rewriting mod_pagespeed's own fetch");
+ return false;
+ }
+
+ return true;
+}
+
+// Create a new bucket from buf using HtmlRewriter.
+// TODO(lsong): the content is copied multiple times. The buf is
+// copied/processed to string output, then output is copied to new bucket.
+apr_bucket* rewrite_html(InstawebContext* context, request_rec* request,
+ RewriteOperation operation, const char* buf, int len) {
+ if (context == NULL) {
+ LOG(DFATAL) << "Context is null";
+ return NULL;
+ }
+ if (buf != NULL) {
+ context->Rewrite(buf, len);
+ }
+ if (operation == REWRITE) {
+ return NULL;
+ } else if (operation == FLUSH) {
+ context->Flush();
+ } else if (operation == FINISH) {
+ context->Finish();
+ }
+
+ const std::string& output = context->output();
+ if (output.empty()) {
+ return NULL;
+ }
+ // Use the rewritten content. Create in heap since output will
+ // be emptied for reuse.
+ apr_bucket* bucket = apr_bucket_heap_create(
+ output.data(), output.size(), NULL,
+ request->connection->bucket_alloc);
+ context->clear();
+ return bucket;
+}
+
+// To support query-specific rewriter sets, scan the query parameters to
+// see whether we have any options we want to set. We will only allow
+// a limited number of options to be set. In particular, some options are
+// risky to set per query, such as image inline threshold, which exposes
+// a DOS vulnerability and a risk of poisoning our internal cache. Domain
+// adjustments can potentially introduce a security vulnerability.
+//
+// So we will check for explicit parameters we want to support.
+bool ScanQueryParamsForRewriterOptions(RewriteDriverFactory* factory,
+ const QueryParams& query_params,
+ RewriteOptions* options) {
+ MessageHandler* handler = factory->message_handler();
+ bool ret = true;
+ int option_count = 0;
+ for (int i = 0; i < query_params.size(); ++i) {
+ const char* name = query_params.name(i);
+ const char* value = query_params.value(i);
+ if (value == NULL) {
+ // Empty; all our options require a value, so skip. It might be a
+ // perfectly legitimate query param for the underlying page.
+ continue;
+ }
+ int64 int_val;
+ if (strcmp(name, kModPagespeed) == 0) {
+ bool is_on = (strcmp(value, "on") == 0);
+ if (is_on || (strcmp(value, "off") == 0)) {
+ options->set_enabled(is_on);
+ ++option_count;
+ } else {
+ // TODO(sligocki): Return 404s instead of logging server errors here
+ // and below.
+ handler->Message(kWarning, "Invalid value for %s: %s "
+ "(should be on or off)", name, value);
+ ret = false;
+ }
+ } else if (strcmp(name, kModPagespeedFilters) == 0) {
+ // When using ModPagespeedFilters query param, only the
+ // specified filters should be enabled.
+ options->SetRewriteLevel(RewriteOptions::kPassThrough);
+ if (options->EnableFiltersByCommaSeparatedList(value, handler)) {
+ ++option_count;
+ } else {
+ handler->Message(kWarning,
+ "Invalid filter name in %s: %s", name, value);
+ ret = false;
+ }
+ // TODO(jmarantz): add js inlinine threshold, outline threshold.
+ } else if (strcmp(name, kModPagespeedCssInlineMaxBytes) == 0) {
+ if (StringToInt64(value, &int_val)) {
+ options->set_css_inline_max_bytes(int_val);
+ ++option_count;
+ } else {
+ handler->Message(kWarning, "Invalid integer value for %s: %s",
+ name, value);
+ ret = false;
+ }
+ }
+ }
+ return ret && (option_count > 0);
+}
+
+// Apache's pool-based cleanup is not effective on process shutdown. To allow
+// valgrind to report clean results, we must take matters into our own hands.
+// We employ a statically allocated class object and rely on its destructor to
+// get a reliable cleanup hook. I am, in general, strongly opposed to this
+// sort of technique, and it violates the C++ Style Guide:
+// http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml
+// However, it is not possible to use valgrind to track memory leaks
+// in our Apache module without this approach.
+//
+// We also need this context hold any data needed for statistics
+// collected in advance of the creation of the Statistics object, such
+// as directives-parsing time.
+class ApacheProcessContext {
+ public:
+ ApacheProcessContext()
+ : merge_time_us_(NULL),
+ parse_time_us_(NULL),
+ html_rewrite_time_us_(NULL),
+ stored_parse_time_us_(0) {
+ }
+
+ ~ApacheProcessContext() {
+ STLDeleteElements(&factories_);
+ STLDeleteElements(&configs_);
+ statistics_.reset(NULL);
+ ApacheRewriteDriverFactory::Terminate();
+ log_message_handler::ShutDown();
+ }
+
+ // Delete the specified factory on process exit.
+ void add_factory(ApacheRewriteDriverFactory* factory) {
+ factories_.insert(factory);
+ }
+
+ // Do not delete the specified factory on process exit -- it
+ // is being deleted on a pool hook.
+ void remove_factory(ApacheRewriteDriverFactory* factory) {
+ factories_.erase(factory);
+ }
+
+ // Delete the specified config on process exit.
+ void add_config(ApacheConfig* config) {
+ configs_.insert(config);
+ }
+
+ // Do not delete the specified config on process exit -- it
+ // is being deleted on a pool hook.
+ void remove_config(ApacheConfig* config) {
+ configs_.erase(config);
+ }
+
+ AprStatistics* InitStatistics(const StringPiece& filename_prefix) {
+ if (statistics_.get() == NULL) {
+ statistics_.reset(new AprStatistics(filename_prefix));
+ RewriteDriverFactory::Initialize(statistics_.get());
+ SerfUrlAsyncFetcher::Initialize(statistics_.get());
+ statistics_->AddVariable("merge_time_us");
+ statistics_->AddVariable("parse_time_us");
+ statistics_->AddVariable("html_rewrite_time_us");
+ statistics_->InitVariables(true);
+ merge_time_us_ = statistics_->GetVariable("merge_time_us");
+ parse_time_us_ = statistics_->GetVariable("parse_time_us");
+ html_rewrite_time_us_ = statistics_->GetVariable("html_rewrite_time_us");
+ parse_time_us_->Add(stored_parse_time_us_);
+ stored_parse_time_us_ = 0;
+ statistics_->InitVariables(true);
+ }
+ return statistics_.get();
+ }
+
+ void AddMergeTimeUs(int64 merge_time_us) {
+ if (statistics_.get() != NULL) {
+ merge_time_us_->Add(merge_time_us);
+ }
+ }
+
+ void AddHtmlRewriteTimeUs(int64 rewrite_time_us) {
+ if (statistics_.get() != NULL) {
+ html_rewrite_time_us_->Add(rewrite_time_us);
+ }
+ }
+
+ // Accumulating the time spent parsing directives requires special handling,
+ // because the parsing of directives precedes the initialization of the
+ // statistics object, which cannot be created until the file_prefix setting
+ // is parsed.
+ //
+ // Thus we need a place to store the accumulated parsing time, so we
+ // store it hera in the ApacheProcessContext, which gets statically
+ // initialized.
+ void AddParseTimeUs(int64 parse_time_us) {
+ if (parse_time_us_ != NULL) {
+ parse_time_us_->Add(parse_time_us);
+ } else {
+ stored_parse_time_us_ += parse_time_us;
+ }
+ }
+
+ std::set<ApacheRewriteDriverFactory*> factories_;
+ std::set<ApacheConfig*> configs_;
+ scoped_ptr<AprStatistics> statistics_;
+ Variable* merge_time_us_;
+ Variable* parse_time_us_;
+ Variable* html_rewrite_time_us_;
+ int64 stored_parse_time_us_;
+};
+ApacheProcessContext apache_process_context;
+
+typedef void (ApacheProcessContext::*AddTimeFn)(int64 delta);
+
+class ScopedTimer {
+ public:
+ explicit ScopedTimer(AddTimeFn add_time_fn)
+ : add_time_fn_(add_time_fn),
+ start_time_us_(timer_.NowUs()) {
+ }
+
+ ~ScopedTimer() {
+ int64 delta_us = timer_.NowUs() - start_time_us_;
+ (apache_process_context.*add_time_fn_)(delta_us);
+ }
+
+ private:
+ AddTimeFn add_time_fn_;
+ AprTimer timer_;
+ int64 start_time_us_;
+};
+
+void MergeOptions(const RewriteOptions& a, const RewriteOptions& b,
+ RewriteOptions* out) {
+ ScopedTimer timer(&ApacheProcessContext::AddMergeTimeUs);
+ out->Merge(a, b);
+}
+
+// Builds a new context for an HTTP request, returning NULL if we decide
+// that we should not handle the request.
+InstawebContext* build_context_for_request(request_rec* request) {
+ ApacheConfig* config = static_cast<ApacheConfig*>
+ ap_get_module_config(request->per_dir_config, &pagespeed_module);
+ ApacheRewriteDriverFactory* factory = InstawebContext::Factory(
+ request->server);
+ scoped_ptr<RewriteOptions> custom_options;
+ const RewriteOptions* options = factory->options();
+ const RewriteOptions* config_options = config->options();
+ bool use_custom_options = false;
+
+ if (config_options->modified()) {
+ custom_options.reset(new RewriteOptions);
+ MergeOptions(*options, *config_options, custom_options.get());
+ options = custom_options.get();
+ use_custom_options = true;
+ }
+
+ if (request->unparsed_uri == NULL) {
+ // TODO(jmarantz): consider adding Debug message if unparsed_uri is NULL,
+ // possibly of request->the_request which was non-null in the case where
+ // I found this in the debugger.
+ return NULL;
+ }
+
+ ap_log_rerror(APLOG_MARK, APLOG_DEBUG, APR_SUCCESS, request,
+ "ModPagespeed OutputFilter called for request %s",
+ request->unparsed_uri);
+
+ QueryParams query_params;
+ if (request->parsed_uri.query != NULL) {
+ query_params.Parse(request->parsed_uri.query);
+ }
+
+ const ContentType* content_type =
+ MimeTypeToContentType(request->content_type);
+ if (content_type == NULL) {
+ return NULL;
+ }
+
+ // Check if pagespeed optimization is applicable.
+ if (!check_pagespeed_applicable(request, *content_type)) {
+ return NULL;
+ }
+
+ // Check if mod_instaweb has already rewritten the HTML. If the server is
+ // setup as both the original and the proxy server, mod_pagespeed filter may
+ // be applied twice. To avoid this, skip the content if it is already
+ // optimized by mod_pagespeed.
+ if (apr_table_get(request->headers_out, kModPagespeedHeader) != NULL) {
+ ap_log_rerror(APLOG_MARK, APLOG_DEBUG, APR_SUCCESS, request,
+ "URL %s already has been processed by mod_pagespeed",
+ request->unparsed_uri);
+ return NULL;
+ }
+
+ // Determine the absolute URL for this request, which might take on different
+ // forms in the request structure depending on whether this request comes
+ // from a browser proxy, or whether mod_proxy is enabled.
+ std::string absolute_url;
+ if (strncmp(request->unparsed_uri, "http://", 7) == 0) {
+ absolute_url = request->unparsed_uri;
+ } else {
+ absolute_url = ap_construct_url(request->pool, request->unparsed_uri,
+ request);
+ }
+ if ((request->filename != NULL) &&
+ (strncmp(request->filename, "proxy:", 6) == 0)) {
+ absolute_url.assign(request->filename + 6, strlen(request->filename) - 6);
+ }
+
+ RewriteOptions query_options;
+ query_options.SetDefaultRewriteLevel(RewriteOptions::kCoreFilters);
+ if (ScanQueryParamsForRewriterOptions(
+ factory, query_params, &query_options)) {
+ use_custom_options = true;
+ RewriteOptions* merged_options = new RewriteOptions;
+ MergeOptions(*options, query_options, merged_options);
+ custom_options.reset(merged_options);
+ options = merged_options;
+ }
+
+ // Is ModPagespeed turned off? We check after parsing query params so that
+ // they can override .conf settings.
+ if (!options->enabled()) {
+ return NULL;
+ }
+
+ // Do ModPagespeedDisallow restrict us from rewriting this URL?
+ if (!options->IsAllowed(absolute_url)) {
+ return NULL;
+ }
+
+ InstawebContext* context = new InstawebContext(
+ request, *content_type, factory, absolute_url,
+ use_custom_options, *options);
+
+ InstawebContext::ContentEncoding encoding =
+ context->content_encoding();
+ if ((encoding == InstawebContext::kGzip) ||
+ (encoding == InstawebContext::kDeflate)) {
+ // Unset the content encoding because the InstawebContext will decode the
+ // content before parsing.
+ apr_table_unset(request->headers_out, HttpAttributes::kContentEncoding);
+ apr_table_unset(request->err_headers_out, HttpAttributes::kContentEncoding);
+ } else if (encoding == InstawebContext::kOther) {
+ // We don't know the encoding, so we cannot rewrite the HTML.
+ return NULL;
+ }
+
+ apr_table_setn(request->headers_out, kModPagespeedHeader,
+ kModPagespeedVersion);
+ SetCacheControl(HttpAttributes::kNoCache, request);
+ SetupCacheRepair(HttpAttributes::kNoCache, request);
+
+ apr_table_unset(request->headers_out, HttpAttributes::kContentLength);
+ apr_table_unset(request->headers_out, "Content-MD5");
+ apr_table_unset(request->headers_out, HttpAttributes::kContentEncoding);
+
+ // Make sure compression is enabled for this response.
+ ap_add_output_filter("DEFLATE", NULL, request, request->connection);
+ return context;
+}
+
+// This returns 'false' if the output filter should stop its loop over
+// the brigade and return an error.
+bool process_bucket(ap_filter_t *filter, request_rec* request,
+ InstawebContext* context, apr_bucket* bucket,
+ apr_status_t* return_code) {
+ // Remove the bucket from the old brigade. We will create new bucket or
+ // reuse the bucket to insert into the new brigade.
+ APR_BUCKET_REMOVE(bucket);
+ *return_code = APR_SUCCESS;
+ apr_bucket_brigade* context_bucket_brigade = context->bucket_brigade();
+ apr_bucket* new_bucket = NULL;
+ if (!APR_BUCKET_IS_METADATA(bucket)) {
+ const char* buf = NULL;
+ size_t bytes = 0;
+ *return_code = apr_bucket_read(bucket, &buf, &bytes, APR_BLOCK_READ);
+ if (*return_code == APR_SUCCESS) {
+ new_bucket = rewrite_html(context, request, REWRITE, buf, bytes);
+ } else {
+ ap_log_rerror(APLOG_MARK, APLOG_ERR, *return_code, request,
+ "Reading bucket failed (rcode=%d)", *return_code);
+ apr_bucket_delete(bucket);
+ return false;
+ }
+ // Processed the bucket, now delete it.
+ apr_bucket_delete(bucket);
+ if (new_bucket != NULL) {
+ APR_BRIGADE_INSERT_TAIL(context_bucket_brigade, new_bucket);
+ }
+ } else if (APR_BUCKET_IS_EOS(bucket)) {
+ new_bucket = rewrite_html(context, request, FINISH, NULL, 0);
+ if (new_bucket != NULL) {
+ APR_BRIGADE_INSERT_TAIL(context_bucket_brigade, new_bucket);
+ }
+ // Insert the EOS bucket to the new brigade.
+ APR_BRIGADE_INSERT_TAIL(context_bucket_brigade, bucket);
+ // OK, we have seen the EOS. Time to pass it along down the chain.
+ *return_code = ap_pass_brigade(filter->next, context_bucket_brigade);
+ return false;
+ } else if (APR_BUCKET_IS_FLUSH(bucket)) {
+ new_bucket = rewrite_html(context, request, FLUSH, NULL, 0);
+ if (new_bucket != NULL) {
+ APR_BRIGADE_INSERT_TAIL(context_bucket_brigade, new_bucket);
+ }
+ APR_BRIGADE_INSERT_TAIL(context_bucket_brigade, bucket);
+ // OK, Time to flush, pass it along down the chain.
+ *return_code = ap_pass_brigade(filter->next, context_bucket_brigade);
+ if (*return_code != APR_SUCCESS) {
+ return false;
+ }
+ } else {
+ // TODO(lsong): remove this log.
+ ap_log_rerror(APLOG_MARK, APLOG_INFO, APR_SUCCESS, request,
+ "Unknown meta data");
+ APR_BRIGADE_INSERT_TAIL(context_bucket_brigade, bucket);
+ }
+ return true;
+}
+
+apr_status_t instaweb_out_filter(ap_filter_t *filter, apr_bucket_brigade *bb) {
+ ScopedTimer timer(&ApacheProcessContext::AddHtmlRewriteTimeUs);
+
+ // Do nothing if there is nothing, and stop passing to other filters.
+ if (APR_BRIGADE_EMPTY(bb)) {
+ return APR_SUCCESS;
+ }
+
+ request_rec* request = filter->r;
+ InstawebContext* context = static_cast<InstawebContext*>(filter->ctx);
+
+ // Initialize per-request context structure. Note that instaweb_out_filter
+ // may get called multiple times per HTTP request, and this occurs only
+ // on the first call.
+ if (context == NULL) {
+ context = build_context_for_request(request);
+ if (context == NULL) {
+ ap_remove_output_filter(filter);
+ return ap_pass_brigade(filter->next, bb);
+ }
+ filter->ctx = context;
+ }
+
+ apr_status_t return_code = APR_SUCCESS;
+ while (!APR_BRIGADE_EMPTY(bb)) {
+ apr_bucket* bucket = APR_BRIGADE_FIRST(bb);
+ if (!process_bucket(filter, request, context, bucket, &return_code)) {
+ return return_code;
+ }
+ }
+
+ apr_brigade_cleanup(bb);
+ return return_code;
+}
+
+void pagespeed_child_init(apr_pool_t* pool, server_rec* server) {
+ // Create PageSpeed context used by instaweb rewrite-driver. This is
+ // per-process, so we initialize all the server's context by iterating the
+ // server lists in server->next.
+ server_rec* next_server = server;
+ while (next_server) {
+ ApacheRewriteDriverFactory* factory = InstawebContext::Factory(next_server);
+ if (factory->statistics()) {
+ factory->statistics()->InitVariables(false);
+ }
+ next_server = next_server->next;
+ }
+}
+
+int pagespeed_post_config(apr_pool_t* pool, apr_pool_t* plog, apr_pool_t* ptemp,
+ server_rec *server_list) {
+ AprStatistics* statistics = NULL;
+
+ // This routine is complicated by the fact that statistics use inter-process
+ // mutexes and have static data, which co-mingles poorly with this otherwise
+ // re-entrant module. The situation that gets interesting is when there are
+ // multiple VirtualHosts, some of which have statistics enabled and some of
+ // which don't. We don't want the behavior to be order-dependent so we
+ // do multiple passes.
+ //
+ // TODO(jmarantz): test VirtualHost
+
+ // In the first pass, we see whether any of the servers have
+ // statistics enabled, if found, do the static initialization of
+ // statistics to establish global memory segments.
+ for (server_rec* server = server_list; server != NULL;
+ server = server->next) {
+ ApacheRewriteDriverFactory* factory = InstawebContext::Factory(server);
+ if (factory->options()->enabled()) {
+ if (factory->filename_prefix().empty() ||
+ factory->file_cache_path().empty()) {
+ std::string buf = StrCat(
+ "mod_pagespeed is enabled. "
+ "The following directives must not be NULL\n",
+ kModPagespeedFileCachePath, "=",
+ StrCat(
+ factory->file_cache_path(), "\n",
+ kModPagespeedGeneratedFilePrefix, "=",
+ factory->filename_prefix(), "\n"));
+ factory->message_handler()->Message(kError, "%s", buf.c_str());
+ return HTTP_INTERNAL_SERVER_ERROR;
+ }
+ if ((factory->statistics_enabled() && (statistics == NULL))) {
+ statistics = apache_process_context.InitStatistics(
+ factory->filename_prefix());
+ }
+ }
+ }
+
+ // Next we do the instance-independent static initialization, once we have
+ // established whether *any* of the servers of stats enabled.
+ RewriteDriverFactory::Initialize(statistics);
+ SerfUrlAsyncFetcher::Initialize(statistics);
+
+ // Do a final pass over the servers and init the server-specific statistics.
+ for (server_rec* server = server_list; server != NULL;
+ server = server->next) {
+ ApacheRewriteDriverFactory* factory = InstawebContext::Factory(server);
+ factory->set_statistics(factory->statistics_enabled() ? statistics : NULL);
+ }
+ return OK;
+}
+
+// Here log transaction will wait for all the asynchronous resource fetchers to
+// finish.
+apr_status_t pagespeed_log_transaction(request_rec *request) {
+ server_rec* server = request->server;
+ ApacheRewriteDriverFactory* factory = InstawebContext::Factory(server);
+ if (factory == NULL) {
+ return DECLINED;
+ }
+ return DECLINED;
+}
+
+// This function is a callback and it declares what
+// other functions should be called for request
+// processing and configuration requests. This
+// callback function declares the Handlers for
+// other events.
+void mod_pagespeed_register_hooks(apr_pool_t *pool) {
+ // Enable logging using pagespeed style
+ log_message_handler::Install(pool);
+
+ // Use instaweb to handle generated resources.
+ ap_hook_handler(instaweb_handler, NULL, NULL, -1);
+ ap_register_output_filter(
+ kModPagespeedFilterName, instaweb_out_filter, NULL, AP_FTYPE_RESOURCE);
+ // We need our repair headers filter to run after mod_headers. The
+ // mod_headers, which is the filter that is used to add the cache settings, is
+ // AP_FTYPE_CONTENT_SET. Using (AP_FTYPE_CONTENT_SET + 2) to make sure that we
+ // run after mod_headers.
+ ap_register_output_filter(
+ InstawebContext::kRepairHeadersFilterName, repair_caching_header, NULL,
+ static_cast<ap_filter_type>(AP_FTYPE_CONTENT_SET + 2));
+ ap_hook_post_config(pagespeed_post_config, NULL, NULL, APR_HOOK_MIDDLE);
+ ap_hook_child_init(pagespeed_child_init, NULL, NULL, APR_HOOK_LAST);
+ ap_hook_log_transaction(pagespeed_log_transaction, NULL, NULL, APR_HOOK_LAST);
+}
+
+apr_status_t pagespeed_child_exit(void* data) {
+ ApacheRewriteDriverFactory* factory =
+ static_cast<ApacheRewriteDriverFactory*>(data);
+ // avoid double-destructing from the cleanup handler on process exit
+ apache_process_context.remove_factory(factory);
+ delete factory;
+ return APR_SUCCESS;
+}
+
+void* mod_pagespeed_create_server_config(apr_pool_t* pool, server_rec* server) {
+ ApacheRewriteDriverFactory* factory = InstawebContext::Factory(server);
+ if (factory == NULL) {
+ factory = new ApacheRewriteDriverFactory(server, kModPagespeedVersion);
+
+ apr_pool_cleanup_register(pool, factory, pagespeed_child_exit,
+ apr_pool_cleanup_null);
+
+ // The pool-based cleanup hooks do not appear to be effective when exiting
+ // the process. The pagespeed_child_exit will *not* be called when the
+ // apache process is shut down. However, the static
+ // apache_process_context's destructor will be.
+ //
+ // This approach is needed to clean up our memory so that valgrind can
+ // report real memory leaks.
+ apache_process_context.add_factory(factory);
+ }
+ return factory;
+}
+
+template<class Options>
+const char* ParseBoolOption(Options* options, cmd_parms* cmd,
+ void (Options::*fn)(bool val),
+ const char* arg) {
+ const char* ret = NULL;
+ if (strcasecmp(arg, "on") == 0) {
+ (options->*fn)(true);
+ } else if (strcasecmp(arg, "off") == 0) {
+ (options->*fn)(false);
+ } else {
+ ret = apr_pstrcat(cmd->pool, cmd->directive->directive, " on|off", NULL);
+ }
+ return ret;
+}
+
+template<class Options>
+const char* ParseInt64Option(Options* options, cmd_parms* cmd,
+ void (Options::*fn)(int64 val),
+ const char* arg) {
+ int64 val;
+ const char* ret = NULL;
+ if (StringToInt64(arg, &val)) {
+ (options->*fn)(val);
+ } else {
+ ret = apr_pstrcat(cmd->pool, cmd->directive->directive,
+ " must specify a 64-bit integer", NULL);
+ }
+ return ret;
+}
+
+template<class Options>
+const char* ParseIntOption(Options* options, cmd_parms* cmd,
+ void (Options::*fn)(int val),
+ const char* arg) {
+ int val;
+ const char* ret = NULL;
+ if (StringToInt(arg, &val)) {
+ (options->*fn)(val);
+ } else {
+ ret = apr_pstrcat(cmd->pool, cmd->directive->directive,
+ " must specify a 32-bit integer", NULL);
+ }
+ return ret;
+}
+
+void warn_deprecated(cmd_parms* cmd, const char* remedy) {
+ ap_log_error(APLOG_MARK, APLOG_WARNING, APR_SUCCESS, cmd->server,
+ "%s is deprecated. %s",
+ cmd->directive->directive, remedy);
+}
+
+// Callback function that parses a single-argument directive. This is called
+// by the Apache config parser.
+static const char* ParseDirective(cmd_parms* cmd, void* data, const char* arg) {
+ ScopedTimer timer(&ApacheProcessContext::AddParseTimeUs);
+ ApacheRewriteDriverFactory* factory = InstawebContext::Factory(cmd->server);
+ MessageHandler* handler = factory->message_handler();
+ const char* directive = cmd->directive->directive;
+ const char* ret = NULL;
+ ApacheConfig* config = static_cast<ApacheConfig*>(data);
+ RewriteOptions* options = factory->options();
+ if (!config->description().empty()) {
+ options = config->options();
+ }
+
+ if (strcasecmp(directive, kModPagespeed) == 0) {
+ ret = ParseBoolOption(options, cmd,
+ &RewriteOptions::set_enabled, arg);
+ } else if (strcasecmp(directive, kModPagespeedUrlPrefix) == 0) {
+ warn_deprecated(cmd, "Please remove it from your configuration.");
+ } else if (strcasecmp(directive, kModPagespeedFetchProxy) == 0) {
+ factory->set_fetcher_proxy(arg);
+ } else if (strcasecmp(directive, kModPagespeedGeneratedFilePrefix) == 0) {
+ if (!factory->set_filename_prefix(arg)) {
+ ret = apr_pstrcat(cmd->pool, "Directory ", arg,
+ " does not exist and can't be created.", NULL);
+ }
+ } else if (strcasecmp(directive, kModPagespeedFileCachePath) == 0) {
+ factory->set_file_cache_path(arg);
+ } else if (strcasecmp(directive, kModPagespeedFileCacheSizeKb) == 0) {
+ ret = ParseInt64Option(factory,
+ cmd, &ApacheRewriteDriverFactory::set_file_cache_clean_size_kb, arg);
+ } else if (strcasecmp(directive,
+ kModPagespeedFileCacheCleanIntervalMs) == 0) {
+ ret = ParseInt64Option(factory,
+ cmd, &ApacheRewriteDriverFactory::set_file_cache_clean_interval_ms,
+ arg);
+ } else if (strcasecmp(directive, kModPagespeedFetcherTimeoutMs) == 0) {
+ ret = ParseInt64Option(factory,
+ cmd, &ApacheRewriteDriverFactory::set_fetcher_time_out_ms, arg);
+ } else if (strcasecmp(directive, kModPagespeedNumShards) == 0) {
+ warn_deprecated(cmd, "Please remove it from your configuration.");
+ } else if (strcasecmp(directive, kModPagespeedCssOutlineMinBytes) == 0) {
+ ret = ParseInt64Option(options,
+ cmd, &RewriteOptions::set_css_outline_min_bytes, arg);
+ } else if (strcasecmp(directive, kModPagespeedJsOutlineMinBytes) == 0) {
+ ret = ParseInt64Option(options,
+ cmd, &RewriteOptions::set_js_outline_min_bytes, arg);
+ } else if (strcasecmp(directive, kModPagespeedImgInlineMaxBytes) == 0) {
+ ret = ParseInt64Option(options,
+ cmd, &RewriteOptions::set_img_inline_max_bytes, arg);
+ } else if (strcasecmp(directive, kModPagespeedJsInlineMaxBytes) == 0) {
+ ret = ParseInt64Option(options,
+ cmd, &RewriteOptions::set_js_inline_max_bytes, arg);
+ } else if (strcasecmp(directive, kModPagespeedCssInlineMaxBytes) == 0) {
+ ret = ParseInt64Option(options,
+ cmd, &RewriteOptions::set_css_inline_max_bytes, arg);
+ } else if (strcasecmp(directive, kModPagespeedLRUCacheKbPerProcess) == 0) {
+ ret = ParseInt64Option(factory,
+ cmd, &ApacheRewriteDriverFactory::set_lru_cache_kb_per_process, arg);
+ } else if (strcasecmp(directive, kModPagespeedLRUCacheByteLimit) == 0) {
+ ret = ParseInt64Option(factory,
+ cmd, &ApacheRewriteDriverFactory::set_lru_cache_byte_limit, arg);
+ } else if (strcasecmp(directive, kModPagespeedImgMaxRewritesAtOnce) == 0) {
+ ret = ParseIntOption(options,
+ cmd, &RewriteOptions::set_img_max_rewrites_at_once, arg);
+ } else if (strcasecmp(directive, kModPagespeedEnableFilters) == 0) {
+ if (!options->EnableFiltersByCommaSeparatedList(arg, handler)) {
+ ret = "Failed to enable some filters.";
+ }
+ } else if (strcasecmp(directive, kModPagespeedDisableFilters) == 0) {
+ if (!options->DisableFiltersByCommaSeparatedList(arg, handler)) {
+ ret = "Failed to disable some filters.";
+ }
+ } else if (strcasecmp(directive, kModPagespeedRewriteLevel) == 0) {
+ RewriteOptions::RewriteLevel level = RewriteOptions::kPassThrough;
+ if (RewriteOptions::ParseRewriteLevel(arg, &level)) {
+ options->SetRewriteLevel(level);
+ } else {
+ ret = "Failed to parse RewriteLevel.";
+ }
+ } else if (strcasecmp(directive, kModPagespeedSlurpDirectory) == 0) {
+ factory->set_slurp_directory(arg);
+ } else if (strcasecmp(directive, kModPagespeedSlurpReadOnly) == 0) {
+ ret = ParseBoolOption(static_cast<RewriteDriverFactory*>(factory),
+ cmd, &ApacheRewriteDriverFactory::set_slurp_read_only, arg);
+ } else if (strcasecmp(directive, kModPagespeedSlurpFlushLimit) == 0) {
+ ret = ParseInt64Option(factory,
+ cmd, &ApacheRewriteDriverFactory::set_slurp_flush_limit, arg);
+ } else if (strcasecmp(directive, kModPagespeedForceCaching) == 0) {
+ ret = ParseBoolOption(static_cast<RewriteDriverFactory*>(factory),
+ cmd, &ApacheRewriteDriverFactory::set_force_caching, arg);
+ } else if (strcasecmp(directive, kModPagespeedBeaconUrl) == 0) {
+ options->set_beacon_url(arg);
+ } else if (strcasecmp(directive, kModPagespeedDomain) == 0) {
+ options->domain_lawyer()->AddDomain(arg, factory->message_handler());
+ } else if (strcasecmp(directive, kModPagespeedAllow) == 0) {
+ options->Allow(arg);
+ } else if (strcasecmp(directive, kModPagespeedDisallow) == 0) {
+ options->Disallow(arg);
+ } else if (strcasecmp(directive, kModPagespeedStatistics) == 0) {
+ ret = ParseBoolOption(factory, cmd,
+ &ApacheRewriteDriverFactory::set_statistics_enabled, arg);
+ } else {
+ return "Unknown directive.";
+ }
+
+ return ret;
+}
+
+// Callback function that parses a two-argument directive. This is called
+// by the Apache config parser.
+static const char* ParseDirective2(cmd_parms* cmd, void* data,
+ const char* arg1, const char* arg2) {
+ ScopedTimer timer(&ApacheProcessContext::AddParseTimeUs);
+ ApacheRewriteDriverFactory* factory = InstawebContext::Factory(cmd->server);
+ RewriteOptions* options = factory->options();
+ const char* directive = cmd->directive->directive;
+ const char* ret = NULL;
+ if (strcasecmp(directive, kModPagespeedMapRewriteDomain) == 0) {
+ options->domain_lawyer()->AddRewriteDomainMapping(
+ arg1, arg2, factory->message_handler());
+ } else if (strcasecmp(directive, kModPagespeedMapOriginDomain) == 0) {
+ options->domain_lawyer()->AddOriginDomainMapping(
+ arg1, arg2, factory->message_handler());
+ } else {
+ return "Unknown directive.";
+ }
+ return ret;
+}
+
+// Setting up Apache options is cumbersome for several reasons:
+//
+// 1. Apache appears to require the option table be entirely constructed
+// using static data. So we cannot use helper functions to create the
+// helper table, so that we can populate it from another table.
+// 2. You have to fill in the table with a function pointer with a K&R
+// C declaration that does not specify its argument types. There appears
+// to be a type-correct union hidden behind an ifdef for
+// AP_HAVE_DESIGNATED_INITIALIZER, but that doesn't work. It gives a
+// syntax error; its comments indicate it is there for Doxygen.
+// 3. Although you have to pre-declare all the options, you need to again
+// dispatch based on the name of the options. You could, conceivably,
+// provide a different function pointer for each call. This might look
+// feasible with the 'mconfig' argument to AP_INIT_TAKE1, but mconfig
+// must be specified in a static initializer. So it wouldn't be that easy
+// to, say, create a C++ object for each config parameter.
+//
+// Googling for AP_MODULE_DECLARE_DATA didn't shed any light on how to do this
+// using a style suitable for programming after 1980. So all we can do is make
+// this a little less ugly with wrapper macros and helper functions.
+//
+// TODO(jmarantz): investigate usage of RSRC_CONF -- perhaps many of these
+// options should be allowable inside a Directory or Location by ORing in
+// ACCESS_CONF to RSRC_CONF.
+
+#define APACHE_CONFIG_OPTION(name, help) \
+ AP_INIT_TAKE1(name, reinterpret_cast<const char*(*)()>(ParseDirective), \
+ NULL, RSRC_CONF, help)
+#define APACHE_CONFIG_DIR_OPTION(name, help) \
+ AP_INIT_TAKE1(name, reinterpret_cast<const char*(*)()>(ParseDirective), \
+ NULL, OR_ALL, help)
+
+// Like APACHE_CONFIG_OPTION, but gets 2 arguments.
+#define APACHE_CONFIG_DIR_OPTION2(name, help) \
+ AP_INIT_TAKE2(name, reinterpret_cast<const char*(*)()>(ParseDirective2), \
+ NULL, OR_ALL, help)
+
+static const command_rec mod_pagespeed_filter_cmds[] = {
+ APACHE_CONFIG_DIR_OPTION(kModPagespeed, "Enable instaweb"),
+ APACHE_CONFIG_OPTION(kModPagespeedUrlPrefix, "Set the url prefix"),
+ APACHE_CONFIG_OPTION(kModPagespeedFetchProxy, "Set the fetch proxy"),
+ APACHE_CONFIG_OPTION(kModPagespeedGeneratedFilePrefix,
+ "Set generated file's prefix"),
+ APACHE_CONFIG_OPTION(kModPagespeedFileCachePath,
+ "Set the path for file cache"),
+ APACHE_CONFIG_OPTION(kModPagespeedFileCacheSizeKb,
+ "Set the target size (in kilobytes) for file cache"),
+ APACHE_CONFIG_OPTION(kModPagespeedFileCacheCleanIntervalMs,
+ "Set the interval (in ms) for cleaning the file cache"),
+ APACHE_CONFIG_OPTION(kModPagespeedFetcherTimeoutMs,
+ "Set internal fetcher timeout in milliseconds"),
+ APACHE_CONFIG_OPTION(kModPagespeedNumShards, "Set number of shards"),
+ APACHE_CONFIG_OPTION(kModPagespeedLRUCacheKbPerProcess,
+ "Set the total size, in KB, of the per-process "
+ "in-memory LRU cache"),
+ APACHE_CONFIG_OPTION(kModPagespeedLRUCacheByteLimit,
+ "Set the maximum byte size entry to store in the per-process "
+ "in-memory LRU cache"),
+ APACHE_CONFIG_DIR_OPTION(kModPagespeedRewriteLevel,
+ "Base level of rewriting (PassThrough, CoreFilters)"),
+ APACHE_CONFIG_DIR_OPTION(kModPagespeedEnableFilters,
+ "Comma-separated list of enabled filters"),
+ APACHE_CONFIG_DIR_OPTION(kModPagespeedDisableFilters,
+ "Comma-separated list of disabled filters"),
+ APACHE_CONFIG_OPTION(kModPagespeedSlurpDirectory,
+ "Directory from which to read slurped resources"),
+ APACHE_CONFIG_OPTION(kModPagespeedSlurpReadOnly,
+ "Only read from the slurped directory, fail to fetch "
+ "URLs not already in the slurped directory"),
+ APACHE_CONFIG_OPTION(kModPagespeedSlurpFlushLimit,
+ "Set the maximum byte size for the slurped content to hold before "
+ "a flush"),
+ APACHE_CONFIG_OPTION(kModPagespeedForceCaching,
+ "Ignore HTTP cache headers and TTLs"),
+ APACHE_CONFIG_DIR_OPTION(kModPagespeedCssOutlineMinBytes,
+ "Number of bytes above which inline "
+ "CSS resources will be outlined."),
+ APACHE_CONFIG_DIR_OPTION(kModPagespeedJsOutlineMinBytes,
+ "Number of bytes above which inline "
+ "Javascript resources will be outlined."),
+ APACHE_CONFIG_DIR_OPTION(kModPagespeedImgInlineMaxBytes,
+ "Number of bytes below which images will be inlined."),
+ APACHE_CONFIG_OPTION(kModPagespeedImgMaxRewritesAtOnce,
+ "Set bound on number of images being rewritten at one time "
+ "(0 = unbounded)."),
+ APACHE_CONFIG_DIR_OPTION(kModPagespeedJsInlineMaxBytes,
+ "Number of bytes below which javascript will be inlined."),
+ APACHE_CONFIG_DIR_OPTION(kModPagespeedCssInlineMaxBytes,
+ "Number of bytes below which stylesheets will be inlined."),
+ APACHE_CONFIG_DIR_OPTION(kModPagespeedBeaconUrl, "URL for beacon callback"
+ " injected by add_instrumentation."),
+ APACHE_CONFIG_DIR_OPTION(kModPagespeedDomain,
+ "Authorize mod_pagespeed to rewrite resources in a domain."),
+ APACHE_CONFIG_DIR_OPTION2(kModPagespeedMapRewriteDomain,
+ "to_domain from_domain[,from_domain]*"),
+ APACHE_CONFIG_DIR_OPTION2(kModPagespeedMapOriginDomain,
+ "to_domain from_domain[,from_domain]*"),
+ APACHE_CONFIG_DIR_OPTION(kModPagespeedAllow,
+ "wildcard_spec for urls"),
+ APACHE_CONFIG_DIR_OPTION(kModPagespeedDisallow,
+ "wildcard_spec for urls"),
+ APACHE_CONFIG_DIR_OPTION(kModPagespeedStatistics,
+ "Whether to collect cross-process statistics."),
+ {NULL}
+};
+
+// We use pool-based cleanup for ApacheConfigs. This is 99% effective.
+// There is at least one base config which is created with create_dir_config,
+// but whose pool is never freed. To allow clean valgrind reports, we
+// must delete that config too. So we keep a backup cleanup-set for
+// configs at end-of-process, and keep that set up-to-date when the
+// pool deletion does work.
+apr_status_t delete_config(void* data) {
+ ApacheConfig* config = static_cast<ApacheConfig*>(data);
+ // avoid double-destructing from the cleanup handler on process exit
+ apache_process_context.remove_config(config);
+ delete config;
+ return APR_SUCCESS;
+}
+
+// Function to allow all modules to create per directory configuration
+// structures.
+// dir is the directory currently being processed.
+// Returns the per-directory structure created.
+void* create_dir_config(apr_pool_t* pool, char* dir) {
+ ApacheConfig* config = new ApacheConfig(dir);
+ config->options()->SetDefaultRewriteLevel(RewriteOptions::kCoreFilters);
+ apache_process_context.add_config(config);
+ apr_pool_cleanup_register(pool, config, delete_config, apr_pool_cleanup_null);
+ apache_process_context.add_config(config);
+ return config;
+}
+
+// Function to allow all modules to merge the per directory configuration
+// structures for two directories.
+// base_conf is the directory structure created for the parent directory.
+// new_conf is the directory structure currently being processed.
+// This function returns the new per-directory structure created
+void* merge_dir_config(apr_pool_t* pool, void* base_conf, void* new_conf) {
+ ApacheConfig* dir1 = static_cast<ApacheConfig*>(base_conf);
+ ApacheConfig* dir2 = static_cast<ApacheConfig*>(new_conf);
+
+ // To make it easier to debug the merged configurations, we store
+ // the name of both input configurations as the description for
+ // the merged configuration.
+ ApacheConfig* dir3 = new ApacheConfig(StrCat(
+ "Combine(", dir1->description(), ", ", dir2->description(), ")"));
+ MergeOptions(*(dir1->options()), *(dir2->options()), dir3->options());
+ apr_pool_cleanup_register(pool, dir3, delete_config, apr_pool_cleanup_null);
+ apache_process_context.add_config(dir3);
+ return dir3;
+}
+
+} // namespace
+
+} // namespace net_instaweb
+
+extern "C" {
+// Export our module so Apache is able to load us.
+// See http://gcc.gnu.org/wiki/Visibility for more information.
+#if defined(__linux)
+#pragma GCC visibility push(default)
+#endif
+
+// Declare and populate the module's data structure. The
+// name of this structure ('pagespeed_module') is important - it
+// must match the name of the module. This structure is the
+// only "glue" between the httpd core and the module.
+module AP_MODULE_DECLARE_DATA pagespeed_module = {
+ // Only one callback function is provided. Real
+ // modules will need to declare callback functions for
+ // server/directory configuration, configuration merging
+ // and other tasks.
+ STANDARD20_MODULE_STUFF,
+ net_instaweb::create_dir_config,
+ net_instaweb::merge_dir_config,
+ net_instaweb::mod_pagespeed_create_server_config,
+ NULL, // merge per-server config structures
+ net_instaweb::mod_pagespeed_filter_cmds,
+ net_instaweb::mod_pagespeed_register_hooks,
+};
+
+#if defined(__linux)
+#pragma GCC visibility pop
+#endif
+} // extern "C"
diff --git a/trunk/src/net/instaweb/apache/mod_instaweb.h b/trunk/src/net/instaweb/apache/mod_instaweb.h
new file mode 100644
index 0000000..2345e2c
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/mod_instaweb.h
@@ -0,0 +1,34 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+
+#ifndef MOD_INSTAWEB_MOD_INSTAWEB_H_
+#define MOD_INSTAWEB_MOD_INSTAWEB_H_
+
+#include "base/basictypes.h"
+
+// Forward declaration.
+struct server_rec;
+
+namespace net_instaweb {
+
+struct PageSpeedConfig;
+class PageSpeedServerContext;
+
+PageSpeedConfig* mod_pagespeed_get_server_config(server_rec* server);
+PageSpeedServerContext* mod_pagespeed_get_config_server_context(
+ server_rec* server);
+} // namespace net_instaweb
+
+#endif // MOD_INSTAWEB_MOD_INSTAWEB_H_
diff --git a/trunk/src/net/instaweb/apache/serf_async_callback.cc b/trunk/src/net/instaweb/apache/serf_async_callback.cc
new file mode 100644
index 0000000..462cd1a
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/serf_async_callback.cc
@@ -0,0 +1,91 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Author: jmarantz@google.com (Joshua Marantz)
+// lsong@google.com (Libo Song)
+
+#include "net/instaweb/apache/serf_async_callback.h"
+#include "net/instaweb/util/public/writer.h"
+
+namespace net_instaweb {
+
+namespace {
+
+class ProtectedWriter : public Writer {
+ public:
+ ProtectedWriter(SerfAsyncCallback* callback, Writer* orig_writer)
+ : callback_(callback),
+ orig_writer_(orig_writer) {
+ }
+
+ virtual bool Write(const StringPiece& buf, MessageHandler* handler) {
+ bool ret = true;
+
+ // If the callback has not timed out and been released, then pass
+ // the data through.
+ if (!callback_->released()) {
+ ret = orig_writer_->Write(buf, handler);
+ }
+ return ret;
+ }
+
+ virtual bool Flush(MessageHandler* handler) {
+ bool ret = true;
+
+ // If the callback has not timed out and been released, then pass
+ // the flush through.
+ if (!callback_->released()) {
+ ret = orig_writer_->Flush(handler);
+ }
+ return ret;
+ }
+
+ private:
+ SerfAsyncCallback* callback_;
+ Writer* orig_writer_;
+
+ DISALLOW_COPY_AND_ASSIGN(ProtectedWriter);
+};
+
+} // namespace
+
+SerfAsyncCallback::SerfAsyncCallback(MetaData* response_headers, Writer* writer)
+ : done_(false),
+ success_(false),
+ released_(false),
+ response_headers_(response_headers),
+ writer_(new ProtectedWriter(this, writer)) {
+}
+
+SerfAsyncCallback::~SerfAsyncCallback() {
+}
+
+void SerfAsyncCallback::Done(bool success) {
+ done_ = true;
+ success_ = success;
+ if (released_) {
+ delete this;
+ } else {
+ response_headers_->CopyFrom(response_headers_buffer_);
+ }
+}
+
+void SerfAsyncCallback::Release() {
+ released_ = true;
+ if (done_) {
+ delete this;
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/serf_async_callback.h b/trunk/src/net/instaweb/apache/serf_async_callback.h
new file mode 100644
index 0000000..ea62e27
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/serf_async_callback.h
@@ -0,0 +1,72 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Author: jmarantz@google.com (Joshua Marantz)
+// lsong@google.com (Libo Song)
+
+#ifndef NET_INSTAWEB_APACHE_SERF_ASYNC_CALLBACK_H_
+#define NET_INSTAWEB_APACHE_SERF_ASYNC_CALLBACK_H_
+
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+
+namespace net_instaweb {
+
+// Class to help run an asynchronous fetch synchronously with a timeout.
+class SerfAsyncCallback : public UrlAsyncFetcher::Callback {
+ public:
+ SerfAsyncCallback(MetaData* response_headers, Writer* writer);
+ virtual ~SerfAsyncCallback();
+ virtual void Done(bool success);
+
+ // When implementing a synchronous fetch with a timeout based on an
+ // underlying asynchronous mechanism, we need to ensure that we don't
+ // write to freed memory if the Done callback fires after the timeout.
+ //
+ // So we need to make sure the Writer and Response Buffers are owned
+ // by this Callback class, which will forward the output and headers
+ // to the caller *if* it has not been released by the time the callback
+ // is called.
+ MetaData* response_headers() { return &response_headers_buffer_; }
+ Writer* writer() { return writer_.get(); }
+
+ // When the 'owner' of this callback -- the code that calls 'new' --
+ // is done with it, it can call release. This will only delete the
+ // callback if Done() has been called. Otherwise it will stay around
+ // waiting for Done() to be called, and only then will it be deleted.
+ //
+ // When Release is called prior to Done(), the writer and response_headers
+ // will beo NULLed out in this structure so they will not be updated when
+ // Done() is finally called.
+ void Release();
+
+ bool done() const { return done_; }
+ bool success() const { return success_; }
+ bool released() const { return released_; }
+
+ private:
+ bool done_;
+ bool success_;
+ bool released_;
+ SimpleMetaData response_headers_buffer_;
+ MetaData* response_headers_;
+ scoped_ptr<Writer> writer_;
+
+ DISALLOW_COPY_AND_ASSIGN(SerfAsyncCallback);
+};
+
+} // namespace
+
+#endif // NET_INSTAWEB_APACHE_SERF_ASYNC_CALLBACK_H_
diff --git a/trunk/src/net/instaweb/apache/serf_url_async_fetcher.cc b/trunk/src/net/instaweb/apache/serf_url_async_fetcher.cc
new file mode 100644
index 0000000..940d520
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/serf_url_async_fetcher.cc
@@ -0,0 +1,878 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// TODO(jmarantz): Avoid initiating fetches for resources already in flight.
+// The challenge is that we would want to call all the callbacks that indicated
+// interest in a particular URL once the callback completed. Alternatively,
+// this could be done in a level above the URL fetcher.
+
+#include "net/instaweb/apache/serf_url_async_fetcher.h"
+
+#include <algorithm>
+#include <string>
+#include <vector>
+
+#include "apr_atomic.h"
+#include "apr_strings.h"
+#include "apr_thread_proc.h"
+#include "apr_version.h"
+#include "base/basictypes.h"
+#include "base/stl_util-inl.h"
+#include "net/instaweb/apache/apr_mutex.h"
+#include "net/instaweb/public/version.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/meta_data.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/statistics.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/timer.h"
+#include "net/instaweb/util/public/writer.h"
+#include "third_party/serf/src/serf.h"
+#include "third_party/serf/src/serf_bucket_util.h"
+
+// Until this fetcher has some mileage on it, it is useful to keep around
+// an easy way to turn on lots of debug messages. But they do get a bit chatty
+// when things are working well.
+#define SERF_DEBUG(x)
+
+namespace {
+const int kBufferSize = 2048;
+const char kFetchMethod[] = "GET";
+} // namespace
+
+extern "C" {
+ // Declares a new function added to
+ // src/third_party/serf/instaweb_context.c
+serf_bucket_t* serf_request_bucket_request_create_for_host(
+ serf_request_t *request,
+ const char *method,
+ const char *uri,
+ serf_bucket_t *body,
+ serf_bucket_alloc_t *allocator, const char* host);
+}
+
+namespace net_instaweb {
+
+const char SerfStats::kSerfFetchRequestCount[] = "serf_fetch_request_count";
+const char SerfStats::kSerfFetchByteCount[] = "serf_fetch_bytes_count";
+const char SerfStats::kSerfFetchTimeDurationMs[] =
+ "serf_fetch_time_duration_ms";
+const char SerfStats::kSerfFetchCancelCount[] = "serf_fetch_cancel_count";
+const char SerfStats::kSerfFetchOutstandingCount[] =
+ "serf_fetch_outstanding_count";
+const char SerfStats::kSerfFetchTimeoutCount[] = "serf_fetch_timeout_count";
+
+std::string GetAprErrorString(apr_status_t status) {
+ char error_str[1024];
+ apr_strerror(status, error_str, sizeof(error_str));
+ return error_str;
+}
+
+// TODO(lsong): Move this to a separate file. Necessary?
+class SerfFetch {
+ public:
+ // TODO(lsong): make use of request_headers.
+ SerfFetch(apr_pool_t* pool,
+ const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* fetched_content_writer,
+ MessageHandler* message_handler,
+ UrlAsyncFetcher::Callback* callback,
+ Timer* timer)
+ : fetcher_(NULL),
+ timer_(timer),
+ str_url_(url),
+ response_headers_(response_headers),
+ fetched_content_writer_(fetched_content_writer),
+ message_handler_(message_handler),
+ callback_(callback),
+ connection_(NULL),
+ byte_received_(0),
+ fetch_start_ms_(0),
+ fetch_end_ms_(0) {
+ request_headers_.CopyFrom(request_headers);
+ apr_pool_create(&pool_, pool);
+ bucket_alloc_ = serf_bucket_allocator_create(pool_, NULL, NULL);
+ }
+
+ ~SerfFetch() {
+ if (connection_ != NULL) {
+ serf_connection_close(connection_);
+ }
+ apr_pool_destroy(pool_);
+ }
+
+ // Start the fetch. It returns immediately. This can only be run when
+ // locked with fetcher->mutex_.
+ bool Start(SerfUrlAsyncFetcher* fetcher);
+
+ const char* str_url() { return str_url_.c_str(); }
+
+ // This must be called while holding SerfUrlAsyncFetcher's mutex_.
+ void Cancel() {
+ CallCallback(false);
+ }
+
+ // Calls the callback supplied by the user. This needs to happen
+ // exactly once. In some error cases it appears that Serf calls
+ // HandleResponse multiple times on the same object.
+ //
+ // This must be called while holding SerfUrlAsyncFetcher's mutex_.
+ void CallCallback(bool success) {
+ if (callback_ == NULL) {
+ LOG(INFO) << "Serf callback more than once on same fetch " << str_url()
+ << " (" << this << ")";
+ } else {
+ UrlAsyncFetcher::Callback* callback = callback_;
+ callback_ = NULL;
+ response_headers_ = NULL;
+ callback->Done(success);
+ fetch_end_ms_ = timer_->NowMs();
+ fetcher_->FetchComplete(this);
+ }
+ }
+
+ int64 TimeDuration() const {
+ if ((fetch_start_ms_ != 0) && (fetch_end_ms_ != 0)) {
+ return fetch_end_ms_ - fetch_start_ms_;
+ } else {
+ return 0;
+ }
+ }
+ int64 fetch_start_ms() const { return fetch_start_ms_; }
+
+ size_t byte_received() const { return byte_received_; }
+ MessageHandler* message_handler() { return message_handler_; }
+
+ private:
+
+ // Static functions used in callbacks.
+ static serf_bucket_t* ConnectionSetup(
+ apr_socket_t* socket, void* setup_baton, apr_pool_t* pool) {
+ SerfFetch* fetch = static_cast<SerfFetch*>(setup_baton);
+ return serf_bucket_socket_create(socket, fetch->bucket_alloc_);
+ }
+
+ static void ClosedConnection(serf_connection_t* conn,
+ void* closed_baton,
+ apr_status_t why,
+ apr_pool_t* pool) {
+ SerfFetch* fetch = static_cast<SerfFetch*>(closed_baton);
+ if (why != APR_SUCCESS) {
+ fetch->message_handler_->Warning(
+ fetch->str_url_.c_str(), 0, "Connection close (code=%d %s).",
+ why, GetAprErrorString(why).c_str());
+ }
+ // Connection is closed.
+ fetch->connection_ = NULL;
+ }
+
+ static serf_bucket_t* AcceptResponse(serf_request_t* request,
+ serf_bucket_t* stream,
+ void* acceptor_baton,
+ apr_pool_t* pool) {
+ // Get the per-request bucket allocator.
+ serf_bucket_alloc_t* bucket_alloc = serf_request_get_alloc(request);
+ // Create a barrier so the response doesn't eat us!
+ // From the comment in Serf:
+ // ### the stream does not have a barrier, this callback should generally
+ // ### add a barrier around the stream before incorporating it into a
+ // ### response bucket stack.
+ // ... i.e. the passed bucket becomes owned rather than
+ // ### borrowed.
+ serf_bucket_t* bucket = serf_bucket_barrier_create(stream, bucket_alloc);
+ return serf_bucket_response_create(bucket, bucket_alloc);
+ }
+
+ static apr_status_t HandleResponse(serf_request_t* request,
+ serf_bucket_t* response,
+ void* handler_baton,
+ apr_pool_t* pool) {
+ SerfFetch* fetch = static_cast<SerfFetch*>(handler_baton);
+ return fetch->HandleResponse(request, response);
+ }
+
+ // The handler MUST process data from the response bucket until the
+ // bucket's read function states it would block (APR_STATUS_IS_EAGAIN).
+ // The handler is invoked only when new data arrives. If no further data
+ // arrives, and the handler does not process all available data, then the
+ // system can result in a deadlock around the unprocessed, but read, data.
+ apr_status_t HandleResponse(serf_request_t* request,
+ serf_bucket_t* response) {
+ apr_status_t status = APR_EGENERAL;
+ if (response_headers_ == NULL) {
+ LOG(INFO) << "HandleResponse called on URL " << str_url()
+ << "(" << this << "), which is already erased";
+ return status;
+ }
+
+ serf_status_line status_line;
+ if ((response != NULL) &&
+ ((status = serf_bucket_response_status(response, &status_line))
+ == APR_SUCCESS)) {
+ response_headers_->SetStatusAndReason(
+ static_cast<HttpStatus::Code>(status_line.code));
+ response_headers_->set_major_version(status_line.version / 1000);
+ response_headers_->set_minor_version(status_line.version % 1000);
+ const char* data = NULL;
+ apr_size_t len = 0;
+ while ((status = serf_bucket_read(response, kBufferSize, &data, &len))
+ == APR_SUCCESS || APR_STATUS_IS_EOF(status) ||
+ APR_STATUS_IS_EAGAIN(status)) {
+ byte_received_ += len;
+ if (len > 0 &&
+ !fetched_content_writer_->Write(
+ StringPiece(data, len), message_handler_)) {
+ status = APR_EGENERAL;
+ break;
+ }
+ if (status != APR_SUCCESS) {
+ break;
+ }
+ }
+ // We could read the headers earlier, but then we have to check if we
+ // have received the headers. At EOF of response, we have the headers
+ // already. Read them.
+ if (APR_STATUS_IS_EOF(status)) {
+ status = ReadHeaders(response);
+ }
+ }
+ if (!APR_STATUS_IS_EAGAIN(status)) {
+ bool success = APR_STATUS_IS_EOF(status);
+ CallCallback(success);
+ }
+ return status;
+ }
+
+ apr_status_t ReadHeaders(serf_bucket_t* response) {
+ apr_status_t status = APR_SUCCESS;
+ serf_bucket_t* headers = serf_bucket_response_get_headers(response);
+ const char* data = NULL;
+ apr_size_t num_bytes = 0;
+ while ((status = serf_bucket_read(headers, kBufferSize, &data, &num_bytes))
+ == APR_SUCCESS || APR_STATUS_IS_EOF(status) ||
+ APR_STATUS_IS_EAGAIN(status)) {
+ if (response_headers_->headers_complete()) {
+ status = APR_EGENERAL;
+ message_handler_->Info(str_url_.c_str(), 0,
+ "headers complete but more data coming");
+ } else {
+ StringPiece str_piece(data, num_bytes);
+ apr_size_t parsed_len =
+ response_headers_->ParseChunk(str_piece, message_handler_);
+ if (parsed_len != num_bytes) {
+ status = APR_EGENERAL;
+ message_handler_->Error(str_url_.c_str(), 0,
+ "unexpected bytes at end of header");
+ }
+ }
+ if (status != APR_SUCCESS) {
+ break;
+ }
+ }
+ if (APR_STATUS_IS_EOF(status)
+ && !response_headers_->headers_complete()) {
+ message_handler_->Error(str_url_.c_str(), 0,
+ "eof on incomplete headers code=%d %s",
+ status, GetAprErrorString(status).c_str());
+ status = APR_EGENERAL;
+ }
+ return status;
+ }
+
+ // Ensures that a user-agent string is included, and that the mod_pagespeed
+ // version is appended.
+ void FixUserAgent() {
+ // Supply a default user-agent if none is present, and in any case
+ // append on a 'serf' suffix.
+ std::string user_agent;
+ CharStarVector v;
+ if (request_headers_.Lookup(HttpAttributes::kUserAgent, &v)) {
+ for (int i = 0, n = v.size(); i < n; ++i) {
+ if (i != 0) {
+ user_agent += " ";
+ }
+ user_agent += v[i];
+ }
+ request_headers_.RemoveAll(HttpAttributes::kUserAgent);
+ }
+ if (user_agent.empty()) {
+ user_agent += "Serf/" SERF_VERSION_STRING;
+ }
+ StringPiece version(" mod_pagespeed/" MOD_PAGESPEED_VERSION_STRING "-"
+ LASTCHANGE_STRING);
+ if (!StringPiece(user_agent).ends_with(version)) {
+ user_agent.append(version.data(), version.size());
+ }
+ request_headers_.Add(HttpAttributes::kUserAgent, user_agent);
+ }
+
+ static apr_status_t SetupRequest(serf_request_t* request,
+ void* setup_baton,
+ serf_bucket_t** req_bkt,
+ serf_response_acceptor_t* acceptor,
+ void** acceptor_baton,
+ serf_response_handler_t* handler,
+ void** handler_baton,
+ apr_pool_t* pool) {
+ SerfFetch* fetch = static_cast<SerfFetch*>(setup_baton);
+ const char* url_path = apr_uri_unparse(pool, &fetch->url_,
+ APR_URI_UNP_OMITSITEPART);
+
+ // If there is an explicit Host header, then override the
+ // host field in the Serf structure, as we will not be able
+ // to override it after it is created; only append to it.
+ //
+ // Serf automatically populates the Host field based on the
+ // URL, and provides no mechanism to override it, except
+ // by hacking source. We hacked source.
+ //
+ // See src/third_party/serf/src/instaweb_context.c
+ CharStarVector v;
+ const char* host = NULL;
+ if (fetch->request_headers_.Lookup(HttpAttributes::kHost, &v) &&
+ (v.size() == 1)) {
+ host = v[0];
+ }
+
+ fetch->FixUserAgent();
+
+ *req_bkt = serf_request_bucket_request_create_for_host(
+ request, kFetchMethod,
+ url_path, NULL,
+ serf_request_get_alloc(request), host);
+ serf_bucket_t* hdrs_bkt = serf_bucket_request_get_headers(*req_bkt);
+
+ for (int i = 0; i < fetch->request_headers_.NumAttributes(); ++i) {
+ const char* name = fetch->request_headers_.Name(i);
+ const char* value = fetch->request_headers_.Value(i);
+ if ((strcasecmp(name, HttpAttributes::kUserAgent) == 0) ||
+ (strcasecmp(name, HttpAttributes::kAcceptEncoding) == 0) ||
+ (strcasecmp(name, HttpAttributes::kReferer) == 0)) {
+ serf_bucket_headers_setn(hdrs_bkt, name, value);
+ }
+ }
+
+ // TODO(jmarantz): add accept-encoding:gzip even if not requested by
+ // the caller, but then decompress in the output handler.
+
+ *acceptor = SerfFetch::AcceptResponse;
+ *acceptor_baton = fetch;
+ *handler = SerfFetch::HandleResponse;
+ *handler_baton = fetch;
+ return APR_SUCCESS;
+ }
+
+ bool ParseUrl() {
+ apr_status_t status = 0;
+ status = apr_uri_parse(pool_, str_url_.c_str(), &url_);
+ if (status != APR_SUCCESS) {
+ return false; // Failed to parse URL.
+ }
+
+ // TODO(lsong): We do not handle HTTPS for now. HTTPS needs authentication
+ // verifying certificates, etc.
+ if (strcasecmp(url_.scheme, "https") == 0) {
+ return false;
+ }
+ if (!url_.port) {
+ url_.port = apr_uri_port_of_scheme(url_.scheme);
+ }
+ if (!url_.path) {
+ url_.path = apr_pstrdup(pool_, "/");
+ }
+ return true;
+ }
+
+ SerfUrlAsyncFetcher* fetcher_;
+ Timer* timer_;
+ const std::string str_url_;
+ SimpleMetaData request_headers_;
+ MetaData* response_headers_;
+ Writer* fetched_content_writer_;
+ MessageHandler* message_handler_;
+ UrlAsyncFetcher::Callback* callback_;
+
+ apr_pool_t* pool_;
+ serf_bucket_alloc_t* bucket_alloc_;
+ apr_uri_t url_;
+ serf_connection_t* connection_;
+ size_t byte_received_;
+ int64 fetch_start_ms_;
+ int64 fetch_end_ms_;
+
+
+ DISALLOW_COPY_AND_ASSIGN(SerfFetch);
+};
+
+class SerfThreadedFetcher : public SerfUrlAsyncFetcher {
+ public:
+ SerfThreadedFetcher(SerfUrlAsyncFetcher* parent, const char* proxy) :
+ SerfUrlAsyncFetcher(parent, proxy),
+ initiate_mutex_(pool_),
+ terminate_mutex_(pool_),
+ thread_done_(false) {
+ terminate_mutex_.Lock();
+ CHECK_EQ(APR_SUCCESS,
+ apr_thread_create(&thread_id_, NULL, SerfThreadFn, this, pool_));
+ }
+
+ ~SerfThreadedFetcher() {
+ // Although Cancel will be called in the base class destructor, we
+ // want to call it here as well, as it will make it easier for the
+ // thread to terminate.
+ CancelOutstandingFetches();
+ STLDeleteElements(&completed_fetches_);
+ STLDeleteElements(&initiate_fetches_);
+
+ // Let the thread terminate naturally by unlocking its mutexes.
+ thread_done_ = true;
+ mutex_->Unlock();
+ LOG(INFO) << "Waiting for threaded serf fetcher to terminate";
+ terminate_mutex_.Lock();
+ terminate_mutex_.Unlock();
+ }
+
+ // Called from mainline to queue up a fetch for the thread. If the
+ // thread is idle then we can unlock it.
+ void InitiateFetch(SerfFetch* fetch) {
+ ScopedMutex lock(&initiate_mutex_);
+ initiate_fetches_.push_back(fetch);
+ }
+
+ private:
+ static void* SerfThreadFn(apr_thread_t* thread_id, void* context) {
+ SerfThreadedFetcher* stc = static_cast<SerfThreadedFetcher*>(context);
+ CHECK_EQ(thread_id, stc->thread_id_);
+ stc->SerfThread();
+ return NULL;
+ }
+
+ // Thread-called function to transfer fetches from initiate_fetches_ vector to
+ // the active_fetches_ queue. Doesn't do any work if initiate_fetches_ is
+ // empty.
+ void TransferFetches() {
+ // Use a temp that to minimize the amount of time we hold the
+ // initiate_mutex_ lock, so that the parent thread doesn't get
+ // blocked trying to initiate fetches.
+ FetchVector xfer_fetches;
+ {
+ ScopedMutex lock(&initiate_mutex_);
+ xfer_fetches.swap(initiate_fetches_);
+ }
+
+ // Now that we've unblocked the parent thread, we can leisurely
+ // queue up the fetches, employing the proper lock for the active_fetches_
+ // set. Actually we expect we wll never have contention on this mutex
+ // from the thread.
+ if (!xfer_fetches.empty()) {
+ int num_started = 0;
+ ScopedMutex lock(mutex_);
+ for (int i = 0, n = xfer_fetches.size(); i < n; ++i) {
+ SerfFetch* fetch = xfer_fetches[i];
+ if (fetch->Start(this)) {
+ SERF_DEBUG(LOG(INFO) << "Adding threaded fetch to url "
+ << fetch->str_url()
+ << " (" << active_fetches_.size() << ")");
+ active_fetches_.push_back(fetch);
+ active_fetch_map_[fetch] = --active_fetches_.end();
+ ++num_started;
+ } else {
+ delete fetch;
+ }
+ }
+ if ((num_started != 0) && (outstanding_count_ != NULL)) {
+ outstanding_count_->Add(num_started);
+ }
+ }
+ }
+
+ void SerfThread() {
+ while (!thread_done_) {
+ // If initiate_fetches is empty, we will not do any work.
+ TransferFetches();
+
+ const int64 kPollIntervalUs = 500000;
+ SERF_DEBUG(LOG(INFO) << "Polling from serf thread (" << this << ")");
+ // If active_fetches_ is empty, we will not do any work.
+ int num_outstanding_fetches = Poll(kPollIntervalUs);
+ SERF_DEBUG(LOG(INFO) << "Finished polling from serf thread ("
+ << this << ")");
+ // We don't want to spin busily waiting for new fetches. We could use a
+ // semaphore, but we're not really concerned with latency here, so we can
+ // just check every once in a while.
+ if (num_outstanding_fetches == 0) {
+ sleep(1);
+ }
+ }
+ terminate_mutex_.Unlock();
+ }
+
+ apr_thread_t* thread_id_;
+
+ // protects initiate_fetches_
+ AprMutex initiate_mutex_;
+ // pushed in the main thread; popped in the serf thread.
+ std::vector<SerfFetch*> initiate_fetches_;
+
+ // Allows parent to block till thread exits
+ AprMutex terminate_mutex_;
+ bool thread_done_;
+
+ DISALLOW_COPY_AND_ASSIGN(SerfThreadedFetcher);
+};
+
+bool SerfFetch::Start(SerfUrlAsyncFetcher* fetcher) {
+ fetch_start_ms_ = timer_->NowMs();
+ fetcher_ = fetcher;
+ // Parse and validate the URL.
+ if (!ParseUrl()) {
+ return false;
+ }
+
+ apr_status_t status = serf_connection_create2(&connection_,
+ fetcher_->serf_context(),
+ url_,
+ ConnectionSetup, this,
+ ClosedConnection, this,
+ pool_);
+ if (status != APR_SUCCESS) {
+ message_handler_->Error(str_url_.c_str(), 0,
+ "Error status=%d (%s) serf_connection_create2",
+ status, GetAprErrorString(status).c_str());
+ return false;
+ }
+ serf_connection_request_create(connection_, SetupRequest, this);
+
+ // Start the fetch. It will connect to the remote host, send the request,
+ // and accept the response, without blocking.
+ status = serf_context_run(fetcher_->serf_context(), 0, fetcher_->pool());
+
+ if (status == APR_SUCCESS || APR_STATUS_IS_TIMEUP(status)) {
+ return true;
+ } else {
+ message_handler_->Error(str_url_.c_str(), 0,
+ "serf_context_run error status=%d (%s)",
+ status, GetAprErrorString(status).c_str());
+ return false;
+ }
+}
+
+
+// Set up the proxy for all the connections in the context. The proxy is in the
+// format of hostname:port.
+bool SerfUrlAsyncFetcher::SetupProxy(const char* proxy) {
+ apr_status_t status = 0;
+ if (proxy == NULL || *proxy == '\0') {
+ return true; // No proxy to be set.
+ }
+
+ apr_sockaddr_t* proxy_address = NULL;
+ apr_port_t proxy_port;
+ char* proxy_host;
+ char* proxy_scope;
+ status = apr_parse_addr_port(&proxy_host, &proxy_scope, &proxy_port, proxy,
+ pool_);
+ if (status != APR_SUCCESS || proxy_host == NULL || proxy_port == 0 ||
+ (status = apr_sockaddr_info_get(&proxy_address, proxy_host, APR_UNSPEC,
+ proxy_port, 0, pool_)) != APR_SUCCESS) {
+ return false;
+ }
+ serf_config_proxy(serf_context_, proxy_address);
+ return true;
+}
+
+SerfUrlAsyncFetcher::SerfUrlAsyncFetcher(const char* proxy, apr_pool_t* pool,
+ Statistics* statistics, Timer* timer,
+ int64 timeout_ms)
+ : pool_(pool),
+ timer_(timer),
+ mutex_(NULL),
+ serf_context_(NULL),
+ threaded_fetcher_(NULL),
+ outstanding_count_(NULL),
+ request_count_(NULL),
+ byte_count_(NULL),
+ time_duration_ms_(NULL),
+ cancel_count_(NULL),
+ timeout_count_(NULL),
+ timeout_ms_(timeout_ms) {
+ if (statistics != NULL) {
+ request_count_ =
+ statistics->GetVariable(SerfStats::kSerfFetchRequestCount);
+ byte_count_ = statistics->GetVariable(SerfStats::kSerfFetchByteCount);
+ time_duration_ms_ =
+ statistics->GetVariable(SerfStats::kSerfFetchTimeDurationMs);
+ cancel_count_ = statistics->GetVariable(SerfStats::kSerfFetchCancelCount);
+ outstanding_count_ = statistics->GetVariable(
+ SerfStats::kSerfFetchOutstandingCount);
+ timeout_count_ = statistics->GetVariable(
+ SerfStats::kSerfFetchTimeoutCount);
+ }
+ mutex_ = new AprMutex(pool_);
+ serf_context_ = serf_context_create(pool_);
+ threaded_fetcher_ = new SerfThreadedFetcher(this, proxy);
+ if (!SetupProxy(proxy)) {
+ LOG(WARNING) << "Proxy failed: " << proxy;
+ }
+}
+
+SerfUrlAsyncFetcher::SerfUrlAsyncFetcher(SerfUrlAsyncFetcher* parent,
+ const char* proxy)
+ : pool_(parent->pool_),
+ timer_(parent->timer_),
+ mutex_(NULL),
+ serf_context_(NULL),
+ threaded_fetcher_(NULL),
+ outstanding_count_(parent->outstanding_count_),
+ request_count_(parent->request_count_),
+ byte_count_(parent->byte_count_),
+ time_duration_ms_(parent->time_duration_ms_),
+ cancel_count_(parent->cancel_count_),
+ timeout_count_(parent->timeout_count_),
+ timeout_ms_(parent->timeout_ms()) {
+ mutex_ = new AprMutex(pool_);
+ serf_context_ = serf_context_create(pool_);
+ threaded_fetcher_ = NULL;
+ if (!SetupProxy(proxy)) {
+ LOG(WARNING) << "Proxy failed: " << proxy;
+ }
+}
+
+SerfUrlAsyncFetcher::~SerfUrlAsyncFetcher() {
+ CancelOutstandingFetches();
+ STLDeleteElements(&completed_fetches_);
+ int orphaned_fetches = active_fetches_.size();
+ if (orphaned_fetches != 0) {
+ LOG(ERROR) << "SerfFecher destructed with " << orphaned_fetches
+ << " orphaned fetches.";
+ if (outstanding_count_ != NULL) {
+ outstanding_count_->Add(-orphaned_fetches);
+ }
+ if (cancel_count_ != NULL) {
+ cancel_count_->Add(orphaned_fetches);
+ }
+ }
+
+ STLDeleteElements(&active_fetches_);
+ active_fetch_map_.clear();
+ if (threaded_fetcher_ != NULL) {
+ delete threaded_fetcher_;
+ }
+ delete mutex_;
+}
+
+void SerfUrlAsyncFetcher::CancelOutstandingFetches() {
+ // If there are still active requests, cancel them.
+ int num_canceled = 0;
+ {
+ ScopedMutex lock(mutex_);
+ while (!active_fetches_.empty()) {
+ FetchQueueEntry p = active_fetches_.begin();
+ SerfFetch* fetch = *p;
+ LOG(WARNING) << "Aborting fetch of " << fetch->str_url();
+ fetch->Cancel();
+ ++num_canceled;
+ }
+ }
+ if (num_canceled != 0) {
+ if (cancel_count_ != NULL) {
+ cancel_count_->Add(num_canceled);
+ }
+ if (outstanding_count_ != NULL) {
+ outstanding_count_->Add(-num_canceled);
+ }
+ }
+}
+
+bool SerfUrlAsyncFetcher::StreamingFetch(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* fetched_content_writer,
+ MessageHandler* message_handler,
+ UrlAsyncFetcher::Callback* callback) {
+ SerfFetch* fetch = new SerfFetch(
+ pool_, url, request_headers, response_headers, fetched_content_writer,
+ message_handler, callback, timer_);
+ if (request_count_ != NULL) {
+ request_count_->Add(1);
+ }
+ if (callback->EnableThreaded()) {
+ message_handler->Message(kInfo, "Initiating async fetch for %s",
+ url.c_str());
+ threaded_fetcher_->InitiateFetch(fetch);
+ } else {
+ message_handler->Message(kInfo, "Initiating blocking fetch for %s",
+ url.c_str());
+ bool started = false;
+ {
+ ScopedMutex mutex(mutex_);
+ started = fetch->Start(this);
+ if (started) {
+ active_fetches_.push_back(fetch);
+ active_fetch_map_[fetch] = --active_fetches_.end();
+ if (outstanding_count_ != NULL) {
+ outstanding_count_->Add(1);
+ }
+ } else {
+ delete fetch;
+ }
+ }
+ }
+ return false;
+}
+
+void SerfUrlAsyncFetcher::PrintOutstandingFetches(
+ MessageHandler* handler) const {
+ ScopedMutex mutex(mutex_);
+ for (FetchQueue::const_iterator p = active_fetches_.begin(),
+ e = active_fetches_.end(); p != e; ++p) {
+ SerfFetch* fetch = *p;
+ handler->Message(kInfo, "Outstanding fetch: %s",
+ fetch->str_url());
+ }
+}
+
+// If active_fetches_ is empty, this does no work and returns 0.
+int SerfUrlAsyncFetcher::Poll(int64 microseconds) {
+ // Run serf polling up to microseconds.
+ ScopedMutex mutex(mutex_);
+ if (!active_fetches_.empty()) {
+ apr_status_t status = serf_context_run(serf_context_, microseconds, pool_);
+ STLDeleteElements(&completed_fetches_);
+ if (APR_STATUS_IS_TIMEUP(status)) {
+ // Remove expired fetches from the front of the queue.
+ int64 stale_cutoff = timer_->NowMs() - timeout_ms_;
+ FetchQueueEntry p = active_fetches_.begin(), e = active_fetches_.end();
+ int timeouts = 0;
+ while ((p != e) && ((*p)->fetch_start_ms() < stale_cutoff)) {
+ SerfFetch* fetch = *p;
+ ++p;
+ LOG(WARNING) << "Fetch timed out: " << fetch->str_url();
+ timeouts++;
+ fetch->Cancel();
+ }
+ if ((timeouts > 0) && (timeout_count_ != NULL)) {
+ timeout_count_->Add(timeouts);
+ }
+ }
+ bool success = ((status == APR_SUCCESS) || APR_STATUS_IS_TIMEUP(status));
+ // TODO(jmarantz): provide the success status to the caller if there is a
+ // need.
+ if (!success && !active_fetches_.empty()) {
+ // TODO(jmarantz): I have a new theory that we are getting
+ // behind when our self-directed URL fetches queue up multiple
+ // requests for the same URL, which might be sending the Serf
+ // library into an n^2 situation with its polling, even though
+ // we are using an rb_tree to hold the outstanding fetches. We
+ // should fix this by keeping a map from url->SerfFetch, where
+ // we'd have to store lists of Callback*, ResponseHeader*, Writer* so
+ // all interested parties were updated if and when the fetch finally
+ // completed.
+ //
+ // In the meantime by putting more detail into the log here, we'll
+ // know whether we are accumulating outstanding fetches to make the
+ // server fall over.
+ LOG(ERROR) << "Serf status " << status << " ("
+ << GetAprErrorString(status) << " ) polling for "
+ << active_fetches_.size()
+ << ((threaded_fetcher_ == NULL) ? ": (threaded)"
+ : ": (non-blocking)")
+ << " (" << this << ") for " << microseconds/1.0e6
+ << " seconds";
+ }
+ }
+ return active_fetches_.size();
+}
+
+void SerfUrlAsyncFetcher::FetchComplete(SerfFetch* fetch) {
+ // We do not have a ScopedMutex in FetchComplete, because it is only
+ // called from Poll and CancelOutstandingFetches, which have ScopedMutexes.
+ // Note that SerfFetch::Cancel is currently not exposed from outside this
+ // class.
+ LOG(WARNING) << "FetchComplete(" << fetch->str_url() << ", " << fetch << ")";
+ FetchMapEntry map_entry = active_fetch_map_.find(fetch);
+ CHECK(map_entry != active_fetch_map_.end());
+ active_fetches_.erase(map_entry->second);
+ active_fetch_map_.erase(map_entry);
+ completed_fetches_.push_back(fetch);
+ fetch->message_handler()->Message(kInfo, "Fetch complete: %s",
+ fetch->str_url());
+ if (time_duration_ms_) {
+ time_duration_ms_->Add(fetch->TimeDuration());
+ }
+ if (byte_count_) {
+ byte_count_->Add(fetch->byte_received());
+ }
+ if (outstanding_count_) {
+ outstanding_count_->Add(-1);
+ }
+}
+
+size_t SerfUrlAsyncFetcher::NumActiveFetches() {
+ ScopedMutex lock(mutex_);
+ return active_fetches_.size();
+}
+
+bool SerfUrlAsyncFetcher::WaitForInProgressFetches(
+ int64 max_ms, MessageHandler* message_handler, WaitChoice wait_choice) {
+ bool ret = true;
+ if ((threaded_fetcher_ != NULL) && (wait_choice != kMainlineOnly)) {
+ ret &= threaded_fetcher_->WaitForInProgressFetchesHelper(
+ max_ms, message_handler);
+ }
+ if (wait_choice != kThreadedOnly) {
+ ret &= WaitForInProgressFetchesHelper(max_ms, message_handler);
+ }
+ return ret;
+}
+
+bool SerfUrlAsyncFetcher::WaitForInProgressFetchesHelper(
+ int64 max_ms, MessageHandler* message_handler) {
+ int num_active_fetches = NumActiveFetches();
+ if (num_active_fetches != 0) {
+ int64 now_ms = timer_->NowMs();
+ int64 end_ms = now_ms + max_ms;
+ while ((now_ms < end_ms) && (num_active_fetches != 0)) {
+ int64 remaining_ms = end_ms - now_ms;
+ SERF_DEBUG(LOG(INFO) << "Blocking process waiting " << remaining_ms
+ << "ms for " << active_fetches_.size() << " to complete");
+ SERF_DEBUG(PrintOutstandingFetches(message_handler));
+ Poll(1000 * remaining_ms);
+ now_ms = timer_->NowMs();
+ num_active_fetches = NumActiveFetches();
+ }
+ if (!active_fetches_.empty()) {
+ message_handler->Message(
+ kError, "Serf timeout waiting for %d to complete",
+ num_active_fetches);
+ return false;
+ }
+ SERF_DEBUG(LOG(INFO) << "Serf successfully completed outstanding fetches");
+ }
+ return true;
+}
+void SerfUrlAsyncFetcher::Initialize(Statistics* statistics) {
+ if (statistics != NULL) {
+ statistics->AddVariable(SerfStats::kSerfFetchRequestCount);
+ statistics->AddVariable(SerfStats::kSerfFetchByteCount);
+ statistics->AddVariable(SerfStats::kSerfFetchTimeDurationMs);
+ statistics->AddVariable(SerfStats::kSerfFetchCancelCount);
+ statistics->AddVariable(SerfStats::kSerfFetchOutstandingCount);
+ statistics->AddVariable(SerfStats::kSerfFetchTimeoutCount);
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/serf_url_async_fetcher.h b/trunk/src/net/instaweb/apache/serf_url_async_fetcher.h
new file mode 100644
index 0000000..99f9351
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/serf_url_async_fetcher.h
@@ -0,0 +1,132 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#ifndef NET_INSTAWEB_APACHE_SERF_URL_ASYNC_FETCHER_H_
+#define NET_INSTAWEB_APACHE_SERF_URL_ASYNC_FETCHER_H_
+
+#include <set>
+#include <string>
+#include <vector>
+#include <list>
+#include <map>
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+
+struct apr_pool_t;
+struct serf_context_t;
+struct apr_thread_mutex_t;
+
+namespace net_instaweb {
+
+class AprMutex;
+class Statistics;
+class SerfFetch;
+class SerfThreadedFetcher;
+class Timer;
+class Variable;
+
+struct SerfStats {
+ static const char kSerfFetchRequestCount[];
+ static const char kSerfFetchByteCount[];
+ static const char kSerfFetchTimeDurationMs[];
+ static const char kSerfFetchCancelCount[];
+ static const char kSerfFetchOutstandingCount[];
+ static const char kSerfFetchTimeoutCount[];
+};
+
+class SerfUrlAsyncFetcher : public UrlAsyncFetcher {
+ public:
+ SerfUrlAsyncFetcher(const char* proxy, apr_pool_t* pool,
+ Statistics* statistics, Timer* timer, int64 timeout_ms);
+ SerfUrlAsyncFetcher(SerfUrlAsyncFetcher* parent, const char* proxy);
+ virtual ~SerfUrlAsyncFetcher();
+ static void Initialize(Statistics* statistics);
+ virtual bool StreamingFetch(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* fetched_content_writer,
+ MessageHandler* message_handler,
+ UrlAsyncFetcher::Callback* callback);
+
+ // Poll the active fetches, returning the number of fetches
+ // still outstanding.
+ int Poll(int64 microseconds);
+
+ enum WaitChoice {
+ kThreadedOnly,
+ kMainlineOnly,
+ kThreadedAndMainline
+ };
+
+ bool WaitForInProgressFetches(int64 max_milliseconds,
+ MessageHandler* message_handler,
+ WaitChoice wait_choice);
+
+ // Remove the completed fetch from the active fetch set, and put it into a
+ // completed fetch list to be cleaned up.
+ void FetchComplete(SerfFetch* fetch);
+ apr_pool_t* pool() const { return pool_; }
+ serf_context_t* serf_context() const { return serf_context_; }
+
+ void PrintOutstandingFetches(MessageHandler* handler) const;
+ virtual int64 timeout_ms() { return timeout_ms_; }
+
+ protected:
+ typedef std::list<SerfFetch*> FetchQueue;
+ typedef std::list<SerfFetch*>::iterator FetchQueueEntry;
+ typedef std::map<SerfFetch*, FetchQueueEntry> FetchMap;
+ typedef std::map<SerfFetch*, FetchQueueEntry>::iterator FetchMapEntry;
+ bool SetupProxy(const char* proxy);
+ size_t NumActiveFetches();
+ void CancelOutstandingFetches();
+ bool WaitForInProgressFetchesHelper(int64 max_ms,
+ MessageHandler* message_handler);
+
+ apr_pool_t* pool_;
+ Timer* timer_;
+
+ // mutex_ protects serf_context_, active_fetches_, and active_fetch_map_.
+ AprMutex* mutex_;
+ serf_context_t* serf_context_;
+ FetchQueue active_fetches_;
+ FetchMap active_fetch_map_;
+
+ typedef std::vector<SerfFetch*> FetchVector;
+ FetchVector completed_fetches_;
+ SerfThreadedFetcher* threaded_fetcher_;
+
+ // This is protected because it's updated along with active_fetches_,
+ // which happens in subclass SerfThreadedFetcher as well as this class.
+ Variable* outstanding_count_;
+
+ private:
+ // Removes the given fetch from both active_fetch_queue_ and
+ // active_fetch_map_. You are expected to already be holding the appropriate
+ // locks. Does not actually delete anything. Returns a queue iterator
+ // pointing to the next fetch after this one in the queue.
+ Variable* request_count_;
+ Variable* byte_count_;
+ Variable* time_duration_ms_;
+ Variable* cancel_count_;
+ Variable* timeout_count_;
+ const int64 timeout_ms_;
+
+ DISALLOW_COPY_AND_ASSIGN(SerfUrlAsyncFetcher);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_APACHE_SERF_URL_ASYNC_FETCHER_H_
diff --git a/trunk/src/net/instaweb/apache/serf_url_async_fetcher_test.cc b/trunk/src/net/instaweb/apache/serf_url_async_fetcher_test.cc
new file mode 100644
index 0000000..b7cedc3
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/serf_url_async_fetcher_test.cc
@@ -0,0 +1,419 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "net/instaweb/apache/serf_url_async_fetcher.h"
+
+#include <algorithm>
+#include <string>
+#include <vector>
+#include "apr_atomic.h"
+#include "apr_pools.h"
+#include "apr_strings.h"
+#include "apr_version.h"
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "base/stl_util-inl.h"
+#include "net/instaweb/apache/apr_file_system.h"
+#include "net/instaweb/apache/apr_mutex.h"
+#include "net/instaweb/apache/apr_timer.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/gzip_inflater.h"
+#include "net/instaweb/util/public/mock_timer.h"
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/simple_stats.h"
+#include "testing/gtest/include/gtest/gtest.h"
+#include "third_party/serf/src/serf.h"
+
+namespace net_instaweb {
+
+namespace {
+const char kProxy[] = "";
+const int kMaxMs = 20000;
+const int kThreadedPollMs = 200;
+const int kWaitTimeoutMs = 5 * 1000;
+const int kTimerAdvanceMs = 10;
+const int kFetcherTimeoutMs = 5 * 1000;
+
+class SerfTestCallback : public UrlAsyncFetcher::Callback {
+ public:
+ explicit SerfTestCallback(AprMutex* mutex, const std::string& url)
+ : done_(false),
+ mutex_(mutex),
+ url_(url),
+ enable_threaded_(false),
+ success_(false) {
+ }
+ virtual ~SerfTestCallback() {}
+ virtual void Done(bool success) {
+ ScopedMutex lock(mutex_);
+ CHECK(!done_);
+ done_ = true;
+ success_ = success;
+ }
+ bool IsDone() const {
+ ScopedMutex lock(mutex_);
+ return done_;
+ }
+ virtual bool EnableThreaded() const {
+ return enable_threaded_;
+ }
+ void set_enable_threaded(bool b) { enable_threaded_ = b; }
+ bool success() const { return success_; }
+ private:
+ bool done_;
+ AprMutex* mutex_;
+ std::string url_;
+ bool enable_threaded_;
+ bool success_;
+
+ DISALLOW_COPY_AND_ASSIGN(SerfTestCallback);
+};
+
+} // namespace
+
+class SerfUrlAsyncFetcherTest: public ::testing::Test {
+ public:
+ static void SetUpTestCase() {
+ apr_initialize();
+ atexit(apr_terminate);
+ }
+
+ protected:
+ SerfUrlAsyncFetcherTest() { }
+
+ virtual void SetUp() {
+ apr_pool_create(&pool_, NULL);
+ timer_.reset(new MockTimer(MockTimer::kApr_5_2010_ms));
+ SerfUrlAsyncFetcher::Initialize(&statistics_);
+ serf_url_async_fetcher_.reset(
+ new SerfUrlAsyncFetcher(kProxy, pool_, &statistics_,
+ timer_.get(), kFetcherTimeoutMs));
+ mutex_ = new AprMutex(pool_);
+ AddTestUrl("http://www.google.com/", "<!doctype html>");
+ AddTestUrl("http://www.google.com/favicon.ico",
+ std::string("\000\000\001\000", 4));
+ AddTestUrl("http://www.google.com/intl/en_ALL/images/logo.gif", "GIF");
+ AddTestUrl("http://stevesouders.com/bin/resource.cgi?type=js&sleep=10",
+ "var");
+ prev_done_count = 0;
+ }
+
+ virtual void TearDown() {
+ // Need to free the fetcher before destroy the pool.
+ serf_url_async_fetcher_.reset(NULL);
+ timer_.reset(NULL);
+ STLDeleteElements(&request_headers_);
+ STLDeleteElements(&response_headers_);
+ STLDeleteElements(&contents_);
+ STLDeleteElements(&writers_);
+ STLDeleteElements(&callbacks_);
+ apr_pool_destroy(pool_);
+ delete mutex_;
+ }
+
+ void AddTestUrl(const std::string& url,
+ const std::string& content_start) {
+ urls_.push_back(url);
+ content_starts_.push_back(content_start);
+ request_headers_.push_back(new SimpleMetaData);
+ response_headers_.push_back(new SimpleMetaData);
+ contents_.push_back(new std::string);
+ writers_.push_back(new StringWriter(contents_.back()));
+ callbacks_.push_back(new SerfTestCallback(mutex_, url));
+ }
+
+ void StartFetches(size_t begin, size_t end, bool enable_threaded) {
+ for (size_t idx = begin; idx < end; ++idx) {
+ SerfTestCallback* callback = callbacks_[idx];
+ callback->set_enable_threaded(enable_threaded);
+ serf_url_async_fetcher_->StreamingFetch(
+ urls_[idx], *request_headers_[idx], response_headers_[idx],
+ writers_[idx], &message_handler_, callback);
+ }
+ }
+
+ int OutstandingFetches() {
+ return statistics_.GetVariable(SerfStats::kSerfFetchOutstandingCount)
+ ->Get();
+ }
+
+ int CountCompletedFetches(size_t begin, size_t end) {
+ int completed = 0;
+ for (size_t idx = begin; idx < end; ++idx) {
+ if (callbacks_[idx]->IsDone()) {
+ ++completed;
+ }
+ }
+ return completed;
+ }
+
+ void ValidateFetches(size_t begin, size_t end) {
+ for (size_t idx = begin; idx < end; ++idx) {
+ ASSERT_TRUE(callbacks_[idx]->IsDone());
+ EXPECT_LT(static_cast<size_t>(0), contents_[idx]->size());
+ EXPECT_EQ(200, response_headers_[idx]->status_code());
+ EXPECT_EQ(content_starts_[idx],
+ contents_[idx]->substr(0, content_starts_[idx].size()));
+ }
+ }
+
+ int WaitTillDone(size_t begin, size_t end, int64 delay_ms) {
+ AprTimer timer;
+ bool done = false;
+ int64 now_ms = timer.NowMs();
+ int64 end_ms = now_ms + delay_ms;
+ size_t done_count = 0;
+ while (!done && (now_ms < end_ms)) {
+ int64 remaining_ms = end_ms - now_ms;
+ serf_url_async_fetcher_->Poll(1000 * remaining_ms);
+ done_count = 0;
+ for (size_t idx = begin; idx < end; ++idx) {
+ if (callbacks_[idx]->IsDone()) {
+ ++done_count;
+ }
+ }
+ if (done_count != prev_done_count) {
+ prev_done_count = done_count;
+ done = (done_count == (end - begin));
+ }
+ now_ms = timer.NowMs();
+ }
+ return done_count;
+ }
+
+ int TestFetch(size_t begin, size_t end) {
+ StartFetches(begin, end, false);
+ timer_->advance_ms(kTimerAdvanceMs);
+ int done = WaitTillDone(begin, end, kMaxMs);
+ ValidateFetches(begin, end);
+ return (done == (end - begin));
+ }
+
+ // Valgrind will not allow the async-fetcher thread to run without a sleep.
+ void YieldToThread() {
+ usleep(1);
+ }
+
+ apr_pool_t* pool_;
+ std::vector<std::string> urls_;
+ std::vector<std::string> content_starts_;
+ std::vector<SimpleMetaData*> request_headers_;
+ std::vector<SimpleMetaData*> response_headers_;
+ std::vector<std::string*> contents_;
+ std::vector<StringWriter*> writers_;
+ std::vector<SerfTestCallback*> callbacks_;
+ // The fetcher to be tested.
+ scoped_ptr<SerfUrlAsyncFetcher> serf_url_async_fetcher_;
+ scoped_ptr<MockTimer> timer_;
+ SimpleStats statistics_; // TODO(jmarantz): make this thread-safe
+ GoogleMessageHandler message_handler_;
+ size_t prev_done_count;
+ AprMutex* mutex_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(SerfUrlAsyncFetcherTest);
+};
+
+TEST_F(SerfUrlAsyncFetcherTest, FetchOneURL) {
+ EXPECT_TRUE(TestFetch(0, 1));
+ EXPECT_FALSE(response_headers_[0]->IsGzipped());
+ int request_count =
+ statistics_.GetVariable(SerfStats::kSerfFetchRequestCount)->Get();
+ EXPECT_EQ(1, request_count);
+ int bytes_count =
+ statistics_.GetVariable(SerfStats::kSerfFetchByteCount)->Get();
+ // google.com is changing every time, check if we get a rough number.
+ EXPECT_LT(7500, bytes_count);
+ int time_duration =
+ statistics_.GetVariable(SerfStats::kSerfFetchTimeDurationMs)->Get();
+ EXPECT_EQ(kTimerAdvanceMs, time_duration);
+}
+
+TEST_F(SerfUrlAsyncFetcherTest, FetchOneURLGzipped) {
+ request_headers_[0]->Add(HttpAttributes::kAcceptEncoding,
+ HttpAttributes::kGzip);
+
+ // www.google.com doesn't respect our 'gzip' encoding request unless
+ // we have a reasonable user agent.
+ const char kDefaultUserAgent[] =
+ "Mozilla/5.0 (X11; U; Linux x86_64; en-US) "
+ "AppleWebKit/534.0 (KHTML, like Gecko) Chrome/6.0.408.1 Safari/534.0";
+
+ request_headers_[0]->Add(HttpAttributes::kUserAgent,
+ kDefaultUserAgent);
+ StartFetches(0, 1, false);
+ EXPECT_EQ(1, OutstandingFetches());
+ ASSERT_EQ(1, WaitTillDone(0, 1, kMaxMs));
+ ASSERT_TRUE(callbacks_[0]->IsDone());
+ EXPECT_LT(static_cast<size_t>(0), contents_[0]->size());
+ EXPECT_EQ(200, response_headers_[0]->status_code());
+ ASSERT_TRUE(response_headers_[0]->IsGzipped());
+
+ GzipInflater inflater(GzipInflater::kGzip);
+ ASSERT_TRUE(inflater.Init());
+ ASSERT_TRUE(inflater.SetInput(contents_[0]->data(), contents_[0]->size()));
+ ASSERT_TRUE(inflater.HasUnconsumedInput());
+ int size = content_starts_[0].size();
+ scoped_array<char> buf(new char[size]);
+ ASSERT_EQ(size, inflater.InflateBytes(buf.get(), size));
+ EXPECT_EQ(content_starts_[0], std::string(buf.get(), size));
+ EXPECT_EQ(0, OutstandingFetches());
+}
+
+TEST_F(SerfUrlAsyncFetcherTest, FetchTwoURLs) {
+ EXPECT_TRUE(TestFetch(1, 3));
+ int request_count =
+ statistics_.GetVariable(SerfStats::kSerfFetchRequestCount)->Get();
+ EXPECT_EQ(2, request_count);
+ int bytes_count =
+ statistics_.GetVariable(SerfStats::kSerfFetchByteCount)->Get();
+ // Maybe also need a rough number here. We will break if google's icon or logo
+ // changes.
+ EXPECT_EQ(9708, bytes_count);
+ int time_duration =
+ statistics_.GetVariable(SerfStats::kSerfFetchTimeDurationMs)->Get();
+ EXPECT_EQ(2 * kTimerAdvanceMs, time_duration);
+ EXPECT_EQ(0, OutstandingFetches());
+}
+
+TEST_F(SerfUrlAsyncFetcherTest, TestCancelThreeThreaded) {
+ StartFetches(0, 3, true);
+}
+
+TEST_F(SerfUrlAsyncFetcherTest, TestCancelOneThreadedTwoSync) {
+ StartFetches(0, 1, true);
+ StartFetches(1, 3, false);
+}
+
+TEST_F(SerfUrlAsyncFetcherTest, TestCancelTwoThreadedOneSync) {
+ StartFetches(0, 1, false),
+ StartFetches(1, 3, true);
+}
+
+TEST_F(SerfUrlAsyncFetcherTest, TestWaitThreeThreaded) {
+ StartFetches(0, 3, true);
+ serf_url_async_fetcher_->WaitForInProgressFetches(
+ kWaitTimeoutMs, &message_handler_,
+ SerfUrlAsyncFetcher::kThreadedOnly);
+ EXPECT_EQ(0, OutstandingFetches());
+}
+
+TEST_F(SerfUrlAsyncFetcherTest, TestThreeThreadedAsync) {
+ StartFetches(0, 1, true);
+ serf_url_async_fetcher_->WaitForInProgressFetches(
+ 10 /* milliseconds */, &message_handler_,
+ SerfUrlAsyncFetcher::kThreadedOnly);
+ StartFetches(1, 3, true);
+
+ // In this test case, we are not going to call the explicit threaded
+ // wait function, WaitForInProgressFetches. We have initiated async
+ // fetches and we are hoping they will complete within a certain amount
+ // of time. If the system is running well then we they will finish
+ // within a 100ms or so, so we'll loop in 50ms sleep intervals until
+ // we hit a max. We'll give it 5 seconds before declaring failure.
+ const int kMaxSeconds = 5;
+ const int kPollTimeUs = 50000;
+ const int kPollsPerSecond = 1000000 / kPollTimeUs;
+ const int kMaxIters = kMaxSeconds * kPollsPerSecond;
+ int completed = 0;
+ for (int i = 0; (completed < 3) && (i < kMaxIters); ++i) {
+ usleep(kPollTimeUs);
+ completed = CountCompletedFetches(0, 3);
+ }
+
+ // TODO(jmarantz): I have seen this test fail; then pass when it was
+ // run a second time. Find the flakiness and fix it.
+ // Value of: completed
+ // Actual: 0
+ // Expected: 3
+ //
+ // In the meantime, if this fails, re-running will help you determine whether
+ // this is due to your CL or not. It's possible this is associated with a
+ // recent change to the thread loop in serf_url_async_fetcher.cc to use
+ // sleep(1) rather than a mutex to keep from spinning when there is nothing
+ // to do. Maybe a little more than 5 seconds is now needed to complete 3
+ // async fetches.
+ ASSERT_EQ(3, completed) << "Async fetches times out before completing";
+ ValidateFetches(0, 3);
+ EXPECT_EQ(0, OutstandingFetches());
+}
+
+TEST_F(SerfUrlAsyncFetcherTest, TestWaitOneThreadedTwoSync) {
+ StartFetches(0, 1, true);
+ StartFetches(1, 3, false);
+ serf_url_async_fetcher_->WaitForInProgressFetches(
+ kWaitTimeoutMs, &message_handler_,
+ SerfUrlAsyncFetcher::kThreadedAndMainline);
+ EXPECT_EQ(0, OutstandingFetches());
+}
+
+TEST_F(SerfUrlAsyncFetcherTest, TestWaitTwoThreadedOneSync) {
+ StartFetches(0, 1, false),
+ StartFetches(1, 3, true);
+ serf_url_async_fetcher_->WaitForInProgressFetches(
+ kWaitTimeoutMs, &message_handler_,
+ SerfUrlAsyncFetcher::kThreadedAndMainline);
+ EXPECT_EQ(0, OutstandingFetches());
+}
+
+TEST_F(SerfUrlAsyncFetcherTest, TestThreeThreaded) {
+ StartFetches(0, 3, true);
+ int done = 0;
+ for (int i = 0; (done < 3) && (i < 100); ++i) {
+ YieldToThread();
+ done = WaitTillDone(0, 3, kThreadedPollMs);
+ }
+ EXPECT_EQ(3, done);
+ ValidateFetches(0, 3);
+}
+
+TEST_F(SerfUrlAsyncFetcherTest, TestOneThreadedTwoSync) {
+ StartFetches(0, 1, true);
+ StartFetches(1, 3, false);
+ int done = 0;
+ for (int i = 0; (done < 3) && (i < 100); ++i) {
+ YieldToThread();
+ done = WaitTillDone(0, 3, kThreadedPollMs);
+ }
+ EXPECT_EQ(3, done);
+ ValidateFetches(0, 3);
+}
+
+TEST_F(SerfUrlAsyncFetcherTest, TestTwoThreadedOneSync) {
+ StartFetches(0, 1, false);
+ StartFetches(1, 3, true);
+ int done = 0;
+ for (int i = 0; (done < 3) && (i < 100); ++i) {
+ YieldToThread();
+ done = WaitTillDone(0, 3, kThreadedPollMs);
+ }
+ EXPECT_EQ(3, done);
+ ValidateFetches(0, 3);
+}
+
+TEST_F(SerfUrlAsyncFetcherTest, TestTimeout) {
+ StartFetches(3, 4, false);
+ int timeouts =
+ statistics_.GetVariable(SerfStats::kSerfFetchTimeoutCount)->Get();
+ ASSERT_EQ(0, WaitTillDone(3, 4, kThreadedPollMs));
+ timer_->advance_ms(2 * kFetcherTimeoutMs);
+ ASSERT_EQ(1, WaitTillDone(3, 4, kThreadedPollMs));
+ ASSERT_TRUE(callbacks_[3]->IsDone());
+ EXPECT_FALSE(callbacks_[3]->success());
+ EXPECT_EQ(timeouts + 1,
+ statistics_.GetVariable(SerfStats::kSerfFetchTimeoutCount)->Get());
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/serf_url_fetcher.cc b/trunk/src/net/instaweb/apache/serf_url_fetcher.cc
new file mode 100644
index 0000000..b489923
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/serf_url_fetcher.cc
@@ -0,0 +1,71 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "net/instaweb/apache/serf_url_fetcher.h"
+
+#include <algorithm>
+#include "base/basictypes.h"
+#include "net/instaweb/apache/apr_timer.h"
+#include "net/instaweb/apache/serf_async_callback.h"
+
+namespace net_instaweb {
+
+SerfUrlFetcher::SerfUrlFetcher(int64 fetcher_timeout_ms,
+ SerfUrlAsyncFetcher* async_fetcher)
+ : fetcher_timeout_ms_(fetcher_timeout_ms),
+ async_fetcher_(async_fetcher) {
+}
+
+SerfUrlFetcher::~SerfUrlFetcher() {
+}
+
+bool SerfUrlFetcher::StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* fetched_content_writer,
+ MessageHandler* message_handler) {
+ SerfAsyncCallback* callback = new SerfAsyncCallback(
+ response_headers, fetched_content_writer);
+ async_fetcher_->StreamingFetch(
+ url, request_headers, callback->response_headers(),
+ callback->writer(), message_handler, callback);
+
+ // We are counting on the serf async fetcher implementing its own timeouts,
+ // using the same timeout that we have in this class. To avoid a race
+ // we double the timeout in the limit set here and CHECK that the
+ // callback got called by the time our timeout loop exits.
+ AprTimer timer;
+ int64 start_ms = timer.NowMs();
+ int64 now_ms = start_ms;
+ for (int64 end_ms = now_ms + 2 * fetcher_timeout_ms_;
+ !callback->done() && (now_ms < end_ms);
+ now_ms = timer.NowMs()) {
+ int64 remaining_us = std::max(static_cast<int64>(0),
+ 1000 * (end_ms - now_ms));
+ async_fetcher_->Poll(remaining_us);
+ }
+ bool ret = false;
+ if (!callback->done()) {
+ message_handler->Message(
+ kWarning,
+ "Async fetcher allowed %dms to expire without calling its callback",
+ static_cast<int>(now_ms - start_ms));
+ } else {
+ ret = callback->success();
+ }
+ callback->Release();
+ return ret;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/apache/serf_url_fetcher.h b/trunk/src/net/instaweb/apache/serf_url_fetcher.h
new file mode 100644
index 0000000..0512e33
--- /dev/null
+++ b/trunk/src/net/instaweb/apache/serf_url_fetcher.h
@@ -0,0 +1,45 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#ifndef NET_INSTAWEB_APACHE_SERF_URL_FETCHER_H_
+#define NET_INSTAWEB_APACHE_SERF_URL_FETCHER_H_
+
+#include <string>
+#include "base/basictypes.h"
+#include "net/instaweb/apache/serf_url_async_fetcher.h"
+#include "net/instaweb/util/public/url_fetcher.h"
+
+namespace net_instaweb {
+
+class SerfUrlFetcher : public UrlFetcher {
+ public:
+ SerfUrlFetcher(int64 fetcher_timeout_ms,
+ SerfUrlAsyncFetcher* async_fetcher);
+ virtual ~SerfUrlFetcher();
+ virtual bool StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* fetched_content_writer,
+ MessageHandler* message_handler);
+
+ private:
+ int64 fetcher_timeout_ms_;
+ SerfUrlAsyncFetcher* async_fetcher_;
+
+ DISALLOW_COPY_AND_ASSIGN(SerfUrlFetcher);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_APACHE_SERF_URL_FETCHER_H_
diff --git a/trunk/src/net/instaweb/htmlparse/doctype.cc b/trunk/src/net/instaweb/htmlparse/doctype.cc
new file mode 100644
index 0000000..634d4dd
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/doctype.cc
@@ -0,0 +1,99 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "net/instaweb/htmlparse/public/doctype.h"
+
+#include <vector>
+
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+const DocType DocType::kUnknown(DocType::UNKNOWN);
+const DocType DocType::kHTML5(DocType::HTML_5);
+const DocType DocType::kXHTML5(DocType::XHTML_5);
+const DocType DocType::kHTML4Strict(DocType::HTML_4_STRICT);
+const DocType DocType::kHTML4Transitional(DocType::HTML_4_TRANSITIONAL);
+const DocType DocType::kXHTML11(DocType::XHTML_1_1);
+const DocType DocType::kXHTML10Strict(DocType::XHTML_1_0_STRICT);
+const DocType DocType::kXHTML10Transitional(DocType::XHTML_1_0_TRANSITIONAL);
+
+bool DocType::IsXhtml() const {
+ switch (doctype_) {
+ case XHTML_5:
+ case XHTML_1_1:
+ case XHTML_1_0_STRICT:
+ case XHTML_1_0_TRANSITIONAL:
+ return true;
+ default:
+ return false;
+ }
+}
+
+bool DocType::IsVersion5() const {
+ switch (doctype_) {
+ case HTML_5:
+ case XHTML_5:
+ return true;
+ default:
+ return false;
+ }
+}
+
+bool DocType::Parse(const StringPiece& directive,
+ const ContentType& content_type) {
+ // Check if this is a doctype directive; don't bother parsing if it isn't.
+ if (!StringCaseStartsWith(directive, "doctype ")) {
+ return false;
+ }
+
+ // Parse the directive.
+ std::vector<std::string> parts;
+ ParseShellLikeString(directive, &parts);
+
+ // Sanity check:
+ DCHECK(parts.size() >= 1);
+ DCHECK(StringCaseEqual(parts[0], "doctype"));
+
+ // Check for known doctypes.
+ // See http://en.wikipedia.org/wiki/DOCTYPE
+ doctype_ = UNKNOWN;
+ if (parts.size() >= 2 && StringCaseEqual(parts[1], "html")) {
+ if (parts.size() == 2) {
+ if (content_type.IsXmlLike()) {
+ doctype_ = XHTML_5;
+ } else {
+ doctype_ = HTML_5;
+ }
+ } else if (parts.size() == 5 && StringCaseEqual(parts[2], "public")) {
+ if (parts[3] == "-//W3C//DTD HTML 4.01//EN") {
+ doctype_ = HTML_4_STRICT;
+ } else if (parts[3] == "-//W3C//DTD HTML 4.01 Transitional//EN") {
+ doctype_ = HTML_4_TRANSITIONAL;
+ } else if (parts[3] == "-//W3C//DTD XHTML 1.1//EN") {
+ doctype_ = XHTML_1_1;
+ } else if (parts[3] == "-//W3C//DTD XHTML 1.0 Strict//EN") {
+ doctype_ = XHTML_1_0_STRICT;
+ } else if (parts[3] == "-//W3C//DTD XHTML 1.0 Transitional//EN") {
+ doctype_ = XHTML_1_0_TRANSITIONAL;
+ }
+ }
+ }
+ return true;
+}
+
+} // net_instaweb
diff --git a/trunk/src/net/instaweb/htmlparse/empty_html_filter.cc b/trunk/src/net/instaweb/htmlparse/empty_html_filter.cc
new file mode 100644
index 0000000..da87896
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/empty_html_filter.cc
@@ -0,0 +1,59 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "public/empty_html_filter.h"
+
+namespace net_instaweb {
+
+EmptyHtmlFilter::EmptyHtmlFilter() {
+}
+
+EmptyHtmlFilter::~EmptyHtmlFilter() {
+}
+
+void EmptyHtmlFilter::StartDocument() {
+}
+
+void EmptyHtmlFilter::EndDocument() {
+}
+
+void EmptyHtmlFilter::StartElement(HtmlElement* element) {
+}
+
+void EmptyHtmlFilter::EndElement(HtmlElement* element) {
+}
+
+void EmptyHtmlFilter::Cdata(HtmlCdataNode* cdata) {
+}
+
+void EmptyHtmlFilter::Comment(HtmlCommentNode* comment) {
+}
+
+void EmptyHtmlFilter::IEDirective(HtmlIEDirectiveNode* directive) {
+}
+
+void EmptyHtmlFilter::Characters(HtmlCharactersNode* characters) {
+}
+
+void EmptyHtmlFilter::Directive(HtmlDirectiveNode* directive) {
+}
+
+void EmptyHtmlFilter::Flush() {
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/htmlparse/file_driver.cc b/trunk/src/net/instaweb/htmlparse/file_driver.cc
new file mode 100644
index 0000000..05d3034
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/file_driver.cc
@@ -0,0 +1,120 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/htmlparse/public/file_driver.h"
+
+#include "net/instaweb/htmlparse/public/file_statistics_log.h"
+#include "net/instaweb/htmlparse/public/html_writer_filter.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/file_writer.h"
+#include "net/instaweb/util/public/message_handler.h"
+
+namespace {
+
+bool GenerateFilename(
+ const char* extension, const bool keep_old_extension,
+ const char* infilename, std::string* outfilename) {
+ bool ret = false;
+ const char* dot = strrchr(infilename, '.');
+ if (dot != NULL) {
+ outfilename->clear();
+ int base_size = dot - infilename;
+ outfilename->append(infilename, base_size);
+ *outfilename += extension;
+ if (keep_old_extension) {
+ *outfilename += dot;
+ }
+ ret = true;
+ }
+ return ret;
+}
+
+} // namespace
+
+namespace net_instaweb {
+
+FileDriver::FileDriver(HtmlParse* html_parse, FileSystem* file_system)
+ : html_parse_(html_parse),
+ logging_filter_(),
+ html_write_filter_(html_parse_),
+ filters_added_(false),
+ file_system_(file_system) {
+}
+
+bool FileDriver::GenerateOutputFilename(
+ const char* infilename, std::string* outfilename) {
+ return GenerateFilename(".out", true, infilename, outfilename);
+}
+
+bool FileDriver::GenerateStatsFilename(
+ const char* infilename, std::string* outfilename) {
+ return GenerateFilename(".stats", false, infilename, outfilename);
+}
+
+bool FileDriver::ParseFile(const char* infilename,
+ const char* outfilename,
+ const char* statsfilename,
+ MessageHandler* message_handler) {
+ FileSystem::OutputFile* outf =
+ file_system_->OpenOutputFile(outfilename, message_handler);
+ bool ret = false;
+
+ if (outf != NULL) {
+ if (!filters_added_) {
+ filters_added_ = true;
+ html_parse_->AddFilter(&logging_filter_);
+ html_parse_->AddFilter(&html_write_filter_);
+ }
+ logging_filter_.Reset();
+ FileWriter file_writer(outf);
+ html_write_filter_.set_writer(&file_writer);
+ FileSystem::InputFile* f =
+ file_system_->OpenInputFile(infilename, message_handler);
+ if (f != NULL) {
+ // HtmlParser needs a valid HTTP URL to evaluate relative paths,
+ // so we create a dummy URL.
+ std::string dummy_url = StrCat("http://file.name/", infilename);
+ html_parse_->StartParseId(dummy_url, infilename, kContentTypeHtml);
+ char buf[1000];
+ int nread;
+ while ((nread = f->Read(buf, sizeof(buf), message_handler)) > 0) {
+ html_parse_->ParseText(buf, nread);
+ }
+ file_system_->Close(f, message_handler);
+ html_parse_->FinishParse();
+ ret = true;
+ if (statsfilename != NULL) {
+ FileSystem::OutputFile* statsfile =
+ file_system_->OpenOutputFile(statsfilename, message_handler);
+ if (statsfile != NULL) {
+ FileStatisticsLog statslog(statsfile, message_handler);
+ logging_filter_.LogStatistics(&statslog);
+ file_system_->Close(statsfile, message_handler);
+ } else {
+ ret = false;
+ }
+ }
+ }
+ file_system_->Close(outf, message_handler);
+ }
+
+ return ret;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/htmlparse/file_statistics_log.cc b/trunk/src/net/instaweb/htmlparse/file_statistics_log.cc
new file mode 100644
index 0000000..1a3fbb7
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/file_statistics_log.cc
@@ -0,0 +1,59 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/htmlparse/public/file_statistics_log.h"
+
+#include <stdio.h>
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+// TODO(jmarantz): convert to statistics interface
+
+namespace net_instaweb {
+
+FileStatisticsLog::FileStatisticsLog(FileSystem::OutputFile* file,
+ MessageHandler* message_handler)
+ : file_(file),
+ message_handler_(message_handler) {
+}
+
+FileStatisticsLog::~FileStatisticsLog() {
+}
+
+void FileStatisticsLog::LogStat(const char *stat_name, int value) {
+ // Buffer whole log entry before writing, in case there's interleaving going
+ // on (ie avoid multiple writes for single log entry)
+ std::string buf(stat_name);
+ buf += StrCat(": ", IntegerToString(value), "\n");
+ file_->Write(buf, message_handler_);
+}
+
+void FileStatisticsLog::LogDifference(const char *stat_name,
+ int value1, int value2) {
+ // Buffer whole log entry before writing, in case there's interleaving going
+ // on (ie avoid multiple writes for single log entry)
+ std::string buf(stat_name);
+ buf += StrCat(":\t", IntegerToString(value1), " vs\t",
+ IntegerToString(value2));
+ buf += StrCat("\tdiffer by\t", IntegerToString(value1 - value2), "\n");
+ file_->Write(buf, message_handler_);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/htmlparse/html_element.cc b/trunk/src/net/instaweb/htmlparse/html_element.cc
new file mode 100644
index 0000000..ebf47a0
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/html_element.cc
@@ -0,0 +1,217 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "public/html_element.h"
+
+#include <stdio.h>
+
+#include "net/instaweb/htmlparse/public/html_escape.h"
+#include "net/instaweb/htmlparse/html_event.h"
+#include <string>
+
+namespace net_instaweb {
+
+HtmlElement::HtmlElement(HtmlElement* parent, Atom tag,
+ const HtmlEventListIterator& begin, const HtmlEventListIterator& end)
+ : HtmlNode(parent),
+ sequence_(-1),
+ tag_(tag),
+ begin_(begin),
+ end_(end),
+ close_style_(AUTO_CLOSE),
+ begin_line_number_(-1),
+ end_line_number_(-1) {
+}
+
+HtmlElement::~HtmlElement() {
+ for (int i = 0, n = attribute_size(); i < n; ++i) {
+ delete attributes_[i];
+ }
+}
+
+void HtmlElement::SynthesizeEvents(const HtmlEventListIterator& iter,
+ HtmlEventList* queue) {
+ // We use -1 as a bogus line number, since these events are synthetic.
+ HtmlEvent* start_tag = new HtmlStartElementEvent(this, -1);
+ set_begin(queue->insert(iter, start_tag));
+ HtmlEvent* end_tag = new HtmlEndElementEvent(this, -1);
+ set_end(queue->insert(iter, end_tag));
+}
+
+void HtmlElement::InvalidateIterators(const HtmlEventListIterator& end) {
+ set_begin(end);
+ set_end(end);
+}
+
+void HtmlElement::DeleteAttribute(int i) {
+ std::vector<Attribute*>::iterator iter = attributes_.begin() + i;
+ delete *iter;
+ attributes_.erase(iter);
+}
+
+bool HtmlElement::DeleteAttribute(Atom name) {
+ for (int i = 0; i < attribute_size(); ++i) {
+ const Attribute* attribute = attributes_[i];
+ if (attribute->name() == name) {
+ DeleteAttribute(i);
+ return true;
+ }
+ }
+ return false;
+}
+
+const HtmlElement::Attribute* HtmlElement::FindAttribute(
+ const Atom name) const {
+ const Attribute* ret = NULL;
+ for (int i = 0; i < attribute_size(); ++i) {
+ const Attribute* attribute = attributes_[i];
+ if (attribute->name() == name) {
+ ret = attribute;
+ break;
+ }
+ }
+ return ret;
+}
+
+void HtmlElement::ToString(std::string* buf) const {
+ *buf += "<";
+ *buf += tag_.c_str();
+ for (int i = 0; i < attribute_size(); ++i) {
+ const Attribute& attribute = *attributes_[i];
+ *buf += ' ';
+ *buf += attribute.name().c_str();
+ if (attribute.value() != NULL) {
+ *buf += "=";
+ const char* quote = (attribute.quote() != NULL) ? attribute.quote() : "?";
+ *buf += quote;
+ *buf += attribute.value();
+ *buf += quote;
+ }
+ }
+ switch (close_style_) {
+ case AUTO_CLOSE: *buf += "> (not yet closed)"; break;
+ case IMPLICIT_CLOSE: *buf += ">"; break;
+ case EXPLICIT_CLOSE: *buf += "></";
+ *buf += tag_.c_str();
+ *buf += ">";
+ break;
+ case BRIEF_CLOSE: *buf += "/>"; break;
+ case UNCLOSED: *buf += "> (unclosed)"; break;
+ }
+ if ((begin_line_number_ != -1) || (end_line_number_ != -1)) {
+ *buf += " ";
+ if (begin_line_number_ != -1) {
+ *buf += IntegerToString(begin_line_number_);
+ }
+ *buf += "...";
+ if (end_line_number_ != -1) {
+ *buf += IntegerToString(end_line_number_);
+ }
+ }
+}
+
+void HtmlElement::DebugPrint() const {
+ std::string buf;
+ ToString(&buf);
+ fprintf(stdout, "%s\n", buf.c_str());
+}
+
+void HtmlElement::AddAttribute(const Attribute& src_attr) {
+ Attribute* attr = new Attribute(src_attr.name(), src_attr.value(),
+ src_attr.escaped_value(), src_attr.quote());
+ attributes_.push_back(attr);
+}
+
+void HtmlElement::AddAttribute(Atom name, const StringPiece& value,
+ const char* quote) {
+ std::string buf;
+ Attribute* attr = new Attribute(name, value, HtmlEscape::Escape(value, &buf),
+ quote);
+ attributes_.push_back(attr);
+}
+
+void HtmlElement::AddAttribute(Atom name, int value) {
+ std::string buf = IntegerToString(value);
+ // We include quotes here because XHTML requires them. If it later turns out
+ // we're in HTML, the remove_quotes filter can take them back out.
+ Attribute* attr = new Attribute(name, buf, buf, "\"");
+ attributes_.push_back(attr);
+}
+
+void HtmlElement::AddEscapedAttribute(Atom name,
+ const StringPiece& escaped_value,
+ const char* quote) {
+ std::string buf;
+ Attribute* attr = new Attribute(name,
+ HtmlEscape::Unescape(escaped_value, &buf),
+ escaped_value, quote);
+ attributes_.push_back(attr);
+}
+
+void HtmlElement::Attribute::CopyValue(const StringPiece& src,
+ scoped_array<char>* dst) {
+ if (src.data() == NULL) {
+ // This case indicates attribute without value <tag attr>, as opposed
+ // to data()=="", which implies an empty value <tag attr=>.
+ dst->reset(NULL);
+ } else {
+ char* buf = new char[src.size() + 1];
+ memcpy(buf, src.data(), src.size());
+ buf[src.size()] = '\0';
+ dst->reset(buf);
+ }
+}
+
+HtmlElement::Attribute::Attribute(Atom name, const StringPiece& value,
+ const StringPiece& escaped_value,
+ const char* quote)
+ : name_(name), quote_(quote) {
+ CopyValue(value, &value_);
+ CopyValue(escaped_value, &escaped_value_);
+}
+
+// Modify value of attribute (eg to rewrite dest of src or href).
+// As with the constructor, copies the string in, so caller retains
+// ownership of value.
+void HtmlElement::Attribute::SetValue(const StringPiece& value) {
+ std::string buf;
+ // Note that we execute the lines in this order in case value
+ // is a substring of value_. This copies the value just prior
+ // to deallocation of the old value_.
+ const char* escaped_chars = escaped_value_.get();
+ DCHECK(value.data() + value.size() < escaped_chars ||
+ escaped_chars + strlen(escaped_chars) < value.data())
+ << "Setting unescaped value from substring of escaped value.";
+ CopyValue(HtmlEscape::Escape(value, &buf), &escaped_value_);
+ CopyValue(value, &value_);
+}
+
+void HtmlElement::Attribute::SetEscapedValue(const StringPiece& escaped_value) {
+ std::string buf;
+ // Note that we execute the lines in this order in case value
+ // is a substring of value_. This copies the value just prior
+ // to deallocation of the old value_.
+ const char* value_chars = value_.get();
+ DCHECK(value_chars + strlen(value_chars) < escaped_value.data() ||
+ escaped_value.data() + escaped_value.size() < value_chars)
+ << "Setting escaped value from substring of unescaped value.";
+ CopyValue(HtmlEscape::Unescape(escaped_value, &buf), &value_);
+ CopyValue(escaped_value, &escaped_value_);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/htmlparse/html_escape.cc b/trunk/src/net/instaweb/htmlparse/html_escape.cc
new file mode 100644
index 0000000..7688a57
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/html_escape.cc
@@ -0,0 +1,308 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/htmlparse/public/html_escape.h"
+#include <set>
+
+namespace {
+
+struct HtmlEscapeSequence {
+ const char* sequence;
+ const unsigned char value[3];
+};
+
+// TODO(jmarantz): the multi-byte sequences are not working yet.
+static HtmlEscapeSequence kHtmlEscapeSequences[] = {
+ { "AElig", {0xC6, 0x0} },
+ { "Aacute", {0xC1, 0x0} },
+ { "Acirc", {0xC2, 0x0} },
+ { "Agrave", {0xC0, 0x0} },
+ { "Aring", {0xC5, 0x0} },
+ { "Atilde", {0xC3, 0x0} },
+ { "Auml", {0xC4, 0x0} },
+ { "Ccedil", {0xC7, 0x0} },
+ { "ETH", {0xD0, 0x0} },
+ { "Eacute", {0xC9, 0x0} },
+ { "Ecirc", {0xCA, 0x0} },
+ { "Egrave", {0xC8, 0x0} },
+ { "Euml", {0xCB, 0x0} },
+ { "Iacute", {0xCD, 0x0} },
+ { "Icirc", {0xCE, 0x0} },
+ { "Igrave", {0xCC, 0x0} },
+ { "Iuml", {0xCF, 0x0} },
+ { "Ntilde", {0xD1, 0x0} },
+ { "Oacute", {0xD3, 0x0} },
+ { "Ocirc", {0xD4, 0x0} },
+ { "Ograve", {0xD2, 0x0} },
+ { "Oslash", {0xD8, 0x0} },
+ { "Otilde", {0xD5, 0x0} },
+ { "Ouml", {0xD6, 0x0} },
+ { "THORN", {0xDE, 0x0} },
+ { "Uacute", {0xDA, 0x0} },
+ { "Ucirc", {0xDB, 0x0} },
+ { "Ugrave", {0xD9, 0x0} },
+ { "Uuml", {0xDC, 0x0} },
+ { "Yacute", {0xDD, 0x0} },
+ { "aacute", {0xE1, 0x0} },
+ { "acirc", {0xE2, 0x0} },
+ { "acute", {0xB4, 0x0} },
+ { "aelig", {0xE6, 0x0} },
+ { "agrave", {0xE0, 0x0} },
+ { "amp", {0x26, 0x0} },
+ { "aring", {0xE5, 0x0} },
+ { "atilde", {0xE3, 0x0} },
+ { "auml", {0xE4, 0x0} },
+ { "brvbar", {0xA6, 0x0} },
+ { "ccedil", {0xE7, 0x0} },
+ { "cedil", {0xB8, 0x0} },
+ { "cent", {0xA2, 0x0} },
+ { "copy", {0xA9, 0x0} },
+ { "curren", {0xA4, 0x0} },
+ { "deg", {0xB0, 0x0} },
+ { "divide", {0xF7, 0x0} },
+ { "eacute", {0xE9, 0x0} },
+ { "ecirc", {0xEA, 0x0} },
+ { "egrave", {0xE8, 0x0} },
+ { "eth", {0xF0, 0x0} },
+ { "euml", {0xEB, 0x0} },
+ { "frac12", {0xBD, 0x0} },
+ { "frac14", {0xBC, 0x0} },
+ { "frac34", {0xBE, 0x0} },
+ { "gt", {0x3E, 0x0} },
+ { "iacute", {0xED, 0x0} },
+ { "icirc", {0xEE, 0x0} },
+ { "iexcl", {0xA1, 0x0} },
+ { "igrave", {0xEC, 0x0} },
+ { "iquest", {0xBF, 0x0} },
+ { "iuml", {0xEF, 0x0} },
+ { "laquo", {0xAB, 0x0} },
+ { "lt", {0x3C, 0x0} },
+ { "macr", {0xAF, 0x0} },
+ { "micro", {0xB5, 0x0} },
+ { "middot", {0xB7, 0x0} },
+ { "nbsp", {0xA0, 0x0} },
+ { "not", {0xAC, 0x0} },
+ { "ntilde", {0xF1, 0x0} },
+ { "oacute", {0xF3, 0x0} },
+ { "ocirc", {0xF4, 0x0} },
+ { "ograve", {0xF2, 0x0} },
+ { "ordf", {0xAA, 0x0} },
+ { "ordm", {0xBA, 0x0} },
+ { "oslash", {0xF8, 0x0} },
+ { "otilde", {0xF5, 0x0} },
+ { "ouml", {0xF6, 0x0} },
+ { "para", {0xB6, 0x0} },
+ { "plusmn", {0xB1, 0x0} },
+ { "pound", {0xA3, 0x0} },
+ { "quot", {0x22, 0x0} },
+ { "raquo", {0xBB, 0x0} },
+ { "reg", {0xAE, 0x0} },
+ { "sect", {0xA7, 0x0} },
+ { "shy", {0xAD, 0x0} },
+ { "sup1", {0xB9, 0x0} },
+ { "sup2", {0xB2, 0x0} },
+ { "sup3", {0xB3, 0x0} },
+ { "szlig", {0xDF, 0x0} },
+ { "thorn", {0xFE, 0x0} },
+ { "times", {0xD7, 0x0} },
+ { "uacute", {0xFA, 0x0} },
+ { "ucirc", {0xFB, 0x0} },
+ { "ugrave", {0xF9, 0x0} },
+ { "uml", {0xA8, 0x0} },
+ { "uuml", {0xFC, 0x0} },
+ { "yacute", {0xFD, 0x0} },
+ { "yen", {0xA5, 0x0} },
+ { "yuml", {0xFF, 0x0} }
+};
+
+} // namespace
+
+namespace net_instaweb {
+
+HtmlEscape* HtmlEscape::singleton_ = NULL;
+
+HtmlEscape::HtmlEscape() {
+ typedef std::set<std::string, StringCompareInsensitive> CaseInsensitiveSet;
+ CaseInsensitiveSet case_sensitive_symbols;
+ for (size_t i = 0; i < arraysize(kHtmlEscapeSequences); ++i) {
+ // Put all symbols in the case-sensitive map
+ const HtmlEscapeSequence& seq = kHtmlEscapeSequences[i];
+ unescape_sensitive_map_[seq.sequence] =
+ reinterpret_cast<const char*>(seq.value);
+
+ // Don't populate the case-insensitive map for symbols that we've
+ // already determined are case-sensitive.
+ if (case_sensitive_symbols.find(seq.sequence) ==
+ case_sensitive_symbols.end()) {
+ // If this symbol is already present in the insensitive map, then it
+ // must be case-sensitive. E.g. Æ and æ are distinct.
+ StringStringMapInsensitive::iterator p =
+ unescape_insensitive_map_.find(seq.sequence);
+ if (p != unescape_insensitive_map_.end()) {
+ // As this symbol is case-sensitive, we must remove it from the
+ // case-insensitive map. This way we will report an error for
+ // &Aelig;, rather than Æ or æ unpredictably. Also,
+ // note that this symbol is case-sensitive, and therefore must
+ // be not be ent
+ unescape_insensitive_map_.erase(p);
+ case_sensitive_symbols.insert(seq.sequence);
+ } else {
+ unescape_insensitive_map_[seq.sequence] =
+ reinterpret_cast<const char*>(seq.value);
+ }
+
+ // For now, we will only generate symbolic escaped-names for
+ // single-byte sequences
+ if (strlen(reinterpret_cast<const char*>(seq.value)) == 1) {
+ escape_map_[reinterpret_cast<const char*>(seq.value)] = seq.sequence;
+ }
+ }
+ }
+}
+
+void HtmlEscape::Init() {
+ if (singleton_ == NULL) {
+ singleton_ = new HtmlEscape();
+ }
+}
+
+void HtmlEscape::ShutDown() {
+ if (singleton_ != NULL) {
+ delete singleton_;
+ singleton_ = NULL;
+ }
+}
+
+StringPiece HtmlEscape::UnescapeHelper(const StringPiece& escaped,
+ std::string* buf) const {
+ if (escaped.data() == NULL) {
+ return escaped;
+ }
+ buf->clear();
+
+ // Attribute values may have HTML escapes in them, e.g.
+ // href="host.com/path?v1&v2"
+ // Un-escape the attribute value here before populating the
+ // attribute data structure.
+ std::string escape;
+ int numeric_value = 0;
+ bool accumulate_numeric_code = false;
+ bool hex_mode = false;
+ bool in_escape = false;
+ for (size_t i = 0; i < escaped.size(); ++i) {
+ char ch = escaped[i];
+ bool bogus_escape = false;
+ if (!in_escape) {
+ if (ch == '&') {
+ in_escape = true;
+ escape.clear();
+ numeric_value = 0;
+ accumulate_numeric_code = false;
+ hex_mode = false;
+ } else {
+ *buf += ch;
+ }
+ } else if (escape.empty() && (ch == '#')) {
+ escape += ch;
+ accumulate_numeric_code = true;
+ if (((i + 1) < escaped.size()) && (toupper(escaped[i + 1]) == 'X')) {
+ hex_mode = true;
+ ++i;
+ }
+ } else if (ch == ';') {
+ if (accumulate_numeric_code && (escape.size() > 1)) {
+ *buf += static_cast<char>(numeric_value);
+ } else {
+ // Some symbols are case-sensitive (AElig vs aelig are different
+ // code-points) where as some are case-insensitive (" and
+ // " both work. So do the case-sensitive lookup first, and
+ // if that fails, do an insensitive lookup.
+ StringStringMapSensitive::const_iterator p =
+ unescape_sensitive_map_.find(escape);
+ if (p != unescape_sensitive_map_.end()) {
+ *buf += p->second;
+ // TODO(jmarantz): fix this code for multi-byte sequences.
+ } else {
+ // The sensitive lookup failed, but allow, for example, " to work
+ // in place of "
+ StringStringMapInsensitive::const_iterator q =
+ unescape_insensitive_map_.find(escape);
+ if (q != unescape_insensitive_map_.end()) {
+ *buf += q->second;
+ } else {
+ bogus_escape = true;
+ }
+ }
+ }
+ in_escape = false;
+ } else if (accumulate_numeric_code &&
+ ((hex_mode && !AccumulateHexValue(ch, &numeric_value)) ||
+ (!hex_mode && !AccumulateDecimalValue(ch, &numeric_value)))) {
+ bogus_escape = true;
+ } else {
+ escape += ch;
+ }
+ if (bogus_escape) {
+ // Error("Invalid escape syntax: %s", escape.c_str());
+ *buf += "&";
+ *buf += escape;
+ in_escape = false;
+ *buf += ch;
+ }
+ }
+ if (in_escape) {
+ // Error("Unterminated escape: %s", escape.c_str());
+ *buf += "&";
+ *buf += escape;
+ }
+ return StringPiece(*buf);
+}
+
+StringPiece HtmlEscape::EscapeHelper(const StringPiece& unescaped,
+ std::string* buf) const {
+ if (unescaped.data() == NULL) {
+ return unescaped;
+ }
+ buf->clear();
+
+ std::string char_to_escape;
+ for (size_t i = 0; i < unescaped.size(); ++i) {
+ int ch = unescaped[i];
+ // See http://www.htmlescape.net/htmlescape_tool.html. Single-quote and
+ // semi-colon do not need to be escaped.
+ if ((ch > 127) || (ch < 32) || (ch == '"') || (ch == '&') || (ch == '<') ||
+ (ch == '>')) {
+ char_to_escape.clear();
+ char_to_escape += ch;
+ StringStringMapSensitive::const_iterator p =
+ escape_map_.find(char_to_escape);
+ if (p == escape_map_.end()) {
+ *buf += StringPrintf("&#%02d;", static_cast<int>(ch));
+ } else {
+ *buf += '&';
+ *buf += p->second;
+ *buf += ';';
+ }
+ } else {
+ *buf += unescaped[i];
+ }
+ }
+ return StringPiece(*buf);
+}
+
+} // namespace
diff --git a/trunk/src/net/instaweb/htmlparse/html_event.cc b/trunk/src/net/instaweb/htmlparse/html_event.cc
new file mode 100644
index 0000000..8403472
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/html_event.cc
@@ -0,0 +1,34 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/htmlparse/html_event.h"
+#include <stdio.h>
+#include <string>
+
+namespace net_instaweb {
+
+HtmlEvent::~HtmlEvent() {
+}
+
+void HtmlEvent::DebugPrint() {
+ std::string buf;
+ ToString(&buf);
+ fprintf(stdout, "%s\n", buf.c_str());
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/htmlparse/html_event.h b/trunk/src/net/instaweb/htmlparse/html_event.h
new file mode 100644
index 0000000..3ad21c8
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/html_event.h
@@ -0,0 +1,220 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_HTML_EVENT_H_
+#define NET_INSTAWEB_HTMLPARSE_HTML_EVENT_H_
+
+#include <list>
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_filter.h"
+#include "net/instaweb/htmlparse/public/html_node.h"
+#include <string>
+
+namespace net_instaweb {
+
+class HtmlEvent {
+ public:
+ explicit HtmlEvent(int line_number) : line_number_(line_number) {
+ }
+ virtual ~HtmlEvent();
+ virtual void Run(HtmlFilter* filter) = 0;
+ virtual void ToString(std::string* buffer) = 0;
+
+ // If this is a StartElement event, returns the HtmlElement that is being
+ // started. Otherwise returns NULL.
+ virtual HtmlElement* GetElementIfStartEvent() { return NULL; }
+
+ // If this is an EndElement event, returns the HtmlElement that is being
+ // ended. Otherwise returns NULL.
+ virtual HtmlElement* GetElementIfEndEvent() { return NULL; }
+
+ virtual HtmlLeafNode* GetLeafNode() { return NULL; }
+ virtual HtmlNode* GetNode() { return NULL; }
+ virtual HtmlCharactersNode* GetCharactersNode() { return NULL; }
+ void DebugPrint();
+
+ int line_number() const { return line_number_; }
+ private:
+ int line_number_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlEvent);
+};
+
+class HtmlStartDocumentEvent: public HtmlEvent {
+ public:
+ explicit HtmlStartDocumentEvent(int line_number) : HtmlEvent(line_number) {}
+ virtual void Run(HtmlFilter* filter) { filter->StartDocument(); }
+ virtual void ToString(std::string* str) { *str += "StartDocument"; }
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(HtmlStartDocumentEvent);
+};
+
+class HtmlEndDocumentEvent: public HtmlEvent {
+ public:
+ explicit HtmlEndDocumentEvent(int line_number) : HtmlEvent(line_number) {}
+ virtual void Run(HtmlFilter* filter) { filter->EndDocument(); }
+ virtual void ToString(std::string* str) { *str += "EndDocument"; }
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(HtmlEndDocumentEvent);
+};
+
+class HtmlStartElementEvent: public HtmlEvent {
+ public:
+ HtmlStartElementEvent(HtmlElement* element, int line_number)
+ : HtmlEvent(line_number),
+ element_(element) {
+ }
+ virtual void Run(HtmlFilter* filter) { filter->StartElement(element_); }
+ virtual void ToString(std::string* str) {
+ *str += "StartElement ";
+ *str += element_->tag().c_str();
+ }
+ virtual HtmlElement* GetElementIfStartEvent() { return element_; }
+ virtual HtmlElement* GetNode() { return element_; }
+ private:
+ HtmlElement* element_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlStartElementEvent);
+};
+
+class HtmlEndElementEvent: public HtmlEvent {
+ public:
+ HtmlEndElementEvent(HtmlElement* element, int line_number)
+ : HtmlEvent(line_number),
+ element_(element) {
+ }
+ virtual void Run(HtmlFilter* filter) { filter->EndElement(element_); }
+ virtual void ToString(std::string* str) {
+ *str += "EndElement ";
+ *str += element_->tag().c_str();
+ }
+ virtual HtmlElement* GetElementIfEndEvent() { return element_; }
+ virtual HtmlElement* GetNode() { return element_; }
+ private:
+ HtmlElement* element_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlEndElementEvent);
+};
+
+class HtmlLeafNodeEvent: public HtmlEvent {
+ public:
+ explicit HtmlLeafNodeEvent(int line_number) : HtmlEvent(line_number) { }
+ virtual HtmlNode* GetNode() { return GetLeafNode(); }
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(HtmlLeafNodeEvent);
+};
+
+class HtmlIEDirectiveEvent: public HtmlLeafNodeEvent {
+ public:
+ HtmlIEDirectiveEvent(HtmlIEDirectiveNode* directive, int line_number)
+ : HtmlLeafNodeEvent(line_number),
+ directive_(directive) {
+ }
+ virtual void Run(HtmlFilter* filter) { filter->IEDirective(directive_); }
+ virtual void ToString(std::string* str) {
+ *str += "IEDirective ";
+ *str += directive_->contents();
+ }
+ virtual HtmlLeafNode* GetLeafNode() { return directive_; }
+ private:
+ HtmlIEDirectiveNode* directive_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlIEDirectiveEvent);
+};
+
+class HtmlCdataEvent: public HtmlLeafNodeEvent {
+ public:
+ HtmlCdataEvent(HtmlCdataNode* cdata, int line_number)
+ : HtmlLeafNodeEvent(line_number),
+ cdata_(cdata) {
+ }
+ virtual void Run(HtmlFilter* filter) { filter->Cdata(cdata_); }
+ virtual void ToString(std::string* str) {
+ *str += "Cdata ";
+ *str += cdata_->contents();
+ }
+ virtual HtmlLeafNode* GetLeafNode() { return cdata_; }
+ private:
+ HtmlCdataNode* cdata_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlCdataEvent);
+};
+
+class HtmlCommentEvent: public HtmlLeafNodeEvent {
+ public:
+ HtmlCommentEvent(HtmlCommentNode* comment, int line_number)
+ : HtmlLeafNodeEvent(line_number),
+ comment_(comment) {
+ }
+ virtual void Run(HtmlFilter* filter) { filter->Comment(comment_); }
+ virtual void ToString(std::string* str) {
+ *str += "Comment ";
+ *str += comment_->contents();
+ }
+ virtual HtmlLeafNode* GetLeafNode() { return comment_; }
+
+ private:
+ HtmlCommentNode* comment_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlCommentEvent);
+};
+
+class HtmlCharactersEvent: public HtmlLeafNodeEvent {
+ public:
+ HtmlCharactersEvent(HtmlCharactersNode* characters, int line_number)
+ : HtmlLeafNodeEvent(line_number),
+ characters_(characters) {
+ }
+ virtual void Run(HtmlFilter* filter) { filter->Characters(characters_); }
+ virtual void ToString(std::string* str) {
+ *str += "Characters ";
+ *str += characters_->contents();
+ }
+ virtual HtmlLeafNode* GetLeafNode() { return characters_; }
+ virtual HtmlCharactersNode* GetCharactersNode() { return characters_; }
+ private:
+ HtmlCharactersNode* characters_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlCharactersEvent);
+};
+
+class HtmlDirectiveEvent: public HtmlLeafNodeEvent {
+ public:
+ HtmlDirectiveEvent(HtmlDirectiveNode* directive, int line_number)
+ : HtmlLeafNodeEvent(line_number),
+ directive_(directive) {
+ }
+ virtual void Run(HtmlFilter* filter) { filter->Directive(directive_); }
+ virtual void ToString(std::string* str) {
+ *str += "Directive: ";
+ *str += directive_->contents();
+ }
+ virtual HtmlLeafNode* GetLeafNode() { return directive_; }
+ private:
+ HtmlDirectiveNode* directive_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlDirectiveEvent);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_HTML_EVENT_H_
diff --git a/trunk/src/net/instaweb/htmlparse/html_filter.cc b/trunk/src/net/instaweb/htmlparse/html_filter.cc
new file mode 100644
index 0000000..90477f2
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/html_filter.cc
@@ -0,0 +1,29 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "public/html_filter.h"
+
+namespace net_instaweb {
+
+HtmlFilter::HtmlFilter() {
+}
+
+HtmlFilter::~HtmlFilter() {
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/htmlparse/html_lexer.cc b/trunk/src/net/instaweb/htmlparse/html_lexer.cc
new file mode 100644
index 0000000..56b014e
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/html_lexer.cc
@@ -0,0 +1,1045 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/htmlparse/html_lexer.h"
+#include <ctype.h>
+#include <stdio.h>
+#include <string.h>
+#include "net/instaweb/htmlparse/html_event.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace {
+// These tags can be specified in documents without a brief "/>",
+// or an explicit </tag>, according to the Chrome Developer Tools console.
+//
+// TODO(jmarantz): Check out
+// http://www.whatwg.org/specs/web-apps/current-work/multipage/
+// syntax.html#optional-tags
+const char* kImplicitlyClosedHtmlTags[] = {
+ "meta", "input", "link", "br", "img", "area", "hr", "wbr", "param", "?xml",
+ NULL
+};
+
+// These tags cannot be closed using the brief syntax; they must
+// be closed by using an explicit </TAG>.
+const char* kNonBriefTerminatedTags[] = {
+ "script", "a", "div", "span", "iframe", "style", "textarea",
+ NULL
+};
+
+// These tags cause the text inside them to retained literally
+// and not interpreted.
+const char* kLiteralTags[] = {
+ "script", "iframe", "textarea", "style",
+ NULL
+};
+
+// These tags do not need to be explicitly closed, but can be. See
+// http://www.w3.org/TR/html5/syntax.html#optional-tags . Note that this
+// is *not* consistent with http://www.w3schools.com/tags/tag_p.asp which
+// claims this tag works the same in XHTML as HTML.
+//
+// TODO(jmarantz): http://www.w3.org/TR/html5/syntax.html#optional-tags
+// specifies complex rules, in some cases, dictating whether the closing
+// elements are optional or not. For now we just will eliminate the
+// messages for any of these tags in any case. These rules are echoed below
+// in case we want to add code to emit the 'unclosed tag' messages when they
+// are appropriate. For now we err on the side of silence.
+const char* kOptionallyClosedTags[] = {
+ // An html element's end tag may be omitted if the html element is not
+ // immediately followed by a comment.
+ "html",
+
+ // A body element's end tag may be omitted if the body element is not
+ // immediately followed by a comment.
+ "body",
+
+ // A li element's end tag may be omitted if the li element is immediately
+ // followed by another li element or if there is no more content in the
+ // parent element.
+ "li",
+
+ // A dt element's end tag may be omitted if the dt element is immediately
+ // followed by another dt element or a dd element.
+ "dt",
+
+ // A dd element's end tag may be omitted if the dd element is immediately
+ // followed by another dd element or a dt element, or if there is no more
+ // content in the parent element.
+ "dd",
+
+ // A p element's end tag may be omitted if the p element is immediately
+ // followed by an address, article, aside, blockquote, dir, div, dl, fieldset,
+ // footer, form, h1, h2, h3, h4, h5, h6, header, hgroup, hr, menu, nav, ol, p,
+ // pre, section, table, or ul, element, or if there is no more content in the
+ // parent element and the parent element is not an a element.
+ "p",
+
+ // An rt element's end tag may be omitted if the rt element is immediately
+ // followed by an rt or rp element, or if there is no more content in the
+ // parent element.
+ "rt",
+
+ // An rp element's end tag may be omitted if the rp element is immediately
+ // followed by an rt or rp element, or if there is no more content in the
+ // parent element.
+ "rp",
+
+ // An optgroup element's end tag may be omitted if the optgroup element is
+ // immediately followed by another optgroup element, or if there is no more
+ // content in the parent element.
+ "optgroup",
+
+ // An option element's end tag may be omitted if the option element is
+ // immediately followed by another option element, or if it is immediately
+ // followed by an optgroup element, or if there is no more content in the
+ // parent element.
+ "option",
+
+ // A colgroup element's end tag may be omitted if the colgroup element is not
+ // immediately followed by a space character or a comment.
+ "colgroup",
+
+ // A thead element's end tag may be omitted if the thead element is
+ // immediately followed by a tbody or tfoot element.
+ "thead",
+
+ // A tbody element's end tag may be omitted if the tbody element is
+ // immediately followed by a tbody or tfoot element, or if there is no more
+ // content in the parent element.
+ "tbody",
+
+ // A tfoot element's end tag may be omitted if the tfoot element is
+ // immediately followed by a tbody element, or if there is no more content in
+ // the parent element.
+ "tfoot",
+
+ // A tr element's end tag may be omitted if the tr element is immediately
+ // followed by another tr element, or if there is no more content in the
+ // parent element.
+ "tr",
+
+ // A td element's end tag may be omitted if the td element is immediately
+ // followed by a td or th element, or if there is no more content in the
+ // parent element.
+ "td",
+
+ // A th element's end tag may be omitted if the th element is immediately
+ // followed by a td or th element, or if there is no more content in the
+ // parent element.
+ "th",
+
+ NULL
+};
+
+// We start our stack-iterations from 1, because we put a NULL into
+// position 0 to reduce special-cases.
+const int kStartStack = 1;
+
+} // namespace
+
+// TODO(jmarantz): support multi-byte encodings
+// TODO(jmarantz): emit close-tags immediately for selected html tags,
+// rather than waiting for the next explicit close-tag to force a rebalance.
+// See http://www.whatwg.org/specs/web-apps/current-work/multipage/
+// syntax.html#optional-tags
+
+namespace net_instaweb {
+
+HtmlLexer::HtmlLexer(HtmlParse* html_parse)
+ : html_parse_(html_parse),
+ state_(START),
+ attr_quote_(""),
+ has_attr_value_(false),
+ element_(NULL),
+ line_(1),
+ tag_start_line_(-1) {
+ for (const char** p = kImplicitlyClosedHtmlTags; *p != NULL; ++p) {
+ implicitly_closed_.insert(html_parse->Intern(*p));
+ }
+ for (const char** p = kNonBriefTerminatedTags; *p != NULL; ++p) {
+ non_brief_terminated_tags_.insert(html_parse->Intern(*p));
+ }
+ for (const char** p = kLiteralTags; *p != NULL; ++p) {
+ literal_tags_.insert(html_parse->Intern(*p));
+ }
+ for (const char** p = kOptionallyClosedTags; *p != NULL; ++p) {
+ optionally_closed_tags_.insert(html_parse->Intern(*p));
+ }
+}
+
+HtmlLexer::~HtmlLexer() {
+}
+
+void HtmlLexer::EvalStart(char c) {
+ if (c == '<') {
+ literal_.resize(literal_.size() - 1);
+ EmitLiteral();
+ literal_ += c;
+ state_ = TAG;
+ tag_start_line_ = line_;
+ } else {
+ state_ = START;
+ }
+}
+
+// Browsers appear to only allow letters for first char in tag name,
+// plus ? for <?xml version="1.0" encoding="UTF-8"?>
+bool HtmlLexer::IsLegalTagFirstChar(char c) {
+ return isalpha(c) || (c == '?');
+}
+
+// ... and letters, digits, unicode and some symbols for subsequent chars.
+// Based on a test of Firefox and Chrome.
+//
+// TODO(jmarantz): revisit these predicates based on
+// http://www.w3.org/TR/REC-xml/#NT-NameChar . This
+// XML spec may or may not inform of us of what we need to do
+// to parse all HTML on the web.
+bool HtmlLexer::IsLegalTagChar(char c) {
+ return (IsI18nChar(c) ||
+ (isalnum(c) || (c == '-') || (c == '#') || (c == '_') || (c == ':')));
+}
+
+bool HtmlLexer::IsLegalAttrNameChar(char c) {
+ return (IsI18nChar(c) ||
+ ((c != '=') && (c != '>') && (c != '/') && (c != '<') &&
+ !isspace(c)));
+}
+
+bool HtmlLexer::IsLegalAttrValChar(char c) {
+ return (IsI18nChar(c) ||
+ ((c != '=') && (c != '>') && (c != '/') && (c != '<') &&
+ (c != '"') && (c != '\'') && !isspace(c)));
+}
+
+// Handle the case where "<" was recently parsed.
+void HtmlLexer::EvalTag(char c) {
+ if (c == '/') {
+ state_ = TAG_CLOSE;
+ } else if (IsLegalTagFirstChar(c)) { // "<x"
+ state_ = TAG_OPEN;
+ token_ += c;
+ } else if (c == '!') {
+ state_ = COMMENT_START1;
+ } else {
+ // Illegal tag syntax; just pass it through as raw characters
+ SyntaxError("Invalid tag syntax: unexpected sequence `<%c'", c);
+ EvalStart(c);
+ }
+}
+
+// Handle the case where "<x" was recently parsed. We will stay in this
+// state as long as we keep seeing legal tag characters, appending to
+// token_ for each character.
+void HtmlLexer::EvalTagOpen(char c) {
+ if (IsLegalTagChar(c)) {
+ token_ += c;
+ } else if (c == '>') {
+ EmitTagOpen(true);
+ } else if (c == '<') {
+ // Chrome transforms "<tag<tag>" into "<tag><tag>";
+ SyntaxError("Invalid tag syntax: expected close tag before opener");
+ EmitTagOpen(true);
+ literal_ = "<"; // will be removed by EvalStart.
+ EvalStart(c);
+ } else if (c == '/') {
+ state_ = TAG_BRIEF_CLOSE;
+ } else if (isspace(c)) {
+ state_ = TAG_ATTRIBUTE;
+ } else {
+ // Some other punctuation. Not sure what to do. Let's run this
+ // on the web and see what breaks & decide what to do. E.g. "<x&"
+ SyntaxError("Invalid character `%c` while parsing tag `%s'",
+ c, token_.c_str());
+ token_.clear();
+ state_ = START;
+ }
+}
+
+// Handle several cases of seeing "/" in the middle of a tag, but after
+// the identifier has been completed. Examples: "<x /" or "<x y/" or "x y=/z".
+void HtmlLexer::EvalTagBriefCloseAttr(char c) {
+ if (c == '>') {
+ FinishAttribute(c, has_attr_value_, true);
+ } else if (isspace(c)) {
+ // "<x y/ ". This can lead to "<x y/ z" where z would be
+ // a new attribute, or "<x y/ >" where the tag would be
+ // closed without adding a new attribute. In either case,
+ // we will be completing this attribute.
+ //
+ // TODO(jmarantz): what about "<x y/ =z>"? I am not sure
+ // sure if this matters, because testing that would require
+ // a browser that could react to a named attribute with a
+ // slash in the name (not the value). But should we wind
+ // up with 1 attributes or 2 for this case? There are probably
+ // more important questions, but if we ever need to answer that
+ // one, this is the place.
+ if (!attr_name_.empty()) {
+ if (has_attr_value_) {
+ // The "/" should be interpreted as the last character in
+ // the attribute, so we must tack it on before making it.
+ attr_value_ += '/';
+ }
+ MakeAttribute(has_attr_value_);
+ }
+ } else {
+ // Slurped www.google.com has
+ // <a href=/advanced_search?hl=en>Advanced Search</a>
+ // So when we first see the "/" it looks like it might
+ // be a brief-close, .e.g. <a href=/>. But when we see
+ // that what follows the '/' is not '>' then we know it's
+ // just part off the attribute name or value. So there's
+ // no need to even warn.
+ if (has_attr_value_) {
+ attr_value_ += '/';
+ state_ = TAG_ATTR_VAL;
+ EvalAttrVal(c);
+ // we know it's not the double-quoted or single-quoted versions
+ // because then we wouldn't have let the '/' get us into the
+ // brief-close state.
+ } else {
+ attr_name_ += '/';
+ state_ = TAG_ATTR_NAME;
+ EvalAttrName(c);
+ }
+ }
+}
+
+// Handle the case where "<x/" was recently parsed, where "x" can
+// be any length tag identifier. Note that we if we anything other
+// than a ">" after this, we will just consider the "/" to be part
+// of the tag identifier, and go back to the TAG_OPEN state.
+void HtmlLexer::EvalTagBriefClose(char c) {
+ if (c == '>') {
+ EmitTagOpen(false);
+ EmitTagBriefClose();
+ } else {
+ std::string expected(literal_.data(), literal_.size() - 1);
+ SyntaxError("Invalid close tag syntax: expected %s>, got %s",
+ expected.c_str(), literal_.c_str());
+ // Recover by returning to the mode from whence we came.
+ if (element_ != NULL) {
+ token_ += '/';
+ state_ = TAG_OPEN;
+ EvalTagOpen(c);
+ } else {
+ // E.g. "<R/A", see testdata/invalid_brief.html.
+ state_ = START;
+ token_.clear();
+ }
+ }
+}
+
+// Handle the case where "</" was recently parsed. This function
+// is also called for "</a ", in which case state will be TAG_CLOSE_TERMINATE.
+// We distinguish that case to report an error on "</a b>".
+void HtmlLexer::EvalTagClose(char c) {
+ if ((state_ != TAG_CLOSE_TERMINATE) && IsLegalTagChar(c)) { // "</x"
+ token_ += c;
+ } else if (isspace(c)) {
+ if (token_.empty()) { // e.g. "</ a>"
+ // just ignore the whitespace. Wait for
+ // the tag-name to begin.
+ } else {
+ // "</a ". Now we are in a state where we can only
+ // accept more whitesapce or a close.
+ state_ = TAG_CLOSE_TERMINATE;
+ }
+ } else if (c == '>') {
+ EmitTagClose(HtmlElement::EXPLICIT_CLOSE);
+ } else {
+ SyntaxError("Invalid tag syntax: expected `>' after `</%s' got `%c'",
+ token_.c_str(), c);
+ token_.clear();
+ EvalStart(c);
+ }
+}
+
+// Handle the case where "<!x" was recently parsed, where x
+// is any illegal tag identifier. We stay in this state until
+// we see the ">", accumulating the directive in token_.
+void HtmlLexer::EvalDirective(char c) {
+ if (c == '>') {
+ EmitDirective();
+ } else {
+ token_ += c;
+ }
+}
+
+// After a partial match of a multi-character lexical sequence, a mismatched
+// character needs to temporarily removed from the retained literal_ before
+// being emitted. Then re-inserted for so that EvalStart can attempt to
+// re-evaluate this character as potentialy starting a new lexical token.
+void HtmlLexer::Restart(char c) {
+ CHECK_LE(1U, literal_.size());
+ CHECK_EQ(c, literal_[literal_.size() - 1]);
+ literal_.resize(literal_.size() - 1);
+ EmitLiteral();
+ literal_ += c;
+ EvalStart(c);
+}
+
+// Handle the case where "<!" was recently parsed.
+void HtmlLexer::EvalCommentStart1(char c) {
+ if (c == '-') {
+ state_ = COMMENT_START2;
+ } else if (c == '[') {
+ state_ = CDATA_START1;
+ } else if (IsLegalTagChar(c)) { // "<!DOCTYPE ... >"
+ state_ = DIRECTIVE;
+ EvalDirective(c);
+ } else {
+ SyntaxError("Invalid comment syntax");
+ Restart(c);
+ }
+}
+
+// Handle the case where "<!-" was recently parsed.
+void HtmlLexer::EvalCommentStart2(char c) {
+ if (c == '-') {
+ state_ = COMMENT_BODY;
+ } else {
+ SyntaxError("Invalid comment syntax");
+ Restart(c);
+ }
+}
+
+// Handle the case where "<!--" was recently parsed. We will stay in
+// this state until we see "-". And even after that we may go back to
+// this state if the "-" is not followed by "->".
+void HtmlLexer::EvalCommentBody(char c) {
+ if (c == '-') {
+ state_ = COMMENT_END1;
+ } else {
+ token_ += c;
+ }
+}
+
+// Handle the case where "-" has been parsed from a comment. If we
+// see another "-" then we go to CommentEnd2, otherwise we go back
+// to the comment state.
+void HtmlLexer::EvalCommentEnd1(char c) {
+ if (c == '-') {
+ state_ = COMMENT_END2;
+ } else {
+ // thought we were ending a comment cause we saw '-', but
+ // now we changed our minds. No worries mate. That
+ // fake-out dash was just part of the comment.
+ token_ += '-';
+ token_ += c;
+ state_ = COMMENT_BODY;
+ }
+}
+
+// Handle the case where "--" has been parsed from a comment.
+void HtmlLexer::EvalCommentEnd2(char c) {
+ if (c == '>') {
+ EmitComment();
+ state_ = START;
+ } else if (c == '-') {
+ // There could be an arbitrarily long stream of dashes before
+ // we see the >. Keep looking.
+ token_ += "-";
+ } else {
+ // thought we were ending a comment cause we saw '--', but
+ // now we changed our minds. No worries mate. Those
+ // fake-out dashes were just part of the comment.
+ token_ += "--";
+ token_ += c;
+ state_ = COMMENT_BODY;
+ }
+}
+
+// Handle the case where "<![" was recently parsed.
+void HtmlLexer::EvalCdataStart1(char c) {
+ if (c == 'C') {
+ state_ = CDATA_START2;
+ } else {
+ SyntaxError("Invalid CDATA syntax");
+ Restart(c);
+ }
+}
+
+// Handle the case where "<![C" was recently parsed.
+void HtmlLexer::EvalCdataStart2(char c) {
+ if (c == 'D') {
+ state_ = CDATA_START3;
+ } else {
+ SyntaxError("Invalid CDATA syntax");
+ Restart(c);
+ }
+}
+
+// Handle the case where "<![CD" was recently parsed.
+void HtmlLexer::EvalCdataStart3(char c) {
+ if (c == 'A') {
+ state_ = CDATA_START4;
+ } else {
+ SyntaxError("Invalid CDATA syntax");
+ Restart(c);
+ }
+}
+
+// Handle the case where "<![CDA" was recently parsed.
+void HtmlLexer::EvalCdataStart4(char c) {
+ if (c == 'T') {
+ state_ = CDATA_START5;
+ } else {
+ SyntaxError("Invalid CDATA syntax");
+ Restart(c);
+ }
+}
+
+// Handle the case where "<![CDAT" was recently parsed.
+void HtmlLexer::EvalCdataStart5(char c) {
+ if (c == 'A') {
+ state_ = CDATA_START6;
+ } else {
+ SyntaxError("Invalid CDATA syntax");
+ Restart(c);
+ }
+}
+
+// Handle the case where "<![CDATA" was recently parsed.
+void HtmlLexer::EvalCdataStart6(char c) {
+ if (c == '[') {
+ state_ = CDATA_BODY;
+ } else {
+ SyntaxError("Invalid CDATA syntax");
+ Restart(c);
+ }
+}
+
+// Handle the case where "<![CDATA[" was recently parsed. We will stay in
+// this state until we see "]". And even after that we may go back to
+// this state if the "]" is not followed by "]>".
+void HtmlLexer::EvalCdataBody(char c) {
+ if (c == ']') {
+ state_ = CDATA_END1;
+ } else {
+ token_ += c;
+ }
+}
+
+// Handle the case where "]" has been parsed from a cdata. If we
+// see another "]" then we go to CdataEnd2, otherwise we go back
+// to the cdata state.
+void HtmlLexer::EvalCdataEnd1(char c) {
+ if (c == ']') {
+ state_ = CDATA_END2;
+ } else {
+ // thought we were ending a cdata cause we saw ']', but
+ // now we changed our minds. No worries mate. That
+ // fake-out bracket was just part of the cdata.
+ token_ += ']';
+ token_ += c;
+ state_ = CDATA_BODY;
+ }
+}
+
+// Handle the case where "]]" has been parsed from a cdata.
+void HtmlLexer::EvalCdataEnd2(char c) {
+ if (c == '>') {
+ EmitCdata();
+ state_ = START;
+ } else {
+ // thought we were ending a cdata cause we saw ']]', but
+ // now we changed our minds. No worries mate. Those
+ // fake-out brackets were just part of the cdata.
+ token_ += "]]";
+ token_ += c;
+ state_ = CDATA_BODY;
+ }
+}
+
+// Handle the case where a literal tag (script, iframe) was started.
+// This is of lexical significance because we ignore all the special
+// characters until we see "</script>" or "</iframe>".
+void HtmlLexer::EvalLiteralTag(char c) {
+ // Look explicitly for </script> in the literal buffer.
+ // TODO(jmarantz): check for whitespace in unexpected places.
+ if (c == '>') {
+ // expecting "</x>" for tag x.
+ html_parse_->message_handler()->Check(
+ literal_close_.size() > 3, "literal_close_.size() <= 3"); // NOLINT
+ int literal_minus_close_size = literal_.size() - literal_close_.size();
+ if ((literal_minus_close_size >= 0) &&
+ (strcasecmp(literal_.c_str() + literal_minus_close_size,
+ literal_close_.c_str()) == 0)) {
+ // The literal actually starts after the "<script>", and we will
+ // also let it finish before, so chop it off.
+ literal_.resize(literal_minus_close_size);
+ EmitLiteral();
+ token_.clear();
+ // Transform "</script>" into "script" to form close tag.
+ token_.append(literal_close_.c_str() + 2, literal_close_.size() - 3);
+ EmitTagClose(HtmlElement::EXPLICIT_CLOSE);
+ }
+ }
+}
+
+// Emits raw uninterpreted characters.
+void HtmlLexer::EmitLiteral() {
+ if (!literal_.empty()) {
+ html_parse_->AddEvent(new HtmlCharactersEvent(
+ html_parse_->NewCharactersNode(Parent(), literal_), tag_start_line_));
+ literal_.clear();
+ }
+ state_ = START;
+}
+
+void HtmlLexer::EmitComment() {
+ literal_.clear();
+ if ((token_.find("[if IE") != std::string::npos) &&
+ (token_.find("<![endif]") != std::string::npos)) {
+ HtmlIEDirectiveNode* node =
+ html_parse_->NewIEDirectiveNode(Parent(), token_);
+ html_parse_->AddEvent(new HtmlIEDirectiveEvent(node, tag_start_line_));
+ } else {
+ HtmlCommentNode* node = html_parse_->NewCommentNode(Parent(), token_);
+ html_parse_->AddEvent(new HtmlCommentEvent(node, tag_start_line_));
+ }
+ token_.clear();
+ state_ = START;
+}
+
+void HtmlLexer::EmitCdata() {
+ literal_.clear();
+ html_parse_->AddEvent(new HtmlCdataEvent(
+ html_parse_->NewCdataNode(Parent(), token_), tag_start_line_));
+ token_.clear();
+ state_ = START;
+}
+
+// If allow_implicit_close is true, and the element type is one which
+// does not require an explicit termination in HTML, then we will
+// automatically emit a matching 'element close' event.
+void HtmlLexer::EmitTagOpen(bool allow_implicit_close) {
+ literal_.clear();
+ MakeElement();
+ html_parse_->AddElement(element_, tag_start_line_);
+ element_stack_.push_back(element_);
+ if (literal_tags_.find(element_->tag()) != literal_tags_.end()) {
+ state_ = LITERAL_TAG;
+ literal_close_ = "</";
+ literal_close_ += element_->tag().c_str();
+ literal_close_ += ">";
+ } else {
+ state_ = START;
+ }
+
+ Atom tag = element_->tag();
+ if (allow_implicit_close && IsImplicitlyClosedTag(tag)) {
+ token_ = tag.c_str();
+ EmitTagClose(HtmlElement::IMPLICIT_CLOSE);
+ }
+
+ element_ = NULL;
+}
+
+void HtmlLexer::EmitTagBriefClose() {
+ HtmlElement* element = PopElement();
+ html_parse_->CloseElement(element, HtmlElement::BRIEF_CLOSE, line_);
+ state_ = START;
+}
+
+static void toLower(std::string* str) {
+ for (int i = 0, n = str->size(); i < n; ++i) {
+ char& c = (*str)[i];
+ c = tolower(c);
+ }
+}
+
+HtmlElement* HtmlLexer::Parent() const {
+ html_parse_->message_handler()->Check(!element_stack_.empty(),
+ "element_stack_.empty()");
+ return element_stack_.back();
+}
+
+void HtmlLexer::MakeElement() {
+ if (element_ == NULL) {
+ if (token_.empty()) {
+ SyntaxError("Making element with empty tag name");
+ }
+ toLower(&token_);
+ element_ = html_parse_->NewElement(Parent(), html_parse_->Intern(token_));
+ element_->set_begin_line_number(tag_start_line_);
+ token_.clear();
+ }
+}
+
+void HtmlLexer::StartParse(const StringPiece& id,
+ const ContentType& content_type) {
+ line_ = 1;
+ tag_start_line_ = -1;
+ id.CopyToString(&id_);
+ content_type_ = content_type;
+ has_attr_value_ = false;
+ attr_quote_ = "";
+ state_ = START;
+ element_stack_.clear();
+ element_stack_.push_back(NULL);
+ element_ = NULL;
+ token_.clear();
+ attr_name_.clear();
+ attr_value_.clear();
+ literal_.clear();
+ // clear buffers
+}
+
+void HtmlLexer::FinishParse() {
+ if (!token_.empty()) {
+ SyntaxError("End-of-file in mid-token: %s", token_.c_str());
+ token_.clear();
+ }
+ if (!attr_name_.empty()) {
+ SyntaxError("End-of-file in mid-attribute-name: %s", attr_name_.c_str());
+ attr_name_.clear();
+ }
+ if (!attr_value_.empty()) {
+ SyntaxError("End-of-file in mid-attribute-value: %s", attr_value_.c_str());
+ attr_value_.clear();
+ }
+
+ if (!literal_.empty()) {
+ EmitLiteral();
+ }
+
+ // Any unclosed tags? These should be noted.
+ html_parse_->message_handler()->Check(!element_stack_.empty(),
+ "element_stack_.empty()");
+ html_parse_->message_handler()->Check(element_stack_[0] == NULL,
+ "element_stack_[0] != NULL");
+ for (size_t i = kStartStack; i < element_stack_.size(); ++i) {
+ HtmlElement* element = element_stack_[i];
+ Atom tag = element->tag();
+ if (!IsOptionallyClosedTag(tag)) {
+ html_parse_->Info(id_.c_str(), element->begin_line_number(),
+ "End-of-file with open tag: %s",
+ tag.c_str());
+ }
+ }
+ element_stack_.clear();
+ element_stack_.push_back(NULL);
+ element_ = NULL;
+}
+
+void HtmlLexer::MakeAttribute(bool has_value) {
+ html_parse_->message_handler()->Check(element_ != NULL, "element_ == NULL");
+ toLower(&attr_name_);
+ Atom name = html_parse_->Intern(attr_name_);
+ attr_name_.clear();
+ const char* value = NULL;
+ html_parse_->message_handler()->Check(has_value == has_attr_value_,
+ "has_value != has_attr_value_");
+ if (has_value) {
+ value = attr_value_.c_str();
+ has_attr_value_ = false;
+ } else {
+ html_parse_->message_handler()->Check(attr_value_.empty(),
+ "!attr_value_.empty()");
+ }
+ element_->AddEscapedAttribute(name, value, attr_quote_);
+ attr_value_.clear();
+ attr_quote_ = "";
+ state_ = TAG_ATTRIBUTE;
+}
+
+void HtmlLexer::EvalAttribute(char c) {
+ MakeElement();
+ attr_name_.clear();
+ attr_value_.clear();
+ if (c == '>') {
+ EmitTagOpen(true);
+ } else if (c == '<') {
+ FinishAttribute(c, false, false);
+ } else if (c == '/') {
+ state_ = TAG_BRIEF_CLOSE_ATTR;
+ } else if (IsLegalAttrNameChar(c)) {
+ attr_name_ += c;
+ state_ = TAG_ATTR_NAME;
+ } else if (!isspace(c)) {
+ SyntaxError("Unexpected char `%c' in attribute list", c);
+ }
+}
+
+// "<x y" or "<x y ".
+void HtmlLexer::EvalAttrName(char c) {
+ if (c == '=') {
+ state_ = TAG_ATTR_EQ;
+ has_attr_value_ = true;
+ } else if (IsLegalAttrNameChar(c) && (state_ != TAG_ATTR_NAME_SPACE)) {
+ attr_name_ += c;
+ } else if (isspace(c)) {
+ state_ = TAG_ATTR_NAME_SPACE;
+ } else if (c == '>') {
+ MakeAttribute(false);
+ EmitTagOpen(true);
+ } else {
+ if (state_ == TAG_ATTR_NAME_SPACE) {
+ // "<x y z". Now that we see the 'z', we need
+ // to finish 'y' as an attribute, then queue up
+ // 'z' (c) as the start of a new attribute.
+ MakeAttribute(false);
+ state_ = TAG_ATTR_NAME;
+ attr_name_ += c;
+ } else {
+ FinishAttribute(c, false, false);
+ }
+ }
+}
+
+void HtmlLexer::FinishAttribute(char c, bool has_value, bool brief_close) {
+ if (isspace(c)) {
+ MakeAttribute(has_value);
+ state_ = TAG_ATTRIBUTE;
+ } else if (c == '/') {
+ // If / was seen terminating an attribute, without
+ // the closing quote or whitespace, it might just be
+ // part of a syntactically dubious attribute. We'll
+ // hold off completing the attribute till we see the
+ // next character.
+ state_ = TAG_BRIEF_CLOSE_ATTR;
+ } else if ((c == '<') || (c == '>')) {
+ if (!attr_name_.empty()) {
+ if (!brief_close &&
+ (strcmp(attr_name_.c_str(), "/") == 0) && !has_value) {
+ brief_close = true;
+ attr_name_.clear();
+ attr_value_.clear();
+ } else {
+ MakeAttribute(has_value);
+ }
+ }
+ EmitTagOpen(!brief_close);
+ if (brief_close) {
+ EmitTagBriefClose();
+ }
+
+ if (c == '<') {
+ // Chrome transforms "<tag a<tag>" into "<tag a><tag>"; we should too.
+ SyntaxError("Invalid tag syntax: expected close tag before opener");
+ literal_ += '<';
+ EvalStart(c);
+ }
+ has_attr_value_ = false;
+ } else {
+ // Some other funny character within a tag. Probably can't
+ // trust the tag at all. Check the web and see when this
+ // happens.
+ SyntaxError("Unexpected character in attribute: %c", c);
+ MakeAttribute(has_value);
+ has_attr_value_ = false;
+ }
+}
+
+void HtmlLexer::EvalAttrEq(char c) {
+ if (IsLegalAttrValChar(c)) {
+ state_ = TAG_ATTR_VAL;
+ attr_quote_ = "";
+ EvalAttrVal(c);
+ } else if (c == '"') {
+ attr_quote_ = "\"";
+ state_ = TAG_ATTR_VALDQ;
+ } else if (c == '\'') {
+ attr_quote_ = "'";
+ state_ = TAG_ATTR_VALSQ;
+ } else if (isspace(c)) {
+ // ignore -- spaces are allowed between "=" and the value
+ } else {
+ FinishAttribute(c, true, false);
+ }
+}
+
+void HtmlLexer::EvalAttrVal(char c) {
+ if (isspace(c) || (c == '>') || (c == '<')) {
+ FinishAttribute(c, true, false);
+ } else {
+ attr_value_ += c;
+ }
+}
+
+void HtmlLexer::EvalAttrValDq(char c) {
+ if (c == '"') {
+ MakeAttribute(true);
+ } else {
+ attr_value_ += c;
+ }
+}
+
+void HtmlLexer::EvalAttrValSq(char c) {
+ if (c == '\'') {
+ MakeAttribute(true);
+ } else {
+ attr_value_ += c;
+ }
+}
+
+void HtmlLexer::EmitTagClose(HtmlElement::CloseStyle close_style) {
+ toLower(&token_);
+ Atom tag = html_parse_->Intern(token_);
+ HtmlElement* element = PopElementMatchingTag(tag);
+ if (element != NULL) {
+ element->set_end_line_number(line_);
+ html_parse_->CloseElement(element, close_style, line_);
+ } else {
+ SyntaxError("Unexpected close-tag `%s', no tags are open", token_.c_str());
+ EmitLiteral();
+ }
+
+ literal_.clear();
+ token_.clear();
+ state_ = START;
+}
+
+void HtmlLexer::EmitDirective() {
+ literal_.clear();
+ html_parse_->AddEvent(new HtmlDirectiveEvent(
+ html_parse_->NewDirectiveNode(Parent(), token_), line_));
+ // Update the doctype; note that if this is not a doctype directive, Parse()
+ // will return false and not alter doctype_.
+ doctype_.Parse(token_, content_type_);
+ token_.clear();
+ state_ = START;
+}
+
+void HtmlLexer::Parse(const char* text, int size) {
+ for (int i = 0; i < size; ++i) {
+ char c = text[i];
+ if (c == '\n') {
+ ++line_;
+ }
+
+ // By default we keep track of every byte as it comes in.
+ // If we can't accurately parse it, we transmit it as
+ // raw characters to be re-serialized without interpretation,
+ // and good luck to the browser. When we do successfully
+ // parse something, we remove it from the literal.
+ literal_ += c;
+
+ switch (state_) {
+ case START: EvalStart(c); break;
+ case TAG: EvalTag(c); break;
+ case TAG_OPEN: EvalTagOpen(c); break;
+ case TAG_CLOSE: EvalTagClose(c); break;
+ case TAG_CLOSE_TERMINATE: EvalTagClose(c); break;
+ case TAG_BRIEF_CLOSE: EvalTagBriefClose(c); break;
+ case TAG_BRIEF_CLOSE_ATTR: EvalTagBriefCloseAttr(c); break;
+ case COMMENT_START1: EvalCommentStart1(c); break;
+ case COMMENT_START2: EvalCommentStart2(c); break;
+ case COMMENT_BODY: EvalCommentBody(c); break;
+ case COMMENT_END1: EvalCommentEnd1(c); break;
+ case COMMENT_END2: EvalCommentEnd2(c); break;
+ case CDATA_START1: EvalCdataStart1(c); break;
+ case CDATA_START2: EvalCdataStart2(c); break;
+ case CDATA_START3: EvalCdataStart3(c); break;
+ case CDATA_START4: EvalCdataStart4(c); break;
+ case CDATA_START5: EvalCdataStart5(c); break;
+ case CDATA_START6: EvalCdataStart6(c); break;
+ case CDATA_BODY: EvalCdataBody(c); break;
+ case CDATA_END1: EvalCdataEnd1(c); break;
+ case CDATA_END2: EvalCdataEnd2(c); break;
+ case TAG_ATTRIBUTE: EvalAttribute(c); break;
+ case TAG_ATTR_NAME: EvalAttrName(c); break;
+ case TAG_ATTR_NAME_SPACE: EvalAttrName(c); break;
+ case TAG_ATTR_EQ: EvalAttrEq(c); break;
+ case TAG_ATTR_VAL: EvalAttrVal(c); break;
+ case TAG_ATTR_VALDQ: EvalAttrValDq(c); break;
+ case TAG_ATTR_VALSQ: EvalAttrValSq(c); break;
+ case LITERAL_TAG: EvalLiteralTag(c); break;
+ case DIRECTIVE: EvalDirective(c); break;
+ }
+ }
+}
+
+bool HtmlLexer::IsImplicitlyClosedTag(Atom tag) const {
+ return (!doctype_.IsXhtml() &&
+ implicitly_closed_.find(tag) != implicitly_closed_.end());
+}
+
+bool HtmlLexer::TagAllowsBriefTermination(Atom tag) const {
+ return (non_brief_terminated_tags_.find(tag) ==
+ non_brief_terminated_tags_.end());
+}
+
+bool HtmlLexer::IsOptionallyClosedTag(Atom tag) const {
+ return (!doctype_.IsXhtml() &&
+ optionally_closed_tags_.find(tag) != optionally_closed_tags_.end());
+}
+
+void HtmlLexer::DebugPrintStack() {
+ for (size_t i = kStartStack; i < element_stack_.size(); ++i) {
+ std::string buf;
+ element_stack_[i]->ToString(&buf);
+ fprintf(stdout, "%s\n", buf.c_str());
+ }
+ fflush(stdout);
+}
+
+HtmlElement* HtmlLexer::PopElement() {
+ HtmlElement* element = NULL;
+ if (!element_stack_.empty()) {
+ element = element_stack_.back();
+ element_stack_.pop_back();
+ }
+ return element;
+}
+
+HtmlElement* HtmlLexer::PopElementMatchingTag(Atom tag) {
+ HtmlElement* element = NULL;
+
+ // Search the stack from top to bottom.
+ for (int i = element_stack_.size() - 1; i >= kStartStack; --i) {
+ element = element_stack_[i];
+ if (element->tag() == tag) {
+ // Emit warnings for the tags we are skipping. We have to do
+ // this in reverse order so that we maintain stack discipline.
+ for (int j = element_stack_.size() - 1; j > i; --j) {
+ HtmlElement* skipped = element_stack_[j];
+ // In fact, should we actually perform this optimization ourselves
+ // in a filter to omit closing tags that can be inferred?
+ Atom tag = skipped->tag();
+ if (!IsOptionallyClosedTag(tag)) {
+ html_parse_->Info(id_.c_str(), skipped->begin_line_number(),
+ "Unclosed element `%s'", skipped->tag().c_str());
+ }
+ // Before closing the skipped element, pop it off the stack. Otherwise,
+ // the parent redundancy check in HtmlParse::AddEvent will fail.
+ element_stack_.resize(j);
+ html_parse_->CloseElement(skipped, HtmlElement::UNCLOSED, line_);
+ }
+ element_stack_.resize(i);
+ break;
+ }
+ element = NULL;
+ }
+ return element;
+}
+
+void HtmlLexer::SyntaxError(const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ html_parse_->InfoV(id_.c_str(), line_, msg, args);
+ va_end(args);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/htmlparse/html_lexer.h b/trunk/src/net/instaweb/htmlparse/html_lexer.h
new file mode 100644
index 0000000..dcf428b
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/html_lexer.h
@@ -0,0 +1,223 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_HTML_LEXER_H_
+#define NET_INSTAWEB_HTMLPARSE_HTML_LEXER_H_
+
+#include <stdarg.h>
+#include <set>
+#include <vector>
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/doctype.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/printf_format.h"
+#include <string>
+
+namespace net_instaweb {
+
+// Constructs a re-entrant HTML lexer. This lexer minimally parses tags,
+// attributes, and comments. It is intended to parse the Wild West of the
+// Web. It's designed to be tolerant of syntactic transgressions, merely
+// passing through unparseable chunks as Characters.
+//
+// TODO(jmarantz): refactor this with html_parse, so that this class owns
+// the symbol table and the event queue, and no longer needs to mutually
+// depend on HtmlParse. That will make it easier to unit-test.
+class HtmlLexer {
+ public:
+ explicit HtmlLexer(HtmlParse* html_parse);
+ ~HtmlLexer();
+
+ // Initialize a new parse session, id is only used for error messages.
+ void StartParse(const StringPiece& id, const ContentType& content_type);
+
+ // Parse a chunk of text, adding events to the parser by calling
+ // html_parse_->AddEvent(...).
+ void Parse(const char* text, int size);
+
+ // Completes parse, reporting any leftover text as a final HtmlCharacterEvent.
+ void FinishParse();
+
+ // Determines whether a tag should be terminated in HTML.
+ bool IsImplicitlyClosedTag(Atom tag) const;
+
+ // Determines whether a tag can be terminated briefly (e.g. <tag/>)
+ bool TagAllowsBriefTermination(Atom tag) const;
+
+ // Determines whether it's OK to leave a tag unclosed.
+ bool IsOptionallyClosedTag(Atom tag) const;
+
+ // Print element stack to stdout (for debugging).
+ void DebugPrintStack();
+
+ // Returns the current lowest-level parent element in the element stack
+ HtmlElement* Parent() const;
+
+ // Return the current assumed doctype of the document (based on the content
+ // type and any HTML directives encountered so far).
+ const DocType& doctype() const { return doctype_; }
+
+ private:
+ // Most of these routines expect c to be the last character of literal_
+ inline void EvalStart(char c);
+ inline void EvalTag(char c);
+ inline void EvalTagOpen(char c);
+ inline void EvalTagClose(char c);
+ inline void EvalTagCloseTerminate(char c);
+ inline void EvalTagBriefClose(char c);
+ inline void EvalTagBriefCloseAttr(char c);
+ inline void EvalCommentStart1(char c);
+ inline void EvalCommentStart2(char c);
+ inline void EvalCommentBody(char c);
+ inline void EvalCommentEnd1(char c);
+ inline void EvalCommentEnd2(char c);
+ inline void EvalCdataStart1(char c);
+ inline void EvalCdataStart2(char c);
+ inline void EvalCdataStart3(char c);
+ inline void EvalCdataStart4(char c);
+ inline void EvalCdataStart5(char c);
+ inline void EvalCdataStart6(char c);
+ inline void EvalCdataBody(char c);
+ inline void EvalCdataEnd1(char c);
+ inline void EvalCdataEnd2(char c);
+ inline void EvalAttribute(char c);
+ inline void EvalAttrName(char c);
+ inline void EvalAttrEq(char c);
+ inline void EvalAttrVal(char c);
+ inline void EvalAttrValSq(char c);
+ inline void EvalAttrValDq(char c);
+ inline void EvalLiteralTag(char c);
+ inline void EvalDirective(char c);
+
+ void MakeElement();
+ void MakeAttribute(bool has_value);
+ void FinishAttribute(char c, bool has_value, bool brief_close);
+
+ void EmitCdata();
+ void EmitComment();
+ void EmitLiteral();
+ void EmitTagOpen(bool allow_implicit_close);
+ void EmitTagClose(HtmlElement::CloseStyle close_style);
+ void EmitTagBriefClose();
+ void EmitDirective();
+ void Restart(char c);
+
+ // Emits a syntax error message.
+ void SyntaxError(const char* format, ...) INSTAWEB_PRINTF_FORMAT(2, 3);
+
+ // Takes an interned tag, and tries to find a matching HTML element on
+ // the stack. If it finds it, it pops all the intervening elements off
+ // the stack, issuing warnings for each discarded tag, the matching element
+ // is also popped off the stack, and returned.
+ //
+ // If the tag is not matched, then no mutations are done to the stack,
+ // and NULL is returned.
+ //
+ // The tag name should be interned.
+ // TODO(jmarantz): use type system
+ HtmlElement* PopElementMatchingTag(Atom tag);
+
+ HtmlElement* PopElement();
+ void CloseElement(HtmlElement* element, HtmlElement::CloseStyle close_style,
+ int line_nubmer);
+
+ // Minimal i18n analysis. With utf-8 and gb2312 we can do this
+ // context-free, and thus the method can be static. If we add
+ // more encodings we may need to turn this into a non-static method.
+ static inline bool IsI18nChar(char c) {return (((c) & 0x80) != 0); }
+
+ // Determines whether a character can be used in a tag name as first char ...
+ static inline bool IsLegalTagFirstChar(char c);
+ // ... or subsequent char.
+ static inline bool IsLegalTagChar(char c);
+
+ // Determines whether a character can be used in an attribute name.
+ static inline bool IsLegalAttrNameChar(char c);
+
+ // Determines whether a character can be used in an attribute value.
+ static inline bool IsLegalAttrValChar(char c);
+
+ // The lexer is implemented as a pure state machine. There is
+ // no lookahead. The state is understood primarily in this
+ // enum, although there are a few state flavors that are managed
+ // by the other member variables, notably: has_attr_value_ and
+ // attr_name_.empty(). Those could be eliminated by adding
+ // a few more explicit states.
+ enum State {
+ START,
+ TAG, // "<"
+ TAG_CLOSE, // "</"
+ TAG_CLOSE_TERMINATE, // "</x "
+ TAG_OPEN, // "<x"
+ TAG_BRIEF_CLOSE, // "<x/"
+ TAG_BRIEF_CLOSE_ATTR, // "<x /" or "<x y/" or "x y=/z" etc
+ COMMENT_START1, // "<!"
+ COMMENT_START2, // "<!-"
+ COMMENT_BODY, // "<!--"
+ COMMENT_END1, // "-"
+ COMMENT_END2, // "--"
+ CDATA_START1, // "<!["
+ CDATA_START2, // "<![C"
+ CDATA_START3, // "<![CD"
+ CDATA_START4, // "<![CDA"
+ CDATA_START5, // "<![CDAT"
+ CDATA_START6, // "<![CDATA"
+ CDATA_BODY, // "<![CDATA["
+ CDATA_END1, // "]"
+ CDATA_END2, // "]]"
+ TAG_ATTRIBUTE, // "<x "
+ TAG_ATTR_NAME, // "<x y"
+ TAG_ATTR_NAME_SPACE, // "<x y "
+ TAG_ATTR_EQ, // "<x y="
+ TAG_ATTR_VAL, // "<x y=x" value terminated by whitespace or >
+ TAG_ATTR_VALDQ, // '<x y="' value terminated by double-quote
+ TAG_ATTR_VALSQ, // "<x y='" value terminated by single-quote
+ LITERAL_TAG, // "<script " or "<iframe "
+ DIRECTIVE // "<!x"
+ };
+
+ HtmlParse* html_parse_;
+ State state_;
+ std::string token_; // accmulates tag names and comments
+ std::string literal_; // accumulates raw text to pass through
+ std::string attr_name_; // accumulates attribute name
+ std::string attr_value_; // accumulates attribute value
+ const char* attr_quote_; // accumulates quote used to delimit attribute
+ bool has_attr_value_; // distinguishes <a n=> from <a n>
+ HtmlElement* element_; // current element; used to collect attributes
+ int line_;
+ int tag_start_line_; // line at which we last transitioned to TAG state
+ std::string id_;
+ std::string literal_close_; // specific tag go close, e.g </script>
+
+ ContentType content_type_;
+ DocType doctype_;
+
+ AtomSet implicitly_closed_;
+ AtomSet non_brief_terminated_tags_;
+ AtomSet literal_tags_;
+ AtomSet optionally_closed_tags_;
+ std::vector<HtmlElement*> element_stack_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlLexer);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_HTML_LEXER_H_
diff --git a/trunk/src/net/instaweb/htmlparse/html_node.cc b/trunk/src/net/instaweb/htmlparse/html_node.cc
new file mode 100644
index 0000000..1f016a6
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/html_node.cc
@@ -0,0 +1,83 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "public/html_node.h"
+
+#include "net/instaweb/htmlparse/html_event.h"
+
+namespace net_instaweb {
+
+HtmlNode::~HtmlNode() {}
+
+void HtmlNode::MarkAsDead(const HtmlEventListIterator& end) {
+ live_ = false;
+ InvalidateIterators(end);
+}
+
+HtmlLeafNode::~HtmlLeafNode() {}
+
+void HtmlLeafNode::InvalidateIterators(const HtmlEventListIterator& end) {
+ set_iter(end);
+}
+
+HtmlCdataNode::~HtmlCdataNode() {}
+
+void HtmlCdataNode::SynthesizeEvents(const HtmlEventListIterator& iter,
+ HtmlEventList* queue) {
+ // We use -1 as a bogus line number, since the event is synthetic.
+ HtmlCdataEvent* event = new HtmlCdataEvent(this, -1);
+ set_iter(queue->insert(iter, event));
+}
+
+HtmlCharactersNode::~HtmlCharactersNode() {}
+
+void HtmlCharactersNode::SynthesizeEvents(const HtmlEventListIterator& iter,
+ HtmlEventList* queue) {
+ // We use -1 as a bogus line number, since the event is synthetic.
+ HtmlCharactersEvent* event = new HtmlCharactersEvent(this, -1);
+ set_iter(queue->insert(iter, event));
+}
+
+HtmlCommentNode::~HtmlCommentNode() {}
+
+void HtmlCommentNode::SynthesizeEvents(const HtmlEventListIterator& iter,
+ HtmlEventList* queue) {
+ // We use -1 as a bogus line number, since the event is synthetic.
+ HtmlCommentEvent* event = new HtmlCommentEvent(this, -1);
+ set_iter(queue->insert(iter, event));
+}
+
+HtmlIEDirectiveNode::~HtmlIEDirectiveNode() {}
+
+void HtmlIEDirectiveNode::SynthesizeEvents(const HtmlEventListIterator& iter,
+ HtmlEventList* queue) {
+ // We use -1 as a bogus line number, since the event is synthetic.
+ HtmlIEDirectiveEvent* event = new HtmlIEDirectiveEvent(this, -1);
+ set_iter(queue->insert(iter, event));
+}
+
+HtmlDirectiveNode::~HtmlDirectiveNode() {}
+
+void HtmlDirectiveNode::SynthesizeEvents(const HtmlEventListIterator& iter,
+ HtmlEventList* queue) {
+ // We use -1 as a bogus line number, since the event is synthetic.
+ HtmlDirectiveEvent* event = new HtmlDirectiveEvent(this, -1);
+ set_iter(queue->insert(iter, event));
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/htmlparse/html_parse.cc b/trunk/src/net/instaweb/htmlparse/html_parse.cc
new file mode 100644
index 0000000..bc3a4bc
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/html_parse.cc
@@ -0,0 +1,750 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/htmlparse/public/html_parse.h"
+
+#include <errno.h>
+#include <stdio.h>
+#include <string.h>
+#include <utility> // for std::pair
+
+#include "base/logging.h"
+#include "net/instaweb/htmlparse/html_event.h"
+#include "net/instaweb/htmlparse/html_lexer.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_escape.h"
+#include "net/instaweb/htmlparse/public/html_filter.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include <string>
+#include "net/instaweb/util/public/timer.h"
+
+namespace net_instaweb {
+
+HtmlParse::HtmlParse(MessageHandler* message_handler)
+ : lexer_(NULL), // Can't initialize here, since "this" should not be used
+ // in the initializer list (it generates an error in
+ // Visual Studio builds).
+ sequence_(0),
+ current_(queue_.end()),
+ deleted_current_(false),
+ message_handler_(message_handler),
+ line_number_(1),
+ need_sanity_check_(false),
+ coalesce_characters_(true),
+ need_coalesce_characters_(false),
+ parse_start_time_us_(0),
+ timer_(NULL) {
+ lexer_ = new HtmlLexer(this);
+ HtmlEscape::Init();
+}
+
+HtmlParse::~HtmlParse() {
+ delete lexer_;
+ ClearElements();
+}
+
+void HtmlParse::AddFilter(HtmlFilter* html_filter) {
+ filters_.push_back(html_filter);
+}
+
+HtmlEventListIterator HtmlParse::Last() {
+ HtmlEventListIterator p = queue_.end();
+ --p;
+ return p;
+}
+
+// Checks that the parent provided when creating the event's Node is
+// consistent with the position in the list. An alternative approach
+// here is to use this code and remove the explicit specification of
+// parents when constructing nodes.
+//
+// A complexity we will run into with that approach is that the queue_ is
+// cleared on a Flush, so we cannot reliably derive the correct parent
+// from the queue. However, the Lexer keeps an element stack across
+// flushes, and therefore can keep correct parent pointers. So we have
+// to inject pessimism in this process.
+//
+// Note that we also have sanity checks that run after each filter.
+void HtmlParse::CheckParentFromAddEvent(HtmlEvent* event) {
+ HtmlNode* node = event->GetNode();
+ if (node != NULL) {
+ message_handler_->Check(lexer_->Parent() == node->parent(),
+ "lexer_->Parent() != node->parent()");
+ }
+}
+
+// Testing helper method
+void HtmlParse::AddEvent(HtmlEvent* event) {
+ CheckParentFromAddEvent(event);
+ queue_.push_back(event);
+ need_sanity_check_ = true;
+ need_coalesce_characters_ = true;
+
+ // If this is a leaf-node event, we need to set the iterator of the
+ // corresponding leaf node to point to this event's position in the queue.
+ // If this is an element event, then the iterators of the element will get
+ // set in HtmlParse::AddElement and HtmlParse::CloseElement, so there's no
+ // need to do it here. If this is some other kind of event, there are no
+ // iterators to set.
+ HtmlLeafNode* leaf = event->GetLeafNode();
+ if (leaf != NULL) {
+ leaf->set_iter(Last());
+ message_handler_->Check(IsRewritable(leaf), "!IsRewritable(leaf)");
+ }
+}
+
+// Testing helper method
+void HtmlParse::SetCurrent(HtmlNode* node) {
+ current_ = node->begin();
+}
+
+HtmlCdataNode* HtmlParse::NewCdataNode(HtmlElement* parent,
+ const StringPiece& contents) {
+ HtmlCdataNode* cdata = new HtmlCdataNode(parent, contents, queue_.end());
+ nodes_.insert(cdata);
+ return cdata;
+}
+
+HtmlCharactersNode* HtmlParse::NewCharactersNode(HtmlElement* parent,
+ const StringPiece& literal) {
+ HtmlCharactersNode* characters =
+ new HtmlCharactersNode(parent, literal, queue_.end());
+ nodes_.insert(characters);
+ return characters;
+}
+
+HtmlCommentNode* HtmlParse::NewCommentNode(HtmlElement* parent,
+ const StringPiece& contents) {
+ HtmlCommentNode* comment = new HtmlCommentNode(parent, contents,
+ queue_.end());
+ nodes_.insert(comment);
+ return comment;
+}
+
+HtmlIEDirectiveNode* HtmlParse::NewIEDirectiveNode(
+ HtmlElement* parent, const StringPiece& contents) {
+ HtmlIEDirectiveNode* directive =
+ new HtmlIEDirectiveNode(parent, contents, queue_.end());
+ nodes_.insert(directive);
+ return directive;
+}
+
+HtmlDirectiveNode* HtmlParse::NewDirectiveNode(HtmlElement* parent,
+ const StringPiece& contents) {
+ HtmlDirectiveNode* directive = new HtmlDirectiveNode(parent, contents,
+ queue_.end());
+ nodes_.insert(directive);
+ return directive;
+}
+
+HtmlElement* HtmlParse::NewElement(HtmlElement* parent, Atom tag) {
+ HtmlElement* element = new HtmlElement(parent, tag, queue_.end(),
+ queue_.end());
+ nodes_.insert(element);
+ element->set_sequence(sequence_++);
+ return element;
+}
+
+void HtmlParse::AddElement(HtmlElement* element, int line_number) {
+ HtmlStartElementEvent* event =
+ new HtmlStartElementEvent(element, line_number);
+ AddEvent(event);
+ element->set_begin(Last());
+ element->set_begin_line_number(line_number);
+}
+
+void HtmlParse::StartParseId(const StringPiece& url, const StringPiece& id,
+ const ContentType& content_type) {
+ url.CopyToString(&url_);
+ GURL gurl(url_);
+ // TODO(jmaessen): warn and propagate upwards. This will require
+ // major changes to the callers.
+ message_handler_->Check(gurl.is_valid(), "Invalid url %s", url_.c_str());
+ gurl_.Swap(&gurl);
+ line_number_ = 1;
+ id.CopyToString(&id_);
+ if (timer_ != NULL) {
+ parse_start_time_us_ = timer_->NowUs();
+ InfoHere("HtmlParse::StartParse");
+ }
+ AddEvent(new HtmlStartDocumentEvent(line_number_));
+ lexer_->StartParse(id, content_type);
+}
+
+void HtmlParse::ShowProgress(const char* message) {
+ if (timer_ != NULL) {
+ long delta = static_cast<long>(timer_->NowUs() - parse_start_time_us_);
+ InfoHere("%ldus: HtmlParse::%s", delta, message);
+ }
+}
+
+void HtmlParse::FinishParse() {
+ lexer_->FinishParse();
+ AddEvent(new HtmlEndDocumentEvent(line_number_));
+ Flush();
+ ClearElements();
+ ShowProgress("FinishParse");
+}
+
+void HtmlParse::ParseText(const char* text, int size) {
+ lexer_->Parse(text, size);
+}
+
+// This is factored out of Flush() for testing purposes.
+void HtmlParse::ApplyFilter(HtmlFilter* filter) {
+ if (coalesce_characters_ && need_coalesce_characters_) {
+ CoalesceAdjacentCharactersNodes();
+ need_coalesce_characters_ = false;
+ }
+
+ ShowProgress(StrCat("ApplyFilter:", filter->Name()).c_str());
+ for (current_ = queue_.begin(); current_ != queue_.end(); ) {
+ HtmlEvent* event = *current_;
+ line_number_ = event->line_number();
+ event->Run(filter);
+ deleted_current_ = false;
+ ++current_;
+ }
+ filter->Flush();
+
+ if (need_sanity_check_) {
+ SanityCheck();
+ need_sanity_check_ = false;
+ }
+}
+
+void HtmlParse::CoalesceAdjacentCharactersNodes() {
+ ShowProgress("CoalesceAdjacentCharactersNodes");
+ HtmlCharactersNode* prev = NULL;
+ for (current_ = queue_.begin(); current_ != queue_.end(); ) {
+ HtmlEvent* event = *current_;
+ HtmlCharactersNode* node = event->GetCharactersNode();
+ if ((node != NULL) && (prev != NULL)) {
+ prev->Append(node->contents());
+ current_ = queue_.erase(current_); // returns element after erased
+ delete event;
+ node->MarkAsDead(queue_.end());
+ need_sanity_check_ = true;
+ } else {
+ ++current_;
+ prev = node;
+ }
+ }
+}
+
+void HtmlParse::CheckEventParent(HtmlEvent* event, HtmlElement* expect,
+ HtmlElement* actual) {
+ if ((expect != NULL) && (actual != expect)) {
+ std::string actual_buf, expect_buf, event_buf;
+ if (actual != NULL) {
+ actual->ToString(&actual_buf);
+ } else {
+ actual_buf = "(null)";
+ }
+ expect->ToString(&expect_buf);
+ event->ToString(&event_buf);
+ FatalErrorHere("HtmlElement Parents of %s do not match:\n"
+ "Actual: %s\n"
+ "Expected: %s\n",
+ event_buf.c_str(), actual_buf.c_str(), expect_buf.c_str());
+ }
+}
+
+void HtmlParse::SanityCheck() {
+ ShowProgress("SanityCheck");
+
+ // Sanity check that the Node parent-pointers are consistent with the
+ // begin/end-element events. This is done in a second pass to avoid
+ // confusion when the filter mutates the event-stream. Also note that
+ // a mid-HTML call to HtmlParse::Flush means that we may pop out beyond
+ // the stack we can detect in this event stream. This is represented
+ // here by an empty stack.
+ std::vector<HtmlElement*> element_stack;
+ HtmlElement* expect_parent = NULL;
+ for (current_ = queue_.begin(); current_ != queue_.end(); ++current_) {
+ HtmlEvent* event = *current_;
+
+ // Determine whether the event we are looking at is a StartElement,
+ // EndElement, or a leaf. We manipulate our temporary stack when
+ // we see StartElement and EndElement, and we always test to make
+ // sure the elements have the expected parent based on context, when
+ // we can figure out what the expected parent is.
+ HtmlElement* start_element = event->GetElementIfStartEvent();
+ HtmlElement* actual_parent = NULL;
+ if (start_element != NULL) {
+ CheckEventParent(event, expect_parent, start_element->parent());
+ message_handler_->Check(start_element->begin() == current_,
+ "start_element->begin() != current_");
+ message_handler_->Check(start_element->live(), "!start_element->live()");
+ element_stack.push_back(start_element);
+ expect_parent = start_element;
+ } else {
+ HtmlElement* end_element = event->GetElementIfEndEvent();
+ if (end_element != NULL) {
+ message_handler_->Check(end_element->end() == current_,
+ "end_element->end() != current_");
+ message_handler_->Check(end_element->live(),
+ "!end_element->live()");
+ if (!element_stack.empty()) {
+ // The element stack can be empty on End can happen due
+ // to this sequence:
+ // <tag1>
+ // FLUSH
+ // </tag1> <!-- tag1 close seen with empty stack -->
+ message_handler_->Check(element_stack.back() == end_element,
+ "element_stack.back() != end_element");
+ element_stack.pop_back();
+ }
+ actual_parent = end_element->parent();
+ expect_parent = element_stack.empty() ? NULL : element_stack.back();
+ CheckEventParent(event, expect_parent, end_element->parent());
+ } else {
+ // We only know for sure what the parents are once we have seen
+ // a start_element.
+ HtmlLeafNode* leaf_node = event->GetLeafNode();
+ if (leaf_node != NULL) { // Start/EndDocument are not leaf nodes
+ message_handler_->Check(leaf_node->live(), "!leaf_node->live()");
+ message_handler_->Check(leaf_node->end() == current_,
+ "leaf_node->end() != current_");
+ CheckEventParent(event, expect_parent, leaf_node->parent());
+ }
+ }
+ }
+ }
+}
+
+void HtmlParse::Flush() {
+ ShowProgress("Flush");
+
+ for (size_t i = 0; i < filters_.size(); ++i) {
+ HtmlFilter* filter = filters_[i];
+ ApplyFilter(filter);
+ }
+
+ // Detach all the elements from their events, as we are now invalidating
+ // the events, but not the elements.
+ for (current_ = queue_.begin(); current_ != queue_.end(); ++current_) {
+ HtmlEvent* event = *current_;
+ line_number_ = event->line_number();
+ HtmlElement* element = event->GetElementIfStartEvent();
+ if (element != NULL) {
+ element->set_begin(queue_.end());
+ } else {
+ element = event->GetElementIfEndEvent();
+ if (element != NULL) {
+ element->set_end(queue_.end());
+ } else {
+ HtmlLeafNode* leaf_node = event->GetLeafNode();
+ if (leaf_node != NULL) {
+ leaf_node->set_iter(queue_.end());
+ }
+ }
+ }
+ delete event;
+ }
+ queue_.clear();
+ need_sanity_check_ = false;
+ need_coalesce_characters_ = false;
+}
+
+void HtmlParse::InsertElementBeforeElement(const HtmlNode* existing_node,
+ HtmlNode* new_node) {
+ // begin() == queue_.end() -> this is an invalid element.
+ // TODO(sligocki): Rather than checks, we should probably return this as
+ // the status.
+ message_handler_->Check(existing_node->begin() != queue_.end(),
+ "InsertElementBeforeElement: existing_node invalid");
+ new_node->set_parent(existing_node->parent());
+ InsertElementBeforeEvent(existing_node->begin(), new_node);
+}
+
+void HtmlParse::InsertElementAfterElement(const HtmlNode* existing_node,
+ HtmlNode* new_node) {
+ message_handler_->Check(existing_node->end() != queue_.end(),
+ "InsertElementAfterElement: existing_node invalid");
+ new_node->set_parent(existing_node->parent());
+ InsertElementAfterEvent(existing_node->end(), new_node);
+}
+
+void HtmlParse::PrependChild(const HtmlElement* existing_parent,
+ HtmlNode* new_child) {
+ message_handler_->Check(existing_parent->begin() != queue_.end(),
+ "PrependChild: existing_parent invalid");
+ new_child->set_parent(const_cast<HtmlElement*>(existing_parent));
+ InsertElementAfterEvent(existing_parent->begin(), new_child);
+}
+
+void HtmlParse::AppendChild(const HtmlElement* existing_parent,
+ HtmlNode* new_child) {
+ message_handler_->Check(existing_parent->end() != queue_.end(),
+ "AppendChild: existing_parent invalid");
+ new_child->set_parent(const_cast<HtmlElement*>(existing_parent));
+ InsertElementBeforeEvent(existing_parent->end(), new_child);
+}
+
+void HtmlParse::InsertElementBeforeCurrent(HtmlNode* new_node) {
+ if (deleted_current_) {
+ FatalErrorHere("InsertElementBeforeCurrent after current has been "
+ "deleted.");
+ }
+ if ((new_node->parent() == NULL) && (current_ != queue_.end())) {
+ // Add a parent if one was not provided in new_node. We figure out
+ // what the parent should be by looking at current_. If that's an
+ // EndElement event, then that means that we are adding a new child
+ // of that element. In all other cases, we are adding a sibling.
+ HtmlEvent* current_event = *current_;
+ HtmlElement* end_element = current_event->GetElementIfEndEvent();
+ if (end_element != NULL) {
+ // The node pointed to by Current will be our new parent.
+ new_node->set_parent(end_element);
+ } else {
+ // The node pointed to by Current will be our new sibling, so
+ // we should grab its parent.
+ HtmlNode* node = current_event->GetNode();
+ message_handler_->Check(node != NULL,
+ "Cannot compute parent for new node");
+ new_node->set_parent(node->parent());
+ }
+ }
+ InsertElementBeforeEvent(current_, new_node);
+}
+
+void HtmlParse::InsertElementBeforeEvent(const HtmlEventListIterator& event,
+ HtmlNode* new_node) {
+ need_sanity_check_ = true;
+ need_coalesce_characters_ = true;
+ new_node->SynthesizeEvents(event, &queue_);
+}
+
+void HtmlParse::InsertElementAfterEvent(const HtmlEventListIterator& event,
+ HtmlNode* new_node) {
+ message_handler_->Check(event != queue_.end(), "event == queue_.end()");
+ HtmlEventListIterator next_event = event;
+ ++next_event;
+ InsertElementBeforeEvent(next_event, new_node);
+}
+
+
+void HtmlParse::InsertElementAfterCurrent(HtmlNode* new_node) {
+ if (deleted_current_) {
+ FatalErrorHere("InsertElementAfterCurrent after current has been "
+ "deleted.");
+ }
+ if (current_ == queue_.end()) {
+ FatalErrorHere("InsertElementAfterCurrent called with queue at end.");
+ }
+ ++current_;
+ InsertElementBeforeEvent(current_, new_node);
+
+ // We want to leave current_ pointing to the newly created element.
+ --current_;
+ message_handler_->Check((*current_)->GetNode() == new_node,
+ "(*current_)->GetNode() != new_node");
+}
+
+bool HtmlParse::AddParentToSequence(HtmlNode* first, HtmlNode* last,
+ HtmlElement* new_parent) {
+ bool added = false;
+ HtmlElement* original_parent = first->parent();
+ if (IsRewritable(first) && IsRewritable(last) &&
+ (last->parent() == original_parent) &&
+ (new_parent->begin() == queue_.end()) &&
+ (new_parent->end() == queue_.end())) {
+ InsertElementBeforeEvent(first->begin(), new_parent);
+ // This sequence of checks culminated in inserting the parent's begin
+ // and end before 'first'. Now we must mutate new_parent's end pointer
+ // to insert it after the last->end(). list::insert(iter) inserts
+ // *before* the iter, so we'll increment last->end().
+ HtmlEvent* end_element_event = *new_parent->end();
+ queue_.erase(new_parent->end());
+ HtmlEventListIterator p = last->end();
+ ++p;
+ new_parent->set_end(queue_.insert(p, end_element_event));
+ FixParents(first->begin(), last->end(), new_parent);
+ added = true;
+ need_sanity_check_ = true;
+ need_coalesce_characters_ = true;
+ }
+ return added;
+}
+
+void HtmlParse::FixParents(const HtmlEventListIterator& begin,
+ const HtmlEventListIterator& end_inclusive,
+ HtmlElement* new_parent) {
+ HtmlEvent* event = *begin;
+ HtmlNode* first = event->GetNode();
+ HtmlElement* original_parent = first->parent();
+ // Loop over all the nodes from begin to end, inclusive,
+ // and set the parent pointer for the node, if there is one. A few
+ // event types don't have HtmlNodes, such as Comments and IEDirectives.
+ message_handler_->Check(end_inclusive != queue_.end(),
+ "end_inclusive == queue_.end()");
+ HtmlEventListIterator end = end_inclusive;
+ ++end;
+ for (HtmlEventListIterator p = begin; p != end; ++p) {
+ HtmlNode* node = (*p)->GetNode();
+ if ((node != NULL) && (node->parent() == original_parent)) {
+ node->set_parent(new_parent);
+ }
+ }
+}
+
+bool HtmlParse::MoveCurrentInto(HtmlElement* new_parent) {
+ bool moved = false;
+ HtmlNode* node = (*current_)->GetNode();
+ if ((node != NULL) && (node != new_parent) &&
+ IsRewritable(node) && IsRewritable(new_parent)) {
+ HtmlEventListIterator begin = node->begin();
+ HtmlEventListIterator end = node->end();
+ ++end; // splice is non-inclusive for the 'end' iterator.
+
+ // Manipulate current_ so that when Flush() iterates it lands
+ // you on object after current_'s original position, rather
+ // than re-iterating over the new_parent's EndElement event.
+ current_ = end;
+ queue_.splice(new_parent->end(), queue_, begin, end);
+ --current_;
+
+ // TODO(jmarantz): According to
+ // http://www.cplusplus.com/reference/stl/list/splice/
+ // the moved iterators are no longer valid, and we
+ // are retaining them in the HtmlNode, so we need to fix them.
+ //
+ // However, in practice they appear to remain valid. And
+ // I can't think of a reason they should be invalidated,
+ // as the iterator is a pointer to a node structure with
+ // next/prev pointers. splice can mutate the next/prev pointers
+ // in place.
+ //
+ // See http://stackoverflow.com/questions/143156
+
+ FixParents(node->begin(), node->end(), new_parent);
+ moved = true;
+ need_sanity_check_ = true;
+ need_coalesce_characters_ = true;
+ }
+ return moved;
+}
+
+bool HtmlParse::DeleteElement(HtmlNode* node) {
+ bool deleted = false;
+ if (IsRewritable(node)) {
+ bool done = false;
+ // If node is an HtmlLeafNode, then begin() and end() might be the same.
+ for (HtmlEventListIterator p = node->begin(); !done; ) {
+ // We want to include end, so once p == end we still have to do one more
+ // iteration.
+ done = (p == node->end());
+
+ // Clean up any nested elements/leaves as we get to their 'end' event.
+ HtmlEvent* event = *p;
+ HtmlNode* nested_node = event->GetElementIfEndEvent();
+ if (nested_node == NULL) {
+ nested_node = event->GetLeafNode();
+ }
+ if (nested_node != NULL) {
+ std::set<HtmlNode*>::iterator iter = nodes_.find(nested_node);
+ message_handler_->Check(iter != nodes_.end(), "iter == nodes_.end()");
+ message_handler_->Check(nested_node->live(), "!nested_node->live()");
+ nested_node->MarkAsDead(queue_.end());
+ }
+
+ // Check if we're about to delete the current event.
+ bool move_current = (p == current_);
+ p = queue_.erase(p);
+ if (move_current) {
+ current_ = p; // p is the event *after* the old current.
+ --current_; // Go to *previous* event so that we don't skip p.
+ deleted_current_ = true;
+ line_number_ = (*current_)->line_number();
+ }
+ delete event;
+ }
+
+ // Our iteration should have covered the passed-in element as well.
+ message_handler_->Check(!node->live(), "node->live()");
+ deleted = true;
+ need_sanity_check_ = true;
+ need_coalesce_characters_ = true;
+ }
+ return deleted;
+}
+
+bool HtmlParse::DeleteSavingChildren(HtmlElement* element) {
+ bool deleted = false;
+ if (IsRewritable(element)) {
+ HtmlElement* new_parent = element->parent();
+ HtmlEventListIterator first = element->begin();
+ ++first;
+ HtmlEventListIterator last = element->end();
+ if (first != last) {
+ --last;
+ FixParents(first, last, new_parent);
+ queue_.splice(element->begin(), queue_, first, element->end());
+ need_sanity_check_ = true;
+ need_coalesce_characters_ = true;
+ }
+ deleted = DeleteElement(element);
+ }
+ return deleted;
+}
+
+bool HtmlParse::ReplaceNode(HtmlNode* existing_node, HtmlNode* new_node) {
+ bool replaced = false;
+ if (IsRewritable(existing_node)) {
+ InsertElementBeforeElement(existing_node, new_node);
+ replaced = DeleteElement(existing_node);
+ message_handler_->Check(replaced, "!replaced");
+ }
+ return replaced;
+}
+
+bool HtmlParse::IsRewritable(const HtmlNode* node) const {
+ return IsInEventWindow(node->begin()) && IsInEventWindow(node->end());
+}
+
+bool HtmlParse::IsInEventWindow(const HtmlEventListIterator& iter) const {
+ return iter != queue_.end();
+}
+
+void HtmlParse::ClearElements() {
+ for (std::set<HtmlNode*>::iterator p = nodes_.begin(),
+ e = nodes_.end(); p != e; ++p) {
+ HtmlNode* node = *p;
+ delete node;
+ }
+ nodes_.clear();
+}
+
+void HtmlParse::DebugPrintQueue() {
+ for (HtmlEventList::iterator p = queue_.begin(), e = queue_.end();
+ p != e; ++p) {
+ std::string buf;
+ HtmlEvent* event = *p;
+ event->ToString(&buf);
+ long node_ptr = reinterpret_cast<long>(event->GetNode());
+ if (p == current_) {
+ fprintf(stdout, "* %s (0x%lx)\n", buf.c_str(), node_ptr);
+ } else {
+ fprintf(stdout, " %s (0x%lx)\n", buf.c_str(), node_ptr);
+ }
+ }
+ fflush(stdout);
+}
+
+bool HtmlParse::IsImplicitlyClosedTag(Atom tag) const {
+ return lexer_->IsImplicitlyClosedTag(tag);
+}
+
+bool HtmlParse::TagAllowsBriefTermination(Atom tag) const {
+ return lexer_->TagAllowsBriefTermination(tag);
+}
+
+const DocType& HtmlParse::doctype() const {
+ return lexer_->doctype();
+}
+
+void HtmlParse::InfoV(
+ const char* file, int line, const char *msg, va_list args) {
+ message_handler_->InfoV(file, line, msg, args);
+}
+
+void HtmlParse::WarningV(
+ const char* file, int line, const char *msg, va_list args) {
+ message_handler_->WarningV(file, line, msg, args);
+}
+
+void HtmlParse::ErrorV(
+ const char* file, int line, const char *msg, va_list args) {
+ message_handler_->ErrorV(file, line, msg, args);
+}
+
+void HtmlParse::FatalErrorV(
+ const char* file, int line, const char* msg, va_list args) {
+ message_handler_->FatalErrorV(file, line, msg, args);
+}
+
+void HtmlParse::Info(const char* file, int line, const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ InfoV(file, line, msg, args);
+ va_end(args);
+}
+
+void HtmlParse::Warning(const char* file, int line, const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ WarningV(file, line, msg, args);
+ va_end(args);
+}
+
+void HtmlParse::Error(const char* file, int line, const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ ErrorV(file, line, msg, args);
+ va_end(args);
+}
+
+void HtmlParse::FatalError(const char* file, int line, const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ FatalErrorV(file, line, msg, args);
+ va_end(args);
+}
+
+void HtmlParse::InfoHere(const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ InfoHereV(msg, args);
+ va_end(args);
+}
+
+void HtmlParse::WarningHere(const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ WarningHereV(msg, args);
+ va_end(args);
+}
+
+void HtmlParse::ErrorHere(const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ ErrorHereV(msg, args);
+ va_end(args);
+}
+
+void HtmlParse::FatalErrorHere(const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ FatalErrorHereV(msg, args);
+ va_end(args);
+}
+
+void HtmlParse::CloseElement(
+ HtmlElement* element, HtmlElement::CloseStyle close_style,
+ int line_number) {
+ HtmlEndElementEvent* end_event =
+ new HtmlEndElementEvent(element, line_number);
+ element->set_close_style(close_style);
+ AddEvent(end_event);
+ element->set_end(Last());
+ element->set_end_line_number(line_number);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/htmlparse/html_parse_test.cc b/trunk/src/net/instaweb/htmlparse/html_parse_test.cc
new file mode 100644
index 0000000..8e03c38
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/html_parse_test.cc
@@ -0,0 +1,724 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the html reader/writer to ensure that a few tricky
+// constructs come through without corruption.
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/htmlparse/public/html_parse_test_base.h"
+#include "net/instaweb/htmlparse/html_event.h"
+#include "net/instaweb/htmlparse/html_testing_peer.h"
+
+namespace net_instaweb {
+
+class HtmlParseTest : public HtmlParseTestBase {
+ virtual bool AddBody() const { return true; }
+};
+
+class HtmlParseTestNoBody : public HtmlParseTestBase {
+ virtual bool AddBody() const { return false; }
+};
+
+TEST_F(HtmlParseTest, AvoidFalseXmlComment) {
+ ValidateNoChanges("avoid_false_xml_comment",
+ "<script type=\"text/javascript\">\n"
+ "// <!-- this looks like a comment but is not\n"
+ "</script>");
+}
+
+TEST_F(HtmlParseTest, RetainBogusEndTag) {
+ ValidateNoChanges("bogus_end_tag",
+ "<script language=\"JavaScript\" type=\"text/javascript\">\n"
+ "<!--\n"
+ "var s = \"</retain_bogus_end_tag>\";\n"
+ "// -->\n"
+ "</script>");
+}
+
+TEST_F(HtmlParseTest, AmpersandInHref) {
+ // Note that we will escape the "&" in the href.
+ ValidateNoChanges("ampersand_in_href",
+ "<a href=\"http://myhost.com/path?arg1=val1&arg2=val2\">Hello</a>");
+}
+
+TEST_F(HtmlParseTest, CorrectTaggify) {
+ // Don't turn <2 -> <2>
+ ValidateNoChanges("no_taggify_digit", "<p>1<2</p>");
+ ValidateNoChanges("no_taggify_unicode", "<p>☃<☕</p>");
+ ValidateExpected("taggify_letter", "<p>x<y</p>", "<p>x<y></p>");
+
+ ValidateExpected("taggify_letter+digit", "<p>x1<y2</p>", "<p>x1<y2></p>");
+ ValidateExpected("taggify_letter+unicode", "<p>x☃<y☕</p>", "<p>x☃<y☕></p>");
+
+ ValidateNoChanges("no_taggify_digit+letter", "<p>1x<2y</p>");
+ ValidateNoChanges("no_taggify_unicode+letter", "<p>☃x<☕y</p>");
+
+ // Found on http://www.taobao.com/
+ // Don't turn <1... -> <1...>
+ ValidateNoChanges("taobao", "<a>1+1<1母婴全场加1元超值购</a>");
+}
+
+TEST_F(HtmlParseTest, BooleanSpaceCloseInTag) {
+ ValidateExpected("bool_space_close", "<a b >foo</a>", "<a b>foo</a>");
+ ValidateNoChanges("bool_close", "<a b>foo</a>");
+ ValidateExpected("space_close_sq", "<a b='c' >foo</a>", "<a b='c'>foo</a>");
+ ValidateExpected("space_close_dq",
+ "<a b=\"c\" >foo</a>", "<a b=\"c\">foo</a>");
+ ValidateExpected("space_close_nq", "<a b=c >foo</a>", "<a b=c>foo</a>");
+ // Distilled from http://www.gougou.com/
+ // Unclear exactly what we should do here, maybe leave it as it was without
+ // the space?
+ ValidateExpected("allow_semicolon",
+ "<a onclick='return m(this)'; >foo</a>",
+ "<a onclick='return m(this)' ;>foo</a>");
+}
+
+class AttrValuesSaverFilter : public EmptyHtmlFilter {
+ public:
+ AttrValuesSaverFilter() { }
+
+ virtual void StartElement(HtmlElement* element) {
+ for (int i = 0; i < element->attribute_size(); ++i) {
+ value_ += element->attribute(i).value();
+ }
+ }
+
+ const std::string& value() { return value_; }
+ virtual const char* Name() const { return "attr_saver"; }
+
+ private:
+ std::string value_;
+
+ DISALLOW_COPY_AND_ASSIGN(AttrValuesSaverFilter);
+};
+
+TEST_F(HtmlParseTest, EscapedSingleQuote) {
+ AttrValuesSaverFilter attr_saver;
+ html_parse_.AddFilter(&attr_saver);
+ Parse("escaped_single_quote",
+ "<img src='my'single_quoted_image.jpg'/>");
+ EXPECT_EQ("my'single_quoted_image.jpg", attr_saver.value());
+}
+
+TEST_F(HtmlParseTest, UnclosedQuote) {
+ // In this test, the system automatically closes the 'a' tag, which
+ // didn't really get closed in the input text. The exact syntax
+ // of the expected results not critical, as long as the parser recovers
+ // and does not crash.
+ //
+ // TODO(jmarantz): test error reporting.
+ ValidateNoChanges("unclosed_quote",
+ "<div>\n"
+ " <a href=\"http://myhost.com/path?arg1=val1&arg2=val2>Hello</a>\n"
+ "</div>\n"
+ "<p>next token</p>"
+ "</body></html>\n"
+ "\"></a></div>");
+}
+
+TEST_F(HtmlParseTest, NestedDivInBr) {
+ ValidateNoChanges("nested_div_in_br",
+ "<br><div>hello</div></br>");
+}
+
+// bug 2465145 - Sequential defaulted attribute tags lost
+TEST_F(HtmlParseTest, SequentialDefaultedTagsLost) {
+ // This test cannot work with libxml, but since we use our own
+ // parser we can make it work. See
+ // https://bugzilla.gnome.org/show_bug.cgi?id=611655
+ ValidateNoChanges("sequential_defaulted_attribute_tags_lost",
+ "<select>\n"
+ " <option value=\"&cat=244\">Other option</option>\n"
+ " <option value selected style=\"color: #ccc;\">Default option"
+ "</option>\n"
+ "</select>");
+
+ // Illegal attribute "http://www.yahoo.com" mangled by parser into
+ // "http:", although if the parser changes how it mangles that somehow
+ // it's fine to regold.
+ ValidateNoChanges("yahoo",
+ "<a href=\"#\" http://www.yahoo.com "
+ "class=\"pa-btn-open hide-textindent\">yahoo</a>");
+
+ // Here's another interesting thing from the bug testcase.
+ // Specifying a literal "&" without a recognized sequence
+ // following it gets parsed correctly by libxml2, and then
+ // re-encoded by our writer as &. That's fine; let's
+ // make sure that doesn't change.
+ ValidateNoChanges("amp_cat",
+ "<option value=\"&cat=244\">other</option>");
+}
+
+// bug 2465201 : some html constructs do not need ';' termination.
+// Fixed by providing own lexer.
+TEST_F(HtmlParseTest, UnterminatedTokens) {
+ // the termination semicolons should be added in the output.
+ ValidateNoChanges("unterminated_tokens",
+ "<p>Look at the non breaking space: \" \"</p>");
+}
+
+// bug 2467040 : keep ampersands and quotes encoded
+TEST_F(HtmlParseTest, EncodeAmpersandsAndQuotes) {
+ ValidateNoChanges("ampersands_in_text",
+ "<p>This should be a string '&amp;' not a single ampersand.</p>");
+ ValidateNoChanges("ampersands_in_values",
+ "<img alt=\"This should be a string '&amp;' "
+ "not a single ampersand.\"/>");
+ ValidateNoChanges("quotes",
+ "<p>Clicking <a href=\"javascript: alert("Alert works!");\">"
+ "here</a> should pop up an alert box.</p>");
+}
+
+// bug 2508334 : encoding unicode in general
+TEST_F(HtmlParseTest, EncodeUnicode) {
+ ValidateNoChanges("unicode_in_text",
+ "<p>Non-breaking space: ' '</p>\n"
+ "<p>Alpha: 'α'</p>\n"
+ "<p>Unicode #54321: '퐱'</p>\n");
+}
+
+TEST_F(HtmlParseTest, ImplicitExplicitClose) {
+ // The lexer/printer preserves the input syntax, making it easier
+ // to diff inputs & outputs.
+ //
+ // TODO(jmarantz): But we can have a rewrite pass that eliminates
+ // the superfluous "/>".
+ ValidateNoChanges("one_brief_one_implicit_input",
+ "<input type=\"text\" name=\"username\">"
+ "<input type=\"password\" name=\"password\"/>");
+}
+
+TEST_F(HtmlParseTest, OpenBracketAfterQuote) {
+ // '<' after '"' in attr value
+ const char input[] =
+ "<input type=\"text\" name=\"username\""
+ "<input type=\"password\" name=\"password\"/>";
+ const char expected[] =
+ "<input type=\"text\" name=\"username\">" // note added '>'
+ "<input type=\"password\" name=\"password\"/>";
+ ValidateExpected("open_bracket_after_quote", input, expected);
+}
+
+TEST_F(HtmlParseTest, OpenBracketUnquoted) {
+ // '<' after after unquoted attr value
+ const char input[] =
+ "<input type=\"text\" name=username"
+ "<input type=\"password\" name=\"password\"/>";
+ const char expected[] =
+ "<input type=\"text\" name=username>" // note added '>'
+ "<input type=\"password\" name=\"password\"/>";
+ ValidateExpected("open_bracket_unquoted", input, expected);
+}
+
+TEST_F(HtmlParseTest, OpenBracketAfterEquals) {
+ // '<' after after unquoted attr value
+ const char input[] =
+ "<input type=\"text\" name="
+ "<input type=\"password\" name=\"password\"/>";
+ const char expected[] =
+ "<input type=\"text\" name=>" // note added '>'
+ "<input type=\"password\" name=\"password\"/>";
+ ValidateExpected("open_brack_after_equals", input, expected);
+}
+
+TEST_F(HtmlParseTest, OpenBracketAfterName) {
+ // '<' after after unquoted attr value
+ const char input[] =
+ "<input type=\"text\" name"
+ "<input type=\"password\" name=\"password\"/>";
+ const char expected[] =
+ "<input type=\"text\" name>" // note added '>'
+ "<input type=\"password\" name=\"password\"/>";
+ ValidateExpected("open_brack_after_name", input, expected);
+}
+
+TEST_F(HtmlParseTest, OpenBracketAfterSpace) {
+ // '<' after after unquoted attr value
+ const char input[] =
+ "<input type=\"text\" "
+ "<input type=\"password\" name=\"password\"/>";
+ const char expected[] =
+ "<input type=\"text\">" // note added '>'
+ "<input type=\"password\" name=\"password\"/>";
+ ValidateExpected("open_brack_after_name", input, expected);
+}
+
+// bug 2508140 : <noscript> in <head>
+TEST_F(HtmlParseTestNoBody, NoscriptInHead) {
+ // Some real websites (ex: google.com) have <noscript> in the <head> even
+ // though this is technically illegal acording to the HTML4 spec.
+ // We should support the case in use.
+ ValidateNoChanges("noscript_in_head",
+ "<head><noscript><title>You don't have JS enabled :(</title>"
+ "</noscript></head>");
+}
+
+
+// Bool that is auto-initialized to false
+class Bool {
+ public:
+ Bool() : value_(false) {}
+ Bool(bool value) : value_(value) {} // Copy constructor // NOLINT
+ const bool Test() const { return value_; }
+
+ private:
+ bool value_;
+};
+
+// Class simply keeps track of which handlers have been called.
+class HandlerCalledFilter : public HtmlFilter {
+ public:
+ HandlerCalledFilter() { }
+
+ virtual void StartDocument() { called_start_document_ = true; }
+ virtual void EndDocument() { called_end_document_ = true;}
+ virtual void StartElement(HtmlElement* element) {
+ called_start_element_ = true;
+ }
+ virtual void EndElement(HtmlElement* element) {
+ called_end_element_ = true;
+ }
+ virtual void Cdata(HtmlCdataNode* cdata) { called_cdata_ = true; }
+ virtual void Comment(HtmlCommentNode* comment) { called_comment_ = true; }
+ virtual void IEDirective(HtmlIEDirectiveNode* directive) {
+ called_ie_directive_ = true;
+ }
+ virtual void Characters(HtmlCharactersNode* characters) {
+ called_characters_ = true;
+ }
+ virtual void Directive(HtmlDirectiveNode* directive) {
+ called_directive_ = true;
+ }
+ virtual void Flush() { called_flush_ = true; }
+ virtual const char* Name() const { return "HandlerCalled"; }
+
+ Bool called_start_document_;
+ Bool called_end_document_;
+ Bool called_start_element_;
+ Bool called_end_element_;
+ Bool called_cdata_;
+ Bool called_comment_;
+ Bool called_ie_directive_;
+ Bool called_characters_;
+ Bool called_directive_;
+ Bool called_flush_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(HandlerCalledFilter);
+};
+
+class HandlerCalledTest : public HtmlParseTest {
+ protected:
+ HandlerCalledTest() {
+ html_parse_.AddFilter(&handler_called_filter_);
+ }
+
+ HandlerCalledFilter handler_called_filter_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(HandlerCalledTest);
+};
+
+// Check that StartDocument and EndDocument were called for filters.
+TEST_F(HandlerCalledTest, StartEndDocumentCalled) {
+ Parse("start_end_document_called", "");
+ EXPECT_TRUE(handler_called_filter_.called_start_document_.Test());
+ EXPECT_TRUE(handler_called_filter_.called_end_document_.Test());
+}
+
+TEST_F(HandlerCalledTest, StartEndElementCalled) {
+ Parse("start_end_element_called", "<p>...</p>");
+ EXPECT_TRUE(handler_called_filter_.called_start_element_.Test());
+ EXPECT_TRUE(handler_called_filter_.called_end_element_.Test());
+}
+
+TEST_F(HandlerCalledTest, CdataCalled) {
+ Parse("cdata_called", "<![CDATA[...]]>");
+ // Looks like a directive, but isn't.
+ EXPECT_FALSE(handler_called_filter_.called_directive_.Test());
+ EXPECT_TRUE(handler_called_filter_.called_cdata_.Test());
+}
+
+TEST_F(HandlerCalledTest, CommentCalled) {
+ Parse("comment_called", "<!--...-->");
+ EXPECT_TRUE(handler_called_filter_.called_comment_.Test());
+}
+
+TEST_F(HandlerCalledTest, IEDirectiveCalled) {
+ Parse("ie_directive_called", "<!--[if IE]>...<![endif]-->");
+ // Looks like a comment, but isn't.
+ EXPECT_FALSE(handler_called_filter_.called_comment_.Test());
+ EXPECT_TRUE(handler_called_filter_.called_ie_directive_.Test());
+}
+
+// Unit tests for event-list manipulation. In these tests, we do not parse
+// HTML input text, but instead create two 'Characters' nodes and use the
+// event-list manipulation methods and make sure they render as expected.
+class EventListManipulationTest : public HtmlParseTest {
+ protected:
+ EventListManipulationTest() { }
+
+ virtual void SetUp() {
+ HtmlParseTest::SetUp();
+ static const char kUrl[] = "http://html.parse.test/event_list_test.html";
+ html_parse_.StartParse(kUrl);
+ node1_ = html_parse_.NewCharactersNode(NULL, "1");
+ HtmlTestingPeer::AddEvent(&html_parse_,
+ new HtmlCharactersEvent(node1_, -1));
+ node2_ = html_parse_.NewCharactersNode(NULL, "2");
+ node3_ = html_parse_.NewCharactersNode(NULL, "3");
+ // Note: the last 2 are not added in SetUp.
+ }
+
+ virtual void TearDown() {
+ html_parse_.FinishParse();
+ HtmlParseTest::TearDown();
+ }
+
+ void CheckExpected(const std::string& expected) {
+ SetupWriter();
+ html_parse()->ApplyFilter(html_writer_filter_.get());
+ EXPECT_EQ(expected, output_buffer_);
+ }
+
+ HtmlCharactersNode* node1_;
+ HtmlCharactersNode* node2_;
+ HtmlCharactersNode* node3_;
+ private:
+ DISALLOW_COPY_AND_ASSIGN(EventListManipulationTest);
+};
+
+TEST_F(EventListManipulationTest, TestReplace) {
+ EXPECT_TRUE(html_parse_.ReplaceNode(node1_, node2_));
+ CheckExpected("2");
+}
+
+TEST_F(EventListManipulationTest, TestInsertElementBeforeElement) {
+ HtmlTestingPeer::set_coalesce_characters(&html_parse_, false);
+ html_parse_.InsertElementBeforeElement(node1_, node2_);
+ CheckExpected("21");
+ html_parse_.InsertElementBeforeElement(node1_, node3_);
+ CheckExpected("231");
+}
+
+TEST_F(EventListManipulationTest, TestInsertElementAfterElement) {
+ HtmlTestingPeer::set_coalesce_characters(&html_parse_, false);
+ html_parse_.InsertElementAfterElement(node1_, node2_);
+ CheckExpected("12");
+ html_parse_.InsertElementAfterElement(node1_, node3_);
+ CheckExpected("132");
+}
+
+TEST_F(EventListManipulationTest, TestInsertElementBeforeCurrent) {
+ HtmlTestingPeer::set_coalesce_characters(&html_parse_, false);
+ html_parse_.InsertElementBeforeCurrent(node2_);
+ // Current is left at queue_.end() after the AddEvent.
+ CheckExpected("12");
+
+ HtmlTestingPeer::SetCurrent(&html_parse_, node1_);
+ html_parse_.InsertElementBeforeCurrent(node3_);
+ CheckExpected("312");
+}
+
+TEST_F(EventListManipulationTest, TestInsertElementAfterCurrent) {
+ HtmlTestingPeer::set_coalesce_characters(&html_parse_, false);
+ HtmlTestingPeer::SetCurrent(&html_parse_, node1_);
+ html_parse_.InsertElementAfterCurrent(node2_);
+ // Note that if we call CheckExpected here it will mutate current_.
+ html_parse_.InsertElementAfterCurrent(node3_);
+ CheckExpected("123");
+}
+
+TEST_F(EventListManipulationTest, TestDeleteOnly) {
+ html_parse_.DeleteElement(node1_);
+ CheckExpected("");
+}
+
+TEST_F(EventListManipulationTest, TestDeleteFirst) {
+ HtmlTestingPeer::set_coalesce_characters(&html_parse_, false);
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node2_, -1));
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node3_, -1));
+ html_parse_.DeleteElement(node1_);
+ CheckExpected("23");
+ html_parse_.DeleteElement(node2_);
+ CheckExpected("3");
+ html_parse_.DeleteElement(node3_);
+ CheckExpected("");
+}
+
+TEST_F(EventListManipulationTest, TestDeleteLast) {
+ HtmlTestingPeer::set_coalesce_characters(&html_parse_, false);
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node2_, -1));
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node3_, -1));
+ html_parse_.DeleteElement(node3_);
+ CheckExpected("12");
+ html_parse_.DeleteElement(node2_);
+ CheckExpected("1");
+ html_parse_.DeleteElement(node1_);
+ CheckExpected("");
+}
+
+TEST_F(EventListManipulationTest, TestDeleteMiddle) {
+ HtmlTestingPeer::set_coalesce_characters(&html_parse_, false);
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node2_, -1));
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node3_, -1));
+ html_parse_.DeleteElement(node2_);
+ CheckExpected("13");
+}
+
+// Note that an unconditionaly sanity check runs after every
+// filter, verifying that all the parent-pointers are correct.
+// CheckExpected applies the HtmlWriterFilter, so it runs the
+// parent-pointer check.
+TEST_F(EventListManipulationTest, TestAddParentToSequence) {
+ HtmlTestingPeer::set_coalesce_characters(&html_parse_, false);
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node2_, -1));
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node3_, -1));
+ HtmlElement* div = html_parse_.NewElement(NULL, html_parse_.Intern("div"));
+ EXPECT_TRUE(html_parse_.AddParentToSequence(node1_, node3_, div));
+ CheckExpected("<div>123</div>");
+
+ // Now interpose a span between the div and the Characeters nodes.
+ HtmlElement* span = html_parse_.NewElement(div, html_parse_.Intern("span"));
+ EXPECT_TRUE(html_parse_.AddParentToSequence(node1_, node2_, span));
+ CheckExpected("<div><span>12</span>3</div>");
+
+ // Next, add an HTML block above the div. Note that we pass 'div' in as
+ // both 'first' and 'last'.
+ HtmlElement* html = html_parse_.NewElement(NULL, html_parse_.Intern("html"));
+ EXPECT_TRUE(html_parse_.AddParentToSequence(div, div, html));
+ CheckExpected("<html><div><span>12</span>3</div></html>");
+}
+
+TEST_F(EventListManipulationTest, TestPrependChild) {
+ HtmlTestingPeer::set_coalesce_characters(&html_parse_, false);
+ HtmlElement* div = html_parse_.NewElement(NULL, html_parse_.Intern("div"));
+ html_parse_.InsertElementBeforeCurrent(div);
+ CheckExpected("1<div></div>");
+
+ html_parse_.PrependChild(div, node2_);
+ CheckExpected("1<div>2</div>");
+ html_parse_.PrependChild(div, node3_);
+ CheckExpected("1<div>32</div>");
+
+ // TODO(sligocki): Test with elements that don't explicitly end like image.
+}
+
+TEST_F(EventListManipulationTest, TestAppendChild) {
+ HtmlTestingPeer::set_coalesce_characters(&html_parse_, false);
+ HtmlElement* div = html_parse_.NewElement(NULL, html_parse_.Intern("div"));
+ html_parse_.InsertElementBeforeCurrent(div);
+ CheckExpected("1<div></div>");
+
+ html_parse_.AppendChild(div, node2_);
+ CheckExpected("1<div>2</div>");
+ html_parse_.AppendChild(div, node3_);
+ CheckExpected("1<div>23</div>");
+
+ // TODO(sligocki): Test with elements that don't explicitly end like image.
+}
+
+TEST_F(EventListManipulationTest, TestAddParentToSequenceDifferentParents) {
+ HtmlTestingPeer::set_coalesce_characters(&html_parse_, false);
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node2_, -1));
+ HtmlElement* div = html_parse_.NewElement(NULL, html_parse_.Intern("div"));
+ EXPECT_TRUE(html_parse_.AddParentToSequence(node1_, node2_, div));
+ CheckExpected("<div>12</div>");
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node3_, -1));
+ CheckExpected("<div>12</div>3");
+ EXPECT_FALSE(html_parse_.AddParentToSequence(node2_, node3_, div));
+}
+
+TEST_F(EventListManipulationTest, TestDeleteGroup) {
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node2_, -1));
+ HtmlElement* div = html_parse_.NewElement(NULL, html_parse_.Intern("div"));
+ EXPECT_TRUE(html_parse_.AddParentToSequence(node1_, node2_, div));
+ CheckExpected("<div>12</div>");
+ html_parse_.DeleteElement(div);
+ CheckExpected("");
+}
+
+TEST_F(EventListManipulationTest, TestMoveElementIntoParent1) {
+ HtmlElement* head = html_parse_.NewElement(NULL, html_parse_.Intern("head"));
+ EXPECT_TRUE(html_parse_.AddParentToSequence(node1_, node1_, head));
+ CheckExpected("<head>1</head>");
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node2_, -1));
+ HtmlElement* div = html_parse_.NewElement(NULL, html_parse_.Intern("div"));
+ EXPECT_TRUE(html_parse_.AddParentToSequence(node2_, node2_, div));
+ CheckExpected("<head>1</head><div>2</div>");
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node3_, -1));
+ CheckExpected("<head>1</head><div>2</div>3");
+ HtmlTestingPeer::SetCurrent(&html_parse_, div);
+ EXPECT_TRUE(html_parse_.MoveCurrentInto(head));
+ CheckExpected("<head>1<div>2</div></head>3");
+}
+
+TEST_F(EventListManipulationTest, TestMoveElementIntoParent2) {
+ HtmlTestingPeer::set_coalesce_characters(&html_parse_, false);
+ HtmlElement* head = html_parse_.NewElement(NULL, html_parse_.Intern("head"));
+ EXPECT_TRUE(html_parse_.AddParentToSequence(node1_, node1_, head));
+ CheckExpected("<head>1</head>");
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node2_, -1));
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node3_, -1));
+ CheckExpected("<head>1</head>23");
+ HtmlElement* div = html_parse_.NewElement(NULL, html_parse_.Intern("div"));
+ EXPECT_TRUE(html_parse_.AddParentToSequence(node3_, node3_, div));
+ CheckExpected("<head>1</head>2<div>3</div>");
+ HtmlTestingPeer::SetCurrent(&html_parse_, div);
+ EXPECT_TRUE(html_parse_.MoveCurrentInto(head));
+ CheckExpected("<head>1<div>3</div></head>2");
+ EXPECT_TRUE(html_parse_.DeleteSavingChildren(div));
+ CheckExpected("<head>13</head>2");
+ EXPECT_TRUE(html_parse_.DeleteSavingChildren(head));
+ CheckExpected("132");
+}
+
+TEST_F(EventListManipulationTest, TestCoalesceOnAdd) {
+ CheckExpected("1");
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node2_, -1));
+ CheckExpected("12");
+
+ // this will coalesce node1 and node2 togethers. So there is only
+ // one node1_="12", and node2_ is gone. Deleting node1_ will now
+ // leave us empty
+ html_parse_.DeleteElement(node1_);
+ CheckExpected("");
+}
+
+TEST_F(EventListManipulationTest, TestCoalesceOnDelete) {
+ CheckExpected("1");
+ HtmlElement* div = html_parse_.NewElement(NULL, html_parse_.Intern("div"));
+ html_parse_.AddElement(div, -1);
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node2_, -1));
+ HtmlTestingPeer testing_peer;
+ testing_peer.SetNodeParent(node2_, div);
+ html_parse_.CloseElement(div, HtmlElement::EXPLICIT_CLOSE, -1);
+ HtmlTestingPeer::AddEvent(&html_parse_, new HtmlCharactersEvent(node3_, -1));
+ CheckExpected("1<div>2</div>3");
+
+ // Removing the div, leaving the children intact...
+ EXPECT_TRUE(html_parse_.DeleteSavingChildren(div));
+ CheckExpected("123");
+
+ // At this point, node1, node2, and node3 are automatically coalesced.
+ // This means when we remove node1, all the content will disappear.
+ html_parse_.DeleteElement(node1_);
+ CheckExpected("");
+}
+
+// Unit tests for attribute manipulation.
+// Goal is to make sure we don't (eg) read deallocated storage
+// while manipulating attribute values.
+class AttributeManipulationTest : public HtmlParseTest {
+ protected:
+ AttributeManipulationTest() { }
+
+ virtual void SetUp() {
+ HtmlParseTest::SetUp();
+ static const char kUrl[] =
+ "http://html.parse.test/attribute_manipulation_test.html";
+ html_parse_.StartParse(kUrl);
+ node_ = html_parse_.NewElement(NULL, MakeAtom("a"));
+ html_parse_.AddElement(node_, 0);
+ node_->AddAttribute(MakeAtom("href"), "http://www.google.com/", "\"");
+ node_->AddAttribute(MakeAtom("id"), "37", "");
+ node_->AddAttribute(MakeAtom("class"), "search!", "'");
+ html_parse_.CloseElement(node_, HtmlElement::BRIEF_CLOSE, 0);
+ }
+
+ virtual void TearDown() {
+ html_parse_.FinishParse();
+ HtmlParseTest::TearDown();
+ }
+
+ Atom MakeAtom(const char *name) {
+ return html_parse_.Intern(name);
+ }
+
+ void CheckExpected(const std::string& expected) {
+ SetupWriter();
+ html_parse_.ApplyFilter(html_writer_filter_.get());
+ EXPECT_EQ(expected, output_buffer_);
+ }
+
+ HtmlElement* node_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(AttributeManipulationTest);
+};
+
+TEST_F(AttributeManipulationTest, PropertiesAndDeserialize) {
+ StringPiece google("http://www.google.com/");
+ StringPiece number37("37");
+ StringPiece search("search!");
+ EXPECT_EQ(3, node_->attribute_size());
+ EXPECT_EQ(google, node_->AttributeValue(MakeAtom("href")));
+ EXPECT_EQ(number37, node_->AttributeValue(MakeAtom("id")));
+ EXPECT_EQ(search, node_->AttributeValue(MakeAtom("class")));
+ EXPECT_TRUE(NULL == node_->AttributeValue(MakeAtom("absent")));
+ int val = -35;
+ EXPECT_FALSE(node_->IntAttributeValue(MakeAtom("absent"), &val));
+ EXPECT_EQ(-35, val);
+ EXPECT_FALSE(node_->IntAttributeValue(MakeAtom("href"), &val));
+ EXPECT_EQ(0, val);
+ EXPECT_TRUE(node_->IntAttributeValue(MakeAtom("id"), &val));
+ EXPECT_EQ(37, val);
+ EXPECT_TRUE(NULL == node_->FindAttribute(MakeAtom("absent")));
+ EXPECT_EQ(google, node_->FindAttribute(MakeAtom("href"))->value());
+ EXPECT_EQ(number37, node_->FindAttribute(MakeAtom("id"))->value());
+ EXPECT_EQ(search, node_->FindAttribute(MakeAtom("class"))->value());
+ EXPECT_EQ(google, node_->FindAttribute(MakeAtom("href"))->escaped_value());
+ EXPECT_EQ(number37, node_->FindAttribute(MakeAtom("id"))->escaped_value());
+ EXPECT_EQ(search, node_->FindAttribute(MakeAtom("class"))->escaped_value());
+ CheckExpected("<a href=\"http://www.google.com/\" id=37 class='search!'/>");
+}
+
+TEST_F(AttributeManipulationTest, AddAttribute) {
+ node_->AddAttribute(MakeAtom("lang"), "ENG-US", "\"");
+ CheckExpected("<a href=\"http://www.google.com/\" id=37 class='search!'"
+ " lang=\"ENG-US\"/>");
+}
+
+TEST_F(AttributeManipulationTest, DeleteAttribute) {
+ node_->DeleteAttribute(1);
+ CheckExpected("<a href=\"http://www.google.com/\" class='search!'/>");
+}
+
+TEST_F(AttributeManipulationTest, ModifyAttribute) {
+ HtmlElement::Attribute* href =
+ node_->FindAttribute(MakeAtom("href"));
+ EXPECT_TRUE(href != NULL);
+ href->SetValue("google");
+ href->set_quote("'");
+ href->set_name(MakeAtom("src"));
+ CheckExpected("<a src='google' id=37 class='search!'/>");
+}
+
+TEST_F(AttributeManipulationTest, ModifyKeepAttribute) {
+ HtmlElement::Attribute* href =
+ node_->FindAttribute(MakeAtom("href"));
+ EXPECT_TRUE(href != NULL);
+ // This apparently do-nothing call to SetValue exposed an allocation bug.
+ href->SetValue(href->value());
+ href->set_quote(href->quote());
+ href->set_name(href->name());
+ CheckExpected("<a href=\"http://www.google.com/\" id=37 class='search!'/>");
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/htmlparse/html_testing_peer.h b/trunk/src/net/instaweb/htmlparse/html_testing_peer.h
new file mode 100644
index 0000000..ecbd435
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/html_testing_peer.h
@@ -0,0 +1,50 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_HTML_TESTING_PEER_H_
+#define NET_INSTAWEB_HTMLPARSE_HTML_TESTING_PEER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_node.h"
+
+namespace net_instaweb {
+
+class HtmlTestingPeer {
+ public:
+ HtmlTestingPeer() { }
+
+ static void SetNodeParent(HtmlNode* node, HtmlElement* parent) {
+ node->set_parent(parent);
+ }
+ static void AddEvent(HtmlParse* parser, HtmlEvent* event) {
+ parser->AddEvent(event);
+ }
+ static void SetCurrent(HtmlParse* parser, HtmlNode* node) {
+ parser->SetCurrent(node);
+ }
+ static void set_coalesce_characters(HtmlParse* parser, bool x) {
+ parser->set_coalesce_characters(x);
+ }
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(HtmlTestingPeer);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_HTML_TESTING_PEER_H_
diff --git a/trunk/src/net/instaweb/htmlparse/html_writer_filter.cc b/trunk/src/net/instaweb/htmlparse/html_writer_filter.cc
new file mode 100644
index 0000000..4d95070
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/html_writer_filter.cc
@@ -0,0 +1,230 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/htmlparse/public/html_writer_filter.h"
+
+#include "base/logging.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include <string>
+#include "net/instaweb/util/public/writer.h"
+
+namespace net_instaweb {
+
+static const int kDefaultMaxColumn = -1;
+
+HtmlWriterFilter::HtmlWriterFilter(HtmlParse* html_parse)
+ : writer_(NULL) {
+ html_parse_ = html_parse;
+
+ // Pre-intern a set of common symbols that can be used for
+ // fast comparisons when matching tags, and for pointer-based
+ // hash-tables.
+ symbol_a_ = html_parse->Intern("a");
+ symbol_link_ = html_parse->Intern("link");
+ symbol_href_ = html_parse->Intern("href");
+ symbol_img_ = html_parse->Intern("img");
+ symbol_script_ = html_parse->Intern("script");
+ symbol_src_ = html_parse->Intern("src");
+ symbol_alt_ = html_parse->Intern("alt");
+ max_column_ = kDefaultMaxColumn;
+ Clear();
+}
+
+HtmlWriterFilter::~HtmlWriterFilter() {
+}
+
+void HtmlWriterFilter::Clear() {
+ lazy_close_element_ = NULL;
+ column_ = 0;
+ write_errors_ = 0;
+}
+
+void HtmlWriterFilter::EmitBytes(const StringPiece& str) {
+ if (lazy_close_element_ != NULL) {
+ lazy_close_element_ = NULL;
+ if (!writer_->Write(">", html_parse_->message_handler())) {
+ ++write_errors_;
+ }
+ ++column_;
+ }
+
+ // Search backward from the end for the last occurrence of a newline.
+ column_ += str.size(); // if there are no newlines, bump up column counter.
+ for (int i = str.size() - 1; i >= 0; --i) {
+ if (str[i] == '\n') {
+ column_ = str.size() - i - 1; // found a newline; so reset the column.
+ break;
+ }
+ }
+ if (!writer_->Write(str, html_parse_->message_handler())) {
+ ++write_errors_;
+ }
+}
+
+void HtmlWriterFilter::StartElement(HtmlElement* element) {
+ EmitBytes("<");
+ EmitBytes(element->tag().c_str());
+
+ for (int i = 0; i < element->attribute_size(); ++i) {
+ const HtmlElement::Attribute& attribute = element->attribute(i);
+ // If the column has grown too large, insert a newline. It's always safe
+ // to insert whitespace in the middle of tag parameters.
+ int attr_length = 1 + attribute.name().size();
+ if (max_column_ > 0) {
+ if (attribute.escaped_value() != NULL) {
+ attr_length += 1 + strlen(attribute.escaped_value());
+ }
+ if ((column_ + attr_length) > max_column_) {
+ EmitBytes("\n");
+ }
+ }
+ EmitBytes(" ");
+ EmitBytes(attribute.name().c_str());
+ if (attribute.escaped_value() != NULL) {
+ EmitBytes("=");
+ EmitBytes(attribute.quote());
+ EmitBytes(attribute.escaped_value());
+ EmitBytes(attribute.quote());
+ }
+ }
+
+ // Attempt to briefly terminate any legal tag that was explicitly terminated
+ // in the input. Note that a rewrite pass might have injected events
+ // between the begin/end of an element that was closed briefly in the input
+ // html. In that case it cannot be closed briefly. It is up to this
+ // code to validate BRIEF_CLOSE on each element.
+ //
+ // TODO(jmarantz): Add a rewrite pass that morphs EXPLICIT_CLOSE into 'brief'
+ // when legal. Such a change will introduce textual diffs between
+ // input and output html that would cause htmlparse unit tests to require
+ // a regold. But the changes could be validated with the normalizer.
+ if (GetCloseStyle(element) == HtmlElement::BRIEF_CLOSE) {
+ lazy_close_element_ = element;
+ } else {
+ EmitBytes(">");
+ }
+}
+
+// Compute the tag-closing style for an element. If the style was specified
+// on construction, then we use that. If the element was synthesized by
+// a rewrite pass, then it's stored as AUTO_CLOSE, and we can determine
+// whether the element is briefly closable or implicitly closed.
+HtmlElement::CloseStyle HtmlWriterFilter::GetCloseStyle(HtmlElement* element) {
+ HtmlElement::CloseStyle style = element->close_style();
+ if (style == HtmlElement::AUTO_CLOSE) {
+ Atom tag = element->tag();
+ if (html_parse_->IsImplicitlyClosedTag(tag)) {
+ style = HtmlElement::IMPLICIT_CLOSE;
+ } else if (html_parse_->TagAllowsBriefTermination(tag)) {
+ style = HtmlElement::BRIEF_CLOSE;
+ } else {
+ style = HtmlElement::EXPLICIT_CLOSE;
+ }
+ }
+ return style;
+}
+
+void HtmlWriterFilter::EndElement(HtmlElement* element) {
+ HtmlElement::CloseStyle style = GetCloseStyle(element);
+ switch (style) {
+ case HtmlElement::AUTO_CLOSE:
+ // This cannot happen because GetCloseStyle prevents won't
+ // return AUTO_CLOSE.
+ html_parse_->message_handler()->FatalError(
+ __FILE__, __LINE__, "GetCloseStyle should never return AUTO_CLOSE.");
+ break;
+ case HtmlElement::IMPLICIT_CLOSE:
+ // Nothing new to write; the ">" was written in StartElement
+ break;
+ case HtmlElement::BRIEF_CLOSE:
+ // even if the element is briefly closeable, if more text
+ // got written after the element open, then we must
+ // explicitly close it, so we fall through.
+ if (lazy_close_element_ == element) {
+ lazy_close_element_ = NULL;
+
+ // If this attribute was unquoted, or lacked a value, then we'll need
+ // to add a space here to ensure that HTML parsers don't interpret the
+ // '/' in the '/>' as part of the attribute.
+ if (element->attribute_size() != 0) {
+ const HtmlElement::Attribute& attribute = element->attribute(
+ element->attribute_size() - 1);
+ if ((attribute.escaped_value() == NULL) ||
+ (attribute.quote()[0] == '\0')) {
+ EmitBytes(" ");
+ }
+ }
+ EmitBytes("/>");
+ break;
+ }
+ // fall through
+ case HtmlElement::EXPLICIT_CLOSE:
+ EmitBytes("</");
+ EmitBytes(element->tag().c_str());
+ EmitBytes(">");
+ break;
+ case HtmlElement::UNCLOSED:
+ // Nothing new to write; the ">" was written in StartElement
+ break;
+ }
+}
+
+void HtmlWriterFilter::Characters(HtmlCharactersNode* chars) {
+ EmitBytes(chars->contents());
+}
+
+void HtmlWriterFilter::Cdata(HtmlCdataNode* cdata) {
+ EmitBytes("<![CDATA[");
+ EmitBytes(cdata->contents());
+ EmitBytes("]]>");
+}
+
+void HtmlWriterFilter::Comment(HtmlCommentNode* comment) {
+ EmitBytes("<!--");
+ EmitBytes(comment->contents());
+ EmitBytes("-->");
+}
+
+void HtmlWriterFilter::IEDirective(HtmlIEDirectiveNode* directive) {
+ EmitBytes("<!--");
+ EmitBytes(directive->contents());
+ EmitBytes("-->");
+}
+
+void HtmlWriterFilter::Directive(HtmlDirectiveNode* directive) {
+ EmitBytes("<!");
+ EmitBytes(directive->contents());
+ EmitBytes(">");
+}
+
+void HtmlWriterFilter::StartDocument() {
+ Clear();
+}
+
+void HtmlWriterFilter::EndDocument() {
+}
+
+void HtmlWriterFilter::Flush() {
+ if (!writer_->Flush(html_parse_->message_handler())) {
+ ++write_errors_;
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/htmlparse/logging_html_filter.cc b/trunk/src/net/instaweb/htmlparse/logging_html_filter.cc
new file mode 100644
index 0000000..b3af089
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/logging_html_filter.cc
@@ -0,0 +1,147 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "public/logging_html_filter.h"
+#include "public/html_element.h"
+#include "public/statistics_log.h"
+
+// TODO(jmarantz): convert to Statistics interface
+
+namespace {
+// Printable names of the statistics.
+// Must match up with enum Statistic in logging_html_filter.h;
+// is this bad for maintenance?
+const char* kStatisticNames[] = {
+ "explicit_close", "implicit_close", "brief_close", "closed", "unclosed",
+ "spurious_close", "tags", "cdata", "comments", "directives", "documents",
+ "IE_directives",
+};
+}
+
+namespace net_instaweb {
+
+LoggingFilter::LoggingFilter() {
+ Reset();
+}
+
+void LoggingFilter::StartDocument() {
+ ++stats_[NUM_DOCUMENTS];
+}
+
+void LoggingFilter::StartElement(HtmlElement* element) {
+ // Does EndElement get called for singleton elements?
+ ++stats_[NUM_UNCLOSED];
+ ++stats_[NUM_TAGS];
+}
+
+void LoggingFilter::EndElement(HtmlElement* element) {
+ // Figure out what's up with the element (implicitly vs explicitly closed)
+ switch (element->close_style()) {
+ case HtmlElement::EXPLICIT_CLOSE: {
+ --stats_[NUM_UNCLOSED];
+ ++stats_[NUM_CLOSED];
+ ++stats_[NUM_EXPLICIT_CLOSED];
+ break;
+ }
+ case HtmlElement::IMPLICIT_CLOSE: {
+ --stats_[NUM_UNCLOSED];
+ ++stats_[NUM_CLOSED];
+ ++stats_[NUM_IMPLICIT_CLOSED];
+ break;
+ }
+ case HtmlElement::BRIEF_CLOSE: {
+ --stats_[NUM_UNCLOSED];
+ ++stats_[NUM_CLOSED];
+ ++stats_[NUM_BRIEF_CLOSED];
+ break;
+ }
+ case HtmlElement::UNCLOSED: {
+ // We assumed unmatchedness at StartElement, so do nothing.
+ break;
+ }
+ case HtmlElement::AUTO_CLOSE: {
+ // Another form of unmated tag, again do nothing.
+ break;
+ }
+ }
+}
+
+void LoggingFilter::Cdata(HtmlCdataNode* cdata) {
+ ++stats_[NUM_CDATA];
+}
+
+void LoggingFilter::Comment(HtmlCommentNode* comment) {
+ ++stats_[NUM_COMMENTS];
+}
+
+void LoggingFilter::IEDirective(HtmlIEDirectiveNode* directive) {
+ ++stats_[NUM_IE_DIRECTIVES];
+}
+
+void LoggingFilter::Directive(HtmlDirectiveNode* directive) {
+ ++stats_[NUM_DIRECTIVES];
+}
+
+// Logging, diffing, and aggregation
+void LoggingFilter::LogStatistics(StatisticsLog *statistics_log) const {
+ for (int statistic = MIN_STAT; statistic < MAX_STAT; ++statistic) {
+ statistics_log->LogStat(kStatisticNames[statistic], stats_[statistic]);
+ }
+}
+
+bool LoggingFilter::Equals(const LoggingFilter &that) const {
+ for (int statistic = MIN_STAT; statistic < MAX_STAT; ++statistic) {
+ if (this->stats_[statistic] != that.stats_[statistic])
+ return false;
+ }
+ return true;
+}
+
+void LoggingFilter::LogDifferences(
+ const LoggingFilter &that, StatisticsLog *statistics_log) const {
+ for (int statistic = MIN_STAT; statistic < MAX_STAT; ++statistic) {
+ if (this->stats_[statistic] != that.stats_[statistic]) {
+ statistics_log->LogDifference(
+ kStatisticNames[statistic],
+ this->stats_[statistic], that.stats_[statistic]);
+ }
+ }
+}
+
+void LoggingFilter::Aggregate(const LoggingFilter &that) {
+ for (int statistic = MIN_STAT; statistic < MAX_STAT; ++statistic) {
+ this->stats_[statistic] += that.stats_[statistic];
+ }
+}
+
+void LoggingFilter::AggregateDifferences(const LoggingFilter &first,
+ const LoggingFilter &second) {
+ for (int statistic = MIN_STAT; statistic < MAX_STAT; ++statistic) {
+ this->stats_[statistic] +=
+ first.stats_[statistic] - second.stats_[statistic];
+ }
+}
+
+void LoggingFilter::Reset() {
+ // Cleaner than memset?
+ for (int statistic = MIN_STAT; statistic < MAX_STAT; ++statistic) {
+ stats_[statistic] = 0;
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/htmlparse/null_filter.cc b/trunk/src/net/instaweb/htmlparse/null_filter.cc
new file mode 100644
index 0000000..7f6ae0f
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/null_filter.cc
@@ -0,0 +1,70 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include <stdio.h>
+#include "net/instaweb/htmlparse/public/file_driver.h"
+#include "net/instaweb/util/public/file_message_handler.h"
+#include "net/instaweb/htmlparse/public/file_statistics_log.h"
+#include "net/instaweb/util/public/stdio_file_system.h"
+
+int null_filter(int argc, char** argv) {
+ int ret = 1;
+
+ if ((argc < 2) || (argc > 4)) {
+ fprintf(stdout, "Usage: %s input_file [- | output_file] [log_file]\n",
+ argv[0]);
+ return ret;
+ }
+
+ const char* infile = argv[1];
+ net_instaweb::FileMessageHandler message_handler(stderr);
+ net_instaweb::StdioFileSystem file_system;
+ net_instaweb::HtmlParse html_parse(&message_handler);
+ net_instaweb::FileDriver file_driver(&html_parse, &file_system);
+ const char* outfile = NULL;
+ std::string outfile_buffer;
+ const char* statsfile = NULL;
+ std::string statsfile_buffer;
+
+ if (argc >= 3) {
+ outfile = argv[2];
+ } else if (net_instaweb::FileDriver::GenerateOutputFilename(
+ infile, &outfile_buffer)) {
+ outfile = outfile_buffer.c_str();
+ fprintf(stdout, "Null rewriting %s into %s\n", infile, outfile);
+ } else {
+ message_handler.FatalError(infile, 0, "Cannot generate output filename");
+ }
+
+ if (argc >= 4) {
+ statsfile = argv[3];
+ } else if (net_instaweb::FileDriver::GenerateStatsFilename(
+ infile, &statsfile_buffer)) {
+ statsfile = statsfile_buffer.c_str();
+ fprintf(stdout, "Logging statistics for %s into %s\n",
+ infile, statsfile);
+ } else {
+ message_handler.FatalError(infile, 0, "Cannot generate stats file name");
+ }
+
+ if (file_driver.ParseFile(infile, outfile, statsfile, &message_handler)) {
+ ret = 0;
+ }
+
+ return ret;
+}
diff --git a/trunk/src/net/instaweb/htmlparse/null_filter.h b/trunk/src/net/instaweb/htmlparse/null_filter.h
new file mode 100644
index 0000000..f94d497
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/null_filter.h
@@ -0,0 +1,24 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_NULL_FILTER_H_
+#define NET_INSTAWEB_HTMLPARSE_NULL_FILTER_H_
+
+int null_filter(int argc, char** argv);
+
+#endif // NET_INSTAWEB_HTMLPARSE_NULL_FILTER_H_
diff --git a/trunk/src/net/instaweb/htmlparse/public/doctype.h b/trunk/src/net/instaweb/htmlparse/public/doctype.h
new file mode 100644
index 0000000..3c8df8c
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/public/doctype.h
@@ -0,0 +1,88 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_PUBLIC_DOCTYPE_H_
+#define NET_INSTAWEB_HTMLPARSE_PUBLIC_DOCTYPE_H_
+
+#include "net/instaweb/util/public/string_util.h" // for StringPiece
+
+#include "net/instaweb/util/public/content_type.h"
+
+namespace net_instaweb {
+
+class DocType {
+ public:
+ DocType() : doctype_(UNKNOWN) {}
+ DocType(const DocType& src) : doctype_(src.doctype_) {}
+ ~DocType() {}
+
+ DocType& operator=(const DocType& src) {
+ if (&src != this) {
+ doctype_ = src.doctype_;
+ }
+ return *this;
+ }
+
+ bool operator==(const DocType& other) const {
+ return doctype_ == other.doctype_;
+ }
+
+ bool operator!=(const DocType& other) const {
+ return doctype_ != other.doctype_;
+ }
+
+ // Return true iff this is a known XHTML doctype (of some version).
+ bool IsXhtml() const;
+ // Return true iff this is an HTML 5 or XHTML 5 doctype.
+ bool IsVersion5() const;
+ // TODO(mdsteele): Add more such methods as necessary.
+
+ static const DocType kUnknown;
+ static const DocType kHTML5;
+ static const DocType kHTML4Strict;
+ static const DocType kHTML4Transitional;
+ static const DocType kXHTML5;
+ static const DocType kXHTML11;
+ static const DocType kXHTML10Strict;
+ static const DocType kXHTML10Transitional;
+
+ // Given the contents of an HTML directive and the content type of the file
+ // it appears in, update this DocType to match that specified by the
+ // directive and return true. If the directive is not a doctype directive,
+ // return false and don't alter the DocType.
+ bool Parse(const StringPiece& directive,
+ const ContentType& content_type);
+
+ private:
+ enum DocTypeEnum {
+ UNKNOWN = 0,
+ HTML_5,
+ HTML_4_STRICT,
+ HTML_4_TRANSITIONAL,
+ XHTML_5,
+ XHTML_1_1,
+ XHTML_1_0_STRICT,
+ XHTML_1_0_TRANSITIONAL,
+ };
+
+ DocType(DocTypeEnum doctype) : doctype_(doctype) {}
+
+ DocTypeEnum doctype_;
+};
+
+} // net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_PUBLIC_DOCTYPE_H_
diff --git a/trunk/src/net/instaweb/htmlparse/public/empty_html_filter.h b/trunk/src/net/instaweb/htmlparse/public/empty_html_filter.h
new file mode 100644
index 0000000..668ffcb
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/public/empty_html_filter.h
@@ -0,0 +1,52 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_PUBLIC_EMPTY_HTML_FILTER_H_
+#define NET_INSTAWEB_HTMLPARSE_PUBLIC_EMPTY_HTML_FILTER_H_
+
+#include <string>
+#include "net/instaweb/htmlparse/public/html_filter.h"
+
+namespace net_instaweb {
+
+// Base class for rewriting filters that don't need to be sure to
+// override every filter method. Other filters that need to be sure
+// they override every method would derive directly from HtmlFilter.
+class EmptyHtmlFilter : public HtmlFilter {
+ public:
+ EmptyHtmlFilter();
+ virtual ~EmptyHtmlFilter();
+
+ virtual void StartDocument();
+ virtual void EndDocument();
+ virtual void StartElement(HtmlElement* element);
+ virtual void EndElement(HtmlElement* element);
+ virtual void Cdata(HtmlCdataNode* cdata);
+ virtual void Comment(HtmlCommentNode* comment);
+ virtual void IEDirective(HtmlIEDirectiveNode* directive);
+ virtual void Characters(HtmlCharactersNode* characters);
+ virtual void Directive(HtmlDirectiveNode* directive);
+ virtual void Flush();
+
+ // Note -- this does not provide an implementation for Name(). This
+ // must be supplied by derived classes.
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_PUBLIC_EMPTY_HTML_FILTER_H_
diff --git a/trunk/src/net/instaweb/htmlparse/public/file_driver.h b/trunk/src/net/instaweb/htmlparse/public/file_driver.h
new file mode 100644
index 0000000..cb9b091
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/public/file_driver.h
@@ -0,0 +1,73 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_PUBLIC_FILE_DRIVER_H_
+#define NET_INSTAWEB_HTMLPARSE_PUBLIC_FILE_DRIVER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_parser_types.h"
+#include "net/instaweb/htmlparse/public/html_writer_filter.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/logging_html_filter.h"
+#include "net/instaweb/htmlparse/public/statistics_log.h"
+#include <string>
+
+namespace net_instaweb {
+
+// Framework for reading an input HTML file, running it through
+// a chain of HTML filters, and writing an output file.
+class FileDriver {
+ public:
+ FileDriver(HtmlParse* html_parse, FileSystem* file_system);
+
+ // Return the parser. This can be used to add filters.
+ HtmlParse* html_parse() { return html_parse_; }
+
+ // Helper function to generate an output .html filename from
+ // an input filename. Given "/a/b/c.html" returns "a/b/c.out.html".
+ // Returns false if the input file does not contain a "."
+ static bool GenerateOutputFilename(
+ const char* infilename, std::string* outfilename);
+
+ // Helper function to generate an output .stats filename from
+ // an input filename. Given "/a/b/c.html" returns "a/b/c.stats".
+ // Returns false if the input file does not contain a "."
+ static bool GenerateStatsFilename(
+ const char* infilename, std::string* statsfilename);
+
+ // Error messages are sent to the message file, true is returned
+ // if the file was parsed successfully.
+ bool ParseFile(const char* infilename,
+ const char* outfilename,
+ const char* statsfilename,
+ MessageHandler* handler);
+
+ private:
+ HtmlParse* html_parse_;
+ LoggingFilter logging_filter_;
+ StatisticsLog* stats_log_;
+ HtmlWriterFilter html_write_filter_;
+ bool filters_added_;
+ FileSystem* file_system_;
+
+ DISALLOW_COPY_AND_ASSIGN(FileDriver);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_PUBLIC_FILE_DRIVER_H_
diff --git a/trunk/src/net/instaweb/htmlparse/public/file_statistics_log.h b/trunk/src/net/instaweb/htmlparse/public/file_statistics_log.h
new file mode 100644
index 0000000..dfd7d55
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/public/file_statistics_log.h
@@ -0,0 +1,49 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_PUBLIC_FILE_STATISTICS_LOG_H_
+#define NET_INSTAWEB_HTMLPARSE_PUBLIC_FILE_STATISTICS_LOG_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/statistics_log.h"
+#include "net/instaweb/util/public/file_system.h"
+
+namespace net_instaweb {
+
+class MessageHandler;
+
+// Statistics logger that sends its output to a file.
+class FileStatisticsLog : public StatisticsLog {
+ public:
+ // Note: calling context responsible for closing & cleaning up file.
+ explicit FileStatisticsLog(FileSystem::OutputFile* file,
+ MessageHandler* message_handler);
+ virtual ~FileStatisticsLog();
+ virtual void LogStat(const char *statName, int value);
+ virtual void LogDifference(const char *statName,
+ int value1, int value2);
+ private:
+ FileSystem::OutputFile* file_;
+ MessageHandler* message_handler_;
+
+ DISALLOW_COPY_AND_ASSIGN(FileStatisticsLog);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_PUBLIC_FILE_STATISTICS_LOG_H_
diff --git a/trunk/src/net/instaweb/htmlparse/public/html_element.h b/trunk/src/net/instaweb/htmlparse/public/html_element.h
new file mode 100644
index 0000000..11888d7
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/public/html_element.h
@@ -0,0 +1,264 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_ELEMENT_H_
+#define NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_ELEMENT_H_
+
+#include <vector>
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/htmlparse/public/html_node.h"
+#include "net/instaweb/htmlparse/public/html_parser_types.h"
+#include "net/instaweb/util/public/atom.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/symbol_table.h"
+
+namespace net_instaweb {
+
+class HtmlElement : public HtmlNode {
+ public:
+ // Tags can be closed in three ways: implicitly (e.g. <img ..>),
+ // briefly (e.g. <br/>), or explicitly (<a...>...</a>). The
+ // Lexer will always record the way it parsed a tag, but synthesized
+ // elements will have AUTO_CLOSE, and rewritten elements may
+ // no longer qualify for the closing style with which they were
+ // parsed.
+ enum CloseStyle {
+ AUTO_CLOSE, // synthesized tag, or not yet closed in source
+ IMPLICIT_CLOSE, // E.g. <img...> <meta...> <link...> <br...> <input...>
+ EXPLICIT_CLOSE, // E.g. <a href=...>anchor</a>
+ BRIEF_CLOSE, // E.g. <head/>
+ UNCLOSED // Was never closed in source
+ };
+
+ class Attribute {
+ public:
+ // A large quantity of HTML in the wild has attributes that are
+ // improperly escaped. Browsers are generally tolerant of this.
+ // But we want to avoid corrupting pages we do not understand.
+
+ // The result of value() and escaped_value() is still owned by this, and
+ // will be invalidated by a subsequent call to SetValue() or
+ // SetUnescapedValue
+
+ Atom name() const { return name_; }
+ void set_name(Atom name) { name_ = name; }
+
+ // Returns the value in its original directly from the HTML source.
+ // This may have HTML escapes in it, such as "&".
+ const char* escaped_value() const { return escaped_value_.get(); }
+
+ // The result of value() is still owned by this, and will be invalidated by
+ // a subsequent call to set_value().
+ //
+ // The result will be a NUL-terminated string containing the value of the
+ // attribute, or NULL if the attribute has no value at all (this is
+ // distinct from having the empty string for a value).
+
+ // Returns the unescaped value, suitable for directly operating on
+ // in filters as URLs or other data.
+ const char* value() const { return value_.get(); }
+
+ // See comment about quote on constructor for Attribute.
+ // Returns the quotation mark associated with this URL, typically
+ // ", ', or an empty string.
+ const char* quote() const { return quote_; }
+
+ // Two related methods to modify the value of attribute (eg to rewrite
+ // dest of src or href). As with the constructor, copies the string in,
+ // so caller retains ownership of value.
+ //
+ // A StringPiece pointing to an empty string (that is, a char array {'\0'})
+ // indicates that the attribute value is the empty string (e.g. <foo
+ // bar="">); however, a StringPiece with a data() pointer of NULL indicates
+ // that the attribute has no value at all (e.g. <foo bar>). This is an
+ // important distinction.
+ //
+ // Note that passing a value containing NULs in the middle will cause
+ // breakage, but this isn't currently checked for.
+ // TODO(mdsteele): Perhaps we should check for this?
+
+ // Sets the value of the attribute. No HTML escaping is expected.
+ // This call causes the HTML-escaped value to be automatically computed
+ // by scanning the value and escaping any characters required in HTML
+ // attributes.
+ void SetValue(const StringPiece& value);
+
+ // Sets the escaped value. This is intended to be called from the HTML
+ // Lexer, and results in the Value being computed automatically by
+ // scanning the value for escape sequences.
+ void SetEscapedValue(const StringPiece& value);
+
+ // See comment about quote on constructor for Attribute.
+ void set_quote(const char *quote) {
+ quote_ = quote;
+ }
+
+ friend class HtmlElement;
+
+ private:
+ // TODO(jmarantz): arg 'quote' must be a static string, or NULL,
+ // if quoting is not yet known (e.g. this is a synthesized attribute.
+ // This is hard-to-describe and we should probably use an Atom for
+ // the quote, and decide how to handle NULL.
+ //
+ // This should only be called from AddAttribute
+ Attribute(Atom name, const StringPiece& value,
+ const StringPiece& escaped_value, const char* quote);
+
+ static inline void CopyValue(const StringPiece& src,
+ scoped_array<char>* dst);
+
+ Atom name_;
+ scoped_array<char> escaped_value_;
+ scoped_array<char> value_;
+ const char* quote_;
+
+ DISALLOW_COPY_AND_ASSIGN(Attribute);
+ };
+
+ virtual ~HtmlElement();
+
+ // Add a copy of an attribute to this element. The attribute may come
+ // from this element, or another one.
+ void AddAttribute(const Attribute& attr);
+
+ // Unconditionally add attribute, copying value.
+ // Quote is assumed to be a static const char *.
+ // Doesn't check for attribute duplication (which is illegal in html).
+ //
+ // The value, if non-null, is assumed to be unescaped. See also
+ // AddEscapedAttribute.
+ void AddAttribute(Atom name, const StringPiece& value, const char* quote);
+ // Unconditionally add attribute with int value.
+ void AddAttribute(Atom name, int value);
+ // As AddAttribute, but assumes value has been escaped for html output.
+ void AddEscapedAttribute(Atom name, const StringPiece& escaped_value,
+ const char* quote);
+
+ // Removes the attribute at the given index, shifting higher-indexed
+ // attributes down. Note that this operation is linear in the number of
+ // attributes.
+ void DeleteAttribute(int i);
+
+ // Remove the attribute with the given name. Return true if the attribute
+ // was deleted, false if it wasn't there to begin with.
+ bool DeleteAttribute(Atom name);
+
+ // Look up attribute by name. NULL if no attribute exists.
+ // Use this for attributes whose value you might want to change
+ // after lookup.
+ const Attribute* FindAttribute(Atom name) const;
+ Attribute* FindAttribute(Atom name) {
+ const HtmlElement* const_this = this;
+ const Attribute* result = const_this->FindAttribute(name);
+ return const_cast<Attribute*>(result);
+ }
+
+ // Look up attribute value by name. NULL if no attribute exists.
+ // Use this only if you don't intend to change the attribute value;
+ // if you might change the attribute value, use FindAttribute instead
+ // (this avoids a double lookup).
+ const char* AttributeValue(Atom name) const {
+ const Attribute* attribute = FindAttribute(name);
+ if (attribute != NULL) {
+ return attribute->value();
+ }
+ return NULL;
+ }
+
+ // Look up attribute value by name. false if no attribute exists,
+ // or attribute value cannot be converted to int. Otherwise
+ // sets *value.
+ bool IntAttributeValue(Atom name, int* value) const {
+ const Attribute* attribute = FindAttribute(name);
+ if (attribute != NULL) {
+ return StringToInt(attribute->value(), value);
+ }
+ return false;
+ }
+
+ // Small integer uniquely identifying the HTML element, primarily
+ // for debugging.
+ void set_sequence(int sequence) { sequence_ = sequence; }
+
+
+ Atom tag() const {return tag_;}
+
+ // Changing that tag of an element should only occur if the caller knows
+ // that the old attributes make sense for the new tag. E.g. a div could
+ // be changed to a span.
+ void set_tag(Atom new_tag) { tag_ = new_tag; }
+
+ int attribute_size() const {return attributes_.size(); }
+ const Attribute& attribute(int i) const { return *attributes_[i]; }
+ Attribute& attribute(int i) { return *attributes_[i]; }
+
+ friend class HtmlParse;
+ friend class HtmlLexer;
+
+ CloseStyle close_style() const { return close_style_; }
+ void set_close_style(CloseStyle style) { close_style_ = style; }
+
+ // Render an element as a string for debugging. This is not
+ // intended as a fully legal serialization.
+ void ToString(std::string* buf) const;
+ void DebugPrint() const;
+
+ int begin_line_number() const { return begin_line_number_; }
+ int end_line_number() const { return end_line_number_; }
+
+ protected:
+ virtual void SynthesizeEvents(const HtmlEventListIterator& iter,
+ HtmlEventList* queue);
+ virtual void InvalidateIterators(const HtmlEventListIterator& end);
+
+ virtual HtmlEventListIterator begin() const { return begin_; }
+ virtual HtmlEventListIterator end() const { return end_; }
+
+ private:
+ // Begin/end event iterators are used by HtmlParse to keep track
+ // of the span of events underneath an element. This is primarily to
+ // help delete the element. Events are not public.
+ void set_begin(const HtmlEventListIterator& begin) { begin_ = begin; }
+ void set_end(const HtmlEventListIterator& end) { end_ = end; }
+
+ void set_begin_line_number(int line) { begin_line_number_ = line; }
+ void set_end_line_number(int line) { end_line_number_ = line; }
+
+ // construct via HtmlParse::NewElement
+ HtmlElement(HtmlElement* parent, Atom tag, const HtmlEventListIterator& begin,
+ const HtmlEventListIterator& end);
+
+ int sequence_;
+ Atom tag_;
+ std::vector<Attribute*> attributes_;
+ HtmlEventListIterator begin_;
+ HtmlEventListIterator end_;
+ CloseStyle close_style_;
+ int begin_line_number_;
+ int end_line_number_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlElement);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_ELEMENT_H_
diff --git a/trunk/src/net/instaweb/htmlparse/public/html_escape.h b/trunk/src/net/instaweb/htmlparse/public/html_escape.h
new file mode 100644
index 0000000..a6eb4b8
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/public/html_escape.h
@@ -0,0 +1,87 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_ESCAPE_H_
+#define NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_ESCAPE_H_
+
+#include <map>
+#include "base/basictypes.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class HtmlEscape {
+ public:
+ // Initialize a singleton instance of this class. This call is
+ // inherently thread unsafe, but only the first time it is called.
+ // If multi-threaded programs call this function before spawning
+ // threads then there will be no races.
+ static void Init();
+
+ // Tear down the singleton instance of this class, freeing any
+ // allocated memory. This call is inherently thread unsafe.
+ static void ShutDown();
+
+ // Take a raw text and escape it so it's safe for an HTML attribute,
+ // e.g. a&b --> a&b
+ static StringPiece Escape(const StringPiece& unescaped, std::string* buf) {
+ return singleton_->EscapeHelper(unescaped, buf);
+ }
+
+ // Take escaped text and unescape it so its value can be interpreted,
+ // e.g. "http://myhost.com/p?v&w" --> "http://myhost.com/p?v&w"
+ static StringPiece Unescape(const StringPiece& escaped, std::string* buf) {
+ return singleton_->UnescapeHelper(escaped, buf);
+ }
+
+ // Note that Escape and Unescape are not guaranteed to be inverses of
+ // one another. For example, Unescape("")=="&", but Escape("&")="&".
+ // However, note that Unescape(Escape(s)) == s.
+ //
+ // Another case to be wary of is when the argument to Unescape is not
+ // properly escaped. The result will be that the string is returned
+ // unmodified. For example, Unescape("a&b")=="a&b", butthen re-escaping
+ // that will give "a&b". Hence, the careful maintainer of an HTML
+ // parsing and rewriting system will need to maintain the original escaped
+ // text parsed from HTML files, and pass that to browsers.
+
+ private:
+ HtmlEscape();
+ const char* UnescapeAttributeValue();
+
+ static HtmlEscape* singleton_;
+
+ StringPiece EscapeHelper(const StringPiece& unescaped,
+ std::string* buf) const;
+ StringPiece UnescapeHelper(const StringPiece& escaped,
+ std::string* buf) const;
+
+ typedef std::map<std::string, std::string,
+ StringCompareInsensitive> StringStringMapInsensitive;
+ typedef std::map<std::string, std::string> StringStringMapSensitive;
+ StringStringMapInsensitive unescape_insensitive_map_;
+ StringStringMapSensitive unescape_sensitive_map_;
+ StringStringMapSensitive escape_map_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlEscape);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_ESCAPE_H_
diff --git a/trunk/src/net/instaweb/htmlparse/public/html_filter.h b/trunk/src/net/instaweb/htmlparse/public/html_filter.h
new file mode 100644
index 0000000..bb6ac2d
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/public/html_filter.h
@@ -0,0 +1,82 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_FILTER_H_
+#define NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_FILTER_H_
+
+#include <string>
+#include "net/instaweb/htmlparse/public/html_parser_types.h"
+
+namespace net_instaweb {
+
+class HtmlFilter {
+ public:
+ HtmlFilter();
+ virtual ~HtmlFilter();
+
+ // Starts a new document. Filters should clear their state in this function,
+ // as the same Filter instance may be used for multiple HTML documents.
+ virtual void StartDocument() = 0;
+ // Note: EndDocument will be called imediately before the last Flush call.
+ virtual void EndDocument() = 0;
+
+ // When an HTML element is encountered during parsing, each filter's
+ // StartElement method is called. The HtmlElement lives for the entire
+ // duration of the document.
+ //
+ // TODO(jmarantz): consider passing handles rather than pointers and
+ // reference-counting them instead to save memory on long documents.
+ virtual void StartElement(HtmlElement* element) = 0;
+ virtual void EndElement(HtmlElement* element) = 0;
+
+ // Called for CDATA blocks (e.g. <![CDATA[foobar]]>)
+ virtual void Cdata(HtmlCdataNode* cdata) = 0;
+
+ // Called for HTML comments that aren't IE directives (e.g. <!--foobar-->).
+ virtual void Comment(HtmlCommentNode* comment) = 0;
+
+ // Called for an IE directive; typically used for CSS styling.
+ // See http://msdn.microsoft.com/en-us/library/ms537512(VS.85).aspx
+ //
+ // TODO(mdsteele): Should we try to maintain the nested structure of
+ // the conditionals, in the same way that we maintain nesting of elements?
+ virtual void IEDirective(HtmlIEDirectiveNode* directive) = 0;
+
+ // Called for raw characters between tags.
+ virtual void Characters(HtmlCharactersNode* characters) = 0;
+
+ // Called for HTML directives (e.g. <!doctype foobar>).
+ virtual void Directive(HtmlDirectiveNode* directive) = 0;
+
+ // Notifies the Filter that a flush is occurring. A filter that's
+ // generating streamed output should flush at this time. A filter
+ // that's mutating elements can mutate any element seen since the
+ // most recent flush; once an element is flushed it is already on
+ // the wire to its destination and it's too late to mutate. Flush
+ // is initiated by an application calling HttpParse::Flush().
+ //
+ // Flush() is called after all other handlers during a HttpParse::Flush().
+ virtual void Flush() = 0;
+
+ // The name of this filter -- used for logging and debugging.
+ virtual const char* Name() const = 0;
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_FILTER_H_
diff --git a/trunk/src/net/instaweb/htmlparse/public/html_node.h b/trunk/src/net/instaweb/htmlparse/public/html_node.h
new file mode 100644
index 0000000..5c78c3e
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/public/html_node.h
@@ -0,0 +1,207 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_NODE_H_
+#define NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_NODE_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_parser_types.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+// Base class for HtmlElement and HtmlLeafNode
+class HtmlNode {
+ public:
+ virtual ~HtmlNode();
+ friend class HtmlParse;
+
+ HtmlElement* parent() const { return parent_; }
+ bool live() const { return live_; }
+
+ // Marks a node as dead. The queue's end iterator should be passed in,
+ // to remove references to stale iterators, and to force IsRewritable to
+ // return false.
+ void MarkAsDead(const HtmlEventListIterator& end);
+
+ protected:
+ // TODO(jmarantz): jmaessen suggests instantiating the html nodes
+ // without parents and computing them from context at the time they
+ // are instantiated from the lexer. This is a little more difficult
+ // when synthesizing new nodes, however. We assert sanity, however,
+ // when calling HtmlParse::ApplyFilter.
+ explicit HtmlNode(HtmlElement* parent) : parent_(parent), live_(true) {}
+
+ // Create new event object(s) representing this node, and insert them into
+ // the queue just before the given iterator; also, update this node object as
+ // necessary so that begin() and end() will return iterators pointing to
+ // the new event(s). The line number for each event should probably be -1.
+ virtual void SynthesizeEvents(const HtmlEventListIterator& iter,
+ HtmlEventList* queue) = 0;
+ virtual void InvalidateIterators(const HtmlEventListIterator& end) = 0;
+
+ // Return an iterator pointing to the first event associated with this node.
+ virtual HtmlEventListIterator begin() const = 0;
+ // Return an iterator pointing to the last event associated with this node.
+ virtual HtmlEventListIterator end() const = 0;
+
+ private:
+ friend class HtmlTestingPeer;
+
+ // Note: setting the parent doesn't change the DOM -- it just updates
+ // the pointer. This is intended to be called only from the DOM manipulation
+ // methods in HtmlParse.
+ void set_parent(HtmlElement* parent) { parent_ = parent; }
+
+ HtmlElement* parent_;
+ bool live_;
+ DISALLOW_COPY_AND_ASSIGN(HtmlNode);
+};
+
+// Base class for leaf nodes (like HtmlCharactersNode and HtmlCommentNode)
+class HtmlLeafNode : public HtmlNode {
+ public:
+ virtual ~HtmlLeafNode();
+ friend class HtmlParse;
+
+ protected:
+ HtmlLeafNode(HtmlElement* parent, const HtmlEventListIterator& iter)
+ : HtmlNode(parent),
+ iter_(iter) {}
+ virtual HtmlEventListIterator begin() const { return iter_; }
+ virtual HtmlEventListIterator end() const { return iter_; }
+ void set_iter(const HtmlEventListIterator& iter) { iter_ = iter; }
+ virtual void InvalidateIterators(const HtmlEventListIterator& end);
+
+ private:
+ HtmlEventListIterator iter_;
+ DISALLOW_COPY_AND_ASSIGN(HtmlLeafNode);
+};
+
+// Leaf node representing a CDATA section
+class HtmlCdataNode : public HtmlLeafNode {
+ public:
+ virtual ~HtmlCdataNode();
+ const std::string& contents() { return contents_; }
+ friend class HtmlParse;
+
+ protected:
+ virtual void SynthesizeEvents(const HtmlEventListIterator& iter,
+ HtmlEventList* queue);
+
+ private:
+ HtmlCdataNode(HtmlElement* parent,
+ const StringPiece& contents,
+ const HtmlEventListIterator& iter)
+ : HtmlLeafNode(parent, iter),
+ contents_(contents.data(), contents.size()) {}
+ const std::string contents_;
+ DISALLOW_COPY_AND_ASSIGN(HtmlCdataNode);
+};
+
+// Leaf node representing raw characters in HTML
+class HtmlCharactersNode : public HtmlLeafNode {
+ public:
+ virtual ~HtmlCharactersNode();
+ const std::string& contents() { return contents_; }
+ void Append(const StringPiece& str) {
+ contents_.append(str.data(), str.size());
+ }
+ friend class HtmlParse;
+
+ protected:
+ virtual void SynthesizeEvents(const HtmlEventListIterator& iter,
+ HtmlEventList* queue);
+
+ private:
+ HtmlCharactersNode(HtmlElement* parent,
+ const StringPiece& contents,
+ const HtmlEventListIterator& iter)
+ : HtmlLeafNode(parent, iter),
+ contents_(contents.data(), contents.size()) {}
+ std::string contents_;
+ DISALLOW_COPY_AND_ASSIGN(HtmlCharactersNode);
+};
+
+// Leaf node representing an HTML comment
+class HtmlCommentNode : public HtmlLeafNode {
+ public:
+ virtual ~HtmlCommentNode();
+ const std::string& contents() { return contents_; }
+ friend class HtmlParse;
+
+ protected:
+ virtual void SynthesizeEvents(const HtmlEventListIterator& iter,
+ HtmlEventList* queue);
+
+ private:
+ HtmlCommentNode(HtmlElement* parent,
+ const StringPiece& contents,
+ const HtmlEventListIterator& iter)
+ : HtmlLeafNode(parent, iter),
+ contents_(contents.data(), contents.size()) {}
+ const std::string contents_;
+ DISALLOW_COPY_AND_ASSIGN(HtmlCommentNode);
+};
+
+// Leaf node representing an HTML IE directive
+class HtmlIEDirectiveNode : public HtmlLeafNode {
+ public:
+ virtual ~HtmlIEDirectiveNode();
+ const std::string& contents() { return contents_; }
+ friend class HtmlParse;
+
+ protected:
+ virtual void SynthesizeEvents(const HtmlEventListIterator& iter,
+ HtmlEventList* queue);
+
+ private:
+ HtmlIEDirectiveNode(HtmlElement* parent,
+ const StringPiece& contents,
+ const HtmlEventListIterator& iter)
+ : HtmlLeafNode(parent, iter),
+ contents_(contents.data(), contents.size()) {}
+ const std::string contents_;
+ DISALLOW_COPY_AND_ASSIGN(HtmlIEDirectiveNode);
+};
+
+// Leaf node representing an HTML directive
+class HtmlDirectiveNode : public HtmlLeafNode {
+ public:
+ virtual ~HtmlDirectiveNode();
+ const std::string& contents() { return contents_; }
+ friend class HtmlParse;
+
+ protected:
+ virtual void SynthesizeEvents(const HtmlEventListIterator& iter,
+ HtmlEventList* queue);
+
+ private:
+ HtmlDirectiveNode(HtmlElement* parent,
+ const StringPiece& contents,
+ const HtmlEventListIterator& iter)
+ : HtmlLeafNode(parent, iter),
+ contents_(contents.data(), contents.size()) {}
+ const std::string contents_;
+ DISALLOW_COPY_AND_ASSIGN(HtmlDirectiveNode);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_NODE_H_
diff --git a/trunk/src/net/instaweb/htmlparse/public/html_parse.h b/trunk/src/net/instaweb/htmlparse/public/html_parse.h
new file mode 100644
index 0000000..c7cb804
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/public/html_parse.h
@@ -0,0 +1,294 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_PARSE_H_
+#define NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_PARSE_H_
+
+#include <stdarg.h>
+#include <set>
+#include <vector>
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/doctype.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_node.h"
+#include "net/instaweb/htmlparse/public/html_parser_types.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/printf_format.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/symbol_table.h"
+
+namespace net_instaweb {
+
+class Timer;
+
+class HtmlParse {
+ public:
+ explicit HtmlParse(MessageHandler* message_handler);
+ ~HtmlParse();
+
+ // Application methods for parsing functions and adding filters
+
+ // Add a new html filter to the filter-chain
+ void AddFilter(HtmlFilter* filter);
+
+ // Initiate a chunked parsing session. Finish with FinishParse. The
+ // url is only used to resolve relative URLs; the contents are not
+ // directly fetched. The caller must supply the text and call ParseText.
+ void StartParse(const StringPiece& url) {
+ StartParseWithType(url, kContentTypeHtml);
+ }
+ void StartParseWithType(const StringPiece& url,
+ const ContentType& content_type) {
+ StartParseId(url, url, content_type);
+ }
+ // Use an error message id that is distinct from the url.
+ // Mostly useful for testing.
+ void StartParseId(const StringPiece& url, const StringPiece& id,
+ const ContentType& content_type);
+
+ // Parses an arbitrary block of an html file, queuing up the events. Call
+ // Flush to send the events through the Filter.
+ //
+ // To parse an entire file, first call StartParse(), then call
+ // ParseText on the file contents (in whatever size chunks are convenient),
+ // then call FinishParse().
+ void ParseText(const char* content, int size);
+ void ParseText(const StringPiece& sp) { ParseText(sp.data(), sp.size()); }
+
+ // Flush the currently queued events through the filters. It is desirable
+ // for large web pages, particularly dynamically generated ones, to start
+ // getting delivered to the browser as soon as they are ready. On the
+ // other hand, rewriting is more powerful when more of the content can
+ // be considered for image/css/js spriting. This method should be called
+ // when the controlling network process wants to induce a new chunk of
+ // output. The less you call this function the better the rewriting will
+ // be.
+ void Flush();
+
+ // Finish a chunked parsing session. This also induces a Flush.
+ void FinishParse();
+
+
+ // Utility methods for implementing filters
+
+ HtmlCdataNode* NewCdataNode(HtmlElement* parent,
+ const StringPiece& contents);
+ HtmlCharactersNode* NewCharactersNode(HtmlElement* parent,
+ const StringPiece& literal);
+ HtmlCommentNode* NewCommentNode(HtmlElement* parent,
+ const StringPiece& contents);
+ HtmlDirectiveNode* NewDirectiveNode(HtmlElement* parent,
+ const StringPiece& contents);
+ HtmlIEDirectiveNode* NewIEDirectiveNode(HtmlElement* parent,
+ const StringPiece& contents);
+
+ // DOM-manipulation methods.
+ // TODO(sligocki): Find Javascript equivalents and list them or even change
+ // our names to be consistent.
+
+ // TODO(mdsteele): Rename these methods to e.g. InsertNodeBeforeNode.
+ // This and downstream filters will then see inserted elements but upstream
+ // filters will not.
+ // Note: In Javascript the first is called insertBefore and takes the arg
+ // in the oposite order.
+ void InsertElementBeforeElement(const HtmlNode* existing_node,
+ HtmlNode* new_node);
+ void InsertElementAfterElement(const HtmlNode* existing_node,
+ HtmlNode* new_node);
+
+ // Add child element at the begining or end of existing_parent's children.
+ // Named after Javascript's appendChild method.
+ void PrependChild(const HtmlElement* existing_parent, HtmlNode* new_child);
+ void AppendChild(const HtmlElement* existing_parent, HtmlNode* new_child);
+
+ // Insert element before the current one. current_ remains unchanged.
+ void InsertElementBeforeCurrent(HtmlNode* node);
+
+ // Insert element after the current one, moving current_ to the new
+ // element. In a Filter, the flush-loop will advance past this on
+ // the next iteration.
+ void InsertElementAfterCurrent(HtmlNode* node);
+
+ // Enclose element around two elements in a sequence. The first
+ // element must be the same as, or precede the last element in the
+ // event-stream, and this is not checked, but the two elements do
+ // not need to be adjacent. They must have the same parent to start
+ // with.
+ //
+ // This differs from MoveSequenceToParent in that the new parent is
+ // not yet in the DOM tree, and will be inserted around the
+ // elements.
+ bool AddParentToSequence(HtmlNode* first, HtmlNode* last,
+ HtmlElement* new_parent);
+
+ // Moves a node-sequence to an already-existing parent, where they
+ // will be placed as the last elements in that parent. Returns false
+ // if the operation could not be performed because either the node
+ // or its parent was partially or wholy flushed.
+ //
+ // This differs from AddParentToSequence in that the parent is already
+ // in the DOM-tree.
+ bool MoveCurrentInto(HtmlElement* new_parent);
+
+ // If the given node is rewritable, delete it and all of its children (if
+ // any) and return true; otherwise, do nothing and return false.
+ // Note: Javascript appears to use removeChild for this.
+ bool DeleteElement(HtmlNode* node);
+
+ // Delete a parent element, retaining any children and moving them to
+ // reside under the parent's parent.
+ bool DeleteSavingChildren(HtmlElement* element);
+
+ // If possible, replace the existing node with the new node and return true;
+ // otherwise, do nothing and return false.
+ bool ReplaceNode(HtmlNode* existing_node, HtmlNode* new_node);
+
+
+ HtmlElement* NewElement(HtmlElement* parent, Atom tag);
+
+ bool IsRewritable(const HtmlNode* node) const;
+
+ void ClearElements();
+
+ void DebugPrintQueue(); // Print queue (for debugging)
+
+ Atom Intern(const std::string& name) {
+ return string_table_.Intern(name);
+ }
+ Atom Intern(const char* name) {
+ return string_table_.Intern(name);
+ }
+
+ // Implementation helper with detailed knowledge of html parsing libraries
+ friend class HtmlLexer;
+
+ // Determines whether a tag should be terminated in HTML.
+ bool IsImplicitlyClosedTag(Atom tag) const;
+
+ // Determines whether a tag allows brief termination in HTML, e.g. <tag/>
+ bool TagAllowsBriefTermination(Atom tag) const;
+
+ MessageHandler* message_handler() const { return message_handler_; }
+ // Gets the current location information; typically to help with error
+ // messages.
+ const char* url() const { return url_.c_str(); }
+ // Gets a parsed GURL& corresponding to url().
+ const GURL& gurl() const { return gurl_; }
+ const char* id() const { return id_.c_str(); }
+ int line_number() const { return line_number_; }
+ // Return the current assumed doctype of the document (based on the content
+ // type and any HTML directives encountered so far).
+ const DocType& doctype() const;
+
+ // Interface for any caller to report an error message via the message handler
+ void Info(const char* filename, int line, const char* msg, ...)
+ INSTAWEB_PRINTF_FORMAT(4, 5);
+ void Warning(const char* filename, int line, const char* msg, ...)
+ INSTAWEB_PRINTF_FORMAT(4, 5);
+ void Error(const char* filename, int line, const char* msg, ...)
+ INSTAWEB_PRINTF_FORMAT(4, 5);
+ void FatalError(const char* filename, int line, const char* msg, ...)
+ INSTAWEB_PRINTF_FORMAT(4, 5);
+
+ void InfoV(const char* file, int line, const char *msg, va_list args);
+ void WarningV(const char* file, int line, const char *msg, va_list args);
+ void ErrorV(const char* file, int line, const char *msg, va_list args);
+ void FatalErrorV(const char* file, int line, const char* msg, va_list args);
+
+ // Report error message with current parsing filename and linenumber.
+ void InfoHere(const char* msg, ...) INSTAWEB_PRINTF_FORMAT(2, 3);
+ void WarningHere(const char* msg, ...) INSTAWEB_PRINTF_FORMAT(2, 3);
+ void ErrorHere(const char* msg, ...) INSTAWEB_PRINTF_FORMAT(2, 3);
+ void FatalErrorHere(const char* msg, ...) INSTAWEB_PRINTF_FORMAT(2, 3);
+
+ void InfoHereV(const char *msg, va_list args) {
+ InfoV(id_.c_str(), line_number_, msg, args);
+ }
+ void WarningHereV(const char *msg, va_list args) {
+ WarningV(id_.c_str(), line_number_, msg, args);
+ }
+ void ErrorHereV(const char *msg, va_list args) {
+ ErrorV(id_.c_str(), line_number_, msg, args);
+ }
+ void FatalErrorHereV(const char* msg, va_list args) {
+ FatalErrorV(id_.c_str(), line_number_, msg, args);
+ }
+
+ void AddElement(HtmlElement* element, int line_number);
+ void CloseElement(HtmlElement* element, HtmlElement::CloseStyle close_style,
+ int line_number);
+
+ // Run a filter on the current queue of parse nodes. This is visible
+ // for testing.
+ void ApplyFilter(HtmlFilter* filter);
+
+ // Provide timer to helping to report timing of each filter. In the absense
+ // of a timer, reporting will be suppressed.
+ void set_timer(Timer* timer) { timer_ = timer; }
+
+ private:
+ HtmlEventListIterator Last(); // Last element in queue
+ bool IsInEventWindow(const HtmlEventListIterator& iter) const;
+ void InsertElementBeforeEvent(const HtmlEventListIterator& event,
+ HtmlNode* new_node);
+ void InsertElementAfterEvent(const HtmlEventListIterator& event,
+ HtmlNode* new_node);
+ void SanityCheck();
+ void CheckEventParent(HtmlEvent* event, HtmlElement* expect,
+ HtmlElement* actual);
+ void CheckParentFromAddEvent(HtmlEvent* event);
+ void FixParents(const HtmlEventListIterator& begin,
+ const HtmlEventListIterator& end_inclusive,
+ HtmlElement* new_parent);
+ void CoalesceAdjacentCharactersNodes();
+ void ShowProgress(const char* message);
+
+ // Visible for testing only, via HtmlTestingPeer
+ friend class HtmlTestingPeer;
+ void AddEvent(HtmlEvent* event);
+ void SetCurrent(HtmlNode* node);
+ void set_coalesce_characters(bool x) { coalesce_characters_ = x; }
+
+ SymbolTableInsensitive string_table_;
+ std::vector<HtmlFilter*> filters_;
+ HtmlLexer* lexer_;
+ int sequence_;
+ std::set<HtmlNode*> nodes_;
+ HtmlEventList queue_;
+ HtmlEventListIterator current_;
+ // Have we deleted current? Then we shouldn't do certain manipulations to it.
+ bool deleted_current_;
+ MessageHandler* message_handler_;
+ std::string url_;
+ GURL gurl_;
+ std::string id_; // Per-request identifier string used in error messages.
+ int line_number_;
+ bool need_sanity_check_;
+ bool coalesce_characters_;
+ bool need_coalesce_characters_;
+ int64 parse_start_time_us_;
+ Timer* timer_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlParse);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_PARSE_H_
diff --git a/trunk/src/net/instaweb/htmlparse/public/html_parse_test_base.h b/trunk/src/net/instaweb/htmlparse/public/html_parse_test_base.h
new file mode 100644
index 0000000..a949307
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/public/html_parse_test_base.h
@@ -0,0 +1,174 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Infrastructure for testing html parsing and rewriting.
+
+#ifndef NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_PARSE_TEST_BASE_H_
+#define NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_PARSE_TEST_BASE_H_
+
+#include <string>
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_writer_filter.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/mock_message_handler.h"
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/writer.h"
+
+namespace net_instaweb {
+
+class HtmlParseTestBaseNoAlloc : public testing::Test {
+ protected:
+ HtmlParseTestBaseNoAlloc()
+ : write_to_string_(&output_buffer_),
+ added_filter_(false) {
+ }
+
+ virtual void TearDown() {
+ output_buffer_.clear();
+ doctype_string_.clear();
+ }
+
+ // To make the tests more concise, we generally omit the <html>...</html>
+ // tags bracketing the input. The libxml parser will add those in
+ // if we don't have them. To avoid having that make the test data more
+ // verbose, we automatically add them in the test infrastructure, both
+ // for stimulus and expected response.
+ //
+ // This flag controls whether we also add <body>...</body> tags. In
+ // the case html_parse_test, we go ahead and add them in. In the
+ // case of the rewriter tests, we want to explicitly control/observe
+ // the head and the body so we don't add the body tags in
+ // automatically. So classes that derive from HtmlParseTestBase must
+ // override this variable to indicate which they prefer.
+ virtual bool AddBody() const = 0;
+
+ // Set a doctype string (e.g. "<!doctype html>") to be inserted before the
+ // rest of the document (for the current test only). If none is set, it
+ // defaults to the empty string.
+ void SetDoctype(const StringPiece& directive) {
+ directive.CopyToString(&doctype_string_);
+ }
+
+ std::string AddHtmlBody(const std::string& html) {
+ std::string ret = AddBody() ? "<html><body>\n" : "<html>\n";
+ ret += html + (AddBody() ? "\n</body></html>\n" : "\n</html>");
+ return ret;
+ }
+
+ // Check that the output HTML is serialized to string-compare
+ // precisely with the input.
+ void ValidateNoChanges(const StringPiece& case_id,
+ const std::string& html_input) {
+ ValidateExpected(case_id, html_input, html_input);
+ }
+
+ // Fail to ValidateNoChanges.
+ void ValidateNoChangesFail(const StringPiece& case_id,
+ const std::string& html_input) {
+ ValidateExpectedFail(case_id, html_input, html_input);
+ }
+
+ void SetupWriter() {
+ output_buffer_.clear();
+ if (html_writer_filter_.get() == NULL) {
+ html_writer_filter_.reset(new HtmlWriterFilter(html_parse()));
+ html_writer_filter_->set_writer(&write_to_string_);
+ html_parse()->AddFilter(html_writer_filter_.get());
+ }
+ }
+
+ // Parse html_input, the result is stored in output_buffer_.
+ void Parse(const StringPiece& case_id, const std::string& html_input) {
+ // HtmlParser needs a valid HTTP URL to evaluate relative paths,
+ // so we create a dummy URL.
+ std::string dummy_url = StrCat("http://test.com/", case_id, ".html");
+ ParseUrl(dummy_url, html_input);
+ }
+
+ // Parse given an explicit URL rather than an id to build URL around.
+ void ParseUrl(const StringPiece& url, const std::string& html_input) {
+ // We don't add the filter in the constructor because it needs to be the
+ // last filter added.
+ SetupWriter();
+ html_parse()->StartParse(url);
+ html_parse()->ParseText(doctype_string_ + AddHtmlBody(html_input));
+ html_parse()->FinishParse();
+ }
+
+ // Validate that the output HTML serializes as specified in
+ // 'expected', which might not be identical to the input.
+ void ValidateExpected(const StringPiece& case_id,
+ const std::string& html_input,
+ const std::string& expected) {
+ Parse(case_id, html_input);
+ std::string xbody = doctype_string_ + AddHtmlBody(expected);
+ EXPECT_EQ(xbody, output_buffer_);
+ output_buffer_.clear();
+ }
+
+ // Same as ValidateExpected, but with an explicit URL rather than an id.
+ void ValidateExpectedUrl(const StringPiece& url,
+ const std::string& html_input,
+ const std::string& expected) {
+ ParseUrl(url, html_input);
+ std::string xbody = doctype_string_ + AddHtmlBody(expected);
+ EXPECT_EQ(xbody, output_buffer_);
+ output_buffer_.clear();
+ }
+
+ // Fail to ValidateExpected.
+ void ValidateExpectedFail(const StringPiece& case_id,
+ const std::string& html_input,
+ const std::string& expected) {
+ Parse(case_id, html_input);
+ std::string xbody = AddHtmlBody(expected);
+ EXPECT_NE(xbody, output_buffer_);
+ output_buffer_.clear();
+ }
+
+ virtual HtmlParse* html_parse() = 0;
+
+ MockMessageHandler message_handler_;
+ StringWriter write_to_string_;
+ std::string output_buffer_;
+ bool added_filter_;
+ scoped_ptr<HtmlWriterFilter> html_writer_filter_;
+ std::string doctype_string_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(HtmlParseTestBaseNoAlloc);
+};
+
+class HtmlParseTestBase : public HtmlParseTestBaseNoAlloc {
+ public:
+ HtmlParseTestBase() : html_parse_(&message_handler_) {
+ };
+ protected:
+ virtual HtmlParse* html_parse() { return &html_parse_; }
+
+ HtmlParse html_parse_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(HtmlParseTestBase);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_PARSE_TEST_BASE_H_
diff --git a/trunk/src/net/instaweb/htmlparse/public/html_parser_types.h b/trunk/src/net/instaweb/htmlparse/public/html_parser_types.h
new file mode 100644
index 0000000..b444ba6
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/public/html_parser_types.h
@@ -0,0 +1,50 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_PARSER_TYPES_H_
+#define NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_PARSER_TYPES_H_
+
+#include <list>
+
+namespace net_instaweb {
+
+class FileSystem;
+class HtmlCdataNode;
+class HtmlCharactersNode;
+class HtmlCommentNode;
+class HtmlDirectiveNode;
+class HtmlElement;
+class HtmlEvent;
+class HtmlFilter;
+class HtmlIEDirectiveNode;
+class HtmlLeafNode;
+class HtmlLexer;
+class HtmlNode;
+class HtmlParse;
+class HtmlStartElementEvent;
+class HtmlWriterFilter;
+class LibxmlAdapter;
+class MessageHandler;
+class Writer;
+
+typedef std::list<HtmlEvent*> HtmlEventList;
+typedef HtmlEventList::iterator HtmlEventListIterator;
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_PARSER_TYPES_H_
diff --git a/trunk/src/net/instaweb/htmlparse/public/html_writer_filter.h b/trunk/src/net/instaweb/htmlparse/public/html_writer_filter.h
new file mode 100644
index 0000000..69d4361
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/public/html_writer_filter.h
@@ -0,0 +1,88 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_WRITER_FILTER_H_
+#define NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_WRITER_FILTER_H_
+
+#include <string.h>
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_filter.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class HtmlWriterFilter : public HtmlFilter {
+ public:
+ explicit HtmlWriterFilter(HtmlParse* html_parse);
+
+ void set_writer(Writer* writer) { writer_ = writer; }
+ virtual ~HtmlWriterFilter();
+
+ virtual void StartDocument();
+ virtual void EndDocument();
+ virtual void StartElement(HtmlElement* element);
+ virtual void EndElement(HtmlElement* element);
+ virtual void Cdata(HtmlCdataNode* cdata);
+ virtual void Comment(HtmlCommentNode* comment);
+ virtual void IEDirective(HtmlIEDirectiveNode* directive);
+ virtual void Characters(HtmlCharactersNode* characters);
+ virtual void Directive(HtmlDirectiveNode* directive);
+ virtual void Flush();
+
+ void set_max_column(int max_column) { max_column_ = max_column; }
+
+ virtual const char* Name() const { return "HtmlWriter"; }
+
+ private:
+ void EmitBytes(const StringPiece& str);
+ HtmlElement::CloseStyle GetCloseStyle(HtmlElement* element);
+ void Clear();
+
+ // Escapes arbitrary text as HTML, e.g. turning & into &. If quoteChar
+ // is non-zero, e.g. '"', then it would escape " as well.
+ void EncodeBytes(const std::string& val, int quoteChar);
+
+ HtmlParse* html_parse_;
+ Writer* writer_;
+
+ // Helps writer exploit shortcuts like <img .../> rather than writing
+ // <img ...></img>. At the end of StartElement, we defer writing the ">"
+ // until we see what's coming next. If it's the matching end_tag, then
+ // we can emit />. If something else comes first, then we have to
+ // first emit the delayed ">" before continuing.
+ HtmlElement* lazy_close_element_;
+
+ Atom symbol_a_;
+ Atom symbol_link_;
+ Atom symbol_href_;
+ Atom symbol_img_;
+ Atom symbol_script_;
+ Atom symbol_src_;
+ Atom symbol_alt_;
+ int column_;
+ int max_column_;
+ int write_errors_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlWriterFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_PUBLIC_HTML_WRITER_FILTER_H_
diff --git a/trunk/src/net/instaweb/htmlparse/public/logging_html_filter.h b/trunk/src/net/instaweb/htmlparse/public/logging_html_filter.h
new file mode 100644
index 0000000..6ff0cd6
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/public/logging_html_filter.h
@@ -0,0 +1,121 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+// html_filter that passes data through unmodified, but
+// logs statistics about the data as it goes by.
+// It should be possible to create many instances of this
+// class and insert them at different points in the rewriting flow
+// Goal is to log:
+// NUM_EXPLICIT_CLOSED - <tag> </tag> pairs
+// NUM_IMPLICIT_CLOSED - <tag> for implicitly-closed tag
+// NUM_BRIEF_CLOSED - </tag>
+// NUM_CLOSED - Sum of above three
+// NUM_UNCLOSED - <tag> without matching </tag>
+// NUM_SPURIOUS_CLOSED - </tag> without preceding <tag>; UNCOUNTED RIGHT NOW!
+// NUM_TAGS - Total number of opening tags
+// NUM_CDATA - cdata sections
+// NUM_COMMENTS - comments
+// NUM_DIRECTIVES - directives
+// NUM_DOCUMENTS - started documents
+// NUM_IE_DIRECTIVES - ie directives
+// Reporting:
+// We report this information via a StatisticsLog: filter.ToString(log)
+// Two sets of statistics (eg before and after processing) can be
+// compared using before.Equals(after),
+
+#ifndef NET_INSTAWEB_HTMLPARSE_PUBLIC_LOGGING_HTML_FILTER_H_
+#define NET_INSTAWEB_HTMLPARSE_PUBLIC_LOGGING_HTML_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include <string>
+
+namespace net_instaweb {
+
+class StatisticsLog;
+
+class LoggingFilter : public EmptyHtmlFilter {
+ public:
+ // internal names of statistics.
+ // NOTE: must match string names in kStatisticNames at top of
+ // logging_html_filter.c
+ enum Statistic {
+ MIN_STAT = 0,
+ NUM_EXPLICIT_CLOSED = 0,
+ NUM_IMPLICIT_CLOSED,
+ NUM_BRIEF_CLOSED,
+ NUM_CLOSED,
+ NUM_UNCLOSED,
+ NUM_SPURIOUS_CLOSED,
+ NUM_TAGS,
+ NUM_CDATA,
+ NUM_COMMENTS,
+ NUM_DIRECTIVES,
+ NUM_DOCUMENTS,
+ NUM_IE_DIRECTIVES,
+ MAX_STAT
+ };
+
+ LoggingFilter();
+
+ // HtmlFilter methods.
+ virtual void StartDocument();
+ virtual void StartElement(HtmlElement* element);
+ virtual void EndElement(HtmlElement* element);
+ virtual void Cdata(HtmlCdataNode* cdata);
+ virtual void Comment(HtmlCommentNode* comment);
+ virtual void IEDirective(HtmlIEDirectiveNode* directive);
+ virtual void Directive(HtmlDirectiveNode* directive);
+ virtual const char* Name() const { return "Logging"; }
+
+ // Getter for individual statistics; NO BOUNDS CHECKS.
+ inline int get(const Statistic statistic) const {
+ return stats_[statistic];
+ }
+
+ // Logging, diffing, and aggregation
+
+ // Report all statistics
+ void LogStatistics(StatisticsLog *statistics_log) const;
+
+ // Return true if all statistics in this are equal to those in that.
+ bool Equals(const LoggingFilter &that) const;
+
+ // Report all statistics in this and that which differ
+ void LogDifferences(
+ const LoggingFilter &that, StatisticsLog *statistics_log) const;
+
+ // Add all statistics in that into this.
+ void Aggregate(const LoggingFilter &that);
+
+ // Aggregate differences between two sets of statistics into this.
+ // this.get(stat) += first.get(stat) - second.get(stat)
+ void AggregateDifferences(const LoggingFilter &first,
+ const LoggingFilter &second);
+
+ void Reset();
+
+ private:
+ int stats_[MAX_STAT];
+
+ DISALLOW_COPY_AND_ASSIGN(LoggingFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_PUBLIC_LOGGING_HTML_FILTER_H_
diff --git a/trunk/src/net/instaweb/htmlparse/public/statistics_log.h b/trunk/src/net/instaweb/htmlparse/public/statistics_log.h
new file mode 100644
index 0000000..9edcb4a
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/public/statistics_log.h
@@ -0,0 +1,40 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_HTMLPARSE_PUBLIC_STATISTICS_LOG_H_
+#define NET_INSTAWEB_HTMLPARSE_PUBLIC_STATISTICS_LOG_H_
+
+#include "base/basictypes.h"
+
+namespace net_instaweb {
+
+class StatisticsLog {
+ public:
+ StatisticsLog() { }
+ virtual ~StatisticsLog();
+ virtual void LogStat(const char *statName, int value) = 0;
+ virtual void LogDifference(const char *statName,
+ int value1, int value2) = 0;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(StatisticsLog);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_HTMLPARSE_PUBLIC_STATISTICS_LOG_H_
diff --git a/trunk/src/net/instaweb/htmlparse/statistics_log.cc b/trunk/src/net/instaweb/htmlparse/statistics_log.cc
new file mode 100644
index 0000000..eddd1c6
--- /dev/null
+++ b/trunk/src/net/instaweb/htmlparse/statistics_log.cc
@@ -0,0 +1,26 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/htmlparse/public/statistics_log.h"
+
+namespace net_instaweb {
+
+StatisticsLog::~StatisticsLog() {
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/instaweb.gyp b/trunk/src/net/instaweb/instaweb.gyp
new file mode 100644
index 0000000..cb243fb
--- /dev/null
+++ b/trunk/src/net/instaweb/instaweb.gyp
@@ -0,0 +1,292 @@
+# Copyright 2009 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'variables': {
+ 'instaweb_root': '../..',
+ 'chromium_code': 1,
+ },
+ 'targets': [
+ {
+ # TODO: break this up into sub-libs (mocks, real, etc)
+ 'target_name': 'instaweb_util',
+ 'type': '<(library)',
+ 'dependencies': [
+ 'instaweb_core.gyp:instaweb_util_core',
+ '<(instaweb_root)/third_party/base64/base64.gyp:base64',
+ '<(DEPTH)/third_party/zlib/zlib.gyp:zlib',
+ '<(DEPTH)/base/base.gyp:base',
+ '<(DEPTH)/third_party/chromium/src/net/tools/dump_cache.gyp:url_to_filename_encoder',
+ '<(DEPTH)/third_party/libpagespeed/src/pagespeed/core/core.gyp:pagespeed_core',
+ '<(DEPTH)/third_party/google-sparsehash/google-sparsehash.gyp:include',
+ ],
+ 'sources': [
+ 'util/abstract_mutex.cc',
+ 'util/cache_interface.cc',
+ 'util/cache_url_async_fetcher.cc',
+ 'util/cache_url_fetcher.cc',
+ 'util/data_url.cc',
+ 'util/dummy_url_fetcher.cc',
+ 'util/fake_url_async_fetcher.cc',
+ 'util/file_cache.cc',
+ 'util/file_system.cc',
+ 'util/file_system_lock_manager.cc',
+ 'util/file_writer.cc',
+ 'util/filename_encoder.cc',
+ 'util/gzip_inflater.cc',
+ 'util/hasher.cc',
+ 'util/http_cache.cc',
+ 'util/http_response_parser.cc',
+ 'util/http_dump_url_async_writer.cc',
+ 'util/http_dump_url_fetcher.cc',
+ 'util/http_dump_url_writer.cc',
+ 'util/http_value.cc',
+ 'util/lru_cache.cc',
+ 'util/meta_data.cc',
+ 'util/md5_hasher.cc',
+ 'util/mock_hasher.cc',
+ 'util/mock_message_handler.cc',
+ 'util/mock_timer.cc',
+ 'util/named_lock_manager.cc',
+ 'util/null_message_handler.cc',
+ 'util/null_writer.cc',
+ 'util/pthread_mutex.cc',
+ 'util/ref_counted.cc',
+ 'util/rolling_hash.cc',
+ 'util/query_params.cc',
+ 'util/simple_meta_data.cc',
+ 'util/simple_stats.cc',
+ 'util/statistics.cc',
+ 'util/statistics_work_bound.cc',
+ 'util/stdio_file_system.cc',
+ 'util/threadsafe_cache.cc',
+ 'util/time_util.cc',
+ 'util/timer_based_abstract_lock.cc',
+ 'util/url_async_fetcher.cc',
+ 'util/url_escaper.cc',
+ 'util/url_fetcher.cc',
+ 'util/url_multipart_encoder.cc',
+ 'util/url_segment_encoder.cc',
+ 'util/user_agent.cc',
+ 'util/wait_url_async_fetcher.cc',
+ 'util/wget_url_fetcher.cc',
+ 'util/wildcard.cc',
+ 'util/wildcard_group.cc',
+ 'util/work_bound.cc',
+ 'util/write_through_cache.cc',
+ ],
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ },
+ },
+ {
+ 'target_name': 'instaweb_htmlparse',
+ 'type': '<(library)',
+ 'dependencies': [
+ 'instaweb_core.gyp:instaweb_htmlparse_core',
+ 'instaweb_util',
+ '<(DEPTH)/base/base.gyp:base',
+ ],
+ 'sources': [
+ 'htmlparse/file_driver.cc',
+ 'htmlparse/file_statistics_log.cc',
+ 'htmlparse/logging_html_filter.cc',
+ 'htmlparse/null_filter.cc',
+ 'htmlparse/statistics_log.cc',
+ ],
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ },
+ },
+ {
+ 'target_name': 'instaweb_rewriter_base',
+ 'type': '<(library)',
+ 'dependencies': [
+ 'instaweb_util',
+ '<(DEPTH)/base/base.gyp:base',
+ ],
+ 'sources': [
+ 'rewriter/domain_lawyer.cc',
+ 'rewriter/resource.cc',
+ 'rewriter/output_resource.cc',
+ 'rewriter/resource_namer.cc',
+ 'rewriter/resource_manager.cc',
+ ],
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ },
+ },
+ {
+ 'variables': {
+ # OpenCV has compile warnings in gcc 4.1 in a header file so turn off
+ # strict checking.
+ #
+ # TODO(jmarantz): disable the specific warning rather than
+ # turning off all warnings, and also scope this down to a
+ # minimal wrapper around the offending header file.
+ #
+ # TODO(jmarantz): figure out how to test for this failure in
+ # checkin tests, as it passes in gcc 4.2 and fails in gcc 4.1.
+ 'chromium_code': 0,
+ },
+ 'target_name': 'instaweb_rewriter_image',
+ 'type': '<(library)',
+ 'dependencies': [
+ 'instaweb_rewriter_base',
+ 'instaweb_util',
+ '<(DEPTH)/base/base.gyp:base',
+ '<(DEPTH)/third_party/libpagespeed/src/pagespeed/image_compression/image_compression.gyp:pagespeed_jpeg_optimizer',
+ '<(DEPTH)/third_party/libpagespeed/src/pagespeed/image_compression/image_compression.gyp:pagespeed_png_optimizer',
+ '<(DEPTH)/third_party/opencv/opencv.gyp:highgui',
+ ],
+ 'sources': [
+ 'rewriter/image.cc',
+ 'rewriter/image_dim.cc',
+ 'rewriter/img_rewrite_filter.cc',
+ 'rewriter/img_tag_scanner.cc',
+ ],
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ },
+ },
+ {
+ 'target_name': 'instaweb_rewriter_javascript',
+ 'type': '<(library)',
+ 'dependencies': [
+ 'instaweb_rewriter_base',
+ 'instaweb_util',
+ '<(DEPTH)/base/base.gyp:base',
+ '<(DEPTH)/third_party/libpagespeed/src/pagespeed/jsminify/js_minify.gyp:pagespeed_jsminify',
+ ],
+ 'sources': [
+ 'rewriter/javascript_code_block.cc',
+ 'rewriter/javascript_filter.cc',
+ 'rewriter/javascript_library_identification.cc',
+ ],
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ },
+ },
+ {
+ 'target_name': 'instaweb_rewriter_css',
+ 'type': '<(library)',
+ 'dependencies': [
+ 'instaweb_rewriter_base',
+ 'instaweb_util',
+ '<(DEPTH)/base/base.gyp:base',
+ '<(DEPTH)/third_party/css_parser/css_parser.gyp:css_parser',
+ ],
+ 'sources': [
+ 'rewriter/css_filter.cc',
+ 'rewriter/css_minify.cc',
+ ],
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ '<(DEPTH)/third_party/css_parser/src',
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ },
+ },
+ {
+ 'target_name': 'instaweb_rewriter',
+ 'type': '<(library)',
+ 'dependencies': [
+ 'instaweb_rewriter_base',
+ 'instaweb_core.gyp:instaweb_rewriter_html',
+ 'instaweb_rewriter_css',
+ 'instaweb_rewriter_image',
+ 'instaweb_rewriter_javascript',
+ 'instaweb_util',
+ '<(DEPTH)/base/base.gyp:base',
+ ],
+ 'sources': [
+ 'rewriter/add_head_filter.cc',
+ 'rewriter/add_instrumentation_filter.cc',
+ 'rewriter/base_tag_filter.cc',
+ 'rewriter/cache_extender.cc',
+ 'rewriter/common_filter.cc',
+ 'rewriter/css_combine_filter.cc',
+ 'rewriter/css_inline_filter.cc',
+ 'rewriter/css_move_to_head_filter.cc',
+ 'rewriter/css_outline_filter.cc',
+ 'rewriter/css_tag_scanner.cc',
+ 'rewriter/data_url_input_resource.cc',
+ 'rewriter/file_input_resource.cc',
+ 'rewriter/js_inline_filter.cc',
+ 'rewriter/js_outline_filter.cc',
+ 'rewriter/resource_tag_scanner.cc',
+ 'rewriter/rewrite_driver.cc',
+ 'rewriter/rewrite_driver_factory.cc',
+ 'rewriter/rewrite_filter.cc',
+ 'rewriter/rewrite_single_resource_filter.cc',
+ 'rewriter/rewrite_options.cc',
+ 'rewriter/script_tag_scanner.cc',
+ 'rewriter/strip_scripts_filter.cc',
+ 'rewriter/url_input_resource.cc',
+ 'rewriter/url_left_trim_filter.cc',
+ 'rewriter/url_partnership.cc',
+ ],
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ },
+ },
+ ],
+}
diff --git a/trunk/src/net/instaweb/instaweb_core.gyp b/trunk/src/net/instaweb/instaweb_core.gyp
new file mode 100644
index 0000000..4624bf8
--- /dev/null
+++ b/trunk/src/net/instaweb/instaweb_core.gyp
@@ -0,0 +1,125 @@
+# Copyright 2009 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'variables': {
+ 'instaweb_root': '../..',
+ 'chromium_code': 1,
+ },
+ 'targets': [
+ {
+ 'target_name': 'instaweb_util_core',
+ 'type': '<(library)',
+ 'dependencies': [
+ '<(DEPTH)/base/base.gyp:base',
+ ],
+ 'sources': [
+ 'util/content_type.cc',
+ 'util/file_message_handler.cc',
+ 'util/google_message_handler.cc',
+ 'util/google_url.cc',
+ 'util/message_handler.cc',
+ 'util/string_convert.cc',
+ 'util/string_util.cc',
+ 'util/string_writer.cc',
+ 'util/timer.cc',
+ 'util/writer.cc',
+ ],
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ },
+ },
+ {
+ 'target_name': 'instaweb_htmlparse_core',
+ 'type': '<(library)',
+ 'dependencies': [
+ 'instaweb_util_core',
+ '<(DEPTH)/base/base.gyp:base',
+ '<(DEPTH)/build/temp_gyp/googleurl.gyp:googleurl',
+ ],
+ 'sources': [
+ 'htmlparse/doctype.cc',
+ 'htmlparse/empty_html_filter.cc',
+ 'htmlparse/html_element.cc',
+ 'htmlparse/html_escape.cc',
+ 'htmlparse/html_event.cc',
+ 'htmlparse/html_filter.cc',
+ 'htmlparse/html_lexer.cc',
+ 'htmlparse/html_node.cc',
+ 'htmlparse/html_parse.cc',
+ 'htmlparse/html_writer_filter.cc',
+ ],
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ },
+ 'export_dependent_settings': [
+ 'instaweb_util_core',
+ ],
+ },
+ {
+ 'target_name': 'instaweb_rewriter_html',
+ 'type': '<(library)',
+ 'dependencies': [
+ 'instaweb_util_core',
+ '<(DEPTH)/base/base.gyp:base',
+ ],
+ 'sources': [
+ 'rewriter/collapse_whitespace_filter.cc',
+ 'rewriter/elide_attributes_filter.cc',
+ 'rewriter/html_attribute_quote_removal.cc',
+ 'rewriter/remove_comments_filter.cc',
+ ],
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '<(instaweb_root)',
+ '<(DEPTH)',
+ ],
+ },
+ },
+ # We build this target to make sure that we don't accidentially
+ # introduce dependencies from the core libraries to non-core
+ # libraries.
+ {
+ 'target_name': 'html_minifier_main',
+ 'type': 'executable',
+ 'dependencies': [
+ '<(DEPTH)/base/base.gyp:base',
+ 'instaweb_util_core',
+ 'instaweb_htmlparse_core',
+ 'instaweb_rewriter_html',
+ ],
+ 'sources': [
+ 'rewriter/html_minifier_main.cc',
+ ],
+ },
+ ],
+}
diff --git a/trunk/src/net/instaweb/instaweb_html_rewriter.gyp b/trunk/src/net/instaweb/instaweb_html_rewriter.gyp
new file mode 100644
index 0000000..4078c6d
--- /dev/null
+++ b/trunk/src/net/instaweb/instaweb_html_rewriter.gyp
@@ -0,0 +1,60 @@
+# Copyright 2010 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'variables': {
+ # chromium_code indicates that the code is not
+ # third-party code and should be subjected to strict compiler
+ # warnings/errors in order to catch programming mistakes.
+ 'chromium_code': 1,
+ 'instaweb_root': '<(DEPTH)/net/instaweb',
+ },
+
+ 'targets': [
+ {
+ 'target_name': 'html_rewriter',
+ 'type': '<(library)',
+ 'dependencies': [
+ '<(DEPTH)/base/base.gyp:base',
+ '<(DEPTH)/build/build_util.gyp:mod_pagespeed_version_header',
+ '<(DEPTH)/third_party/apache/httpd/httpd.gyp:include',
+ '<(DEPTH)/third_party/serf/serf.gyp:serf',
+ '<(instaweb_root)/instaweb.gyp:instaweb_htmlparse',
+ '<(instaweb_root)/instaweb.gyp:instaweb_rewriter',
+ '<(instaweb_root)/instaweb.gyp:instaweb_util',
+ ],
+ 'include_dirs': [
+ '<(DEPTH)',
+ ],
+ 'sources': [
+ 'apache/apache_message_handler.cc',
+ 'apache/apache_rewrite_driver_factory.cc',
+ 'apache/apache_slurp.cc',
+ 'apache/apr_file_system.cc',
+ 'apache/apr_mutex.cc',
+ 'apache/apr_statistics.cc',
+ 'apache/apr_timer.cc',
+ 'apache/header_util.cc',
+ 'apache/instaweb_context.cc',
+ 'apache/log_message_handler.cc',
+ 'apache/serf_url_async_fetcher.cc',
+ 'apache/serf_url_fetcher.cc',
+ 'apache/serf_async_callback.cc',
+ ],
+ 'export_dependent_settings': [
+ '<(instaweb_root)/instaweb.gyp:instaweb_util',
+ ],
+ },
+ ],
+}
diff --git a/trunk/src/net/instaweb/instaweb_mod_instaweb.gyp b/trunk/src/net/instaweb/instaweb_mod_instaweb.gyp
new file mode 100644
index 0000000..02a850c
--- /dev/null
+++ b/trunk/src/net/instaweb/instaweb_mod_instaweb.gyp
@@ -0,0 +1,128 @@
+# Copyright 2010 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'variables': {
+ # chromium_code indicates that the code is not
+ # third-party code and should be subjected to strict compiler
+ # warnings/errors in order to catch programming mistakes.
+ 'chromium_code': 1,
+ },
+
+ 'targets': [
+ {
+ 'target_name': 'mod_instaweb',
+ 'type': 'loadable_module',
+ 'dependencies': [
+ 'instaweb_html_rewriter.gyp:html_rewriter',
+ '<(DEPTH)/base/base.gyp:base',
+ '<(DEPTH)/third_party/apache/httpd/httpd.gyp:include',
+ ],
+ 'include_dirs': [
+ '<(DEPTH)',
+ ],
+ 'sources': [
+ 'apache/instaweb_handler.cc',
+ 'apache/log_message_handler.cc',
+ 'apache/mod_instaweb.cc',
+ ],
+ },
+ {
+ 'variables': {
+ # OpenCV has compile warnings in gcc 4.1 in a header file so turn off
+ # strict checking.
+ #
+ # TODO(jmarantz): disable the specific warning rather than
+ # turning off all warnings, and also scope this down to a
+ # minimal wrapper around the offending header file.
+ #
+ # TODO(jmarantz): figure out how to test for this failure in
+ # checkin tests, as it passes in gcc 4.2 and fails in gcc 4.1.
+ 'chromium_code': 0,
+ },
+ 'target_name': 'mod_instaweb_test',
+ 'type': 'executable',
+ 'dependencies': [
+ 'instaweb_html_rewriter.gyp:html_rewriter',
+ '<(DEPTH)/base/base.gyp:base',
+ '<(DEPTH)/testing/gtest.gyp:gtest',
+ '<(DEPTH)/testing/gtest.gyp:gtestmain',
+ '<(DEPTH)/third_party/apache/apr/apr.gyp:apr',
+ '<(DEPTH)/third_party/apache/aprutil/aprutil.gyp:aprutil',
+ '<(DEPTH)/third_party/apache/httpd/httpd.gyp:include',
+ '<(DEPTH)/third_party/google-sparsehash/google-sparsehash.gyp:include',
+ ],
+ 'include_dirs': [
+ '<(DEPTH)',
+ ],
+ 'sources': [
+ 'apache/serf_url_async_fetcher_test.cc',
+ 'apache/apr_file_system_test.cc',
+ 'util/base64_test.cc',
+ 'util/cache_fetcher_test.cc',
+ 'util/cache_url_async_fetcher_test.cc',
+ 'util/cache_url_fetcher_test.cc',
+ 'util/data_url_test.cc',
+ 'util/fetcher_test.cc',
+ 'util/file_cache_test.cc',
+ 'util/file_system_test.cc',
+ 'util/filename_encoder_test.cc',
+ 'util/gtest.cc',
+ 'util/gzip_inflater_test.cc',
+ 'util/http_cache_test.cc',
+ 'util/http_dump_url_async_writer_test.cc',
+ 'util/http_dump_url_fetcher_test.cc',
+ 'util/http_dump_url_writer_test.cc',
+ 'util/http_value_test.cc',
+ 'util/lru_cache_test.cc',
+ 'util/mem_file_system.cc',
+ 'util/mem_file_system_test.cc',
+ 'util/message_handler_test.cc',
+ 'util/mock_url_fetcher_test.cc',
+# 'util/simple_meta_data_test.cc',
+# 'util/simple_stats_test.cc',
+# 'util/split_writer_test.cc',
+# 'util/stdio_file_system_test.cc',
+ 'util/string_buffer_test.cc',
+ 'util/string_util_test.cc',
+ 'util/symbol_table_test.cc',
+# 'util/threadsafe_cache_test.cc',
+ 'util/time_util_test.cc',
+ 'util/url_escaper_test.cc',
+ 'util/write_through_cache_test.cc',
+ 'rewriter/collapse_whitespace_filter_test.cc',
+# TODO(sligocki): Fix. Fails in chromium/src/base/at_exit.cc(40).
+# 'rewriter/css_filter_test.cc',
+ 'rewriter/css_tag_scanner_test.cc',
+ 'rewriter/elide_attributes_filter_test.cc',
+ 'rewriter/html_attribute_quote_removal_test.cc',
+ 'rewriter/image_endian_test.cc',
+# 'rewriter/image_test.cc',
+ 'rewriter/javascript_code_block_test.cc',
+ 'rewriter/remove_comments_filter_test.cc',
+ 'rewriter/resource_encoder_test.cc',
+ 'rewriter/resource_manager_test.cc',
+ 'rewriter/resource_tag_scanner_test.cc',
+ 'rewriter/rewrite_driver_test.cc',
+ 'rewriter/rewriter_test.cc',
+ 'rewriter/script_tag_scanner_test.cc',
+ 'rewriter/strip_scripts_filter_test.cc',
+ 'rewriter/url_left_trim_filter_test.cc',
+# 'htmlparse/html_escape_test.cc',
+ 'htmlparse/html_parse_test.cc',
+ ],
+ },
+
+ ],
+}
diff --git a/trunk/src/net/instaweb/instaweb_mod_slurp.gyp b/trunk/src/net/instaweb/instaweb_mod_slurp.gyp
new file mode 100644
index 0000000..875dfa4
--- /dev/null
+++ b/trunk/src/net/instaweb/instaweb_mod_slurp.gyp
@@ -0,0 +1,38 @@
+# Copyright 2010 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'variables': {
+ # chromium_code indicates that the code is not
+ # third-party code and should be subjected to strict compiler
+ # warnings/errors in order to catch programming mistakes.
+ 'chromium_code': 1,
+ },
+
+ 'targets': [
+ {
+ 'target_name': 'mod_slurp',
+ 'type': 'loadable_module',
+ 'dependencies': [
+ 'instaweb_html_rewriter.gyp:html_rewriter',
+ '<(DEPTH)/base/base.gyp:base',
+ '<(DEPTH)/third_party/apache/httpd/httpd.gyp:include',
+ ],
+ 'sources': [
+ 'apache/mod_slurp.cc',
+ 'apache/log_message_handler.cc',
+ ],
+ },
+ ],
+}
diff --git a/trunk/src/net/instaweb/instaweb_protobuf_gzip.gyp b/trunk/src/net/instaweb/instaweb_protobuf_gzip.gyp
new file mode 100644
index 0000000..05b4966
--- /dev/null
+++ b/trunk/src/net/instaweb/instaweb_protobuf_gzip.gyp
@@ -0,0 +1,37 @@
+# Copyright 2009 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'targets': [
+ {
+ # Unfortunately, the inherited protobuf target in protobuf.gyp
+ # does not build gzip_stream.cc, which we require. Thus we're
+ # required to define our own target that includes protobuf as
+ # well as gzip_stream.cc.
+ 'target_name': 'instaweb_protobuf_gzip',
+ 'type': '<(library)',
+ 'dependencies': [
+ '<(DEPTH)/third_party/protobuf2/protobuf.gyp:protobuf',
+ '<(DEPTH)/third_party/zlib/zlib.gyp:zlib',
+ ],
+ 'sources': [
+ '<(DEPTH)/third_party/protobuf2/src/src/google/protobuf/io/gzip_stream.cc',
+ ],
+ 'export_dependent_settings': [
+ '<(DEPTH)/third_party/protobuf2/protobuf.gyp:protobuf',
+ '<(DEPTH)/third_party/zlib/zlib.gyp:zlib',
+ ],
+ },
+ ],
+}
diff --git a/trunk/src/net/instaweb/mod_pagespeed.gyp b/trunk/src/net/instaweb/mod_pagespeed.gyp
new file mode 100644
index 0000000..e0557ac
--- /dev/null
+++ b/trunk/src/net/instaweb/mod_pagespeed.gyp
@@ -0,0 +1,160 @@
+# Copyright 2010 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# TODO(sligocki): This is a confusing name for the gyp file where tests live.
+
+{
+ 'variables': {
+ # chromium_code indicates that the code is not
+ # third-party code and should be subjected to strict compiler
+ # warnings/errors in order to catch programming mistakes.
+ 'chromium_code': 1,
+ },
+
+ 'targets': [
+ {
+ 'target_name': 'mod_pagespeed',
+ 'type': 'loadable_module',
+ 'dependencies': [
+ 'instaweb_html_rewriter.gyp:html_rewriter',
+ '<(DEPTH)/base/base.gyp:base',
+ '<(DEPTH)/third_party/apache/httpd/httpd.gyp:include',
+ '<(DEPTH)/build/build_util.gyp:mod_pagespeed_version_header',
+ ],
+ 'include_dirs': [
+ '<(DEPTH)',
+ ],
+ 'sources': [
+ 'apache/apache_config.cc',
+ 'apache/instaweb_handler.cc',
+ 'apache/log_message_handler.cc',
+ "apache/mem_clean_up.cc",
+ 'apache/mem_debug.cc',
+ 'apache/mod_instaweb.cc',
+ ],
+ },
+ {
+ 'variables': {
+ # OpenCV has compile warnings in gcc 4.1 in a header file so turn off
+ # strict checking.
+ #
+ # TODO(jmarantz): disable the specific warning rather than
+ # turning off all warnings, and also scope this down to a
+ # minimal wrapper around the offending header file.
+ #
+ # TODO(jmarantz): figure out how to test for this failure in
+ # checkin tests, as it passes in gcc 4.2 and fails in gcc 4.1.
+ 'chromium_code': 0,
+ },
+ 'target_name': 'mod_pagespeed_test',
+ 'type': 'executable',
+ 'dependencies': [
+ 'instaweb_html_rewriter.gyp:html_rewriter',
+ '<(DEPTH)/base/base.gyp:base',
+ '<(DEPTH)/build/build_util.gyp:mod_pagespeed_version_header',
+ '<(DEPTH)/testing/gtest.gyp:gtest',
+ '<(DEPTH)/testing/gtest.gyp:gtestmain',
+ '<(DEPTH)/third_party/apache/apr/apr.gyp:apr',
+ '<(DEPTH)/third_party/apache/aprutil/aprutil.gyp:aprutil',
+ '<(DEPTH)/third_party/apache/httpd/httpd.gyp:include',
+ '<(DEPTH)/third_party/google-sparsehash/google-sparsehash.gyp:include',
+ ],
+ 'include_dirs': [
+ '<(DEPTH)',
+ ],
+ 'sources': [
+ 'apache/serf_url_async_fetcher_test.cc',
+ 'apache/apr_file_system_test.cc',
+ "apache/mem_clean_up.cc",
+ 'apache/mem_debug.cc',
+ 'util/base64_test.cc',
+ 'util/cache_fetcher_test.cc',
+ 'util/cache_url_async_fetcher_test.cc',
+ 'util/cache_url_fetcher_test.cc',
+ 'util/content_type_test.cc',
+ 'util/data_url_test.cc',
+ 'util/fetcher_test.cc',
+ 'util/file_cache_test.cc',
+ 'util/file_system_test.cc',
+ 'util/filename_encoder_test.cc',
+ 'util/google_url_test.cc',
+ 'util/gtest.cc',
+ 'util/gzip_inflater_test.cc',
+ 'util/http_cache_test.cc',
+ 'util/http_dump_url_async_writer_test.cc',
+ 'util/http_dump_url_fetcher_test.cc',
+ 'util/http_dump_url_writer_test.cc',
+ 'util/http_value_test.cc',
+ 'util/lru_cache_test.cc',
+ 'util/mem_file_system.cc',
+ 'util/mem_file_system_test.cc',
+ 'util/message_handler_test.cc',
+ 'util/md5_hasher_test.cc',
+ 'util/mock_message_handler_test.cc',
+ 'util/mock_url_fetcher.cc',
+ 'util/mock_url_fetcher_test.cc',
+# 'util/simple_meta_data_test.cc',
+# 'util/simple_stats_test.cc',
+ 'util/statistics_work_bound_test.cc',
+# 'util/split_writer_test.cc',
+# 'util/stdio_file_system_test.cc',
+ 'util/string_util_test.cc',
+ 'util/symbol_table_test.cc',
+# 'util/threadsafe_cache_test.cc',
+ 'util/time_util_test.cc',
+ 'util/url_escaper_test.cc',
+ 'util/url_multipart_encoder_test.cc',
+ 'util/user_agent_test.cc',
+ 'util/wait_url_async_fetcher_test.cc',
+ 'util/wildcard_test.cc',
+ 'util/wildcard_group_test.cc',
+ 'util/write_through_cache_test.cc',
+ 'rewriter/cache_extender_test.cc',
+ 'rewriter/collapse_whitespace_filter_test.cc',
+ 'rewriter/common_filter_test.cc',
+ 'rewriter/css_combine_filter_test.cc',
+ 'rewriter/css_inline_filter_test.cc',
+ 'rewriter/css_outline_filter_test.cc',
+ 'rewriter/css_filter_test.cc',
+ 'rewriter/css_move_to_head_filter_test.cc',
+ 'rewriter/css_tag_scanner_test.cc',
+ 'rewriter/domain_lawyer_test.cc',
+ 'rewriter/elide_attributes_filter_test.cc',
+ 'rewriter/html_attribute_quote_removal_test.cc',
+ 'rewriter/image_endian_test.cc',
+ 'rewriter/image_rewrite_filter_test.cc',
+# 'rewriter/image_test.cc',
+ 'rewriter/javascript_code_block_test.cc',
+ 'rewriter/javascript_filter_test.cc',
+ 'rewriter/js_inline_filter_test.cc',
+ 'rewriter/js_outline_filter_test.cc',
+ 'rewriter/remove_comments_filter_test.cc',
+ 'rewriter/resource_namer_test.cc',
+ 'rewriter/resource_manager_test.cc',
+ 'rewriter/resource_manager_test_base.cc',
+ 'rewriter/resource_tag_scanner_test.cc',
+ 'rewriter/rewrite_driver_test.cc',
+ 'rewriter/rewrite_options_test.cc',
+ 'rewriter/rewriter_test.cc',
+ 'rewriter/script_tag_scanner_test.cc',
+ 'rewriter/strip_scripts_filter_test.cc',
+ 'rewriter/url_left_trim_filter_test.cc',
+ 'rewriter/url_partnership_test.cc',
+# 'htmlparse/html_escape_test.cc',
+ 'htmlparse/html_parse_test.cc',
+ ],
+ },
+
+ ],
+}
diff --git a/trunk/src/net/instaweb/public/VERSION b/trunk/src/net/instaweb/public/VERSION
new file mode 100644
index 0000000..a23377b
--- /dev/null
+++ b/trunk/src/net/instaweb/public/VERSION
@@ -0,0 +1,4 @@
+MAJOR=0
+MINOR=9
+BUILD=0
+PATCH=0
diff --git a/trunk/src/net/instaweb/public/global_constants.h b/trunk/src/net/instaweb/public/global_constants.h
new file mode 100644
index 0000000..fd98deb
--- /dev/null
+++ b/trunk/src/net/instaweb/public/global_constants.h
@@ -0,0 +1,34 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: pradnya@google.com (Pradnya Karbhari)
+
+// Header file that includes global constants to be used in net/instaweb.
+
+#ifndef NET_INSTAWEB_PUBLIC_GLOBAL_CONSTANTS_H_
+#define NET_INSTAWEB_PUBLIC_GLOBAL_CONSTANTS_H_
+
+namespace {
+
+// Time of day used in Chromium when running javascript in deterministic mode
+// (with flag --no-js-randomness). We use the same time of day for slurping,
+// validation and measurement in order to maintain consistency.
+static const double kChromiumTimeOfDay = 1204251968254LL;
+const char kModPagespeedHeader[] = "X-Mod-Pagespeed";
+
+} // namespace
+
+#endif // NET_INSTAWEB_PUBLIC_GLOBAL_CONSTANTS_H_
diff --git a/trunk/src/net/instaweb/public/version.h.in b/trunk/src/net/instaweb/public/version.h.in
new file mode 100644
index 0000000..f9db8bb
--- /dev/null
+++ b/trunk/src/net/instaweb/public/version.h.in
@@ -0,0 +1,25 @@
+// Copyright (c) 2010 The Chromium Authors. All rights reserved.
+// Use of this source is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+// version.h is generated from version.h.in. Edit the source!
+
+#pragma once
+
+// Version Information
+
+#define MOD_PAGESPEED_VERSION @MAJOR@,@MINOR@,@BUILD@,@PATCH@
+#define MOD_PAGESPEED_VERSION_STRING "@MAJOR@.@MINOR@.@BUILD@.@PATCH@"
+
+// Branding Information
+
+#define COMPANY_FULLNAME_STRING "@COMPANY_FULLNAME@"
+#define COMPANY_SHORTNAME_STRING "@COMPANY_SHORTNAME@"
+#define PRODUCT_FULLNAME_STRING "@PRODUCT_FULLNAME@"
+#define PRODUCT_SHORTNAME_STRING "@PRODUCT_SHORTNAME@"
+#define COPYRIGHT_STRING "@COPYRIGHT@"
+#define OFFICIAL_BUILD_STRING "@OFFICIAL_BUILD@"
+
+// Changelist Information
+
+#define LASTCHANGE_STRING "@LASTCHANGE@"
diff --git a/trunk/src/net/instaweb/rewriter/add_head_filter.cc b/trunk/src/net/instaweb/rewriter/add_head_filter.cc
new file mode 100644
index 0000000..7bcafe0
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/add_head_filter.cc
@@ -0,0 +1,84 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "public/add_head_filter.h"
+
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+
+namespace net_instaweb {
+
+AddHeadFilter::AddHeadFilter(HtmlParse* html_parse, bool combine_multiple_heads)
+ : html_parse_(html_parse),
+ combine_multiple_heads_(combine_multiple_heads),
+ found_head_(false),
+ s_head_(html_parse->Intern("head")),
+ s_body_(html_parse->Intern("body")),
+ head_element_(NULL) {
+}
+
+void AddHeadFilter::StartDocument() {
+ found_head_ = false;
+ head_element_ = NULL;
+}
+
+void AddHeadFilter::StartElement(HtmlElement* element) {
+ if (!found_head_) {
+ if (element->tag() == s_body_) {
+ head_element_ = html_parse_->NewElement(
+ element->parent(), s_head_);
+ html_parse_->InsertElementBeforeElement(element, head_element_);
+ found_head_ = true;
+ } else if (element->tag() == s_head_) {
+ found_head_ = true;
+ head_element_ = element;
+ }
+ }
+}
+
+void AddHeadFilter::EndElement(HtmlElement* element) {
+ // Detect elements that are children of a subsequent head. Move them
+ // into the first head if possible.
+ if (combine_multiple_heads_ && found_head_ && (head_element_ != NULL) &&
+ html_parse_->IsRewritable(head_element_)) {
+ if ((element->tag() == s_head_) && (element != head_element_)) {
+ // There should be no elements left in the subsequent head, so
+ // remove it.
+ bool moved = html_parse_->MoveCurrentInto(head_element_);
+ CHECK(moved) << "failed to move new head under old head";
+ bool deleted = html_parse_->DeleteSavingChildren(element);
+ CHECK(deleted) << "failed to delete extra head";
+ }
+ }
+}
+
+void AddHeadFilter::Flush() {
+ head_element_ = NULL;
+}
+
+void AddHeadFilter::EndDocument() {
+ if (!found_head_) {
+ // In order to insert a <head> in a docucument that lacks one, we
+ // must first find the body. If we get through the whole doc without
+ // finding a <head> or a <body> then this filter will have failed to
+ // add a head.
+ html_parse_->InfoHere("Reached end of document without finding <body>");
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/add_instrumentation_filter.cc b/trunk/src/net/instaweb/rewriter/add_instrumentation_filter.cc
new file mode 100644
index 0000000..021262a
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/add_instrumentation_filter.cc
@@ -0,0 +1,129 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: abliss@google.com (Adam Bliss)
+
+#include "public/add_instrumentation_filter.h"
+
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/util/public/statistics.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// The javascript tag to insert in the top of the <head> element. We want this
+// as early as possible in the html. It must be short and fast.
+const char kHeadScript[] =
+ "<script type='text/javascript'>"
+ "window.mod_pagespeed_start = Number(new Date());"
+ "</script>";
+
+// The javascript tag to insert at the bottom of document. The %s will be
+// replaced with the custom beacon url, by default
+// "./mod_pagespeed_beacon?ets=". Then our timing info, e.g. "load:123", will
+// be appended.
+const char kTailScript[] =
+ "<script type='text/javascript'>"
+ "function g(){new Image().src='%sload:'+"
+ "(Number(new Date())-window.mod_pagespeed_start);};"
+ "var f=window.addEventListener;if(f){f('load',g,false);}else{"
+ "f=window.attachEvent;if(f){f('onload',g);}}"
+ "</script>";
+
+// Timing tag for total page load time. Also embedded in kTailScript above!
+const char kLoadTag[] = "load:";
+
+// Variables for the beacon to increment. These are currently handled in
+// mod_pagespeed_handler on apache. The average load time in milliseconds is
+// total_page_load_ms / page_load_count. Note that these are not updated
+// together atomically, so you might get a slightly bogus value.
+const char kTotalPageLoadMs[] = "total_page_load_ms";
+const char kPageLoadCount[] = "page_load_count";
+
+} // namespace
+
+AddInstrumentationFilter::AddInstrumentationFilter(
+ HtmlParse* html_parse, const StringPiece& beacon_url, Statistics* stats)
+ : html_parse_(html_parse),
+ found_head_(false),
+ s_head_(html_parse->Intern("head")),
+ s_body_(html_parse->Intern("body")),
+ total_page_load_ms_((stats == NULL) ? NULL :
+ stats->GetVariable(kTotalPageLoadMs)),
+ page_load_count_((stats == NULL) ? NULL :
+ stats->GetVariable(kPageLoadCount)) {
+ beacon_url.CopyToString(&beacon_url_);
+}
+
+void AddInstrumentationFilter::Initialize(Statistics* statistics) {
+ statistics->AddVariable(kTotalPageLoadMs);
+ statistics->AddVariable(kPageLoadCount);
+}
+
+void AddInstrumentationFilter::StartDocument() {
+ found_head_ = false;
+}
+
+void AddInstrumentationFilter::StartElement(HtmlElement* element) {
+ if (!found_head_) {
+ if (element->tag() == s_head_) {
+ found_head_ = true;
+ // TODO(abliss): add an actual element instead, so other filters can
+ // rewrite this JS
+ HtmlCharactersNode* script =
+ html_parse_->NewCharactersNode(element, kHeadScript);
+ html_parse_->InsertElementAfterCurrent(script);
+ }
+ }
+}
+
+void AddInstrumentationFilter::EndElement(HtmlElement* element) {
+ if (element->tag() == s_body_) {
+ // We relied on the existence of a <head> element. This should have been
+ // assured by add_head_filter.
+ CHECK(found_head_) << "Reached end of document without finding <head>."
+ " Please turn on the add_head filter.";
+ std::string tailScript = StringPrintf(kTailScript, beacon_url_.c_str());
+ HtmlCharactersNode* script =
+ html_parse_->NewCharactersNode(element, tailScript);
+ html_parse_->InsertElementBeforeCurrent(script);
+ }
+}
+
+bool AddInstrumentationFilter::HandleBeacon(const StringPiece& unparsed_url) {
+ if ((total_page_load_ms_ == NULL) || (page_load_count_ == NULL)) {
+ return false;
+ }
+ std::string url = unparsed_url.as_string();
+ // TODO(abliss): proper query parsing
+ size_t index = url.find(kLoadTag);
+ if (index == std::string::npos) {
+ return false;
+ }
+ url = url.substr(index + strlen(kLoadTag));
+ int value = 0;
+ if (!StringToInt(url, &value)) {
+ return false;
+ }
+ total_page_load_ms_->Add(value);
+ page_load_count_->Add(1);
+ return true;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/base_tag_filter.cc b/trunk/src/net/instaweb/rewriter/base_tag_filter.cc
new file mode 100644
index 0000000..6fb1cdb
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/base_tag_filter.cc
@@ -0,0 +1,65 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "public/base_tag_filter.h"
+
+#include "public/add_head_filter.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include <string>
+
+namespace net_instaweb {
+BaseTagFilter::BaseTagFilter(HtmlParse* html_parse) {
+ s_head_ = html_parse->Intern("head");
+ s_base_ = html_parse->Intern("base");
+ s_href_ = html_parse->Intern("href");
+ found_head_ = false;
+ html_parse_ = html_parse;
+}
+
+void BaseTagFilter::StartDocument() {
+ found_head_ = false;
+}
+
+// In a proxy server, we will want to set a base tag according to the current
+// URL being processed. But we need to add the BaseTagFilter upstream of
+// the HtmlWriterFilter, so we'll need to establish it at init time before
+// we know a URL. So in that mode, where we've installed the filter but
+// have no specific URL to set the base tag to, then we should avoid
+// adding an empty base tag.
+void BaseTagFilter::StartElement(HtmlElement* element) {
+ if ((element->tag() == s_head_) && !found_head_) {
+ found_head_ = true;
+ HtmlElement* new_element = html_parse_->NewElement(element, s_base_);
+ new_element->set_close_style(HtmlElement::IMPLICIT_CLOSE);
+ new_element->AddAttribute(s_href_, base_url_.c_str(), "\"");
+ html_parse_->InsertElementAfterCurrent(new_element);
+ } else if (element->tag() == s_base_) {
+ // There is was a pre-existing base tag. See if it specifies an href.
+ // If so, delete it, as it's now superseded by the one we added above.
+ for (int i = 0; i < element->attribute_size(); ++i) {
+ HtmlElement::Attribute& attribute = element->attribute(i);
+ if (attribute.name() == s_href_) {
+ html_parse_->DeleteElement(element);
+ break;
+ }
+ }
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/cache_extender.cc b/trunk/src/net/instaweb/rewriter/cache_extender.cc
new file mode 100644
index 0000000..9b90769
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/cache_extender.cc
@@ -0,0 +1,164 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/cache_extender.h"
+
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/css_tag_scanner.h"
+#include "net/instaweb/rewriter/public/output_resource.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/util/public/hasher.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/meta_data.h"
+#include "net/instaweb/util/public/statistics.h"
+#include <string>
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/timer.h"
+#include "net/instaweb/util/public/url_escaper.h"
+
+namespace {
+
+// names for Statistics variables.
+const char kCacheExtensions[] = "cache_extensions";
+const char kNotCacheable[] = "not_cacheable";
+
+} // namespace
+
+namespace net_instaweb {
+
+// We do not want to bother to extend the cache lifetime for any resource
+// that is already cached for a month.
+const int64 kMinThresholdMs = Timer::kMonthMs;
+
+// TODO(jmarantz): consider factoring out the code that finds external resources
+
+CacheExtender::CacheExtender(RewriteDriver* driver, const char* filter_prefix)
+ : RewriteSingleResourceFilter(driver, filter_prefix),
+ html_parse_(driver->html_parse()),
+ resource_manager_(driver->resource_manager()),
+ tag_scanner_(html_parse_),
+ extension_count_(NULL),
+ not_cacheable_count_(NULL) {
+ Statistics* stats = resource_manager_->statistics();
+ if (stats != NULL) {
+ extension_count_ = stats->GetVariable(kCacheExtensions);
+ not_cacheable_count_ = stats->GetVariable(kNotCacheable);
+ }
+}
+
+void CacheExtender::Initialize(Statistics* statistics) {
+ statistics->AddVariable(kCacheExtensions);
+ statistics->AddVariable(kNotCacheable);
+}
+
+void CacheExtender::StartElementImpl(HtmlElement* element) {
+ MessageHandler* message_handler = html_parse_->message_handler();
+ HtmlElement::Attribute* href = tag_scanner_.ScanElement(element);
+ if ((href != NULL) && html_parse_->IsRewritable(element)) {
+ scoped_ptr<Resource> input_resource(CreateInputResource(href->value()));
+ if ((input_resource.get() != NULL) &&
+ !IsRewrittenResource(input_resource->url()) &&
+ resource_manager_->ReadIfCached(input_resource.get(),
+ message_handler)) {
+ const MetaData* headers = input_resource->metadata();
+ int64 now_ms = resource_manager_->timer()->NowMs();
+
+ // We cannot cache-extend a resource that's completely uncacheable,
+ // as our serving-side image would b
+ if (!resource_manager_->http_cache()->force_caching() &&
+ !headers->IsCacheable()) {
+ if (not_cacheable_count_ != NULL) {
+ not_cacheable_count_->Add(1);
+ }
+ } else if (((headers->CacheExpirationTimeMs() - now_ms) <
+ kMinThresholdMs) &&
+ (input_resource->type() != NULL)) {
+ scoped_ptr<OutputResource> output(
+ resource_manager_->CreateOutputResourceFromResource(
+ filter_prefix_, input_resource->type(),
+ resource_manager_->url_escaper(), input_resource.get(),
+ message_handler));
+ if (output.get() != NULL) {
+ CHECK(!output->IsWritten());
+
+ // TODO(sligocki): Shouldn't we be rewriting if !IsWritten?
+ if (!output->HasValidUrl()) {
+ // Transfer the input resource to output resource.
+ RewriteLoadedResource(input_resource.get(), output.get());
+ }
+
+ if (output->HasValidUrl()) {
+ href->SetValue(output->url());
+ if (extension_count_ != NULL) {
+ extension_count_->Add(1);
+ }
+ }
+ }
+ }
+ }
+ }
+}
+
+// Just based on the pattern of the URL, see if we think this was
+// already the result of a rewrite. It should, in general, be
+// functionally correct to apply a new filter to an
+// already-rewritten resoure. However, in the case of cache
+// extension, there is no benefit because every rewriter generates
+// URLs that are served with long cache lifetimes. This filter
+// just wants to pick up the scraps. Note that we would discover
+// this anywahy in the cache expiration time below, but it's worth
+// going to the extra trouble to reduce the cache lookups since this
+// happens for basically every resource.
+bool CacheExtender::IsRewrittenResource(const StringPiece& url) const {
+ RewriteFilter* filter;
+ scoped_ptr<OutputResource> output_resource(driver_->DecodeOutputResource(
+ url, &filter));
+ return (output_resource.get() != NULL);
+}
+
+bool CacheExtender::RewriteLoadedResource(const Resource* input_resource,
+ OutputResource* output_resource) {
+ CHECK(input_resource->loaded());
+
+ MessageHandler* message_handler = html_parse_->message_handler();
+ StringPiece contents(input_resource->contents());
+ std::string absolutified;
+ std::string input_dir =
+ GoogleUrl::AllExceptLeaf(GoogleUrl::Create(input_resource->url()));
+ if ((input_resource->type() == &kContentTypeCss) &&
+ (input_dir != output_resource->resolved_base())) {
+ // TODO(jmarantz): find a mechanism to write this directly into
+ // the HTTPValue so we can reduce the number of times that we
+ // copy entire resources.
+ StringWriter writer(&absolutified);
+ CssTagScanner::AbsolutifyUrls(contents, input_resource->url(),
+ &writer, message_handler);
+ contents = absolutified;
+ }
+ // TODO(sligocki): Should we preserve the response headers from the
+ // original resource?
+ // TODO(sligocki): Maybe we shouldn't cache the rewritten resource,
+ // just the input_resource.
+ return resource_manager_->Write(
+ HttpStatus::kOK, contents, output_resource,
+ input_resource->metadata()->CacheExpirationTimeMs(), message_handler);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/cache_extender_test.cc b/trunk/src/net/instaweb/rewriter/cache_extender_test.cc
new file mode 100644
index 0000000..1589bd8
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/cache_extender_test.cc
@@ -0,0 +1,148 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the cache extender.
+
+
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/cache_extender.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+#include "net/instaweb/rewriter/public/resource_namer.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/string_writer.h"
+
+namespace net_instaweb {
+
+namespace {
+
+static const char kDomain[] = "http://test.com/";
+
+const char kHtmlFormat[] =
+ "<link rel='stylesheet' href='%s' type='text/css'>\n"
+ "<img src='%s'/>\n"
+ "<script type='text/javascript' src='%s'></script>\n";
+
+const char kCssData[] = ".blue {color: blue;}";
+const char kImageData[] = "Invalid JPEG but it does not matter for this test";
+const char kJsData[] = "alert('hello, world!')";
+const char kFilterId[] = "ce";
+
+class CacheExtenderTest : public ResourceManagerTestBase {
+ protected:
+
+ virtual void SetUp() {
+ ResourceManagerTestBase::SetUp();
+ }
+
+ void InitTest(int64 ttl) {
+ AddFilter(RewriteOptions::kExtendCache);
+ InitMetaData("a.css", kContentTypeCss, kCssData, ttl);
+ InitMetaData("b.jpg", kContentTypeJpeg, kImageData, ttl);
+ InitMetaData("c.js", kContentTypeJavascript, kJsData, ttl);
+ }
+
+ // Generate HTML loading 3 resources with the specified URLs
+ std::string GenerateHtml(const std::string& a,
+ const std::string& b,
+ const std::string& c) {
+ return StringPrintf(kHtmlFormat, a.c_str(), b.c_str(), c.c_str());
+ }
+};
+
+TEST_F(CacheExtenderTest, DoExtend) {
+ InitTest(100);
+ for (int i = 0; i < 3; i++) {
+ ValidateExpected(
+ "do_extend",
+ GenerateHtml("a.css", "b.jpg", "c.js"),
+ GenerateHtml(
+ Encode(kDomain, "ce", "0", "a.css", "css"),
+ Encode(kDomain, "ce", "0", "b.jpg", "jpg"),
+ Encode(kDomain, "ce", "0", "c.js", "js")));
+ }
+}
+
+TEST_F(CacheExtenderTest, NoExtendAlreadyCachedProperly) {
+ InitTest(100000000); // cached for a long time to begin with
+ ValidateExpected("no_extend_cached_properly",
+ GenerateHtml("a.css", "b.jpg", "c.js"),
+ GenerateHtml("a.css", "b.jpg", "c.js"));
+}
+
+TEST_F(CacheExtenderTest, NoExtendOriginUncacheable) {
+ InitTest(0); // origin not cacheable
+ ValidateExpected("no_extend_origin_not_cacheable",
+ GenerateHtml("a.css", "b.jpg", "c.js"),
+ GenerateHtml("a.css", "b.jpg", "c.js"));
+}
+
+TEST_F(CacheExtenderTest, ServeFiles) {
+ std::string content;
+
+ InitTest(100);
+ ASSERT_TRUE(ServeResource(kDomain, kFilterId, "a.css", "css", &content));
+ EXPECT_EQ(std::string(kCssData), content);
+ ASSERT_TRUE(ServeResource(kDomain, kFilterId, "b.jpg", "jpg", &content));
+ EXPECT_EQ(std::string(kImageData), content);
+ ASSERT_TRUE(ServeResource(kDomain, kFilterId, "c.js", "js", &content));
+ EXPECT_EQ(std::string(kJsData), content);
+}
+
+TEST_F(CacheExtenderTest, ServeFilesFromDelayedFetch) {
+ InitTest(100);
+ ServeResourceFromManyContexts(Encode(kDomain, "ce", "0", "a.css", "css"),
+ RewriteOptions::kExtendCache,
+ &mock_hasher_, kCssData);
+ ServeResourceFromManyContexts(Encode(kDomain, "ce", "0", "b.jpg", "jpg"),
+ RewriteOptions::kExtendCache,
+ &mock_hasher_, kImageData);
+ ServeResourceFromManyContexts(Encode(kDomain, "ce", "0", "c.js", "js"),
+ RewriteOptions::kExtendCache,
+ &mock_hasher_, kJsData);
+
+ // TODO(jmarantz): make ServeResourceFromManyContexts check:
+ // 1. Gets the data from the cache, with no mock fetchers, null file system
+ // 2. Gets the data from the file system, with no cache, no mock fetchers.
+ // 3. Gets the data from the mock fetchers: no cache, no file system.
+}
+
+TEST_F(CacheExtenderTest, MinimizeCacheHits) {
+ options_.EnableFilter(RewriteOptions::kOutlineCss);
+ options_.EnableFilter(RewriteOptions::kExtendCache);
+ options_.set_css_outline_min_bytes(1);
+ rewrite_driver_.AddFilters();
+ std::string html_input = StrCat("<style>", kCssData, "</style>");
+ std::string html_output = StringPrintf(
+ "<link rel='stylesheet' href='%s'>",
+ Encode(kDomain, "co", "0", "_", "css").c_str());
+ ValidateExpected("no_extend_origin_not_cacheable", html_input, html_output);
+
+ // The key thing about this test is that the CacheExtendFilter should
+ // not pound the cache looking to see if it's already rewritten this
+ // resource. If we try, in the cache extend filter, to this already-optimized
+ // resource from the cache, then we'll get a cache-hit and decide that
+ // it's already got a long cache lifetime. But we should know, just from
+ // the name of the resource, that it should not be cache extended.
+ EXPECT_EQ(0, lru_cache_->num_hits());
+ EXPECT_EQ(1, lru_cache_->num_misses());
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/collapse_whitespace_filter.cc b/trunk/src/net/instaweb/rewriter/collapse_whitespace_filter.cc
new file mode 100644
index 0000000..d12520a
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/collapse_whitespace_filter.cc
@@ -0,0 +1,112 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "net/instaweb/rewriter/public/collapse_whitespace_filter.h"
+
+#include "base/logging.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_node.h"
+#include <string>
+
+namespace {
+
+// Tags within which we should never try to collapse whitespace (note that this
+// is not _quite_ the same thing as kLiteralTags in html_lexer.cc):
+const char* const kSensitiveTags[] = {"pre", "script", "style", "textarea"};
+
+bool IsHtmlWhiteSpace(char ch) {
+ // See http://www.w3.org/TR/html401/struct/text.html#h-9.1
+ return ch == ' ' || ch == '\n' || ch == '\r' || ch == '\t' || ch == '\f';
+}
+
+// Sentinel value for use in the CollapseWhitespace function:
+const char kNotInWhitespace = '\0';
+
+// Append the input to the output with whitespace collapsed. Specifically,
+// each contiguous sequence of whitespace is replaced with the first
+// (whitespace) character in the sequence, except that any sequence containing
+// a newline is collapsed to a newline.
+void CollapseWhitespace(const std::string& input, std::string* output) {
+ // This variable stores the first whitespace character in each whitespace
+ // sequence, or kNotInWhitespace.
+ char whitespace = kNotInWhitespace;
+ for (std::string::const_iterator iter = input.begin(), end = input.end();
+ iter != end; ++iter) {
+ const char ch = *iter;
+ if (IsHtmlWhiteSpace(ch)) {
+ // We let newlines take precedence over other kinds of whitespace, for
+ // aesthetic reasons.
+ if (whitespace == kNotInWhitespace || ch == '\n') {
+ whitespace = ch;
+ }
+ } else {
+ if (whitespace != kNotInWhitespace) {
+ *output += whitespace;
+ whitespace = kNotInWhitespace;
+ }
+ *output += ch;
+ }
+ }
+ if (whitespace != kNotInWhitespace) {
+ *output += whitespace;
+ }
+}
+
+} // namespace
+
+namespace net_instaweb {
+
+CollapseWhitespaceFilter::CollapseWhitespaceFilter(HtmlParse* html_parse)
+ : html_parse_(html_parse) {
+ for (size_t i = 0; i < arraysize(kSensitiveTags); ++i) {
+ sensitive_tags_.insert(html_parse->Intern(kSensitiveTags[i]));
+ }
+}
+
+void CollapseWhitespaceFilter::StartDocument() {
+ atom_stack_.clear();
+}
+
+void CollapseWhitespaceFilter::StartElement(HtmlElement* element) {
+ const Atom tag = element->tag();
+ if (sensitive_tags_.count(tag) > 0) {
+ atom_stack_.push_back(tag);
+ }
+}
+
+void CollapseWhitespaceFilter::EndElement(HtmlElement* element) {
+ const Atom tag = element->tag();
+ if (!atom_stack_.empty() && tag == atom_stack_.back()) {
+ atom_stack_.pop_back();
+ } else {
+ DCHECK(sensitive_tags_.count(tag) == 0);
+ }
+}
+
+void CollapseWhitespaceFilter::Characters(HtmlCharactersNode* characters) {
+ if (atom_stack_.empty()) {
+ std::string minified;
+ CollapseWhitespace(characters->contents(), &minified);
+ html_parse_->ReplaceNode(
+ characters,
+ html_parse_->NewCharactersNode(characters->parent(), minified));
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/collapse_whitespace_filter_test.cc b/trunk/src/net/instaweb/rewriter/collapse_whitespace_filter_test.cc
new file mode 100644
index 0000000..76a6c43
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/collapse_whitespace_filter_test.cc
@@ -0,0 +1,96 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_parse_test_base.h"
+#include "net/instaweb/rewriter/public/collapse_whitespace_filter.h"
+
+namespace net_instaweb {
+
+class CollapseWhitespaceFilterTest : public HtmlParseTestBase {
+ protected:
+ CollapseWhitespaceFilterTest()
+ : collapse_whitespace_filter_(&html_parse_) {
+ html_parse_.AddFilter(&collapse_whitespace_filter_);
+ }
+
+ virtual bool AddBody() const { return false; }
+
+ private:
+ CollapseWhitespaceFilter collapse_whitespace_filter_;
+
+ DISALLOW_COPY_AND_ASSIGN(CollapseWhitespaceFilterTest);
+};
+
+TEST_F(CollapseWhitespaceFilterTest, NoChange) {
+ ValidateNoChanges("no_change",
+ "<head><title>Hello</title></head>"
+ "<body>Why, hello there!</body>");
+}
+
+TEST_F(CollapseWhitespaceFilterTest, CollapseWhitespace) {
+ ValidateExpected("collapse_whitespace",
+ "<body>hello world, it\n"
+ " is good to see you </body>",
+ "<body>hello world, it\n"
+ "is good to see you </body>");
+}
+
+TEST_F(CollapseWhitespaceFilterTest, NewlineTakesPrecedence) {
+ ValidateExpected("newline_takes_precedence",
+ "<body>hello world, it \n"
+ " is good to see you</body>",
+ "<body>hello world, it\n"
+ "is good to see you</body>");
+}
+
+TEST_F(CollapseWhitespaceFilterTest, DoNotCollapseWithinPre) {
+ ValidateNoChanges("do_not_collapse_within_pre",
+ "<body><pre>hello world, it\n"
+ " is good to see you </pre></body>");
+}
+
+TEST_F(CollapseWhitespaceFilterTest, CollapseAfterNestedPre) {
+ ValidateExpected("collapse_after_nested_pre",
+ "<body><pre>hello <pre>world, it</pre>\n"
+ " is good</pre> to see you </body>",
+ "<body><pre>hello <pre>world, it</pre>\n"
+ " is good</pre> to see you </body>");
+}
+
+TEST_F(CollapseWhitespaceFilterTest, DoNotCollapseWithinScript) {
+ ValidateExpected("do_not_collapse_within_script",
+ "<head><script>x = \"don't collapse\"</script></head>"
+ "<body>do collapse</body>",
+ "<head><script>x = \"don't collapse\"</script></head>"
+ "<body>do collapse</body>");
+}
+
+TEST_F(CollapseWhitespaceFilterTest, DoNotCollapseWithinStyle) {
+ ValidateNoChanges("do_not_collapse_within_style",
+ "<head><style>P{font-family:\"don't collapse\";}</style>"
+ "</head><body></body>");
+}
+
+TEST_F(CollapseWhitespaceFilterTest, DoNotCollapseWithinTextarea) {
+ ValidateNoChanges("do_not_collapse_within_textarea",
+ "<body><textarea>hello world, it\n"
+ " is good to see you </textarea></body>");
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/common_filter.cc b/trunk/src/net/instaweb/rewriter/common_filter.cc
new file mode 100644
index 0000000..d4e4cb2
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/common_filter.cc
@@ -0,0 +1,112 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/rewriter/public/common_filter.h"
+
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+
+namespace net_instaweb {
+
+CommonFilter::CommonFilter(RewriteDriver* driver)
+ : driver_(driver),
+ html_parse_(driver->html_parse()),
+ s_base_(html_parse_->Intern("base")),
+ s_href_(html_parse_->Intern("href")),
+ s_noscript_(html_parse_->Intern("noscript")) {}
+
+CommonFilter::~CommonFilter() {}
+
+void CommonFilter::StartDocument() {
+ // Base URL starts as document URL.
+ base_gurl_ = html_parse_->gurl();
+ noscript_element_ = NULL;
+ // Run the actual filter's StartDocumentImpl.
+ StartDocumentImpl();
+}
+
+void CommonFilter::StartElement(HtmlElement* element) {
+ // <base>
+ if (element->tag() == s_base_) {
+ HtmlElement::Attribute* href = element->FindAttribute(s_href_);
+ if (href != NULL) {
+ GURL temp_url(href->value());
+ if (temp_url.is_valid()) {
+ base_gurl_.Swap(&temp_url);
+ }
+ }
+
+ // <noscript>
+ } else if (element->tag() == s_noscript_) {
+ if (noscript_element_ == NULL) {
+ noscript_element_ = element; // Record top-level <noscript>
+ }
+ }
+
+ // Run actual filter's StartElementImpl
+ StartElementImpl(element);
+}
+
+void CommonFilter::EndElement(HtmlElement* element) {
+ if (element == noscript_element_) {
+ noscript_element_ = NULL; // We are exitting the top-level <noscript>
+ }
+
+ // Run actual filter's EndElementImpl
+ EndElementImpl(element);
+}
+
+// TODO(jmarantz): While it is expedient to route the input-resource creation
+// through this base class, it's not clear this is the right place. I think
+// it would be easier to argue that it should be RewriteDriver. But this
+// fitler had access to the base_gurl which is very convenient, and was already
+// the base class of all filters that needed this. We should revisit and
+// decide where this should go.
+//
+// jmaessen also points out that we have not been symmetric: we have ignored
+// OutputResources here. In any case, we should strive for consistency.
+Resource* CommonFilter::CreateInputResource(const StringPiece& url) {
+ ResourceManager* resource_manager = driver_->resource_manager();
+ return resource_manager->CreateInputResource(
+ base_gurl(), url, driver_->options(), html_parse()->message_handler());
+}
+
+Resource* CommonFilter::CreateInputResourceAbsolute(const StringPiece& url) {
+ ResourceManager* resource_manager = driver_->resource_manager();
+ return resource_manager->CreateInputResourceAbsolute(
+ url, driver_->options(), html_parse()->message_handler());
+}
+
+Resource* CommonFilter::CreateInputResourceAndReadIfCached(
+ const StringPiece& url) {
+ ResourceManager* resource_manager = driver_->resource_manager();
+ return resource_manager->CreateInputResourceAndReadIfCached(
+ base_gurl(), url, driver_->options(), html_parse_->message_handler());
+}
+
+Resource* CommonFilter::CreateInputResourceFromOutputResource(
+ UrlSegmentEncoder* encoder, OutputResource* output_resource) {
+ ResourceManager* resource_manager = driver_->resource_manager();
+ return resource_manager->CreateInputResourceFromOutputResource(
+ encoder, output_resource, driver_->options(),
+ html_parse_->message_handler());
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/common_filter_test.cc b/trunk/src/net/instaweb/rewriter/common_filter_test.cc
new file mode 100644
index 0000000..39d4734
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/common_filter_test.cc
@@ -0,0 +1,198 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/rewriter/public/common_filter.h"
+
+#include "base/scoped_ptr.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+
+namespace net_instaweb {
+
+namespace {
+
+class CountingFilter : public CommonFilter {
+ public:
+ CountingFilter(RewriteDriver* driver) : CommonFilter(driver),
+ start_doc_calls_(0),
+ start_element_calls_(0),
+ end_element_calls_(0) {}
+
+ virtual void StartDocumentImpl() { ++start_doc_calls_; }
+ virtual void StartElementImpl(HtmlElement* element) {
+ ++start_element_calls_;
+ }
+ virtual void EndElementImpl(HtmlElement* element) { ++end_element_calls_; }
+
+ virtual const char* Name() const { return "CommonFilterTest.CountingFilter"; }
+
+ int start_doc_calls_;
+ int start_element_calls_;
+ int end_element_calls_;
+};
+
+class CommonFilterTest : public ResourceManagerTestBase {
+ protected:
+ CommonFilterTest() : html_parse_(rewrite_driver_.html_parse()),
+ filter_(&rewrite_driver_) {
+ html_parse_->AddFilter(&filter_);
+ }
+
+ void ExpectUrl(const std::string& expected_url, const GURL& actual_gurl) {
+ LOG(INFO) << actual_gurl;
+ EXPECT_EQ(expected_url, GoogleUrl::Spec(actual_gurl));
+ }
+
+ bool CanRewriteResource(CommonFilter* filter, const StringPiece& url) {
+ scoped_ptr<Resource> resource(filter->CreateInputResource(url));
+ return (resource.get() != NULL);
+ }
+
+ CommonFilter* MakeFilter(const StringPiece& base_url,
+ const StringPiece& domain,
+ RewriteOptions* options,
+ RewriteDriver* driver) {
+ options->domain_lawyer()->AddDomain(domain, &message_handler_);
+ CountingFilter* filter = new CountingFilter(driver);
+ driver->AddFilter(filter);
+ driver->html_parse()->StartParse(base_url);
+ driver->html_parse()->Flush();
+ return filter;
+ }
+
+ GoogleMessageHandler handler_;
+ HtmlParse* html_parse_;
+ CountingFilter filter_;
+};
+
+TEST_F(CommonFilterTest, DoesCallImpls) {
+ EXPECT_EQ(0, filter_.start_doc_calls_);
+ filter_.StartDocument();
+ EXPECT_EQ(1, filter_.start_doc_calls_);
+
+ HtmlElement* element =
+ html_parse_->NewElement(NULL, html_parse_->Intern("foo"));
+ EXPECT_EQ(0, filter_.start_element_calls_);
+ filter_.StartElement(element);
+ EXPECT_EQ(1, filter_.start_element_calls_);
+
+ EXPECT_EQ(0, filter_.end_element_calls_);
+ filter_.EndElement(element);
+ EXPECT_EQ(1, filter_.end_element_calls_);
+}
+
+TEST_F(CommonFilterTest, StoresCorrectBaseUrl) {
+ std::string doc_url = "http://www.example.com/";
+ html_parse_->StartParse(doc_url);
+ html_parse_->Flush();
+ // Base URL starts out as document URL.
+ ExpectUrl(doc_url, html_parse_->gurl());
+ ExpectUrl(doc_url, filter_.base_gurl());
+
+ html_parse_->ParseText("<html><head><link rel='stylesheet' href='foo.css'>");
+ html_parse_->Flush();
+ ExpectUrl(doc_url, filter_.base_gurl());
+
+ std::string base_url = "http://www.baseurl.com/foo/";
+ html_parse_->ParseText("<base href='");
+ html_parse_->ParseText(base_url);
+ html_parse_->ParseText("' />");
+ html_parse_->Flush();
+ // Update to base URL.
+ ExpectUrl(base_url, filter_.base_gurl());
+ // Make sure we didn't change the document URL.
+ ExpectUrl(doc_url, html_parse_->gurl());
+
+ html_parse_->ParseText("<link rel='stylesheet' href='foo.css'>");
+ html_parse_->Flush();
+ ExpectUrl(base_url, filter_.base_gurl());
+
+ std::string new_base_url = "http://www.somewhere-else.com/";
+ html_parse_->ParseText("<base href='");
+ html_parse_->ParseText(new_base_url);
+ html_parse_->ParseText("' />");
+ html_parse_->Flush();
+ // Uses new base URL.
+ ExpectUrl(new_base_url, filter_.base_gurl());
+
+ html_parse_->ParseText("</head></html>");
+ html_parse_->FinishParse();
+ ExpectUrl(new_base_url, filter_.base_gurl());
+ ExpectUrl(doc_url, html_parse_->gurl());
+}
+
+TEST_F(CommonFilterTest, DetectsNoScriptCorrectly) {
+ std::string doc_url = "http://www.example.com/";
+ html_parse_->StartParse(doc_url);
+ html_parse_->Flush();
+ EXPECT_TRUE(filter_.noscript_element() == NULL);
+
+ html_parse_->ParseText("<html><head><title>Example Site");
+ html_parse_->Flush();
+ EXPECT_TRUE(filter_.noscript_element() == NULL);
+
+ html_parse_->ParseText("</title><noscript>");
+ html_parse_->Flush();
+ EXPECT_TRUE(filter_.noscript_element() != NULL);
+
+ // Nested <noscript> elements
+ html_parse_->ParseText("Blah blah blah <noscript><noscript> do-de-do-do ");
+ html_parse_->Flush();
+ EXPECT_TRUE(filter_.noscript_element() != NULL);
+
+ html_parse_->ParseText("<link href='style.css'>");
+ html_parse_->Flush();
+ EXPECT_TRUE(filter_.noscript_element() != NULL);
+
+ // Close inner <noscript>s
+ html_parse_->ParseText("</noscript></noscript>");
+ html_parse_->Flush();
+ EXPECT_TRUE(filter_.noscript_element() != NULL);
+
+ // Close outter <noscript>
+ html_parse_->ParseText("</noscript>");
+ html_parse_->Flush();
+ EXPECT_TRUE(filter_.noscript_element() == NULL);
+
+ html_parse_->ParseText("</head></html>");
+ html_parse_->FinishParse();
+ EXPECT_TRUE(filter_.noscript_element() == NULL);
+
+}
+
+TEST_F(CommonFilterTest, TestTwoDomainLawyers) {
+ static const char kBaseUrl[] = "http://www.base.com/";
+ CommonFilter* a = MakeFilter(kBaseUrl, "a.com", &options_, &rewrite_driver_);
+ CommonFilter* b = MakeFilter(kBaseUrl, "b.com", &other_options_,
+ &other_rewrite_driver_);
+
+ // Either filter can rewrite resources from the base URL
+ EXPECT_TRUE(CanRewriteResource(a, StrCat(kBaseUrl, "base.css")));
+ EXPECT_TRUE(CanRewriteResource(b, StrCat(kBaseUrl, "base.css")));
+
+ // But the other domains are specific to the two different drivers/filters
+ EXPECT_TRUE(CanRewriteResource(a, "http://a.com/a.css"));
+ EXPECT_FALSE(CanRewriteResource(a, "http://b.com/b.css"));
+ EXPECT_FALSE(CanRewriteResource(b, "http://a.com/a.css"));
+ EXPECT_TRUE(CanRewriteResource(b, "http://b.com/b.css"));
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/css_combine_filter.cc b/trunk/src/net/instaweb/rewriter/css_combine_filter.cc
new file mode 100644
index 0000000..c40fa6b
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/css_combine_filter.cc
@@ -0,0 +1,560 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/css_combine_filter.h"
+
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/output_resource.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/resource_namer.h"
+#include "net/instaweb/rewriter/public/rewrite_options.h"
+#include "net/instaweb/rewriter/public/url_partnership.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/hasher.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/statistics.h"
+#include "net/instaweb/util/public/stl_util.h"
+#include <string>
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/url_escaper.h"
+#include "net/instaweb/util/public/url_multipart_encoder.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// names for Statistics variables.
+const char kCssFileCountReduction[] = "css_file_count_reduction";
+
+// TODO(jmarantz): This is arguably fragile.
+//
+// Another approach is to put a CHECK that the final URL with the
+// resource naming does not exceed the limit.
+//
+// Another option too is to just instantiate a ResourceNamer and a
+// hasher put in the correct ID and EXT and leave the name blank and
+// take size of that.
+const int kIdOverhead = 2; // strlen("cc")
+const int kExtOverhead = 3; // strlen("css")
+const int kUrlOverhead = kIdOverhead + ResourceNamer::kOverhead + kExtOverhead;
+
+} // namespace
+
+class CssCombineFilter::Partnership : public UrlPartnership {
+ public:
+ Partnership(RewriteDriver* driver, const GURL& gurl)
+ : UrlPartnership(driver->options(), gurl),
+ rewrite_options_(driver->options()),
+ resource_manager_(driver->resource_manager()),
+ prev_num_components_(0),
+ accumulated_leaf_size_(0) {
+ }
+
+ virtual ~Partnership() {
+ STLDeleteElements(&resources_);
+ }
+
+ bool AddElement(HtmlElement* element, const StringPiece& href,
+ const StringPiece& media, scoped_ptr<Resource>* resource,
+ MessageHandler* handler) {
+ // Assert the sanity of three parallel vectors.
+ CHECK_EQ(num_urls(), static_cast<int>(resources_.size()));
+ CHECK_EQ(num_urls(), static_cast<int>(css_elements_.size()));
+ CHECK_EQ(num_urls(), static_cast<int>(multipart_encoder_.num_urls()));
+
+ bool added = true;
+ if (num_urls() == 0) {
+ // TODO(jmarantz): do media='' and media='display mean the same
+ // thing? sligocki thinks mdsteele looked into this and it
+ // depended on HTML version. In one display was default, in the
+ // other screen was IIRC.
+ media.CopyToString(&media_);
+ } else {
+ // After the first CSS file, subsequent CSS files must have matching
+ // media and no @import tags.
+ added = ((media_ == media) &&
+ !CssTagScanner::HasImport((*resource)->contents(), handler));
+ }
+ if (added) {
+ added = AddUrl(href, handler);
+ }
+ if (added) {
+ int index = num_urls() - 1;
+ CHECK_EQ(index, static_cast<int>(css_elements_.size()));
+
+ if (num_components() != prev_num_components_) {
+ UpdateResolvedBase();
+ }
+ const std::string relative_path = RelativePath(index);
+ multipart_encoder_.AddUrl(relative_path);
+
+ if (accumulated_leaf_size_ == 0) {
+ ComputeLeafSize();
+ } else {
+ AccumulateLeafSize(relative_path);
+ }
+
+ if (UrlTooBig()) {
+ added = false;
+ RemoveLast();
+ multipart_encoder_.pop_back();
+
+ // The base might have changed again
+ if (num_components() != prev_num_components_) {
+ UpdateResolvedBase();
+ }
+ } else {
+ css_elements_.push_back(element);
+ resources_.push_back(resource->release());
+ }
+ }
+ return added;
+ }
+
+ // Computes a name for the URL that meets all known character-set and
+ // size restrictions.
+ std::string UrlSafeId() const {
+ UrlEscaper* escaper = resource_manager_->url_escaper();
+ std::string segment;
+ escaper->EncodeToUrlSegment(multipart_encoder_.Encode(), &segment);
+ return segment;
+ }
+
+ // Computes the total size
+ void ComputeLeafSize() {
+ std::string segment = UrlSafeId();
+ // TODO(sligocki): Use hasher for custom overhead.
+ accumulated_leaf_size_ = segment.size() + kUrlOverhead
+ + resource_manager_->hasher()->HashSizeInChars();
+ }
+
+ // Incrementally updates the accumulated leaf size without re-examining
+ // every element in the combined css file.
+ void AccumulateLeafSize(const StringPiece& url) {
+ std::string segment;
+ UrlEscaper* escaper = resource_manager_->url_escaper();
+ escaper->EncodeToUrlSegment(url, &segment);
+ const int kMultipartOverhead = 1; // for the '+'
+ accumulated_leaf_size_ += segment.size() + kMultipartOverhead;
+ }
+
+ // Determines whether our accumulated leaf size is too big, taking into
+ // account both per-segment and total-url limitations.
+ bool UrlTooBig() {
+ if (accumulated_leaf_size_ > rewrite_options_->max_url_segment_size()) {
+ return true;
+ }
+ if ((accumulated_leaf_size_ + static_cast<int>(resolved_base_.size())) >
+ rewrite_options_->max_url_size()) {
+ return true;
+ }
+ return false;
+ }
+
+ void UpdateResolvedBase() {
+ // If the addition of this URL changes the base path,
+ // then we will have to recompute the multi-part encoding.
+ // This is n^2 in the pathalogical case and if this code
+ // is copied from CSS combining to image spriting then this
+ // algorithm should be revisited. For CSS we expect N to
+ // be relatively small.
+ prev_num_components_ = num_components();
+ resolved_base_ = ResolvedBase();
+ multipart_encoder_.clear();
+ for (size_t i = 0; i < css_elements_.size(); ++i) {
+ multipart_encoder_.AddUrl(RelativePath(i));
+ }
+
+ accumulated_leaf_size_ = 0;
+ }
+
+ HtmlElement* element(int i) { return css_elements_[i]; }
+ const ResourceVector& resources() const { return resources_; }
+ const std::string& media() const { return media_; }
+
+ private:
+ const RewriteOptions* rewrite_options_;
+ ResourceManager* resource_manager_;
+ std::vector<HtmlElement*> css_elements_;
+ std::vector<Resource*> resources_;
+ UrlMultipartEncoder multipart_encoder_;
+ int prev_num_components_;
+ int accumulated_leaf_size_;
+ std::string resolved_base_;
+ std::string media_;
+};
+
+// TODO(jmarantz) We exhibit zero intelligence about which css files to
+// combine; we combine whatever is possible. This can reduce performance
+// by combining highly cacheable shared resources with transient ones.
+//
+// TODO(jmarantz): We do not recognize IE directives as spriting boundaries.
+// We should supply a meaningful IEDirective method as a boundary.
+//
+// TODO(jmarantz): allow combining of CSS elements found in the body, whether
+// or not the head has already been flushed.
+
+CssCombineFilter::CssCombineFilter(RewriteDriver* driver,
+ const char* filter_prefix)
+ : RewriteFilter(driver, filter_prefix),
+ html_parse_(driver->html_parse()),
+ resource_manager_(driver->resource_manager()),
+ css_tag_scanner_(html_parse_),
+ css_file_count_reduction_(NULL) {
+ // This CHECK is here because RewriteDriver is constructed with it's
+ // resource_manager_ == NULL.
+ // TODO(sligocki): Construct RewriteDriver with a ResourceManager.
+ CHECK(resource_manager_ != NULL);
+ s_link_ = html_parse_->Intern("link");
+ s_href_ = html_parse_->Intern("href");
+ s_type_ = html_parse_->Intern("type");
+ s_rel_ = html_parse_->Intern("rel");
+ s_media_ = html_parse_->Intern("media");
+ s_style_ = html_parse_->Intern("style");
+ Statistics* stats = resource_manager_->statistics();
+ if (stats != NULL) {
+ css_file_count_reduction_ = stats->GetVariable(kCssFileCountReduction);
+ }
+}
+
+CssCombineFilter::~CssCombineFilter() {
+}
+
+void CssCombineFilter::Initialize(Statistics* statistics) {
+ statistics->AddVariable(kCssFileCountReduction);
+}
+
+void CssCombineFilter::StartDocumentImpl() {
+ // This should already be clear, but just in case.
+ partnership_.reset(new Partnership(driver_, base_gurl()));
+}
+
+void CssCombineFilter::EndElementImpl(HtmlElement* element) {
+ HtmlElement::Attribute* href;
+ const char* media;
+ if (css_tag_scanner_.ParseCssElement(element, &href, &media)) {
+ // We cannot combine with a link in <noscript> tag and we cannot combine
+ // over a link in a <noscript> tag, so this is a barrier.
+ if (noscript_element() != NULL) {
+ TryCombineAccumulated();
+ } else {
+ const char* url = href->value();
+ MessageHandler* handler = html_parse_->message_handler();
+ scoped_ptr<Resource> resource(CreateInputResource(url));
+ if (resource.get() == NULL) {
+ TryCombineAccumulated();
+ } else {
+ if (!resource_manager_->ReadIfCached(resource.get(), handler) ||
+ !resource->ContentsValid()) {
+ TryCombineAccumulated();
+ } else if (!partnership_->AddElement(element, url, media, &resource,
+ handler)) {
+ TryCombineAccumulated();
+
+ // Now we'll try to start a new partnership with this CSS file --
+ // perhaps we ran out out of space in the previous combination
+ // or this file is simply in a different authorized domain, or
+ // contained @Import.
+ partnership_->AddElement(element, url, media, &resource, handler);
+ }
+ }
+ }
+ } else if (element->tag() == s_style_) {
+ // We can't reorder styles on a page, so if we are only combining <link>
+ // tags, we can't combine them across a <style> tag.
+ // TODO(sligocki): Maybe we should just combine <style>s too?
+ // We can run outline_css first for now to make all <style>s into <link>s.
+ TryCombineAccumulated();
+ }
+}
+
+// An IE directive that includes any stylesheet info should be a barrier
+// for css combining. It's OK to emit the combination we've seen so far.
+void CssCombineFilter::IEDirective(HtmlIEDirectiveNode* directive) {
+ // TODO(sligocki): Figure out how to safely parse IEDirectives, for now we
+ // just consider them black boxes / solid barriers.
+ TryCombineAccumulated();
+}
+
+void CssCombineFilter::Flush() {
+ TryCombineAccumulated();
+}
+
+void CssCombineFilter::TryCombineAccumulated() {
+ CHECK(partnership_.get() != NULL);
+ MessageHandler* handler = html_parse_->message_handler();
+ if (partnership_->num_urls() > 1) {
+ // Ideally like to have a data-driven service tell us which elements should
+ // be combined together. Note that both the resources and the elements
+ // are managed, so we don't delete them even if the spriting fails.
+
+ // First, compute the name of the new resource based on the names of
+ // the CSS files.
+ std::string url_safe_id = partnership_->UrlSafeId();
+ HtmlElement* combine_element = html_parse_->NewElement(NULL, s_link_);
+ combine_element->AddAttribute(s_rel_, "stylesheet", "\"");
+ combine_element->AddAttribute(s_type_, "text/css", "\"");
+ StringPiece media = partnership_->media();
+ if (!media.empty()) {
+ combine_element->AddAttribute(s_media_, media, "\"");
+ }
+
+ // Start building up the combination. At this point we are still
+ // not committed to the combination, because the 'write' can fail.
+ // TODO(jmaessen, jmarantz): encode based on partnership
+ scoped_ptr<OutputResource> combination(
+ resource_manager_->CreateOutputResourceWithPath(
+ partnership_->ResolvedBase(),
+ filter_prefix_, url_safe_id, &kContentTypeCss, handler));
+ bool written = combination->IsWritten() ||
+ WriteCombination(partnership_->resources(), combination.get(), handler);
+
+ // We've collected at least two CSS files to combine, and whose
+ // HTML elements are in the current flush window. Last step
+ // is to write the combination.
+ if (written && combination->IsWritten()) {
+ // Commit by adding combine element ...
+ combine_element->AddAttribute(s_href_, combination->url(), "\"");
+ // TODO(sligocki): Put at top of head/flush-window.
+ // Right now we're putting it where the first original element used to be.
+ html_parse_->InsertElementBeforeElement(partnership_->element(0),
+ combine_element);
+ // ... and removing originals from the DOM.
+ for (int i = 0; i < partnership_->num_urls(); ++i) {
+ html_parse_->DeleteElement(partnership_->element(i));
+ }
+ html_parse_->InfoHere("Combined %d CSS files into one at %s",
+ partnership_->num_urls(),
+ combination->url().c_str());
+ if (css_file_count_reduction_ != NULL) {
+ css_file_count_reduction_->Add(partnership_->num_urls() - 1);
+ }
+ }
+ }
+ partnership_.reset(new Partnership(driver_, base_gurl()));
+}
+
+bool CssCombineFilter::WriteCombination(const ResourceVector& combine_resources,
+ OutputResource* combination,
+ MessageHandler* handler) {
+ bool written = true;
+ // TODO(sligocki): Write directly to a temp file rather than doing the extra
+ // string copy.
+ std::string combined_contents;
+ StringWriter writer(&combined_contents);
+ int64 min_origin_expiration_time_ms = 0;
+
+ for (int i = 0, n = combine_resources.size(); written && (i < n); ++i) {
+ Resource* input = combine_resources[i];
+ StringPiece contents = input->contents();
+ int64 input_expire_time_ms = input->CacheExpirationTimeMs();
+ if ((min_origin_expiration_time_ms == 0) ||
+ (input_expire_time_ms < min_origin_expiration_time_ms)) {
+ min_origin_expiration_time_ms = input_expire_time_ms;
+ }
+
+ std::string input_dir =
+ GoogleUrl::AllExceptLeaf(GoogleUrl::Create(input->url()));
+ if (input_dir == combination->resolved_base()) {
+ // We don't need to absolutify URLs if input directory is same as output.
+ written = writer.Write(contents, handler);
+ } else {
+ // If they are different directories, we need to absolutify.
+ // TODO(sligocki): Perhaps we should use the real CSS parser.
+ written = css_tag_scanner_.AbsolutifyUrls(contents, input->url(),
+ &writer, handler);
+ }
+ }
+ if (written) {
+ written =
+ resource_manager_->Write(
+ HttpStatus::kOK, combined_contents, combination,
+ min_origin_expiration_time_ms, handler);
+ }
+ return written;
+}
+
+// Callback to run whenever a CSS resource is collected. This keeps a
+// count of the resources collected so far. When the last one is collected,
+// it aggregates the results and calls the final callback with the result.
+class CssCombiner : public Resource::AsyncCallback {
+ public:
+ CssCombiner(CssCombineFilter* filter,
+ MessageHandler* handler,
+ UrlAsyncFetcher::Callback* callback,
+ OutputResource* combination,
+ Writer* writer,
+ MetaData* response_headers) :
+ enable_completion_(false),
+ emit_done_(true),
+ done_count_(0),
+ fail_count_(0),
+ filter_(filter),
+ message_handler_(handler),
+ callback_(callback),
+ combination_(combination),
+ writer_(writer),
+ response_headers_(response_headers) {
+ }
+
+ virtual ~CssCombiner() {
+ STLDeleteContainerPointers(combine_resources_.begin(),
+ combine_resources_.end());
+ }
+
+ // Note that the passed-in resource might be NULL; this gives us a chance
+ // to note failure.
+ bool AddResource(Resource* resource) {
+ bool ret = false;
+ if (resource == NULL) {
+ // Whoops, we've failed to even obtain a resource.
+ ++fail_count_;
+ } else if (fail_count_ > 0) {
+ // Another of the resource fetches failed. Drop this resource
+ // and don't fetch it.
+ delete resource;
+ } else {
+ combine_resources_.push_back(resource);
+ ret = true;
+ }
+ return ret;
+ }
+
+ virtual void Done(bool success, Resource* resource) {
+ if (!success) {
+ ++fail_count_;
+ }
+ ++done_count_;
+
+ if (Ready()) {
+ DoCombination();
+ }
+ }
+
+ bool Ready() {
+ return (enable_completion_ &&
+ (done_count_ == combine_resources_.size()));
+ }
+
+ void EnableCompletion() {
+ enable_completion_ = true;
+ if (Ready()) {
+ DoCombination();
+ }
+ }
+
+ void DoCombination() {
+ bool ok = fail_count_ == 0;
+ for (size_t i = 0; ok && (i < combine_resources_.size()); ++i) {
+ Resource* css_resource = combine_resources_[i];
+ ok = css_resource->ContentsValid();
+ }
+ if (ok) {
+ ok = (filter_->WriteCombination(combine_resources_, combination_,
+ message_handler_) &&
+ combination_->IsWritten() &&
+ ((writer_ == NULL) ||
+ writer_->Write(combination_->contents(), message_handler_)));
+ // Above code fills in combination_->metadata(); now propagate to
+ // repsonse_headers_.
+ }
+ if (ok) {
+ response_headers_->CopyFrom(*combination_->metadata());
+ } else {
+ response_headers_->SetStatusAndReason(HttpStatus::kNotFound);
+ }
+
+ if (emit_done_) {
+ callback_->Done(ok);
+ }
+ delete this;
+ }
+
+ void set_emit_done(bool new_emit_done) {
+ emit_done_ = new_emit_done;
+ }
+
+ private:
+ bool enable_completion_;
+ bool emit_done_;
+ size_t done_count_;
+ size_t fail_count_;
+ CssCombineFilter* filter_;
+ MessageHandler* message_handler_;
+ UrlAsyncFetcher::Callback* callback_;
+ OutputResource* combination_;
+ CssCombineFilter::ResourceVector combine_resources_;
+ Writer* writer_;
+ MetaData* response_headers_;
+
+ DISALLOW_COPY_AND_ASSIGN(CssCombiner);
+};
+
+bool CssCombineFilter::Fetch(OutputResource* combination,
+ Writer* writer,
+ const MetaData& request_header,
+ MetaData* response_headers,
+ MessageHandler* message_handler,
+ UrlAsyncFetcher::Callback* callback) {
+ CHECK(response_headers != NULL);
+ bool ret = false;
+ StringPiece url_safe_id = combination->name();
+ UrlMultipartEncoder multipart_encoder;
+ UrlEscaper* escaper = resource_manager_->url_escaper();
+ std::string multipart_encoding;
+ GURL gurl(combination->url());
+ if (gurl.is_valid() &&
+ escaper->DecodeFromUrlSegment(url_safe_id, &multipart_encoding) &&
+ multipart_encoder.Decode(multipart_encoding, message_handler)) {
+ std::string url, decoded_resource;
+ ret = true;
+ CssCombiner* combiner = new CssCombiner(
+ this, message_handler, callback, combination, writer, response_headers);
+
+ std::string root = GoogleUrl::AllExceptLeaf(gurl);
+ for (int i = 0; ret && (i < multipart_encoder.num_urls()); ++i) {
+ std::string url = StrCat(root, multipart_encoder.url(i));
+ Resource* css_resource = CreateInputResourceAbsolute(url);
+ ret = combiner->AddResource(css_resource);
+ if (ret) {
+ resource_manager_->ReadAsync(css_resource, combiner, message_handler);
+ }
+ }
+
+ // If we're about to return false, we do not want the combiner to emit
+ // Done as RewriteDriver::FetchResource will do it as well.
+ combiner->set_emit_done(ret);
+
+ // In the case where the first input CSS files is already cached,
+ // ReadAsync will directly call the CssCombineCallback, which, if
+ // already enabled, would think it was complete and run DoCombination
+ // prematurely. So we wait until the resources are all added before
+ // enabling the callback to complete.
+ combiner->EnableCompletion();
+ } else {
+ message_handler->Error(url_safe_id.as_string().c_str(), 0,
+ "Unable to decode resource string");
+ }
+ return ret;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/css_combine_filter_test.cc b/trunk/src/net/instaweb/rewriter/css_combine_filter_test.cc
new file mode 100644
index 0000000..bb8b29e
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/css_combine_filter_test.cc
@@ -0,0 +1,837 @@
+/*
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+// and sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/rewriter/public/css_tag_scanner.h"
+#include "net/instaweb/util/public/url_multipart_encoder.h"
+#include "net/instaweb/util/public/url_escaper.h"
+
+namespace net_instaweb {
+
+namespace {
+
+const char kDomain[] = "http://combine_css.test/";
+const char kYellow[] = ".yellow {background-color: yellow;}";
+const char kBlue[] = ".blue {color: blue;}\n";
+
+class CssCombineFilterTest : public ResourceManagerTestBase {
+ protected:
+ // Test spriting CSS with options to write headers and use a hasher.
+ void CombineCss(const StringPiece& id, Hasher* hasher,
+ const char* barrier_text, bool is_barrier) {
+ CombineCssWithNames(id, hasher, barrier_text, is_barrier, "a.css", "b.css");
+ }
+
+ void CombineCssWithNames(const StringPiece& id, Hasher* hasher,
+ const char* barrier_text, bool is_barrier,
+ const char* a_css_name,
+ const char* b_css_name) {
+ resource_manager_->set_hasher(hasher);
+ other_resource_manager_.set_hasher(hasher);
+
+ AddFilter(RewriteOptions::kCombineCss);
+ AddOtherFilter(RewriteOptions::kCombineCss);
+
+ // URLs and content for HTML document and resources.
+ CHECK_EQ(StringPiece::npos, id.find("/"));
+ std::string html_url = StrCat(kDomain, id, ".html");
+ std::string a_css_url = StrCat(kDomain, a_css_name);
+ std::string b_css_url = StrCat(kDomain, b_css_name);
+ std::string c_css_url = StrCat(kDomain, "c.css");
+
+ static const char html_input_format[] =
+ "<head>\n"
+ " <link rel='stylesheet' href='%s' type='text/css'>\n"
+ " <link rel='stylesheet' href='%s' type='text/css'>\n"
+ " <title>Hello, Instaweb</title>\n"
+ "%s"
+ "</head>\n"
+ "<body>\n"
+ " <div class=\"c1\">\n"
+ " <div class=\"c2\">\n"
+ " Yellow on Blue\n"
+ " </div>\n"
+ " </div>\n"
+ " <link rel='stylesheet' href='c.css' type='text/css'>\n"
+ "</body>\n";
+ std::string html_input =
+ StringPrintf(html_input_format, a_css_name, b_css_name, barrier_text);
+ const char a_css_body[] = ".c1 {\n background-color: blue;\n}\n";
+ const char b_css_body[] = ".c2 {\n color: yellow;\n}\n";
+ const char c_css_body[] = ".c3 {\n font-weight: bold;\n}\n";
+
+ // Put original CSS files into our fetcher.
+ SimpleMetaData default_css_header;
+ resource_manager_->SetDefaultHeaders(&kContentTypeCss, &default_css_header);
+ mock_url_fetcher_.SetResponse(a_css_url, default_css_header, a_css_body);
+ mock_url_fetcher_.SetResponse(b_css_url, default_css_header, b_css_body);
+ mock_url_fetcher_.SetResponse(c_css_url, default_css_header, c_css_body);
+
+ ParseUrl(html_url, html_input);
+
+ std::string headers;
+ AppendDefaultHeaders(kContentTypeCss, resource_manager_, &headers);
+
+ // Check for CSS files in the rewritten page.
+ StringVector css_urls;
+ CollectCssLinks(id, output_buffer_, &css_urls);
+ EXPECT_LE(1UL, css_urls.size());
+ const std::string& combine_url = css_urls[0];
+
+ GURL gurl(combine_url);
+ std::string path = GoogleUrl::PathAndLeaf(gurl);
+
+ std::string combine_filename;
+ filename_encoder_.Encode(file_prefix_, combine_url, &combine_filename);
+
+ // Expected CSS combination.
+ // This syntax must match that in css_combine_filter
+ std::string expected_combination = StrCat(a_css_body, b_css_body);
+ if (!is_barrier) {
+ expected_combination.append(c_css_body);
+ }
+
+ static const char expected_output_format[] =
+ "<head>\n"
+ " <link rel=\"stylesheet\" type=\"text/css\" href=\"%s\">\n"
+ " \n" // The whitespace from the original link is preserved here ...
+ " <title>Hello, Instaweb</title>\n"
+ "%s"
+ "</head>\n"
+ "<body>\n"
+ " <div class=\"c1\">\n"
+ " <div class=\"c2\">\n"
+ " Yellow on Blue\n"
+ " </div>\n"
+ " </div>\n"
+ " %s\n"
+ "</body>\n";
+ std::string expected_output = StringPrintf(
+ expected_output_format, combine_url.c_str(), barrier_text,
+ is_barrier ? "<link rel='stylesheet' href='c.css' type='text/css'>"
+ : "");
+ EXPECT_EQ(AddHtmlBody(expected_output), output_buffer_);
+
+ std::string actual_combination;
+ ASSERT_TRUE(file_system_.ReadFile(combine_filename.c_str(),
+ &actual_combination, &message_handler_));
+ EXPECT_EQ(headers + expected_combination, actual_combination);
+
+ // Fetch the combination to make sure we can serve the result from above.
+ SimpleMetaData request_headers, response_headers;
+ std::string fetched_resource_content;
+ StringWriter writer(&fetched_resource_content);
+ DummyCallback dummy_callback(true);
+ rewrite_driver_.FetchResource(combine_url, request_headers,
+ &response_headers, &writer,
+ &message_handler_, &dummy_callback);
+ EXPECT_EQ(HttpStatus::kOK, response_headers.status_code()) << combine_url;
+ EXPECT_EQ(expected_combination, fetched_resource_content);
+
+ // Now try to fetch from another server (other_rewrite_driver_) that
+ // does not already have the combination cached.
+ // TODO(sligocki): This has too much shared state with the first server.
+ // See RewriteImage for details.
+ SimpleMetaData other_response_headers;
+ fetched_resource_content.clear();
+ message_handler_.Message(kInfo, "Now with serving.");
+ file_system_.Enable();
+ dummy_callback.Reset();
+ other_rewrite_driver_.FetchResource(combine_url, request_headers,
+ &other_response_headers, &writer,
+ &message_handler_, &dummy_callback);
+ EXPECT_EQ(HttpStatus::kOK, other_response_headers.status_code());
+ EXPECT_EQ(expected_combination, fetched_resource_content);
+
+ // Try to fetch from an independent server.
+ ServeResourceFromManyContexts(combine_url, RewriteOptions::kCombineCss,
+ hasher, fetched_resource_content);
+ }
+
+ // Test what happens when CSS combine can't find a previously-rewritten
+ // resource during a subsequent resource fetch. This used to segfault.
+ void CssCombineMissingResource() {
+ std::string a_css_url = StrCat(kDomain, "a.css");
+ std::string c_css_url = StrCat(kDomain, "c.css");
+
+ const char a_css_body[] = ".c1 {\n background-color: blue;\n}\n";
+ const char c_css_body[] = ".c3 {\n font-weight: bold;\n}\n";
+ std::string expected_combination = StrCat(a_css_body, c_css_body);
+
+ AddFilter(RewriteOptions::kCombineCss);
+
+ // Put original CSS files into our fetcher.
+ SimpleMetaData default_css_header;
+ resource_manager_->SetDefaultHeaders(&kContentTypeCss, &default_css_header);
+ mock_url_fetcher_.SetResponse(a_css_url, default_css_header, a_css_body);
+ mock_url_fetcher_.SetResponse(c_css_url, default_css_header, c_css_body);
+
+ // First make sure we can serve the combination of a & c. This is to avoid
+ // spurious test successes.
+
+ std::string kACUrl = Encode(kDomain, "cc", "0", "a.css+c.css", "css");
+ std::string kABCUrl = Encode(kDomain, "cc", "0", "a.css+bbb.css+c.css",
+ "css");
+ SimpleMetaData request_headers, response_headers;
+ std::string fetched_resource_content;
+ StringWriter writer(&fetched_resource_content);
+ DummyCallback dummy_callback(true);
+
+ // NOTE: This first fetch used to return status 0 because response_headers
+ // weren't initialized by the first resource fetch (but were cached
+ // correctly). Content was correct.
+ EXPECT_TRUE(
+ rewrite_driver_.FetchResource(kACUrl, request_headers,
+ &response_headers, &writer,
+ &message_handler_, &dummy_callback));
+ EXPECT_EQ(HttpStatus::kOK, response_headers.status_code());
+ EXPECT_EQ(expected_combination, fetched_resource_content);
+
+ // We repeat the fetch to prove that it succeeds from cache:
+ fetched_resource_content.clear();
+ dummy_callback.Reset();
+ response_headers.Clear();
+ EXPECT_TRUE(
+ rewrite_driver_.FetchResource(kACUrl, request_headers,
+ &response_headers, &writer,
+ &message_handler_, &dummy_callback));
+ EXPECT_EQ(HttpStatus::kOK, response_headers.status_code());
+ EXPECT_EQ(expected_combination, fetched_resource_content);
+
+ // Now let's try fetching the url that references a missing resource
+ // (bbb.css) in addition to the two that do exist, a.css and c.css. Using
+ // an entirely non-existent resource appears to test a strict superset of
+ // filter code paths when compared with returning a 404 for the resource.
+ mock_url_fetcher_.set_fail_on_unexpected(false);
+ DummyCallback fail_callback(false);
+ fetched_resource_content.clear();
+ response_headers.Clear();
+ EXPECT_TRUE(
+ rewrite_driver_.FetchResource(kABCUrl, request_headers,
+ &response_headers, &writer,
+ &message_handler_, &fail_callback));
+ EXPECT_EQ(HttpStatus::kNotFound, response_headers.status_code());
+ EXPECT_EQ("", fetched_resource_content);
+ }
+
+ // Representation for a CSS tag.
+ class CssLink {
+ public:
+ CssLink(const StringPiece& url, const StringPiece& content,
+ const StringPiece& media, bool supply_mock)
+ : url_(url.data(), url.size()),
+ content_(content.data(), content.size()),
+ media_(media.data(), media.size()),
+ supply_mock_(supply_mock) {
+ }
+
+ // A vector of CssLink* should know how to accumulate and add.
+ class Vector : public std::vector<CssLink*> {
+ public:
+ ~Vector() {
+ STLDeleteElements(this);
+ }
+
+ void Add(const StringPiece& url, const StringPiece& content,
+ const StringPiece& media, bool supply_mock) {
+ push_back(new CssLink(url, content, media, supply_mock));
+ }
+ };
+
+ // Parses a combined CSS elementand provides the segments from which
+ // it came.
+ bool DecomposeCombinedUrl(std::string* base, StringVector* segments,
+ MessageHandler* handler) {
+ GURL gurl = GoogleUrl::Create(url_);
+ bool ret = false;
+ if (gurl.is_valid()) {
+ *base = GoogleUrl::AllExceptLeaf(gurl);
+ ResourceNamer namer;
+ if (namer.Decode(GoogleUrl::Leaf(gurl)) &&
+ (namer.id() == "cc")) { // TODO(jmarantz): Share this literal
+ UrlEscaper escaper;
+ UrlMultipartEncoder multipart_encoder;
+ std::string segment;
+ if (escaper.DecodeFromUrlSegment(namer.name(), &segment) &&
+ multipart_encoder.Decode(segment, handler)) {
+ ret = true;
+ for (int i = 0; i < multipart_encoder.num_urls(); ++i) {
+ segments->push_back(multipart_encoder.url(i));
+ }
+ }
+ }
+ }
+ return ret;
+ }
+
+ std::string url_;
+ std::string content_;
+ std::string media_;
+ bool supply_mock_;
+ };
+
+ // Helper class to collect CSS hrefs.
+ class CssCollector : public EmptyHtmlFilter {
+ public:
+ CssCollector(HtmlParse* html_parse, CssLink::Vector* css_links)
+ : css_links_(css_links),
+ css_tag_scanner_(html_parse) {
+ }
+
+ virtual void EndElement(HtmlElement* element) {
+ HtmlElement::Attribute* href;
+ const char* media;
+ if (css_tag_scanner_.ParseCssElement(element, &href, &media)) {
+ // TODO(jmarantz): collect content of the CSS files, before and
+ // after combination, so we can diff.
+ const char* content = "";
+ css_links_->Add(href->value(), content, media, false);
+ }
+ }
+
+ virtual const char* Name() const { return "CssCollector"; }
+
+ private:
+ CssLink::Vector* css_links_;
+ CssTagScanner css_tag_scanner_;
+
+ DISALLOW_COPY_AND_ASSIGN(CssCollector);
+ };
+
+ // Collects just the hrefs from CSS links into a string vector.
+ void CollectCssLinks(const StringPiece& id, const StringPiece& html,
+ StringVector* css_links) {
+ CssLink::Vector v;
+ CollectCssLinks(id, html, &v);
+ for (int i = 0, n = v.size(); i < n; ++i) {
+ css_links->push_back(v[i]->url_);
+ }
+ }
+
+ // Collects all information about CSS links into a CssLink::Vector.
+ void CollectCssLinks(const StringPiece& id, const StringPiece& html,
+ CssLink::Vector* css_links) {
+ HtmlParse html_parse(&message_handler_);
+ CssCollector collector(&html_parse, css_links);
+ html_parse.AddFilter(&collector);
+ std::string dummy_url = StrCat("http://collect.css.links/", id, ".html");
+ html_parse.StartParse(dummy_url);
+ html_parse.ParseText(html.data(), html.size());
+ html_parse.FinishParse();
+ }
+
+ // Common framework for testing barriers. A null-terminated set of css
+ // names is specified, with optional media tags. E.g.
+ // static const char* link[] {
+ // "a.css",
+ // "styles/b.css",
+ // "print.css media=print",
+ // }
+ //
+ // The output of this function is the collected CSS links after rewrite.
+ void BarrierTestHelper(
+ const StringPiece& id,
+ const CssLink::Vector& input_css_links,
+ CssLink::Vector* output_css_links) {
+ // TODO(sligocki): Allow other domains (this is constrained right now b/c
+ // of InitMetaData.
+ std::string html_url = StrCat("http://test.com/", id, ".html");
+ std::string html_input("<head>\n");
+ for (int i = 0, n = input_css_links.size(); i < n; ++i) {
+ const CssLink* link = input_css_links[i];
+ if (!link->url_.empty()) {
+ if (link->supply_mock_) {
+ // If the css-vector contains a 'true' for this, then we supply the
+ // mock fetcher with headers and content for the CSS file.
+ InitMetaData(link->url_, kContentTypeCss, link->content_, 600);
+ }
+ std::string style(" <");
+ style += StringPrintf("link rel='stylesheet' type='text/css' href='%s'",
+ link->url_.c_str());
+ if (!link->media_.empty()) {
+ style += StrCat(" media='", link->media_, "'");
+ }
+ style += ">\n";
+ html_input += style;
+ } else {
+ html_input += link->content_;
+ }
+ }
+ html_input += "</head>\n<body>\n <div class='yellow'>\n";
+ html_input += " Hello, mod_pagespeed!\n </div>\n</body>\n";
+
+ AddFilter(RewriteOptions::kCombineCss);
+ ParseUrl(html_url, html_input);
+ CollectCssLinks("combine_css_missing_files", output_buffer_,
+ output_css_links);
+
+ // TODO(jmarantz): fetch all content and provide output as text.
+ }
+};
+
+TEST_F(CssCombineFilterTest, CombineCss) {
+ CombineCss("combine_css_no_hash", &mock_hasher_, "", false);
+}
+
+TEST_F(CssCombineFilterTest, CombineCssMD5) {
+ CombineCss("combine_css_md5", &md5_hasher_, "", false);
+}
+
+
+// http://code.google.com/p/modpagespeed/issues/detail?q=css&id=39
+TEST_F(CssCombineFilterTest, DealWithParams) {
+ CombineCssWithNames("deal_with_params", &mock_hasher_, "", false,
+ "a.css?U", "b.css?rev=138");
+}
+
+TEST_F(CssCombineFilterTest, CombineCssWithIEDirective) {
+ const char ie_directive_barrier[] =
+ "<!--[if IE]>\n"
+ "<link rel=\"stylesheet\" type=\"text/css\" "
+ "href=\"http://graphics8.nytimes.com/css/"
+ "0.1/screen/build/homepage/ie.css\">\n"
+ "<![endif]-->";
+ CombineCss("combine_css_ie", &md5_hasher_, ie_directive_barrier, true);
+}
+
+TEST_F(CssCombineFilterTest, CombineCssWithStyle) {
+ const char style_barrier[] = "<style>a { color: red }</style>\n";
+ CombineCss("combine_css_style", &md5_hasher_, style_barrier, true);
+}
+
+TEST_F(CssCombineFilterTest, CombineCssWithBogusLink) {
+ const char bogus_barrier[] = "<link rel='stylesheet' type='text/css' "
+ "href='crazee://big/blue/fake'>\n";
+ CombineCss("combine_css_bogus_link", &md5_hasher_, bogus_barrier, true);
+}
+
+TEST_F(CssCombineFilterTest, CombineCssWithImportInFirst) {
+ CssLink::Vector css_in, css_out;
+ css_in.Add("1.css", "@Import \"1a.css\"", "", true);
+ css_in.Add("2.css", kYellow, "", true);
+ css_in.Add("3.css", kYellow, "", true);
+ BarrierTestHelper("combine_css_with_import1", css_in, &css_out);
+ EXPECT_EQ(1, css_out.size());
+}
+
+TEST_F(CssCombineFilterTest, CombineCssWithImportInSecond) {
+ CssLink::Vector css_in, css_out;
+ css_in.Add("1.css", kYellow, "", true);
+ css_in.Add("2.css", "@Import \"2a.css\"", "", true);
+ css_in.Add("3.css", kYellow, "", true);
+ BarrierTestHelper("combine_css_with_import1", css_in, &css_out);
+ EXPECT_EQ("1.css", css_out[0]->url_);
+ EXPECT_EQ(2, css_out.size());
+}
+
+TEST_F(CssCombineFilterTest, CombineCssWithNoscriptBarrier) {
+ const char noscript_barrier[] =
+ "<noscript>\n"
+ " <link rel='stylesheet' type='text/css' href='d.css'>\n"
+ "</noscript>\n";
+
+ // Put this in the Test class to remove repetition here and below.
+ std::string d_css_url = StrCat(kDomain, "d.css");
+ const char d_css_body[] = ".c4 {\n color: green;\n}\n";
+ SimpleMetaData default_css_header;
+ resource_manager_->SetDefaultHeaders(&kContentTypeCss, &default_css_header);
+ mock_url_fetcher_.SetResponse(d_css_url, default_css_header, d_css_body);
+
+ CombineCss("combine_css_noscript", &md5_hasher_, noscript_barrier, true);
+}
+
+TEST_F(CssCombineFilterTest, CombineCssWithFakeNoscriptBarrier) {
+ const char non_barrier[] =
+ "<noscript>\n"
+ " <p>You have no scripts installed</p>\n"
+ "</noscript>\n";
+ CombineCss("combine_css_fake_noscript", &md5_hasher_, non_barrier, false);
+}
+
+TEST_F(CssCombineFilterTest, CombineCssWithMediaBarrier) {
+ const char media_barrier[] =
+ "<link rel='stylesheet' type='text/css' href='d.css' media='print'>\n";
+
+ std::string d_css_url = StrCat(kDomain, "d.css");
+ const char d_css_body[] = ".c4 {\n color: green;\n}\n";
+ SimpleMetaData default_css_header;
+ resource_manager_->SetDefaultHeaders(&kContentTypeCss, &default_css_header);
+ mock_url_fetcher_.SetResponse(d_css_url, default_css_header, d_css_body);
+
+ CombineCss("combine_css_media", &md5_hasher_, media_barrier, true);
+}
+
+TEST_F(CssCombineFilterTest, CombineCssWithNonMediaBarrier) {
+ // Put original CSS files into our fetcher.
+ std::string html_url = StrCat(kDomain, "no_media_barrier.html");
+ std::string a_css_url = StrCat(kDomain, "a.css");
+ std::string b_css_url = StrCat(kDomain, "b.css");
+ std::string c_css_url = StrCat(kDomain, "c.css");
+ std::string d_css_url = StrCat(kDomain, "d.css");
+
+ const char a_css_body[] = ".c1 {\n background-color: blue;\n}\n";
+ const char b_css_body[] = ".c2 {\n color: yellow;\n}\n";
+ const char c_css_body[] = ".c3 {\n font-weight: bold;\n}\n";
+ const char d_css_body[] = ".c4 {\n color: green;\n}\n";
+
+ SimpleMetaData default_css_header;
+ resource_manager_->SetDefaultHeaders(&kContentTypeCss, &default_css_header);
+ mock_url_fetcher_.SetResponse(a_css_url, default_css_header, a_css_body);
+ mock_url_fetcher_.SetResponse(b_css_url, default_css_header, b_css_body);
+ mock_url_fetcher_.SetResponse(c_css_url, default_css_header, c_css_body);
+ mock_url_fetcher_.SetResponse(d_css_url, default_css_header, d_css_body);
+
+ // Only the first two CSS files should be combined.
+ const char html_input[] =
+ "<head>\n"
+ " <link rel='stylesheet' type='text/css' href='a.css' media='print'>\n"
+ " <link rel='stylesheet' type='text/css' href='b.css' media='print'>\n"
+ " <link rel='stylesheet' type='text/css' href='c.css'>\n"
+ " <link rel='stylesheet' type='text/css' href='d.css' media='print'>\n"
+ "</head>";
+
+ // Rewrite
+ AddFilter(RewriteOptions::kCombineCss);
+ ParseUrl(html_url, html_input);
+
+ // Check for CSS files in the rewritten page.
+ StringVector css_urls;
+ CollectCssLinks("combine_css_no_media-links", output_buffer_, &css_urls);
+ EXPECT_EQ(3UL, css_urls.size());
+ const std::string& combine_url = css_urls[0];
+
+ const char expected_output_format[] =
+ "<head>\n"
+ " <link rel=\"stylesheet\" type=\"text/css\" media=\"print\" "
+ "href=\"%s\">\n"
+ " \n"
+ " <link rel='stylesheet' type='text/css' href='c.css'>\n"
+ " <link rel='stylesheet' type='text/css' href='d.css' media='print'>\n"
+ "</head>";
+ std::string expected_output = StringPrintf(expected_output_format,
+ combine_url.c_str());
+
+ EXPECT_EQ(AddHtmlBody(expected_output), output_buffer_);
+}
+
+TEST_F(CssCombineFilterTest, CombineCssBaseUrl) {
+ // Put original CSS files into our fetcher.
+ std::string html_url = StrCat(kDomain, "base_url.html");
+ std::string a_css_url = StrCat(kDomain, "a.css");
+ const char b_css_url[] = "http://other_domain.test/foo/b.css";
+
+ const char a_css_body[] = ".c1 {\n background-color: blue;\n}\n";
+ const char b_css_body[] = ".c2 {\n color: yellow;\n}\n";
+
+ SimpleMetaData default_css_header;
+ resource_manager_->SetDefaultHeaders(&kContentTypeCss, &default_css_header);
+ mock_url_fetcher_.SetResponse(a_css_url, default_css_header, a_css_body);
+ mock_url_fetcher_.SetResponse(b_css_url, default_css_header, b_css_body);
+
+ // Second stylesheet is on other domain.
+ const char html_input[] =
+ "<head>\n"
+ " <link rel='stylesheet' type='text/css' href='a.css'>\n"
+ " <base href='http://other_domain.test/foo/'>\n"
+ " <link rel='stylesheet' type='text/css' href='b.css'>\n"
+ "</head>\n";
+
+ // Rewrite
+ AddFilter(RewriteOptions::kCombineCss);
+ ParseUrl(html_url, html_input);
+
+ // Check for CSS files in the rewritten page.
+ StringVector css_urls;
+ CollectCssLinks("combine_css_no_media-links", output_buffer_, &css_urls);
+ EXPECT_EQ(1UL, css_urls.size());
+ const std::string& combine_url = css_urls[0];
+
+ const char expected_output_format[] =
+ "<head>\n"
+ " <link rel=\"stylesheet\" type=\"text/css\" href=\"%s\">\n"
+ " <base href='http://other_domain.test/foo/'>\n"
+ " \n"
+ "</head>\n";
+ std::string expected_output = StringPrintf(expected_output_format,
+ combine_url.c_str());
+
+ EXPECT_EQ(AddHtmlBody(expected_output), output_buffer_);
+
+ EXPECT_TRUE(GURL(combine_url.c_str()).is_valid());
+}
+
+// TODO(jmaessen): Re-write for new sharding story when it exists.
+// TEST_F(CssCombineFilterTest, CombineCssShards) {
+// num_shards_ = 10;
+// url_prefix_ = "http://mysite%d/";
+// CombineCss("combine_css_sha1", &mock_hasher_, "", false);
+// }
+
+TEST_F(CssCombineFilterTest, CombineCssNoInput) {
+ // TODO(sligocki): This is probably not working correctly, we need to put
+ // a_broken.css and b.css in mock_fetcher_ ... not sure why this isn't
+ // crashing right now.
+ static const char html_input[] =
+ "<head>\n"
+ " <link rel='stylesheet' href='a_broken.css' type='text/css'>\n"
+ " <link rel='stylesheet' href='b.css' type='text/css'>\n"
+ "</head>\n"
+ "<body><div class=\"c1\"><div class=\"c2\"><p>\n"
+ " Yellow on Blue</p></div></div></body>";
+ ValidateNoChanges("combine_css_missing_input", html_input);
+}
+
+TEST_F(CssCombineFilterTest, CombineCssMissingResource) {
+ CssCombineMissingResource();
+}
+
+TEST_F(CssCombineFilterTest, CombineCssManyFiles) {
+ // Prepare an HTML fragment with too many CSS files to combine,
+ // exceeding the 250 char limit currently the default in rewrite_options.cc.
+ //
+ // It looks like we can fit a limited number of encodings of
+ // "yellow%d.css" in the buffer. It might be more general to base
+ // this on the constant declared in RewriteOptions but I think it's
+ // easier to understand leaving these exposed as constants; we can
+ // abstract them later.
+ const int kNumCssLinks = 31;
+ const int kNumCssInCombination = 18; // based on how we encode "yellow%d.css"
+ CssLink::Vector css_in, css_out;
+ for (int i = 0; i < kNumCssLinks; ++i) {
+ css_in.Add(StringPrintf("styles/yellow%d.css", i),
+ kYellow, "", true);
+ }
+ BarrierTestHelper("combine_css_many_files", css_in, &css_out);
+ ASSERT_EQ(2, css_out.size());
+
+ // Check that the first element is really a combination.
+ std::string base;
+ StringVector segments;
+ ASSERT_TRUE(css_out[0]->DecomposeCombinedUrl(&base, &segments,
+ &message_handler_));
+ EXPECT_EQ("http://test.com/styles/", base);
+ EXPECT_EQ(kNumCssInCombination, segments.size());
+
+ segments.clear();
+ ASSERT_TRUE(css_out[1]->DecomposeCombinedUrl(&base, &segments,
+ &message_handler_));
+ EXPECT_EQ("http://test.com/styles/", base);
+ EXPECT_EQ(kNumCssLinks - kNumCssInCombination, segments.size());
+}
+
+TEST_F(CssCombineFilterTest, CombineCssManyFilesOneOrphan) {
+ // This test differs from the previous test in we have exactly one CSS file
+ // that stays on its own.
+ const int kNumCssInCombination = 18; // based on how we encode "yellow%d.css"
+ const int kNumCssLinks = kNumCssInCombination + 1;
+ CssLink::Vector css_in, css_out;
+ for (int i = 0; i < kNumCssLinks - 1; ++i) {
+ css_in.Add(StringPrintf("styles/yellow%d.css", i),
+ kYellow, "", true);
+ }
+ css_in.Add("styles/last_one.css",
+ kYellow, "", true);
+ BarrierTestHelper("combine_css_many_files", css_in, &css_out);
+ ASSERT_EQ(2, css_out.size());
+
+ // Check that the first element is really a combination.
+ std::string base;
+ StringVector segments;
+ ASSERT_TRUE(css_out[0]->DecomposeCombinedUrl(&base, &segments,
+ &message_handler_));
+ EXPECT_EQ("http://test.com/styles/", base);
+ EXPECT_EQ(kNumCssInCombination, segments.size());
+ EXPECT_EQ("styles/last_one.css", css_out[1]->url_);
+}
+
+// Note -- this test is redundant with CombineCssMissingResource -- this
+// is a taste test. This new mechanism is more code per test but I think
+// the failures are more obvious and the expect/assert tests are in the
+// top level of the test which might make it easier to debug.
+TEST_F(CssCombineFilterTest, CombineCssNotCached) {
+ CssLink::Vector css_in, css_out;
+ css_in.Add("1.css", kYellow, "", true);
+ css_in.Add("2.css", kYellow, "", true);
+ css_in.Add("3.css", kYellow, "", false);
+ css_in.Add("4.css", kYellow, "", true);
+ mock_url_fetcher_.set_fail_on_unexpected(false);
+ BarrierTestHelper("combine_css_not_cached", css_in, &css_out);
+ EXPECT_EQ(3, css_out.size());
+ std::string base;
+ StringVector segments;
+ ASSERT_TRUE(css_out[0]->DecomposeCombinedUrl(&base, &segments,
+ &message_handler_));
+ EXPECT_EQ(2, segments.size());
+ EXPECT_EQ("1.css", segments[0]);
+ EXPECT_EQ("2.css", segments[1]);
+ EXPECT_EQ("3.css", css_out[1]->url_);
+ EXPECT_EQ("4.css", css_out[2]->url_);
+}
+
+// Note -- this test is redundant with CombineCssWithIEDirective -- this
+// is a taste test.
+TEST_F(CssCombineFilterTest, CombineStyleTag) {
+ CssLink::Vector css_in, css_out;
+ css_in.Add("1.css", kYellow, "", true);
+ css_in.Add("2.css", kYellow, "", true);
+ css_in.Add("", "<style>a { color: red }</style>\n", "", false);
+ css_in.Add("4.css", kYellow, "", true);
+ BarrierTestHelper("combine_css_with_style", css_in, &css_out);
+ EXPECT_EQ(2, css_out.size());
+ std::string base;
+ StringVector segments;
+ ASSERT_TRUE(css_out[0]->DecomposeCombinedUrl(&base, &segments,
+ &message_handler_));
+ EXPECT_EQ(2, segments.size());
+ EXPECT_EQ("1.css", segments[0]);
+ EXPECT_EQ("2.css", segments[1]);
+ EXPECT_EQ("4.css", css_out[1]->url_);
+}
+
+TEST_F(CssCombineFilterTest, NoAbsolutifySameDir) {
+ CssLink::Vector css_in, css_out;
+ css_in.Add("1.css", ".yellow {background-image: url('1.png');}\n", "", true);
+ css_in.Add("2.css", ".yellow {background-image: url('2.png');}\n", "", true);
+ BarrierTestHelper("combine_css_with_style", css_in, &css_out);
+ EXPECT_EQ(1, css_out.size());
+
+ // Note: the urls are not absolutified.
+ std::string expected_combination =
+ ".yellow {background-image: url('1.png');}\n"
+ ".yellow {background-image: url('2.png');}\n";
+
+ // Check fetched resource.
+ std::string actual_combination;
+ EXPECT_TRUE(ServeResourceUrl(css_out[0]->url_, &actual_combination));
+ // TODO(sligocki): Check headers?
+ EXPECT_EQ(expected_combination, actual_combination);
+}
+
+TEST_F(CssCombineFilterTest, DoAbsolutifyDifferentDir) {
+ CssLink::Vector css_in, css_out;
+ css_in.Add("1.css", ".yellow {background-image: url('1.png');}\n", "", true);
+ css_in.Add("foo/2.css", ".yellow {background-image: url('2.png');}\n",
+ "", true);
+ BarrierTestHelper("combine_css_with_style", css_in, &css_out);
+ EXPECT_EQ(1, css_out.size());
+
+ std::string expected_combination =
+ ".yellow {background-image: url('1.png');}\n"
+ ".yellow {background-image: url('http://test.com/foo/2.png');}\n";
+
+ // Check fetched resource.
+ std::string actual_combination;
+ EXPECT_TRUE(ServeResourceUrl(css_out[0]->url_, &actual_combination));
+ // TODO(sligocki): Check headers?
+ EXPECT_EQ(expected_combination, actual_combination);
+}
+
+// Verifies that when we combine across paths in a certain pattern we get
+// the correct results.
+TEST_F(CssCombineFilterTest, CrossAcrossPathsExceedingUrlSize) {
+ CssLink::Vector css_in, css_out;
+ css_in.Add("modules/acquia/fivestar/css/fivestar.css?3", "a", "", true);
+ css_in.Add("modules/node/node.css?3", "b", "", true);
+ css_in.Add("modules/poll/poll.css?3", "c", "", true);
+ css_in.Add("modules/system/defaults.css?3", "d", "", true);
+ css_in.Add("modules/system/system.css?3", "e", "", true);
+ css_in.Add("modules/system/system-menus.css?3", "f", "", true);
+ css_in.Add("modules/user/user.css?3", "g", "", true);
+
+ // This last 'Add' causes the resolved path to change from "/modules/" to "/".
+ css_in.Add("sites/all/modules/ckeditor/ckeditor.css?3", "h", "", true);
+ BarrierTestHelper("cross_paths", css_in, &css_out);
+ EXPECT_EQ(2, css_out.size());
+ std::string actual_combination;
+ EXPECT_TRUE(ServeResourceUrl(css_out[0]->url_, &actual_combination));
+ GURL gurl = GoogleUrl::Create(css_out[0]->url_);
+ EXPECT_EQ("/modules/", GoogleUrl::PathSansLeaf(gurl));
+ ResourceNamer namer;
+ ASSERT_TRUE(namer.Decode(GoogleUrl::Leaf(gurl)));
+ EXPECT_EQ("acquia,_fivestar,_css,_fivestar.css,q3+"
+ "node,_node.css,q3+"
+ "poll,_poll.css,q3+"
+ "system,_defaults.css,q3+"
+ "system,_system.css,q3+"
+ "system,_system-menus.css,q3+"
+ "user,_user.css,q3",
+ namer.name());
+ EXPECT_EQ("abcdefg", actual_combination);
+}
+
+TEST_F(CssCombineFilterTest, CrossMappedDomain) {
+ CssLink::Vector css_in, css_out;
+ DomainLawyer* laywer = options_.domain_lawyer();
+ laywer->AddRewriteDomainMapping("a.com", "b.com", &message_handler_);
+ bool supply_mock = false;
+ css_in.Add("http://a.com/1.css", kYellow, "", supply_mock);
+ css_in.Add("http://b.com/2.css", kBlue, "", supply_mock);
+ SimpleMetaData default_css_header;
+ resource_manager_->SetDefaultHeaders(&kContentTypeCss, &default_css_header);
+ mock_url_fetcher_.SetResponse("http://a.com/1.css", default_css_header,
+ kYellow);
+ mock_url_fetcher_.SetResponse("http://a.com/2.css", default_css_header,
+ kBlue);
+ BarrierTestHelper("combine_css_with_style", css_in, &css_out);
+ EXPECT_EQ(1, css_out.size());
+ std::string actual_combination;
+ EXPECT_TRUE(ServeResourceUrl(css_out[0]->url_, &actual_combination));
+ EXPECT_EQ(StrCat(kYellow, kBlue), actual_combination);
+}
+
+// Verifies that we cannot do the same cross-domain combo when we lack
+// the domain mapping.
+TEST_F(CssCombineFilterTest, CrossUnmappedDomain) {
+ CssLink::Vector css_in, css_out;
+ DomainLawyer* laywer = options_.domain_lawyer();
+ laywer->AddDomain("a.com", &message_handler_);
+ laywer->AddDomain("b.com", &message_handler_);
+ bool supply_mock = false;
+ const char kUrl1[] = "http://a.com/1.css";
+ const char kUrl2[] = "http://b.com/2.css";
+ css_in.Add(kUrl1, kYellow, "", supply_mock);
+ css_in.Add(kUrl2, kBlue, "", supply_mock);
+ SimpleMetaData default_css_header;
+ resource_manager_->SetDefaultHeaders(&kContentTypeCss, &default_css_header);
+ mock_url_fetcher_.SetResponse(kUrl1, default_css_header, kYellow);
+ mock_url_fetcher_.SetResponse(kUrl2, default_css_header, kBlue);
+ BarrierTestHelper("combine_css_with_style", css_in, &css_out);
+ EXPECT_EQ(2, css_out.size());
+ std::string actual_combination;
+ EXPECT_EQ(kUrl1, css_out[0]->url_);
+ EXPECT_EQ(kUrl2, css_out[1]->url_);
+}
+
+/*
+ TODO(jmarantz): cover intervening FLUSH
+ TODO(jmarantz): consider converting some of the existing tests to this
+ format, covering
+ IE Directive
+ @Import in any css element except the first
+ link in noscript tag
+ change in 'media'
+ incompatible domain
+ intervening inline style tag (TODO: outline first?)
+*/
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/css_filter.cc b/trunk/src/net/instaweb/rewriter/css_filter.cc
new file mode 100644
index 0000000..17980c3
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/css_filter.cc
@@ -0,0 +1,352 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/rewriter/public/css_filter.h"
+
+#include "base/at_exit.h"
+
+#include "base/scoped_ptr.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/rewriter/public/css_minify.h"
+#include "net/instaweb/rewriter/public/output_resource.h"
+#include "net/instaweb/rewriter/public/resource.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/statistics.h"
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/url_escaper.h"
+#include "net/instaweb/util/public/writer.h"
+#include "webutil/css/parser.h"
+
+namespace {
+
+base::AtExitManager* at_exit_manager = NULL;
+
+} // namespace
+
+namespace net_instaweb {
+
+namespace {
+
+const char kStylesheet[] = "stylesheet";
+
+} // namespace
+
+// Statistics variable names.
+const char CssFilter::kFilesMinified[] = "css_filter_files_minified";
+const char CssFilter::kMinifiedBytesSaved[] = "css_filter_minified_bytes_saved";
+const char CssFilter::kParseFailures[] = "css_filter_parse_failures";
+
+CssFilter::CssFilter(RewriteDriver* driver, const StringPiece& path_prefix)
+ : RewriteSingleResourceFilter(driver, path_prefix),
+ html_parse_(driver->html_parse()),
+ resource_manager_(driver->resource_manager()),
+ in_style_element_(false),
+ s_style_(html_parse_->Intern("style")),
+ s_link_(html_parse_->Intern("link")),
+ s_rel_(html_parse_->Intern("rel")),
+ s_href_(html_parse_->Intern("href")),
+ num_files_minified_(NULL),
+ minified_bytes_saved_(NULL),
+ num_parse_failures_(NULL) {
+ Statistics* stats = resource_manager_->statistics();
+ if (stats != NULL) {
+ num_files_minified_ = stats->GetVariable(CssFilter::kFilesMinified);
+ minified_bytes_saved_ = stats->GetVariable(CssFilter::kMinifiedBytesSaved);
+ num_parse_failures_ = stats->GetVariable(CssFilter::kParseFailures);
+ }
+}
+
+void CssFilter::Initialize(Statistics* statistics) {
+ if (statistics != NULL) {
+ statistics->AddVariable(CssFilter::kFilesMinified);
+ statistics->AddVariable(CssFilter::kMinifiedBytesSaved);
+ statistics->AddVariable(CssFilter::kParseFailures);
+ }
+
+ InitializeAtExitManager();
+}
+
+void CssFilter::Terminate() {
+ // Note: This is not thread-safe, but I don't believe we need it to be.
+ if (at_exit_manager != NULL) {
+ delete at_exit_manager;
+ at_exit_manager = NULL;
+ }
+}
+
+void CssFilter::InitializeAtExitManager() {
+ // Note: This is not thread-safe, but I don't believe we need it to be.
+ if (at_exit_manager == NULL) {
+ at_exit_manager = new base::AtExitManager;
+ }
+}
+
+void CssFilter::StartDocumentImpl() {
+ in_style_element_ = false;
+}
+
+void CssFilter::StartElementImpl(HtmlElement* element) {
+ // HtmlParse should not pass us elements inside a style element.
+ CHECK(!in_style_element_);
+ if (element->tag() == s_style_) {
+ in_style_element_ = true;
+ style_element_ = element;
+ style_char_node_ = NULL;
+ }
+ // We deal with <link> elements in EndElement.
+}
+
+void CssFilter::Characters(HtmlCharactersNode* characters_node) {
+ if (in_style_element_) {
+ if (style_char_node_ == NULL) {
+ style_char_node_ = characters_node;
+ } else {
+ html_parse_->ErrorHere("Multiple character nodes in style.");
+ in_style_element_ = false;
+ }
+ }
+}
+
+void CssFilter::EndElementImpl(HtmlElement* element) {
+ // Rewrite an inline style.
+ if (in_style_element_) {
+ CHECK(style_element_ == element); // HtmlParse should not pass unmatching.
+
+ if (html_parse_->IsRewritable(element) && style_char_node_ != NULL) {
+ CHECK(element == style_char_node_->parent()); // Sanity check.
+ std::string new_content;
+ if (RewriteCssText(style_char_node_->contents(), &new_content,
+ StrCat("inline CSS in ", html_parse_->url()),
+ html_parse_->message_handler())) {
+ // Note: Copy of new_content here.
+ HtmlCharactersNode* new_style_char_node =
+ html_parse_->NewCharactersNode(element, new_content);
+ html_parse_->ReplaceNode(style_char_node_, new_style_char_node);
+ }
+ }
+ in_style_element_ = false;
+
+ // Rewrite an external style.
+ } else if (element->tag() == s_link_ && html_parse_->IsRewritable(element)) {
+ StringPiece relation(element->AttributeValue(s_rel_));
+ if (relation == kStylesheet) {
+ HtmlElement::Attribute* element_href = element->FindAttribute(s_href_);
+ if (element_href != NULL) {
+ // If it has a href= attribute
+ std::string new_url;
+ if (RewriteExternalCss(element_href->value(), &new_url)) {
+ element_href->SetValue(new_url); // Update the href= attribute.
+ }
+ } else {
+ html_parse_->ErrorHere("Link element with no href.");
+ }
+ }
+ }
+}
+
+// Return value answers the question: May we rewrite?
+// If return false, out_text is undefined.
+// id should be the URL for external CSS and other identifying info
+// for inline CSS. It is used to log where the CSS parsing error was.
+bool CssFilter::RewriteCssText(const StringPiece& in_text,
+ std::string* out_text,
+ const std::string& id,
+ MessageHandler* handler) {
+ // Load stylesheet w/o expanding background attributes.
+ Css::Parser parser(in_text);
+ scoped_ptr<Css::Stylesheet> stylesheet(parser.ParseRawStylesheet());
+
+ bool ret = false;
+ if (parser.errors_seen_mask() != Css::Parser::kNoError) {
+ html_parse_->InfoHere("CSS parsing error in %s", id.c_str());
+ if (num_parse_failures_ != NULL) {
+ num_parse_failures_->Add(1);
+ }
+ } else {
+ // TODO(sligocki): Edit stylesheet.
+
+ // Re-serialize stylesheet.
+ StringWriter writer(out_text);
+ CssMinify::Stylesheet(*stylesheet, &writer, handler);
+
+ // Get signed versions so that we can subtract them.
+ int64 out_text_size = static_cast<int64>(out_text->size());
+ int64 in_text_size = static_cast<int64>(in_text.size());
+
+ // Don't rewrite if we don't make it smaller.
+ ret = (out_text_size < in_text_size);
+
+ // Don't rewrite if we blanked the CSS file! (This is a parse error)
+ // TODO(sligocki): Don't error if in_text is all whitespace.
+ if (out_text_size == 0 && in_text_size != 0) {
+ ret = false;
+ html_parse_->InfoHere("CSS parsing error in %s", id.c_str());
+ if (num_parse_failures_ != NULL) {
+ num_parse_failures_->Add(1);
+ }
+ }
+
+ // Statistics
+ if (ret && num_files_minified_ != NULL) {
+ num_files_minified_->Add(1);
+ minified_bytes_saved_->Add(in_text_size - out_text_size);
+ }
+ // TODO(sligocki): Do we want to save the AST 'stylesheet' somewhere?
+ // It currently, deletes itself at the end of the function.
+ }
+
+ return ret;
+}
+
+
+// Combine all 'original_stylesheets' (and all their sub stylescripts) into a
+// single returned stylesheet which has no @imports or returns NULL if we fail
+// to load some sub-resources.
+//
+// Note: we must cannibalize input stylesheets or we will have ownership
+// problems or a lot of deep-copying.
+Css::Stylesheet* CssFilter::CombineStylesheets(
+ std::vector<Css::Stylesheet*>* original_stylesheets) {
+ // Load all sub-stylesheets to assure that we can do the combination.
+ std::vector<Css::Stylesheet*> stylesheets;
+ std::vector<Css::Stylesheet*>::const_iterator iter;
+ for (iter = original_stylesheets->begin();
+ iter < original_stylesheets->end(); ++iter) {
+ Css::Stylesheet* stylesheet = *iter;
+ if (!LoadAllSubStylesheets(stylesheet, &stylesheets)) {
+ return NULL;
+ }
+ }
+
+ // Once all sub-stylesheets are loaded in memory, combine them.
+ Css::Stylesheet* combination = new Css::Stylesheet;
+ // TODO(sligocki): combination->rulesets().reserve(...);
+ for (std::vector<Css::Stylesheet*>::const_iterator iter = stylesheets.begin();
+ iter < stylesheets.end(); ++iter) {
+ Css::Stylesheet* stylesheet = *iter;
+ // Append all rulesets from 'stylesheet' to 'combination' ...
+ combination->mutable_rulesets().insert(
+ combination->mutable_rulesets().end(),
+ stylesheet->rulesets().begin(),
+ stylesheet->rulesets().end());
+ // ... and then clear rules from 'stylesheet' to avoid double ownership.
+ stylesheet->mutable_rulesets().clear();
+ }
+ return combination;
+}
+
+// Collect a list of all stylesheets @imported by base_stylesheet directly or
+// indirectly in the order that they will be dealt with by a CSS parser and
+// append them to vector 'all_stylesheets'.
+bool CssFilter::LoadAllSubStylesheets(
+ Css::Stylesheet* base_stylesheet,
+ std::vector<Css::Stylesheet*>* all_stylesheets) {
+ const Css::Imports& imports = base_stylesheet->imports();
+ for (Css::Imports::const_iterator iter = imports.begin();
+ iter < imports.end(); ++iter) {
+ Css::Import* import = *iter;
+ StringPiece url(import->link.utf8_data(), import->link.utf8_length());
+
+ // Fetch external stylesheet from url ...
+ Css::Stylesheet* sub_stylesheet = LoadStylesheet(url);
+ if (sub_stylesheet == NULL) {
+ html_parse_->ErrorHere("Failed to load sub-resource %s",
+ url.as_string().c_str());
+ return false;
+ }
+
+ // ... and recursively add all its sub-stylesheets (and it) to vector.
+ if (!LoadAllSubStylesheets(sub_stylesheet, all_stylesheets)) {
+ return false;
+ }
+ }
+ // Add base stylesheet after all imports have been added.
+ all_stylesheets->push_back(base_stylesheet);
+ return true;
+}
+
+
+// Read an external CSS file, rewrite it and write a new external CSS file.
+bool CssFilter::RewriteExternalCss(const StringPiece& in_url,
+ std::string* out_url) {
+ bool ret = false;
+ scoped_ptr<Resource> input_resource(CreateInputResource(in_url));
+ scoped_ptr<OutputResource> output_resource(
+ resource_manager_->CreateOutputResourceFromResource(
+ filter_prefix_, &kContentTypeCss, resource_manager_->url_escaper(),
+ input_resource.get(), html_parse_->message_handler()));
+ if (output_resource.get() != NULL &&
+ RewriteExternalCssToResource(input_resource.get(),
+ output_resource.get())) {
+ ret = true;
+ *out_url = output_resource->url();
+ }
+ return ret;
+}
+
+// Rewrite in input_resource once it has already been loaded.
+bool CssFilter::RewriteExternalCssToResource(Resource* input_resource,
+ OutputResource* output_resource) {
+ bool ret = false;
+ // If this OutputResource has not already been created, create it.
+ if (!output_resource->IsWritten()) {
+ // Load input stylesheet.
+ MessageHandler* handler = html_parse_->message_handler();
+ if (input_resource != NULL &&
+ resource_manager_->ReadIfCached(input_resource, handler)) {
+ if (input_resource->ContentsValid()) {
+ ret = RewriteLoadedResource(input_resource, output_resource);
+ } else {
+ // TODO(sligocki): Should these really be HtmlParse warnings?
+ html_parse_->WarningHere("CSS resource fetch failed: %s",
+ input_resource->url().c_str());
+ }
+ }
+ }
+
+ return ret;
+}
+
+bool CssFilter::RewriteLoadedResource(const Resource* input_resource,
+ OutputResource* output_resource) {
+ CHECK(input_resource->loaded());
+ if (input_resource->ContentsValid()) {
+ // Rewrite stylesheet.
+ StringPiece in_contents = input_resource->contents();
+ std::string out_contents;
+ if (!RewriteCssText(in_contents, &out_contents, input_resource->url(),
+ html_parse_->message_handler())) {
+ return false;
+ }
+
+ // Write new stylesheet.
+ // TODO(sligocki): Set expire time.
+ if (!resource_manager_->Write(HttpStatus::kOK, out_contents,
+ output_resource, -1,
+ html_parse_->message_handler())) {
+ return false;
+ }
+ }
+
+ return output_resource->IsWritten();
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/css_filter_test.cc b/trunk/src/net/instaweb/rewriter/css_filter_test.cc
new file mode 100644
index 0000000..fa24a1a
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/css_filter_test.cc
@@ -0,0 +1,405 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+// and sligocki@google.com (Shawn Ligocki)
+
+// Unit-test the html rewriter
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/css_filter.h"
+#include "net/instaweb/rewriter/public/css_move_to_head_filter.h"
+#include "net/instaweb/rewriter/public/css_tag_scanner.h"
+#include "net/instaweb/rewriter/public/img_tag_scanner.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+#include "net/instaweb/rewriter/public/rewrite_options.h"
+#include "net/instaweb/util/public/simple_stats.h"
+#include <string>
+
+namespace net_instaweb {
+
+class CssFilterTest : public ResourceManagerTestBase {
+ protected:
+ CssFilterTest() {
+ RewriteDriver::Initialize(&statistics_);
+
+ num_files_minified_ = statistics_.GetVariable(CssFilter::kFilesMinified);
+ minified_bytes_saved_ =
+ statistics_.GetVariable(CssFilter::kMinifiedBytesSaved);
+ num_parse_failures_ = statistics_.GetVariable(CssFilter::kParseFailures);
+ }
+
+ virtual void SetUp() {
+ ResourceManagerTestBase::SetUp();
+ resource_manager_->set_statistics(&statistics_);
+ AddFilter(RewriteOptions::kRewriteCss);
+ }
+
+ // Check that inline CSS get's rewritten correctly.
+ void ValidateRewriteInlineCss(const StringPiece& id,
+ const StringPiece& css_input,
+ const StringPiece& expected_css_output,
+ bool expect_change,
+ bool expect_failure) {
+ static const char prefix[] =
+ "<head>\n"
+ " <title>Example style outline</title>\n"
+ " <!-- Style starts here -->\n"
+ " <style type='text/css'>";
+ static const char suffix[] = "</style>\n"
+ " <!-- Style ends here -->\n"
+ "</head>";
+
+ std::string html_input = StrCat(prefix, css_input, suffix);
+ std::string html_output = StrCat(prefix, expected_css_output, suffix);
+
+ // Reset stats
+ num_files_minified_->Set(0);
+ minified_bytes_saved_->Set(0);
+ num_parse_failures_->Set(0);
+
+ // Rewrite
+ ValidateExpected(id, html_input, html_output);
+
+ // Check stats
+ if (expect_change) {
+ EXPECT_EQ(1, num_files_minified_->Get());
+ EXPECT_EQ(css_input.size() - expected_css_output.size(),
+ minified_bytes_saved_->Get());
+ EXPECT_EQ(0, num_parse_failures_->Get());
+ } else {
+ EXPECT_EQ(0, num_files_minified_->Get());
+ EXPECT_EQ(0, minified_bytes_saved_->Get());
+ EXPECT_EQ((expect_failure ? 1 : 0), num_parse_failures_->Get()) << id;
+ }
+ }
+
+ // Check that external CSS gets rewritten correctly.
+ void ValidateRewriteExternalCss(const StringPiece& id,
+ const std::string& css_input,
+ const std::string& expected_css_output,
+ bool expect_change,
+ bool expect_failure) {
+ // TODO(sligocki): Allow arbitrary URLs.
+ std::string css_url = StrCat("http://test.com/", id, ".css");
+
+ Hasher* hasher = &mock_hasher_;
+ // TODO(sligocki): Allow testing with other hashers.
+ //resource_manager_->SetHasher(hasher);
+
+ // Set input file.
+ mock_url_fetcher_.Clear();
+ InitMetaData(StrCat(id, ".css"), kContentTypeCss, css_input, 300);
+
+ static const char html_template[] =
+ "<head>\n"
+ " <title>Example style outline</title>\n"
+ " <!-- Style starts here -->\n"
+ " <link rel='stylesheet' type='text/css' href='%s'>\n"
+ " <!-- Style ends here -->\n"
+ "</head>";
+
+ std::string html_input = StringPrintf(html_template, css_url.c_str());
+
+ std::string html_output;
+
+ ResourceNamer namer;
+ namer.set_id("cf");
+ namer.set_hash(hasher->Hash(expected_css_output));
+ namer.set_ext("css");
+ // TODO(sligocki): Derive these from css_url the "right" way.
+ std::string url_prefix = "http://test.com/";
+ namer.set_name(StrCat(id, ".css"));
+
+ std::string expected_new_url = StrCat(url_prefix, namer.Encode());
+
+ if (expect_change) {
+ html_output = StringPrintf(html_template, expected_new_url.c_str());
+ } else {
+ html_output = html_input;
+ }
+
+ // Reset stats
+ num_files_minified_->Set(0);
+ minified_bytes_saved_->Set(0);
+ num_parse_failures_->Set(0);
+
+ // Rewrite
+ ValidateExpected(id, html_input, html_output);
+
+ // Check stats
+ if (expect_change) {
+ EXPECT_EQ(1, num_files_minified_->Get());
+ EXPECT_EQ(css_input.size() - expected_css_output.size(),
+ minified_bytes_saved_->Get());
+ EXPECT_EQ(0, num_parse_failures_->Get());
+ } else {
+ EXPECT_EQ(0, num_files_minified_->Get());
+ EXPECT_EQ(0, minified_bytes_saved_->Get());
+ EXPECT_EQ((expect_failure ? 1 : 0), num_parse_failures_->Get()) << id;
+ }
+
+ // Check CSS output.
+ if (expect_change) {
+ std::string actual_output;
+ // TODO(sligocki): This will only work with mock_hasher.
+ EXPECT_TRUE(ServeResource(url_prefix,
+ namer.id(), namer.name(), namer.ext(),
+ &actual_output));
+ EXPECT_EQ(expected_css_output, actual_output);
+
+ // Serve from new context.
+ ServeResourceFromManyContexts(expected_new_url,
+ RewriteOptions::kRewriteCss,
+ hasher, expected_css_output);
+ }
+ }
+
+ void ValidateRewrite(const StringPiece& id,
+ const std::string& css_input,
+ const std::string& gold_output) {
+ ValidateRewriteInlineCss(StrCat(id, "-inline"),
+ css_input, gold_output, true, false);
+ ValidateRewriteExternalCss(StrCat(id, "-external"),
+ css_input, gold_output, true, false);
+ }
+
+ void ValidateNoChange(const StringPiece& id, const std::string& css_input) {
+ ValidateRewriteInlineCss(StrCat(id, "-inline"),
+ css_input, css_input, false, false);
+ ValidateRewriteExternalCss(StrCat(id, "-external"),
+ css_input, "", false, false);
+ }
+
+ void ValidateFailParse(const StringPiece& id, const std::string& css_input) {
+ ValidateRewriteInlineCss(StrCat(id, "-inline"),
+ css_input, css_input, false, true);
+ ValidateRewriteExternalCss(StrCat(id, "-external"),
+ css_input, "", false, true);
+ }
+
+
+ SimpleStats statistics_;
+ Variable* num_files_minified_;
+ Variable* minified_bytes_saved_;
+ Variable* num_parse_failures_;
+};
+
+
+TEST_F(CssFilterTest, SimpleRewriteCssTest) {
+ std::string input_style =
+ ".background_blue { background-color: #f00; }\n"
+ ".foreground_yellow { color: yellow; }\n";
+ std::string output_style =
+ ".background_blue{background-color:red}"
+ ".foreground_yellow{color:#ff0}";
+
+ ValidateRewrite("rewrite_css", input_style, output_style);
+}
+
+// Make sure we can deal with 0 character nodes between open and close of style.
+TEST_F(CssFilterTest, RewriteEmptyCssTest) {
+ ValidateNoChange("rewrite_empty_css", "");
+}
+
+// Make sure we don't change CSS with errors. Note: We can move these tests
+// to expected rewrites if we find safe ways to edit them.
+TEST_F(CssFilterTest, NoRewriteParseError) {
+ ValidateFailParse("non_unicode_charset",
+ "a { font-family: \"\xCB\xCE\xCC\xE5\"; }");
+ // From http://www.baidu.com/
+ ValidateFailParse("non_unicode_baidu",
+ "#lk span {font:14px \"\xCB\xCE\xCC\xE5\"}");
+ // From http://www.yahoo.com/
+ const char confusing_value[] =
+ "a { background-image:-webkit-gradient(linear, 50% 0%, 50% 100%,"
+ " from(rgb(232, 237, 240)), to(rgb(252, 252, 253)));}";
+ ValidateFailParse("non_standard_value", confusing_value);
+
+ ValidateFailParse("bad_char_in_selector", ".bold: { font-weight: bold }");
+}
+
+TEST_F(CssFilterTest, RewriteVariousCss) {
+ // Distilled examples.
+ const char* good_examples[] = {
+ "a.b #c.d e#d,f:g>h+i>j{color:red}", // .#,>+: in selectors
+ "a{border:solid 1px #ccc}", // Multiple values declaration
+ "a{border:none!important}", // !important
+ "a{background-image:url(foo.png)}", // url
+ "a{background-position:-19px 60%}", // negative position
+ "a{margin:0}", // 0 w/ no units
+ "a{padding:0.01em 0.25em}", // fractions and em
+ "a{-moz-border-radius-topleft:0}", // Browser-specific (-moz)
+ "a{background:none}", // CSS Parser used to expand this.
+ // http://code.google.com/p/modpagespeed/issues/detail?id=5
+ "a{font-family:trebuchet ms}", // Keep space between trebuchet and ms.
+ // http://code.google.com/p/modpagespeed/issues/detail?id=121
+ "a{color:inherit}",
+ };
+
+ for (int i = 0; i < arraysize(good_examples); ++i) {
+ std::string id = StringPrintf("distilled_css_good%d", i);
+ ValidateNoChange(id, good_examples[i]);
+ }
+
+ const char* fail_examples[] = {
+ // http://code.google.com/p/modpagespeed/issues/detail?id=50
+ "@media screen and (max-width:290px){a{color:red}}", // CSS3 "and (...)"
+ // http://code.google.com/p/modpagespeed/issues/detail?id=51
+ "a{box-shadow:-1px -2px 2px rgba(0, 0, 0, .15)}", // CSS3 rgba
+ // http://code.google.com/p/modpagespeed/issues/detail?id=66
+ "a{-moz-transform:rotate(7deg)}",
+ };
+
+ for (int i = 0; i < arraysize(fail_examples); ++i) {
+ std::string id = StringPrintf("distilled_css_fail%d", i);
+ ValidateFailParse(id, fail_examples[i]);
+ }
+}
+
+
+// Test more complicated CSS.
+TEST_F(CssFilterTest, ComplexCssTest) {
+ // Real-world examples. Picked out of Wikipedia's CSS.
+ const char* examples[][2] = {
+ { "#userlogin, #userloginForm {\n"
+ " border: solid 1px #cccccc;\n"
+ " padding: 1.2em;\n"
+ " float: left;\n"
+ "}\n",
+
+ "#userlogin,#userloginForm{border:solid 1px #ccc;padding:1.2em;"
+ "float:left}"},
+
+ { "h3 .editsection { font-size: 76%; font-weight: normal; }\n",
+ "h3 .editsection{font-size:76%;font-weight:normal}"},
+
+ { "div.magnify a, div.magnify img {\n"
+ " display: block;\n"
+ " border: none !important;\n"
+ " background: none !important;\n"
+ "}\n",
+
+ "div.magnify a,div.magnify img{display:block;border:none!important;"
+ "background:none!important}"},
+
+ { "#ca-watch.icon a:hover {\n"
+ " background-image: url('images/watch-icons.png?1');\n"
+ " background-position: -19px 60%;\n"
+ "}\n",
+
+ "#ca-watch.icon a:hover{background-image:url(images/watch-icons.png?1);"
+ "background-position:-19px 60%}"},
+
+ { "body {\n"
+ " background: White;\n"
+ " /*font-size: 11pt !important;*/\n"
+ " color: Black;\n"
+ " margin: 0;\n"
+ " padding: 0;\n"
+ "}\n",
+
+ "body{background:#fff;color:#000;margin:0;padding:0}"},
+
+ { ".suggestions-result{\n"
+ " color:black;\n"
+ " color:WindowText;\n"
+ " padding:0.01em 0.25em;\n"
+ "}\n",
+
+ // TODO(sligocki): Do we care about color:WindowText?
+ //".suggestions-result{color:#000;color:WindowText;padding:0.01em 0.25em}"
+
+ ".suggestions-result{color:#000;color:#000;padding:0.01em 0.25em}"},
+
+ { ".ui-corner-tl { -moz-border-radius-topleft: 0; -webkit-border-top-left"
+ "-radius: 0; }\n",
+
+ ".ui-corner-tl{-moz-border-radius-topleft:0;-webkit-border-top-left"
+ "-radius:0}"},
+
+ { ".ui-tabs .ui-tabs-nav li.ui-tabs-selected a, .ui-tabs .ui-tabs-nav li."
+ "ui-state-disabled a, .ui-tabs .ui-tabs-nav li.ui-state-processing a { "
+ "cursor: pointer; }\n",
+
+ ".ui-tabs .ui-tabs-nav li.ui-tabs-selected a,.ui-tabs .ui-tabs-nav "
+ "li.ui-state-disabled a,.ui-tabs .ui-tabs-nav li.ui-state-processing a{"
+ "cursor:pointer}"},
+
+ // http://code.google.com/p/modpagespeed/issues/detail?id=121
+ { "body { font: 2em sans-serif; }", "body{font:2em sans-serif}" },
+ { "body { font: 0.75em sans-serif; }", "body{font:0.75em sans-serif}" },
+
+ // http://code.google.com/p/modpagespeed/issues/detail?id=128
+ { "#breadcrumbs ul { list-style-type: none; }",
+ "#breadcrumbs ul{list-style-type:none}" },
+
+ // http://code.google.com/p/modpagespeed/issues/detail?id=126
+ // Extra spaces assure that we actually rewrite the first arg even if
+ // font: is expanded by parser.
+ { ".menu { font: menu; } ", ".menu{font:menu}" },
+ };
+
+ for (int i = 0; i < arraysize(examples); ++i) {
+ std::string id = StringPrintf("complex_css%d", i);
+ ValidateRewrite(id, examples[i][0], examples[i][1]);
+ }
+
+ const char* parse_fail_examples[] = {
+ ".ui-datepicker-cover {\n"
+ " display: none; /*sorry for IE5*/\n"
+ " display/**/: block; /*sorry for IE5*/\n"
+ " position: absolute; /*must have*/\n"
+ " z-index: -1; /*must have*/\n"
+ " filter: mask(); /*must have*/\n"
+ " top: -4px; /*must have*/\n"
+ " left: -4px; /*must have*/\n"
+ " width: 200px; /*must have*/\n"
+ " height: 200px; /*must have*/\n"
+ "}\n",
+
+ // Right now we bail on parsing the above. Could probably be minified to:
+ //".ui-datepicker-cover{display:none;display/**/:block;position:absolute;"
+ //"z-index:-1;filter:mask();top:-4px;left:-4px;width:200px;height:200px}"
+ // TODO(sligocki): When this is parsed correctly, move it up to examples[][]
+
+ ".shift {\n"
+ " -moz-transform: rotate(7deg);\n"
+ " -webkit-transform: rotate(7deg);\n"
+ " -moz-transform: skew(-25deg);\n"
+ " -webkit-transform: skew(-25deg);\n"
+ " -moz-transform: scale(0.5);\n"
+ " -webkit-transform: scale(0.5);\n"
+ " -moz-transform: translate(3em, 0);\n"
+ " -webkit-transform: translate(3em, 0);\n"
+ "}\n",
+
+ // Right now we bail on parsing the above. Could probably be minified to:
+ //".shift{-moz-transform:rotate(7deg);-webkit-transform:rotate(7deg);"
+ //"-moz-transform:skew(-25deg);-webkit-transform:skew(-25deg);"
+ //"-moz-transform:scale(0.5);-webkit-transform:scale(0.5);"
+ //"-moz-transform:translate(3em,0);-webkit-transform:translate(3em,0);}"
+ // TODO(sligocki): When this is parsed correctly, move it up to examples[][]
+ };
+
+ for (int i = 0; i < arraysize(parse_fail_examples); ++i) {
+ std::string id = StringPrintf("complex_css_parse_fail%d", i);
+ ValidateFailParse(id, parse_fail_examples[i]);
+ }
+
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/css_inline_filter.cc b/trunk/src/net/instaweb/rewriter/css_inline_filter.cc
new file mode 100644
index 0000000..c6e0e40
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/css_inline_filter.cc
@@ -0,0 +1,122 @@
+// Copyright 2010 Google Inc. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "net/instaweb/rewriter/public/css_inline_filter.h"
+
+#include "base/logging.h"
+#include "base/scoped_ptr.h"
+#include "base/string_util.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_node.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/rewriter/public/css_tag_scanner.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/string_writer.h"
+
+namespace net_instaweb {
+
+CssInlineFilter::CssInlineFilter(RewriteDriver* driver)
+ : CommonFilter(driver),
+ html_parse_(driver->html_parse()),
+ href_atom_(html_parse_->Intern("href")),
+ link_atom_(html_parse_->Intern("link")),
+ media_atom_(html_parse_->Intern("media")),
+ rel_atom_(html_parse_->Intern("rel")),
+ style_atom_(html_parse_->Intern("style")),
+ size_threshold_bytes_(driver->options()->css_inline_max_bytes()) {}
+
+void CssInlineFilter::StartDocumentImpl() {
+ // TODO(sligocki): Domain lawyerify.
+ domain_ = html_parse_->gurl().host();
+}
+
+void CssInlineFilter::EndDocument() {
+ domain_.clear();
+}
+
+void CssInlineFilter::EndElementImpl(HtmlElement* element) {
+ if (element->tag() == link_atom_) {
+ const char* rel = element->AttributeValue(rel_atom_);
+ if (rel == NULL || strcmp(rel, "stylesheet")) {
+ return;
+ }
+
+ // If the link tag has a media attribute whose value isn't "all", don't
+ // inline. (Note that "all" is equivalent to having no media attribute;
+ // see http://www.w3.org/TR/html5/semantics.html#the-style-element)
+ const char* media = element->AttributeValue(media_atom_);
+ if (media != NULL && strcmp(media, "all") != 0) {
+ return;
+ }
+
+ // Get the URL where the external script is stored
+ const char* href = element->AttributeValue(href_atom_);
+ if (href == NULL) {
+ return; // We obviously can't inline if the URL isn't there.
+ }
+
+ // Make sure we're not moving across domains -- CSS can potentially contain
+ // Javascript expressions.
+ // TODO(jmaessen): Is the domain lawyer policy the appropriate one here?
+ // Or do we still have to check for strict domain equivalence?
+ // If so, add an inline-in-page policy to domainlawyer in some form,
+ // as we make a similar policy decision in js_inline_filter.
+ MessageHandler* message_handler = html_parse_->message_handler();
+ scoped_ptr<Resource> resource(CreateInputResourceAndReadIfCached(href));
+ if (resource == NULL || !resource->ContentsValid()) {
+ return;
+ }
+
+ // Check that the file is small enough to inline.
+ StringPiece contents = resource->contents();
+ if (contents.size() > size_threshold_bytes_) {
+ return;
+ }
+
+ // Absolutify the URLs in the CSS -- relative URLs will break otherwise.
+ std::string rewritten_contents;
+ StringWriter writer(&rewritten_contents);
+ std::string input_dir =
+ GoogleUrl::AllExceptLeaf(GoogleUrl::Create(resource->url()));
+ std::string base_dir = GoogleUrl::AllExceptLeaf(base_gurl());
+ bool written;
+ if (input_dir == base_dir) {
+ // We don't need to absolutify URLs if input directory is same as base.
+ written = writer.Write(contents, message_handler);
+ } else {
+ // If they are different directories, we need to absolutify.
+ // TODO(sligocki): Perhaps we should use the real CSS parser.
+ written = CssTagScanner::AbsolutifyUrls(contents, resource->url(),
+ &writer, message_handler);
+ }
+ if (!written) {
+ return;
+ }
+
+ // Inline the CSS.
+ HtmlElement* style_element =
+ html_parse_->NewElement(element->parent(), style_atom_);
+ if (html_parse_->ReplaceNode(element, style_element)) {
+ html_parse_->AppendChild(
+ style_element, html_parse_->NewCharactersNode(element,
+ rewritten_contents));
+ }
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/css_inline_filter_test.cc b/trunk/src/net/instaweb/rewriter/css_inline_filter_test.cc
new file mode 100644
index 0000000..798e4c2
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/css_inline_filter_test.cc
@@ -0,0 +1,133 @@
+/*
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+
+namespace net_instaweb {
+
+namespace {
+
+class CssInlineFilterTest : public ResourceManagerTestBase {
+ protected:
+ void TestInlineCss(const std::string& html_url,
+ const std::string& css_url,
+ const std::string& other_attrs,
+ const std::string& css_original_body,
+ bool expect_inline,
+ const std::string& css_rewritten_body) {
+ AddFilter(RewriteOptions::kInlineCss);
+
+ const std::string html_input =
+ "<head>\n"
+ " <link rel=\"stylesheet\" href=\"" + css_url + "\"" +
+ (other_attrs.empty() ? "" : " " + other_attrs) + ">\n"
+ "</head>\n"
+ "<body>Hello, world!</body>\n";
+
+ // Put original CSS file into our fetcher.
+ SimpleMetaData default_css_header;
+ resource_manager_->SetDefaultHeaders(&kContentTypeCss,
+ &default_css_header);
+ mock_url_fetcher_.SetResponse(css_url, default_css_header,
+ css_original_body);
+
+ // Rewrite the HTML page.
+ ParseUrl(html_url, html_input);
+
+ const std::string expected_output =
+ (!expect_inline ? html_input :
+ "<head>\n"
+ " <style>" + css_rewritten_body + "</style>\n"
+ "</head>\n"
+ "<body>Hello, world!</body>\n");
+ EXPECT_EQ(AddHtmlBody(expected_output), output_buffer_);
+ }
+};
+
+TEST_F(CssInlineFilterTest, InlineCssSimple) {
+ const std::string css = "BODY { color: red; }\n";
+ TestInlineCss("http://www.example.com/index.html",
+ "http://www.example.com/styles.css",
+ "", css, true, css);
+}
+
+TEST_F(CssInlineFilterTest, InlineCssAbsolutifyUrls1) {
+ // CSS with a relative URL that needs to be changed:
+ const std::string css1 =
+ "BODY { background-image: url('bg.png'); }\n";
+ const std::string css2 =
+ "BODY { background-image: "
+ "url('http://www.example.com/foo/bar/bg.png'); }\n";
+ TestInlineCss("http://www.example.com/index.html",
+ "http://www.example.com/foo/bar/baz.css",
+ "", css1, true, css2);
+}
+
+TEST_F(CssInlineFilterTest, InlineCssAbsolutifyUrls2) {
+ // CSS with a relative URL, this time with ".." in it:
+ const std::string css1 =
+ "BODY { background-image: url('../quux/bg.png'); }\n";
+ const std::string css2 =
+ "BODY { background-image: "
+ "url('http://www.example.com/foo/quux/bg.png'); }\n";
+ TestInlineCss("http://www.example.com/index.html",
+ "http://www.example.com/foo/bar/baz.css",
+ "", css1, true, css2);
+}
+
+TEST_F(CssInlineFilterTest, NoAbsolutifyUrlsSameDir) {
+ const std::string css = "BODY { background-image: url('bg.png'); }\n";
+ TestInlineCss("http://www.example.com/index.html",
+ "http://www.example.com/baz.css",
+ "", css, true, css);
+}
+
+TEST_F(CssInlineFilterTest, DoNotInlineCssWithMediaAttr) {
+ const std::string css = "BODY { color: red; }\n";
+ TestInlineCss("http://www.example.com/index.html",
+ "http://www.example.com/styles.css",
+ "media=\"print\"", css, false, "");
+}
+
+TEST_F(CssInlineFilterTest, DoInlineCssWithMediaAll) {
+ const std::string css = "BODY { color: red; }\n";
+ TestInlineCss("http://www.example.com/index.html",
+ "http://www.example.com/styles.css",
+ "media=\"all\"", css, true, css);
+}
+
+TEST_F(CssInlineFilterTest, DoNotInlineCssTooBig) {
+ // CSS too large to inline:
+ const int64 length = 2 * RewriteOptions::kDefaultCssInlineMaxBytes;
+ TestInlineCss("http://www.example.com/index.html",
+ "http://www.example.com/styles.css", "",
+ ("BODY { background-image: url('" +
+ std::string(length, 'z') + ".png'); }\n"),
+ false, "");
+}
+
+TEST_F(CssInlineFilterTest, DoNotInlineCssDifferentDomain) {
+ // TODO(mdsteele): Is switching domains in fact an issue for CSS?
+ TestInlineCss("http://www.example.com/index.html",
+ "http://www.example.org/styles.css",
+ "", "BODY { color: red; }\n", false, "");
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/css_minify.cc b/trunk/src/net/instaweb/rewriter/css_minify.cc
new file mode 100644
index 0000000..af1e78f
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/css_minify.cc
@@ -0,0 +1,298 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/rewriter/public/css_minify.h"
+
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/writer.h"
+#include "webutil/css/parser.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// Escape [(), \t\r\n\\'"]
+std::string CSSEscapeString(const StringPiece& src) {
+ const int dest_length = src.size() * 2 + 1; // Maximum possible expansion
+ scoped_array<char> dest(new char[dest_length]);
+
+ const char* src_end = src.data() + src.size();
+ int used = 0;
+
+ for (const char* p = src.data(); p < src_end; p++) {
+ switch (*p) {
+ case '\n': dest[used++] = '\\'; dest[used++] = 'n'; break;
+ case '\r': dest[used++] = '\\'; dest[used++] = 'r'; break;
+ case '\t': dest[used++] = '\\'; dest[used++] = 't'; break;
+ case '\"': case '\'': case '\\': case ',': case '(': case ')':
+ dest[used++] = '\\';
+ dest[used++] = *p;
+ break;
+ default: dest[used++] = *p; break;
+ }
+ }
+
+ return std::string(dest.get(), used);
+}
+
+std::string CSSEscapeString(const UnicodeText& src) {
+ return CSSEscapeString(StringPiece(src.utf8_data(), src.utf8_length()));
+}
+
+} // namespace
+
+bool CssMinify::Stylesheet(const Css::Stylesheet& stylesheet,
+ Writer* writer,
+ MessageHandler* handler) {
+ // Get an object to encapsulate writing.
+ CssMinify minifier(writer, handler);
+ minifier.Minify(stylesheet);
+ return minifier.ok_;
+}
+
+CssMinify::CssMinify(Writer* writer, MessageHandler* handler)
+ : writer_(writer), handler_(handler), ok_(true) {
+}
+
+CssMinify::~CssMinify() {
+}
+
+// Write if we have not encountered write error yet.
+void CssMinify::Write(const StringPiece& str) {
+ if (ok_) {
+ ok_ &= writer_->Write(str, handler_);
+ }
+}
+
+// Write out minified version of each element of vector using supplied function
+// seperated by sep.
+template<typename Container>
+void CssMinify::JoinMinify(const Container& container, const StringPiece& sep) {
+ JoinMinifyIter(container.begin(), container.end(), sep);
+}
+
+template<typename Iterator>
+void CssMinify::JoinMinifyIter(const Iterator& begin, const Iterator& end,
+ const StringPiece& sep) {
+ for (Iterator iter = begin; iter != end; ++iter) {
+ if (iter != begin) {
+ Write(sep);
+ }
+ Minify(**iter);
+ }
+}
+
+template<typename Container>
+void CssMinify::JoinMediaMinify(const Container& container,
+ const StringPiece& sep) {
+ for (typename Container::const_iterator iter = container.begin();
+ iter != container.end(); ++iter) {
+ if (iter != container.begin()) {
+ Write(sep);
+ }
+ Write(CSSEscapeString(*iter));
+ }
+}
+
+
+// Write the minified versions of each type. Most of these are called via
+// templated instantiations of JoinMinify (or JoinMinifyIter) so that we can
+// abstract the idea of minifying all sub-elements of a vector and joining them
+// together.
+// Adapted from webutil/css/tostring.cc
+
+void CssMinify::Minify(const Css::Stylesheet& stylesheet) {
+ // We might want to add in unnecessary newlines between rules and imports
+ // so that some readability is preserved.
+ JoinMinify(stylesheet.imports(), "");
+ JoinMinify(stylesheet.rulesets(), "");
+}
+
+void CssMinify::Minify(const Css::Import& import) {
+ Write("@import url(");
+ // TODO(sligocki): Make a URL printer method that absolutifies and prints.
+ Write(CSSEscapeString(import.link));
+ Write(") ");
+ JoinMediaMinify(import.media, ",");
+ Write(";");
+}
+
+void CssMinify::Minify(const Css::Ruleset& ruleset) {
+ if (!ruleset.media().empty()) {
+ Write("@media ");
+ JoinMediaMinify(ruleset.media(), ",");
+ Write("{");
+ }
+
+ JoinMinify(ruleset.selectors(), ",");
+ Write("{");
+ JoinMinify(ruleset.declarations(), ";");
+ Write("}");
+
+ if (!ruleset.media().empty()) {
+ Write("}");
+ }
+}
+
+void CssMinify::Minify(const Css::Selector& selector) {
+ // Note Css::Selector == std::vector<Css::SimpleSelectors*>
+ Css::Selector::const_iterator iter = selector.begin();
+ if (iter != selector.end()) {
+ bool isfirst = true;
+ Minify(**iter, isfirst);
+ ++iter;
+ JoinMinifyIter(iter, selector.end(), "");
+ }
+}
+
+void CssMinify::Minify(const Css::SimpleSelectors& sselectors, bool isfirst) {
+ if (sselectors.combinator() == Css::SimpleSelectors::CHILD) {
+ Write(">");
+ } else if (sselectors.combinator() == Css::SimpleSelectors::SIBLING) {
+ Write("+");
+ } else if (!isfirst) {
+ Write(" ");
+ }
+ // Note Css::SimpleSelectors == std::vector<Css::SimpleSelector*>
+ JoinMinify(sselectors, "");
+}
+
+void CssMinify::Minify(const Css::SimpleSelector& sselector) {
+ // SimpleSelector::ToString is already basically minified.
+ Write(sselector.ToString());
+}
+
+namespace {
+
+// TODO(sligocki): Either make this an accessible function in
+// webutil/css/tostring or specialize it for minifier.
+//
+// Note that currently the style is terrible and it will crash the program if
+// we have >= 5 args.
+std::string FontToString(const Css::Values& font_values) {
+ CHECK_LE(5U, font_values.size());
+ std::string tmp, result;
+
+ // font-style: defaults to normal
+ tmp = font_values.get(0)->ToString();
+ if (tmp != "normal") result += tmp + " ";
+ // font-variant: defaults to normal
+ tmp = font_values.get(1)->ToString();
+ if (tmp != "normal") result += tmp + " ";
+ // font-weight: defaults to normal
+ tmp = font_values.get(2)->ToString();
+ if (tmp != "normal") result += tmp + " ";
+ // font-size is required
+ result += font_values.get(3)->ToString();
+ // line-height: defaults to normal
+ tmp = font_values.get(4)->ToString();
+ if (tmp != "normal") result += "/" + tmp;
+ // font-family:
+ for (int i = 5, n = font_values.size(); i < n; ++i)
+ result += (i == 5 ? " " : ",") + font_values.get(i)->ToString();
+
+ return result;
+}
+
+} // namespace
+
+void CssMinify::Minify(const Css::Declaration& declaration) {
+ Write(declaration.prop_text());
+ Write(":");
+ switch (declaration.prop()) {
+ case Css::Property::FONT_FAMILY:
+ JoinMinify(*declaration.values(), ",");
+ break;
+ case Css::Property::FONT:
+ // font: menu special case.
+ if (declaration.values()->size() == 1) {
+ JoinMinify(*declaration.values(), " ");
+ // Normal font notation.
+ } else if (declaration.values()->size() >= 5) {
+ Write(FontToString(*declaration.values()));
+ } else {
+ handler_->Message(kError, "Unexpected number of values in "
+ "font declaration: %d",
+ static_cast<int>(declaration.values()->size()));
+ ok_ = false;
+ }
+ break;
+ default:
+ JoinMinify(*declaration.values(), " ");
+ break;
+ }
+ if (declaration.IsImportant()) {
+ Write("!important");
+ }
+}
+
+void CssMinify::Minify(const Css::Value& value) {
+ switch (value.GetLexicalUnitType()) {
+ case Css::Value::NUMBER:
+ // TODO(sligocki): Minify number
+ // TODO(sligocki): Check that exponential notation is appropriate.
+ Write(StringPrintf("%g%s",
+ value.GetFloatValue(),
+ value.GetDimensionUnitText().c_str()));
+ break;
+ case Css::Value::URI:
+ // TODO(sligocki): Make a URL printer method that absolutifies and prints.
+ Write("url(");
+ Write(CSSEscapeString(value.GetStringValue()));
+ Write(")");
+ break;
+ case Css::Value::COUNTER:
+ Write("counter(");
+ Write(value.GetParameters()->ToString());
+ Write(")");
+ break;
+ case Css::Value::FUNCTION:
+ Write(CSSEscapeString(value.GetFunctionName()));
+ Write("(");
+ Write(value.GetParameters()->ToString());
+ Write(")");
+ break;
+ case Css::Value::RECT:
+ Write("rect(");
+ Write(value.GetParameters()->ToString());
+ Write(")");
+ break;
+ case Css::Value::COLOR:
+ // TODO(sligocki): Can we assert, or might this happen in the wild?
+ CHECK(value.GetColorValue().IsDefined());
+ Write(HtmlColorUtils::MaybeConvertToCssShorthand(
+ value.GetColorValue()));
+ break;
+ case Css::Value::STRING:
+ Write("\"");
+ Write(CSSEscapeString(value.GetStringValue()));
+ Write("\"");
+ break;
+ case Css::Value::IDENT:
+ Write(CSSEscapeString(value.GetIdentifierText()));
+ break;
+ case Css::Value::UNKNOWN:
+ handler_->Message(kError, "Unknown attribute");
+ ok_ = false;
+ break;
+ case Css::Value::DEFAULT:
+ break;
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/css_move_to_head_filter.cc b/trunk/src/net/instaweb/rewriter/css_move_to_head_filter.cc
new file mode 100644
index 0000000..11994e9
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/css_move_to_head_filter.cc
@@ -0,0 +1,86 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/css_move_to_head_filter.h"
+
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/util/public/statistics.h"
+
+namespace {
+
+// names for Statistics variables.
+const char kCssElements[] = "css_elements";
+
+} // namespace
+
+namespace net_instaweb {
+
+CssMoveToHeadFilter::CssMoveToHeadFilter(HtmlParse* html_parse,
+ Statistics* statistics)
+ : html_parse_(html_parse),
+ css_tag_scanner_(html_parse),
+ counter_(NULL) {
+ s_head_ = html_parse->Intern("head");
+ s_noscript_ = html_parse->Intern("noscript");
+ s_style_ = html_parse->Intern("style");
+ if (statistics != NULL) {
+ counter_ = statistics->GetVariable(kCssElements);
+ }
+}
+
+void CssMoveToHeadFilter::Initialize(Statistics* statistics) {
+ statistics->AddVariable(kCssElements);
+}
+
+void CssMoveToHeadFilter::StartDocument() {
+ head_element_ = NULL;
+ noscript_element_ = NULL;
+}
+
+void CssMoveToHeadFilter::StartElement(HtmlElement* element) {
+ if (noscript_element_ == NULL && element->tag() == s_noscript_) {
+ noscript_element_ = element; // Record top-level <noscript>.
+ }
+}
+
+void CssMoveToHeadFilter::EndElement(HtmlElement* element) {
+ if ((head_element_ == NULL) && (element->tag() == s_head_)) {
+ head_element_ = element;
+
+ } else if (element == noscript_element_) {
+ noscript_element_ = NULL; // We are exitting the top level </noscript>.
+
+ // Do not move anything out of a <noscript> element and we can only move
+ // this to <head> if <head> is still rewritable.
+ } else if (noscript_element_ == NULL && head_element_ != NULL &&
+ html_parse_->IsRewritable(head_element_)){
+ HtmlElement::Attribute* href;
+ const char* media;
+ if (element->tag() == s_style_ ||
+ css_tag_scanner_.ParseCssElement(element, &href, &media)) {
+ html_parse_->MoveCurrentInto(head_element_);
+ // TODO(sligocki): It'd be nice to have this pattern simplified.
+ if (counter_ != NULL) {
+ counter_->Add(1);
+ }
+ }
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/css_move_to_head_filter_test.cc b/trunk/src/net/instaweb/rewriter/css_move_to_head_filter_test.cc
new file mode 100644
index 0000000..015a1d9
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/css_move_to_head_filter_test.cc
@@ -0,0 +1,105 @@
+// Copyright 2010 Google Inc. All Rights Reserved.
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/rewriter/public/css_move_to_head_filter.h"
+
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace net_instaweb {
+
+namespace {
+
+class CssMoveToHeadFilterTest : public ResourceManagerTestBase {
+ protected:
+ CssMoveToHeadFilterTest()
+ : move_to_head_filter_(rewrite_driver_.html_parse(), NULL) {
+ rewrite_driver_.html_parse()->AddFilter(&move_to_head_filter_);
+ }
+
+ CssMoveToHeadFilter move_to_head_filter_;
+};
+
+TEST_F(CssMoveToHeadFilterTest, MovesCssToHead) {
+ static const char html_input[] =
+ "<head>\n"
+ " <title>Example</title>\n"
+ "</head>\n"
+ "<body>\n"
+ " Hello,\n"
+ " <link rel='stylesheet' href='a.css' type='text/css'>" // no newline
+ "<link rel='stylesheet' href='b.css' type='text/css'>\n" // no indent
+ " <style type='text/css'>a {color: red }</style>\n"
+ " World!\n"
+ " <link rel='stylesheet' href='c.css' type='text/css'>\n"
+ "</body>\n";
+
+ static const char expected_output[] =
+ "<head>\n"
+ " <title>Example</title>\n"
+ "<link rel='stylesheet' href='a.css' type='text/css'>" // no newline
+ "<link rel='stylesheet' href='b.css' type='text/css'>" // no newline
+ "<style type='text/css'>a {color: red }</style>" // no newline
+ "<link rel='stylesheet' href='c.css' type='text/css'>" // no newline
+ "</head>\n"
+ "<body>\n"
+ " Hello,\n"
+ " \n"
+ " \n"
+ " World!\n"
+ " \n"
+ "</body>\n";
+
+ ValidateExpected("move_css_to_head", html_input, expected_output);
+}
+
+TEST_F(CssMoveToHeadFilterTest, DoesntMoveOutOfNoScript) {
+ static const char html[] =
+ "<head>\n"
+ " <title>Example</title>\n"
+ "</head>\n"
+ "<body>\n"
+ " <noscript>\n"
+ " <link rel='stylesheet' href='a.css' type='text/css'>\n"
+ " </noscript>\n"
+ "</body>\n";
+
+ ValidateNoChanges("noscript", html);
+}
+
+
+TEST_F(CssMoveToHeadFilterTest, DoesntReorderCss) {
+ static const char html[] =
+ "<head>\n"
+ " <title>Example</title>\n"
+ "</head>\n"
+ "<body>\n"
+ " <link rel='stylesheet' href='a.css' type='text/css'>\n"
+ " <link rel='stylesheet' href='b.css' type='text/css'>\n"
+ " <style type='text/css'>a { color: red }</style>\n"
+ " <link rel='stylesheet' href='d.css' type='text/css'>\n"
+ "</body>\n";
+
+ Parse("no_reorder_css", html);
+ LOG(INFO) << "output_buffer_ = " << output_buffer_;
+ size_t a_loc = output_buffer_.find("href='a.css'");
+ size_t b_loc = output_buffer_.find("href='b.css'");
+ size_t c_loc = output_buffer_.find("a { color: red }");
+ size_t d_loc = output_buffer_.find("href='d.css'");
+
+ // Make sure that all attributes are in output_buffer_ ...
+ EXPECT_NE(output_buffer_.npos, a_loc);
+ EXPECT_NE(output_buffer_.npos, b_loc);
+ EXPECT_NE(output_buffer_.npos, c_loc);
+ EXPECT_NE(output_buffer_.npos, d_loc);
+
+ // ... and that they are still in the right order (specifically, that
+ // the last link wasn't moved above the style).
+ EXPECT_LE(a_loc, b_loc);
+ EXPECT_LE(b_loc, c_loc);
+ EXPECT_LE(c_loc, d_loc);
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/css_outline_filter.cc b/trunk/src/net/instaweb/rewriter/css_outline_filter.cc
new file mode 100644
index 0000000..7c82a71
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/css_outline_filter.cc
@@ -0,0 +1,195 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/rewriter/public/css_outline_filter.h"
+
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/css_tag_scanner.h"
+#include "net/instaweb/rewriter/public/output_resource.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include <string>
+#include "net/instaweb/util/public/string_writer.h"
+
+namespace net_instaweb {
+
+const char kStylesheet[] = "stylesheet";
+
+const char CssOutlineFilter::kFilterId[] = "co";
+
+CssOutlineFilter::CssOutlineFilter(RewriteDriver* driver)
+ : CommonFilter(driver),
+ inline_element_(NULL),
+ html_parse_(driver->html_parse()),
+ resource_manager_(driver->resource_manager()),
+ size_threshold_bytes_(driver->options()->css_outline_min_bytes()),
+ s_link_(html_parse_->Intern("link")),
+ s_style_(html_parse_->Intern("style")),
+ s_rel_(html_parse_->Intern("rel")),
+ s_href_(html_parse_->Intern("href")),
+ s_type_(html_parse_->Intern("type")) { }
+
+void CssOutlineFilter::StartDocumentImpl() {
+ inline_element_ = NULL;
+ buffer_.clear();
+}
+
+void CssOutlineFilter::StartElementImpl(HtmlElement* element) {
+ // No tags allowed inside style element.
+ if (inline_element_ != NULL) {
+ // TODO(sligocki): Add negative unit tests to hit these errors.
+ html_parse_->ErrorHere("Tag '%s' found inside style.",
+ element->tag().c_str());
+ inline_element_ = NULL; // Don't outline what we don't understand.
+ buffer_.clear();
+ }
+ if (element->tag() == s_style_) {
+ inline_element_ = element;
+ buffer_.clear();
+ }
+}
+
+void CssOutlineFilter::EndElementImpl(HtmlElement* element) {
+ if (inline_element_ != NULL) {
+ if (element != inline_element_) {
+ // No other tags allowed inside style element.
+ html_parse_->ErrorHere("Tag '%s' found inside style.",
+ element->tag().c_str());
+
+ } else if (buffer_.size() >= size_threshold_bytes_) {
+ OutlineStyle(inline_element_, buffer_);
+ } else {
+ html_parse_->InfoHere("Inline element not outlined because its size %d, "
+ "is below threshold %d",
+ static_cast<int>(buffer_.size()),
+ static_cast<int>(size_threshold_bytes_));
+ }
+ inline_element_ = NULL;
+ buffer_.clear();
+ }
+}
+
+void CssOutlineFilter::Flush() {
+ // If we were flushed in a style element, we cannot outline it.
+ inline_element_ = NULL;
+ buffer_.clear();
+}
+
+void CssOutlineFilter::Characters(HtmlCharactersNode* characters) {
+ if (inline_element_ != NULL) {
+ buffer_ += characters->contents();
+ }
+}
+
+void CssOutlineFilter::Comment(HtmlCommentNode* comment) {
+ if (inline_element_ != NULL) {
+ html_parse_->ErrorHere("Comment found inside style.");
+ inline_element_ = NULL; // Don't outline what we don't understand.
+ buffer_.clear();
+ }
+}
+
+void CssOutlineFilter::Cdata(HtmlCdataNode* cdata) {
+ if (inline_element_ != NULL) {
+ html_parse_->ErrorHere("CDATA found inside style.");
+ inline_element_ = NULL; // Don't outline what we don't understand.
+ buffer_.clear();
+ }
+}
+
+void CssOutlineFilter::IEDirective(HtmlIEDirectiveNode* directive) {
+ if (inline_element_ != NULL) {
+ html_parse_->ErrorHere("IE Directive found inside style.");
+ inline_element_ = NULL; // Don't outline what we don't understand.
+ buffer_.clear();
+ }
+}
+
+// Try to write content and possibly header to resource.
+bool CssOutlineFilter::WriteResource(const StringPiece& content,
+ OutputResource* resource,
+ MessageHandler* handler) {
+ // We set the TTL of the origin->hashed_name map to 0 because this is
+ // derived from the inlined HTML.
+ int64 origin_expire_time_ms = 0;
+ return resource_manager_->Write(HttpStatus::kOK, content, resource,
+ origin_expire_time_ms, handler);
+}
+
+// Create file with style content and remove that element from DOM.
+void CssOutlineFilter::OutlineStyle(HtmlElement* style_element,
+ const std::string& content_str) {
+ StringPiece content(content_str);
+ if (html_parse_->IsRewritable(style_element)) {
+ // Create style file from content.
+ const char* type = style_element->AttributeValue(s_type_);
+ // We only deal with CSS styles. If no type specified, CSS is assumed.
+ // See http://www.w3.org/TR/html5/semantics.html#the-style-element
+ if (type == NULL || strcmp(type, kContentTypeCss.mime_type()) == 0) {
+ MessageHandler* handler = html_parse_->message_handler();
+ // Create outline resource at the document location, not base URL location
+ scoped_ptr<OutputResource> output_resource(
+ resource_manager_->CreateOutputResourceWithPath(
+ GoogleUrl::AllExceptLeaf(html_parse_->gurl()),
+ kFilterId, "_",
+ &kContentTypeCss, handler));
+
+ // Absolutify URLs in content.
+ std::string absolute_content;
+ StringWriter absolute_writer(&absolute_content);
+ std::string base_dir = GoogleUrl::AllExceptLeaf(base_gurl());
+ bool content_valid = true;
+ if (base_dir != output_resource->resolved_base()) {
+ // TODO(sligocki): Use CssParser instead of CssTagScanner hack.
+ content_valid = CssTagScanner::AbsolutifyUrls(
+ content, GoogleUrl::Spec(base_gurl()), &absolute_writer, handler);
+ content = absolute_content; // StringPiece point to the new string.
+
+ }
+ if (content_valid &&
+ WriteResource(content, output_resource.get(), handler)) {
+ HtmlElement* link_element = html_parse_->NewElement(
+ style_element->parent(), s_link_);
+ link_element->AddAttribute(s_rel_, kStylesheet, "'");
+ link_element->AddAttribute(s_href_, output_resource->url(), "'");
+ // Add all style atrributes to link.
+ for (int i = 0; i < style_element->attribute_size(); ++i) {
+ const HtmlElement::Attribute& attr = style_element->attribute(i);
+ link_element->AddAttribute(attr);
+ }
+ // Add link to DOM.
+ html_parse_->InsertElementAfterElement(style_element, link_element);
+ // Remove style element from DOM.
+ if (!html_parse_->DeleteElement(style_element)) {
+ html_parse_->FatalErrorHere("Failed to delete inline sytle element");
+ }
+ }
+ } else {
+ std::string element_string;
+ style_element->ToString(&element_string);
+ html_parse_->InfoHere("Cannot outline non-css stylesheet %s",
+ element_string.c_str());
+ }
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/css_outline_filter_test.cc b/trunk/src/net/instaweb/rewriter/css_outline_filter_test.cc
new file mode 100644
index 0000000..77e8fff
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/css_outline_filter_test.cc
@@ -0,0 +1,128 @@
+/*
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/rewriter/public/css_outline_filter.h"
+
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+
+namespace net_instaweb {
+
+namespace {
+
+class CssOutlineFilterTest : public ResourceManagerTestBase {
+ protected:
+ // Test general situations.
+ void TestOutlineCss(const std::string& html_url,
+ const std::string& other_content, // E.g. <base href>
+ const std::string& css_original_body,
+ bool expect_outline,
+ const std::string& css_rewritten_body) {
+ // TODO(sligocki): Test with outline threshold > 0.
+ options_.set_css_outline_min_bytes(0);
+ AddFilter(RewriteOptions::kOutlineCss);
+
+ // Figure out outline_url.
+ std::string hash = resource_manager_->hasher()->Hash(css_rewritten_body);
+ GURL html_gurl(html_url);
+ GURL outline_gurl = html_gurl.Resolve(
+ Encode("", CssOutlineFilter::kFilterId, hash, "_", "css"));
+ std::string outline_url = GoogleUrl::Spec(outline_gurl);
+ // Figure out outline_filename.
+ std::string outline_filename;
+ filename_encoder_.Encode(file_prefix_, outline_url, &outline_filename);
+ // Make sure the file we check later was written this time, rm any old one.
+ DeleteFileIfExists(outline_filename);
+
+ const std::string html_input =
+ "<head>\n" +
+ other_content +
+ " <style>" + css_original_body + "</style>\n"
+ "</head>\n"
+ "<body>Hello, world!</body>\n";
+
+ // Rewrite the HTML page.
+ ParseUrl(html_url, html_input);
+
+ // Check output HTML.
+ const std::string expected_output =
+ (!expect_outline ? html_input :
+ "<head>\n" +
+ other_content +
+ " <link rel='stylesheet' href='" + outline_url + "'>\n"
+ "</head>\n"
+ "<body>Hello, world!</body>\n");
+ EXPECT_EQ(AddHtmlBody(expected_output), output_buffer_);
+
+ // Expected headers.
+ std::string expected_headers;
+ AppendDefaultHeaders(kContentTypeCss, resource_manager_, &expected_headers);
+
+ // Check file was written.
+ // TODO(sligocki): Should we check this or only the fetch below?
+ std::string actual_outline;
+ ASSERT_TRUE(file_system_.ReadFile(outline_filename.c_str(),
+ &actual_outline,
+ &message_handler_));
+ EXPECT_EQ(expected_headers + css_rewritten_body, actual_outline);
+
+ // Check fetched resource.
+ actual_outline.clear();
+ EXPECT_TRUE(ServeResourceUrl(outline_url, &actual_outline));
+ // TODO(sligocki): Check headers?
+ EXPECT_EQ(css_rewritten_body, actual_outline);
+ }
+
+ // Test with different hashers for a specific situation.
+ void OutlineStyle(const StringPiece& id, Hasher* hasher) {
+ resource_manager_->set_hasher(hasher);
+
+ std::string html_url = StrCat("http://outline_style.test/", id, ".html");
+ std::string style_text = "background_blue { background-color: blue; }\n"
+ "foreground_yellow { color: yellow; }\n";
+ TestOutlineCss(html_url, "", style_text, true, style_text);
+ }
+};
+
+// Tests for Outlining styles.
+TEST_F(CssOutlineFilterTest, OutlineStyle) {
+ OutlineStyle("outline_styles_no_hash", &mock_hasher_);
+}
+
+TEST_F(CssOutlineFilterTest, OutlineStyleMD5) {
+ OutlineStyle("outline_styles_md5", &md5_hasher_);
+}
+
+
+TEST_F(CssOutlineFilterTest, NoAbsolutifySameDir) {
+ const std::string css = "body { background-image: url('bg.png'); }";
+ TestOutlineCss("http://outline_style.test/index.html", "",
+ css, true, css);
+}
+
+TEST_F(CssOutlineFilterTest, AbsolutifyDifferentDir) {
+ const std::string css1 = "body { background-image: url('bg.png'); }";
+ const std::string css2 =
+ "body { background-image: url('http://other_site.test/foo/bg.png'); }";
+ TestOutlineCss("http://outline_style.test/index.html",
+ " <base href='http://other_site.test/foo/'>\n",
+ css1, true, css2);
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/css_tag_scanner.cc b/trunk/src/net/instaweb/rewriter/css_tag_scanner.cc
new file mode 100644
index 0000000..e8d08b0
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/css_tag_scanner.cc
@@ -0,0 +1,201 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/css_tag_scanner.h"
+
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include <string>
+#include "net/instaweb/util/public/writer.h"
+
+namespace {
+const char kStylesheet[] = "stylesheet";
+const char kTextCss[] = "text/css";
+}
+
+namespace net_instaweb {
+
+// Finds CSS files and calls another filter.
+CssTagScanner::CssTagScanner(HtmlParse* html_parse) {
+ s_link_ = html_parse->Intern("link");
+ s_href_ = html_parse->Intern("href");
+ s_type_ = html_parse->Intern("type");
+ s_rel_ = html_parse->Intern("rel");
+ s_media_ = html_parse->Intern("media");
+}
+
+// TODO(jmarantz): add test for this method to css_tag_scanner_test.cc
+bool CssTagScanner::ParseCssElement(
+ HtmlElement* element, HtmlElement::Attribute** href, const char** media) {
+ int num_required_attributes_found = 0;
+ *media = "";
+ *href = NULL;
+ if (element->tag() == s_link_) {
+ // We must have all attributes rel='stylesheet' href='name.css', and
+ // type='text/css', although they can be in any order. If there are,
+ // other attributes, we better learn about them so we don't lose them
+ // in css_combine_filter.cc.
+ int num_attrs = element->attribute_size();
+
+ // 'media=' is optional, but our filter requires href=*, and rel=stylesheet,
+ // and type=text/css.
+ //
+ // type should be "text/css", but if it's omitted, that's OK.
+ //
+ // TODO(jmarantz): Consider recognizing a wider variety of CSS references,
+ // including inline css so that the outline_filter can use it.
+ if ((num_attrs >= 2) || (num_attrs <= 4)) {
+ for (int i = 0; i < num_attrs; ++i) {
+ HtmlElement::Attribute& attr = element->attribute(i);
+ if (attr.name() == s_href_) {
+ *href = &attr;
+ ++num_required_attributes_found;
+ } else if (attr.name() == s_rel_) {
+ if (strcasecmp(attr.value(), kStylesheet) == 0) {
+ ++num_required_attributes_found;
+ } else {
+ // rel=something_else. abort.
+ num_required_attributes_found = 0;
+ break;
+ }
+ } else if (attr.name() == s_media_) {
+ *media = attr.value();
+ } else {
+ // The only other attribute we should see is type=text/css. This
+ // attribute is not required, but if the attribute we are
+ // finding here is anything else then abort.
+ if ((attr.name() != s_type_) ||
+ (strcasecmp(attr.value(), kTextCss) != 0)) {
+ num_required_attributes_found = 0;
+ break;
+ }
+ }
+ }
+ }
+ }
+
+ // we require both 'href=...' and 'rel=stylesheet'.
+ // TODO(jmarantz): warn when CSS elements aren't quite what we expect?
+ return (num_required_attributes_found >= 2);
+}
+
+namespace {
+
+bool ExtractQuote(std::string* url, char* quote) {
+ bool ret = false;
+ int size = url->size();
+ if (size > 2) {
+ *quote = (*url)[0];
+ if (((*quote == '\'') || (*quote == '"')) && (*quote == (*url)[size - 1])) {
+ ret = true;
+ *url = url->substr(1, size - 2);
+ }
+ }
+ return ret;
+}
+
+} // namespace
+
+// TODO(jmarantz): replace this scan-and-replace-in-one-shot methdology with
+// a proper scanner/parser/filtering mechanism akin to HtmlParse/HtmlLexer.
+// See http://www.w3.org/Style/CSS/SAC/ for the C Parser.
+bool CssTagScanner::AbsolutifyUrls(
+ const StringPiece& contents, const std::string& base_url,
+ Writer* writer, MessageHandler* handler) {
+ size_t pos = 0;
+ size_t prev_pos = 0;
+ bool ok = true;
+
+ // If the CSS url was specified with an absolute path, use that to
+ // absolutify any URLs referenced in the CSS text.
+ //
+ // TODO(jmarantz): Consider pasting in any CSS resources found in an import
+ // statement, rather than merely absolutifying in the references. This would
+ // require a few changes in this class API.
+ //
+ // TODO(jmarantz): Consider calling image optimization, if enabled, on any
+ // images found.
+ GURL base_gurl(base_url);
+ if (base_gurl.is_valid()) {
+ while (ok && ((pos = contents.find("url(", pos)) != StringPiece::npos)) {
+ ok = writer->Write(contents.substr(prev_pos, pos - prev_pos), handler);
+ prev_pos = pos;
+ pos += 4;
+ size_t end_of_url = contents.find(')', pos);
+ if ((end_of_url != StringPiece::npos) && (end_of_url != pos)) {
+ std::string url;
+ TrimWhitespace(contents.substr(pos, end_of_url - pos), &url);
+ char quote;
+ bool is_quoted = ExtractQuote(&url, "e);
+ std::string url_string(url.data(), url.size());
+ GURL gurl(url_string);
+
+ // Relative paths are considered invalid by GURL, and those are the
+ // ones we need to resolve.
+ if (!gurl.is_valid()) {
+ GURL resolved = base_gurl.Resolve(url_string.c_str());
+ if (resolved.is_valid()) {
+ ok = writer->Write("url(", handler);
+ if (is_quoted) {
+ writer->Write(StringPiece("e, 1), handler);
+ }
+ ok = writer->Write(resolved.spec().c_str(), handler);
+ if (is_quoted) {
+ writer->Write(StringPiece("e, 1), handler);
+ }
+ ok = writer->Write(")", handler);
+ prev_pos = end_of_url + 1;
+ } else {
+ int line = 1;
+ for (size_t i = 0; i < pos; ++i) {
+ line += (contents[i] == '\n');
+ }
+ handler->Error(
+ base_url.c_str(), line,
+ "CSS URL resolution failed: %s", url_string.c_str());
+ }
+ }
+ }
+ }
+ }
+ if (ok) {
+ ok = writer->Write(contents.substr(prev_pos), handler);
+ }
+ return ok;
+}
+
+bool CssTagScanner::HasImport(const StringPiece& contents,
+ MessageHandler* handler) {
+ // Search for case insensitive @import.
+ size_t pos = -1; // So that pos + 1 == 0 below.
+ static const char kImport[] = "import";
+ static const size_t kImportSize = sizeof(kImport) - 1;
+ while ((pos = contents.find("@", pos + 1)) != StringPiece::npos) {
+ // Rest is everything past the @ (non-inclusive).
+ StringPiece rest = contents.substr(pos + 1);
+ if (rest.size() >= kImportSize &&
+ (strncasecmp(kImport, rest.data(), kImportSize) == 0)) {
+ return true;
+ }
+ }
+ return false;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/css_tag_scanner_test.cc b/trunk/src/net/instaweb/rewriter/css_tag_scanner_test.cc
new file mode 100644
index 0000000..93b4a4e
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/css_tag_scanner_test.cc
@@ -0,0 +1,212 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the css filter
+
+#include "net/instaweb/rewriter/public/css_tag_scanner.h"
+
+#include <string>
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/string_writer.h"
+
+#include "net/instaweb/util/public/google_url.h"
+
+namespace net_instaweb {
+
+class CssTagScannerTest : public testing::Test {
+ protected:
+ CssTagScannerTest() : writer_(&output_buffer_) { }
+
+ void Check(const StringPiece& input, const StringPiece& expected) {
+ ASSERT_TRUE(CssTagScanner::AbsolutifyUrls(
+ input, "http://base/dir/styles.css", &writer_, &message_handler_));
+ EXPECT_EQ(expected, output_buffer_);
+ }
+
+ void CheckNoChange(const StringPiece& value) {
+ Check(value, value);
+ }
+
+ void CheckGurlResolve(const GURL& base, const char* relative_path,
+ const char* abs_path) {
+ GURL resolved = base.Resolve(relative_path);
+ EXPECT_TRUE(resolved.is_valid());
+ EXPECT_EQ(std::string(abs_path), resolved.spec());
+ }
+
+ std::string output_buffer_;
+ StringWriter writer_;
+ GoogleMessageHandler message_handler_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(CssTagScannerTest);
+};
+
+TEST_F(CssTagScannerTest, TestAbsolutifyUrlsEmpty) {
+ CheckNoChange("");
+}
+
+TEST_F(CssTagScannerTest, TestAbsolutifyUrlsNoMatch) {
+ CheckNoChange("hello");
+}
+
+TEST_F(CssTagScannerTest, TestAbsolutifyUrlsAbsolute) {
+ const char css_with_abs_path[] = "a url(http://other_base/image.png) b";
+ CheckNoChange(css_with_abs_path);
+}
+
+TEST_F(CssTagScannerTest, TestAbsolutifyUrlsAbsoluteSQuote) {
+ const char css_with_abs_path[] = "a url('http://other_base/image.png') b";
+ CheckNoChange(css_with_abs_path);
+}
+
+TEST_F(CssTagScannerTest, TestAbsolutifyUrlsAbsoluteDQuote) {
+ CheckNoChange("a url(\"http://other_base/image.png\") b");
+}
+
+TEST_F(CssTagScannerTest, TestAbsolutifyUrlsRelative) {
+ Check("a url(subdir/image.png) b",
+ "a url(http://base/dir/subdir/image.png) b");
+}
+
+TEST_F(CssTagScannerTest, TestAbsolutifyUrlsRelativeSQuote) {
+ Check("a url('subdir/image.png') b",
+ "a url('http://base/dir/subdir/image.png') b");
+}
+
+// Testcase for Issue 60.
+TEST_F(CssTagScannerTest, TestAbsolutifyUrlsRelativeSQuoteSpaced) {
+ Check("a url( 'subdir/image.png' ) b",
+ "a url('http://base/dir/subdir/image.png') b");
+}
+
+TEST_F(CssTagScannerTest, TestAbsolutifyUrlsRelativeDQuote) {
+ Check("a url(\"subdir/image.png\") b",
+ "a url(\"http://base/dir/subdir/image.png\") b");
+}
+
+TEST_F(CssTagScannerTest, TestAbsolutifyUrls2Relative1Abs) {
+ Check("a url(s/1.png) b url(2.png) c url(http://a/3.png) d",
+ "a url(http://base/dir/s/1.png) b url(http://base/dir/2.png) c "
+ "url(http://a/3.png) d");
+}
+
+// This test verifies that we understand how GURL::Resolve works.
+TEST_F(CssTagScannerTest, TestGurl) {
+ GURL base_slash("http://base/");
+ EXPECT_TRUE(base_slash.is_valid());
+ CheckGurlResolve(base_slash, "r/path.ext", "http://base/r/path.ext");
+ CheckGurlResolve(base_slash, "/r/path.ext", "http://base/r/path.ext");
+ CheckGurlResolve(base_slash, "../r/path.ext", "http://base/r/path.ext");
+ CheckGurlResolve(base_slash, "./r/path.ext", "http://base/r/path.ext");
+
+ GURL base_no_slash("http://base");
+ EXPECT_TRUE(base_no_slash.is_valid());
+ CheckGurlResolve(base_no_slash, "r/path.ext", "http://base/r/path.ext");
+ CheckGurlResolve(base_no_slash, "/r/path.ext", "http://base/r/path.ext");
+ CheckGurlResolve(base_no_slash, "../r/path.ext", "http://base/r/path.ext");
+ CheckGurlResolve(base_no_slash, "./r/path.ext", "http://base/r/path.ext");
+}
+
+// This test makes sure we can identify a few different forms of CSS tags we've
+// seen.
+TEST_F(CssTagScannerTest, TestFull) {
+ HtmlParse html_parse(&message_handler_);
+ Atom s_link = html_parse.Intern("link");
+ Atom s_href = html_parse.Intern("href");
+ Atom s_type = html_parse.Intern("type");
+ Atom s_rel = html_parse.Intern("rel");
+ Atom s_media = html_parse.Intern("media");
+ Atom s_other = html_parse.Intern("other");
+ HtmlElement* link = html_parse.NewElement(NULL, s_link);
+ const char kUrl[] = "http://www.myhost.com/static/mycss.css";
+ const char kPrint[] = "print";
+ link->AddAttribute(s_rel, "stylesheet", "\"");
+ link->AddAttribute(s_href, kUrl, "\"");
+ HtmlElement::Attribute* href = NULL;
+ const char* media = NULL;
+ CssTagScanner scanner(&html_parse);
+
+ // We can parse css even lacking a 'type' attribute. Default to text/css.
+ EXPECT_TRUE(scanner.ParseCssElement(link, &href, &media));
+ EXPECT_EQ("", std::string(media));
+ EXPECT_EQ(kUrl, std::string(href->value()));
+
+ // Add an unexpected attribute. Now we don't know what to do with it.
+ link->AddAttribute(s_other, "value", "\"");
+ EXPECT_FALSE(scanner.ParseCssElement(link, &href, &media));
+
+ // Mutate it to the correct attribute.
+ HtmlElement::Attribute* attr = link->FindAttribute(s_other);
+ ASSERT_TRUE(attr != NULL);
+ attr->set_name(s_type);
+ attr->SetValue("text/css");
+ EXPECT_TRUE(scanner.ParseCssElement(link, &href, &media));
+ EXPECT_EQ("", std::string(media));
+ EXPECT_EQ(kUrl, std::string(href->value()));
+
+ // Add a media attribute. It should still pass, yielding media.
+ link->AddAttribute(s_media, kPrint, "\"");
+ EXPECT_TRUE(scanner.ParseCssElement(link, &href, &media));
+ EXPECT_EQ(kPrint, std::string(media));
+ EXPECT_EQ(kUrl, std::string(href->value()));
+
+ // TODO(jmarantz): test removal of 'rel' and 'href' attributes
+}
+
+TEST_F(CssTagScannerTest, TestHasImport) {
+ // Should work.
+ EXPECT_TRUE(CssTagScanner::HasImport("@import", &message_handler_));
+ EXPECT_TRUE(CssTagScanner::HasImport("@Import", &message_handler_));
+ EXPECT_TRUE(CssTagScanner::HasImport(
+ "@charset 'iso-8859-1';\n"
+ "@import url('http://foo.com');\n", &message_handler_));
+ EXPECT_TRUE(CssTagScanner::HasImport(
+ "@charset 'iso-8859-1';\n"
+ "@iMPorT url('http://foo.com');\n", &message_handler_));
+
+ // Should fail.
+ EXPECT_FALSE(CssTagScanner::HasImport("", &message_handler_));
+ EXPECT_FALSE(CssTagScanner::HasImport("@impor", &message_handler_));
+ EXPECT_FALSE(CssTagScanner::HasImport(
+ "@charset 'iso-8859-1';\n"
+ "@impor", &message_handler_));
+ // Make sure we aren't overflowing the buffer.
+ std::string import_string = "@import";
+ StringPiece truncated_import(import_string.data(), import_string.size() - 1);
+ EXPECT_FALSE(CssTagScanner::HasImport(truncated_import, &message_handler_));
+
+ // False positives.
+ EXPECT_TRUE(CssTagScanner::HasImport(
+ "@charset 'iso-8859-1';\n"
+ "@importinvalid url('http://foo.com');\n", &message_handler_));
+ EXPECT_TRUE(CssTagScanner::HasImport(
+ "@charset 'iso-8859-1';\n"
+ "/* @import url('http://foo.com'); */\n", &message_handler_));
+ EXPECT_TRUE(CssTagScanner::HasImport(
+ "@charset 'iso-8859-1';\n"
+ "a { color: pink; }\n"
+ "/* @import after rulesets is invalid */\n"
+ "@import url('http://foo.com');\n", &message_handler_));
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/data_url_input_resource.cc b/trunk/src/net/instaweb/rewriter/data_url_input_resource.cc
new file mode 100644
index 0000000..484c5d7
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/data_url_input_resource.cc
@@ -0,0 +1,42 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/rewriter/public/data_url_input_resource.h"
+
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/util/public/data_url.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+
+namespace net_instaweb {
+
+bool DataUrlInputResource::Load(MessageHandler* message_handler) {
+ if (DecodeDataUrlContent(encoding_, encoded_contents_,
+ &decoded_contents_) &&
+ value_.Write(decoded_contents_, message_handler)) {
+ resource_manager_->SetDefaultHeaders(type_, &meta_data_);
+ value_.SetHeaders(meta_data_);
+ }
+ return loaded();
+}
+
+bool DataUrlInputResource::IsCacheable() const {
+ return false;
+}
+
+}
diff --git a/trunk/src/net/instaweb/rewriter/domain_lawyer.cc b/trunk/src/net/instaweb/rewriter/domain_lawyer.cc
new file mode 100644
index 0000000..6874c19
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/domain_lawyer.cc
@@ -0,0 +1,286 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/domain_lawyer.h"
+
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/stl_util.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/wildcard.h"
+
+namespace net_instaweb {
+
+class DomainLawyer::Domain {
+ public:
+ explicit Domain(const StringPiece& name)
+ : wildcard_(name),
+ name_(name.data(), name.size()),
+ num_shards_(0),
+ rewrite_domain_(NULL),
+ origin_domain_(NULL) {
+ }
+
+ bool IsWildcarded() const { return !wildcard_.IsSimple(); }
+ bool Match(const StringPiece& domain) { return wildcard_.Match(domain); }
+ Domain* rewrite_domain() const { return rewrite_domain_; }
+ Domain* origin_domain() const { return origin_domain_; }
+ StringPiece name() const { return name_; }
+
+ // TODO(jmarantz): check for cycles
+ void set_rewrite_domain(Domain* x) { rewrite_domain_ = x; }
+ void set_origin_domain(Domain* x) { origin_domain_ = x; }
+
+ private:
+ Wildcard wildcard_;
+ std::string name_;
+ int num_shards_;
+
+ // The rewrite_domain, if non-null, gives the location of where this
+ // Domain should be rewritten. This can be used to move resources onto
+ // a CDN or onto a cookieless domain.
+ Domain* rewrite_domain_;
+
+ // The origin_domain, if non-null, gives the location of where
+ // resources should be fetched from by mod_pagespeed, in lieu of how
+ // it is specified in the HTML. This allows, for example, a CDN to
+ // fetch content from an origin domain, or an origin server behind a
+ // load-balancer to specify localhost or an IP address of a host to
+ // go to directly, skipping DNS resolution and reducing outbound
+ // traffic.
+ Domain* origin_domain_;
+};
+
+DomainLawyer::~DomainLawyer() {
+ STLDeleteValues(&domain_map_);
+}
+
+bool DomainLawyer::AddDomain(const StringPiece& domain_name,
+ MessageHandler* handler) {
+ return (AddDomainHelper(domain_name, true, handler) != NULL);
+}
+
+DomainLawyer::Domain* DomainLawyer::AddDomainHelper(
+ const StringPiece& domain_name, bool warn_on_duplicate,
+ MessageHandler* handler) {
+ if (domain_name.empty()) {
+ // handler will be NULL only when called from Merge, which should
+ // only have pre-validated (non-empty) domains. So it should not
+ // be possible to get here from Merge.
+ if (handler != NULL) {
+ handler->Message(kWarning, "Empty domain passed to AddDomain");
+ }
+ return NULL;
+ }
+
+ // Ensure that the following specifications are treated identically:
+ // www.google.com
+ // http://www.google.com
+ // www.google.com/
+ // http://www.google.com/
+ // all come out the same.
+ std::string domain_name_str;
+ if (domain_name.find("://") == std::string::npos) {
+ domain_name_str = StrCat("http://", domain_name);
+ } else {
+ domain_name.CopyToString(&domain_name_str);
+ }
+ EnsureEndsInSlash(&domain_name_str);
+ Domain* domain = NULL;
+ std::pair<DomainMap::iterator, bool> p = domain_map_.insert(
+ DomainMap::value_type(domain_name_str, domain));
+ DomainMap::iterator iter = p.first;
+ if (p.second) {
+ domain = new Domain(domain_name_str);
+ iter->second = domain;
+ if (domain->IsWildcarded()) {
+ wildcarded_domains_.push_back(domain);
+ }
+ iter->second = domain;
+ } else if (warn_on_duplicate) {
+ handler->Message(kWarning, "AddDomain of domain already in map: %s",
+ domain_name_str.c_str());
+ } else {
+ domain = iter->second;
+ }
+ return domain;
+}
+
+// Looks up the Domain* object by name. From the Domain object
+// we can tell if it's wildcarded, in which case it cannot be
+// the 'to' field for a map, and whether resources from it should
+// be mapped to a different domain, either for rewriting or for
+// fetching.
+DomainLawyer::Domain* DomainLawyer::FindDomain(
+ const std::string& domain_name) const {
+ DomainMap::const_iterator p = domain_map_.find(domain_name);
+ Domain* domain = NULL;
+ if (p != domain_map_.end()) {
+ domain = p->second;
+ } else {
+ // TODO(jmarantz): use a better lookup structure for this
+ for (int i = 0, n = wildcarded_domains_.size(); i < n; ++i) {
+ domain = wildcarded_domains_[i];
+ if (domain->Match(domain_name)) {
+ break;
+ } else {
+ domain = NULL;
+ }
+ }
+ }
+ return domain;
+}
+
+bool DomainLawyer::MapRequestToDomain(
+ const GURL& original_request,
+ const StringPiece& resource_url, // relative to original_request
+ std::string* mapped_domain_name,
+ GURL* resolved_request,
+ MessageHandler* handler) const {
+ CHECK(original_request.is_valid());
+ GURL original_origin = original_request.GetOrigin();
+ *resolved_request = GoogleUrl::Resolve(original_request, resource_url);
+ bool ret = false;
+ // At present we're not sure about appropriate resource
+ // policies for https: etc., so we only permit http resources
+ // to be rewritten.
+ // TODO(jmaessen): Figure out if this is appropriate.
+ if (resolved_request->is_valid() && resolved_request->SchemeIs("http")) {
+ GURL resolved_origin = resolved_request->GetOrigin();
+ std::string resolved_domain_name = GoogleUrl::Spec(resolved_origin);
+
+ // Looks at the resovled domain name from the original request and
+ // the resource_url (which might override the original request).
+ // Gets the Domain* object out of that.
+ Domain* resolved_domain = FindDomain(resolved_domain_name);
+
+ // The origin domain is authorized by default.
+ if ((resolved_origin == original_origin) || (resolved_domain != NULL)) {
+ *mapped_domain_name = resolved_domain_name;
+ ret = true;
+
+ // If we actually got a Domain* out of the lookups so far, then a
+ // mapping to a different rewrite_domain may be contained there. This
+ // helps move resources to CDNs or cookieless domains.
+ //
+ // Note that at this point, we are not really caring where we fetch
+ // from. We are only concerned here with what URLs we will write into
+ // HTML files. See MapOrigin below which is used to redirect fetch
+ // requests to a different domain (e.g. localhost).
+ if (resolved_domain != NULL) {
+ Domain* mapped_domain = resolved_domain->rewrite_domain();
+ if (mapped_domain != NULL) {
+ CHECK(!mapped_domain->IsWildcarded());
+ mapped_domain->name().CopyToString(mapped_domain_name);
+ *resolved_request = GoogleUrl::Create(*mapped_domain_name).Resolve(
+ GoogleUrl::PathAndLeaf(*resolved_request));
+ }
+ }
+ }
+ }
+ return ret;
+}
+
+bool DomainLawyer::MapOrigin(const StringPiece& in, std::string* out) const {
+ bool ret = false;
+ GURL gurl = GoogleUrl::Create(in);
+ // At present we're not sure about appropriate resource
+ // policies for https: etc., so we only permit http resources
+ // to be rewritten.
+ if (gurl.is_valid() && gurl.SchemeIs("http")) {
+ ret = true;
+ in.CopyToString(out);
+ GURL origin = gurl.GetOrigin();
+ std::string origin_name = GoogleUrl::Spec(origin);
+ Domain* domain = FindDomain(origin_name);
+ if (domain != NULL) {
+ Domain* origin_domain = domain->origin_domain();
+ if (origin_domain != NULL) {
+ CHECK(!origin_domain->IsWildcarded());
+ GURL mapped_gurl = GoogleUrl::Create(origin_domain->name()).Resolve(
+ GoogleUrl::PathAndLeaf(gurl));
+ if (mapped_gurl.is_valid()) {
+ *out = GoogleUrl::Spec(mapped_gurl);
+ }
+ }
+ }
+ }
+ return ret;
+}
+
+bool DomainLawyer::AddRewriteDomainMapping(
+ const StringPiece& to_domain_name,
+ const StringPiece& comma_separated_from_domains,
+ MessageHandler* handler) {
+ return MapDomainHelper(to_domain_name, comma_separated_from_domains,
+ &Domain::set_rewrite_domain, handler);
+}
+
+bool DomainLawyer::AddOriginDomainMapping(
+ const StringPiece& to_domain_name,
+ const StringPiece& comma_separated_from_domains,
+ MessageHandler* handler) {
+ return MapDomainHelper(to_domain_name, comma_separated_from_domains,
+ &Domain::set_origin_domain, handler);
+}
+
+bool DomainLawyer::MapDomainHelper(
+ const StringPiece& to_domain_name,
+ const StringPiece& comma_separated_from_domains,
+ SetDomainFn set_domain_fn,
+ MessageHandler* handler) {
+ Domain* to_domain = AddDomainHelper(to_domain_name, false, handler);
+ bool ret = false;
+ if (to_domain->IsWildcarded()) {
+ handler->Message(kError, "Cannot map to a wildcarded domain: %s",
+ to_domain_name.as_string().c_str());
+ } else if (to_domain != NULL) {
+ std::vector<StringPiece> domains;
+ SplitStringPieceToVector(comma_separated_from_domains, ",", &domains, true);
+ for (int i = 0, n = domains.size(); i < n; ++i) {
+ const StringPiece& domain_name = domains[i];
+ Domain* from_domain = AddDomainHelper(domain_name, false, handler);
+ if (from_domain != NULL) {
+ (from_domain->*set_domain_fn)(to_domain);
+ ret = true;
+ }
+ }
+ }
+ return ret;
+}
+
+void DomainLawyer::Merge(const DomainLawyer& src) {
+ for (DomainMap::const_iterator
+ p = src.domain_map_.begin(),
+ e = src.domain_map_.end();
+ p != e; ++p) {
+ Domain* src_domain = p->second;
+ Domain* dst_domain = AddDomainHelper(src_domain->name(), false, NULL);
+ Domain* src_rewrite_domain = src_domain->rewrite_domain();
+ if (src_rewrite_domain != NULL) {
+ dst_domain->set_rewrite_domain(
+ AddDomainHelper(src_rewrite_domain->name(), false, NULL));
+ }
+ Domain* src_origin_domain = src_domain->origin_domain();
+ if (src_origin_domain != NULL) {
+ dst_domain->set_origin_domain(
+ AddDomainHelper(src_origin_domain->name(), false, NULL));
+ }
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/domain_lawyer_test.cc b/trunk/src/net/instaweb/rewriter/domain_lawyer_test.cc
new file mode 100644
index 0000000..87ed189
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/domain_lawyer_test.cc
@@ -0,0 +1,337 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/domain_lawyer.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace {
+
+const char kResourceUrl[] = "styles/style.css?appearance=reader";
+const char kCdnPrefix[] = "http://graphics8.nytimes.com/";
+const char kRequestDomain[] = "http://www.nytimes.com/";
+const char kRequestDomainPort[] = "http://www.nytimes.com:8080/";
+
+} // namespace
+
+namespace net_instaweb {
+
+class DomainLawyerTest : public testing::Test {
+ protected:
+ DomainLawyerTest()
+ : orig_request_("http://www.nytimes.com/index.html"),
+ port_request_("http://www.nytimes.com:8080/index.html"),
+ https_request_("https://www.nytimes.com/index.html") {
+ }
+
+ // Syntactic sugar to map a request.
+ bool MapRequest(const GURL& original_request,
+ const StringPiece& resource_url,
+ std::string* mapped_domain_name) {
+ GURL resolved_request;
+ return domain_lawyer_.MapRequestToDomain(
+ original_request, resource_url, mapped_domain_name, &resolved_request,
+ &message_handler_);
+ }
+
+ GURL orig_request_;
+ GURL port_request_;
+ GURL https_request_;
+ DomainLawyer domain_lawyer_;
+ GoogleMessageHandler message_handler_;
+};
+
+TEST_F(DomainLawyerTest, RelativeDomain) {
+ std::string mapped_domain_name;
+ ASSERT_TRUE(MapRequest(
+ orig_request_, kResourceUrl, &mapped_domain_name));
+ EXPECT_EQ(kRequestDomain, mapped_domain_name);
+}
+
+TEST_F(DomainLawyerTest, AbsoluteDomain) {
+ std::string mapped_domain_name;
+ ASSERT_TRUE(MapRequest(
+ orig_request_, StrCat(kRequestDomain, kResourceUrl),
+ &mapped_domain_name));
+ EXPECT_EQ(kRequestDomain, mapped_domain_name);
+}
+
+TEST_F(DomainLawyerTest, ExternalDomainNotDeclared) {
+ std::string mapped_domain_name;
+ EXPECT_FALSE(MapRequest(
+ orig_request_, StrCat(kCdnPrefix, kResourceUrl), &mapped_domain_name));
+}
+
+TEST_F(DomainLawyerTest, ExternalDomainDeclared) {
+ StringPiece cdn_domain(kCdnPrefix, sizeof(kCdnPrefix) - 1);
+ ASSERT_TRUE(domain_lawyer_.AddDomain(cdn_domain, &message_handler_));
+ std::string mapped_domain_name;
+ ASSERT_TRUE(MapRequest(
+ orig_request_, StrCat(kCdnPrefix, kResourceUrl), &mapped_domain_name));
+ EXPECT_EQ(cdn_domain, mapped_domain_name);
+
+ // Make sure that we do not allow requests when the port is present; we've
+ // only authorized origin "http://www.nytimes.com/",
+ // not "http://www.nytimes.com:8080/
+ std::string orig_cdn_domain(kCdnPrefix, sizeof(kCdnPrefix) - 2);
+ std::string port_cdn_domain(cdn_domain.data(), cdn_domain.size() - 1);
+ port_cdn_domain += ":8080/";
+ EXPECT_FALSE(MapRequest(
+ orig_request_, StrCat(port_cdn_domain, "/", kResourceUrl),
+ &mapped_domain_name));
+}
+
+TEST_F(DomainLawyerTest, ExternalDomainDeclaredWithoutScheme) {
+ StringPiece cdn_domain(kCdnPrefix, sizeof(kCdnPrefix) - 1);
+ ASSERT_TRUE(domain_lawyer_.AddDomain(kCdnPrefix + strlen("http://"),
+ &message_handler_));
+ std::string mapped_domain_name;
+ ASSERT_TRUE(MapRequest(
+ orig_request_, StrCat(kCdnPrefix, kResourceUrl), &mapped_domain_name));
+ EXPECT_EQ(cdn_domain, mapped_domain_name);
+}
+
+TEST_F(DomainLawyerTest, ExternalDomainDeclaredWithoutTrailingSlash) {
+ StringPiece cdn_domain(kCdnPrefix, sizeof(kCdnPrefix) - 1);
+ StringPiece cdn_domain_no_slash(kCdnPrefix, sizeof(kCdnPrefix) - 2);
+ ASSERT_TRUE(domain_lawyer_.AddDomain(cdn_domain_no_slash, &message_handler_));
+ std::string mapped_domain_name;
+ ASSERT_TRUE(MapRequest(
+ orig_request_, StrCat(kCdnPrefix, kResourceUrl), &mapped_domain_name));
+ EXPECT_EQ(cdn_domain, mapped_domain_name);
+}
+
+TEST_F(DomainLawyerTest, WildcardDomainDeclared) {
+ StringPiece cdn_domain(kCdnPrefix, sizeof(kCdnPrefix) - 1);
+ ASSERT_TRUE(domain_lawyer_.AddDomain("*.nytimes.com", &message_handler_));
+ std::string mapped_domain_name;
+ ASSERT_TRUE(MapRequest(
+ orig_request_, StrCat(kCdnPrefix, kResourceUrl), &mapped_domain_name));
+ EXPECT_EQ(cdn_domain, mapped_domain_name);
+}
+
+TEST_F(DomainLawyerTest, RelativeDomainPort) {
+ std::string mapped_domain_name;
+ ASSERT_TRUE(MapRequest(
+ port_request_, kResourceUrl, &mapped_domain_name));
+ EXPECT_EQ(kRequestDomainPort, mapped_domain_name);
+}
+
+TEST_F(DomainLawyerTest, AbsoluteDomainPort) {
+ std::string mapped_domain_name;
+ ASSERT_TRUE(MapRequest(
+ port_request_, StrCat(kRequestDomainPort, kResourceUrl),
+ &mapped_domain_name));
+ EXPECT_EQ(kRequestDomainPort, mapped_domain_name);
+}
+
+TEST_F(DomainLawyerTest, PortExternalDomainNotDeclared) {
+ std::string mapped_domain_name;
+ EXPECT_FALSE(MapRequest(
+ port_request_, StrCat(kCdnPrefix, kResourceUrl), &mapped_domain_name));
+}
+
+TEST_F(DomainLawyerTest, PortExternalDomainDeclared) {
+ std::string port_cdn_domain(kCdnPrefix, sizeof(kCdnPrefix) - 2);
+ port_cdn_domain += ":8080/";
+ ASSERT_TRUE(domain_lawyer_.AddDomain(port_cdn_domain, &message_handler_));
+ std::string mapped_domain_name;
+ ASSERT_TRUE(MapRequest(
+ port_request_, StrCat(port_cdn_domain, kResourceUrl),
+ &mapped_domain_name));
+ EXPECT_EQ(port_cdn_domain, mapped_domain_name);
+
+ // Make sure that we do not allow requests when the port is missing; we've
+ // only authorized origin "http://www.nytimes.com:8080/",
+ // not "http://www.nytimes.com:8080
+ std::string orig_cdn_domain(kCdnPrefix, sizeof(kCdnPrefix) - 2);
+ orig_cdn_domain += "/";
+ EXPECT_FALSE(MapRequest(
+ port_request_, StrCat(orig_cdn_domain, kResourceUrl),
+ &mapped_domain_name));
+}
+
+TEST_F(DomainLawyerTest, PortWildcardDomainDeclared) {
+ std::string port_cdn_domain(kCdnPrefix, sizeof(kCdnPrefix) - 2);
+ port_cdn_domain += ":8080/";
+ ASSERT_TRUE(domain_lawyer_.AddDomain("*.nytimes.com:*", &message_handler_));
+ std::string mapped_domain_name;
+ ASSERT_TRUE(MapRequest(
+ port_request_, StrCat(port_cdn_domain, kResourceUrl),
+ &mapped_domain_name));
+ EXPECT_EQ(port_cdn_domain, mapped_domain_name);
+}
+
+TEST_F(DomainLawyerTest, ResourceFromHttpsPage) {
+ ASSERT_TRUE(domain_lawyer_.AddDomain("www.nytimes.com", &message_handler_));
+ std::string mapped_domain_name;
+
+ // When a relative resource is requested from an https page we will fail.
+ ASSERT_FALSE(MapRequest(
+ https_request_, kResourceUrl,
+ &mapped_domain_name));
+ ASSERT_TRUE(MapRequest(
+ https_request_, StrCat(kRequestDomain, kResourceUrl),
+ &mapped_domain_name));
+}
+
+TEST_F(DomainLawyerTest, AddDomainRedundantly) {
+ ASSERT_TRUE(domain_lawyer_.AddDomain("www.nytimes.com", &message_handler_));
+ ASSERT_FALSE(domain_lawyer_.AddDomain("www.nytimes.com", &message_handler_));
+ ASSERT_TRUE(domain_lawyer_.AddDomain("*", &message_handler_));
+ ASSERT_FALSE(domain_lawyer_.AddDomain("*", &message_handler_));
+}
+
+TEST_F(DomainLawyerTest, VerifyPortIsDistinct1) {
+ ASSERT_TRUE(domain_lawyer_.AddDomain("www.example.com", &message_handler_));
+ std::string mapped_domain_name;
+ EXPECT_FALSE(MapRequest(
+ GURL("http://www.other.com/index.html"),
+ "http://www.example.com:81/styles.css",
+ &mapped_domain_name));
+}
+
+TEST_F(DomainLawyerTest, VerifyPortIsDistinct2) {
+ ASSERT_TRUE(domain_lawyer_.AddDomain("www.example.com:81", &message_handler_));
+ std::string mapped_domain_name;
+ EXPECT_FALSE(MapRequest(
+ GURL("http://www.other.com/index.html"),
+ "http://www.example.com/styles.css",
+ &mapped_domain_name));
+}
+
+TEST_F(DomainLawyerTest, VerifyWildcardedPortSpec) {
+ ASSERT_TRUE(domain_lawyer_.AddDomain("www.example.com*", &message_handler_));
+ std::string mapped_domain_name;
+ EXPECT_TRUE(MapRequest(
+ GURL("http://www.other.com/index.html"),
+ "http://www.example.com/styles.css",
+ &mapped_domain_name));
+ EXPECT_TRUE(MapRequest(
+ GURL("http://www.other.com/index.html"),
+ "http://www.example.com:81/styles.css",
+ &mapped_domain_name));
+}
+
+TEST_F(DomainLawyerTest, MapRewriteDomain) {
+ ASSERT_TRUE(domain_lawyer_.AddDomain("http://cdn.com/", &message_handler_));
+ ASSERT_TRUE(domain_lawyer_.AddDomain("http://origin.com/",
+ &message_handler_));
+ ASSERT_TRUE(domain_lawyer_.AddRewriteDomainMapping("http://cdn.com",
+ "http://origin.com",
+ &message_handler_));
+ // First try the mapping from origin.com to cdn.com
+ std::string mapped_domain_name;
+ ASSERT_TRUE(MapRequest(
+ GoogleUrl::Create(StringPiece("http://www.origin.com/index.html")),
+ "http://origin.com/styles/blue.css",
+ &mapped_domain_name));
+ EXPECT_EQ("http://cdn.com/", mapped_domain_name);
+
+ // But a relative reference will not map because we mapped origin.com,
+ // not www.origin.com
+ ASSERT_TRUE(MapRequest(
+ GoogleUrl::Create(StringPiece("http://www.origin.com/index.html")),
+ "styles/blue.css",
+ &mapped_domain_name));
+ EXPECT_EQ("http://www.origin.com/", mapped_domain_name);
+
+ // Now add the mapping from www.
+ ASSERT_TRUE(domain_lawyer_.AddRewriteDomainMapping("http://cdn.com",
+ "http://www.origin.com",
+ &message_handler_));
+
+ ASSERT_TRUE(MapRequest(
+ GoogleUrl::Create(StringPiece("http://www.origin.com/index.html")),
+ "styles/blue.css",
+ &mapped_domain_name));
+ EXPECT_EQ("http://cdn.com/", mapped_domain_name);
+}
+
+TEST_F(DomainLawyerTest, MapOriginDomain) {
+ ASSERT_TRUE(domain_lawyer_.AddOriginDomainMapping(
+ "http://localhost:8080", "http://origin.com:8080", &message_handler_));
+ std::string mapped;
+ ASSERT_TRUE(domain_lawyer_.MapOrigin("http://origin.com:8080/a/b/c?d=f",
+ &mapped));
+ EXPECT_EQ("http://localhost:8080/a/b/c?d=f", mapped);
+}
+
+TEST_F(DomainLawyerTest, Merge) {
+ // Add some mappings for domain_lawywer_.
+ ASSERT_TRUE(domain_lawyer_.AddDomain("http://d1.com/", &message_handler_));
+ ASSERT_TRUE(domain_lawyer_.AddRewriteDomainMapping(
+ "http://cdn1.com", "http://www.o1.com", &message_handler_));
+ ASSERT_TRUE(domain_lawyer_.AddOriginDomainMapping(
+ "http://localhost:8080", "http://o1.com:8080", &message_handler_));
+
+ // We'll also a mapping that will conflict, and one that won't
+ ASSERT_TRUE(domain_lawyer_.AddOriginDomainMapping(
+ "http://dest1/", "http://common_src1", &message_handler_));
+ ASSERT_TRUE(domain_lawyer_.AddOriginDomainMapping(
+ "http://dest2/", "http://common_src2", &message_handler_));
+
+ // Now add a similar set of mappings for another lawyer.
+ DomainLawyer merged;
+ ASSERT_TRUE(merged.AddDomain("http://d2.com/", &message_handler_));
+ ASSERT_TRUE(merged.AddRewriteDomainMapping(
+ "http://cdn2.com", "http://www.o2.com", &message_handler_));
+ ASSERT_TRUE(merged.AddOriginDomainMapping(
+ "http://localhost:8080", "http://o2.com:8080", &message_handler_));
+
+ // Here's a different mapping for the same source.
+ ASSERT_TRUE(merged.AddOriginDomainMapping(
+ "http://dest3/", "http://common_src1", &message_handler_));
+ ASSERT_TRUE(domain_lawyer_.AddOriginDomainMapping(
+ "http://dest4/", "http://common_src3", &message_handler_));
+
+ merged.Merge(domain_lawyer_);
+
+ // Now the tests for both domains should work post-merger.
+
+ std::string mapped;
+ GURL resolved_request;
+ ASSERT_TRUE(merged.MapRequestToDomain(
+ GoogleUrl::Create(StringPiece("http://www.o1.com/index.html")),
+ "styles/blue.css", &mapped, &resolved_request, &message_handler_));
+ EXPECT_EQ("http://cdn1.com/", mapped);
+ ASSERT_TRUE(merged.MapRequestToDomain(
+ GoogleUrl::Create(StringPiece("http://www.o2.com/index.html")),
+ "styles/blue.css", &mapped, &resolved_request, &message_handler_));
+ EXPECT_EQ("http://cdn2.com/", mapped);
+
+ ASSERT_TRUE(merged.MapOrigin("http://o1.com:8080/a/b/c?d=f", &mapped));
+ EXPECT_EQ("http://localhost:8080/a/b/c?d=f", mapped);
+ ASSERT_TRUE(merged.MapOrigin("http://o2.com:8080/a/b/c?d=f", &mapped));
+ EXPECT_EQ("http://localhost:8080/a/b/c?d=f", mapped);
+
+ // The conflict will be silently resolved to prefer the mapping from
+ // the domain that got merged, which is domain_laywer_1, overriding
+ // what was previously in the target.
+ ASSERT_TRUE(merged.MapOrigin("http://common_src1", &mapped));
+ EXPECT_EQ("http://dest1/", mapped);
+
+ // Now check the domains that were added.
+ ASSERT_TRUE(merged.MapOrigin("http://common_src2", &mapped));
+ EXPECT_EQ("http://dest2/", mapped);
+
+ ASSERT_TRUE(merged.MapOrigin("http://common_src3", &mapped));
+ EXPECT_EQ("http://dest4/", mapped);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/elide_attributes_filter.cc b/trunk/src/net/instaweb/rewriter/elide_attributes_filter.cc
new file mode 100644
index 0000000..c8b6372
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/elide_attributes_filter.cc
@@ -0,0 +1,265 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "net/instaweb/rewriter/public/elide_attributes_filter.h"
+
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_node.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace {
+
+// An attribute can be simplified if it is a "boolean attribute".
+// See http://www.w3.org/TR/html5/common-microsyntaxes.html#boolean-attribute
+// For example, <option selected="selected"> can become <option selected>.
+
+struct TagAttr {
+ const char* tag_name;
+ const char* attr_name;
+};
+
+const TagAttr kBooleanAttrs[] = {
+ // http://www.w3.org/TR/html4/struct/objects.html#h-13.6.1
+ {"area", "nohref"},
+ // http://www.w3.org/TR/html5/video.html#media-elements
+ {"audio", "autoplay"},
+ {"audio", "controls"},
+ {"audio", "loop"},
+ {"audio", "muted"},
+ // http://www.w3.org/TR/html5/the-button-element.html#the-button-element
+ {"button", "autofocus"},
+ {"button", "disabled"},
+ // http://www.w3.org/TR/html5/interactive-elements.html#the-command
+ {"command", "checked"},
+ {"command", "disabled"},
+ // http://www.w3.org/TR/html5/interactive-elements.html#the-details-element
+ {"details", "open"},
+ // http://www.w3.org/TR/html5/association-of-controls-and-forms.html#
+ // attributes-for-form-submission
+ {"form", "novalidate"},
+ // http://www.w3.org/TR/html4/present/frames.html#h-16.2.2
+ {"frame", "noresize"},
+ // http://www.w3.org/TR/html5/the-button-element.html#the-keygen-element
+ {"keygen", "autofocus"},
+ {"keygen", "disabled"},
+ // http://www.w3.org/TR/html5/the-iframe-element.html#the-iframe-element
+ {"iframe", "seamless"},
+ // http://www.w3.org/TR/html5/embedded-content-1.html#the-img-element
+ {"img", "ismap"},
+ // http://www.w3.org/TR/html5/the-input-element.html#the-input-element
+ {"input", "autofocus"},
+ {"input", "checked"},
+ {"input", "defaultchecked"},
+ {"input", "disabled"},
+ {"input", "formnovalidate"},
+ {"input", "indeterminate"},
+ {"input", "multiple"},
+ {"input", "readonly"},
+ {"input", "required"},
+ // http://www.w3.org/TR/html4/struct/objects.html#h-13.3
+ {"object", "declare"},
+ // http://www.w3.org/TR/html5/grouping-content.html#the-ol-element
+ {"ol", "reversed"},
+ // http://www.w3.org/TR/html5/the-button-element.html#the-optgroup-element
+ {"optgroup", "disabled"},
+ // http://www.w3.org/TR/html5/the-button-element.html#the-option-element
+ {"option", "defaultselected"},
+ {"option", "disabled"},
+ {"option", "selected"},
+ // http://www.w3.org/TR/html5/scripting-1.html#script
+ {"script", "async"},
+ {"script", "defer"},
+ // http://www.w3.org/TR/html5/the-button-element.html#the-select-element
+ {"select", "autofocus"},
+ {"select", "disabled"},
+ {"select", "multiple"},
+ {"select", "required"},
+ // http://www.w3.org/TR/html5/semantics.html#the-style-element
+ {"style", "scoped"},
+ // http://www.w3.org/TR/html5/the-button-element.html#the-textarea-element
+ {"textarea", "autofocus"},
+ {"textarea", "disabled"},
+ {"textarea", "readonly"},
+ {"textarea", "required"},
+ // http://www.w3.org/TR/html5/video.html#media-elements
+ {"video", "autoplay"},
+ {"video", "controls"},
+ {"video", "loop"},
+ {"video", "muted"},
+};
+
+// An attribute can be removed from a tag if its name and value is in
+// kDefaultList.
+//
+// Note: It is important that this list not include attributes that can be
+// inherited. Otherwise something like this could fail:
+// <div attr="non_default_value">
+// <div attr="default_value"> <!-- must not be elided -->
+// </div>
+// </div>
+
+struct TagAttrValue {
+ const char* tag_name;
+ const char* attr_name;
+ const char* attr_value;
+ bool requires_version_5; // Default value only exists in (X)HTML 5.
+};
+
+// References for HTML 4 and HTML 5 are included below, with extra notes for
+// entries that apply differently to HTML 4 and HTML 5 (i.e. those with 4th
+// argument true). If you are so inclined, you are encouraged to carefully
+// verify the references and make changes to any errors in this data.
+const TagAttrValue kDefaultList[] = {
+ // 4: http://www.w3.org/TR/html4/struct/links.html#h-12.2
+ // 5: Note that the <a> tag's shape attribute is deprecated in HTML5.
+ // http://www.w3.org/TR/html5/obsolete.html#non-conforming-features
+ {"a", "shape", "rect", false},
+ // 4: http://www.w3.org/TR/html4/struct/objects.html#h-13.6.1
+ // 5: http://www.w3.org/TR/html5/the-map-element.html#the-area-element
+ {"area", "shape", "rect", false},
+ // 4: http://www.w3.org/TR/html4/interact/forms.html#h-17.5
+ // 5: http://www.w3.org/TR/html5/the-button-element.html#the-button-element
+ {"button", "type", "submit", false},
+ // 4: The <command> tag does not exist in HTML 4.
+ // 5: http://www.w3.org/TR/html5/interactive-elements.html#the-command
+ {"command", "type", "command", true},
+ // 4: The <form> tag's autocomplete attribute does not exist in HTML 4.
+ // 5: http://www.w3.org/TR/html5/forms.html#the-form-element
+ {"form", "autocomplete", "on", true},
+ // 4: http://www.w3.org/TR/html4/interact/forms.html#h-17.3
+ // 5: http://www.w3.org/TR/html5/association-of-controls-and-forms.html#
+ // attributes-for-form-submission
+ {"form", "enctype", "application/x-www-form-urlencoded", false},
+ {"form", "method", "get", false},
+ // 4: http://www.w3.org/TR/html4/present/frames.html#h-16.2.2
+ // 5: Note that the <frame> tag is deprecated in HTML5.
+ // http://www.w3.org/TR/html5/obsolete.html#non-conforming-features
+ {"frame", "frameborder", "1", false},
+ {"frame", "scrolling", "auto", false},
+ // 4: http://www.w3.org/TR/html4/present/frames.html#h-16.5
+ // 5: Note that these attributes are deprecated in HTML5.
+ // http://www.w3.org/TR/html5/obsolete.html#non-conforming-features
+ {"iframe", "frameborder", "1", false},
+ {"iframe", "scrolling", "auto", false},
+ // 4: http://www.w3.org/TR/html4/interact/forms.html#h-17.4
+ // 5: http://www.w3.org/TR/html5/the-input-element.html#the-input-element
+ {"input", "type", "text", false},
+ // 4: The <keygen> tag does not exist in HTML 4.
+ // 5: http://www.w3.org/TR/html5/the-button-element.html#the-keygen-element
+ {"keygen", "keytype", "rsa", true},
+ // 4: The <menu> tag seems to mean something different in HTML 4.
+ // 5: http://www.w3.org/TR/html5/interactive-elements.html#menus
+ {"menu", "type", "list", true},
+ // 4: http://www.w3.org/TR/html4/struct/objects.html#h-13.3.2
+ // 5: Note that the <param> tag's valuetype attribute is deprecated in HTML5.
+ // http://www.w3.org/TR/html5/obsolete.html#non-conforming-features
+ {"param", "valuetype", "data", false},
+ // 4: These attributes have no default values in HTML 4.
+ // http://www.w3.org/TR/html4/interact/scripts.html#h-18.2.1
+ // 5: http://www.w3.org/TR/html5/scripting-1.html
+ {"script", "language", "javascript", true},
+ {"script", "type", "text/javascript", true},
+ // 4: The <source> tag does not exist in HTML 4.
+ // 5: http://www.w3.org/TR/html5/video.html#the-source-element
+ {"source", "media", "all", true},
+ // 4: This attribute has no default value in HTML 4.
+ // http://www.w3.org/TR/html4/present/styles.html#h-14.2.3
+ // 5: http://www.w3.org/TR/html5/semantics.html#the-style-element
+ {"style", "type", "text/css", true},
+ // 4: This attributes has a _different_ default value in HTML 4!
+ // http://www.w3.org/TR/html4/present/styles.html#h-14.2.3
+ // 5: http://www.w3.org/TR/html5/semantics.html#the-style-element
+ {"style", "media", "all", true},
+ // 4: The <textarea> tag's wrap attribute does not exist in HTML 4.
+ // 5: http://www.w3.org/TR/html5/the-button-element.html#the-textarea-element
+ {"textarea", "wrap", "soft", true},
+ // 4: http://www.w3.org/TR/html4/struct/tables.html
+ // 5: http://www.w3.org/TR/html5/tabular-data.html#table-model
+ {"col", "span", "1", false},
+ {"colgroup", "span", "1", false},
+ {"td", "colspan", "1", false},
+ {"td", "rowspan", "1", false},
+ {"th", "colspan", "1", false},
+ {"th", "rowspan", "1", false},
+};
+
+} // namespace
+
+namespace net_instaweb {
+
+ElideAttributesFilter::ElideAttributesFilter(HtmlParse* html_parse)
+ : html_parse_(html_parse) {
+ // Populate one_value_attrs_map_
+ for (size_t i = 0; i < arraysize(kBooleanAttrs); ++i) {
+ const TagAttr& entry = kBooleanAttrs[i];
+ one_value_attrs_map_[html_parse->Intern(entry.tag_name)].insert(
+ html_parse->Intern(entry.attr_name));
+ }
+ // Populate default_value_map_
+ for (size_t i = 0; i < arraysize(kDefaultList); ++i) {
+ const TagAttrValue& entry = kDefaultList[i];
+ AttrValue& value = default_value_map_[html_parse->Intern(entry.tag_name)]
+ [html_parse->Intern(entry.attr_name)];
+ value.attr_value = entry.attr_value;
+ value.requires_version_5 = entry.requires_version_5;
+ }
+}
+
+void ElideAttributesFilter::StartElement(HtmlElement* element) {
+ const DocType& doctype = html_parse_->doctype();
+
+ if (!doctype.IsXhtml()) {
+ // Check for boolean attributes.
+ AtomSetMap::const_iterator iter =
+ one_value_attrs_map_.find(element->tag());
+ if (iter != one_value_attrs_map_.end()) {
+ const AtomSet& oneValueAttrs = iter->second;
+ for (int i = 0, end = element->attribute_size(); i < end; ++i) {
+ HtmlElement::Attribute& attribute = element->attribute(i);
+ if (attribute.value() != NULL &&
+ oneValueAttrs.count(attribute.name()) > 0) {
+ attribute.SetValue(NULL);
+ }
+ }
+ }
+ }
+
+ // Check for attributes with default values.
+ ValueMapMap::const_iterator iter1 = default_value_map_.find(element->tag());
+ if (iter1 != default_value_map_.end()) {
+ const ValueMap& default_values = iter1->second;
+ for (int i = 0; i < element->attribute_size(); ++i) {
+ HtmlElement::Attribute& attribute = element->attribute(i);
+ if (attribute.value() != NULL) {
+ ValueMap::const_iterator iter2 = default_values.find(attribute.name());
+ if (iter2 != default_values.end()) {
+ const AttrValue& value = iter2->second;
+ if ((!value.requires_version_5 || doctype.IsVersion5()) &&
+ StringCaseEqual(attribute.value(), value.attr_value)) {
+ element->DeleteAttribute(i);
+ --i;
+ }
+ }
+ }
+ }
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/elide_attributes_filter_test.cc b/trunk/src/net/instaweb/rewriter/elide_attributes_filter_test.cc
new file mode 100644
index 0000000..57dff31
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/elide_attributes_filter_test.cc
@@ -0,0 +1,104 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_parse_test_base.h"
+#include "net/instaweb/rewriter/public/elide_attributes_filter.h"
+
+namespace net_instaweb {
+
+class ElideAttributesFilterTest : public HtmlParseTestBase {
+ protected:
+ ElideAttributesFilterTest()
+ : elide_attributes_filter_(&html_parse_) {
+ html_parse_.AddFilter(&elide_attributes_filter_);
+ }
+
+ virtual bool AddBody() const { return false; }
+
+ private:
+ ElideAttributesFilter elide_attributes_filter_;
+
+ DISALLOW_COPY_AND_ASSIGN(ElideAttributesFilterTest);
+};
+
+TEST_F(ElideAttributesFilterTest, NoChanges) {
+ ValidateNoChanges("no_changes",
+ "<head><script src=\"foo.js\"></script></head>"
+ "<body><form method=\"post\">"
+ "<input type=\"checkbox\" checked>"
+ "</form></body>");
+}
+
+TEST_F(ElideAttributesFilterTest, RemoveAttrWithDefaultValue) {
+ ValidateExpected("remove_attr_with_default_value",
+ "<head></head><body><form method=get></form></body>",
+ "<head></head><body><form></form></body>");
+}
+
+TEST_F(ElideAttributesFilterTest, RemoveValueFromAttr) {
+ SetDoctype("<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01//EN\" "
+ "\"http://www.w3.org/TR/html4/strict.dtd\">");
+ ValidateExpected("remove_value_from_attr",
+ "<head></head><body><form>"
+ "<input type=checkbox checked=checked></form></body>",
+ "<head></head><body><form>"
+ "<input type=checkbox checked></form></body>");
+}
+
+TEST_F(ElideAttributesFilterTest, DoNotRemoveValueFromAttrInXhtml) {
+ SetDoctype("<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.1//EN\" "
+ "\"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd\">");
+ ValidateNoChanges("do_not_remove_value_from_attr_in_xhtml",
+ "<head></head><body><form>"
+ "<input type=checkbox checked=checked></form></body>");
+}
+
+TEST_F(ElideAttributesFilterTest, DoNotBreakVBScript) {
+ SetDoctype("<!doctype html>");
+ ValidateExpected("do_not_break_vbscript",
+ "<head><script language=\"JavaScript\">var x=1;</script>"
+ "<script language=\"VBScript\">"
+ "Sub foo(ByVal bar)\n call baz(bar)\nend sub"
+ "</script></head><body></body>",
+ // Remove language="JavaScript", but not the VBScript one:
+ "<head><script>var x=1;</script>"
+ "<script language=\"VBScript\">"
+ "Sub foo(ByVal bar)\n call baz(bar)\nend sub"
+ "</script></head><body></body>");
+}
+
+TEST_F(ElideAttributesFilterTest, RemoveScriptTypeInHtml5) {
+ SetDoctype("<!doctype html>");
+ ValidateExpected("remove_script_type_in_html_5",
+ "<head><script src=\"foo.js\" type=\"text/javascript\">"
+ "</script></head><body></body>",
+ "<head><script src=\"foo.js\">"
+ "</script></head><body></body>");
+}
+
+// See http://code.google.com/p/modpagespeed/issues/detail?id=59
+TEST_F(ElideAttributesFilterTest, DoNotRemoveScriptTypeInHtml4) {
+ SetDoctype("<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01//EN\" "
+ "\"http://www.w3.org/TR/html4/strict.dtd\">");
+ ValidateNoChanges("do_not_remove_script_type_in_html_4",
+ "<head><script src=\"foo.js\" type=\"text/javascript\">"
+ "</script></head><body></body>");
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/file_input_resource.cc b/trunk/src/net/instaweb/rewriter/file_input_resource.cc
new file mode 100644
index 0000000..d4ece8b
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/file_input_resource.cc
@@ -0,0 +1,39 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/rewriter/public/file_input_resource.h"
+
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+
+namespace net_instaweb {
+
+FileInputResource::~FileInputResource() {
+}
+
+bool FileInputResource::Load(MessageHandler* message_handler) {
+ FileSystem* file_system = resource_manager_->file_system();
+ if (file_system->ReadFile(filename_.c_str(), &value_, message_handler)) {
+ resource_manager_->SetDefaultHeaders(type_, &meta_data_);
+ value_.SetHeaders(meta_data_);
+ }
+ return loaded();
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/html_attribute_quote_removal.cc b/trunk/src/net/instaweb/rewriter/html_attribute_quote_removal.cc
new file mode 100644
index 0000000..677ea8d
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/html_attribute_quote_removal.cc
@@ -0,0 +1,95 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/rewriter/public/html_attribute_quote_removal.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace {
+
+// Explicit about signedness because we are
+// loading a 0-indexed lookup table.
+const unsigned char kNoQuoteChars[] =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-._:";
+
+} // namespace
+
+// TODO(jmaessen): Make adjustable.
+const bool kLogQuoteRemoval = false;
+
+namespace net_instaweb {
+// Remove quotes; see description in .h file.
+
+HtmlAttributeQuoteRemoval::HtmlAttributeQuoteRemoval(HtmlParse* html_parse)
+ : total_quotes_removed_(0),
+ html_parse_(html_parse) {
+ // In pidgin Python:
+ // needs_no_quotes[:] = false
+ // needs_no_quotes[kNoQuoteChars] = true
+ memset(&needs_no_quotes_, 0, sizeof(needs_no_quotes_));
+ for (int i = 0; kNoQuoteChars[i] != '\0'; ++i) {
+ needs_no_quotes_[kNoQuoteChars[i]] = true;
+ }
+}
+
+bool HtmlAttributeQuoteRemoval::NeedsQuotes(const char *val) {
+ bool needs_quotes = false;
+ int i = 0;
+ if (val != NULL) {
+ for (; val[i] != '\0'; ++i) {
+ // Explicit cast to unsigned char ensures that our offset
+ // into needs_no_quotes_ is positive.
+ needs_quotes = !needs_no_quotes_[static_cast<unsigned char>(val[i])];
+ if (needs_quotes) {
+ break;
+ }
+ }
+ }
+ // Note that due to inconsistencies in empty attribute parsing between Firefox
+ // and Chrome (Chrome seems to parse the next thing it sees after whitespace
+ // as the attribute value) we leave empty attributes intact.
+ return needs_quotes || i == 0;
+}
+
+void HtmlAttributeQuoteRemoval::StartElement(HtmlElement* element) {
+ if (html_parse_->doctype().IsXhtml()) {
+ return; // XHTML doctypes require quotes, so don't remove any.
+ }
+ int rewritten = 0;
+ for (int i = 0; i < element->attribute_size(); ++i) {
+ HtmlElement::Attribute& attr = element->attribute(i);
+ if (attr.quote() != NULL && attr.quote()[0] != '\0' &&
+ !NeedsQuotes(attr.value())) {
+ attr.set_quote("");
+ rewritten++;
+ }
+ }
+ if (rewritten > 0) {
+ total_quotes_removed_ += rewritten;
+ if (kLogQuoteRemoval) {
+ const char* plural = (rewritten == 1) ? "" : "s";
+ html_parse_->InfoHere("Scrubbed quotes from %d attribute%s",
+ rewritten, plural);
+ }
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/html_attribute_quote_removal_test.cc b/trunk/src/net/instaweb/rewriter/html_attribute_quote_removal_test.cc
new file mode 100644
index 0000000..bdb22dc
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/html_attribute_quote_removal_test.cc
@@ -0,0 +1,73 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_parse_test_base.h"
+#include "net/instaweb/rewriter/public/html_attribute_quote_removal.h"
+
+namespace net_instaweb {
+
+class HtmlAttributeQuoteRemovalTest : public HtmlParseTestBase {
+ protected:
+ HtmlAttributeQuoteRemovalTest()
+ : html_attribute_quote_removal_(&html_parse_) {
+ html_parse_.AddFilter(&html_attribute_quote_removal_);
+ }
+
+ virtual bool AddBody() const { return true; }
+
+ private:
+ HtmlAttributeQuoteRemoval html_attribute_quote_removal_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlAttributeQuoteRemovalTest);
+};
+
+TEST_F(HtmlAttributeQuoteRemovalTest, NoQuotesNoChange) {
+ ValidateNoChanges("no_quotes_no_change",
+ "<div class=foo id=bar>foobar</div>");
+}
+
+TEST_F(HtmlAttributeQuoteRemovalTest, DoNotRemoveNeededQuotes) {
+ ValidateNoChanges("do_not_remove_needed_quotes",
+ "<a href=\"http://www.example.com/\">foobar</a>");
+}
+
+TEST_F(HtmlAttributeQuoteRemovalTest, DoNotDeleteEmptyAttrs) {
+ ValidateNoChanges("do_not_delete_empty_attrs",
+ "<div id=''></div>");
+}
+
+TEST_F(HtmlAttributeQuoteRemovalTest, RemoveUnneededQuotes) {
+ ValidateExpected("remove_unneeded_quotes",
+ "<div class=\"foo\" id='bar'>foobar</div>",
+ "<div class=foo id=bar>foobar</div>");
+}
+
+TEST_F(HtmlAttributeQuoteRemovalTest, NoValueNoChange) {
+ ValidateNoChanges("no_value_no_change",
+ "<input checked type=checkbox>");
+}
+
+TEST_F(HtmlAttributeQuoteRemovalTest, DoNotRemoveQuotesInXhtml) {
+ SetDoctype("<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.1//EN\" "
+ "\"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd\">");
+ ValidateNoChanges("do_not_remove_quotes_in_xhtml",
+ "<div class=\"foo\" id='bar'>foobar</div>");
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/html_minifier_main.cc b/trunk/src/net/instaweb/rewriter/html_minifier_main.cc
new file mode 100644
index 0000000..2e70720
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/html_minifier_main.cc
@@ -0,0 +1,120 @@
+// Copyright 2010 Google Inc. All Rights Reserved.
+// Author: bmcquade@google.com (Bryan McQuade)
+//
+// Simple HTML minifier, based on pagespeed's minify_html.cc
+
+#include <stdio.h>
+
+#include <fstream>
+#include <string>
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_writer_filter.h"
+#include "net/instaweb/rewriter/public/collapse_whitespace_filter.h"
+#include "net/instaweb/rewriter/public/elide_attributes_filter.h"
+#include "net/instaweb/rewriter/public/html_attribute_quote_removal.h"
+#include "net/instaweb/rewriter/public/remove_comments_filter.h"
+#include "net/instaweb/util/public/file_message_handler.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/string_writer.h"
+
+namespace {
+
+using net_instaweb::StrCat;
+
+void ReadFileToStringOrDie(const char* filename,
+ std::string* dest) {
+ std::ifstream file_stream;
+ file_stream.open(filename, std::ifstream::in | std::ifstream::binary);
+ CHECK(!file_stream.fail());
+ dest->assign(std::istreambuf_iterator<char>(file_stream),
+ std::istreambuf_iterator<char>());
+ file_stream.close();
+ CHECK(!file_stream.fail());
+}
+
+void WriteStringToFileOrDie(const std::string& src,
+ const char* filename) {
+ std::ofstream file_stream;
+ file_stream.open(filename, std::ifstream::out | std::ifstream::binary);
+ CHECK(!file_stream.fail());
+ file_stream.write(src.c_str(), src.size());
+ file_stream.close();
+ CHECK(!file_stream.fail());
+}
+
+class HtmlMinifier {
+ public:
+ explicit HtmlMinifier();
+ ~HtmlMinifier();
+
+ // Return true if successful, false on error.
+ bool MinifyHtml(const std::string& input_name,
+ const std::string& input,
+ std::string* output);
+
+ private:
+ net_instaweb::FileMessageHandler message_handler_;
+ net_instaweb::HtmlParse html_parse_;
+ net_instaweb::RemoveCommentsFilter remove_comments_filter_;
+ net_instaweb::ElideAttributesFilter elide_attributes_filter_;
+ net_instaweb::HtmlAttributeQuoteRemoval quote_removal_filter_;
+ net_instaweb::CollapseWhitespaceFilter collapse_whitespace_filter_;
+ net_instaweb::HtmlWriterFilter html_writer_filter_;
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlMinifier);
+};
+
+HtmlMinifier::HtmlMinifier()
+ : message_handler_(stderr),
+ html_parse_(&message_handler_),
+ remove_comments_filter_(&html_parse_),
+ elide_attributes_filter_(&html_parse_),
+ quote_removal_filter_(&html_parse_),
+ collapse_whitespace_filter_(&html_parse_),
+ html_writer_filter_(&html_parse_) {
+ html_parse_.AddFilter(&remove_comments_filter_);
+ html_parse_.AddFilter(&elide_attributes_filter_);
+ html_parse_.AddFilter("e_removal_filter_);
+ html_parse_.AddFilter(&collapse_whitespace_filter_);
+ html_parse_.AddFilter(&html_writer_filter_);
+}
+
+HtmlMinifier::~HtmlMinifier() {}
+
+bool HtmlMinifier::MinifyHtml(const std::string& input_name,
+ const std::string& input,
+ std::string* output) {
+ net_instaweb::StringWriter string_writer(output);
+ html_writer_filter_.set_writer(&string_writer);
+
+ std::string url = StrCat("http://html_minifier.com/", input_name, ".html");
+ html_parse_.StartParse(url);
+ html_parse_.ParseText(input.data(), input.size());
+ html_parse_.FinishParse();
+
+ html_writer_filter_.set_writer(NULL);
+
+ return true;
+}
+
+} // namespace
+
+int main(int argc, char** argv) {
+ if (argc != 3) {
+ fprintf(stderr, "Usage: minify_html <input> <output>\n");
+ return 1;
+ }
+
+ std::string original;
+ ReadFileToStringOrDie(argv[1], &original);
+
+ std::string minified;
+ HtmlMinifier html_minifier;
+ html_minifier.MinifyHtml(argv[1], original, &minified);
+
+ WriteStringToFileOrDie(minified, argv[2]);
+ return 0;
+}
diff --git a/trunk/src/net/instaweb/rewriter/image.cc b/trunk/src/net/instaweb/rewriter/image.cc
new file mode 100644
index 0000000..d1aada4
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/image.cc
@@ -0,0 +1,480 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/rewriter/public/image.h"
+
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/resource.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/writer.h"
+#undef PAGESPEED_PNG_OPTIMIZER_GIF_READER
+#define PAGESPEED_PNG_OPTIMIZER_GIF_READER 0
+#ifdef USE_SYSTEM_OPENCV
+#include "cv.h"
+#include "highgui.h"
+#else
+#include "third_party/opencv/src/opencv/include/opencv/cv.h"
+#include "third_party/opencv/src/opencv/include/opencv/highgui.h"
+#endif
+#include "pagespeed/image_compression/gif_reader.h"
+#include "pagespeed/image_compression/jpeg_optimizer.h"
+#include "pagespeed/image_compression/png_optimizer.h"
+
+namespace net_instaweb {
+
+namespace ImageHeaders {
+
+const char kPngHeader[] = "\x89PNG\r\n\x1a\n";
+const size_t kPngHeaderLength = sizeof(kPngHeader) - 1;
+const char kPngIHDR[] = "\0\0\0\x0dIHDR";
+const size_t kPngIntSize = 4;
+const size_t kPngSectionHeaderLength = 2 * kPngIntSize;
+const size_t kIHDRDataStart = kPngHeaderLength + kPngSectionHeaderLength;
+const size_t kPngSectionMinSize = kPngSectionHeaderLength + kPngIntSize;
+const size_t kPngColourTypeOffset = kIHDRDataStart + 2 * kPngIntSize + 1;
+const char kPngAlphaChannel = 0x4; // bit of ColourType set for alpha channel
+const char kPngIDAT[] = "IDAT";
+const char kPngtRNS[] = "tRNS";
+
+const char kGifHeader[] = "GIF8";
+const size_t kGifHeaderLength = sizeof(kGifHeader) - 1;
+const size_t kGifDimStart = kGifHeaderLength + 2;
+const size_t kGifIntSize = 2;
+
+const size_t kJpegIntSize = 2;
+
+} // namespace ImageHeaders
+
+namespace {
+
+bool WriteTempFileWithContentType(
+ const StringPiece& prefix_name, const ContentType& content_type,
+ const StringPiece& buffer, std::string* filename,
+ FileSystem* file_system, MessageHandler* handler) {
+ std::string tmp_filename;
+ bool ok = file_system->WriteTempFile(
+ prefix_name.as_string().c_str(), buffer, &tmp_filename, handler);
+ if (ok) {
+ *filename = StrCat(tmp_filename, content_type.file_extension());
+ ok = file_system->RenameFile(
+ tmp_filename.c_str(), filename->c_str(), handler);
+ }
+ return ok;
+}
+
+} // namespace
+
+Image::Image(const StringPiece& original_contents,
+ const std::string& url,
+ const StringPiece& file_prefix,
+ FileSystem* file_system,
+ MessageHandler* handler)
+ : file_prefix_(file_prefix.data(), file_prefix.size()),
+ file_system_(file_system),
+ handler_(handler),
+ image_type_(IMAGE_UNKNOWN),
+ original_contents_(original_contents),
+ output_contents_(),
+ output_valid_(false),
+ opencv_filename_(),
+ opencv_image_(NULL),
+ opencv_load_possible_(true),
+ resized_(false),
+ url_(url) { }
+
+Image::~Image() {
+ CleanOpenCV();
+}
+
+// Looks through blocks of jpeg stream to find SOFn block
+// indicating encoding and dimensions of image.
+// Loosely based on code and FAQs found here:
+// http://www.faqs.org/faqs/jpeg-faq/part1/
+void Image::FindJpegSize() {
+ const StringPiece& buf = original_contents_;
+ size_t pos = 2; // Position of first data block after header.
+ while (pos < buf.size()) {
+ // Read block identifier
+ int id = CharToInt(buf[pos++]);
+ if (id == 0xff) { // Padding byte
+ continue;
+ }
+ // At this point pos points to first data byte in block. In any block,
+ // first two data bytes are size (including these 2 bytes). But first,
+ // make sure block wasn't truncated on download.
+ if (pos + ImageHeaders::kJpegIntSize > buf.size()) {
+ break;
+ }
+ int length = JpegIntAtPosition(buf, pos);
+ // Now check for a SOFn header, which describes image dimensions.
+ if (0xc0 <= id && id <= 0xcf && // SOFn header
+ length >= 8 && // Valid SOFn block size
+ pos + 1 + 3 * ImageHeaders::kJpegIntSize <= buf.size() &&
+ // Above avoids case where dimension data was truncated
+ id != 0xc4 && id != 0xc8 && id != 0xcc) {
+ // 0xc4, 0xc8, 0xcc aren't actually valid SOFn headers.
+ // NOTE: we don't care if we have the whole SOFn block,
+ // just that we can fetch both dimensions without trouble.
+ // Our image download could be truncated at this point for
+ // all we care.
+ // We're a bit sloppy about SOFn block size, as it's
+ // actually 8 + 3 * buf[pos+2], but for our purposes this
+ // will suffice as we don't parse subsequent metadata (which
+ // describes the formatting of chunks of image data).
+ int height = JpegIntAtPosition(buf, pos + 1 + ImageHeaders::kJpegIntSize);
+ int width = JpegIntAtPosition(buf,
+ pos + 1 + 2 * ImageHeaders::kJpegIntSize);
+ dims_.set_dims(width, height);
+ break;
+ }
+ pos += length;
+ }
+ if (dims_.height() <= 0 || dims_.width() <= 0) {
+ dims_.invalidate();
+ handler_->Error(url_.c_str(), 0,
+ "Couldn't find jpeg dimensions (data truncated?).");
+ }
+}
+
+// Looks at first (IHDR) block of png stream to find image dimensions.
+// See also: http://www.w3.org/TR/PNG/
+void Image::FindPngSize() {
+ const StringPiece& buf = original_contents_;
+ // Here we make sure that buf contains at least enough data that we'll be able
+ // to decipher the image dimensions first, before we actually check for the
+ // headers and attempt to decode the dimensions (which are the first two ints
+ // after the IHDR section label).
+ if ((buf.size() >= // Not truncated
+ ImageHeaders::kIHDRDataStart + 2 * ImageHeaders::kPngIntSize) &&
+ (StringPiece(buf.data() + ImageHeaders::kPngHeaderLength,
+ ImageHeaders::kPngSectionHeaderLength) ==
+ StringPiece(ImageHeaders::kPngIHDR,
+ ImageHeaders::kPngSectionHeaderLength))) {
+ int width = PngIntAtPosition(buf, ImageHeaders::kIHDRDataStart);
+ int height = PngIntAtPosition(
+ buf, ImageHeaders::kIHDRDataStart + ImageHeaders::kPngIntSize);
+ dims_.set_dims(width, height);
+ } else {
+ handler_->Error(url_.c_str(), 0,
+ "Couldn't find png dimensions "
+ "(data truncated or IHDR missing).");
+ }
+}
+
+// Looks at header of GIF file to extract image dimensions.
+// See also: http://en.wikipedia.org/wiki/Graphics_Interchange_Format
+void Image::FindGifSize() {
+ const StringPiece& buf = original_contents_;
+ // Make sure that buf contains enough data that we'll be able to
+ // decipher the image dimensions before we attempt to do so.
+ if (buf.size() >=
+ ImageHeaders::kGifDimStart + 2 * ImageHeaders::kGifIntSize) {
+ // Not truncated
+ int width = GifIntAtPosition(buf, ImageHeaders::kGifDimStart);
+ int height = GifIntAtPosition(
+ buf, ImageHeaders::kGifDimStart + ImageHeaders::kGifIntSize);
+ dims_.set_dims(width, height);
+ } else {
+ handler_->Error(url_.c_str(), 0,
+ "Couldn't find gif dimensions (data truncated)");
+ }
+}
+
+// Looks at image data in order to determine image type, and also fills in any
+// dimension information it can (setting image_type_ and dims_).
+void Image::ComputeImageType() {
+ // Image classification based on buffer contents gakked from leptonica,
+ // but based on well-documented headers (see Wikipedia etc.).
+ // Note that we can be fooled if we're passed random binary data;
+ // we make the call based on as few as two bytes (JPEG).
+ const StringPiece& buf = original_contents_;
+ if (buf.size() >= 8) {
+ // Note that gcc rightly complains about constant ranges with the
+ // negative char constants unless we cast.
+ switch (CharToInt(buf[0])) {
+ case 0xff:
+ // Either jpeg or jpeg2
+ // (the latter we don't handle yet, and don't bother looking for).
+ if (CharToInt(buf[1]) == 0xd8) {
+ image_type_ = IMAGE_JPEG;
+ FindJpegSize();
+ }
+ break;
+ case 0x89:
+ // Possible png.
+ if (StringPiece(buf.data(), ImageHeaders::kPngHeaderLength) ==
+ StringPiece(ImageHeaders::kPngHeader,
+ ImageHeaders::kPngHeaderLength)) {
+ image_type_ = IMAGE_PNG;
+ FindPngSize();
+ }
+ break;
+ case 'G':
+ // Possible gif.
+ if ((StringPiece(buf.data(), ImageHeaders::kGifHeaderLength) ==
+ StringPiece(ImageHeaders::kGifHeader,
+ ImageHeaders::kGifHeaderLength)) &&
+ (buf[4] == '7' || buf[4] == '9') && buf[5] == 'a') {
+ image_type_ = IMAGE_GIF;
+ FindGifSize();
+ }
+ break;
+ default:
+ break;
+ }
+ }
+}
+
+const ContentType* Image::content_type() {
+ const ContentType* res = NULL;
+ switch (image_type()) {
+ case IMAGE_UNKNOWN:
+ break;
+ case IMAGE_JPEG:
+ res = &kContentTypeJpeg;
+ break;
+ case IMAGE_PNG:
+ res = &kContentTypePng;
+ break;
+ case IMAGE_GIF:
+ res = &kContentTypeGif;
+ break;
+ }
+ return res;
+}
+
+// Compute whether a PNG can have transparent / semi-transparent pixels
+// by walking the image data in accordance with the spec:
+// http://www.w3.org/TR/PNG/
+// If the colour type (UK spelling from spec) includes an alpha channel, or
+// there is a tRNS section with at least one entry before IDAT, then we assume
+// the image contains non-opaque pixels and return true.
+bool Image::ComputePngTransparency() {
+ // We assume the image has transparency until we prove otherwise.
+ // This allows us to deal conservatively with truncation etc.
+ bool has_transparency = true;
+ const StringPiece& buf = original_contents_;
+ if (buf.size() > ImageHeaders::kPngColourTypeOffset &&
+ ((buf[ImageHeaders::kPngColourTypeOffset] &
+ ImageHeaders::kPngAlphaChannel) == 0)) {
+ // The colour type indicates that there is no dedicated alpha channel. Now
+ // we must look for a tRNS section indicating the existence of transparent
+ // colors or palette entries.
+ size_t section_start = ImageHeaders::kPngHeaderLength;
+ while (section_start + ImageHeaders::kPngSectionHeaderLength < buf.size()) {
+ size_t section_size = PngIntAtPosition(buf, section_start);
+ if (PngSectionIdIs(ImageHeaders::kPngIDAT, buf, section_start)) {
+ // tRNS section must occur before first IDAT. This image doesn't have a
+ // tRNS section, and thus doesn't have transparency.
+ has_transparency = false;
+ break;
+ } else if (PngSectionIdIs(ImageHeaders::kPngtRNS, buf, section_start) &&
+ section_size > 0) {
+ // Found a nonempty tRNS section. This image has_transparency.
+ break;
+ } else {
+ // Move on to next section.
+ section_start += section_size + ImageHeaders::kPngSectionMinSize;
+ }
+ }
+ }
+ return has_transparency;
+}
+
+bool Image::HasTransparency() {
+ bool result;
+ switch (image_type()) {
+ case IMAGE_PNG:
+ result = ComputePngTransparency();
+ break;
+ case IMAGE_GIF:
+ // Conservative.
+ // TODO(jmaessen): fix when gif transcoding is enabled.
+ result = true;
+ break;
+ default:
+ result = false;
+ break;
+ }
+ return result;
+}
+
+// Makes sure OpenCV version of image is loaded if that is possible.
+// Returns value of opencv_load_possible_ after load attempted.
+// Note that if the load fails, opencv_load_possible_ will be false
+// and future calls to LoadOpenCV will fail fast.
+bool Image::LoadOpenCV() {
+ if (opencv_image_ == NULL && opencv_load_possible_) {
+ Image::Type image_type = this->image_type();
+ const ContentType* content_type = this->content_type();
+ opencv_load_possible_ = (content_type != NULL &&
+ image_type != IMAGE_GIF &&
+ !HasTransparency());
+ if (opencv_load_possible_) {
+ opencv_load_possible_ =
+ WriteTempFileWithContentType(
+ file_prefix_, *content_type,
+ original_contents_, &opencv_filename_,
+ file_system_, handler_);
+ }
+ if (opencv_load_possible_) {
+ opencv_image_ = cvLoadImage(opencv_filename_.c_str());
+ file_system_->RemoveFile(opencv_filename_.c_str(), handler_);
+ opencv_load_possible_ = (opencv_image_ != NULL);
+ }
+ }
+ if (opencv_load_possible_) {
+ // A bit of belt and suspenders dimension checking. We used to do this for
+ // every image we loaded, but now we only do it when we're already paying
+ // the cost of OpenCV image conversion.
+ if (dims_.valid() && dims_.width() != opencv_image_->width) {
+ handler_->Error(url_.c_str(), 0,
+ "Computed width %d doesn't match OpenCV %d",
+ dims_.width(), opencv_image_->width);
+ }
+ if (dims_.valid() && dims_.height() != opencv_image_->height) {
+ handler_->Error(url_.c_str(), 0,
+ "Computed height %d doesn't match OpenCV %d",
+ dims_.height(), opencv_image_->height);
+ }
+ }
+ return opencv_load_possible_;
+}
+
+// Get rid of OpenCV image data gracefully (requires a call to OpenCV).
+void Image::CleanOpenCV() {
+ if (opencv_image_ != NULL) {
+ cvReleaseImage(&opencv_image_);
+ }
+}
+
+void Image::Dimensions(ImageDim* natural_dim) {
+ if (dims_.valid()) {
+ natural_dim->set_dims(dims_.width(), dims_.height());
+ } else {
+ natural_dim->invalidate();
+ }
+}
+
+bool Image::ResizeTo(const ImageDim& new_dim) {
+ CHECK(new_dim.valid());
+ if (resized_) {
+ // If we already resized, drop data and work with original image.
+ UndoResize();
+ }
+ bool ok = opencv_image_ != NULL || LoadOpenCV();
+ if (ok) {
+ IplImage* rescaled_image =
+ cvCreateImage(cvSize(new_dim.width(), new_dim.height()),
+ opencv_image_->depth,
+ opencv_image_->nChannels);
+ ok = rescaled_image != NULL;
+ if (ok) {
+ cvResize(opencv_image_, rescaled_image);
+ cvReleaseImage(&opencv_image_);
+ opencv_image_ = rescaled_image;
+ }
+ resized_ = ok;
+ }
+ return resized_;
+}
+
+void Image::UndoResize() {
+ if (resized_) {
+ CleanOpenCV();
+ output_valid_ = false;
+ image_type_ = IMAGE_UNKNOWN;
+ resized_ = false;
+ }
+}
+
+// Performs image optimization and output
+bool Image::ComputeOutputContents() {
+ if (!output_valid_) {
+ bool ok = true;
+ std::string opencv_contents;
+ StringPiece contents = original_contents_;
+ // Choose appropriate source for image contents.
+ // Favor original contents if image unchanged.
+ if (resized_ && opencv_image_ != NULL) {
+ cvSaveImage(opencv_filename_.c_str(), opencv_image_);
+ ok = file_system_->ReadFile(opencv_filename_.c_str(),
+ &opencv_contents, handler_);
+ file_system_->RemoveFile(opencv_filename_.c_str(), handler_);
+ if (ok) {
+ contents = opencv_contents;
+ }
+ }
+ // Take image contents and re-compress them.
+ if (ok) {
+ // If we can't optimize the image, we'll fail.
+ ok = false;
+ switch (image_type()) {
+ case IMAGE_UNKNOWN:
+ break;
+ case IMAGE_JPEG:
+ // TODO(jmarantz): The PageSpeed library should, ideally, take
+ // StringPiece args rather than const string&. We would save
+ // lots of string-copying if we made that change.
+ ok = pagespeed::image_compression::OptimizeJpeg(
+ std::string(contents.data(), contents.size()),
+ &output_contents_);
+ break;
+ case IMAGE_PNG: {
+ pagespeed::image_compression::PngReader png_reader;
+ ok = pagespeed::image_compression::PngOptimizer::OptimizePng(
+ png_reader,
+ std::string(contents.data(), contents.size()),
+ &output_contents_);
+ break;
+ }
+ case IMAGE_GIF: {
+#if PAGESPEED_PNG_OPTIMIZER_GIF_READER
+ pagespeed::image_compression::GifReader gif_reader;
+ ok = pagespeed::image_compression::PngOptimizer::OptimizePng(
+ gif_reader, contents, &output_contents_);
+ if (ok) {
+ image_type_ = IMAGE_PNG;
+ }
+#endif
+ break;
+ }
+ }
+ }
+ output_valid_ = ok;
+ }
+ return output_valid_;
+}
+
+StringPiece Image::Contents() {
+ StringPiece contents;
+ if (this->image_type() != IMAGE_UNKNOWN) {
+ contents = original_contents_;
+ if (output_valid_ || ComputeOutputContents()) {
+ contents = output_contents_;
+ }
+ }
+ return contents;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/image_dim.cc b/trunk/src/net/instaweb/rewriter/image_dim.cc
new file mode 100644
index 0000000..b50f83e
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/image_dim.cc
@@ -0,0 +1,80 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/rewriter/public/image_dim.h"
+
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// decodes decimal int followed by x at start of source, removing them
+// from source. Returns true on success. This is a utility for the
+// decoding of image dimensions.
+bool DecodeIntX(StringPiece* in, int *result) {
+ *result = 0;
+ bool ok = false;
+ for (; !in->empty(); in->remove_prefix(1)) {
+ // TODO(jmaessen): roll strtol-like functionality for StringPiece in util
+ if (!AccumulateDecimalValue((*in)[0], result)) {
+ break;
+ }
+ ok = true;
+ }
+ if (in->empty()) {
+ ok = false;
+ } else if ((*in)[0] != 'x') {
+ ok = false;
+ } else {
+ in->remove_prefix(1);
+ }
+ return ok;
+}
+
+} // namespace
+
+void ImageDim::EncodeTo(std::string* out) const {
+ if (valid_) {
+ out->append(StrCat(IntegerToString(width_), "x",
+ IntegerToString(height_)));
+ }
+ out->append("x");
+}
+
+bool ImageDim::DecodeFrom(StringPiece* in) {
+ valid_ = false;
+ bool ok;
+ if (in->empty()) {
+ ok = false;
+ } else if ((*in)[0] == 'x') {
+ // No dimensions.
+ in->remove_prefix(1);
+ ok = true;
+ } else if (DecodeIntX(in, &width_) &&
+ DecodeIntX(in, &height_)) {
+ // Dimensions.
+ valid_ = true;
+ ok = true;
+ } else {
+ ok = false;
+ }
+ return ok;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/image_endian_test.cc b/trunk/src/net/instaweb/rewriter/image_endian_test.cc
new file mode 100644
index 0000000..a93cb3c
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/image_endian_test.cc
@@ -0,0 +1,45 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+// Unit tests for endian-dependent operations used in image rewriting.
+
+#include "net/instaweb/rewriter/public/image.h"
+
+#include "net/instaweb/htmlparse/public/html_parse_test_base.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace net_instaweb {
+namespace {
+
+TEST(ImageEndianTest, CharToIntTest) {
+ EXPECT_EQ(0xff, CharToInt(-1));
+ // Worried about C++ implicit conversions to/from int for arg passing:
+ EXPECT_EQ(0x05, CharToInt(static_cast<char>(0xffffff05)));
+ EXPECT_EQ(0x83, CharToInt(static_cast<char>(0xffffff83)));
+ EXPECT_EQ(0x33, CharToInt(0x33));
+ // Now test deserialization from buffer full of negative values
+ const char buf[] = { 0xf1, 0xf2, 0xf3, 0xf4 };
+ EXPECT_EQ(0xf1f2, JpegIntAtPosition(buf, 0));
+ EXPECT_EQ(0xf2f3, JpegIntAtPosition(buf, 1));
+ EXPECT_EQ(0xf2f1, GifIntAtPosition(buf, 0));
+ EXPECT_EQ(0xf4f3, GifIntAtPosition(buf, 2));
+ EXPECT_EQ(static_cast<int>(0xf1f2f3f4), PngIntAtPosition(buf, 0));
+}
+
+} // namespace
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/image_rewrite_filter_test.cc b/trunk/src/net/instaweb/rewriter/image_rewrite_filter_test.cc
new file mode 100644
index 0000000..e82e2e3
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/image_rewrite_filter_test.cc
@@ -0,0 +1,348 @@
+/*
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/rewriter/public/img_tag_scanner.h"
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+#include "net/instaweb/rewriter/public/rewrite_options.h"
+#include "net/instaweb/util/public/mock_timer.h"
+
+namespace net_instaweb {
+
+namespace {
+
+class ImageRewriteTest : public ResourceManagerTestBase {
+ protected:
+ // Simple image rewrite test to check resource fetching functionality.
+ void RewriteImage(const std::string& tag_string) {
+ options_.EnableFilter(RewriteOptions::kRewriteImages);
+ options_.EnableFilter(RewriteOptions::kInsertImgDimensions);
+ options_.set_img_inline_max_bytes(2000);
+ rewrite_driver_.AddFilters();
+
+ AddOtherFilter(RewriteOptions::kRewriteImages);
+
+ // URLs and content for HTML document and resources.
+ const char domain[] = "http://rewrite_image.test/";
+ const char html_url[] = "http://rewrite_image.test/RewriteImage.html";
+ const char image_url[] = "http://rewrite_image.test/Puzzle.jpg";
+
+ const std::string image_html =
+ StrCat("<head/><body><", tag_string, " src=\"Puzzle.jpg\"/></body>");
+
+ // Store image contents into fetcher.
+ const std::string image_filename =
+ StrCat(GTestSrcDir(), kTestData, "Puzzle.jpg");
+ AddFileToMockFetcher(image_url, image_filename, kContentTypeJpeg);
+
+ // Rewrite the HTML page.
+ ParseUrl(html_url, image_html);
+ StringVector img_srcs;
+ CollectImgSrcs("RewriteImage/collect_sources", output_buffer_, &img_srcs);
+ // output_buffer_ should have exactly one image file (Puzzle.jpg).
+ EXPECT_EQ(1UL, img_srcs.size());
+ const std::string& src_string = img_srcs[0];
+ EXPECT_EQ(domain, src_string.substr(0, strlen(domain)));
+ EXPECT_EQ(".jpg", src_string.substr(src_string.size() - 4, 4));
+
+ std::string rewritten_data;
+
+ const std::string expected_output =
+ StrCat("<head/><body><", tag_string, " src=\"", src_string,
+ "\" width=\"1023\" height=\"766\"/></body>");
+ EXPECT_EQ(AddHtmlBody(expected_output), output_buffer_);
+
+ std::string rewritten_filename;
+ filename_encoder_.Encode(file_prefix_, src_string, &rewritten_filename);
+
+ std::string rewritten_image_data;
+ ASSERT_TRUE(file_system_.ReadFile(rewritten_filename.c_str(),
+ &rewritten_image_data,
+ &message_handler_));
+
+ // Also fetch the resource to ensure it can be created dynamically
+ SimpleMetaData request_headers, response_headers;
+ std::string fetched_resource_content;
+ StringWriter writer(&fetched_resource_content);
+ DummyCallback dummy_callback(true);
+
+ std::string headers;
+ AppendDefaultHeaders(kContentTypeJpeg, resource_manager_, &headers);
+
+ writer.Write(headers, &message_handler_);
+ EXPECT_TRUE(
+ rewrite_driver_.FetchResource(src_string, request_headers,
+ &response_headers, &writer,
+ &message_handler_, &dummy_callback));
+ EXPECT_EQ(HttpStatus::kOK, response_headers.status_code()) <<
+ "Looking for " << src_string;
+ // For readability, only do EXPECT_EQ on initial portions of data
+ // as most of it isn't human-readable. This will show us the headers
+ // and the start of the image data. So far every failure fails this
+ // first, and we caught doubled headers this way.
+ EXPECT_EQ(rewritten_image_data.substr(0, 250),
+ fetched_resource_content.substr(0, 250)) <<
+ "In " << src_string;
+ EXPECT_TRUE(rewritten_image_data == fetched_resource_content) <<
+ "In " << src_string;
+
+ // Now we fetch from the "other" server. To simulate first fetch, we
+ // need to:
+ // 1) Clear the cache, so we don't just find the result there.
+ // TODO(sligocki): We should just use a separate cache.
+ lru_cache_->Clear();
+ // 2) Disable the file system, so we don't find the resource there.
+ // TODO(sligocki): We should just use a separate mem_file_system.
+ file_system_.Disable();
+ // 3) Disable the fetcher, so that the fetch doesn't finish imidiately.
+ // TODO(sligocki): We could use the CallCallbacks() trick to get a more
+ // correct response, rather than effectively shutting off our network.
+ mock_url_fetcher_.Disable();
+
+ fetched_resource_content.clear();
+ dummy_callback.Reset();
+ SimpleMetaData redirect_headers;
+ other_rewrite_driver_.FetchResource(src_string, request_headers,
+ &redirect_headers, &writer,
+ &message_handler_, &dummy_callback);
+ std::string expected_redirect =
+ StrCat("<img src=\"", image_url, "\" alt=\"Temporarily Moved\"/>");
+
+ EXPECT_EQ(HttpStatus::kTemporaryRedirect, redirect_headers.status_code());
+ EXPECT_EQ(expected_redirect, fetched_resource_content);
+
+ // Now we switch the file system and fetcher back on, corresponding to
+ // the case where a prior async fetch completed and we're ready to rewrite.
+ // This should yield the same results as the original html- driven rewrite.
+ file_system_.Enable();
+ mock_url_fetcher_.Enable();
+
+ // TODO(jmarantz): Refactor this code which tries the serving-fetch twice,
+ // the second after the cache stops 'remembering' that the origin fetch
+ // failed.
+ message_handler_.Message(
+ kInfo, "Now with serving, but with a recent fetch failure.");
+ SimpleMetaData other_headers;
+ // size_t header_size = fetched_resource_content.size();
+ dummy_callback.Reset();
+
+ // Trying this twice. The first time, we will again fail to rewrite
+ // the resource because the cache will 'remembered' that the image origin
+ // fetch failed and so the resource manager will not try again for 5
+ // minutes.
+ fetched_resource_content.clear();
+ other_rewrite_driver_.FetchResource(src_string, request_headers,
+ &other_headers, &writer,
+ &message_handler_, &dummy_callback);
+ EXPECT_EQ(HttpStatus::kTemporaryRedirect, redirect_headers.status_code());
+ EXPECT_EQ(expected_redirect, fetched_resource_content);
+
+ // Now let some time go by and try again.
+ message_handler_.Message(
+ kInfo, "Now with serving, but 5 minutes expired since fetch failure.");
+ other_file_system_.timer()->advance_ms(
+ 5 * Timer::kMinuteMs + 1 * Timer::kSecondMs);
+ fetched_resource_content.clear();
+ writer.Write(headers, &message_handler_);
+ EXPECT_EQ(headers, fetched_resource_content);
+ dummy_callback.Reset();
+ other_rewrite_driver_.FetchResource(src_string, request_headers,
+ &other_headers, &writer,
+ &message_handler_, &dummy_callback);
+ EXPECT_EQ(HttpStatus::kOK, other_headers.status_code());
+ EXPECT_EQ(rewritten_image_data.substr(0, 100),
+ fetched_resource_content.substr(0, 100));
+ EXPECT_TRUE(rewritten_image_data == fetched_resource_content);
+
+ std::string secondary_image_data;
+ ASSERT_TRUE(file_system_.ReadFile(rewritten_filename.c_str(),
+ &secondary_image_data,
+ &message_handler_));
+ EXPECT_EQ(rewritten_image_data.substr(0, 100),
+ secondary_image_data.substr(0, 100));
+ EXPECT_EQ(rewritten_image_data, secondary_image_data);
+
+ // Try to fetch from an independent server.
+ /* TODO(sligocki): Get this working. Right now it returns a redirect.
+ ServeResourceFromManyContexts(src_string, RewriteOptions::kRewriteImages,
+ &mock_hasher_,
+ fetched_resource_content.substr(header_size));
+ */
+ }
+
+ // Helper class to collect img srcs.
+ class ImgCollector : public EmptyHtmlFilter {
+ public:
+ ImgCollector(HtmlParse* html_parse, StringVector* img_srcs)
+ : img_srcs_(img_srcs),
+ img_filter_(html_parse) {
+ }
+
+ virtual void StartElement(HtmlElement* element) {
+ HtmlElement::Attribute* src = img_filter_.ParseImgElement(element);
+ if (src != NULL) {
+ img_srcs_->push_back(src->value());
+ }
+ }
+
+ virtual const char* Name() const { return "ImgCollector"; }
+
+ private:
+ StringVector* img_srcs_;
+ ImgTagScanner img_filter_;
+
+ DISALLOW_COPY_AND_ASSIGN(ImgCollector);
+ };
+
+ // Fills `img_srcs` with the urls in img src attributes in `html`
+ void CollectImgSrcs(const StringPiece& id, const StringPiece& html,
+ StringVector* img_srcs) {
+ HtmlParse html_parse(&message_handler_);
+ ImgCollector collector(&html_parse, img_srcs);
+ html_parse.AddFilter(&collector);
+ std::string dummy_url = StrCat("http://collect.css.links/", id, ".html");
+ html_parse.StartParse(dummy_url);
+ html_parse.ParseText(html.data(), html.size());
+ html_parse.FinishParse();
+ }
+
+ void DataUrlResource() {
+ static const char* kCuppaData = "data:image/png;base64,"
+ "iVBORw0KGgoAAAANSUhEUgAAAEEAAABGCAAAAAC2maYhAAAC00lEQVQY0+3PTUhUYR"
+ "QG4HdmMhUaC6FaKSqEZS2MsEJEsaKSwMKgot2QkkKFUFBYWgSpGIhSZH+0yAgLDQ3p"
+ "ByoLRS2DjCjEfm0MzQhK08wZ5/Sde12kc8f5DrXLs3lfPs55uBf0t4MZ4X8QLjeY2X"
+ "C80cieUq9M6MB6I7tDcMgoRWgVCb5VyDLKFuCK8RCHMpFwEzjA+coGdHJ5COwRCSnA"
+ "Jc4cwOnlshs4KhFeA+jib48A1hovK4A6iXADiOB8oyQXF28Y0CIRKgDHsMoeJaTyw6"
+ "gDOC0RGtXlPS5RQOgAlwQgWSK4lZDDZacqxVyOqNIpECgSiBxTeVsdRo/z/9iBXImw"
+ "TV3eUemLU6WRXzYCziGB0KAOs7kUqLKZS40qVwVCr9qP4vJElblc3KocFAi+cMD2U5"
+ "VBdYhPqgyp3CcQKEYdDHCZDYT/mviYa5JvCANiubxTh2u4XAAcfQLhgzrM51KjSjmX"
+ "FGAvCYRTQGgvlwwggX/iGbDwm0RIAwo439tga+biAqpJIHy2I36Uyxkgl7MnBJkkEV"
+ "4AtUbJQvwP86/m94uE71juM8piPDayDOdJJNDKFjMzNpl5fcmYUPBMZIfbzBE3CQXB"
+ "TBIuHtaYwo5phHToTMk0QqaWUNxUUXrui7XggvZEFI9YCfu1AQeQbiWc0LrOe9D11Z"
+ "cNtFsIVVpCG696YrHVQqjVAezDxm4hEi2ElzpCvLl7EkkWwliIhrDD3K1EsoVASzWE"
+ "UnM1DbushO0aQpux2Qw8shJKggPzvLzYl4BYn5XQHVzI4r2Pi4CzZCVQUlChimi0cg"
+ "GQR9ZCRVDhbl1RtIoNngBC/yzozLJqLwUQqCjotTPR1fTnxVTBs3ra89T6/ikHfgK9"
+ "dQa+t1eS//gJVB8WUCgnLYHaYwIAeaQp0GC25S8cG9cWiOrm+AHrnhMJBLplmwLkE8"
+ "kEenp/8oyIBf2ZEWaEfyv8BsICdAZ/XeTCAAAAAElFTkSuQmCC";
+ std::string cuppa_string(kCuppaData);
+ scoped_ptr<Resource> cuppa_resource(
+ resource_manager_->CreateInputResourceAbsolute(cuppa_string,
+ &options_,
+ &message_handler_));
+ ASSERT_TRUE(cuppa_resource != NULL);
+ EXPECT_TRUE(resource_manager_->ReadIfCached(cuppa_resource.get(),
+ &message_handler_));
+ std::string cuppa_contents;
+ cuppa_resource->contents().CopyToString(&cuppa_contents);
+ // Now make sure axing the original cuppa_string doesn't affect the
+ // internals of the cuppa_resource.
+ scoped_ptr<Resource> other_resource(
+ resource_manager_->CreateInputResourceAbsolute(cuppa_string,
+ &options_,
+ &message_handler_));
+ ASSERT_TRUE(other_resource != NULL);
+ cuppa_string.clear();
+ EXPECT_TRUE(resource_manager_->ReadIfCached(other_resource.get(),
+ &message_handler_));
+ std::string other_contents;
+ cuppa_resource->contents().CopyToString(&other_contents);
+ ASSERT_EQ(cuppa_contents, other_contents);
+ }
+};
+
+TEST_F(ImageRewriteTest, ImgTag) {
+ RewriteImage("img");
+}
+
+TEST_F(ImageRewriteTest, InputTag) {
+ RewriteImage("input type=\"image\"");
+}
+
+TEST_F(ImageRewriteTest, DataUrlTest) {
+ DataUrlResource();
+}
+
+TEST_F(ImageRewriteTest, RespectsBaseUrl) {
+ // Put original files into our fetcher.
+ const char html_url[] = "http://image.test/base_url.html";
+ const char png_url[] = "http://other_domain.test/foo/bar/a.png";
+ const char jpeg_url[] = "http://other_domain.test/baz/b.jpeg";
+
+ AddFileToMockFetcher(png_url,
+ StrCat(GTestSrcDir(), kTestData, "BikeCrashIcn.png"),
+ kContentTypePng);
+ AddFileToMockFetcher(jpeg_url,
+ StrCat(GTestSrcDir(), kTestData, "Puzzle.jpg"),
+ kContentTypeJpeg);
+
+ // Second stylesheet is on other domain.
+ const char html_input[] =
+ "<head>\n"
+ " <base href='http://other_domain.test/foo/'>\n"
+ "</head>\n"
+ "<body>\n"
+ " <img src='bar/a.png'>\n"
+ " <img src='/baz/b.jpeg'>\n"
+ "</body>";
+
+ // Rewrite
+ AddFilter(RewriteOptions::kRewriteImages);
+ ParseUrl(html_url, html_input);
+
+ // Check for CSS files in the rewritten page.
+ StringVector image_urls;
+ CollectImgSrcs("base_url-links", output_buffer_, &image_urls);
+ EXPECT_EQ(2UL, image_urls.size());
+ const std::string& new_png_url = image_urls[0];
+ const std::string& new_jpeg_url = image_urls[1];
+
+ // Sanity check that we changed the URL.
+ EXPECT_NE("a.png", new_png_url);
+ EXPECT_NE("b.jpeg", new_jpeg_url);
+
+ LOG(INFO) << "new_png_url: " << new_png_url;
+ LOG(INFO) << "new_jpeg_url: " << new_jpeg_url;
+
+ const char expected_output_format[] =
+ "<head>\n"
+ " <base href='http://other_domain.test/foo/'>\n"
+ "</head>\n"
+ "<body>\n"
+ " <img src='%s'>\n"
+ " <img src='%s'>\n"
+ "</body>";
+ std::string expected_output = StringPrintf(expected_output_format,
+ new_png_url.c_str(),
+ new_jpeg_url.c_str());
+
+ EXPECT_EQ(AddHtmlBody(expected_output), output_buffer_);
+
+ GURL new_png_gurl = GoogleUrl::Create(new_png_url);
+ EXPECT_TRUE(new_png_gurl.is_valid());
+ EXPECT_EQ("other_domain.test", new_png_gurl.host());
+
+ GURL new_jpeg_gurl = GoogleUrl::Create(new_jpeg_url);
+ EXPECT_TRUE(new_jpeg_gurl.is_valid());
+ EXPECT_EQ("other_domain.test", new_jpeg_gurl.host());
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/img_rewrite_filter.cc b/trunk/src/net/instaweb/rewriter/img_rewrite_filter.cc
new file mode 100644
index 0000000..2b1ffce
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/img_rewrite_filter.cc
@@ -0,0 +1,474 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/rewriter/public/img_rewrite_filter.h"
+
+#include "base/logging.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/rewriter/public/image.h"
+#include "net/instaweb/rewriter/public/image_dim.h"
+#include "net/instaweb/rewriter/public/img_tag_scanner.h"
+#include "net/instaweb/rewriter/public/output_resource.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/util/public/atom.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/data_url.h"
+#include "net/instaweb/util/public/file_system.h"
+#include <string>
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/meta_data.h"
+#include "net/instaweb/util/public/statistics.h"
+#include "net/instaweb/util/public/statistics_work_bound.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/url_escaper.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// Rewritten image must be < kMaxRewrittenRatio * origSize to be worth
+// redirecting references to it.
+// TODO(jmaessen): Make this ratio adjustable.
+const double kMaxRewrittenRatio = 1.0;
+
+// Re-scale image if area / originalArea < kMaxAreaRatio
+// Should probably be much less than 1 due to jpeg quality loss.
+// Might need to differ depending upon img format.
+// TODO(jmaessen): Make adjustable.
+const double kMaxAreaRatio = 1.0;
+
+// We overload some http status codes for our own purposes
+
+// This is used to retain the knowledge that a particular image is not
+// profitable to optimize. According to pagespeed, 200, 203, 206, and
+// 304 are cacheable. So we must select from those.
+const HttpStatus::Code kNotOptimizable = HttpStatus::kNotModified; // 304
+
+// names for Statistics variables.
+const char kImageRewrites[] = "image_rewrites";
+const char kImageRewriteSavedBytes[] = "image_rewrite_saved_bytes";
+const char kImageInline[] = "image_inline";
+
+// name for statistic used to bound rewriting work.
+const char kImageOngoingRewrites[] = "image_ongoing_rewrites";
+
+} // namespace
+
+
+ImageUrlEncoder::ImageUrlEncoder(UrlEscaper* url_escaper, ImageDim* stored_dim)
+ : url_escaper_(url_escaper), stored_dim_(stored_dim) { }
+
+ImageUrlEncoder::~ImageUrlEncoder() { }
+
+void ImageUrlEncoder::EncodeToUrlSegment(
+ const StringPiece& origin_url, std::string* rewritten_url) {
+ stored_dim_->EncodeTo(rewritten_url);
+ url_escaper_->EncodeToUrlSegment(origin_url, rewritten_url);
+}
+
+bool ImageUrlEncoder::DecodeFromUrlSegment(
+ const StringPiece& rewritten_url, std::string* origin_url) {
+ // Note that "remaining" is shortened from the left as we parse.
+ StringPiece remaining(rewritten_url.data(), rewritten_url.size());
+ return (stored_dim_->DecodeFrom(&remaining) &&
+ url_escaper_->DecodeFromUrlSegment(remaining, origin_url));
+}
+
+ImgRewriteFilter::ImgRewriteFilter(RewriteDriver* driver,
+ bool log_image_elements,
+ bool insert_image_dimensions,
+ StringPiece path_prefix,
+ size_t img_inline_max_bytes,
+ size_t img_max_rewrites_at_once)
+ : RewriteFilter(driver, path_prefix),
+ file_system_(driver->file_system()),
+ html_parse_(driver->html_parse()),
+ img_filter_(new ImgTagScanner(html_parse_)),
+ resource_manager_(driver->resource_manager()),
+ img_inline_max_bytes_(img_inline_max_bytes),
+ log_image_elements_(log_image_elements),
+ insert_image_dimensions_(insert_image_dimensions),
+ s_width_(html_parse_->Intern("width")),
+ s_height_(html_parse_->Intern("height")),
+ rewrite_count_(NULL),
+ inline_count_(NULL),
+ rewrite_saved_bytes_(NULL) {
+ Statistics* stats = resource_manager_->statistics();
+ Variable* ongoing_rewrites = NULL;
+ if (stats != NULL) {
+ rewrite_count_ = stats->GetVariable(kImageRewrites);
+ rewrite_saved_bytes_ = stats->GetVariable(
+ kImageRewriteSavedBytes);
+ inline_count_ = stats->GetVariable(kImageInline);
+ ongoing_rewrites = stats->GetVariable(kImageOngoingRewrites);
+ }
+ work_bound_.reset(
+ new StatisticsWorkBound(ongoing_rewrites, img_max_rewrites_at_once));
+}
+
+void ImgRewriteFilter::Initialize(Statistics* statistics) {
+ statistics->AddVariable(kImageInline);
+ statistics->AddVariable(kImageRewriteSavedBytes);
+ statistics->AddVariable(kImageRewrites);
+
+ statistics->AddVariable(kImageOngoingRewrites);
+}
+
+void ImgRewriteFilter::OptimizeImage(
+ const Resource& input_resource, const ImageDim& page_dim,
+ Image* image, OutputResource* result) {
+ if (result != NULL && !result->IsWritten() && image != NULL &&
+ work_bound_->TryToWork()) {
+ ImageDim img_dim;
+ image->Dimensions(&img_dim);
+ const char* message; // Informational message for logging only.
+ if (page_dim.valid() && img_dim.valid()) {
+ int64 page_area =
+ static_cast<int64>(page_dim.width()) * page_dim.height();
+ int64 img_area = static_cast<int64>(img_dim.width()) * img_dim.height();
+ if (page_area < img_area * kMaxAreaRatio) {
+ if (image->ResizeTo(page_dim)) {
+ message = "Resized image";
+ } else {
+ message = "Couldn't resize image";
+ }
+ } else {
+ message = "Not worth resizing image";
+ }
+ html_parse_->InfoHere("%s `%s' from %dx%d to %dx%d", message,
+ input_resource.url().c_str(),
+ img_dim.width(), img_dim.height(),
+ page_dim.width(), page_dim.height());
+ }
+ // Unconditionally write resource back so we don't re-attempt optimization.
+ MessageHandler* message_handler = html_parse_->message_handler();
+
+ int64 origin_expire_time_ms = input_resource.CacheExpirationTimeMs();
+ if (image->output_size() <
+ image->input_size() * kMaxRewrittenRatio) {
+ if (resource_manager_->Write(
+ HttpStatus::kOK, image->Contents(), result,
+ origin_expire_time_ms, message_handler)) {
+ html_parse_->InfoHere(
+ "Shrinking image `%s' (%u bytes) to `%s' (%u bytes)",
+ input_resource.url().c_str(),
+ static_cast<unsigned>(image->input_size()),
+ result->url().c_str(),
+ static_cast<unsigned>(image->output_size()));
+
+ if (rewrite_saved_bytes_ != NULL) {
+ // Note: if we are serving a request from a different server
+ // than the server that rewrote the <img> tag, and they don't
+ // share a file system, then we will be bumping the byte-count
+ // here without bumping the rewrite count. This seems ok,
+ // though perhaps we may need to revisit.
+ //
+ // Currently this will be a problem even when serving on a
+ // different file that *does* share a filesystem,
+ // HashResourceManager does not yet load its internal map
+ // by scanning the filesystem on startup.
+ rewrite_saved_bytes_->Add(
+ image->input_size() - image->output_size());
+ }
+ }
+ } else {
+ // Write nothing and set status code to indicate not to rewrite
+ // in future.
+ resource_manager_->Write(kNotOptimizable, "", result,
+ origin_expire_time_ms, message_handler);
+ }
+ work_bound_->WorkComplete();
+ }
+}
+
+Image* ImgRewriteFilter::GetImage(const StringPiece& origin_url,
+ Resource* img_resource) {
+ Image* image = NULL;
+ MessageHandler* message_handler = html_parse_->message_handler();
+ if (img_resource == NULL) {
+ html_parse_->WarningHere("no input resource for %s",
+ origin_url.as_string().c_str());
+ } else if (!resource_manager_->ReadIfCached(img_resource, message_handler)) {
+ html_parse_->WarningHere("%s wasn't loaded",
+ img_resource->url().c_str());
+ } else if (!img_resource->ContentsValid()) {
+ html_parse_->WarningHere("Img contents from %s are invalid.",
+ img_resource->url().c_str());
+ } else {
+ image = new Image(img_resource->contents(), img_resource->url(),
+ resource_manager_->filename_prefix(), file_system_,
+ message_handler);
+ }
+ return image;
+}
+
+bool ImgRewriteFilter::OptimizedImageFor(
+ const StringPiece& origin_url, const ImageDim& page_dim,
+ Resource* img_resource, OutputResource* output) {
+ scoped_ptr<Image> image(GetImage(origin_url, img_resource));
+ bool ok = image.get() != NULL && image->image_type() != Image::IMAGE_UNKNOWN;
+ if (ok) {
+ OptimizeImage(*img_resource, page_dim, image.get(), output);
+ }
+ return ok;
+}
+
+// Convert (possibly NULL) Image* to corresponding (possibly NULL) ContentType*
+const ContentType* ImgRewriteFilter::ImageToContentType(
+ const std::string& origin_url, Image* image) {
+ const ContentType* content_type = NULL;
+ if (image != NULL) {
+ // Even if we know the content type from the extension coming
+ // in, the content-type can change as a result of compression,
+ // e.g. gif to png, or anything to vp8.
+ switch (image->image_type()) {
+ case Image::IMAGE_JPEG:
+ content_type = &kContentTypeJpeg;
+ break;
+ case Image::IMAGE_PNG:
+ content_type = &kContentTypePng;
+ break;
+ case Image::IMAGE_GIF:
+ content_type = &kContentTypeGif;
+ break;
+ default:
+ html_parse_->InfoHere(
+ "Cannot detect content type of image url `%s`",
+ origin_url.c_str());
+ break;
+ }
+ }
+ return content_type;
+}
+
+void ImgRewriteFilter::RewriteImageUrl(HtmlElement* element,
+ HtmlElement::Attribute* src) {
+ // TODO(jmaessen): content type can change after re-compression.
+ // How do we deal with that given only URL?
+ // Separate input and output content type?
+ MessageHandler* message_handler = html_parse_->message_handler();
+ scoped_ptr<Resource> input_resource(CreateInputResourceAndReadIfCached(
+ src->value()));
+ if (input_resource != NULL && input_resource->ContentsValid()) {
+ ImageDim page_dim;
+ // Always rewrite to absolute url used to obtain resource.
+ // This lets us do context-free fetches of content.
+ int width, height;
+ if (element->IntAttributeValue(s_width_, &width) &&
+ element->IntAttributeValue(s_height_, &height)) {
+ // Specific image size is called for. Rewrite to that size.
+ page_dim.set_dims(width, height);
+ }
+ scoped_ptr<Image> image(GetImage(src->value(), input_resource.get()));
+ const ContentType* content_type =
+ ImageToContentType(src->value(), image.get());
+
+ if (content_type != NULL) {
+ ImageDim actual_dim;
+ image->Dimensions(&actual_dim);
+ // Create an output resource and fetch it, as that will tell
+ // us if we have already optimized it, or determined that it was not
+ // worth optimizing.
+ std::string rewritten_name;
+ ImageUrlEncoder encoder(resource_manager_->url_escaper(), &page_dim);
+ scoped_ptr<OutputResource> output_resource(
+ resource_manager_->CreateOutputResourceFromResource(
+ filter_prefix_, content_type, &encoder, input_resource.get(),
+ message_handler));
+ if (output_resource.get() != NULL) {
+ if (!resource_manager_->FetchOutputResource(
+ output_resource.get(), NULL, NULL, message_handler,
+ ResourceManager::kNeverBlock)) {
+ OptimizeImage(*input_resource, page_dim, image.get(),
+ output_resource.get());
+ }
+ if (output_resource->IsWritten()) {
+ UpdateTargetElement(*input_resource, *output_resource,
+ page_dim, actual_dim, element, src);
+ }
+ }
+ }
+ }
+}
+
+bool ImgRewriteFilter::CanInline(
+ int img_inline_max_bytes, const StringPiece& contents,
+ const ContentType* content_type, std::string* data_url) {
+ bool ok = false;
+ if (content_type != NULL && contents.size() <= img_inline_max_bytes) {
+ DataUrl(*content_type, BASE64, contents, data_url);
+ ok = true;
+ }
+ return ok;
+}
+
+// Given image processing reflected in the already-written output_resource,
+// actually update the element (particularly the src attribute), and log
+// statistics on what happened.
+void ImgRewriteFilter::UpdateTargetElement(
+ const Resource& input_resource, const OutputResource& output_resource,
+ const ImageDim& page_dim, const ImageDim& actual_dim,
+ HtmlElement* element, HtmlElement::Attribute* src) {
+
+ if (actual_dim.valid() &&
+ (actual_dim.width() > 1 || actual_dim.height() > 1)) {
+ std::string inlined_url;
+ bool output_ok =
+ output_resource.metadata()->status_code() == HttpStatus::kOK;
+ bool ie6or7 = driver_->user_agent().IsIe6or7();
+ if (!ie6or7 &&
+ ((output_ok &&
+ CanInline(img_inline_max_bytes_, output_resource.contents(),
+ output_resource.type(), &inlined_url)) ||
+ CanInline(img_inline_max_bytes_, input_resource.contents(),
+ input_resource.type(), &inlined_url))) {
+ src->SetValue(inlined_url);
+ if (inline_count_ != NULL) {
+ inline_count_->Add(1);
+ }
+ } else {
+ if (output_ok) {
+ src->SetValue(output_resource.url());
+ if (rewrite_count_ != NULL) {
+ rewrite_count_->Add(1);
+ }
+ }
+ if (insert_image_dimensions_ && actual_dim.valid() && !page_dim.valid() &&
+ !element->FindAttribute(s_width_) &&
+ !element->FindAttribute(s_height_)) {
+ // Add image dimensions. We don't bother if even a single image
+ // dimension is already specified---even though we don't resize in that
+ // case, either, because we might be off by a pixel in the other
+ // dimension from the size chosen by the browser. We also don't bother
+ // to resize if either dimension is specified with units (px, em, %)
+ // rather than as absolute pixes. But note that we DO attempt to
+ // include image dimensions even if we otherwise choose not to optimize
+ // an image. This may require examining the image contents if we didn't
+ // just perform the image processing.
+ element->AddAttribute(s_width_, actual_dim.width());
+ element->AddAttribute(s_height_, actual_dim.height());
+ }
+ }
+ }
+}
+
+void ImgRewriteFilter::EndElementImpl(HtmlElement* element) {
+ HtmlElement::Attribute *src = img_filter_->ParseImgElement(element);
+ if (src != NULL) {
+ if (log_image_elements_) {
+ // We now know that element is an img tag.
+ // Log the element in its original form.
+ std::string tagstring;
+ element->ToString(&tagstring);
+ html_parse_->Info(
+ html_parse_->id(), element->begin_line_number(),
+ "Found image: %s", tagstring.c_str());
+ }
+ RewriteImageUrl(element, src);
+ }
+}
+
+void ImgRewriteFilter::Flush() {
+ // TODO(jmaessen): wait here for resources to have been rewritten??
+}
+
+bool ImgRewriteFilter::Fetch(OutputResource* resource,
+ Writer* writer,
+ const MetaData& request_header,
+ MetaData* response_headers,
+ MessageHandler* message_handler,
+ UrlAsyncFetcher::Callback* callback) {
+ bool ok = true;
+ const char* failure_reason = "";
+ const ContentType *content_type = resource->type();
+ if (content_type != NULL) {
+ ImageDim page_dim; // Dimensions given in source page (invalid if absent).
+ ImageUrlEncoder encoder(resource_manager_->url_escaper(), &page_dim);
+ scoped_ptr<Resource> input_image(
+ CreateInputResourceFromOutputResource(&encoder, resource));
+ if (input_image.get() != NULL) {
+ std::string origin_url = input_image->url();
+ // TODO(jmarantz): this needs to be refactored slightly to allow for
+ // asynchronous fetches of the input image, if it's not obtained via cache
+ // or local filesystem read.
+ if (!OptimizedImageFor(
+ origin_url, page_dim, input_image.get(), resource)) {
+ ok = false;
+ failure_reason = "Server could not find source image.";
+ } else if (!resource_manager_->FetchOutputResource(
+ resource, writer, response_headers, message_handler,
+ ResourceManager::kNeverBlock)) {
+ ok = false;
+ failure_reason = "Server could not read image resource.";
+ } else {
+ if (resource->metadata()->status_code() != HttpStatus::kOK) {
+ // Note that this should not happen, because the url should not have
+ // escaped into the wild. We're content serving an empty response if
+ // it does. We *could* serve / redirect to the origin_url as a fail
+ // safe, but it's probably not worth it. Instead we log and hope that
+ // this causes us to find and fix the problem.
+ message_handler->Error(resource->url().c_str(), 0,
+ "Rewriting %s rejected, "
+ "but URL requested (mistaken rewriting?).",
+ origin_url.c_str());
+ }
+ callback->Done(true);
+ }
+ // Image processing has failed, forward the original image data.
+ if (!ok && input_image != NULL) {
+ if (input_image->ContentsValid()) {
+ ok = writer->Write(input_image->contents(), message_handler);
+ }
+ if (ok) {
+ resource_manager_->SetDefaultHeaders(content_type, response_headers);
+ } else {
+ message_handler->Error(resource->name().as_string().c_str(), 0,
+ "%s", failure_reason);
+ ok = writer->Write("<img src=\"", message_handler);
+ ok &= writer->Write(origin_url, message_handler);
+ ok &= writer->Write("\" alt=\"Temporarily Moved\"/>",
+ message_handler);
+ response_headers->set_major_version(1);
+ response_headers->set_minor_version(1);
+ response_headers->SetStatusAndReason(HttpStatus::kTemporaryRedirect);
+ response_headers->Add(HttpAttributes::kLocation, origin_url.c_str());
+ response_headers->Add(HttpAttributes::kContentType, "text/html");
+ }
+ if (ok) {
+ callback->Done(true);
+ }
+ }
+ }
+ } else {
+ ok = false;
+ failure_reason = "Unrecognized image content type.";
+ }
+ if (!ok) {
+ writer->Write(failure_reason, message_handler);
+ response_headers->set_status_code(HttpStatus::kNotFound);
+ response_headers->set_reason_phrase(failure_reason);
+ message_handler->Error(resource->name().as_string().c_str(), 0,
+ "%s", failure_reason);
+ }
+ return ok;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/img_tag_scanner.cc b/trunk/src/net/instaweb/rewriter/img_tag_scanner.cc
new file mode 100644
index 0000000..c6db89f
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/img_tag_scanner.cc
@@ -0,0 +1,51 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/rewriter/public/img_tag_scanner.h"
+
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/util/public/atom.h"
+
+namespace net_instaweb {
+
+ImgTagScanner::ImgTagScanner(HtmlParse* html_parse)
+ : s_img_(html_parse->Intern("img")),
+ s_input_(html_parse->Intern("input")),
+ s_src_(html_parse->Intern("src")),
+ s_type_(html_parse->Intern("type")) {
+}
+
+HtmlElement::Attribute*
+ImgTagScanner::ParseImgElement(HtmlElement* element) const {
+ // Return the src attribute of an <img> tag.
+ if (element->tag() == s_img_) {
+ return element->FindAttribute(s_src_);
+ }
+ // Return the src attribute of an <input type="image"> tag.
+ // See http://code.google.com/p/modpagespeed/issues/detail?id=86
+ if (element->tag() == s_input_) {
+ const char* type = element->AttributeValue(s_type_);
+ if (type != NULL && strcmp(type, "image") == 0) {
+ return element->FindAttribute(s_src_);
+ }
+ }
+ return NULL;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/javascript_code_block.cc b/trunk/src/net/instaweb/rewriter/javascript_code_block.cc
new file mode 100644
index 0000000..0cc2968
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/javascript_code_block.cc
@@ -0,0 +1,107 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/rewriter/public/javascript_code_block.h"
+
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/statistics.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "pagespeed/jsminify/js_minify.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// Statistics names
+const char kJavascriptBlocksMinified[] = "javascript_blocks_minified";
+const char kJavascriptBytesSaved[] = "javascript_bytes_saved";
+const char kJavascriptMinificationFailures[] =
+ "javascript_minification_failures";
+const char kJavascriptTotalBlocks[] = "javascript_total_blocks";
+
+} // namespace
+
+JavascriptRewriteConfig::JavascriptRewriteConfig(Statistics* stats)
+ : minify_(true),
+ redirect_(true),
+ blocks_minified_(NULL),
+ bytes_saved_(NULL),
+ minification_failures_(NULL),
+ total_blocks_(NULL) {
+ if (stats != NULL) {
+ blocks_minified_ = stats->GetVariable(kJavascriptBlocksMinified);
+ bytes_saved_ = stats->GetVariable(kJavascriptBytesSaved);
+ minification_failures_ =
+ stats->GetVariable(kJavascriptMinificationFailures);
+ total_blocks_ = stats->GetVariable(kJavascriptTotalBlocks);
+ }
+}
+
+void JavascriptRewriteConfig::Initialize(Statistics* statistics) {
+ statistics->AddVariable(kJavascriptBlocksMinified);
+ statistics->AddVariable(kJavascriptBytesSaved);
+ statistics->AddVariable(kJavascriptMinificationFailures);
+ statistics->AddVariable(kJavascriptTotalBlocks);
+}
+
+JavascriptCodeBlock::JavascriptCodeBlock(
+ const StringPiece& original_code, JavascriptRewriteConfig* config,
+ MessageHandler* handler)
+ : config_(config),
+ handler_(handler),
+ original_code_(original_code.data(), original_code.size()),
+ output_code_(original_code_),
+ rewritten_(false) { }
+
+JavascriptCodeBlock::~JavascriptCodeBlock() { }
+
+const JavascriptLibraryId JavascriptCodeBlock::ComputeJavascriptLibrary() {
+ // We always RewriteIfNecessary just to provide a degree of
+ // predictability to the rewrite flow.
+ RewriteIfNecessary();
+ if (!config_->redirect()) {
+ return JavascriptLibraryId();
+ }
+ return JavascriptLibraryId::Find(rewritten_code_);
+}
+
+void JavascriptCodeBlock::Rewrite() {
+ // Before attempting library identification, we minify. However, we only
+ // point output_code_ at the minified code if we actually want to serve it to
+ // the rest of the universe.
+ config_->AddBlock();
+ if ((config_->minify() || config_->redirect())) {
+ if (!pagespeed::jsminify::MinifyJs(original_code_, &rewritten_code_)) {
+ handler_->Message(kError, "Minification failed. Preserving old code.");
+ config_->AddMinificationFailure();
+ TrimWhitespace(original_code_, &rewritten_code_);
+ }
+ } else {
+ TrimWhitespace(original_code_, &rewritten_code_);
+ }
+
+ if (config_->minify()) {
+ output_code_ = rewritten_code_;
+ if (rewritten_code_.size() < original_code_.size()) {
+ size_t savings = original_code_.size() - rewritten_code_.size();
+ config_->AddBytesSaved(savings);
+ }
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/javascript_code_block_test.cc b/trunk/src/net/instaweb/rewriter/javascript_code_block_test.cc
new file mode 100644
index 0000000..261f31b
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/javascript_code_block_test.cc
@@ -0,0 +1,213 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/rewriter/public/javascript_code_block.h"
+
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/simple_stats.h"
+#include <string>
+
+namespace net_instaweb {
+
+namespace {
+
+// This sample code comes from Douglas Crockford's jsmin example.
+// The same code is used to test jsminify in pagespeed.
+const std::string kBeforeCompilation =
+ "// is.js\n"
+ "\n"
+ "// (c) 2001 Douglas Crockford\n"
+ "// 2001 June 3\n"
+ "\n"
+ "\n"
+ "// is\n"
+ "\n"
+ "// The -is- object is used to identify the browser. "
+ "Every browser edition\n"
+ "// identifies itself, but there is no standard way of doing it, "
+ "and some of\n"
+ "// the identification is deceptive. This is because the authors of web\n"
+ "// browsers are liars. For example, Microsoft's IE browsers claim to be\n"
+ "// Mozilla 4. Netscape 6 claims to be version 5.\n"
+ "\n"
+ "var is = {\n"
+ " ie: navigator.appName == 'Microsoft Internet Explorer',\n"
+ " java: navigator.javaEnabled(),\n"
+ " ns: navigator.appName == 'Netscape',\n"
+ " ua: navigator.userAgent.toLowerCase(),\n"
+ " version: parseFloat(navigator.appVersion.substr(21)) ||\n"
+ " parseFloat(navigator.appVersion),\n"
+ " win: navigator.platform == 'Win32'\n"
+ "}\n"
+ "is.mac = is.ua.indexOf('mac') >= 0;\n"
+ "if (is.ua.indexOf('opera') >= 0) {\n"
+ " is.ie = is.ns = false;\n"
+ " is.opera = true;\n"
+ "}\n"
+ "if (is.ua.indexOf('gecko') >= 0) {\n"
+ " is.ie = is.ns = false;\n"
+ " is.gecko = true;\n"
+ "}\n";
+
+const std::string kTruncatedComment =
+ "// is.js\n"
+ "\n"
+ "// (c) 2001 Douglas Crockford\n"
+ "// 2001 June 3\n"
+ "\n"
+ "\n"
+ "// is\n"
+ "\n"
+ "/* The -is- object is used to identify the browser. "
+ "Every browser edition\n"
+ " identifies itself, but there is no standard way of doing it, "
+ "and some of\n";
+
+const std::string kTruncatedRewritten =
+ "// is.js\n"
+ "\n"
+ "// (c) 2001 Douglas Crockford\n"
+ "// 2001 June 3\n"
+ "\n"
+ "\n"
+ "// is\n"
+ "\n"
+ "/* The -is- object is used to identify the browser. "
+ "Every browser edition\n"
+ " identifies itself, but there is no standard way of doing it, "
+ "and some of";
+
+const std::string kTruncatedString =
+ "var is = {\n"
+ " ie: navigator.appName == 'Microsoft Internet Explo";
+
+const std::string kAfterCompilation =
+ "var is={ie:navigator.appName=='Microsoft Internet Explorer',"
+ "java:navigator.javaEnabled(),ns:navigator.appName=='Netscape',"
+ "ua:navigator.userAgent.toLowerCase(),version:parseFloat("
+ "navigator.appVersion.substr(21))||parseFloat(navigator.appVersion)"
+ ",win:navigator.platform=='Win32'}\n"
+ "is.mac=is.ua.indexOf('mac')>=0;if(is.ua.indexOf('opera')>=0){"
+ "is.ie=is.ns=false;is.opera=true;}\n"
+ "if(is.ua.indexOf('gecko')>=0){is.ie=is.ns=false;is.gecko=true;}";
+
+const char kJavascriptBlocksMinified[] = "javascript_blocks_minified";
+const char kJavascriptBytesSaved[] = "javascript_bytes_saved";
+const char kJavascriptMinificationFailures[] =
+ "javascript_minification_failures";
+const char kJavascriptTotalBlocks[] = "javascript_total_blocks";
+
+void ExpectStats(const SimpleStats& stats,
+ int total_blocks, int minified_blocks, int failures,
+ int saved_bytes) {
+ EXPECT_EQ(minified_blocks,
+ stats.GetVariable(kJavascriptBlocksMinified)->Get());
+ EXPECT_EQ(total_blocks,
+ stats.GetVariable(kJavascriptTotalBlocks)->Get());
+ EXPECT_EQ(failures,
+ stats.GetVariable(kJavascriptMinificationFailures)->Get());
+ EXPECT_EQ(saved_bytes,
+ stats.GetVariable(kJavascriptBytesSaved)->Get());
+}
+
+TEST(JsCodeBlockTest, Config) {
+ SimpleStats stats;
+ JavascriptRewriteConfig::Initialize(&stats);
+ JavascriptRewriteConfig config(&stats);
+ EXPECT_TRUE(config.minify());
+ config.set_minify(false);
+ EXPECT_FALSE(config.minify());
+ config.set_minify(true);
+ EXPECT_TRUE(config.minify());
+ ExpectStats(stats, 0, 0, 0, 0);
+}
+
+TEST(JsCodeBlockTest, Rewrite) {
+ SimpleStats stats;
+ JavascriptRewriteConfig::Initialize(&stats);
+ JavascriptRewriteConfig config(&stats);
+ GoogleMessageHandler handler;
+ JavascriptCodeBlock block(kBeforeCompilation, &config, &handler);
+ EXPECT_TRUE(block.ProfitableToRewrite());
+ EXPECT_EQ(kAfterCompilation, block.Rewritten());
+ ExpectStats(stats, 1, 1, 0,
+ kBeforeCompilation.size() - kAfterCompilation.size());
+}
+
+TEST(JsCodeBlockTest, NoRewrite) {
+ SimpleStats stats;
+ JavascriptRewriteConfig::Initialize(&stats);
+ JavascriptRewriteConfig config(&stats);
+ GoogleMessageHandler handler;
+ JavascriptCodeBlock block(kAfterCompilation, &config, &handler);
+ EXPECT_FALSE(block.ProfitableToRewrite());
+ EXPECT_EQ(kAfterCompilation, block.Rewritten());
+ ExpectStats(stats, 1, 0, 0, 0);
+}
+
+TEST(JsCodeBlockTest, TruncatedComment) {
+ SimpleStats stats;
+ JavascriptRewriteConfig::Initialize(&stats);
+ JavascriptRewriteConfig config(&stats);
+ GoogleMessageHandler handler;
+ JavascriptCodeBlock block(kTruncatedComment, &config, &handler);
+ EXPECT_TRUE(block.ProfitableToRewrite());
+ EXPECT_EQ(kTruncatedRewritten, block.Rewritten());
+ ExpectStats(stats, 1, 1, 1,
+ kTruncatedComment.size() - kTruncatedRewritten.size());
+}
+
+TEST(JsCodeBlockTest, TruncatedString) {
+ SimpleStats stats;
+ JavascriptRewriteConfig::Initialize(&stats);
+ JavascriptRewriteConfig config(&stats);
+ GoogleMessageHandler handler;
+ JavascriptCodeBlock block(kTruncatedString, &config, &handler);
+ EXPECT_FALSE(block.ProfitableToRewrite());
+ EXPECT_EQ(kTruncatedString, block.Rewritten());
+ ExpectStats(stats, 1, 0, 1, 0);
+}
+
+TEST(JsCodeBlockTest, NoMinification) {
+ SimpleStats stats;
+ JavascriptRewriteConfig::Initialize(&stats);
+ JavascriptRewriteConfig config(&stats);
+ config.set_minify(false);
+ GoogleMessageHandler handler;
+ JavascriptCodeBlock block(kBeforeCompilation, &config, &handler);
+ EXPECT_FALSE(block.ProfitableToRewrite());
+ EXPECT_EQ(kBeforeCompilation, block.Rewritten());
+ ExpectStats(stats, 1, 0, 0, 0);
+}
+
+TEST(JsCodeBlockTest, DealWithSgmlComment) {
+ SimpleStats stats;
+ JavascriptRewriteConfig::Initialize(&stats);
+ JavascriptRewriteConfig config(&stats);
+ GoogleMessageHandler handler;
+ const std::string original = " <!-- \nvar x = 1;\n //--> ";
+ const std::string expected = "var x=1;";
+ JavascriptCodeBlock block(original, &config, &handler);
+ EXPECT_TRUE(block.ProfitableToRewrite());
+ EXPECT_EQ(expected, block.Rewritten());
+ ExpectStats(stats, 1, 1, 0, original.size() - expected.size());
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/javascript_filter.cc b/trunk/src/net/instaweb/rewriter/javascript_filter.cc
new file mode 100644
index 0000000..f763585
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/javascript_filter.cc
@@ -0,0 +1,308 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/rewriter/public/javascript_filter.h"
+
+#include <ctype.h>
+#include "base/scoped_ptr.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_node.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/rewriter/public/javascript_code_block.h"
+#include "net/instaweb/rewriter/public/javascript_library_identification.h"
+#include "net/instaweb/rewriter/public/resource.h"
+#include "net/instaweb/rewriter/public/output_resource.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/util/public/atom.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/meta_data.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+#include "net/instaweb/util/public/url_escaper.h"
+#include "net/instaweb/util/public/writer.h"
+
+namespace net_instaweb {
+
+namespace {
+const HttpStatus::Code kNotOptimizable = HttpStatus::kNotModified;
+} // namespace
+
+
+JavascriptFilter::JavascriptFilter(RewriteDriver* driver,
+ const StringPiece& path_prefix)
+ : RewriteSingleResourceFilter(driver, path_prefix),
+ html_parse_(driver->html_parse()),
+ script_in_progress_(NULL),
+ script_src_(NULL),
+ resource_manager_(driver->resource_manager()),
+ some_missing_scripts_(false),
+ config_(driver->resource_manager()->statistics()),
+ s_script_(html_parse_->Intern("script")),
+ s_src_(html_parse_->Intern("src")),
+ s_type_(html_parse_->Intern("type")),
+ script_tag_scanner_(html_parse_) { }
+
+JavascriptFilter::~JavascriptFilter() { }
+
+void JavascriptFilter::Initialize(Statistics* statistics) {
+ JavascriptRewriteConfig::Initialize(statistics);
+}
+
+void JavascriptFilter::StartElementImpl(HtmlElement* element) {
+ CHECK(script_in_progress_ == NULL);
+
+ switch (script_tag_scanner_.ParseScriptElement(element, &script_src_)) {
+ case ScriptTagScanner::kJavaScript: {
+ script_in_progress_ = element;
+ if (script_src_ != NULL) {
+ html_parse_->InfoHere("Found script with src %s", script_src_->value());
+ }
+ break;
+ }
+ case ScriptTagScanner::kUnknownScript: {
+ std::string script_dump;
+ element->ToString(&script_dump);
+ html_parse_->InfoHere("Unrecognized script:'%s'", script_dump.c_str());
+ break;
+ }
+ case ScriptTagScanner::kNonScript:
+ break;
+ }
+}
+
+void JavascriptFilter::Characters(HtmlCharactersNode* characters) {
+ if (script_in_progress_ != NULL) {
+ // Note that we're keeping a vector of nodes here,
+ // and appending them lazily at the end. This is
+ // because there's usually only 1 HtmlCharactersNode involved,
+ // and we end up not actually needing to copy the string.
+ buffer_.push_back(characters);
+ }
+}
+
+// Flatten script fragments in buffer_, using script_buffer to hold
+// the data if necessary. Return a StringPiece referring to the data.
+const StringPiece JavascriptFilter::FlattenBuffer(std::string* script_buffer) {
+ const int buffer_size = buffer_.size();
+ if (buffer_.size() == 1) {
+ StringPiece result(buffer_[0]->contents());
+ return result;
+ } else {
+ for (int i = 0; i < buffer_size; i++) {
+ script_buffer->append(buffer_[i]->contents());
+ }
+ StringPiece result(*script_buffer);
+ return result;
+ }
+}
+
+void JavascriptFilter::RewriteInlineScript() {
+ const int buffer_size = buffer_.size();
+ if (buffer_size > 0) {
+ // First buffer up script data and minify it.
+ std::string script_buffer;
+ const StringPiece script = FlattenBuffer(&script_buffer);
+ MessageHandler* message_handler = html_parse_->message_handler();
+ JavascriptCodeBlock code_block(script, &config_, message_handler);
+ JavascriptLibraryId library = code_block.ComputeJavascriptLibrary();
+ if (library.recognized()) {
+ html_parse_->InfoHere("Script is %s %s",
+ library.name(), library.version());
+ }
+ if (code_block.ProfitableToRewrite()) {
+ // Now replace all CharactersNodes with a single CharactersNode containing
+ // the minified script.
+ HtmlCharactersNode* new_script =
+ html_parse_->NewCharactersNode(buffer_[0]->parent(),
+ code_block.Rewritten());
+ html_parse_->ReplaceNode(buffer_[0], new_script);
+ for (int i = 1; i < buffer_size; i++) {
+ html_parse_->DeleteElement(buffer_[i]);
+ }
+ }
+ }
+}
+
+// Load script resource located at the given URL,
+// on error report & return NULL (caller need not report)
+Resource* JavascriptFilter::ScriptAtUrl(const StringPiece& script_url) {
+ Resource* script_input = CreateInputResourceAndReadIfCached(script_url);
+ return script_input;
+}
+
+// Take script_out, which is derived from the script at script_url,
+// and write it to script_dest.
+// Returns true on success, reports failures itself.
+bool JavascriptFilter::WriteExternalScriptTo(
+ const Resource* script_resource,
+ const StringPiece& script_out, OutputResource* script_dest) {
+ bool ok = false;
+ MessageHandler* message_handler = html_parse_->message_handler();
+ int64 origin_expire_time_ms = script_resource->CacheExpirationTimeMs();
+ if (resource_manager_->Write(HttpStatus::kOK, script_out, script_dest,
+ origin_expire_time_ms, message_handler)) {
+ ok = true;
+ html_parse_->InfoHere("Rewrite script %s to %s",
+ script_resource->url().c_str(),
+ script_dest->url().c_str());
+ }
+ return ok;
+}
+
+// External script; minify and replace with rewritten version (also external).
+void JavascriptFilter::RewriteExternalScript() {
+ const StringPiece script_url(script_src_->value());
+ MessageHandler* message_handler = html_parse_->message_handler();
+ scoped_ptr<OutputResource> script_dest(
+ resource_manager_->CreateOutputResourceForRewrittenUrl(
+ base_gurl(), filter_prefix_, script_url,
+ &kContentTypeJavascript, resource_manager_->url_escaper(),
+ driver_->options(), message_handler));
+ if (script_dest != NULL) {
+ bool ok;
+ if (resource_manager_->FetchOutputResource(
+ script_dest.get(), NULL, NULL, message_handler,
+ ResourceManager::kNeverBlock)) {
+ // Only rewrite URL if we have usable rewritten data.
+ ok = script_dest->metadata()->status_code() == HttpStatus::kOK;
+ } else {
+ scoped_ptr<Resource> script_input(ScriptAtUrl(script_url));
+ ok = script_input != NULL;
+ if (ok) {
+ StringPiece script = script_input->contents();
+ MessageHandler* message_handler = html_parse_->message_handler();
+ JavascriptCodeBlock code_block(script, &config_, message_handler);
+ JavascriptLibraryId library = code_block.ComputeJavascriptLibrary();
+ if (library.recognized()) {
+ html_parse_->InfoHere("Script %s is %s %s",
+ script_input->url().c_str(),
+ library.name(), library.version());
+ }
+ ok = code_block.ProfitableToRewrite();
+ if (ok) {
+ ok = WriteExternalScriptTo(script_input.get(), code_block.Rewritten(),
+ script_dest.get());
+ } else {
+ // Rewriting happened but wasn't useful; remember this for later
+ // so we don't attempt to rewrite twice.
+ html_parse_->InfoHere("Script %s didn't shrink",
+ script_input->url().c_str());
+ int64 origin_expire_time_ms = script_input->CacheExpirationTimeMs();
+
+ // TODO(jmarantz): currently this will not work, because HTTPCache
+ // will not report a 'hit' on any status other than OK. This should
+ // be fixed by either:
+ // 1. adding a few other codes that HTTPCache will return hits for
+ // 2. using a special header to indicate failed-to-optimize.
+ resource_manager_->Write(
+ kNotOptimizable, "",
+ script_dest.get(), origin_expire_time_ms, message_handler);
+ }
+ } else {
+ some_missing_scripts_ = true;
+ }
+ }
+ if (ok) {
+ script_src_->SetValue(script_dest->url());
+ }
+ }
+ // Finally, note that the script might contain body data.
+ // We erase this if it is just whitespace; otherwise we leave it alone.
+ // The script body is ignored by all browsers we know of.
+ // However, various sources have encouraged using the body of an
+ // external script element to store a post-load callback.
+ // As this technique is preferable to storing callbacks in, say, html
+ // comments, we support it for now.
+ bool allSpaces = true;
+ for (size_t i = 0; allSpaces && i < buffer_.size(); ++i) {
+ const std::string& contents = buffer_[i]->contents();
+ for (size_t j = 0; allSpaces && j < contents.size(); ++j) {
+ char c = contents[j];
+ if (!isspace(c) && c != 0) {
+ html_parse_->WarningHere("Retaining contents of script tag"
+ " even though script is external.");
+ allSpaces = false;
+ }
+ }
+ }
+ for (size_t i = 0; allSpaces && i < buffer_.size(); ++i) {
+ html_parse_->DeleteElement(buffer_[i]);
+ }
+}
+
+// Reset state at end of script.
+void JavascriptFilter::CompleteScriptInProgress() {
+ buffer_.clear();
+ script_in_progress_ = NULL;
+ script_src_ = NULL;
+}
+
+void JavascriptFilter::EndElementImpl(HtmlElement* element) {
+ if (script_in_progress_ != NULL &&
+ html_parse_->IsRewritable(script_in_progress_) &&
+ html_parse_->IsRewritable(element)) {
+ if (element->tag() == s_script_) {
+ if (element->close_style() == HtmlElement::BRIEF_CLOSE) {
+ html_parse_->ErrorHere("Brief close of script tag (non-portable)");
+ }
+ if (script_src_ == NULL) {
+ RewriteInlineScript();
+ } else {
+ RewriteExternalScript();
+ }
+ CompleteScriptInProgress();
+ } else {
+ // Should not happen by construction (parser should not have tags here).
+ // Note that if we get here, this test *Will* fail; it is written
+ // out longhand to make diagnosis easier.
+ CHECK(script_in_progress_ == NULL);
+ }
+ }
+}
+
+void JavascriptFilter::Flush() {
+ // TODO(jmaessen): We can be smarter here if it turns out to be necessary (eg
+ // by buffering an in-progress script across the flush boundary).
+ if (script_in_progress_ != NULL) {
+ // Not actually an error!
+ html_parse_->InfoHere("Flush in mid-script; leaving script untouched.");
+ CompleteScriptInProgress();
+ some_missing_scripts_ = true;
+ }
+}
+
+void JavascriptFilter::IEDirective(HtmlIEDirectiveNode* directive) {
+ CHECK(script_in_progress_ == NULL);
+ // We presume an IE directive is concealing some js code.
+ some_missing_scripts_ = true;
+}
+
+bool JavascriptFilter::RewriteLoadedResource(const Resource* script_input,
+ OutputResource* output_resource) {
+ StringPiece script = script_input->contents();
+ std::string script_out;
+ JavascriptCodeBlock code_block(script, &config_,
+ html_parse_->message_handler());
+ return WriteExternalScriptTo(script_input,
+ code_block.Rewritten(), output_resource);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/javascript_filter_test.cc b/trunk/src/net/instaweb/rewriter/javascript_filter_test.cc
new file mode 100644
index 0000000..8514814
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/javascript_filter_test.cc
@@ -0,0 +1,159 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the javascript filter
+
+
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/cache_extender.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+#include "net/instaweb/rewriter/public/resource_namer.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/string_writer.h"
+
+namespace {
+
+const char kHtmlFormat[] =
+ "<script type='text/javascript' src='%s'></script>\n";
+
+const char kJsData[] = "alert ( 'hello, world!' ) ";
+const char kJsMinData[] = "alert('hello, world!')";
+const char kFilterId[] = "jm";
+const char kOrigJsName[] = "hello.js";
+const char kRewrittenJsName[] = "hello.js";
+const char kSourcePrefix[] = "http://test.com/";
+
+} // namespace
+
+namespace net_instaweb {
+
+class JavascriptFilterTest : public ResourceManagerTestBase {
+ protected:
+ virtual void SetUp() {
+ ResourceManagerTestBase::SetUp();
+ AddFilter(RewriteOptions::kRewriteJavascript);
+ ResourceNamer namer;
+ namer.set_id(kFilterId);
+ namer.set_name(kRewrittenJsName);
+ namer.set_ext("js");
+ namer.set_hash("0");
+ expected_rewritten_path_ = StrCat(kSourcePrefix, namer.Encode());
+ }
+
+ void InitTest(int64 ttl) {
+ InitMetaData(kOrigJsName, kContentTypeJavascript, kJsData, ttl);
+ }
+
+ // Generate HTML loading 3 resources with the specified URLs
+ std::string GenerateHtml(const char* a) {
+ return StringPrintf(kHtmlFormat, a);
+ }
+
+ std::string expected_rewritten_path_;
+};
+
+TEST_F(JavascriptFilterTest, DoRewrite) {
+ InitTest(100);
+ ValidateExpected("do_rewrite",
+ GenerateHtml(kOrigJsName).c_str(),
+ GenerateHtml(expected_rewritten_path_.c_str()).c_str());
+}
+
+TEST_F(JavascriptFilterTest, RewriteAlreadyCachedProperly) {
+ InitTest(100000000); // cached for a long time to begin with
+ // But we will rewrite because we can make the data smaller.
+ ValidateExpected("rewrite_despite_being_cached_properly",
+ GenerateHtml(kOrigJsName).c_str(),
+ GenerateHtml(expected_rewritten_path_.c_str()).c_str());
+}
+
+TEST_F(JavascriptFilterTest, NoRewriteOriginUncacheable) {
+ InitTest(0); // origin not cacheable
+ ValidateExpected("no_extend_origin_not_cacheable",
+ GenerateHtml(kOrigJsName).c_str(),
+ GenerateHtml(kOrigJsName).c_str());
+}
+
+TEST_F(JavascriptFilterTest, ServeFiles) {
+ std::string content;
+
+ // TODO(jmarantz): Factor some of this logic-flow out so that
+ // cache_extender_test.cc can share it.
+
+ // When we start, there are no mock fetchers, so we'll need to get it
+ // from the cache or the disk. Start with the cache.
+ file_system_.Disable();
+ SimpleMetaData headers;
+ resource_manager_->SetDefaultHeaders(&kContentTypeJavascript, &headers);
+ http_cache_.Put(expected_rewritten_path_, headers, kJsMinData,
+ &message_handler_);
+ EXPECT_EQ(0, lru_cache_->num_hits());
+ ASSERT_TRUE(ServeResource(kSourcePrefix, kFilterId,
+ kRewrittenJsName, "js", &content));
+ EXPECT_EQ(1, lru_cache_->num_hits());
+ EXPECT_EQ(std::string(kJsMinData), content);
+
+ // Now remove it from the cache, but put it in the file system. Make sure
+ // that works. Still there is no mock fetcher.
+ file_system_.Enable();
+ lru_cache_->Clear();
+
+ // Getting the filename is kind of a drag, isn't it. But someone's
+ // gotta do it.
+ // TODO(jmarantz): refactor this and share it with other filter_tests.
+ std::string filename;
+ FilenameEncoder* encoder = resource_manager_->filename_encoder();
+ encoder->Encode(resource_manager_->filename_prefix(),
+ expected_rewritten_path_, &filename);
+ std::string data = StrCat(headers.ToString(), kJsMinData);
+ ASSERT_TRUE(file_system_.WriteFile(filename.c_str(), data,
+ &message_handler_));
+
+ ASSERT_TRUE(ServeResource(kSourcePrefix, kFilterId,
+ kRewrittenJsName, "js", &content));
+ EXPECT_EQ(std::string(kJsMinData), content);
+
+ // After serving from the disk, we should have seeded our cache. Check it.
+ EXPECT_EQ(CacheInterface::kAvailable, http_cache_.Query(
+ expected_rewritten_path_));
+
+ // Finally, nuke the file, nuke the cache, get it via a fetch.
+ file_system_.Disable();
+ ASSERT_TRUE(file_system_.RemoveFile(filename.c_str(), &message_handler_));
+ lru_cache_->Clear();
+ InitTest(100);
+ ASSERT_TRUE(ServeResource(kSourcePrefix, kFilterId,
+ kRewrittenJsName, "js", &content));
+ EXPECT_EQ(std::string(kJsMinData), content);
+
+ // Now we expect both the file and the cache entry to be there.
+ EXPECT_EQ(CacheInterface::kAvailable, http_cache_.Query(
+ expected_rewritten_path_));
+ file_system_.Enable();
+ EXPECT_TRUE(file_system_.Exists(filename.c_str(), &message_handler_)
+ .is_true());
+
+ // Finally, serve from a completely separate server.
+ ServeResourceFromManyContexts(expected_rewritten_path_,
+ RewriteOptions::kRewriteJavascript,
+ &mock_hasher_,
+ kJsMinData);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/javascript_library_identification.cc b/trunk/src/net/instaweb/rewriter/javascript_library_identification.cc
new file mode 100644
index 0000000..7404d2a
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/javascript_library_identification.cc
@@ -0,0 +1,56 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/rewriter/public/javascript_library_identification.h"
+
+#include "net/instaweb/util/public/rolling_hash.h"
+#include "net/instaweb/util/public/string_util.h"
+
+// Include separately-generated library metadata (kLibraryMetadata)
+// whose 0th entry is unrecognized (NULL name).
+#include "net/instaweb/rewriter/javascript_metadata.cc"
+
+namespace net_instaweb {
+
+static const LibraryInfo& kUnrecognizedLibraryInfo = kLibraryMetadata[0];
+static const size_t kLibrarySize = arraysize(kLibraryMetadata);
+
+JavascriptLibraryId JavascriptLibraryId::Find(
+ const StringPiece& minified_code) {
+ const LibraryInfo* library = &kUnrecognizedLibraryInfo;
+ if (minified_code.size() >= kJavascriptHashIdBlockSize) {
+ uint64 block_hash =
+ RollingHash(minified_code.data(), 0, kJavascriptHashIdBlockSize);
+ // Right now we use a naive linear search.
+ // TODO(jmaessen): lazily-initialized search structure of some sort.
+ for (library = &kLibraryMetadata[kLibrarySize - 1];
+ library->name != NULL; --library) {
+ if (library->first_block_hash == block_hash &&
+ library->full_size == minified_code.size() &&
+ library->full_hash ==
+ RollingHash(minified_code.data(), 0, library->full_size)) {
+ break;
+ }
+ }
+ }
+ return JavascriptLibraryId(library);
+}
+
+JavascriptLibraryId::JavascriptLibraryId()
+ : info_(&kUnrecognizedLibraryInfo) { }
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/javascript_metadata.cc b/trunk/src/net/instaweb/rewriter/javascript_metadata.cc
new file mode 100644
index 0000000..d187157
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/javascript_metadata.cc
@@ -0,0 +1,32 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+// Generated by:
+// ./rewriter/generate_javascript_metadata.sh /home/jmaessen/jslibs/
+// Intended for inclusion in javascript_library_identification.cc
+
+namespace net_instaweb {
+
+static const LibraryInfo kLibraryMetadata[] = {
+ {NULL, NULL, 0ULL, 0ULL, 0},
+ { "prototype", "1.6.0.2",
+ 0x84e682bac0395621ULL, 0x754d81bbcddf212aULL, 93706},
+ { "prototype", "1.6.0.3",
+ 0x6d056fb74c5f131cULL, 0xe63b907ad7e2163cULL, 95347},
+ { "prototype", "1.6.1.0",
+ 0x6ddb883a16ace985ULL, 0xea6f76a9731dc118ULL, 105569}
+};
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/js_inline_filter.cc b/trunk/src/net/instaweb/rewriter/js_inline_filter.cc
new file mode 100644
index 0000000..3e65054
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/js_inline_filter.cc
@@ -0,0 +1,138 @@
+// Copyright 2010 Google Inc. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "net/instaweb/rewriter/public/js_inline_filter.h"
+
+#include "base/logging.h"
+#include "base/scoped_ptr.h"
+#include "base/string_util.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_node.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+#include "net/instaweb/util/public/google_url.h"
+
+namespace net_instaweb {
+
+JsInlineFilter::JsInlineFilter(RewriteDriver* driver)
+ : CommonFilter(driver),
+ html_parse_(driver->html_parse()),
+ script_atom_(html_parse_->Intern("script")),
+ src_atom_(html_parse_->Intern("src")),
+ size_threshold_bytes_(driver->options()->js_inline_max_bytes()),
+ script_tag_scanner_(html_parse_),
+ should_inline_(false) {}
+
+JsInlineFilter::~JsInlineFilter() {}
+
+void JsInlineFilter::StartDocumentImpl() {
+ // TODO(sligocki): This should go in the domain lawyer, right?
+ domain_ = html_parse_->gurl().host();
+ should_inline_ = false;
+}
+
+void JsInlineFilter::EndDocument() {
+ domain_.clear();
+}
+
+void JsInlineFilter::StartElementImpl(HtmlElement* element) {
+ DCHECK(!should_inline_);
+
+ HtmlElement::Attribute* src;
+ if (script_tag_scanner_.ParseScriptElement(element, &src) ==
+ ScriptTagScanner::kJavaScript) {
+ should_inline_ = (src != NULL) && (src->value() != NULL);
+ }
+}
+
+void JsInlineFilter::EndElementImpl(HtmlElement* element) {
+ if (should_inline_ && html_parse_->IsRewritable(element)) {
+ DCHECK(element->tag() == script_atom_);
+ const char* src = element->AttributeValue(src_atom_);
+ DCHECK(src != NULL);
+ should_inline_ = false;
+
+ // TODO(morlovich): Consider async/defer here; it may not be a good
+ // idea to inline async scripts in particular
+
+ scoped_ptr<Resource> resource(CreateInputResourceAndReadIfCached(src));
+ // TODO(jmaessen): Is the domain lawyer policy the appropriate one here?
+ // Or do we still have to check for strict domain equivalence?
+ // If so, add an inline-in-page policy to domainlawyer in some form,
+ // as we make a similar policy decision in css_inline_filter.
+ if (resource != NULL && resource->ContentsValid()) {
+ StringPiece contents = resource->contents();
+ // Only inline if it's small enough, and if it doesn't contain
+ // "</script>" anywhere. If we inline an external script containing
+ // "</script>", the <script> tag will be ended early.
+ // See http://code.google.com/p/modpagespeed/issues/detail?id=106
+ // TODO(mdsteele): We should consider rewriting "</script>" to
+ // "<\/script>" instead of just bailing. But we can't blindly search
+ // and replace because that would break legal (if contrived) code such
+ // as "if(x</script>/){...}", which is comparing x to a regex literal.
+ if (contents.size() <= size_threshold_bytes_ &&
+ contents.find("</script>") == StringPiece::npos) {
+ // If we're in XHTML, we should wrap the script in a <!CDATA[...]]>
+ // block to ensure that we don't break well-formedness. Since XHTML is
+ // sometimes interpreted as HTML (which will ignore CDATA delimiters),
+ // we have to hide the CDATA delimiters behind Javascript comments.
+ // See http://lachy.id.au/log/2006/11/xhtml-script
+ // and http://code.google.com/p/modpagespeed/issues/detail?id=125
+ if (html_parse_->doctype().IsXhtml()) {
+ // CDATA sections cannot be nested because they end with the first
+ // occurance of "]]>", so if the script contains that string
+ // anywhere (and we're in XHTML) we can't inline.
+ // TODO(mdsteele): Again, we should consider escaping somehow.
+ if (contents.find("]]>") == StringPiece::npos) {
+ HtmlCharactersNode* node =
+ html_parse_->NewCharactersNode(element, "//<![CDATA[\n");
+ node->Append(contents);
+ node->Append("\n//]]>");
+ html_parse_->InsertElementBeforeCurrent(node);
+ element->DeleteAttribute(src_atom_);
+ }
+ }
+ // If we're not in XHTML, we can simply paste in the external script
+ // verbatim.
+ else {
+ html_parse_->InsertElementBeforeCurrent(
+ html_parse_->NewCharactersNode(element, contents));
+ element->DeleteAttribute(src_atom_);
+ }
+ }
+ }
+ }
+}
+
+void JsInlineFilter::Characters(HtmlCharactersNode* characters) {
+ if (should_inline_) {
+ DCHECK(characters->parent() != NULL);
+ DCHECK(characters->parent()->tag() == script_atom_);
+ if (OnlyWhitespace(characters->contents())) {
+ // If it's just whitespace inside the script tag, it's (probably) safe to
+ // just remove it.
+ html_parse_->DeleteElement(characters);
+ } else {
+ // This script tag isn't empty, despite having a src field. The contents
+ // won't be executed by the browser, but will still be in the DOM; some
+ // external scripts like to use this as a place to store data. So, we'd
+ // better not try to inline in this case.
+ should_inline_ = false;
+ }
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/js_inline_filter_test.cc b/trunk/src/net/instaweb/rewriter/js_inline_filter_test.cc
new file mode 100644
index 0000000..10992b8
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/js_inline_filter_test.cc
@@ -0,0 +1,169 @@
+/*
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+
+namespace net_instaweb {
+
+namespace {
+
+class JsInlineFilterTest : public ResourceManagerTestBase {
+ protected:
+ void TestInlineJavascript(const std::string& html_url,
+ const std::string& js_url,
+ const std::string& js_original_inline_body,
+ const std::string& js_outline_body,
+ bool expect_inline) {
+ TestInlineJavascriptGeneral(
+ html_url,
+ "", // don't use a doctype for these tests
+ js_url,
+ js_original_inline_body,
+ js_outline_body,
+ js_outline_body, // expect ouline body to be inlined verbatim
+ expect_inline);
+ }
+
+ void TestInlineJavascriptXhtml(const std::string& html_url,
+ const std::string& js_url,
+ const std::string& js_outline_body,
+ bool expect_inline) {
+ TestInlineJavascriptGeneral(
+ html_url,
+ "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.1//EN\" "
+ "\"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd\">",
+ js_url,
+ "", // use an empty original inline body for these tests
+ js_outline_body,
+ // Expect outline body to get surrounded by a CDATA block:
+ "//<![CDATA[\n" + js_outline_body + "\n//]]>",
+ expect_inline);
+ }
+
+ void TestInlineJavascriptGeneral(const std::string& html_url,
+ const std::string& doctype,
+ const std::string& js_url,
+ const std::string& js_original_inline_body,
+ const std::string& js_outline_body,
+ const std::string& js_expected_inline_body,
+ bool expect_inline) {
+ AddFilter(RewriteOptions::kInlineJavascript);
+
+ // Specify the input and expected output.
+ if (!doctype.empty()) {
+ SetDoctype(doctype);
+ }
+ const std::string html_input =
+ "<head>\n"
+ " <script src=\"" + js_url + "\">" +
+ js_original_inline_body + "</script>\n"
+ "</head>\n"
+ "<body>Hello, world!</body>\n";
+ const std::string expected_output =
+ (!expect_inline ? html_input :
+ "<head>\n"
+ " <script>" + js_expected_inline_body + "</script>\n"
+ "</head>\n"
+ "<body>Hello, world!</body>\n");
+
+ // Put original Javascript file into our fetcher.
+ SimpleMetaData default_js_header;
+ resource_manager_->SetDefaultHeaders(&kContentTypeJavascript,
+ &default_js_header);
+ mock_url_fetcher_.SetResponse(js_url, default_js_header, js_outline_body);
+
+ // Rewrite the HTML page.
+ ValidateExpectedUrl(html_url, html_input, expected_output);
+ }
+};
+
+TEST_F(JsInlineFilterTest, DoInlineJavascriptSimple) {
+ // Simple case:
+ TestInlineJavascript("http://www.example.com/index.html",
+ "http://www.example.com/script.js",
+ "",
+ "function id(x) { return x; }\n",
+ true);
+}
+
+TEST_F(JsInlineFilterTest, DoInlineJavascriptWhitespace) {
+ // Whitespace between <script> and </script>:
+ TestInlineJavascript("http://www.example.com/index2.html",
+ "http://www.example.com/script2.js",
+ "\n \n ",
+ "function id(x) { return x; }\n",
+ true);
+}
+
+TEST_F(JsInlineFilterTest, DoNotInlineJavascriptDifferentDomain) {
+ // Different domains:
+ TestInlineJavascript("http://www.example.net/index.html",
+ "http://scripts.example.org/script.js",
+ "",
+ "function id(x) { return x; }\n",
+ false);
+}
+
+TEST_F(JsInlineFilterTest, DoNotInlineJavascriptInlineContents) {
+ // Inline contents:
+ TestInlineJavascript("http://www.example.com/index.html",
+ "http://www.example.com/script.js",
+ "{\"json\": true}",
+ "function id(x) { return x; }\n",
+ false);
+}
+
+TEST_F(JsInlineFilterTest, DoNotInlineJavascriptTooBig) {
+ // Javascript too long:
+ const int64 length = 2 * RewriteOptions::kDefaultJsInlineMaxBytes;
+ TestInlineJavascript("http://www.example.com/index.html",
+ "http://www.example.com/script.js",
+ "",
+ ("function longstr() { return '" +
+ std::string(length, 'z') + "'; }\n"),
+ false);
+}
+
+TEST_F(JsInlineFilterTest, DoNotInlineJavascriptWithCloseTag) {
+ // External script contains "</script>":
+ TestInlineJavascript("http://www.example.com/index.html",
+ "http://www.example.com/script.js",
+ "",
+ "function close() { return '</script>'; }\n",
+ false);
+}
+
+TEST_F(JsInlineFilterTest, DoInlineJavascriptXhtml) {
+ // Simple case:
+ TestInlineJavascriptXhtml("http://www.example.com/index.html",
+ "http://www.example.com/script.js",
+ "function id(x) { return x; }\n",
+ true);
+}
+
+TEST_F(JsInlineFilterTest, DoNotInlineJavascriptXhtmlWithCdataEnd) {
+ // External script contains "]]>":
+ TestInlineJavascriptXhtml("http://www.example.com/index.html",
+ "http://www.example.com/script.js",
+ "function end(x) { return ']]>'; }\n",
+ false);
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/js_outline_filter.cc b/trunk/src/net/instaweb/rewriter/js_outline_filter.cc
new file mode 100644
index 0000000..0ecf15a
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/js_outline_filter.cc
@@ -0,0 +1,174 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/rewriter/public/js_outline_filter.h"
+
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/output_resource.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include <string>
+#include "net/instaweb/util/public/string_writer.h"
+
+namespace net_instaweb {
+
+const char JsOutlineFilter::kFilterId[] = "jo";
+
+JsOutlineFilter::JsOutlineFilter(RewriteDriver* driver)
+ : inline_element_(NULL),
+ html_parse_(driver->html_parse()),
+ resource_manager_(driver->resource_manager()),
+ size_threshold_bytes_(driver->options()->js_outline_min_bytes()),
+ s_script_(html_parse_->Intern("script")),
+ s_src_(html_parse_->Intern("src")),
+ s_type_(html_parse_->Intern("type")),
+ script_tag_scanner_(html_parse_) { }
+
+void JsOutlineFilter::StartDocument() {
+ inline_element_ = NULL;
+ buffer_.clear();
+}
+
+void JsOutlineFilter::StartElement(HtmlElement* element) {
+ // No tags allowed inside script element.
+ if (inline_element_ != NULL) {
+ // TODO(sligocki): Add negative unit tests to hit these errors.
+ html_parse_->ErrorHere("Tag '%s' found inside script.",
+ element->tag().c_str());
+ inline_element_ = NULL; // Don't outline what we don't understand.
+ buffer_.clear();
+ }
+
+ HtmlElement::Attribute* src;
+ // We only deal with Javascript
+ if (script_tag_scanner_.ParseScriptElement(element, &src) ==
+ ScriptTagScanner::kJavaScript) {
+ inline_element_ = element;
+ buffer_.clear();
+ // script elements which already have a src should not be outlined.
+ if (src != NULL) {
+ inline_element_ = NULL;
+ }
+ }
+}
+
+void JsOutlineFilter::EndElement(HtmlElement* element) {
+ if (inline_element_ != NULL) {
+ if (element != inline_element_) {
+ // No other tags allowed inside script element.
+ html_parse_->ErrorHere("Tag '%s' found inside script.",
+ element->tag().c_str());
+
+ } else if (buffer_.size() >= size_threshold_bytes_) {
+ OutlineScript(inline_element_, buffer_);
+ } else {
+ html_parse_->InfoHere("Inline element not outlined because its size %d, "
+ "is below threshold %d",
+ static_cast<int>(buffer_.size()),
+ static_cast<int>(size_threshold_bytes_));
+ }
+ inline_element_ = NULL;
+ buffer_.clear();
+ }
+}
+
+void JsOutlineFilter::Flush() {
+ // If we were flushed in a script element, we cannot outline it.
+ inline_element_ = NULL;
+ buffer_.clear();
+}
+
+void JsOutlineFilter::Characters(HtmlCharactersNode* characters) {
+ if (inline_element_ != NULL) {
+ buffer_ += characters->contents();
+ }
+}
+
+void JsOutlineFilter::Comment(HtmlCommentNode* comment) {
+ if (inline_element_ != NULL) {
+ html_parse_->ErrorHere("Comment found inside script.");
+ inline_element_ = NULL; // Don't outline what we don't understand.
+ buffer_.clear();
+ }
+}
+
+void JsOutlineFilter::Cdata(HtmlCdataNode* cdata) {
+ if (inline_element_ != NULL) {
+ html_parse_->ErrorHere("CDATA found inside script.");
+ inline_element_ = NULL; // Don't outline what we don't understand.
+ buffer_.clear();
+ }
+}
+
+void JsOutlineFilter::IEDirective(HtmlIEDirectiveNode* directive) {
+ if (inline_element_ != NULL) {
+ html_parse_->ErrorHere("IE Directive found inside script.");
+ inline_element_ = NULL; // Don't outline what we don't understand.
+ buffer_.clear();
+ }
+}
+
+// Try to write content and possibly header to resource.
+bool JsOutlineFilter::WriteResource(const std::string& content,
+ OutputResource* resource,
+ MessageHandler* handler) {
+ // We set the TTL of the origin->hashed_name map to 0 because this is
+ // derived from the inlined HTML.
+ int64 origin_expire_time_ms = 0;
+ return resource_manager_->Write(HttpStatus::kOK, content, resource,
+ origin_expire_time_ms, handler);
+}
+
+// Create file with script content and remove that element from DOM.
+// TODO(sligocki): We probably will break any relative URL references here.
+void JsOutlineFilter::OutlineScript(HtmlElement* inline_element,
+ const std::string& content) {
+ if (html_parse_->IsRewritable(inline_element)) {
+ // Create script file from content.
+ MessageHandler* handler = html_parse_->message_handler();
+ // Create outline resource at the document location, not base URL location
+ scoped_ptr<OutputResource> resource(
+ resource_manager_->CreateOutputResourceWithPath(
+ GoogleUrl::AllExceptLeaf(html_parse_->gurl()), kFilterId, "_",
+ &kContentTypeJavascript, handler));
+ if (WriteResource(content, resource.get(), handler)) {
+ HtmlElement* outline_element = html_parse_->NewElement(
+ inline_element->parent(), s_script_);
+ outline_element->AddAttribute(s_src_, resource->url(), "'");
+ // Add all atrributes from old script element to new script src element.
+ for (int i = 0; i < inline_element->attribute_size(); ++i) {
+ const HtmlElement::Attribute& attr = inline_element->attribute(i);
+ outline_element->AddAttribute(attr);
+ }
+ // Add <script src=...> element to DOM.
+ html_parse_->InsertElementBeforeElement(inline_element,
+ outline_element);
+ // Remove original script element from DOM.
+ if (!html_parse_->DeleteElement(inline_element)) {
+ html_parse_->FatalErrorHere("Failed to delete inline script element");
+ }
+ } else {
+ html_parse_->ErrorHere("Failed to write outlined script resource.");
+ }
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/js_outline_filter_test.cc b/trunk/src/net/instaweb/rewriter/js_outline_filter_test.cc
new file mode 100644
index 0000000..de3f730
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/js_outline_filter_test.cc
@@ -0,0 +1,118 @@
+/*
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/rewriter/public/js_outline_filter.h"
+
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+
+namespace net_instaweb {
+
+namespace {
+
+class JsOutlineFilterTest : public ResourceManagerTestBase {
+ protected:
+ // TODO(sligocki): factor out common elements in OutlineStyle and Script.
+ // Test outlining scripts with options to write headers and use a hasher.
+ void OutlineScript(const StringPiece& id, Hasher* hasher) {
+ resource_manager_->set_hasher(hasher);
+
+ options_.set_js_outline_min_bytes(0);
+ AddFilter(RewriteOptions::kOutlineJavascript);
+
+ std::string script_text = "FOOBAR";
+ std::string outline_text;
+ AppendDefaultHeaders(kContentTypeJavascript, resource_manager_,
+ &outline_text);
+ outline_text += script_text;
+
+ std::string hash = hasher->Hash(script_text);
+ std::string outline_filename;
+ std::string outline_url = Encode(
+ "http://test.com/", JsOutlineFilter::kFilterId, hash, "_", "js");
+ filename_encoder_.Encode(file_prefix_, outline_url, &outline_filename);
+
+ // Make sure the file we check later was written this time, rm any old one.
+ DeleteFileIfExists(outline_filename);
+
+ std::string html_input =
+ "<head>\n"
+ " <title>Example style outline</title>\n"
+ " <!-- Script starts here -->\n"
+ " <script type='text/javascript'>" + script_text + "</script>\n"
+ " <!-- Script ends here -->\n"
+ "</head>";
+ std::string expected_output =
+ "<head>\n"
+ " <title>Example style outline</title>\n"
+ " <!-- Script starts here -->\n"
+ " <script src='" + outline_url + "' type='text/javascript'></script>\n"
+ " <!-- Script ends here -->\n"
+ "</head>";
+ ValidateExpected(id, html_input, expected_output);
+
+ std::string actual_outline;
+ ASSERT_TRUE(file_system_.ReadFile(outline_filename.c_str(),
+ &actual_outline,
+ &message_handler_));
+ EXPECT_EQ(outline_text, actual_outline);
+ }
+};
+
+// Tests for outlining scripts.
+TEST_F(JsOutlineFilterTest, OutlineScript) {
+ OutlineScript("outline_scripts_no_hash_with_headers", &mock_hasher_);
+}
+
+// Negative test.
+TEST_F(JsOutlineFilterTest, NoOutlineScript) {
+ std::string file_prefix = GTestTempDir() + "/no_outline";
+ std::string url_prefix = "http://mysite/no_outline";
+
+ // TODO(sligocki): Maybe test with other hashers.
+ //resource_manager_->set_hasher(hasher);
+
+ options_.EnableFilter(RewriteOptions::kOutlineCss);
+ options_.EnableFilter(RewriteOptions::kOutlineJavascript);
+ rewrite_driver_.AddFilters();
+
+ // We need to make sure we don't create this file, so rm any old one
+ std::string filename = Encode(file_prefix, JsOutlineFilter::kFilterId, "0",
+ "_", "js");
+ DeleteFileIfExists(filename.c_str());
+
+ static const char html_input[] =
+ "<head>\n"
+ " <title>Example style outline</title>\n"
+ " <!-- Script starts here -->\n"
+ " <script type='text/javascript' src='http://othersite/script.js'>"
+ "</script>\n"
+ " <!-- Script ends here -->\n"
+ "</head>";
+ ValidateNoChanges("no_outline_script", html_input);
+
+ // Check that it did *NOT* create the file.
+ // TODO(jmarantz): this is pretty brittle, and perhaps obsolete.
+ // We just change the test to ensure that we are not outlining when
+ // we don't want to.
+ EXPECT_FALSE(file_system_.Exists(filename.c_str(),
+ &message_handler_).is_true());
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/output_resource.cc b/trunk/src/net/instaweb/rewriter/output_resource.cc
new file mode 100644
index 0000000..821c227
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/output_resource.cc
@@ -0,0 +1,275 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+// jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/output_resource.h"
+
+#include "base/logging.h"
+#include "net/instaweb/rewriter/public/resource_namer.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/rewrite_filter.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/hasher.h"
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/filename_encoder.h"
+#include "net/instaweb/util/public/named_lock_manager.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/timer.h"
+#include "net/instaweb/util/stack_buffer.h"
+
+namespace net_instaweb {
+
+namespace {
+
+const char kLockSuffix[] = ".outputlock";
+
+} // namespace
+
+OutputResource::OutputResource(ResourceManager* manager,
+ const StringPiece& resolved_base,
+ const ResourceNamer& full_name,
+ const ContentType* type)
+ : Resource(manager, type),
+ output_file_(NULL),
+ writing_complete_(false),
+ generated_(false),
+ resolved_base_(resolved_base.data(), resolved_base.size()),
+ full_name_() {
+ full_name_.CopyFrom(full_name);
+ if (type == NULL) {
+ std::string ext_with_dot = StrCat(".", full_name.ext());
+ type_ = NameExtensionToContentType(ext_with_dot);
+ } else {
+ // This if + check used to be a 1-liner, but it was failing and this
+ // yields debuggable output.
+ // TODO(jmaessen): The addition of 1 below avoids the leading ".";
+ // make this convention consistent and fix all code.
+ CHECK_EQ((type->file_extension() + 1), full_name.ext());
+ }
+ CHECK(EndsInSlash(resolved_base)) << "resolved_base must end in a slash.";
+}
+
+OutputResource::~OutputResource() {
+}
+
+bool OutputResource::OutputWriter::Write(const StringPiece& data,
+ MessageHandler* handler) {
+ bool ret = http_value_->Write(data, handler);
+ if (file_writer_.get() != NULL) {
+ ret &= file_writer_->Write(data, handler);
+ }
+ return ret;
+}
+
+OutputResource::OutputWriter* OutputResource::BeginWrite(
+ MessageHandler* handler) {
+ value_.Clear();
+ full_name_.ClearHash();
+ CHECK(!writing_complete_);
+ CHECK(output_file_ == NULL);
+ if (resource_manager_->store_outputs_in_file_system()) {
+ FileSystem* file_system = resource_manager_->file_system();
+ // Always write to a tempfile, so that if we get interrupted in the middle
+ // we won't leave a half-baked file in the serving path.
+ std::string temp_prefix = TempPrefix();
+ output_file_ = file_system->OpenTempFile(
+ temp_prefix.c_str(), handler);
+ bool success = (output_file_ != NULL);
+ if (success) {
+ std::string header;
+ StringWriter string_writer(&header);
+ meta_data_.Write(&string_writer, handler); // Serialize header.
+ // It does not make sense to have the headers in the hash.
+ // call output_file_->Write directly, rather than going through
+ // OutputWriter.
+ //
+ // TODO(jmarantz): consider refactoring to split out the header-file
+ // writing in a different way, e.g. to a separate file.
+ success &= output_file_->Write(header, handler);
+ }
+ OutputWriter* writer = NULL;
+ if (success) {
+ writer = new OutputWriter(output_file_, &value_);
+ }
+ return writer;
+ } else {
+ return new OutputWriter(NULL, &value_);
+ };
+}
+
+bool OutputResource::EndWrite(OutputWriter* writer, MessageHandler* handler) {
+ CHECK(!writing_complete_);
+ value_.SetHeaders(meta_data_);
+ Hasher* hasher = resource_manager_->hasher();
+ full_name_.set_hash(hasher->Hash(contents()));
+ writing_complete_ = true;
+ bool ret = true;
+ if (output_file_ != NULL) {
+ FileSystem* file_system = resource_manager_->file_system();
+ CHECK(file_system != NULL);
+ std::string temp_filename = output_file_->filename();
+ ret = file_system->Close(output_file_, handler);
+
+ // Now that we are done writing, we can rename to the filename we
+ // really want.
+ if (ret) {
+ ret = file_system->RenameFile(temp_filename.c_str(), filename().c_str(),
+ handler);
+ }
+
+ // TODO(jmarantz): Consider writing to the HTTP cache as we write the
+ // file, so same-process write-then-read never has to read from disk.
+ // This is moderately inconvenient because HTTPCache lacks a streaming
+ // Put interface.
+
+ output_file_ = NULL;
+ }
+ if (creation_lock_.get() != NULL) {
+ // We've created the data, never need to lock again.
+ creation_lock_->Unlock();
+ creation_lock_.reset(NULL);
+ }
+ return ret;
+}
+
+// Called by FilenameOutputResource::BeginWrite to determine how
+// to start writing the tmpfile.
+std::string OutputResource::TempPrefix() const {
+ return StrCat(resource_manager_->filename_prefix(), "temp_");
+}
+
+StringPiece OutputResource::suffix() const {
+ CHECK(type_ != NULL);
+ return type_->file_extension();
+}
+
+void OutputResource::set_suffix(const StringPiece& ext) {
+ type_ = NameExtensionToContentType(ext);
+ if (type_ != NULL) {
+ // TODO(jmaessen): The addition of 1 below avoids the leading ".";
+ // make this convention consistent and fix all code.
+ full_name_.set_ext(type_->file_extension() + 1);
+ } else {
+ full_name_.set_ext(ext.substr(1));
+ }
+}
+
+std::string OutputResource::filename() const {
+ std::string filename;
+ resource_manager_->filename_encoder()->Encode(
+ resource_manager_->filename_prefix(), url(), &filename);
+ return filename;
+}
+
+std::string OutputResource::name_key() const {
+ std::string id_name = full_name_.EncodeIdName();
+ std::string result;
+ CHECK(!resolved_base_.empty()); // Corresponding path in url() is dead code
+ result = StrCat(resolved_base_, id_name);
+ return result;
+}
+
+std::string OutputResource::hash_ext() const {
+ return full_name_.EncodeHashExt();
+}
+
+// TODO(jmarantz): change the name to reflect the fact that it is not
+// just an accessor now.
+std::string OutputResource::url() const {
+ std::string encoded = full_name_.Encode();
+ encoded = StrCat(resolved_base_, encoded);
+ return encoded;
+}
+
+void OutputResource::SetHash(const StringPiece& hash) {
+ CHECK(!writing_complete_);
+ CHECK(!has_hash());
+ full_name_.set_hash(hash);
+}
+
+bool OutputResource::Load(MessageHandler* handler) {
+ if (!writing_complete_ && resource_manager_->store_outputs_in_file_system()) {
+ FileSystem* file_system = resource_manager_->file_system();
+ FileSystem::InputFile* file = file_system->OpenInputFile(
+ filename().c_str(), handler);
+ if (file != NULL) {
+ char buf[kStackBufferSize];
+ int nread = 0, num_consumed = 0;
+ // TODO(jmarantz): this logic is duplicated in util/wget_url_fetcher.cc,
+ // consider a refactor to merge it.
+ meta_data_.Clear();
+ value_.Clear();
+ while (!meta_data_.headers_complete() &&
+ ((nread = file->Read(buf, sizeof(buf), handler)) != 0)) {
+ num_consumed = meta_data_.ParseChunk(
+ StringPiece(buf, nread), handler);
+ }
+ value_.SetHeaders(meta_data_);
+ writing_complete_ = value_.Write(
+ StringPiece(buf + num_consumed, nread - num_consumed),
+ handler);
+ while (writing_complete_ &&
+ ((nread = file->Read(buf, sizeof(buf), handler)) != 0)) {
+ writing_complete_ = value_.Write(StringPiece(buf, nread), handler);
+ }
+ file_system->Close(file, handler);
+ }
+ }
+ return writing_complete_;
+}
+
+bool OutputResource::IsWritten() const {
+ return writing_complete_;
+}
+
+void OutputResource::SetType(const ContentType* content_type) {
+ Resource::SetType(content_type);
+ // TODO(jmaessen): The addition of 1 below avoids the leading ".";
+ // make this convention consistent and fix all code.
+ full_name_.set_ext(content_type->file_extension() + 1);
+}
+
+bool OutputResource::LockForCreation(const ResourceManager* resource_manager,
+ ResourceManager::BlockingBehavior block) {
+ const int64 break_lock_ms = 30 * Timer::kSecondMs;
+ const int64 block_lock_ms = 5 * Timer::kSecondMs;
+ bool result = true;
+ if (creation_lock_.get() == NULL) {
+ std::string lock_name =
+ StrCat(resource_manager->filename_prefix(),
+ resource_manager->hasher()->Hash(name_key()),
+ kLockSuffix);
+ creation_lock_.reset(resource_manager->lock_manager()->
+ CreateNamedLock(lock_name));
+ }
+ switch (block) {
+ case ResourceManager::kNeverBlock:
+ // TODO(jmaessen): When caller retries properly in all cases, use
+ // LockTimedWaitStealOld with a sub-second timeout to try to catch
+ // rewritten data.
+ result = creation_lock_->TryLockStealOld(break_lock_ms);
+ break;
+ case ResourceManager::kMayBlock:
+ creation_lock_->LockStealOld(block_lock_ms);
+ break;
+ }
+ return result;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/public/add_head_filter.h b/trunk/src/net/instaweb/rewriter/public/add_head_filter.h
new file mode 100644
index 0000000..7f6d955
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/add_head_filter.h
@@ -0,0 +1,55 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_ADD_HEAD_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_ADD_HEAD_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/util/public/atom.h"
+
+namespace net_instaweb {
+
+// Adds a 'head' element before the 'body', if none was found
+// during parsing. This enables downstream filters to assume
+// that there will be a head.
+class AddHeadFilter : public EmptyHtmlFilter {
+ public:
+ explicit AddHeadFilter(HtmlParse* parser, bool combine_multiple_heads);
+
+ virtual void StartDocument();
+ virtual void StartElement(HtmlElement* element);
+ virtual void EndDocument();
+ virtual void EndElement(HtmlElement* element);
+ virtual void Flush();
+ virtual const char* Name() const { return "AddHead"; }
+
+ private:
+ HtmlParse* html_parse_;
+ bool combine_multiple_heads_;
+ bool found_head_;
+ Atom s_head_;
+ Atom s_body_;
+ HtmlElement* head_element_;
+
+ DISALLOW_COPY_AND_ASSIGN(AddHeadFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_ADD_HEAD_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/add_instrumentation_filter.h b/trunk/src/net/instaweb/rewriter/public/add_instrumentation_filter.h
new file mode 100644
index 0000000..6748b4a
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/add_instrumentation_filter.h
@@ -0,0 +1,62 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: abliss@google.com (Adam Bliss)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_ADD_INSTRUMENTATION_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_ADD_INSTRUMENTATION_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/util/public/atom.h"
+
+namespace net_instaweb {
+
+class Statistics;
+class Variable;
+
+// Injects javascript instrumentation for monitoring page-rendering time.
+class AddInstrumentationFilter : public EmptyHtmlFilter {
+ public:
+ explicit AddInstrumentationFilter(HtmlParse* parser,
+ const StringPiece& beacon_url,
+ Statistics* statistics);
+
+ static void Initialize(Statistics* statistics);
+ virtual void StartDocument();
+ virtual void StartElement(HtmlElement* element);
+ virtual void EndElement(HtmlElement* element);
+ virtual const char* Name() const { return "AddInstrumentation"; }
+ // Handles an incoming beacon request by incrementing the appropriate
+ // variables. Returns true if the url was parsed and handled correctly; in
+ // this case a 204 No Content response should be sent. Returns false if the
+ // url could not be parsed; in this case the request should be declined.
+ bool HandleBeacon(const StringPiece& unparsed_url);
+ private:
+ HtmlParse* html_parse_;
+ bool found_head_;
+ Atom s_head_;
+ Atom s_body_;
+ Variable* total_page_load_ms_;
+ Variable* page_load_count_;
+ std::string beacon_url_;
+
+ DISALLOW_COPY_AND_ASSIGN(AddInstrumentationFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_ADD_INSTRUMENTATION_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/base_tag_filter.h b/trunk/src/net/instaweb/rewriter/public/base_tag_filter.h
new file mode 100644
index 0000000..82a87ad
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/base_tag_filter.h
@@ -0,0 +1,64 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_BASE_TAG_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_BASE_TAG_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/util/public/atom.h"
+#include <string>
+
+namespace net_instaweb {
+
+// Add this filter into the HtmlParse chain to add a base
+// tag into the head section of an HTML document.
+//
+// e.g.
+//
+// HtmlParser* parser = ...
+// ...
+// BaseTagFilter* base_tag_filter = new BaseTagFilter(parser);
+// parser->AddFilter(base_tag_filter);
+// base_tag_filter->set_base("http://my_new_base.com");
+// ...
+// parser->StartParse()...
+class BaseTagFilter : public EmptyHtmlFilter {
+ public:
+ explicit BaseTagFilter(HtmlParse* parser);
+
+ virtual void StartDocument();
+ virtual void StartElement(HtmlElement* element);
+ virtual const char* Name() const { return "BaseTag"; }
+
+ void set_base_url(const StringPiece& url) { url.CopyToString(&base_url_); }
+
+ private:
+ Atom s_head_;
+ Atom s_base_;
+ Atom s_href_;
+ bool found_head_;
+ std::string base_url_;
+ HtmlParse* html_parse_;
+
+ DISALLOW_COPY_AND_ASSIGN(BaseTagFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_BASE_TAG_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/cache_extender.h b/trunk/src/net/instaweb/rewriter/public/cache_extender.h
new file mode 100644
index 0000000..389e3ee
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/cache_extender.h
@@ -0,0 +1,70 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_CACHE_EXTENDER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_CACHE_EXTENDER_H_
+
+#include <vector>
+
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/resource_tag_scanner.h"
+#include "net/instaweb/rewriter/public/rewrite_single_resource_filter.h"
+#include "net/instaweb/util/public/atom.h"
+#include <string>
+
+namespace net_instaweb {
+
+class Hasher;
+class ResourceManager;
+class Timer;
+class Variable;
+
+// Rewrites resources to extend their cache lifetime, encoding the
+// content hash into the new URL to ensure we do not serve stale
+// data.
+class CacheExtender : public RewriteSingleResourceFilter {
+ public:
+ CacheExtender(RewriteDriver* driver, const char* path_prefix);
+
+ static void Initialize(Statistics* statistics);
+
+ virtual void StartDocumentImpl() {}
+ virtual void StartElementImpl(HtmlElement* element);
+ virtual void EndElementImpl(HtmlElement* element) {}
+
+ virtual const char* Name() const { return "CacheExtender"; }
+
+ protected:
+ virtual bool RewriteLoadedResource(const Resource* input_resource,
+ OutputResource* output_resource);
+
+ private:
+ bool IsRewrittenResource(const StringPiece& url) const;
+
+ HtmlParse* html_parse_;
+ ResourceManager* resource_manager_;
+ ResourceTagScanner tag_scanner_;
+ Variable* extension_count_;
+ Variable* not_cacheable_count_;
+
+ DISALLOW_COPY_AND_ASSIGN(CacheExtender);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_CACHE_EXTENDER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/collapse_whitespace_filter.h b/trunk/src/net/instaweb/rewriter/public/collapse_whitespace_filter.h
new file mode 100644
index 0000000..9adbbf0
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/collapse_whitespace_filter.h
@@ -0,0 +1,59 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_COLLAPSE_WHITESPACE_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_COLLAPSE_WHITESPACE_FILTER_H_
+
+#include <vector>
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/util/public/atom.h"
+
+namespace net_instaweb {
+
+// Reduce the size of the HTML by collapsing whitespace (except within certain
+// tags, e.g. <pre> and <script>). Note that this is a dangerous filter, as
+// CSS can be used to make the HTML whitespace-sensitive in unpredictable
+// places; thus, it should only be used for content that you are sure will not
+// do this.
+//
+// TODO(mdsteele): Use the CSS parser (once it's finished) to try to
+// intelligently determine when the CSS "white-space: pre" property is in use;
+// that would make this filter much safer.
+class CollapseWhitespaceFilter : public EmptyHtmlFilter {
+ public:
+ explicit CollapseWhitespaceFilter(HtmlParse* html_parse);
+
+ virtual void StartDocument();
+ virtual void StartElement(HtmlElement* element);
+ virtual void EndElement(HtmlElement* element);
+ virtual void Characters(HtmlCharactersNode* characters);
+ virtual const char* Name() const { return "CollapseWhitespace"; }
+
+ private:
+ HtmlParse* html_parse_;
+ std::vector<Atom> atom_stack_;
+ AtomSet sensitive_tags_;
+
+ DISALLOW_COPY_AND_ASSIGN(CollapseWhitespaceFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_COLLAPSE_WHITESPACE_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/common_filter.h b/trunk/src/net/instaweb/rewriter/public/common_filter.h
new file mode 100644
index 0000000..4eca80d
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/common_filter.h
@@ -0,0 +1,90 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_BASE_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_BASE_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/util/public/atom.h"
+#include "net/instaweb/util/public/google_url.h"
+
+namespace net_instaweb {
+
+class HtmlElement;
+class HtmlParse;
+class Resource;
+class RewriteDriver;
+class OutputResource;
+class UrlSegmentEncoder;
+
+// CommonFilter encapsulates useful functionality that many filters will want.
+// All filters who want this functionality should inherit from CommonFilter and
+// define the Helper methods rather than the main methods.
+//
+// Currently, it stores the current base URL (which can depend on where you
+// are on a page since the <base> element does not have to be first)
+// and whether we are in a <noscript> element (in which case, we should be
+// careful about moving things out of this element).
+class CommonFilter : public EmptyHtmlFilter {
+ public:
+ CommonFilter(RewriteDriver* driver);
+ virtual ~CommonFilter();
+
+ // Getters
+ HtmlParse* html_parse() { return html_parse_; }
+ const GURL& base_gurl() { return base_gurl_; }
+ HtmlElement* noscript_element() { return noscript_element_; }
+
+ // Note: Don't overload these methods, overload the implementers instead!
+ virtual void StartDocument();
+ virtual void StartElement(HtmlElement* element);
+ virtual void EndElement(HtmlElement* element);
+
+ Resource* CreateInputResource(const StringPiece& url);
+ Resource* CreateInputResourceAbsolute(const StringPiece& url);
+ Resource* CreateInputResourceAndReadIfCached(const StringPiece& url);
+ Resource* CreateInputResourceFromOutputResource(
+ UrlSegmentEncoder* encoder, OutputResource* output_resource);
+
+ protected:
+ // Overload these implementer methods:
+ // Intentionally left abstract so that implementers don't forget to change
+ // the name from Blah to BlahImpl.
+ virtual void StartDocumentImpl() = 0;
+ virtual void StartElementImpl(HtmlElement* element) = 0;
+ virtual void EndElementImpl(HtmlElement* element) = 0;
+
+ private:
+ RewriteDriver* driver_;
+ HtmlParse* html_parse_;
+ // TODO(sligocki): Maybe: don't store a separate GURL in each filter.
+ GURL base_gurl_;
+ HtmlElement* noscript_element_;
+
+ const Atom s_base_;
+ const Atom s_href_;
+ const Atom s_noscript_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(CommonFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_BASE_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/css_combine_filter.h b/trunk/src/net/instaweb/rewriter/public/css_combine_filter.h
new file mode 100644
index 0000000..89d0136
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/css_combine_filter.h
@@ -0,0 +1,94 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_CSS_COMBINE_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_CSS_COMBINE_FILTER_H_
+
+#include <vector>
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/css_tag_scanner.h"
+#include "net/instaweb/rewriter/public/rewrite_filter.h"
+#include "net/instaweb/util/public/atom.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class OutputResource;
+class Resource;
+class ResourceManager;
+class Variable;
+
+class CssCombineFilter : public RewriteFilter {
+ public:
+ CssCombineFilter(RewriteDriver* rewrite_driver, const char* path_prefix);
+ virtual ~CssCombineFilter();
+
+ static void Initialize(Statistics* statistics);
+ virtual void StartDocumentImpl();
+ virtual void StartElementImpl(HtmlElement* element) {}
+ virtual void EndElementImpl(HtmlElement* element);
+ virtual void Flush();
+ virtual void IEDirective(HtmlIEDirectiveNode* directive);
+ virtual bool Fetch(OutputResource* resource,
+ Writer* writer,
+ const MetaData& request_header,
+ MetaData* response_headers,
+ MessageHandler* message_handler,
+ UrlAsyncFetcher::Callback* callback);
+ virtual const char* Name() const { return "CssCombine"; }
+
+ private:
+ friend class CssCombiner;
+ typedef std::vector<Resource*> ResourceVector;
+
+ // Try to combine all the CSS files we have seen so far.
+ // Insert the combined resource where the first original CSS link was.
+ void TryCombineAccumulated();
+
+ bool WriteWithAbsoluteUrls(const StringPiece& contents,
+ OutputResource* combination,
+ const std::string& base_url,
+ MessageHandler* handler);
+ bool WriteCombination(const ResourceVector& combine_resources,
+ OutputResource* combination,
+ MessageHandler* handler);
+
+ Atom s_type_;
+ Atom s_link_;
+ Atom s_href_;
+ Atom s_rel_;
+ Atom s_media_;
+ Atom s_style_;
+
+ class Partnership;
+
+ scoped_ptr<Partnership> partnership_;
+ HtmlParse* html_parse_;
+ ResourceManager* resource_manager_;
+ CssTagScanner css_tag_scanner_;
+ Variable* css_file_count_reduction_;
+
+ DISALLOW_COPY_AND_ASSIGN(CssCombineFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_CSS_COMBINE_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/css_filter.h b/trunk/src/net/instaweb/rewriter/public/css_filter.h
new file mode 100644
index 0000000..09e4799
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/css_filter.h
@@ -0,0 +1,117 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_CSS_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_CSS_FILTER_H_
+
+#include <vector>
+
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/rewrite_single_resource_filter.h"
+#include "net/instaweb/util/public/atom.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace Css {
+
+class Stylesheet;
+
+} // namespace Css
+
+namespace net_instaweb {
+
+class HtmlParse;
+class MessageHandler;
+class OutputResource;
+class Resource;
+class ResourceManager;
+
+// Find and parse all CSS in the page and apply transformations including:
+// minification, combining, refactoring, and optimizing sub-resources.
+//
+// Currently only does basic minification.
+//
+// Note that CssCombineFilter currently does combining (although there is a bug)
+// but CssFilter will eventually replace this.
+//
+// Currently only deals with inline <style> tags and external <link> resources.
+// It does not consider style= attributes on arbitrary elements.
+class CssFilter : public RewriteSingleResourceFilter {
+ public:
+ CssFilter(RewriteDriver* driver, const StringPiece& filter_prefix);
+
+ static void Initialize(Statistics* statistics);
+ static void Terminate();
+
+ // Note: AtExitManager needs to be initialized or you get a nasty error:
+ // Check failed: false. Tried to RegisterCallback without an AtExitManager.
+ // This is called by Initialize.
+ static void InitializeAtExitManager();
+
+ virtual void StartDocumentImpl();
+ virtual void StartElementImpl(HtmlElement* element);
+ virtual void Characters(HtmlCharactersNode* characters);
+ virtual void EndElementImpl(HtmlElement* element);
+
+ virtual const char* Name() const { return "CssFilter"; }
+
+ static const char kFilesMinified[];
+ static const char kMinifiedBytesSaved[];
+ static const char kParseFailures[];
+
+ private:
+ bool RewriteCssText(const StringPiece& in_text, std::string* out_text,
+ const std::string& id, MessageHandler* handler);
+ bool RewriteExternalCss(const StringPiece& in_url, std::string* out_url);
+ bool RewriteExternalCssToResource(Resource* input_resource,
+ OutputResource* output_resource);
+
+ virtual bool RewriteLoadedResource(const Resource* input_resource,
+ OutputResource* output_resource);
+
+ Css::Stylesheet* CombineStylesheets(
+ std::vector<Css::Stylesheet*>* stylesheets);
+ bool LoadAllSubStylesheets(Css::Stylesheet* stylesheet_with_imports,
+ std::vector<Css::Stylesheet*>* result_stylesheets);
+
+ Css::Stylesheet* LoadStylesheet(const StringPiece& url) { return NULL; }
+
+ HtmlParse* html_parse_;
+ ResourceManager* resource_manager_;
+
+ bool in_style_element_; // Are we in a style element?
+ // These are meaningless if in_style_element_ is false:
+ HtmlElement* style_element_; // The element we are in.
+ HtmlCharactersNode* style_char_node_; // The single character node in style.
+
+ Atom s_style_;
+ Atom s_link_;
+ Atom s_rel_;
+ Atom s_href_;
+
+ // Statistics
+ Variable* num_files_minified_;
+ Variable* minified_bytes_saved_;
+ Variable* num_parse_failures_;
+
+ DISALLOW_COPY_AND_ASSIGN(CssFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_CSS_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/css_inline_filter.h b/trunk/src/net/instaweb/rewriter/public/css_inline_filter.h
new file mode 100644
index 0000000..5a74985
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/css_inline_filter.h
@@ -0,0 +1,59 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_CSS_INLINE_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_CSS_INLINE_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/common_filter.h"
+#include "net/instaweb/util/public/atom.h"
+#include <string>
+
+namespace net_instaweb {
+
+class ResourceManager;
+class RewriteDriver;
+
+// Inline small CSS files.
+class CssInlineFilter : public CommonFilter {
+ public:
+ explicit CssInlineFilter(RewriteDriver* driver);
+
+ virtual void StartDocumentImpl();
+ virtual void EndDocument();
+ virtual void StartElementImpl(HtmlElement* element) {}
+ virtual void EndElementImpl(HtmlElement* element);
+ virtual const char* Name() const { return "InlineCss"; }
+
+ private:
+ HtmlParse* const html_parse_;
+ const Atom href_atom_;
+ const Atom link_atom_;
+ const Atom media_atom_;
+ const Atom rel_atom_;
+ const Atom style_atom_;
+ const size_t size_threshold_bytes_;
+
+ std::string domain_;
+
+ DISALLOW_COPY_AND_ASSIGN(CssInlineFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_CSS_INLINE_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/css_minify.h b/trunk/src/net/instaweb/rewriter/public/css_minify.h
new file mode 100644
index 0000000..d654265
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/css_minify.h
@@ -0,0 +1,82 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_CSS_MINIFY_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_CSS_MINIFY_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace Css {
+class Stylesheet;
+class Import;
+class Ruleset;
+class Selector;
+class SimpleSelector;
+class SimpleSelectors;
+class Declaration;
+class Value;
+}
+
+namespace net_instaweb {
+
+class MessageHandler;
+class Writer;
+
+class CssMinify {
+ public:
+ // Write minified Stylesheet.
+ static bool Stylesheet(const Css::Stylesheet& stylesheet,
+ Writer* writer,
+ MessageHandler* handler);
+
+ private:
+ CssMinify(Writer* writer, MessageHandler* handler);
+ ~CssMinify();
+
+ void Write(const StringPiece& str);
+
+ template<typename Container>
+ void JoinMinify(const Container& container, const StringPiece& sep);
+ template<typename Iterator>
+ void JoinMinifyIter(const Iterator& begin, const Iterator& end,
+ const StringPiece& sep);
+ template<typename Container>
+ void JoinMediaMinify(const Container& container, const StringPiece& sep);
+
+ // We name all of these methods identically to simplify the writing of the
+ // templated Join* methods.
+ void Minify(const Css::Stylesheet& stylesheet);
+ void Minify(const Css::Import& import);
+ void Minify(const Css::Ruleset& ruleset);
+ void Minify(const Css::Selector& selector);
+ void Minify(const Css::SimpleSelectors& sselectors, bool isfirst = false);
+ void Minify(const Css::SimpleSelector& sselector);
+ void Minify(const Css::Declaration& declaration);
+ void Minify(const Css::Value& value);
+
+ Writer* writer_;
+ MessageHandler* handler_;
+ bool ok_;
+
+ DISALLOW_COPY_AND_ASSIGN(CssMinify);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_CSS_MINIFY_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/css_move_to_head_filter.h b/trunk/src/net/instaweb/rewriter/public/css_move_to_head_filter.h
new file mode 100644
index 0000000..cc98714
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/css_move_to_head_filter.h
@@ -0,0 +1,58 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_CSS_MOVE_TO_HEAD_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_CSS_MOVE_TO_HEAD_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/rewriter/public/css_tag_scanner.h"
+#include "net/instaweb/util/public/atom.h"
+
+namespace net_instaweb {
+
+class Statistics;
+class Variable;
+
+class CssMoveToHeadFilter : public EmptyHtmlFilter {
+ public:
+ CssMoveToHeadFilter(HtmlParse* html_parse, Statistics* statistics);
+
+ static void Initialize(Statistics* statistics);
+ virtual void StartDocument();
+ virtual void StartElement(HtmlElement* element);
+ virtual void EndElement(HtmlElement* element);
+ virtual const char* Name() const { return "CssMoveToHead"; }
+
+ private:
+ Atom s_head_;
+ Atom s_noscript_;
+ Atom s_style_;
+
+ HtmlParse* html_parse_;
+ HtmlElement* head_element_;
+ HtmlElement* noscript_element_;
+ CssTagScanner css_tag_scanner_;
+ Variable* counter_;
+
+ DISALLOW_COPY_AND_ASSIGN(CssMoveToHeadFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_CSS_MOVE_TO_HEAD_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/css_outline_filter.h b/trunk/src/net/instaweb/rewriter/public/css_outline_filter.h
new file mode 100644
index 0000000..9332fa6
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/css_outline_filter.h
@@ -0,0 +1,86 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_CSS_OUTLINE_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_CSS_OUTLINE_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/common_filter.h"
+#include "net/instaweb/util/public/atom.h"
+#include <string>
+
+namespace net_instaweb {
+
+class MessageHandler;
+class MetaData;
+class OutputResource;
+class ResourceManager;
+
+// Filter to take explicit <style> and <script> tags and outline them to files.
+class CssOutlineFilter : public CommonFilter {
+ public:
+ explicit CssOutlineFilter(RewriteDriver* driver);
+ static const char kFilterId[];
+
+ virtual void StartDocumentImpl();
+
+ virtual void StartElementImpl(HtmlElement* element);
+ virtual void EndElementImpl(HtmlElement* element);
+
+ virtual void Flush();
+
+ // HTML Events we expect to be in <style> elements.
+ virtual void Characters(HtmlCharactersNode* characters);
+
+ // HTML Events we do not expect to be in <style> and <script> elements.
+ virtual void Comment(HtmlCommentNode* comment);
+ virtual void Cdata(HtmlCdataNode* cdata);
+ virtual void IEDirective(HtmlIEDirectiveNode* directive);
+
+ // Ignored HTML Events.
+ virtual void EndDocument() {}
+ virtual void Directive(HtmlDirectiveNode* directive) {}
+
+ virtual const char* Name() const { return "OutlineCss"; }
+
+ private:
+ bool WriteResource(const StringPiece& content, OutputResource* resource,
+ MessageHandler* handler);
+ void OutlineStyle(HtmlElement* element, const std::string& content);
+
+ // The style or script element we are in (if it hasn't been flushed).
+ // If we are not in a script or style element, inline_element_ == NULL.
+ HtmlElement* inline_element_;
+ // Temporarily buffers the content between open and close of inline_element_.
+ std::string buffer_;
+ HtmlParse* html_parse_;
+ ResourceManager* resource_manager_;
+ size_t size_threshold_bytes_;
+ // HTML strings interned into a symbol table.
+ Atom s_link_;
+ Atom s_style_;
+ Atom s_rel_;
+ Atom s_href_;
+ Atom s_type_;
+
+ DISALLOW_COPY_AND_ASSIGN(CssOutlineFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_CSS_OUTLINE_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/css_tag_scanner.h b/trunk/src/net/instaweb/rewriter/public/css_tag_scanner.h
new file mode 100644
index 0000000..602a3b6
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/css_tag_scanner.h
@@ -0,0 +1,65 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_CSS_TAG_SCANNER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_CSS_TAG_SCANNER_H_
+
+#include <vector>
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_parser_types.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/util/public/atom.h"
+#include <string>
+
+namespace net_instaweb {
+
+class CssTagScanner {
+ public:
+ explicit CssTagScanner(HtmlParse* html_parse);
+
+ // Examines an HTML element to determine if it's a CSS link,
+ // extracting out the HREF and the media-type.
+ bool ParseCssElement(
+ HtmlElement* element, HtmlElement::Attribute** href, const char** media);
+
+ // Scans the contents of a CSS file, looking for the pattern url(xxx).
+ // If xxx is a relative URL, it absolutifies it based on the passed-in base
+ // path. If xxx is quoted with single-quotes or double-quotes, those are
+ // retained and the URL inside is absolutified.
+ static bool AbsolutifyUrls(const StringPiece& contents,
+ const std::string& base_url,
+ Writer* writer, MessageHandler* handler);
+
+ // Does this CSS file contain @import? If so, it cannot be combined with
+ // previous CSS files. This may give false-positives, but no false-negatives.
+ static bool HasImport(const StringPiece& contents, MessageHandler* handler);
+
+ private:
+ Atom s_link_;
+ Atom s_href_;
+ Atom s_type_;
+ Atom s_rel_;
+ Atom s_media_;
+
+ DISALLOW_COPY_AND_ASSIGN(CssTagScanner);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_CSS_TAG_SCANNER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/data_url_input_resource.h b/trunk/src/net/instaweb/rewriter/public/data_url_input_resource.h
new file mode 100644
index 0000000..7bb2a1e
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/data_url_input_resource.h
@@ -0,0 +1,87 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+//
+// An input resource representing a data: url. This is uncommon in web
+// pages, but we generate these urls as a result of image inlining and
+// this confuses subsequent filters in certain cases.
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/resource.h"
+#include "net/instaweb/util/public/data_url.h"
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_DATA_URL_INPUT_RESOURCE_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_DATA_URL_INPUT_RESOURCE_H_
+
+namespace net_instaweb {
+
+class ResourceManager;
+class ContentType;
+enum Encoding;
+
+class DataUrlInputResource : public Resource {
+ public:
+ // We expose a factory; parse failure returns NULL.
+ static DataUrlInputResource* Make(const StringPiece& url,
+ ResourceManager* manager) {
+ const ContentType* type;
+ Encoding encoding;
+ StringPiece encoded_contents;
+ // We create the local copy of the url early, because
+ // encoded_contents will in general be a substring of this
+ // local copy and must have the same lifetime.
+ std::string* url_copy = new std::string();
+ url.CopyToString(url_copy);
+ if (!ParseDataUrl(*url_copy, &type, &encoding, &encoded_contents)) {
+ return NULL;
+ }
+ return new DataUrlInputResource(url_copy, encoding, type, encoded_contents,
+ manager);
+ }
+
+ virtual ~DataUrlInputResource() { }
+
+ virtual std::string url() const { return *url_.get(); }
+
+ protected:
+ virtual bool Load(MessageHandler* message_handler);
+ virtual bool IsCacheable() const;
+
+ private:
+ DataUrlInputResource(const std::string* url,
+ Encoding encoding,
+ const ContentType* type,
+ const StringPiece& encoded_contents,
+ ResourceManager* manager)
+ : Resource(manager, type),
+ url_(url),
+ encoding_(encoding),
+ encoded_contents_(encoded_contents) {
+ }
+
+ scoped_ptr<const std::string> url_;
+ const Encoding encoding_;
+ const StringPiece encoded_contents_; // substring of url.
+ std::string decoded_contents_;
+
+ DISALLOW_COPY_AND_ASSIGN(DataUrlInputResource);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_DATA_URL_INPUT_RESOURCE_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/domain_lawyer.h b/trunk/src/net/instaweb/rewriter/public/domain_lawyer.h
new file mode 100644
index 0000000..168cfea
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/domain_lawyer.h
@@ -0,0 +1,164 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+//
+// This class manages the relationships between domains and resources.
+//
+// The Lawyer keeps track of which domains we are allowed to rewrite, including
+// whether multiple resources can be bundled together.
+//
+// The Lawyer keeps track of domain mappings to move resources onto a CDN or
+// onto a cookieless domain.
+//
+// The Lawyer keeps track of domain sharding, for distributing resources across
+// equivalent domains to improve browser download parallelism.
+//
+// The class here holds state based on the configuration files
+// (e.g. Apache .conf).
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_DOMAIN_LAWYER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_DOMAIN_LAWYER_H_
+
+#include <map>
+#include <vector>
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/google_url.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class MessageHandler;
+
+class DomainLawyer {
+ public:
+ DomainLawyer() {}
+ ~DomainLawyer();
+
+ // Determines whether a resource can be rewritten, and returns the domain
+ // that it should be written to. Only the domain of the resolved request
+ // is considered. If the resource_url is relative (has no domain) then
+ // the resource can always be written, and will share the domain of the
+ // original request.
+ //
+ // The resource_url is considered relative to original_request. Generally
+ // it is always accessible to rewrite resources in the same domain as the
+ // original.
+ //
+ // TODO(jmarantz): the mapped domain name will not incorporate any sharding.
+ // This must be handled by another mapping function which has not yet
+ // been implemented.
+ //
+ // The returned mapped_domain_name will always end with a slash on success.
+ // The returned resolved_request incorporates rewrite-domain mapping and
+ // the original URL.
+ //
+ // Returns false on failure.
+ bool MapRequestToDomain(const GURL& original_request,
+ const StringPiece& resource_url,
+ std::string* mapped_domain_name,
+ GURL* resolved_request,
+ MessageHandler* handler) const;
+
+ // Maps an origin resource; just prior to fetching it. This fails
+ // if the input URL is not valid. It succeeds even if there is no
+ // mapping done. You must compare 'in' to 'out' to determine if
+ // mapping was done.
+ bool MapOrigin(const StringPiece& in, std::string* out) const;
+
+ // The methods below this comment are intended only to be run only
+ // at configuration time.
+
+ // Adds a simple domain to the set that can be rewritten. No
+ // mapping or sharding will be performed. Returns false if the
+ // domain syntax was not acceptable. Wildcards (*, ?) may be used in
+ // the domain_name. Careless use of wildcards can expose the user to
+ // XSS attacks.
+ bool AddDomain(const StringPiece& domain_name, MessageHandler* handler);
+
+ // Adds a domain mapping, to assist with serving resources from
+ // cookieless domains or CDNs. This implicitly calls AddDomain(to_domain)
+ // and AddDomain(from_domain) if necessary. If either 'to' or 'from' has
+ // invalid syntax then this function returns false and has no effect.
+ //
+ // Wildcards may not be used in the to_domain, but they can be used
+ // in the from_domains.
+ //
+ // This routine can be called multiple times for the same to_domain. If
+ // the 'from' domains overlap due to wildcards, this will not be detected.
+ bool AddRewriteDomainMapping(const StringPiece& to_domain,
+ const StringPiece& comma_separated_from_domains,
+ MessageHandler* handler);
+
+ // Adds a domain mapping, to assist with fetching resources from locally
+ // signficant names/ip-addresses.
+ //
+ // Wildcards may not be used in the to_domain, but they can be used
+ // in the from_domains.
+ //
+ // This routine can be called multiple times for the same to_domain. If
+ // the 'from' domains overlap due to wildcards, this will not be detected.
+ bool AddOriginDomainMapping(const StringPiece& to_domain,
+ const StringPiece& comma_separated_from_domains,
+ MessageHandler* handler);
+
+ // Specifies domain-sharding. This implicitly calls AddDomain(to_domain).
+ // The shard_pattern must include exactly one '%d'.
+ //
+ // Wildcards may not be used in the to_domain or the from_domain.
+ //
+ // TODO(jmarantz): implement this
+ bool ShardDomain(const StringPiece& to_domain,
+ const StringPiece& shard_pattern,
+ int num_shards, MessageHandler* handler);
+
+ // Merge the domains declared in src into this. There are no exclusions, so
+ // this is really just aggregating the mappings and authorizations declared in
+ // both domains. When the same domain is mapped in 'this' and 'src', 'src'
+ // wins.
+ void Merge(const DomainLawyer& src);
+
+ private:
+ class Domain;
+ typedef void (Domain::*SetDomainFn)(Domain* domain);
+
+ bool MapDomainHelper(
+ const StringPiece& to_domain_name,
+ const StringPiece& comma_separated_from_domains,
+ SetDomainFn set_domain_fn,
+ MessageHandler* handler);
+
+ Domain* AddDomainHelper(const StringPiece& domain_name,
+ bool warn_on_duplicate,
+ MessageHandler* handler);
+
+ Domain* FindDomain(const std::string& domain_name) const;
+
+ typedef std::map<std::string, Domain*> DomainMap;
+ DomainMap domain_map_;
+ typedef std::vector<Domain*> DomainVector;
+ DomainVector wildcarded_domains_;
+ // If you add more fields here, please be sure to update Merge().
+
+ DISALLOW_COPY_AND_ASSIGN(DomainLawyer);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_DOMAIN_LAWYER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/elide_attributes_filter.h b/trunk/src/net/instaweb/rewriter/public/elide_attributes_filter.h
new file mode 100644
index 0000000..cf227ae
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/elide_attributes_filter.h
@@ -0,0 +1,57 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_ELIDE_ATTRIBUTES_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_ELIDE_ATTRIBUTES_FILTER_H_
+
+#include <map>
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/util/public/atom.h"
+
+namespace net_instaweb {
+
+// Remove attributes and attribute values that can be safely elided.
+class ElideAttributesFilter : public EmptyHtmlFilter {
+ public:
+ explicit ElideAttributesFilter(HtmlParse* html_parse);
+
+ virtual void StartElement(HtmlElement* element);
+ virtual const char* Name() const { return "ElideAttributes"; }
+
+ private:
+ struct AttrValue {
+ const char* attr_value;
+ bool requires_version_5; // Default value only exists in (X)HTML 5.
+ };
+
+ typedef std::map<Atom, AtomSet, AtomCompare> AtomSetMap;
+ typedef std::map<Atom, AttrValue, AtomCompare> ValueMap;
+ typedef std::map<Atom, ValueMap, AtomCompare> ValueMapMap;
+
+ HtmlParse* html_parse_;
+ AtomSetMap one_value_attrs_map_; // tag/attrs with only one possible value
+ ValueMapMap default_value_map_; // tag/attrs with default values
+
+ DISALLOW_COPY_AND_ASSIGN(ElideAttributesFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_ELIDE_ATTRIBUTES_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/file_input_resource.h b/trunk/src/net/instaweb/rewriter/public/file_input_resource.h
new file mode 100644
index 0000000..1ec2fdd
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/file_input_resource.h
@@ -0,0 +1,56 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+//
+// Input resource created based on a local file.
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_FILE_INPUT_RESOURCE_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_FILE_INPUT_RESOURCE_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/resource.h"
+
+namespace net_instaweb {
+
+class FileInputResource : public Resource {
+ public:
+ FileInputResource(ResourceManager* manager,
+ const ContentType* type,
+ const StringPiece& url,
+ const StringPiece& filename)
+ : Resource(manager, type),
+ url_(url.data(), url.size()),
+ filename_(filename.data(), filename.size()) {
+ }
+
+ virtual ~FileInputResource();
+
+ virtual std::string url() const { return url_; }
+
+ protected:
+ virtual bool Load(MessageHandler* message_handler);
+
+ private:
+ std::string url_;
+ std::string filename_;
+
+ DISALLOW_COPY_AND_ASSIGN(FileInputResource);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_FILE_INPUT_RESOURCE_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/html_attribute_quote_removal.h b/trunk/src/net/instaweb/rewriter/public/html_attribute_quote_removal.h
new file mode 100644
index 0000000..eb0f4fa
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/html_attribute_quote_removal.h
@@ -0,0 +1,67 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_HTML_ATTRIBUTE_QUOTE_REMOVAL_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_HTML_ATTRIBUTE_QUOTE_REMOVAL_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+
+namespace net_instaweb {
+// Very simple html filter that removes quotes from attributes that don't need
+// them.
+//
+// From http://www.w3.org/TR/REC-html40/intro/sgmltut.html#h-3.2.2:
+// In certain cases, authors may specify the value of an attribute without any
+// quotation marks. The attribute value may only contain letters (a-z and A-Z),
+// digits (0-9), hyphens (ASCII decimal 45), periods (ASCII decimal 46),
+// underscores (ASCII decimal 95), and colons (ASCII decimal 58).
+//
+// This is an experiment, to see if quote removal *actually* saves bandwidth.
+// After compression it may not (or may not save enough). In that case we
+// should not bother with quote removal.
+
+class HtmlElement;
+class HtmlParse;
+class ResourceManager;
+
+class HtmlAttributeQuoteRemoval : public EmptyHtmlFilter {
+ public:
+ explicit HtmlAttributeQuoteRemoval(HtmlParse* html_parse);
+ // Given context in object, does attribute value val require quotes?
+ bool NeedsQuotes(const char *val);
+ virtual void StartElement(HtmlElement* element);
+ // # of quote pairs removed from attributes in *all* documents processed.
+ int total_quotes_removed() const {
+ return total_quotes_removed_;
+ }
+
+ virtual const char* Name() const { return "HtmlAttributeQuoteRemoval"; }
+
+ private:
+ int total_quotes_removed_;
+ HtmlParse* html_parse_;
+ bool needs_no_quotes_[256]; // lookup chars for quotability
+ // should be const, but C++ initializer rules are broken.
+
+ DISALLOW_COPY_AND_ASSIGN(HtmlAttributeQuoteRemoval);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_HTML_ATTRIBUTE_QUOTE_REMOVAL_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/image.h b/trunk/src/net/instaweb/rewriter/public/image.h
new file mode 100644
index 0000000..6935b22
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/image.h
@@ -0,0 +1,210 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_IMAGE_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_IMAGE_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/image_dim.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#ifdef USE_SYSTEM_OPENCV
+#include "cv.h"
+#else
+#include "third_party/opencv/src/opencv/include/opencv/cv.h"
+#endif
+
+namespace net_instaweb {
+
+struct ContentType;
+struct ImageDim;
+class FileSystem;
+class MessageHandler;
+class Writer;
+
+// The following four helper functions were moved here for testability. We ran
+// into problems with sign extension under different compiler versions, and we'd
+// like to catch regressions on that front in the future.
+
+// char to int *without sign extension*.
+inline int CharToInt(char c) {
+ uint8 uc = static_cast<uint8>(c);
+ return static_cast<int>(uc);
+}
+
+inline int JpegIntAtPosition(const StringPiece& buf, size_t pos) {
+ return (CharToInt(buf[pos]) << 8) |
+ (CharToInt(buf[pos + 1]));
+}
+
+inline int GifIntAtPosition(const StringPiece& buf, size_t pos) {
+ return (CharToInt(buf[pos + 1]) << 8) |
+ (CharToInt(buf[pos]));
+}
+
+inline int PngIntAtPosition(const StringPiece& buf, size_t pos) {
+ return (CharToInt(buf[pos ]) << 24) |
+ (CharToInt(buf[pos + 1]) << 16) |
+ (CharToInt(buf[pos + 2]) << 8) |
+ (CharToInt(buf[pos + 3]));
+}
+
+inline bool PngSectionIdIs(const char* hdr,
+ const StringPiece& buf, size_t pos) {
+ return ((buf[pos + 4] == hdr[0]) &&
+ (buf[pos + 5] == hdr[1]) &&
+ (buf[pos + 6] == hdr[2]) &&
+ (buf[pos + 7] == hdr[3]));
+}
+
+namespace ImageHeaders {
+ // Constants that are shared by Image and its tests.
+ extern const char kPngHeader[];
+ extern const size_t kPngHeaderLength;
+ extern const char kPngIHDR[];
+ extern const size_t kPngIHDRLength;
+ extern const size_t kIHDRDataStart;
+ extern const size_t kPngIntSize;
+
+ extern const char kGifHeader[];
+ extern const size_t kGifHeaderLength;
+ extern const size_t kGifDimStart;
+ extern const size_t kGifIntSize;
+
+ extern const size_t kJpegIntSize;
+} // namespace ImageHeaders
+
+class Image {
+ public:
+ // Images that are in the process of being transformed are represented by an
+ // Image. This class encapsulates various operations that are sensitive to
+ // the format of the compressed image file and of the image libraries we are
+ // using. In particular, the timing of compression and decompression
+ // operations may be a bit unexpected, because we may do these operations
+ // early in order to retrieve image metadata, or we may choose to skip them
+ // entirely if we don't need them or don't understand how to do them.
+ //
+ // In future we may need to plumb this to other data sources or change how
+ // metadata is retrieved; the object is to do so locally in this class without
+ // disrupting any of its clients.
+
+ enum Type {
+ IMAGE_UNKNOWN = 0,
+ IMAGE_JPEG,
+ IMAGE_PNG,
+ IMAGE_GIF,
+ };
+
+ // Image owns none of its inputs. All of the arguments to Image(...) (the
+ // original_contents in particular) must outlive the Image object itself. The
+ // intent is that an Image is created in a scoped fashion from an existing
+ // known resource.
+ Image(const StringPiece& original_contents,
+ const std::string& url,
+ const StringPiece& file_prefix,
+ FileSystem* file_system,
+ MessageHandler* handler);
+
+ ~Image();
+
+ // Stores the image dimensions in natural_dim (on success, sets
+ // natural_dim->{width, height} and natural_dim->valid = true). This method
+ // can fail (natural_dim->valid == false) for various reasons: we don't
+ // understand the image format (eg a gif), we can't find the headers, the
+ // library doesn't support a particular encoding, etc. In that case the other
+ // fields are left alone.
+ void Dimensions(ImageDim* natural_dim);
+
+ // Returns the size of original input in bytes.
+ size_t input_size() const {
+ return original_contents_.size();
+ }
+
+ // Returns the size of output image in bytes.
+ size_t output_size() {
+ size_t ret;
+ if (output_valid_ || ComputeOutputContents()) {
+ ret = output_contents_.size();
+ } else {
+ ret = input_size();
+ }
+ return ret;
+ }
+
+ Type image_type() {
+ if (image_type_ == IMAGE_UNKNOWN) {
+ ComputeImageType();
+ }
+ return image_type_;
+ }
+
+ // Returns true if the image has transparency (an alpha channel, or a
+ // transparent color). Note that certain ambiguously-formatted images might
+ // yield false positive results here; we don't check whether alpha channels
+ // contain non-opaque data, nor do we check if a distinguished transparent
+ // color is actually used in an image. We assume that if the image file
+ // contains flags for transparency, it does so for a reason.
+ bool HasTransparency();
+
+ // Changes the size of the image to the given width and height. This will run
+ // image processing on the image, and return false if the image processing
+ // fails. Otherwise the image contents and type can change.
+ bool ResizeTo(const ImageDim& new_dim);
+
+ // UndoResize lets us bail out if a resize actually cost space!
+ void UndoResize();
+
+ // Returns image-appropriate content type, or NULL if no content type is
+ // known. Result is a top-level const pointer and should not be deleted etc.
+ const ContentType* content_type();
+
+ // Returns the best known image contents. If image type is not understood,
+ // then Contents() will have NULL data().
+ StringPiece Contents();
+
+ private:
+ // Internal methods used only in image.cc (see there for more).
+ void ComputeImageType();
+ void FindJpegSize();
+ inline void FindPngSize();
+ bool ComputePngTransparency();
+ inline void FindGifSize();
+ bool LoadOpenCV();
+ void CleanOpenCV();
+ bool ComputeOutputContents();
+
+ const std::string file_prefix_;
+ FileSystem* file_system_;
+ MessageHandler* handler_;
+ Type image_type_; // Lazily initialized, initially IMAGE_UNKNOWN.
+ const StringPiece original_contents_;
+ std::string output_contents_; // Lazily filled.
+ bool output_valid_; // Indicates output_contents_ now correct.
+ std::string opencv_filename_; // Lazily holds generated name of temp file.
+ IplImage* opencv_image_; // Lazily filled on OpenCV load.
+ bool opencv_load_possible_; // Attempt opencv_load in future?
+ bool resized_;
+ const std::string url_;
+ ImageDim dims_;
+
+ DISALLOW_COPY_AND_ASSIGN(Image);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_IMAGE_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/image_dim.h b/trunk/src/net/instaweb/rewriter/public/image_dim.h
new file mode 100644
index 0000000..0fc375e
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/image_dim.h
@@ -0,0 +1,60 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_IMAGE_DIM_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_IMAGE_DIM_H_
+
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+// Specification for image dimensions, either in an image file
+// or in the page source. This is 1 method away from a struct.
+class ImageDim {
+ public:
+ // Constructor ensures repeatable field values.
+ ImageDim() : valid_(false), width_(-1), height_(-1) { }
+
+ int width() const { return width_; }
+ int height() const { return height_; }
+ bool valid() const { return valid_; }
+
+ void invalidate() { valid_ = false; }
+ void set_dims(int width, int height) {
+ width_ = width;
+ height_ = height;
+ valid_ = true;
+ }
+
+ // *Append* encoding of this to *out.
+ void EncodeTo(std::string* out) const;
+
+ // Decode encoding from "in", truncating it to remove consumed data.
+ // Invalidate this and return false on parse failure; "in" is in an
+ // arbitrary state in this case.
+ bool DecodeFrom(StringPiece* in);
+
+ private:
+ bool valid_; // If false, other two fields have arbitrary values.
+ int width_;
+ int height_;
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_IMAGE_DIM_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/img_rewrite_filter.h b/trunk/src/net/instaweb/rewriter/public/img_rewrite_filter.h
new file mode 100644
index 0000000..dbc2e42
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/img_rewrite_filter.h
@@ -0,0 +1,152 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_IMG_REWRITE_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_IMG_REWRITE_FILTER_H_
+
+#include "net/instaweb/rewriter/public/rewrite_filter.h"
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/rewriter/public/img_tag_scanner.h"
+#include "net/instaweb/rewriter/public/resource.h"
+#include "net/instaweb/util/public/atom.h"
+#include <string>
+#include "net/instaweb/util/public/url_segment_encoder.h"
+#include "net/instaweb/util/public/work_bound.h"
+
+namespace net_instaweb {
+
+class ContentType;
+class FileSystem;
+class HtmlParse;
+class Image;
+class ImageDim;
+class OutputResource;
+class ResourceManager;
+class UrlEscaper;
+class Variable;
+
+// This class supports the encoding of image urls with optional
+// additional dimension metadata. The passed-in stored_dim is used as
+// the source and/or destination of this metadata during encode/decode
+// respectively.
+class ImageUrlEncoder : public UrlSegmentEncoder {
+ public:
+ ImageUrlEncoder(UrlEscaper* url_escaper, ImageDim* stored_dim);
+ virtual ~ImageUrlEncoder();
+
+ // Encode an origin_url and stored_dim from origin page to a rewritten_url.
+ virtual void EncodeToUrlSegment(
+ const StringPiece& origin_url, std::string* rewritten_url);
+
+ // Decode an origin_url and stored_dim from a rewritten_url, returning false
+ // on parse failure (invalidating output vars).
+ virtual bool DecodeFromUrlSegment(const StringPiece& rewritten_url,
+ std::string* origin_url);
+
+ private:
+ UrlEscaper* url_escaper_;
+ ImageDim* stored_dim_;
+};
+
+// Identify img tags in html and optimize them.
+// TODO(jmaessen): See which ones have immediately-obvious size info.
+// TODO(jmaessen): Provide alternate resources at rewritten urls
+// asynchronously somehow.
+// TODO(jmaessen): Big open question: how best to link pulled-in resources to
+// rewritten urls, when in general those urls will be in a different domain.
+class ImgRewriteFilter : public RewriteFilter {
+ public:
+ ImgRewriteFilter(RewriteDriver* driver,
+ bool log_image_elements,
+ bool insert_image_dimensions,
+ StringPiece path_prefix,
+ size_t img_inline_max_bytes,
+ size_t img_max_rewrites_at_once);
+ static void Initialize(Statistics* statistics);
+ virtual void StartDocumentImpl() {}
+ virtual void StartElementImpl(HtmlElement* element) {}
+ virtual void EndElementImpl(HtmlElement* element);
+ virtual void Flush();
+ virtual bool Fetch(OutputResource* resource,
+ Writer* writer,
+ const MetaData& request_header,
+ MetaData* response_headers,
+ MessageHandler* message_handler,
+ UrlAsyncFetcher::Callback* callback);
+ virtual const char* Name() const { return "ImgRewrite"; }
+
+ // Can we inline resource? If so, encode its contents into the data_url,
+ // otherwise leave data_url alone.
+ static bool CanInline(
+ int img_inline_max_bytes, const StringPiece& contents,
+ const ContentType* content_type, std::string* data_url);
+
+ private:
+ // Helper methods.
+ Image* GetImage(const StringPiece& origin_url, Resource* img_resource);
+ OutputResource* ImageOutputResource(const std::string& url_string,
+ Image* image);
+ const ContentType* ImageToContentType(const std::string& origin_url,
+ Image* image);
+ void OptimizeImage(const Resource& input_resource, const ImageDim& page_dim,
+ Image* image, OutputResource* result);
+ bool OptimizedImageFor(
+ const StringPiece& origin_url, const ImageDim& page_dim,
+ Resource* img_resource, OutputResource* output);
+ void RewriteImageUrl(HtmlElement* element, HtmlElement::Attribute* src);
+ void UpdateTargetElement(const Resource& input_resource,
+ const OutputResource& output_resource,
+ const ImageDim& page_dim, const ImageDim& actual_dim,
+ HtmlElement* element, HtmlElement::Attribute* src);
+
+ FileSystem* file_system_;
+ HtmlParse* html_parse_;
+ scoped_ptr<const ImgTagScanner> img_filter_;
+ ResourceManager* resource_manager_;
+ scoped_ptr<WorkBound> work_bound_;
+ // Threshold size (in bytes) below which we should just inline images
+ // encountered.
+ // TODO(jmaessen): Heuristic must be more sophisticated. Does this image
+ // touch a fresh domain? Require opening a new connection? If so we can
+ // afford to inline quite large images (basically anything we could transmit
+ // in the resulting RTTs)---but of course we don't know about RTT here. In
+ // the absence of such information, we ought to inline if header length + url
+ // size can be saved by inlining image, without increasing the size in packets
+ // of the html. Otherwise we end up loading the image in favor of the html,
+ // which might be a lose. More work is needed here to figure out the exact
+ // tradeoffs involved, especially as we also undermine image cacheability.
+ size_t img_inline_max_bytes_;
+ // Should we log each image element as we encounter it? Handy for debug.
+ bool log_image_elements_;
+ // Should we insert image dimensions into html if they are absent?
+ bool insert_image_dimensions_;
+ const Atom s_width_;
+ const Atom s_height_;
+ Variable* rewrite_count_;
+ Variable* inline_count_;
+ Variable* rewrite_saved_bytes_;
+
+ DISALLOW_COPY_AND_ASSIGN(ImgRewriteFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_IMG_REWRITE_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/img_tag_scanner.h b/trunk/src/net/instaweb/rewriter/public/img_tag_scanner.h
new file mode 100644
index 0000000..739356c
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/img_tag_scanner.h
@@ -0,0 +1,48 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_IMG_TAG_SCANNER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_IMG_TAG_SCANNER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/util/public/atom.h"
+
+namespace net_instaweb {
+class HtmlParse;
+
+class ImgTagScanner {
+ public:
+ explicit ImgTagScanner(HtmlParse* html_parse);
+
+ // Examine HTML element and determine if it is an img with a src. If so
+ // extract the src attribute and return it, otherwise return NULL.
+ HtmlElement::Attribute* ParseImgElement(HtmlElement* element) const;
+
+ private:
+ const Atom s_img_;
+ const Atom s_input_;
+ const Atom s_src_;
+ const Atom s_type_;
+
+ DISALLOW_COPY_AND_ASSIGN(ImgTagScanner);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_IMG_TAG_SCANNER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/javascript_code_block.h b/trunk/src/net/instaweb/rewriter/public/javascript_code_block.h
new file mode 100644
index 0000000..03af5bc
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/javascript_code_block.h
@@ -0,0 +1,144 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_JAVASCRIPT_CODE_BLOCK_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_JAVASCRIPT_CODE_BLOCK_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/javascript_library_identification.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/statistics.h"
+
+namespace net_instaweb {
+
+class MessageHandler;
+class Statistics;
+class Variable;
+
+// Class wrapping up configuration information for javascript
+// rewriting, in order to minimize footprint of later changes
+// to javascript rewriting.
+class JavascriptRewriteConfig {
+ public:
+ explicit JavascriptRewriteConfig(Statistics* statistics);
+ static void Initialize(Statistics* statistics);
+ // Whether to minify javascript output (using jsminify).
+ // true by default.
+ bool minify() const {
+ return minify_;
+ }
+ void set_minify(bool minify) {
+ minify_ = minify;
+ }
+ // whether to redirect external javascript libraries to
+ // Google-as-a-CDN
+ bool redirect() const {
+ return redirect_;
+ }
+ void set_redirect(bool redirect) {
+ redirect_ = redirect;
+ }
+ void AddBytesSaved(size_t bytes) {
+ if (bytes_saved_ != NULL) {
+ bytes_saved_->Add(bytes);
+ blocks_minified_->Add(1);
+ }
+ }
+ void AddMinificationFailure() {
+ if (minification_failures_ != NULL) {
+ minification_failures_->Add(1);
+ }
+ }
+ void AddBlock() {
+ if (total_blocks_ != NULL) {
+ total_blocks_->Add(1);
+ }
+ }
+ private:
+ bool minify_;
+ bool redirect_;
+ Variable* blocks_minified_;
+ Variable* bytes_saved_;
+ Variable* minification_failures_;
+ Variable* total_blocks_;
+
+ DISALLOW_COPY_AND_ASSIGN(JavascriptRewriteConfig);
+};
+
+// Object representing a block of Javascript code that might be a
+// candidate for rewriting.
+// TODO(jmaessen): Does this architecture make sense when we have
+// multiple scripts on a page and the ability to move code around
+// a bunch? How do we maintain JS context in that setting?
+//
+// For now, we're content just being able to pull data in and parse it at all.
+class JavascriptCodeBlock {
+ public:
+ JavascriptCodeBlock(const StringPiece& original_code,
+ JavascriptRewriteConfig* config,
+ MessageHandler* handler);
+
+ virtual ~JavascriptCodeBlock();
+
+ // Is it profitable to replace js code with rewritten version?
+ bool ProfitableToRewrite() {
+ RewriteIfNecessary();
+ return (output_code_.size() < original_code_.size());
+ }
+ // TODO(jmaessen): Other questions we might reasonably ask:
+ // Can this code be floated downwards?
+
+ // Returns the current (maximally-rewritten) contents of the
+ // code block.
+ const StringPiece Rewritten() {
+ RewriteIfNecessary();
+ return output_code_;
+ }
+
+ // Is the current block a JS library that can be redirected to Google?
+ // If so, return the info necessary to do so. Otherwise returns a
+ // block for which .recognized() is false.
+ const JavascriptLibraryId ComputeJavascriptLibrary();
+
+ private:
+ void RewriteIfNecessary() {
+ if (!rewritten_) {
+ Rewrite();
+ rewritten_ = true;
+ }
+ }
+
+ protected:
+ void Rewrite();
+
+ JavascriptRewriteConfig* config_;
+ MessageHandler* handler_;
+ const std::string original_code_;
+ // Note that output_code_ points to either original_code_ or
+ // to rewritten_code_ depending upon the results of processing
+ // (ie it's an indirection to locally-owned data).
+ StringPiece output_code_;
+ bool rewritten_;
+ std::string rewritten_code_;
+
+ DISALLOW_COPY_AND_ASSIGN(JavascriptCodeBlock);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_JAVASCRIPT_CODE_BLOCK_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/javascript_filter.h b/trunk/src/net/instaweb/rewriter/public/javascript_filter.h
new file mode 100644
index 0000000..59367eb
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/javascript_filter.h
@@ -0,0 +1,121 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_JAVASCRIPT_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_JAVASCRIPT_FILTER_H_
+
+#include <vector>
+
+#include "net/instaweb/rewriter/public/rewrite_single_resource_filter.h"
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/rewriter/public/javascript_code_block.h"
+#include "net/instaweb/rewriter/public/script_tag_scanner.h"
+#include "net/instaweb/util/public/atom.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+
+namespace net_instaweb {
+class HtmlParse;
+class MessageHandler;
+class MetaData;
+class OutputResource;
+class Resource;
+class ResourceManager;
+class Statistics;
+class Writer;
+
+/**
+ * Find Javascript elements (either inline scripts or imported js files) and
+ * rewrite them. This can involve any combination of minifaction,
+ * concatenation, renaming, reordering, and incrementalization that accomplishes
+ * our goals.
+ *
+ * For the moment we keep it simple and just minify any scripts that we find.
+ *
+ * Challenges:
+ * * Identifying everywhere js is invoked, in particular event handlers on
+ * elements that might be found in css or in variously-randomly-named
+ * html properties.
+ * * Analysis of eval() contexts. Actually less hard than the last, assuming
+ * constant strings. Otherwise hard.
+ * * Figuring out where to re-inject code after analysis.
+ *
+ * We will probably need to do an end run around the need for js analysis by
+ * instrumenting and incrementally loading code, then probably using dynamic
+ * feedback to change the runtime instrumentation in future pages as we serve
+ * them.
+ */
+class JavascriptFilter : public RewriteSingleResourceFilter {
+ public:
+ JavascriptFilter(RewriteDriver* rewrite_driver,
+ const StringPiece& path_prefix);
+ virtual ~JavascriptFilter();
+ static void Initialize(Statistics* statistics);
+
+ virtual void StartDocumentImpl() {}
+ virtual void StartElementImpl(HtmlElement* element);
+ virtual void Characters(HtmlCharactersNode* characters);
+ virtual void EndElementImpl(HtmlElement* element);
+ virtual void Flush();
+ virtual void IEDirective(HtmlIEDirectiveNode* directive);
+
+ // Configuration settings for javascript filtering:
+ // Set whether to minify javascript code blocks encountered.
+ void set_minify(bool minify) {
+ config_.set_minify(minify);
+ }
+
+ virtual const char* Name() const { return "Javascript"; }
+
+ protected:
+ virtual bool RewriteLoadedResource(const Resource* input_resource,
+ OutputResource* output_resource);
+
+ private:
+ inline void CompleteScriptInProgress();
+ inline void RewriteInlineScript();
+ inline void RewriteExternalScript();
+ inline Resource* ScriptAtUrl(const StringPiece& script_url);
+ const StringPiece FlattenBuffer(std::string* script_buffer);
+ bool WriteExternalScriptTo(const Resource* script_resource,
+ const StringPiece& script_out,
+ OutputResource* script_dest);
+
+ std::vector<HtmlCharactersNode*> buffer_;
+ HtmlParse* html_parse_;
+ HtmlElement* script_in_progress_;
+ HtmlElement::Attribute* script_src_;
+ ResourceManager* resource_manager_;
+ // some_missing_scripts indicates that we stopped processing a script and
+ // therefore can't assume we know all of the Javascript on a page.
+ bool some_missing_scripts_;
+ JavascriptRewriteConfig config_;
+ const Atom s_script_;
+ const Atom s_src_;
+ const Atom s_type_;
+ ScriptTagScanner script_tag_scanner_;
+
+ DISALLOW_COPY_AND_ASSIGN(JavascriptFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_JAVASCRIPT_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/javascript_library_identification.h b/trunk/src/net/instaweb/rewriter/public/javascript_library_identification.h
new file mode 100644
index 0000000..0e6e53c
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/javascript_library_identification.h
@@ -0,0 +1,76 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_JAVASCRIPT_LIBRARY_IDENTIFICATION_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_JAVASCRIPT_LIBRARY_IDENTIFICATION_H_
+
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+// Size of regions that is hashed to do initial detection
+const size_t kJavascriptHashIdBlockSize = 512;
+
+// Internal data used by JavascriptLibraryId.
+// We expose it here to allow the metadata to be
+// constructed sensibly by generate_javascript_metadata.
+struct LibraryInfo {
+ const char* name;
+ const char* version;
+ uint64 first_block_hash;
+ uint64 full_hash;
+ size_t full_size;
+};
+
+// A JavascriptLibraryId contains information about a single third-party
+// Javascript library. Given a block of minified Javascript with leading and
+// trailing whitespace removed, the Find(...) method returns a corresponding
+// JavascriptLibraryId. Note that JavascriptLibraryId is intended to be passed
+// by value.
+class JavascriptLibraryId {
+ public:
+ // Constructs an unrecognized library.
+ JavascriptLibraryId();
+
+ // Find the JavascriptLibraryId object associated with the given
+ // minified_code. This might be an unrecognized library.
+ static JavascriptLibraryId Find(const StringPiece& minified_code);
+
+ // Is this a recognized library? Otherwise we should ignore it.
+ bool recognized() const {
+ return info_->name != NULL;
+ }
+
+ // Canonical name of javascript library (NULL if unrecognized)
+ const char* name() const {
+ return info_->name;
+ }
+
+ // Version number of javascript library (NULL if unrecognized)
+ const char* version() const {
+ return info_->version;
+ }
+
+ private:
+ explicit JavascriptLibraryId(const LibraryInfo* info) : info_(info) { }
+
+ const LibraryInfo* info_;
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_JAVASCRIPT_LIBRARY_IDENTIFICATION_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/js_inline_filter.h b/trunk/src/net/instaweb/rewriter/public/js_inline_filter.h
new file mode 100644
index 0000000..a39c3df
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/js_inline_filter.h
@@ -0,0 +1,67 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_JS_INLINE_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_JS_INLINE_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/common_filter.h"
+#include "net/instaweb/rewriter/public/script_tag_scanner.h"
+#include "net/instaweb/util/public/atom.h"
+#include <string>
+
+namespace net_instaweb {
+
+class ResourceManager;
+class RewriteDriver;
+
+// Inline small Javascript files.
+class JsInlineFilter : public CommonFilter {
+ public:
+ explicit JsInlineFilter(RewriteDriver* driver);
+ virtual ~JsInlineFilter();
+
+ virtual void StartDocumentImpl();
+ virtual void EndDocument();
+ virtual void StartElementImpl(HtmlElement* element);
+ virtual void EndElementImpl(HtmlElement* element);
+ virtual void Characters(HtmlCharactersNode* characters);
+ virtual const char* Name() const { return "InlineJs"; }
+
+ private:
+ HtmlParse* const html_parse_;
+ const Atom script_atom_;
+ const Atom src_atom_;
+ const size_t size_threshold_bytes_;
+ ScriptTagScanner script_tag_scanner_;
+
+ std::string domain_; // The domain of the HTML file we're parsing
+
+ // This is set to true during StartElement() for a <script> tag that we
+ // should maybe inline, but may be set back to false by Characters(). If it
+ // is still true when we hit the corresponding EndElement(), then we'll
+ // inline the script (and set it back to false). It should never be true
+ // outside of <script> and </script>.
+ bool should_inline_;
+
+ DISALLOW_COPY_AND_ASSIGN(JsInlineFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_JS_INLINE_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/js_outline_filter.h b/trunk/src/net/instaweb/rewriter/public/js_outline_filter.h
new file mode 100644
index 0000000..1cefc2d
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/js_outline_filter.h
@@ -0,0 +1,87 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_JS_OUTLINE_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_JS_OUTLINE_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+#include "net/instaweb/rewriter/public/script_tag_scanner.h"
+#include "net/instaweb/util/public/atom.h"
+#include <string>
+
+namespace net_instaweb {
+
+class MessageHandler;
+class MetaData;
+class OutputResource;
+class ResourceManager;
+
+// Filter to take explicit <style> and <script> tags and outline them to files.
+class JsOutlineFilter : public HtmlFilter {
+ public:
+ explicit JsOutlineFilter(RewriteDriver* driver);
+ static const char kFilterId[];
+
+ virtual void StartDocument();
+
+ virtual void StartElement(HtmlElement* element);
+ virtual void EndElement(HtmlElement* element);
+
+ virtual void Flush();
+
+ // HTML Events we expect to be in <script> elements.
+ virtual void Characters(HtmlCharactersNode* characters);
+
+ // HTML Events we do not expect to be in <style> and <script> elements.
+ virtual void Comment(HtmlCommentNode* comment);
+ virtual void Cdata(HtmlCdataNode* cdata);
+ virtual void IEDirective(HtmlIEDirectiveNode* directive);
+
+ // Ignored HTML Events.
+ virtual void EndDocument() {}
+ virtual void Directive(HtmlDirectiveNode* directive) {}
+
+ virtual const char* Name() const { return "OutlineJs"; }
+
+ private:
+ bool WriteResource(const std::string& content, OutputResource* resource,
+ MessageHandler* handler);
+ void OutlineScript(HtmlElement* element, const std::string& content);
+
+ // The style or script element we are in (if it hasn't been flushed).
+ // If we are not in a script or style element, inline_element_ == NULL.
+ HtmlElement* inline_element_;
+ // Temporarily buffers the content between open and close of inline_element_.
+ std::string buffer_;
+ HtmlParse* html_parse_;
+ ResourceManager* resource_manager_;
+ size_t size_threshold_bytes_;
+ // HTML strings interned into a symbol table.
+ Atom s_script_;
+ Atom s_src_;
+ Atom s_type_;
+ ScriptTagScanner script_tag_scanner_;
+
+ DISALLOW_COPY_AND_ASSIGN(JsOutlineFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_JS_OUTLINE_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/output_resource.h b/trunk/src/net/instaweb/rewriter/public/output_resource.h
new file mode 100644
index 0000000..c381dee
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/output_resource.h
@@ -0,0 +1,179 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+//
+// Output resources are created by a ResourceManager. They must be able to
+// write contents and return their url (so that it can be href'd on a page).
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_OUTPUT_RESOURCE_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_OUTPUT_RESOURCE_H_
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/file_writer.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/rewriter/public/resource.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/resource_namer.h"
+
+namespace net_instaweb {
+
+class AbstractLock;
+class MessageHandler;
+class NamedLockManager;
+
+class OutputResource : public Resource {
+ public:
+ // Construct an OutputResource. For the moment, we pass in type redundantly
+ // even though full_name embeds an extension. This reflects current code
+ // structure rather than a principled stand on anything.
+ // TODO(jmaessen): remove redundancy.
+ OutputResource(ResourceManager* manager,
+ const StringPiece& resolved_base,
+ const ResourceNamer& resource_id,
+ const ContentType* type);
+ ~OutputResource();
+
+ virtual bool Load(MessageHandler* message_handler);
+ virtual std::string url() const;
+
+ // The NameKey describes the source url and rewriter used, without hash and
+ // content type information. This is used to find previously-computed filter
+ // results whose output hash and content type is unknown. The full name of a
+ // resource is of the form
+ // path/prefix.encoded_resource_name.hash.extension
+ // we know prefix and name, but not the hash, and we don't always even have
+ // the extension, which might have changes as the result of, for example image
+ // optimization (e.g. gif->png). But We can "remember" the hash/extension for
+ // as long as the origin URL was cacheable. So we construct this as a key:
+ // path/prefix.encoded_resource_name
+ // and use that to map to the hash-code and extension. If we know the
+ // hash-code then we may also be able to look up the contents in the same
+ // cache.
+ virtual std::string name_key() const;
+ // The hash_ext describes the hash and content type of the resource;
+ // to index already-computed resources we lookup name_key() and obtain
+ // the corresponding hash_ext().
+ virtual std::string hash_ext() const;
+
+ // output-specific
+ const std::string& resolved_base() const { return resolved_base_; }
+ const ResourceNamer& full_name() const { return full_name_; }
+ StringPiece name() const { return full_name_.name(); }
+ std::string filename() const;
+ StringPiece suffix() const;
+ StringPiece filter_prefix() const { return full_name_.id(); }
+
+ // In a scalable installation where the sprites must be kept in a
+ // database, we cannot serve HTML that references new resources
+ // that have not been committed yet, and committing to a database
+ // may take too long to block on the HTML rewrite. So we will want
+ // to refactor this to check to see whether the desired resource is
+ // already known. For now we'll assume we can commit to serving the
+ // resource during the HTML rewriter.
+ bool IsWritten() const;
+
+ // Sets the suffix for an output resource. This must be called prior
+ // to Write if the content_type ctor arg was NULL. This can happen if
+ // we are managing a resource whose content-type is not known to us.
+ // CacheExtender is currently the only place where we need this.
+ void set_suffix(const StringPiece& ext);
+
+ // Sets the type of the output resource, and thus also its suffix.
+ virtual void SetType(const ContentType* type);
+
+ // Determines whether the output resource has a valid URL. If so,
+ // we don't need to actually load the output-resource content from
+ // cache during the Rewriting process -- we can immediately rewrite
+ // the href to it.
+ //
+ // Note that when serving content, we must actually load it, but
+ // when rewriting it we can, in some cases, exploit a URL swap.
+ bool HasValidUrl() const { return has_hash(); }
+
+ // Resources rewritten via a UrlPartnership will have a resolved
+ // base to use in lieu of the legacy UrlPrefix held by the resource
+ // manager.
+ void set_resolved_base(const StringPiece& base) {
+ base.CopyToString(&resolved_base_);
+ CHECK(EndsInSlash(base)) << "resolved_base must end in a slash.";
+ }
+
+ private:
+ friend class ResourceManager;
+ friend class ResourceManagerTestingPeer;
+ class OutputWriter {
+ public:
+ // file may be null if we shouldn't write to the filesystem.
+ OutputWriter(FileSystem::OutputFile* file, HTTPValue* http_value)
+ : http_value_(http_value) {
+ if (file != NULL) {
+ file_writer_.reset(new FileWriter(file));
+ } else {
+ file_writer_.reset(NULL);
+ }
+ }
+
+ // Adds the given data to our http_value, and, if
+ // non-null, our file.
+ bool Write(const StringPiece& data, MessageHandler* handler);
+ private:
+ scoped_ptr<FileWriter> file_writer_;
+ HTTPValue* http_value_;
+ };
+
+ void SetHash(const StringPiece& hash);
+ StringPiece hash() const { return full_name_.hash(); }
+ bool has_hash() const { return !hash().empty(); }
+ void set_written(bool written) { writing_complete_ = true; }
+ void set_generated(bool x) { generated_ = x; }
+ bool generated() const { return generated_; }
+ std::string TempPrefix() const;
+
+ OutputWriter* BeginWrite(MessageHandler* message_handler);
+ bool EndWrite(OutputWriter* writer, MessageHandler* message_handler);
+ // Attempt to obtain a named lock for the resource. Return true if we do so.
+ bool LockForCreation(const ResourceManager* resource_manager,
+ ResourceManager::BlockingBehavior block);
+
+ FileSystem::OutputFile* output_file_;
+ bool writing_complete_;
+
+ // Generated via ResourceManager::CreateGeneratedOutputResource,
+ // meaning that it does not have a name that is derived from an
+ // input URL. We must regenerate it every time, but the output name
+ // will be distinct because it's based on the hash of the content.
+ bool generated_;
+
+ // If this output url was created via a partnership then this field
+ // will be non-empty, and we will not need to use the resource manager's
+ // prefix.
+ std::string resolved_base_;
+ ResourceNamer full_name_;
+
+ // Lock guarding resource creation. Lazily initialized by LockForCreation,
+ // unlocked on destruction or EndWrite.
+ scoped_ptr<AbstractLock> creation_lock_;
+
+ DISALLOW_COPY_AND_ASSIGN(OutputResource);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_OUTPUT_RESOURCE_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/remove_comments_filter.h b/trunk/src/net/instaweb/rewriter/public/remove_comments_filter.h
new file mode 100644
index 0000000..27857c2
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/remove_comments_filter.h
@@ -0,0 +1,46 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_REMOVE_COMMENTS_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_REMOVE_COMMENTS_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+
+namespace net_instaweb {
+
+// Reduce the size of the HTML by removing all HTML comments (except those
+// which are IE directives). Note that this is a potentially dangerous
+// optimization; if a site is using comments for some squirrelly purpose, then
+// removing those comments might break something.
+class RemoveCommentsFilter : public EmptyHtmlFilter {
+ public:
+ explicit RemoveCommentsFilter(HtmlParse* html_parse);
+
+ virtual void Comment(HtmlCommentNode* comment);
+ virtual const char* Name() const { return "RemoveComments"; }
+
+ private:
+ HtmlParse* html_parse_;
+
+ DISALLOW_COPY_AND_ASSIGN(RemoveCommentsFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_REMOVE_COMMENTS_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/resource.h b/trunk/src/net/instaweb/rewriter/public/resource.h
new file mode 100644
index 0000000..4a57e56
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/resource.h
@@ -0,0 +1,115 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+// jmarantz@google.com (Joshua Marantz)
+//
+// Resources are created by a ResourceManager. Input resources are
+// read from URLs or the file system. Output resources are constructed
+// programatically, usually by transforming one or more existing
+// resources. Both input and output resources inherit from this class
+// so they can be used interchangably in successive rewrite passes.
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/http_value.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+
+namespace net_instaweb {
+
+class ResourceManager;
+
+class Resource {
+ public:
+ static const int64 kDefaultExpireTimeMs;
+
+ Resource(ResourceManager* manager, const ContentType* type)
+ : resource_manager_(manager),
+ type_(type) {
+ }
+ virtual ~Resource();
+
+ // Common methods across all deriviations
+ ResourceManager* resource_manager() const { return resource_manager_; }
+ bool loaded() const { return meta_data_.status_code() != 0; }
+ // TODO(sligocki): Change name to HttpStatusOk?
+ bool ContentsValid() const {
+ return (meta_data_.status_code() == HttpStatus::kOK);
+ }
+ int64 CacheExpirationTimeMs() const;
+ StringPiece contents() const {
+ StringPiece val;
+ bool got_contents = value_.ExtractContents(&val);
+ CHECK(got_contents) << "Resource contents read before loading";
+ return val;
+ }
+ MetaData* metadata() { return &meta_data_; }
+ const MetaData* metadata() const { return &meta_data_; }
+ const ContentType* type() const { return type_; }
+ virtual void SetType(const ContentType* type);
+ virtual bool IsCacheable() const;
+
+ // Gets the absolute URL of the resource
+ virtual std::string url() const = 0;
+
+ virtual void DetermineContentType();
+
+ // We define a new Callback type here because we need to
+ // pass in the Resource to the Done callback so it can
+ // collect the fetched data.
+ class AsyncCallback {
+ public:
+ virtual ~AsyncCallback();
+ virtual void Done(bool success, Resource* resource) = 0;
+ };
+
+ // Links in the HTTP contents and header from a fetched value.
+ // The contents are linked by sharing. The HTTPValue also
+ // contains a serialization of the headers, and this routine
+ // parses them into meta_data_ and return whether that was
+ // successful.
+ bool Link(HTTPValue* source, MessageHandler* handler);
+
+ protected:
+ friend class ResourceManager;
+ friend class UrlReadAsyncFetchCallback;
+
+ // Load the resource asynchronously, storing MetaData and contents in cache.
+ // Returns true, if the resource is already loaded or loaded synchronously.
+ virtual bool Load(MessageHandler* message_handler) = 0;
+ // Same as Load, but calls a callback when finished.
+ virtual void LoadAndCallback(AsyncCallback* callback,
+ MessageHandler* message_handler);
+
+ ResourceManager* resource_manager_;
+
+ const ContentType* type_;
+ HTTPValue value_; // contains contents and meta-data
+ SimpleMetaData meta_data_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(Resource);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/resource_manager.h b/trunk/src/net/instaweb/rewriter/public/resource_manager.h
new file mode 100644
index 0000000..eb9432f
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/resource_manager.h
@@ -0,0 +1,287 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+// and sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_MANAGER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_MANAGER_H_
+
+#include <map>
+#include <vector>
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/http_cache.h"
+#include "net/instaweb/util/public/meta_data.h"
+#include "net/instaweb/rewriter/public/resource.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+#include "net/instaweb/util/public/url_segment_encoder.h"
+
+class GURL;
+
+namespace net_instaweb {
+
+class ContentType;
+class DomainLawyer;
+class FileSystem;
+class FilenameEncoder;
+class HTTPCache;
+class HTTPValue;
+class Hasher;
+class MessageHandler;
+class MetaData;
+class NamedLockManager;
+class OutputResource;
+class ResourceNamer;
+class RewriteOptions;
+class Statistics;
+class UrlAsyncFetcher;
+class UrlEscaper;
+class Variable;
+class Writer;
+
+class ResourceManager {
+ public:
+ enum BlockingBehavior { kNeverBlock, kMayBlock };
+
+ static const int kNotSharded;
+
+ // This value is a shared constant so that it can also be used in
+ // the Apache-specific code that repairs our caching headers downstream
+ // of mod_headers.
+ static const char kResourceEtagValue[];
+
+ ResourceManager(const StringPiece& file_prefix,
+ FileSystem* file_system,
+ FilenameEncoder* filename_encoder,
+ UrlAsyncFetcher* url_async_fetcher,
+ Hasher* hasher,
+ HTTPCache* http_cache,
+ NamedLockManager* lock_manager);
+ ~ResourceManager();
+
+ // Initialize statistics gathering.
+ static void Initialize(Statistics* statistics);
+
+ // Created resources are managed by ResourceManager and eventually deleted by
+ // ResourceManager's destructor. Every time a Create...Resource... method is
+ // called, a fresh Resource object is generated (or the creation fails and
+ // NULL is returned). All content_type arguments can be NULL if the content
+ // type isn't known or isn't covered by the ContentType library. Where
+ // necessary, the extension is used to infer a content type if one is needed
+ // and none is provided. It is faster and more reliable to provide one
+ // explicitly when it is known.
+
+ // Constructs an output resource corresponding to the specified input resource
+ // and encoded using the provided encoder. Assumes permissions checking
+ // occurred when the input resource was constructed, and does not do it again.
+ // To avoid if-chains, tolerates a NULL input_resource (by returning NULL).
+ // TODO(jmaessen, jmarantz): Do we want to permit NULL input_resources here?
+ // jmarantz has evinced a distaste.
+ OutputResource* CreateOutputResourceFromResource(
+ const StringPiece& filter_prefix,
+ const ContentType* content_type,
+ UrlSegmentEncoder* encoder,
+ Resource* input_resource,
+ MessageHandler* handler);
+
+ // Constructs and permissions-checks an output resource for the specified url,
+ // which occurs in the context of document_gurl. Returns NULL on failure.
+ // The content_type argument cannot be NULL. The resource name will be
+ // encoded using the provided encoder.
+ OutputResource* CreateOutputResourceForRewrittenUrl(
+ const GURL& document_gurl,
+ const StringPiece& filter_prefix,
+ const StringPiece& resource_url,
+ const ContentType* content_type,
+ UrlSegmentEncoder* encoder,
+ const RewriteOptions* rewrite_options,
+ MessageHandler* handler);
+
+ // Creates an output resource where the name is provided by the rewriter.
+ // The intent is to be able to derive the content from the name, for example,
+ // by encoding URLs and metadata.
+ //
+ // This method is not dependent on shared persistent storage, and always
+ // succeeds.
+ //
+ // This name is prepended with path for writing hrefs, and the resulting url
+ // is encoded and stored at file_prefix when working with the file system. So
+ // hrefs are:
+ // $(PATH)/$(FILTER_PREFIX).$(HASH).$(NAME).$(CONTENT_TYPE_EXT)
+ //
+ // 'type' arg can be null if it's not known, or is not in our ContentType
+ // library.
+ OutputResource* CreateOutputResourceWithPath(
+ const StringPiece& path, const StringPiece& filter_prefix,
+ const StringPiece& name, const ContentType* type,
+ MessageHandler* handler);
+
+ // Creates a resource based on a URL. This is used for serving rewritten
+ // resources. No permission checks are performed on the url, though it
+ // is parsed to see if it looks like the url of a generated resource (which
+ // should mean checking the hash to ensure we generated it ourselves).
+ // TODO(jmaessen): add url hash & check thereof.
+ OutputResource* CreateOutputResourceForFetch(
+ const StringPiece& url);
+
+ // Creates an input resource with the url evaluated based on input_url
+ // which may need to be absolutified relative to base_url. Returns NULL if
+ // the input resource url isn't valid, or can't legally be rewritten in the
+ // context of this page.
+ Resource* CreateInputResource(const GURL& base_url,
+ const StringPiece& input_url,
+ const RewriteOptions* rewrite_options,
+ MessageHandler* handler);
+
+ // Create input resource from input_url, if it is legal in the context of
+ // base_gurl, and if the resource can be read from cache. If it's not in
+ // cache, initiate an asynchronous fetch so it will be on next access. This
+ // is a common case for filters.
+ Resource* CreateInputResourceAndReadIfCached(
+ const GURL& base_gurl, const StringPiece& input_url,
+ const RewriteOptions* rewrite_options, MessageHandler* handler);
+
+ // Create an input resource by decoding output_resource using the given
+ // encoder. Assures legality by checking hash signatures, rather than
+ // explicitly permission-checking the result.
+ Resource* CreateInputResourceFromOutputResource(
+ UrlSegmentEncoder* encoder,
+ OutputResource* output_resource,
+ const RewriteOptions* rewrite_options,
+ MessageHandler* handler);
+
+ // Creates an input resource from the given absolute url. Requires that the
+ // provided url has been checked, and can legally be rewritten in the current
+ // page context. If you have a GURL, prefer CreateInputResourceUnchecked,
+ // otherwise use this.
+ Resource* CreateInputResourceAbsolute(const StringPiece& absolute_url,
+ const RewriteOptions* rewrite_options,
+ MessageHandler* handler);
+
+ // Creates an input resource with the given gurl, already absolute and valid.
+ // Use only for resource fetches that lack a page context, or in places where
+ // permission checking has been done explicitly on the caller side (for
+ // example css_combine_filter, which constructs its own url_partnership).
+ Resource* CreateInputResourceUnchecked(const GURL& gurl,
+ const RewriteOptions* rewrite_options,
+ MessageHandler* handler);
+
+ // Set up a basic header for a given content_type.
+ // If content_type is null, the Content-Type is omitted.
+ // This method may only be called once on a header.
+ void SetDefaultHeaders(const ContentType* content_type,
+ MetaData* header) const;
+
+ // Changes the content type of a pre-initialized header.
+ void SetContentType(const ContentType* content_type, MetaData* header);
+
+ StringPiece filename_prefix() const { return file_prefix_; }
+ void set_filename_prefix(const StringPiece& file_prefix);
+ Statistics* statistics() const { return statistics_; }
+ void set_statistics(Statistics* s) {
+ statistics_ = s;
+ resource_url_domain_rejections_ = NULL; // Lazily initialized.
+ }
+ void set_relative_path(bool x) { relative_path_ = x; }
+ NamedLockManager* lock_manager() const { return lock_manager_; }
+
+ // Attempt to fetch extant version of an OutputResource. Returns false if the
+ // resource must be created by the caller. If true is returned, the resulting
+ // data could still be empty (eg because the resource is being rewritten in
+ // another thread, or rewriting results in errors), so
+ // output_resource->IsWritten() must be checked if this call succeeds. When
+ // blocking=kNeverBlock (the normal case for rewriting html), the call returns
+ // quickly if another thread is rewriting. When blocking=kMayBlock (the
+ // normal case for serving resources), the call blocks until the ongoing
+ // rewrite completes, or until the lock times out and can be seized by the
+ // serving thread.
+ bool FetchOutputResource(
+ OutputResource* output_resource,
+ Writer* writer, MetaData* response_headers,
+ MessageHandler* handler, BlockingBehavior blocking) const;
+
+ // Writes the specified contents into the output resource, retaining
+ // both a name->filename map and the filename->contents map.
+ //
+ // TODO(jmarantz): add last_modified arg.
+ bool Write(HttpStatus::Code status_code,
+ const StringPiece& contents, OutputResource* output,
+ int64 origin_expire_time_ms, MessageHandler* handler);
+
+ // Load the resource if it is cached (or if it can be fetched quickly).
+ // If not send off an asynchronous fetch and store the result in the cache.
+ //
+ // Returns true if the resource is loaded.
+ //
+ // The resource remains owned by the caller.
+ bool ReadIfCached(Resource* resource, MessageHandler* message_handler) const;
+
+ // Loads contents of resource asynchronously, calling callback when
+ // done. If the resource contents is cached, the callback will
+ // be called directly, rather than asynchronously. The resource
+ // will be passed to the callback, which will be responsible for
+ // ultimately freeing the resource. The resource will have its
+ // contents and headers filled in.
+ //
+ // The resource can be deleted only after the callback is called.
+ void ReadAsync(Resource* resource, Resource::AsyncCallback* callback,
+ MessageHandler* message_handler);
+
+ // TODO(jmarantz): check thread safety in Apache.
+ Hasher* hasher() const { return hasher_; }
+ // This setter should probably only be used in testing.
+ void set_hasher(Hasher* hasher) { hasher_ = hasher; }
+
+ FileSystem* file_system() { return file_system_; }
+ FilenameEncoder* filename_encoder() const { return filename_encoder_; }
+ UrlAsyncFetcher* url_async_fetcher() { return url_async_fetcher_; }
+ Timer* timer() { return http_cache_->timer(); }
+ HTTPCache* http_cache() { return http_cache_; }
+ UrlEscaper* url_escaper() { return url_escaper_.get(); }
+
+ // Whether or not resources should hit the filesystem.
+ bool store_outputs_in_file_system() { return store_outputs_in_file_system_; }
+ void set_store_outputs_in_file_system(bool store) {
+ store_outputs_in_file_system_ = store;
+ }
+
+ private:
+ inline void IncrementResourceUrlDomainRejections();
+
+ std::string file_prefix_;
+ int resource_id_; // Sequential ids for temporary Resource filenames.
+ FileSystem* file_system_;
+ FilenameEncoder* filename_encoder_;
+ UrlAsyncFetcher* url_async_fetcher_;
+ Hasher* hasher_;
+ Statistics* statistics_;
+ Variable* resource_url_domain_rejections_;
+ HTTPCache* http_cache_;
+ scoped_ptr<UrlEscaper> url_escaper_;
+ bool relative_path_;
+ bool store_outputs_in_file_system_;
+ NamedLockManager* lock_manager_;
+ std::string max_age_string_;
+
+ DISALLOW_COPY_AND_ASSIGN(ResourceManager);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_MANAGER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/resource_manager_test_base.h b/trunk/src/net/instaweb/rewriter/public/resource_manager_test_base.h
new file mode 100644
index 0000000..fab7006
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/resource_manager_test_base.h
@@ -0,0 +1,323 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: abliss@google.com (Adam Bliss)
+
+// Base class for tests which want a ResourceManager.
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_MANAGER_TEST_BASE_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_MANAGER_TEST_BASE_H_
+
+#include "net/instaweb/htmlparse/public/html_parse_test_base.h"
+#include "net/instaweb/rewriter/public/domain_lawyer.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/resource_namer.h"
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+#include "net/instaweb/util/public/fake_url_async_fetcher.h"
+#include "net/instaweb/util/public/filename_encoder.h"
+#include "net/instaweb/util/public/file_system_lock_manager.h"
+#include "net/instaweb/util/public/hasher.h"
+#include "net/instaweb/util/public/http_cache.h"
+#include "net/instaweb/util/public/lru_cache.h"
+#include "net/instaweb/util/public/mem_file_system.h"
+#include "net/instaweb/util/public/md5_hasher.h"
+#include "net/instaweb/util/public/mock_hasher.h"
+#include "net/instaweb/util/public/mock_timer.h"
+#include "net/instaweb/util/public/mock_url_fetcher.h"
+#include "net/instaweb/util/public/null_writer.h"
+#include "net/instaweb/util/public/simple_stats.h"
+#include "net/instaweb/util/public/stdio_file_system.h"
+#include <string>
+#include "net/instaweb/util/public/wait_url_async_fetcher.h"
+
+#define URL_PREFIX "http://www.example.com/"
+
+namespace net_instaweb {
+
+const int kCacheSize = 100 * 1000 * 1000;
+
+class ResourceManagerTestBase : public HtmlParseTestBaseNoAlloc {
+ protected:
+ ResourceManagerTestBase()
+ : mock_url_async_fetcher_(&mock_url_fetcher_),
+ file_prefix_(StrCat(GTestTempDir(), "/")),
+ url_prefix_(URL_PREFIX),
+
+ lru_cache_(new LRUCache(kCacheSize)),
+ http_cache_(lru_cache_, file_system_.timer()),
+ // TODO(jmaessen): Pull timer out of file_system_ and make it
+ // standalone.
+ lock_manager_(&file_system_, file_system_.timer(), &message_handler_),
+ // TODO(sligocki): Why can't I init it here ...
+ // resource_manager_(new ResourceManager(
+ // file_prefix_, &file_system_,
+ // &filename_encoder_, &mock_url_async_fetcher_, &mock_hasher_,
+ // &http_cache_)),
+ rewrite_driver_(&message_handler_, &file_system_,
+ &mock_url_async_fetcher_, options_),
+
+ other_lru_cache_(new LRUCache(kCacheSize)),
+ other_http_cache_(other_lru_cache_, other_file_system_.timer()),
+ other_lock_manager_(
+ &other_file_system_, other_file_system_.timer(), &message_handler_),
+ other_resource_manager_(
+ file_prefix_, &other_file_system_,
+ &filename_encoder_, &mock_url_async_fetcher_, &mock_hasher_,
+ &other_http_cache_, &other_lock_manager_),
+ other_rewrite_driver_(&message_handler_, &other_file_system_,
+ &mock_url_async_fetcher_, other_options_) {
+ // rewrite_driver_.SetResourceManager(resource_manager_);
+ other_rewrite_driver_.SetResourceManager(&other_resource_manager_);
+ }
+
+ virtual void SetUp() {
+ HtmlParseTestBaseNoAlloc::SetUp();
+ // TODO(sligocki): Init this in constructor.
+ resource_manager_ = new ResourceManager(
+ file_prefix_, &file_system_,
+ &filename_encoder_, &mock_url_async_fetcher_, &mock_hasher_,
+ &http_cache_, &lock_manager_);
+ rewrite_driver_.SetResourceManager(resource_manager_);
+ }
+
+ virtual void TearDown() {
+ delete resource_manager_;
+ HtmlParseTestBaseNoAlloc::TearDown();
+ }
+
+ // In this set of tests, we will provide explicit body tags, so
+ // the test harness should not add them in for our convenience.
+ // It can go ahead and add the <html> and </html>, however.
+ virtual bool AddBody() const {
+ return false;
+ }
+
+ // Add a single rewrite filter to rewrite_driver_.
+ void AddFilter(RewriteOptions::Filter filter) {
+ options_.EnableFilter(filter);
+ rewrite_driver_.AddFilters();
+ }
+
+ // Add a single rewrite filter to rewrite_driver_.
+ void AddOtherFilter(RewriteOptions::Filter filter) {
+ other_options_.EnableFilter(filter);
+ other_rewrite_driver_.AddFilters();
+ }
+
+ // The async fetchers in these tests are really fake async fetchers, and
+ // will call their callbacks directly. Hence we don't really need
+ // any functionality in the async callback.
+ class DummyCallback : public UrlAsyncFetcher::Callback {
+ public:
+ explicit DummyCallback(bool expect_success)
+ : done_(false),
+ expect_success_(expect_success) {}
+ virtual ~DummyCallback() {
+ EXPECT_TRUE(done_);
+ }
+ virtual void Done(bool success) {
+ EXPECT_FALSE(done_) << "Already Done; perhaps you reused without Reset()";
+ done_ = true;
+ EXPECT_EQ(expect_success_, success);
+ }
+ void Reset() {
+ done_ = false;
+ }
+
+ bool done_;
+ bool expect_success_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(DummyCallback);
+ };
+
+ MockTimer* mock_timer() { return file_system_.timer(); }
+
+ void DeleteFileIfExists(const std::string& filename) {
+ if (file_system_.Exists(filename.c_str(), &message_handler_).is_true()) {
+ ASSERT_TRUE(file_system_.RemoveFile(filename.c_str(), &message_handler_));
+ }
+ }
+
+ void AppendDefaultHeaders(const ContentType& content_type,
+ ResourceManager* resource_manager,
+ std::string* text) {
+ SimpleMetaData header;
+ int64 time = mock_timer()->NowUs();
+ // Reset mock timer so synthetic headers match original.
+ mock_timer()->set_time_us(0);
+ resource_manager->SetDefaultHeaders(&content_type, &header);
+ // Then set it back
+ mock_timer()->set_time_us(time);
+ StringWriter writer(text);
+ header.Write(&writer, &message_handler_);
+ }
+
+ void ServeResourceFromManyContexts(const std::string& resource_url,
+ RewriteOptions::Filter filter,
+ Hasher* hasher,
+ const StringPiece& expected_content) {
+ // TODO(sligocki): Serve the resource under several contexts. For example:
+ // 1) With output-resource cached,
+ // 2) With output-resource not cached, but in a file,
+ // 3) With output-resource unavailable, but input-resource cached,
+ // 4) With output-resource unavailable and input-resource not cached,
+ // but still fetchable,
+ ServeResourceFromNewContext(resource_url, filter, hasher, expected_content);
+ // 5) With nothing available (failure).
+ }
+
+ // Test that a resource can be served from an new server that has not already
+ // constructed it.
+ void ServeResourceFromNewContext(
+ const std::string& resource_url,
+ RewriteOptions::Filter filter,
+ Hasher* hasher,
+ const StringPiece& expected_content);
+
+ virtual HtmlParse* html_parse() { return rewrite_driver_.html_parse(); }
+
+ // Initializes a resource for mock fetching.
+ void InitMetaData(const StringPiece& resource_name,
+ const ContentType& content_type,
+ const StringPiece& content,
+ int64 ttl) {
+ std::string name = StrCat("http://test.com/", resource_name);
+ SimpleMetaData response_headers;
+ resource_manager_->SetDefaultHeaders(&content_type, &response_headers);
+ response_headers.RemoveAll(HttpAttributes::kCacheControl);
+ response_headers.Add(
+ HttpAttributes::kCacheControl,
+ StringPrintf("public, max-age=%ld", static_cast<long>(ttl)).c_str());
+ mock_url_fetcher_.SetResponse(name, response_headers, content);
+ }
+
+ // TODO(sligocki): Take a ttl and share code with InitMetaData.
+ void AddFileToMockFetcher(const StringPiece& url,
+ const std::string& filename,
+ const ContentType& content_type) {
+ // TODO(sligocki): There's probably a lot of wasteful copying here.
+
+ // We need to load a file from the testdata directory. Don't use this
+ // physical filesystem for anything else, use file_system_ which can be
+ // abstracted as a MemFileSystem instead.
+ std::string contents;
+ StdioFileSystem stdio_file_system;
+ ASSERT_TRUE(stdio_file_system.ReadFile(filename.c_str(), &contents,
+ &message_handler_));
+
+ // Put file into our fetcher.
+ SimpleMetaData default_header;
+ resource_manager_->SetDefaultHeaders(&content_type, &default_header);
+ mock_url_fetcher_.SetResponse(url, default_header, contents);
+ }
+
+ // Callback that can be used for testing resource fetches. As all the
+ // async fetchers in unit-tests call their callbacks immediately, it
+ // is safe to put this on the stack, rather than having it self-delete.
+ // TODO(sligocki): Do we need this and DummyCallback, could we give this
+ // a more descriptive name? MockCallback?
+ class FetchCallback : public UrlAsyncFetcher::Callback {
+ public:
+ FetchCallback() : success_(false), done_(false) {}
+ virtual void Done(bool success) {
+ success_ = success;
+ done_ = true;
+ }
+
+ bool success() const { return success_; }
+ bool done() const { return done_; }
+
+ private:
+ bool success_;
+ bool done_;
+ };
+
+ // Helper function to test resource fetching, returning true if the fetch
+ // succeeded, and modifying content. It is up to the caller to EXPECT_TRUE
+ // on the status and EXPECT_EQ on the content.
+ bool ServeResource(const StringPiece& path, const StringPiece& filter_id,
+ const StringPiece& name, const StringPiece& ext,
+ std::string* content) {
+ std::string url = Encode(path, filter_id, "0", name, ext);
+ return ServeResourceUrl(url, content);
+ }
+
+ bool ServeResourceUrl(const StringPiece& url, std::string* content) {
+ content->clear();
+ SimpleMetaData request_headers, response_headers;
+ StringWriter writer(content);
+ FetchCallback callback;
+ bool fetched = rewrite_driver_.FetchResource(
+ url, request_headers, &response_headers, &writer, &message_handler_,
+ &callback);
+ EXPECT_TRUE(callback.done());
+ return fetched && callback.success();
+ }
+
+ // Just check if we can fetch a resource successfully, ignore response.
+ bool TryFetchResource(const StringPiece& url) {
+ std::string contents;
+ return ServeResourceUrl(url, &contents);
+ }
+
+ // Helper function to encode a resource name from its pieces.
+ std::string Encode(const StringPiece& path,
+ const StringPiece& filter_id, const StringPiece& hash,
+ const StringPiece& name, const StringPiece& ext);
+
+ // Testdata directory.
+ static const char kTestData[];
+
+ MockUrlFetcher mock_url_fetcher_;
+ FakeUrlAsyncFetcher mock_url_async_fetcher_;
+ FilenameEncoder filename_encoder_;
+
+ MockHasher mock_hasher_;
+ MD5Hasher md5_hasher_;
+
+ std::string file_prefix_;
+ std::string url_prefix_;
+
+ // We have two independent RewriteDrivers representing two completely
+ // separate servers for the same domain (say behind a load-balancer).
+ //
+ // Server A runs rewrite_driver_ and will be used to rewrite pages and
+ // served the rewritten resources.
+ MemFileSystem file_system_;
+ LRUCache* lru_cache_; // Owned by http_cache_
+ HTTPCache http_cache_;
+ FileSystemLockManager lock_manager_;
+ ResourceManager* resource_manager_; // TODO(sligocki): Make not a pointer.
+ RewriteOptions options_;
+ RewriteDriver rewrite_driver_;
+
+ // Server B runs other_rewrite_driver_ and will get a request for
+ // resources that server A has rewritten, but server B has not heard
+ // of yet. Thus, server B will have to decode the instructions on how
+ // to rewrite the resource just from the request.
+ MemFileSystem other_file_system_;
+ LRUCache* other_lru_cache_; // Owned by other_http_cache_
+ HTTPCache other_http_cache_;
+ FileSystemLockManager other_lock_manager_;
+ ResourceManager other_resource_manager_;
+ RewriteOptions other_options_;
+ RewriteDriver other_rewrite_driver_;
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_MANAGER_TEST_BASE_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/resource_namer.h b/trunk/src/net/instaweb/rewriter/public/resource_namer.h
new file mode 100644
index 0000000..1c3d2b7
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/resource_namer.h
@@ -0,0 +1,111 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_NAMER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_NAMER_H_
+
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class ContentType;
+class ResourceManager;
+
+// Encapsulates the naming of resource URL leafs. The class holds the context
+// of a single resource, and is not intended for re-use. We could, of course,
+// add a Clear(), but it is a stateful class.
+class ResourceNamer {
+ public:
+ // This determines the overhead imposed on each URL by the ResourceNamer
+ // syntax, such as separators.
+ static const int kOverhead;
+
+ ResourceNamer() {}
+ ~ResourceNamer() {}
+
+ // Encoding and decoding in various formats.
+
+ // Decodes an entire resource name (ID.HASH.NAME.EXT), placing
+ // the result in the fields in this encoder.
+ bool Decode(const StringPiece& encoded_string);
+
+ // Encodes the fields in this encoder into an absolute url, with the
+ // trailing portion "ID.HASH.NAME.EXT".
+ std::string Encode() const;
+
+ // Encode a key that can used to do a lookup based on an id
+ // and the name. This key can be used to find the hash-code for a
+ // resource within the origin TTL.
+ //
+ // The 'id' is a short code indicating which Instaweb rewriter was
+ // used to generate the resource.
+ std::string EncodeIdName() const;
+
+ // Note: there is no need at this time to decode the name key.
+
+ // Encode/decode the hash and extension, which is used as the value
+ // in the origin-TTL-bounded cache.
+ std::string EncodeHashExt() const;
+ bool DecodeHashExt(const StringPiece& encoded_hash_ext);
+
+ // Simple getters
+ StringPiece id() const { return id_; }
+ StringPiece name() const { return name_; }
+ StringPiece hash() const { return hash_; }
+ StringPiece ext() const { return ext_; }
+
+ // Simple setters
+ void set_id(const StringPiece& p) { p.CopyToString(&id_); }
+ void set_name(const StringPiece& n) { n.CopyToString(&name_); }
+ void set_hash(const StringPiece& h) { h.CopyToString(&hash_); }
+ void set_ext(const StringPiece& e) {
+ // TODO(jmaessen): Remove check after transitioning to undotted extensions
+ // everywhere.
+ CHECK(e.empty() || e[0] != '.');
+ e.CopyToString(&ext_);
+ }
+
+ // Other setter-like operations
+ void ClearHash() { hash_.clear(); }
+ void CopyFrom(const ResourceNamer& other);
+
+ // Utility functions
+
+ // Name suitable for debugging and logging
+ std::string PrettyName() const {return InternalEncode(); }
+
+ // Compute a hash of non-path portions of the represented url.
+ size_t Hash() const;
+
+ // Compute a content-type based on ext(). NULL if unrecognized.
+ const ContentType* ContentTypeFromExt() const;
+
+ private:
+ std::string InternalEncode() const;
+ bool LegacyDecode(const StringPiece& encoded_string);
+
+ std::string id_;
+ std::string name_;
+ std::string hash_;
+ std::string ext_;
+
+ DISALLOW_COPY_AND_ASSIGN(ResourceNamer);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_NAMER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/resource_tag_scanner.h b/trunk/src/net/instaweb/rewriter/public/resource_tag_scanner.h
new file mode 100644
index 0000000..3355880
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/resource_tag_scanner.h
@@ -0,0 +1,47 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_TAG_SCANNER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_TAG_SCANNER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/css_tag_scanner.h"
+#include "net/instaweb/rewriter/public/img_tag_scanner.h"
+#include "net/instaweb/rewriter/public/script_tag_scanner.h"
+
+namespace net_instaweb {
+
+class ResourceTagScanner {
+ public:
+ explicit ResourceTagScanner(HtmlParse* html_parse);
+
+ // Examines an HTML element to determine if it's a link to any sort
+ // of resource, extracting out the HREF or SRC.
+ HtmlElement::Attribute* ScanElement(HtmlElement* element);
+
+ private:
+ CssTagScanner css_tag_scanner_;
+ ImgTagScanner img_tag_scanner_;
+ ScriptTagScanner script_tag_scanner_;
+
+ DISALLOW_COPY_AND_ASSIGN(ResourceTagScanner);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_RESOURCE_TAG_SCANNER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/rewrite_driver.h b/trunk/src/net/instaweb/rewriter/public/rewrite_driver.h
new file mode 100644
index 0000000..4b6f89e
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/rewrite_driver.h
@@ -0,0 +1,217 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_DRIVER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_DRIVER_H_
+
+#include <map>
+#include <vector>
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/rewrite_options.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+#include "net/instaweb/util/public/user_agent.h"
+
+namespace net_instaweb {
+
+class AddInstrumentationFilter;
+class BaseTagFilter;
+class FileSystem;
+class Hasher;
+class HtmlFilter;
+class HtmlParse;
+class HtmlWriterFilter;
+class ResourceNamer;
+class RewriteFilter;
+class Statistics;
+class Timer;
+class UrlAsyncFetcher;
+class UrlFetcher;
+class UrlLeftTrimFilter;
+class Variable;
+class Writer;
+
+class RewriteDriver {
+ public:
+ static const char kCssCombinerId[];
+ static const char kCssFilterId[];
+ static const char kCacheExtenderId[];
+ static const char kImageCompressionId[];
+ static const char kJavascriptMinId[];
+
+ // A list of HTTP request headers. These are the headers which should be
+ // passed through from the client request into the MetaData request_headers
+ // sent to the rewrite driver. Headers not in this list will be ignored so
+ // there is no need to copy them over.
+ static const char* kPassThroughRequestAttributes[3];
+
+ RewriteDriver(MessageHandler* message_handler,
+ FileSystem* file_system,
+ UrlAsyncFetcher* url_async_fetcher,
+ const RewriteOptions& options);
+
+ // Need explicit destructors to allow destruction of scoped_ptr-controlled
+ // instances without propagating the include files.
+ ~RewriteDriver();
+
+ // Calls Initialize on all known rewrite_drivers.
+ static void Initialize(Statistics* statistics);
+
+ // Adds a resource manager and/or resource_server, enabling the rewriting of
+ // resources. This will replace any previous resource managers.
+ void SetResourceManager(ResourceManager* resource_manager);
+
+
+ void SetUserAgent(const char* user_agent_string) {
+ user_agent_.set_user_agent(user_agent_string);
+ }
+
+ const UserAgent& user_agent() const {
+ return user_agent_;
+ }
+
+ // Adds the filters from the options, specified by name in enabled_filters.
+ // This must be called explicitly after object construction to provide an
+ // opportunity to programatically add custom filters beyond those defined
+ // in RewriteOptions, via AddFilter(HtmlFilter* filter) (below).
+ void AddFilters();
+
+ // Add any HtmlFilter to the HtmlParse chain and take ownership of the filter.
+ void AddFilter(HtmlFilter* filter);
+
+ // Controls how HTML output is written. Be sure to call this last, after
+ // all other filters have been established.
+ //
+ // TODO(jmarantz): fix this in the implementation so that the caller can
+ // install filters in any order and the writer will always be last.
+ void SetWriter(Writer* writer);
+
+ // Sets the url that BaseTagFilter will set as the base for the document.
+ // This is an no-op if you haven't added BaseTagFilter.
+ // Call this for each new document you are processing or the old values will
+ // leak through.
+ // Note: Use this only to *change* the base URL, not to note the original
+ // base URL.
+ // TODO(sligocki): Do we need this? We should have some way of noting the
+ // base URL of a site if it is explicitly set.
+ void SetBaseUrl(const StringPiece& base);
+
+ // Initiates an async fetch for a rewritten resource with the specified name.
+ // If resource matches the pattern of what the driver is authorized to serve,
+ // then true is returned and the caller must listen on the callback for the
+ // completion of the request.
+ //
+ // If the pattern does not match, then false is returned, and the request
+ // should be passed to another handler, and the callback will *not* be
+ // called. In other words there are four outcomes for this routine:
+ //
+ // 1. the request was handled immediately and the callback called
+ // before the method returns. true is returned.
+ // 2. the request looks good but was queued because some other resource
+ // fetch is needed to satisfy it. true is returned.
+ // 3. the request looks like one it belongs to Instaweb, but the resource
+ // could not be decoded. The callback is called immediately with
+ // 'false', but true is returned.
+ // 4. the request does not look like it belongs to Instaweb. The callback
+ // will not be called, and false will be returned.
+ //
+ // In other words, if this routine returns 'false' then the callback
+ // will not be called. If the callback is called, then this should be the
+ // 'final word' on this request, whether it was called with success=true or
+ // success=false.
+ bool FetchResource(const StringPiece& resource,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* writer,
+ MessageHandler* message_handler,
+ UrlAsyncFetcher::Callback* callback);
+
+ // Attempts to decodes an output resource based on the URL pattern
+ // without actually rewriting it.
+ OutputResource* DecodeOutputResource(const StringPiece& url,
+ RewriteFilter** filter);
+
+ HtmlParse* html_parse() { return &html_parse_; }
+ FileSystem* file_system() { return file_system_; }
+ void set_async_fetcher(UrlAsyncFetcher* f) { url_async_fetcher_ = f; }
+
+ ResourceManager* resource_manager() const { return resource_manager_; }
+ Statistics* statistics() const;
+
+ AddInstrumentationFilter* add_instrumentation_filter() {
+ return add_instrumentation_filter_;
+ }
+
+ const RewriteOptions* options() { return &options_; }
+
+ private:
+ friend class ResourceManagerTestBase;
+ typedef std::map<std::string, RewriteFilter*> StringFilterMap;
+ typedef void (RewriteDriver::*SetStringMethod)(const StringPiece& value);
+ typedef void (RewriteDriver::*SetInt64Method)(int64 value);
+
+ bool ParseKeyString(const StringPiece& key, SetStringMethod m,
+ const std::string& flag);
+ bool ParseKeyInt64(const StringPiece& key, SetInt64Method m,
+ const std::string& flag);
+
+ // Registers RewriteFilter in the map, but does not put it in the
+ // html parse filter filter chain. This allows it to serve resource
+ // requests.
+ void RegisterRewriteFilter(RewriteFilter* filter);
+
+ // Adds a pre-added rewrite filter to the html parse chain.
+ void EnableRewriteFilter(const char* id);
+
+ StringFilterMap resource_filter_map_;
+
+ // These objects are provided on construction or later, and are
+ // owned by the caller.
+ HtmlParse html_parse_;
+ FileSystem* file_system_;
+ UrlAsyncFetcher* url_async_fetcher_;
+ ResourceManager* resource_manager_;
+
+ AddInstrumentationFilter* add_instrumentation_filter_;
+ scoped_ptr<HtmlWriterFilter> html_writer_filter_;
+ scoped_ptr<BaseTagFilter> base_tag_filter_;
+ scoped_ptr<UrlLeftTrimFilter> left_trim_filter_;
+ UserAgent user_agent_;
+ std::vector<HtmlFilter*> filters_;
+
+ // Statistics
+ static const char kResourceFetchesCached[];
+ static const char kResourceFetchConstructSuccesses[];
+ static const char kResourceFetchConstructFailures[];
+
+ Variable* cached_resource_fetches_;
+ Variable* succeeded_filter_resource_fetches_;
+ Variable* failed_filter_resource_fetches_;
+
+ const RewriteOptions& options_;
+
+ DISALLOW_COPY_AND_ASSIGN(RewriteDriver);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_DRIVER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/rewrite_driver_factory.h b/trunk/src/net/instaweb/rewriter/public/rewrite_driver_factory.h
new file mode 100644
index 0000000..c1f1a38
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/rewrite_driver_factory.h
@@ -0,0 +1,244 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_DRIVER_FACTORY_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_DRIVER_FACTORY_H_
+
+#include <set>
+#include <vector>
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/domain_lawyer.h"
+#include "net/instaweb/rewriter/public/rewrite_options.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class AbstractMutex;
+class CacheInterface;
+class CacheUrlAsyncFetcher;
+class CacheUrlFetcher;
+class DelayController;
+class FileDriver;
+class FileSystem;
+class FilenameEncoder;
+class Hasher;
+class HtmlParse;
+class HTTPCache;
+class LRUCache;
+class MessageHandler;
+class NamedLockManager;
+class ResourceManager;
+class RewriteDriver;
+class Statistics;
+class Timer;
+class UrlAsyncFetcher;
+class UrlFetcher;
+class Variable;
+
+// A base RewriteDriverFactory.
+class RewriteDriverFactory {
+ public:
+ RewriteDriverFactory();
+ virtual ~RewriteDriverFactory();
+
+ // The RewriteDriveFactory will create objects of default type through the
+ // New* method from drived classs. Here are the objects that can be
+ // replaced before creating the RewriteDriver.
+ // Note: RewriteDriver takes ownership of these.
+ void set_html_parse_message_handler(MessageHandler* message_handler);
+ void set_message_handler(MessageHandler* message_handler);
+ void set_file_system(FileSystem* file_system);
+ void set_hasher(Hasher* hasher);
+ void set_filename_encoder(FilenameEncoder* filename_encoder);
+ void set_timer(Timer* timer);
+
+ // Set up a directory for slurped files for HTML and resources. If
+ // read_only is true, then it will only read from these files, and
+ // this will eliminate the usage of any other url_fetcher. If
+ // read_only is false, then the existing url fetcher will be used as
+ // a fallback if the slurped file is not found, and slurped files will
+ // be subsequently written so they don't have to be fetched from
+ // the Internet again.
+ //
+ // You must set the slurp directory prior to calling ComputeUrlFetcher
+ // or ComputeUrlAsyncFetcher.
+ void set_slurp_directory(const StringPiece& directory);
+ void set_slurp_read_only(bool read_only);
+ void set_slurp_print_urls(bool read_only);
+
+ // Determines whether Slurping is enabled.
+ bool slurping_enabled() const { return !slurp_directory_.empty(); }
+
+ // Setting HTTP caching on causes both the fetcher and the async
+ // fecher to return cached versions.
+ void set_force_caching(bool u) { force_caching_ = u; }
+
+ // You should either call set_base_url_fetcher,
+ // set_base_url_async_fetcher, or neither. Do not set both. If you
+ // want to enable real async fetching, because you are serving or
+ // want to model live traffic, then call set_base_url_async_fetcher
+ // before calling url_fetcher.
+ //
+ // These fetchers may be used directly when serving traffic, or they
+ // may be aggregated with other fetchers (e.g. for slurping).
+ //
+ // You cannot set either base URL fetcher once ComputeUrlFetcher has
+ // been called.
+ void set_base_url_fetcher(UrlFetcher* url_fetcher);
+ void set_base_url_async_fetcher(UrlAsyncFetcher* url_fetcher);
+
+ bool set_filename_prefix(StringPiece p);
+
+ RewriteOptions* options() { return &options_; }
+ MessageHandler* html_parse_message_handler();
+ MessageHandler* message_handler();
+ FileSystem* file_system();
+ // TODO(sligocki): Remove hasher() and force people to make a NewHasher when
+ // they need one.
+ Hasher* hasher();
+ FilenameEncoder* filename_encoder();
+ Timer* timer();
+ HTTPCache* http_cache();
+ NamedLockManager* lock_manager();
+
+ StringPiece filename_prefix();
+
+ // Computes URL fetchers using the based fetcher, and optionally,
+ // slurp_directory and slurp_read_only.
+ virtual UrlFetcher* ComputeUrlFetcher();
+ virtual UrlAsyncFetcher* ComputeUrlAsyncFetcher();
+ virtual ResourceManager* ComputeResourceManager();
+
+ // Generates a new mutex, hasher.
+ virtual AbstractMutex* NewMutex() = 0;
+ virtual Hasher* NewHasher() = 0;
+
+ // Generates a new managed RewriteDriver using the RewriteOptions
+ // managed by this class. Each RewriteDriver is not thread-safe,
+ // but you can generate a RewriteDriver* for each thread. The
+ // returned drivers are deleted by the factory; they do not need to
+ // be deleted by the allocator.
+ RewriteDriver* NewRewriteDriver();
+
+ // Releases a rewrite driver back into the pool. These are free-listed
+ // because they are not cheap to construct.
+ void ReleaseRewriteDriver(RewriteDriver* rewrite_driver);
+
+ // Generates a custom RewriteDriver using the passed-in options. This
+ // driver is *not* managed by the factory: you must delete it after
+ // you are done with it.
+ RewriteDriver* NewCustomRewriteDriver(const RewriteOptions& options);
+
+ // Initialize statistics variables for 404 responses.
+ static void Initialize(Statistics* statistics);
+ // Increment the count of resource returning 404.
+ void Increment404Count();
+ // Increment the cournt of slurp returning 404.
+ void IncrementSlurpCount();
+
+ protected:
+ virtual void AddPlatformSpecificRewritePasses(RewriteDriver* driver);
+ bool FetchersComputed() const;
+
+ // Implementors of RewriteDriverFactory must supply default definitions
+ // for each of these methods, although they may be overridden via set_
+ // methods above
+ virtual UrlFetcher* DefaultUrlFetcher() = 0;
+ virtual UrlAsyncFetcher* DefaultAsyncUrlFetcher() = 0;
+ virtual MessageHandler* DefaultHtmlParseMessageHandler() = 0;
+ virtual MessageHandler* DefaultMessageHandler() = 0;
+ virtual FileSystem* DefaultFileSystem() = 0;
+ virtual Timer* DefaultTimer() = 0;
+ virtual CacheInterface* DefaultCacheInterface() = 0;
+
+ // Implementors of RewriteDriverFactory must supply two mutexes.
+ virtual AbstractMutex* cache_mutex() = 0;
+ virtual AbstractMutex* rewrite_drivers_mutex() = 0;
+
+ // Clean up all the resources. When shutdown Apache, and destroy the process
+ // sub-pool. The RewriteDriverFactory owns some elements that were created
+ // from that sub-pool. The sub-pool is destroyed in ApacheRewriteFactory,
+ // which happens before the destruction of the base class. When the base class
+ // destroys, the sub-pool has been destroyed, but the elements in base class
+ // are still trying to destroy the sub-pool of the sub-pool. Call this
+ // function before destroying the process sub-pool.
+ void ShutDown();
+
+ // Called before creating the url fetchers.
+ virtual void FetcherSetupHooks();
+
+ // Override this to return false if you don't want the resource
+ // manager to write resources to the filesystem.
+ virtual bool ShouldWriteResourcesToFileSystem() { return true; }
+
+ private:
+ void SetupSlurpDirectories();
+
+ scoped_ptr<MessageHandler> html_parse_message_handler_;
+ scoped_ptr<MessageHandler> message_handler_;
+ scoped_ptr<FileSystem> file_system_;
+ UrlFetcher* url_fetcher_;
+ UrlAsyncFetcher* url_async_fetcher_;
+ scoped_ptr<UrlFetcher> base_url_fetcher_;
+ scoped_ptr<UrlAsyncFetcher> base_url_async_fetcher_;
+ scoped_ptr<Hasher> hasher_;
+ scoped_ptr<FilenameEncoder> filename_encoder_;
+ scoped_ptr<Timer> timer_;
+ HtmlParse* html_parse_;
+
+ std::string filename_prefix_;
+ std::string slurp_directory_;
+ RewriteOptions options_;
+ bool force_caching_;
+ bool slurp_read_only_;
+ bool slurp_print_urls_;
+
+ scoped_ptr<ResourceManager> resource_manager_;
+
+ // RewriteDrivers that were previously allocated, but have
+ // been released with ReleaseRewriteDriver, and are ready
+ // for re-use with NewRewriteDriver.
+ std::vector<RewriteDriver*> available_rewrite_drivers_;
+
+ // RewriteDrivers that are currently in use. This is retained
+ // as a sanity check to make sure our system is coherent,
+ // and to facilitate complete cleanup if a Shutdown occurs
+ // while a request is in flight.
+ std::set<RewriteDriver*> active_rewrite_drivers_;
+
+ // Caching support
+ scoped_ptr<HTTPCache> http_cache_;
+ scoped_ptr<CacheUrlFetcher> cache_fetcher_;
+ scoped_ptr<CacheUrlAsyncFetcher> cache_async_fetcher_;
+ Variable* resource_404_count_;
+ Variable* slurp_404_count_;
+
+ // Keep track of authorized domains, sharding, and mappings.
+ DomainLawyer domain_lawyer_;
+
+ // Manage locks for output resources.
+ scoped_ptr<NamedLockManager> lock_manager_;
+
+ DISALLOW_COPY_AND_ASSIGN(RewriteDriverFactory);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_DRIVER_FACTORY_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/rewrite_filter.h b/trunk/src/net/instaweb/rewriter/public/rewrite_filter.h
new file mode 100644
index 0000000..9a52f30
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/rewrite_filter.h
@@ -0,0 +1,83 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/common_filter.h"
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+
+namespace net_instaweb {
+
+class HtmlParse;
+class OutputResource;
+class ResourceManager;
+class RewriteDriver;
+class UrlAsyncFetcher;
+class Writer;
+
+class RewriteFilter : public CommonFilter {
+ public:
+ explicit RewriteFilter(RewriteDriver* driver, StringPiece filter_prefix)
+ : CommonFilter(driver),
+ filter_prefix_(filter_prefix.data(), filter_prefix.size()),
+ driver_(driver) {
+ }
+ virtual ~RewriteFilter();
+
+ // Fetches a resource written using the filter. Filters that
+ // encode all the data (URLs, meta-data) needed to reconstruct
+ // a rewritten resource in a URL component, this method is the
+ // mechanism for the filter to serve the rewritten resource.
+ //
+ // The flow is that a RewriteFilter is instantiated with
+ // a path prefix, e.g. a two letter abbreviation of the
+ // filter, like "ce" for CacheExtender. When it rewrites a
+ // resource, it replaces the href with a url constructed as
+ // HOST://PATH/ENCODED_NAME.pagespeed.FILTER_ID.HASH.EXT
+ // Most ENCODED_NAMEs are just the original URL with a few
+ // characters, notably '?' and '&' esacped. For "ic" (ImgRewriterFilter)
+ // the encoding includes the original image URL, plus the pixel-dimensions
+ // to which the image was resized. For combine_css it includes
+ // all the original URLs separated by '+'.
+ virtual bool Fetch(OutputResource* output_resource,
+ Writer* response_writer,
+ const MetaData& request_header,
+ MetaData* response_headers,
+ MessageHandler* message_handler,
+ UrlAsyncFetcher::Callback* callback) = 0;
+
+ const std::string& id() const { return filter_prefix_; }
+ HtmlParse* html_parse() { return driver_->html_parse(); }
+ ResourceManager* resource_manager() { return driver_->resource_manager(); }
+
+ protected:
+ std::string filter_prefix_; // Prefix that should be used in front of all
+ // rewritten URLs
+ RewriteDriver* driver_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(RewriteFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/rewrite_options.h b/trunk/src/net/instaweb/rewriter/public/rewrite_options.h
new file mode 100644
index 0000000..4db0b72
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/rewrite_options.h
@@ -0,0 +1,337 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_OPTIONS_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_OPTIONS_H_
+
+#include <map>
+#include <set>
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/domain_lawyer.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/wildcard_group.h"
+
+namespace net_instaweb {
+
+class MessageHandler;
+
+class RewriteOptions {
+ public:
+ enum Filter {
+ kAddBaseTag, // Update kFirstEnumFilter if you add something before this.
+ kAddHead,
+ kAddInstrumentation,
+ kCollapseWhitespace,
+ kCombineCss,
+ kCombineHeads,
+ kDebugLogImgTags,
+ kElideAttributes,
+ kExtendCache,
+ kInlineCss,
+ kInlineJavascript,
+ kInsertImgDimensions,
+ kLeftTrimUrls,
+ kMoveCssToHead,
+ kOutlineCss,
+ kOutlineJavascript,
+ kRemoveComments,
+ kRemoveQuotes,
+ kRewriteCss,
+ kRewriteImages,
+ kRewriteJavascript,
+ kStripScripts, // Update kLastEnumFilter if you add something after this.
+ };
+
+ private:
+ // Needed by kAllFilters.
+ static const Filter kFirstEnumFilter = kAddBaseTag;
+ static const Filter kLastEnumFilter = kStripScripts;
+
+ public:
+ enum RewriteLevel {
+ // Enable no filters. Parse HTML but do not perform any
+ // transformations. This is the default value. Most users should
+ // explcitly enable the kCoreFilters level by calling
+ // SetRewriteLevel(kCoreFilters).
+ kPassThrough,
+
+ // Enable the core set of filters. These filters are considered
+ // generally safe for most sites, though even safe filters can
+ // break some sites. Most users should specify this option, and
+ // then optionally add or remove specific filters based on
+ // specific needs.
+ kCoreFilters,
+
+ // Enable all filters intended for core, but some of which might
+ // need more testing. Good for if users are willing to test out
+ // the results of the rewrite more closely.
+ kTestingCoreFilters,
+
+ // Enable all filters.
+ kAllFilters,
+ };
+
+ // Used for enumerating over all entries in the Filter enum.
+ static const Filter kFirstFilter = kAddBaseTag;
+ static const Filter kLastFilter = kStripScripts;
+
+ static const int64 kDefaultCssInlineMaxBytes;
+ static const int64 kDefaultImgInlineMaxBytes;
+ static const int64 kDefaultJsInlineMaxBytes;
+ static const int64 kDefaultCssOutlineMinBytes;
+ static const int64 kDefaultJsOutlineMinBytes;
+ static const std::string kDefaultBeaconUrl;
+
+ // IE limits URL size overall to about 2k characters. See
+ // http://support.microsoft.com/kb/208427/EN-US
+ static const int kMaxUrlSize;
+
+ static const int kDefaultImgMaxRewritesAtOnce;
+
+ // See http://code.google.com/p/modpagespeed/issues/detail?id=9
+ // Apache evidently limits each URL path segment (between /) to
+ // about 256 characters. This is not fundamental URL limitation
+ // but is Apache specific.
+ static const int64 kMaxUrlSegmentSize;
+
+ static bool ParseRewriteLevel(const StringPiece& in, RewriteLevel* out);
+
+ RewriteOptions();
+ ~RewriteOptions();
+
+ bool modified() const { return modified_; }
+
+ void SetDefaultRewriteLevel(RewriteLevel level) {
+ // Do not set the modified bit -- we are only changing the default.
+ level_.set_default(level);
+ }
+ void SetRewriteLevel(RewriteLevel level) {
+ modified_ = true;
+ level_.set(level);
+ }
+ RewriteLevel level() const { return level_.value();}
+
+ // Adds a set of filters to the enabled set. Returns false if any
+ // of the filter names are invalid, but all the valid ones will be
+ // added anyway.
+ bool EnableFiltersByCommaSeparatedList(const StringPiece& filters,
+ MessageHandler* handler);
+
+ // Adds a set of filters to the disabled set. Returns false if any
+ // of the filter names are invalid, but all the valid ones will be
+ // added anyway.
+ bool DisableFiltersByCommaSeparatedList(const StringPiece& filters,
+ MessageHandler* handler);
+ void EnableFilter(Filter filter);
+ void DisableFilter(Filter filter);
+
+ bool Enabled(Filter filter) const;
+
+ // TODO(jmarantz): consider setting flags in the set_ methods so that
+ // first's explicit settings can override default values from second.
+
+ int64 css_outline_min_bytes() const { return css_outline_min_bytes_.value(); }
+ void set_css_outline_min_bytes(int64 x) {
+ modified_ = true;
+ css_outline_min_bytes_.set(x);
+ }
+ int64 js_outline_min_bytes() const { return js_outline_min_bytes_.value(); }
+ void set_js_outline_min_bytes(int64 x) {
+ modified_ = true;
+ js_outline_min_bytes_.set(x);
+ }
+ int64 img_inline_max_bytes() const { return img_inline_max_bytes_.value(); }
+ void set_img_inline_max_bytes(int64 x) {
+ modified_ = true;
+ img_inline_max_bytes_.set(x);
+ }
+ int64 css_inline_max_bytes() const { return css_inline_max_bytes_.value(); }
+ void set_css_inline_max_bytes(int64 x) {
+ modified_ = true;
+ css_inline_max_bytes_.set(x);
+ }
+ int64 js_inline_max_bytes() const { return js_inline_max_bytes_.value(); }
+ void set_js_inline_max_bytes(int64 x) {
+ modified_ = true;
+ js_inline_max_bytes_.set(x);
+ }
+ int num_shards() const { return num_shards_.value(); }
+ void set_num_shards(int x) {
+ modified_ = true;
+ num_shards_.set(x);
+ }
+ const std::string& beacon_url() const { return beacon_url_.value(); }
+ void set_beacon_url(const StringPiece& p) {
+ modified_ = true;
+ beacon_url_.set(std::string(p.data(), p.size()));
+
+ }
+ // The maximum length of a URL segment.
+ // for http://a/b/c.d, this is == strlen("c.d")
+ int max_url_segment_size() const { return max_url_segment_size_.value(); }
+ void set_max_url_segment_size(int x) {
+ modified_ = true;
+ max_url_segment_size_.set(x);
+ }
+
+ int img_max_rewrites_at_once() const {
+ return img_max_rewrites_at_once_.value();
+ }
+ void set_img_max_rewrites_at_once(int x) {
+ modified_ = true;
+ img_max_rewrites_at_once_.set(x);
+ }
+
+ // The maximum size of the entire URL. If '0', this is left unlimited.
+ int max_url_size() const { return max_url_size_.value(); }
+ void set_max_url_size(int x) {
+ modified_ = true;
+ max_url_size_.set(x);
+ }
+
+ void set_enabled(bool x) {
+ modified_ = true;
+ enabled_.set(x);
+ }
+ bool enabled() const { return enabled_.value(); }
+
+ // Merge together two source RewriteOptions to populate this. The order
+ // is significant: the second will override the first. One semantic
+ // subject to interpretation is when a core-filter is disabled in the
+ // first set and not in the second. In this case, my judgement is that
+ // the 'disable' from the first should override the core-set membership
+ // in the second, but not an 'enable' in the second.
+ void Merge(const RewriteOptions& first, const RewriteOptions& second);
+
+ // Registers a wildcard pattern for to be allowed, potentially overriding
+ // previous Disallow wildcards.
+ void Allow(const StringPiece& wildcard_pattern) {
+ modified_ = true;
+ allow_resources_.Allow(wildcard_pattern);
+ }
+
+ // Registers a wildcard pattern for to be disallowed, potentially overriding
+ // previous Allow wildcards.
+ void Disallow(const StringPiece& wildcard_pattern) {
+ modified_ = true;
+ allow_resources_.Disallow(wildcard_pattern);
+ }
+
+ DomainLawyer* domain_lawyer() { return &domain_lawyer_; }
+ const DomainLawyer* domain_lawyer() const { return &domain_lawyer_; }
+
+ // Determines, based on the sequence of Allow/Disallow calls above, whether
+ // a url is allowed.
+ bool IsAllowed(const StringPiece& url) const {
+ return allow_resources_.Match(url);
+ }
+
+ void CopyFrom(const RewriteOptions& src) {
+ Merge(src, src); // We lack a better implementation of Copy.
+ }
+
+ private:
+ // Helper class to represent an Option, whose value is held in some class T.
+ // An option is explicitly initialized with its default value, although the
+ // default value can be altered later. It keeps track of whether a
+ // value has been explicitly set (independent of whether that happens to
+ // coincide with the default value).
+ //
+ // It can use this knowledge to intelligently merge a 'base' option value
+ // into a 'new' option value, allowing explicitly set values from 'base'
+ // to override default values from 'new'.
+ template<class T> class Option {
+ public:
+ explicit Option(const T& default_value)
+ : value_(default_value),
+ was_set_(false) {
+ }
+
+ void set(const T& val) {
+ was_set_ = true;
+ value_ = val;
+ }
+
+ void set_default(const T& val) {
+ if (!was_set_) {
+ value_ = val;
+ }
+ }
+
+ const T& value() const { return value_; }
+
+ void Merge(const Option& one, const Option& two) {
+ if (two.was_set_ || !one.was_set_) {
+ value_ = two.value_;
+ was_set_ = two.was_set_;
+ } else {
+ value_ = one.value_;
+ was_set_ = true; // this stmt is reached only if one.was_set_==true
+ }
+ }
+
+ private:
+ T value_;
+ bool was_set_;
+
+ DISALLOW_COPY_AND_ASSIGN(Option);
+ };
+
+ typedef std::set<Filter> FilterSet;
+ typedef std::map<std::string, Filter> NameToFilterMap;
+ typedef std::map<RewriteLevel, FilterSet> RewriteLevelToFilterSetMap;
+
+ void SetUp();
+ bool AddCommaSeparatedListToFilterSet(
+ const StringPiece& filters, MessageHandler* handler, FilterSet* set);
+
+ bool modified_;
+ NameToFilterMap name_filter_map_;
+ RewriteLevelToFilterSetMap level_filter_set_map_;
+ FilterSet enabled_filters_;
+ FilterSet disabled_filters_;
+
+ // Note: using the template class Option here saves a lot of repeated
+ // and error-prone merging code. However, it is not space efficient as
+ // we are alternating int64s and bools in the structure. If we cared
+ // about that, then we would keep the bools in a bitmask. But since
+ // we don't really care we'll try to keep the code structured better.
+ Option<RewriteLevel> level_;
+ Option<int64> css_inline_max_bytes_;
+ Option<int64> img_inline_max_bytes_;
+ Option<int64> img_max_rewrites_at_once_;
+ Option<int64> js_inline_max_bytes_;
+ Option<int64> css_outline_min_bytes_;
+ Option<int64> js_outline_min_bytes_;
+ Option<int> num_shards_;
+ Option<std::string> beacon_url_;
+ Option<int> max_url_segment_size_; // for http://a/b/c.d, use strlen("c.d")
+ Option<int> max_url_size_; // but this is strlen("http://a/b/c.d")
+ Option<bool> enabled_;
+ DomainLawyer domain_lawyer_;
+ // Be sure to update Merge() if a new field is added.
+
+ WildcardGroup allow_resources_;
+
+ DISALLOW_COPY_AND_ASSIGN(RewriteOptions);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_OPTIONS_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/rewrite_single_resource_filter.h b/trunk/src/net/instaweb/rewriter/public/rewrite_single_resource_filter.h
new file mode 100644
index 0000000..5baad2b
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/rewrite_single_resource_filter.h
@@ -0,0 +1,62 @@
+/*
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_SINGLE_RESOURCE_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_SINGLE_RESOURCE_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/rewrite_filter.h"
+
+namespace net_instaweb {
+
+// Simpler interface for RewriteFilters which only convert one input resource
+// to one output resource.
+//
+// To derive from this class, implement RewriteLoadedResource.
+class RewriteSingleResourceFilter : public RewriteFilter {
+ public:
+ explicit RewriteSingleResourceFilter(
+ RewriteDriver* driver, StringPiece filter_prefix)
+ : RewriteFilter(driver, filter_prefix),
+ resource_manager_(driver->resource_manager()) {
+ }
+ virtual ~RewriteSingleResourceFilter();
+
+ virtual bool Fetch(OutputResource* output_resource,
+ Writer* response_writer,
+ const MetaData& request_header,
+ MetaData* response_headers,
+ MessageHandler* message_handler,
+ UrlAsyncFetcher::Callback* callback);
+
+ protected:
+ // Derived classes must implement this function instead of Fetch.
+ virtual bool RewriteLoadedResource(const Resource* input_resource,
+ OutputResource* output_resource) = 0;
+
+ private:
+ class FetchCallback;
+
+ ResourceManager* resource_manager_;
+
+ DISALLOW_COPY_AND_ASSIGN(RewriteSingleResourceFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_REWRITE_SINGLE_RESOURCE_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/script_tag_scanner.h b/trunk/src/net/instaweb/rewriter/public/script_tag_scanner.h
new file mode 100644
index 0000000..d3c173f
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/script_tag_scanner.h
@@ -0,0 +1,83 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_SCRIPT_TAG_SCANNER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_SCRIPT_TAG_SCANNER_H_
+
+#include <set>
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/util/public/atom.h"
+
+namespace net_instaweb {
+class HtmlParse;
+
+class ScriptTagScanner {
+ public:
+ enum ScriptClassification {
+ kNonScript,
+ kUnknownScript,
+ kJavaScript
+ };
+
+ // Bit flags that specify when the script is to be run
+ enum ExecutionModeFlags {
+ kExecuteSync = 0,
+ kExecuteDefer = 1,
+ kExecuteAsync = 2,
+ kExecuteForEvent = 4 // IE extension. If this is set,
+ // script will not run in browsers following HTML5,
+ // and will run at hard-to-describe time in IE.
+ };
+
+ explicit ScriptTagScanner(HtmlParse* html_parse);
+
+ // Examines an HTML element and determine if it is a script.
+ // If it's not, it returns kNonScript and doesn't touch *src.
+ // If it is a script, it returns whether it is JavaScript or not,
+ // and sets *src to the src attribute (perhaps NULL)
+ ScriptClassification ParseScriptElement(HtmlElement* element,
+ HtmlElement::Attribute** src);
+
+ // Returns which execution model attributes are set.
+ // Keep in mind, however, that HTML5 browsers will ignore
+ // kExecuteDefer and kExecuteAsync on elements without src=''
+ int ExecutionMode(const HtmlElement* element) const;
+ private:
+ // Normalizes the input str by trimming whitespace and lowercasing.
+ static std::string Normalized(const StringPiece& str);
+
+ bool IsJsMime(const std::string& type_str);
+
+ const Atom s_async_;
+ const Atom s_defer_;
+ const Atom s_event_;
+ const Atom s_for_;
+ const Atom s_language_;
+ const Atom s_script_;
+ const Atom s_src_;
+ const Atom s_type_;
+ std::set<std::string> javascript_mimetypes_;
+
+ DISALLOW_COPY_AND_ASSIGN(ScriptTagScanner);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_SCRIPT_TAG_SCANNER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/strip_scripts_filter.h b/trunk/src/net/instaweb/rewriter/public/strip_scripts_filter.h
new file mode 100644
index 0000000..f755b1f
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/strip_scripts_filter.h
@@ -0,0 +1,45 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_STRIP_SCRIPTS_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_STRIP_SCRIPTS_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/util/public/atom.h"
+
+namespace net_instaweb {
+
+// Remove all scripts from a web page.
+class StripScriptsFilter : public EmptyHtmlFilter {
+ public:
+ explicit StripScriptsFilter(HtmlParse* html_parse);
+
+ virtual void EndElement(HtmlElement* element);
+ virtual const char* Name() const { return "StripScripts"; }
+
+ private:
+ HtmlParse* html_parse_;
+ Atom s_script_;
+
+ DISALLOW_COPY_AND_ASSIGN(StripScriptsFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_STRIP_SCRIPTS_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/url_input_resource.h b/trunk/src/net/instaweb/rewriter/public/url_input_resource.h
new file mode 100644
index 0000000..5d6114c
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/url_input_resource.h
@@ -0,0 +1,63 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+//
+// Input resource created based on a network resource.
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_URL_INPUT_RESOURCE_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_URL_INPUT_RESOURCE_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/resource.h"
+
+namespace net_instaweb {
+
+class MessageHandler;
+class MetaData;
+class RewriteOptions;
+class UrlFetcher;
+
+class UrlInputResource : public Resource {
+ public:
+ UrlInputResource(ResourceManager* manager,
+ const RewriteOptions* options,
+ const ContentType* type,
+ const StringPiece& url)
+ : Resource(manager, type),
+ url_(url.data(), url.size()),
+ rewrite_options_(options) {
+ }
+ virtual ~UrlInputResource();
+
+ virtual std::string url() const { return url_; }
+ const RewriteOptions* rewrite_options() const { return rewrite_options_; }
+
+ protected:
+ virtual bool Load(MessageHandler* message_handler);
+ virtual void LoadAndCallback(AsyncCallback* callback,
+ MessageHandler* message_handler);
+
+ private:
+ std::string url_;
+ const RewriteOptions* rewrite_options_;
+
+ DISALLOW_COPY_AND_ASSIGN(UrlInputResource);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_URL_INPUT_RESOURCE_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/url_left_trim_filter.h b/trunk/src/net/instaweb/rewriter/public/url_left_trim_filter.h
new file mode 100644
index 0000000..a92cd68
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/url_left_trim_filter.h
@@ -0,0 +1,82 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_URL_LEFT_TRIM_FILTER_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_URL_LEFT_TRIM_FILTER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/rewriter/public/resource_tag_scanner.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+// Relatively simple filter that trims redundant information from the left end
+// of each url. In particular, we can drop http: from any url that is on a page
+// served via http (etc. for other protocols). By the same token, a page often
+// contains fully-qualified urls that can be made base-relative (especially as
+// we may do this as a result of rewriting itself), or root-relative. We should
+// strip the leading portions of these urls.
+//
+// We actually register the base url of a page. This in turn registers
+// individual trimmings for the protocol, host, and path in that order. These
+// portions of the url are then trimmed off in order by the Trim(...) operation.
+//
+// TODO(jmaessen): url references in css / outside src= and href= properties
+// TODO(jmaessen): do we need a generic filter base class that just finds urls
+// and calls a class method? Or do we need context information for any
+// transform other than the sort of thing you see here?
+// TODO(jmaessen): Do we care to introduce ../ in order to relativize more urls?
+// Do we have a library solution to do so with minimal effort?
+
+class Statistics;
+class Variable;
+
+class UrlLeftTrimFilter : public EmptyHtmlFilter {
+ public:
+ UrlLeftTrimFilter(HtmlParse* html_parse, Statistics* resource_manager);
+ static void Initialize(Statistics* statistics);
+ virtual void StartElement(HtmlElement* element);
+ // TODO(sligocki): This is broken and only adds base_urls. We need to be able
+ // to ResetBaseUrl to a new value. There is only one base_url at a time.
+ virtual void AddBaseUrl(const StringPiece& base_url);
+ virtual const char* Name() const { return "UrlLeftTrim"; }
+
+ protected:
+ friend class UrlLeftTrimFilterTest;
+ bool Trim(StringPiece* url);
+ void AddTrimming(const StringPiece& trimming);
+
+ private:
+ HtmlParse* html_parse_;
+ StringVector left_trim_strings_;
+ const Atom s_base_;
+ const Atom s_href_;
+ const Atom s_src_;
+ Variable* trim_count_;
+ Variable* trim_saved_bytes_;
+
+ void TrimAttribute(HtmlElement::Attribute* attr);
+
+ DISALLOW_COPY_AND_ASSIGN(UrlLeftTrimFilter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_URL_LEFT_TRIM_FILTER_H_
diff --git a/trunk/src/net/instaweb/rewriter/public/url_partnership.h b/trunk/src/net/instaweb/rewriter/public/url_partnership.h
new file mode 100644
index 0000000..c9d504c
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/public/url_partnership.h
@@ -0,0 +1,87 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef NET_INSTAWEB_REWRITER_PUBLIC_URL_PARTNERSHIP_H_
+#define NET_INSTAWEB_REWRITER_PUBLIC_URL_PARTNERSHIP_H_
+
+#include <vector>
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/string_util.h"
+#include <string>
+
+class GURL;
+
+namespace net_instaweb {
+
+class MessageHandler;
+class RewriteOptions;
+
+// A URL partnership should be estalished in order to combine resources,
+// such as in CSS combination, JS combination, or image spriting. This
+// class centralizes the handling of such combinations, answering two
+// questions:
+// 1. Is it legal for a new URL to enter into the partnership
+// 2. What is the greatest common prefix
+// 3. What are the unique suffices for the elements.
+class UrlPartnership {
+ public:
+ UrlPartnership(const RewriteOptions* options, const GURL& original_request);
+ ~UrlPartnership();
+
+ // Adds a URL to a combination. If it can be legally added, consulting
+ // the DomainLaywer, then true is returned.
+ bool AddUrl(const StringPiece& resource_url, MessageHandler* handler);
+
+ // Computes the resolved base common to all URLs. This will always
+ // have a trailing slash.
+ std::string ResolvedBase() const;
+
+ // Returns the number of URLs that have been successfully added.
+ int num_urls() { return gurl_vector_.size(); }
+
+ // Returns the relative path of a particular URL that was added into
+ // the partnership. This requires that Resolve() be called first.
+ std::string RelativePath(int index) const;
+
+ // Returns the full resolved path
+ const GURL* FullPath(int index) const { return gurl_vector_[index]; }
+
+ // Removes the last URL that was added to the partnership.
+ void RemoveLast();
+
+ protected:
+ int num_components() const { return common_components_.size(); }
+
+ private:
+ void IncrementalResolve(int index);
+
+ typedef std::vector<GURL*> GurlVector;
+ GurlVector gurl_vector_;
+ std::string domain_;
+ GURL domain_gurl_;
+ const RewriteOptions* rewrite_options_;
+ GURL original_origin_and_path_;
+
+ // common_components_ is updated while adding Urls to support incremental
+ // resolution.
+ StringVector common_components_;
+
+ DISALLOW_COPY_AND_ASSIGN(UrlPartnership);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_PUBLIC_URL_PARTNERSHIP_H_
diff --git a/trunk/src/net/instaweb/rewriter/remove_comments_filter.cc b/trunk/src/net/instaweb/rewriter/remove_comments_filter.cc
new file mode 100644
index 0000000..0d1cb38
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/remove_comments_filter.cc
@@ -0,0 +1,32 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "net/instaweb/rewriter/public/remove_comments_filter.h"
+
+#include "net/instaweb/htmlparse/public/html_parse.h"
+
+namespace net_instaweb {
+
+RemoveCommentsFilter::RemoveCommentsFilter(HtmlParse* html_parse)
+ : html_parse_(html_parse) {}
+
+void RemoveCommentsFilter::Comment(HtmlCommentNode* comment) {
+ html_parse_->DeleteElement(comment);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/remove_comments_filter_test.cc b/trunk/src/net/instaweb/rewriter/remove_comments_filter_test.cc
new file mode 100644
index 0000000..e628d69
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/remove_comments_filter_test.cc
@@ -0,0 +1,65 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_parse_test_base.h"
+#include "net/instaweb/rewriter/public/remove_comments_filter.h"
+
+namespace net_instaweb {
+
+class RemoveCommentsFilterTest : public HtmlParseTestBase {
+ protected:
+ RemoveCommentsFilterTest()
+ : remove_comments_filter_(&html_parse_) {
+ html_parse_.AddFilter(&remove_comments_filter_);
+ }
+
+ virtual bool AddBody() const { return false; }
+
+ private:
+ RemoveCommentsFilter remove_comments_filter_;
+
+ DISALLOW_COPY_AND_ASSIGN(RemoveCommentsFilterTest);
+};
+
+TEST_F(RemoveCommentsFilterTest, NoComments) {
+ ValidateNoChanges("no_comments",
+ "<head><title>Hello</title></head>"
+ "<body>Why, hello there!</body>");
+}
+
+TEST_F(RemoveCommentsFilterTest, RemoveComment) {
+ ValidateExpected("remove_comment",
+ "<body>hello <!--world--></body>",
+ "<body>hello </body>");
+}
+
+TEST_F(RemoveCommentsFilterTest, RemoveMultipleComments) {
+ ValidateExpected("remove_multiple_comments",
+ "<head><!--1--><title>Hi<!--2--></title></head>"
+ "<body><!--3-->hello<!--4--><!--5--></body>",
+ "<head><title>Hi</title></head>"
+ "<body>hello</body>");
+}
+
+TEST_F(RemoveCommentsFilterTest, DoNotRemoveIEDirective) {
+ ValidateNoChanges("do_not_remove_ie_directive",
+ "<body>hello <!--[if IE 8]>world<![endif]--></body>");
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/resource.cc b/trunk/src/net/instaweb/rewriter/resource.cc
new file mode 100644
index 0000000..2957739
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/resource.cc
@@ -0,0 +1,87 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+// jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/resource.h"
+#include "net/instaweb/util/public/content_type.h"
+
+namespace net_instaweb {
+
+const int64 Resource::kDefaultExpireTimeMs = 5 * 60 * 1000;
+
+Resource::~Resource() {
+}
+
+int64 Resource::CacheExpirationTimeMs() const {
+ int64 input_expire_time_ms = kDefaultExpireTimeMs;
+ if (meta_data_.IsCacheable()) {
+ input_expire_time_ms = meta_data_.CacheExpirationTimeMs();
+ }
+ return input_expire_time_ms;
+}
+
+// Note: OutputResource overrides this to also set the file extension.
+void Resource::SetType(const ContentType* type) {
+ type_ = type;
+}
+
+void Resource::DetermineContentType() {
+ // Try to determine the content type from the URL extension, or
+ // the response headers.
+ CharStarVector content_types;
+ MetaData* headers = metadata();
+ const ContentType* content_type = NULL;
+ if (headers->Lookup("Content-type", &content_types)) {
+ for (int i = 0, n = content_types.size(); (i < n) && content_type == NULL;
+ ++i) {
+ content_type = MimeTypeToContentType(content_types[i]);
+ }
+ }
+
+ if (content_type == NULL) {
+ // If there is no content type in input headers, then try to
+ // determine it from the name.
+ std::string trimmed_url;
+ TrimWhitespace(url(), &trimmed_url);
+ content_type = NameExtensionToContentType(trimmed_url);
+ }
+ if (content_type != NULL) {
+ SetType(content_type);
+ }
+}
+
+// Default, blocking implementation which calls Load.
+// Resources which can fetch asynchronously should override this.
+void Resource::LoadAndCallback(AsyncCallback* callback,
+ MessageHandler* message_handler) {
+ callback->Done(Load(message_handler), this);
+}
+
+Resource::AsyncCallback::~AsyncCallback() {
+}
+
+bool Resource::Link(HTTPValue* value, MessageHandler* handler) {
+ SharedString* contents_and_headers = value->share();
+ return value_.Link(contents_and_headers, &meta_data_, handler);
+}
+
+bool Resource::IsCacheable() const {
+ return true;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/resource_manager.cc b/trunk/src/net/instaweb/rewriter/resource_manager.cc
new file mode 100644
index 0000000..94e7f8e
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/resource_manager.cc
@@ -0,0 +1,653 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/resource_manager.h"
+
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/data_url_input_resource.h"
+#include "net/instaweb/rewriter/public/file_input_resource.h"
+#include "net/instaweb/rewriter/public/output_resource.h"
+#include "net/instaweb/rewriter/public/resource_namer.h"
+#include "net/instaweb/rewriter/public/rewrite_driver_factory.h"
+#include "net/instaweb/rewriter/public/rewrite_filter.h"
+#include "net/instaweb/rewriter/public/rewrite_options.h"
+#include "net/instaweb/rewriter/public/url_input_resource.h"
+#include "net/instaweb/rewriter/public/url_partnership.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/hasher.h"
+#include "net/instaweb/util/public/http_cache.h"
+#include "net/instaweb/util/public/http_value.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/named_lock_manager.h"
+#include "net/instaweb/util/public/statistics.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/time_util.h"
+#include "net/instaweb/util/public/timer.h"
+#include "net/instaweb/util/public/url_escaper.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// resource_url_domain_rejections counts the number of urls on a page that we
+// could have rewritten, except that they lay in a domain that did not
+// permit resource rewriting relative to the current page.
+const char kResourceUrlDomainRejections[] = "resource_url_domain_rejections";
+
+const int64 kGeneratedMaxAgeMs = Timer::kYearMs;
+const int64 kGeneratedMaxAgeSec = Timer::kYearMs / Timer::kSecondMs;
+
+// Our HTTP cache mostly stores full URLs, including the http: prefix,
+// mapping them into the URL contents and HTTP headers. However, we
+// also put name->hash mappings into the HTTP cache, and we prefix
+// these with "ResourceName:" to disambiguate them.
+//
+// Cache entries prefixed this way map the base name of a resource
+// into the hash-code of the contents. This mapping has a TTL based
+// on the minimum TTL of the input resources used to construct the
+// resource. After that TTL has expired, we will need to re-fetch the
+// resources from their origin, and recompute the hash.
+//
+// Whenever we change the hashing function we can bust caches by
+// changing this prefix.
+//
+// TODO(jmarantz): inject the SVN version number here to automatically bust
+// caches whenever pagespeed is upgraded.
+const char kCacheKeyPrefix[] = "rname/";
+
+} // namespace
+
+const int ResourceManager::kNotSharded = -1;
+
+// We set etags for our output resources to "W/0". The "W" means
+// that this etag indicates a functional consistency, but is not
+// guaranteeing byte-consistency. This distinction is important because
+// we serve different bytes for clients that do not accept gzip.
+//
+// This value is a shared constant so that it can also be used in
+// the Apache-specific code that repairs headers after mod_headers
+// alters them.
+const char ResourceManager::kResourceEtagValue[] = "W/0";
+
+ResourceManager::ResourceManager(const StringPiece& file_prefix,
+ FileSystem* file_system,
+ FilenameEncoder* filename_encoder,
+ UrlAsyncFetcher* url_async_fetcher,
+ Hasher* hasher,
+ HTTPCache* http_cache,
+ NamedLockManager* lock_manager)
+ : file_prefix_(file_prefix.data(), file_prefix.size()),
+ resource_id_(0),
+ file_system_(file_system),
+ filename_encoder_(filename_encoder),
+ url_async_fetcher_(url_async_fetcher),
+ hasher_(hasher),
+ statistics_(NULL),
+ resource_url_domain_rejections_(NULL),
+ http_cache_(http_cache),
+ url_escaper_(new UrlEscaper()),
+ relative_path_(false),
+ store_outputs_in_file_system_(true),
+ lock_manager_(lock_manager),
+ max_age_string_(StringPrintf("max-age=%d",
+ static_cast<int>(kGeneratedMaxAgeSec))) {
+}
+
+ResourceManager::~ResourceManager() {
+}
+
+void ResourceManager::Initialize(Statistics* statistics) {
+ statistics->AddVariable(kResourceUrlDomainRejections);
+}
+
+#if 0
+// Preserved for the sake of making it easier to revive sharding.
+
+std::string ResourceManager::UrlPrefixFor(const ResourceNamer& namer) const {
+ CHECK(!namer.hash().empty());
+ std::string url_prefix;
+ if (num_shards_ == 0) {
+ url_prefix = url_prefix_pattern_;
+ } else {
+ size_t hash = namer.Hash();
+ int shard = hash % num_shards_;
+ CHECK_NE(std::string::npos, url_prefix_pattern_.find("%d"));
+ // The following uses a user-provided printf format string; this would be
+ // really dangerous if we did not validate url_prefix_pattern_ by calling
+ // ValidateShardsAgainstUrlPrefixPattern() below.
+ url_prefix = StringPrintf(url_prefix_pattern_.c_str(), shard); // NOLINT
+ }
+ return url_prefix;
+}
+
+// Decode a base path into a shard number and canonical base url.
+// Right now the canonical base url is empty for the old resource
+// naming scheme, and non-empty otherwise.
+// TODO(jmaessen): Either axe or adapt to sharding post-url_prefix.
+std::string ResourceManager::CanonicalizeBase(
+ const StringPiece& base, int* shard) const {
+ std::string base_str = base.as_string();
+ base_str += "/";
+ std::string result;
+ if (num_shards_ == 0) {
+ CHECK_EQ(std::string::npos, url_prefix_pattern_.find("%d"));
+ if (url_prefix_pattern_.compare(base_str) != 0) {
+ base.CopyToString(&result);
+ }
+ } else {
+ CHECK_NE(std::string::npos, url_prefix_pattern_.find("%d"));
+ // TODO(jmaessen): Ugh. Lint hates this sscanf call and so do I. Can parse
+ // based on the results of the above find.
+ if (!sscanf(base_str.c_str(), url_prefix_pattern_.c_str(), shard) == 1) {
+ base.CopyToString(&result);
+ }
+ }
+ return result;
+}
+
+void ResourceManager::ValidateShardsAgainstUrlPrefixPattern() {
+ std::string::size_type pos = url_prefix_pattern_.find('%');
+ if (num_shards_ == 0) {
+ CHECK(pos == StringPiece::npos) << "URL prefix should not have a percent "
+ << "when num_shards==0";
+ } else {
+ // Ensure that the % is followed by a 'd'. But be careful because
+ // the percent may have appeared at the end of the string, which
+ // is not necessarily null-terminated.
+ if ((pos == std::string::npos) ||
+ ((pos + 1) == url_prefix_pattern_.size()) ||
+ (url_prefix_pattern_.substr(pos + 1, 1) != "d")) {
+ CHECK(false) << "url_prefix must contain exactly one %d";
+ } else {
+ // make sure there is not another percent
+ pos = url_prefix_pattern_.find('%', pos + 2);
+ CHECK(pos == std::string::npos) << "Extra % found in url_prefix_pattern";
+ }
+ }
+}
+#endif
+
+// TODO(jmarantz): consider moving this method to MetaData
+void ResourceManager::SetDefaultHeaders(const ContentType* content_type,
+ MetaData* header) const {
+ CHECK_EQ(0, header->major_version());
+ CHECK_EQ(0, header->NumAttributes());
+ header->set_major_version(1);
+ header->set_minor_version(1);
+ header->SetStatusAndReason(HttpStatus::kOK);
+ if (content_type != NULL) {
+ header->Add(HttpAttributes::kContentType, content_type->mime_type());
+ }
+ int64 now_ms = http_cache_->timer()->NowMs();
+ header->Add(HttpAttributes::kCacheControl, max_age_string_);
+ std::string expires_string;
+ if (ConvertTimeToString(now_ms + kGeneratedMaxAgeMs, &expires_string)) {
+ header->Add(HttpAttributes::kExpires, expires_string);
+ }
+
+ // While PageSpeed claims the "Vary" header is needed to avoid proxy cache
+ // issues for clients where some accept gzipped content and some don't, it
+ // should not be done here. It should instead be done by whatever code is
+ // conditionally gzipping the content based on user-agent, e.g. mod_deflate.
+ // header->Add(HttpAttributes::kVary, HttpAttributes::kAcceptEncoding);
+
+ // ETag is superfluous for mod_pagespeed as we sign the URL with the
+ // content hash. However, we have seen evidence that IE8 will not
+ // serve images from its cache when the image lacks an ETag. Since
+ // we sign URLs, there is no reason to have a unique signature in
+ // the ETag.
+ header->Add(HttpAttributes::kEtag, kResourceEtagValue);
+
+ // TODO(jmarantz): add date/last-modified headers by default.
+ CharStarVector v;
+ if (!header->Lookup(HttpAttributes::kDate, &v)) {
+ header->SetDate(now_ms);
+ }
+ if (!header->Lookup(HttpAttributes::kLastModified, &v)) {
+ header->SetLastModified(now_ms);
+ }
+
+ // TODO(jmarantz): Page-speed suggested adding a "Last-Modified" header
+ // for cache validation. To do this we must track the max of all
+ // Last-Modified values for all input resources that are used to
+ // create this output resource. For now we are using the current
+ // time.
+
+ header->ComputeCaching();
+}
+
+// TODO(jmarantz): consider moving this method to MetaData
+void ResourceManager::SetContentType(const ContentType* content_type,
+ MetaData* header) {
+ CHECK(content_type != NULL);
+ header->RemoveAll(HttpAttributes::kContentType);
+ header->Add(HttpAttributes::kContentType, content_type->mime_type());
+ header->ComputeCaching();
+}
+
+// Constructs an output resource corresponding to the specified input resource
+// and encoded using the provided encoder.
+OutputResource* ResourceManager::CreateOutputResourceFromResource(
+ const StringPiece& filter_prefix,
+ const ContentType* content_type,
+ UrlSegmentEncoder* encoder,
+ Resource* input_resource,
+ MessageHandler* handler) {
+ OutputResource* result = NULL;
+ if (input_resource != NULL) {
+ std::string url = input_resource->url();
+ GURL input_gurl(url);
+ CHECK(input_gurl.is_valid()); // or input_resource should have been NULL.
+ std::string name;
+ encoder->EncodeToUrlSegment(GoogleUrl::Leaf(input_gurl), &name);
+ result = CreateOutputResourceWithPath(
+ GoogleUrl::AllExceptLeaf(input_gurl),
+ filter_prefix, name, content_type, handler);
+ }
+ return result;
+}
+
+OutputResource* ResourceManager::CreateOutputResourceForRewrittenUrl(
+ const GURL& document_gurl,
+ const StringPiece& filter_prefix,
+ const StringPiece& resource_url,
+ const ContentType* content_type,
+ UrlSegmentEncoder* encoder,
+ const RewriteOptions* rewrite_options,
+ MessageHandler* handler) {
+ OutputResource* output_resource = NULL;
+ UrlPartnership partnership(rewrite_options, document_gurl);
+ if (partnership.AddUrl(resource_url, handler)) {
+ std::string base = partnership.ResolvedBase();
+ std::string relative_url = partnership.RelativePath(0);
+ std::string name;
+ encoder->EncodeToUrlSegment(relative_url, &name);
+ output_resource = CreateOutputResourceWithPath(
+ base, filter_prefix, name, content_type, handler);
+ }
+ return output_resource;
+}
+
+OutputResource* ResourceManager::CreateOutputResourceWithPath(
+ const StringPiece& path,
+ const StringPiece& filter_prefix,
+ const StringPiece& name,
+ const ContentType* content_type,
+ MessageHandler* handler) {
+ CHECK(content_type != NULL);
+ ResourceNamer full_name;
+ full_name.set_id(filter_prefix);
+ full_name.set_name(name);
+ // TODO(jmaessen): The addition of 1 below avoids the leading ".";
+ // make this convention consistent and fix all code.
+ full_name.set_ext(content_type->file_extension() + 1);
+ OutputResource* resource =
+ new OutputResource(this, path, full_name, content_type);
+
+ // Determine whether this output resource is still valid by looking
+ // up by hash in the http cache. Note that this cache entry will
+ // expire when any of the origin resources expire.
+ SimpleMetaData meta_data;
+ StringPiece hash_extension;
+ HTTPValue value;
+ std::string name_key = StrCat(kCacheKeyPrefix, resource->name_key());
+ if ((http_cache_->Find(name_key, &value, &meta_data, handler)
+ == HTTPCache::kFound) &&
+ value.ExtractContents(&hash_extension)) {
+ ResourceNamer hash_ext;
+ if (hash_ext.DecodeHashExt(hash_extension)) {
+ resource->SetHash(hash_ext.hash());
+ // Note that the '.' must be included in the suffix
+ // TODO(jmarantz): remove this from the suffix.
+ resource->set_suffix(StrCat(".", hash_ext.ext()));
+ }
+ }
+ return resource;
+}
+
+OutputResource* ResourceManager::CreateOutputResourceForFetch(
+ const StringPiece& url) {
+ OutputResource* resource = NULL;
+ std::string url_string(url.data(), url.size());
+ GURL gurl(url_string);
+ if (gurl.is_valid()) {
+ std::string name = GoogleUrl::Leaf(gurl);
+ ResourceNamer namer;
+ if (namer.Decode(name)) {
+ std::string base = GoogleUrl::AllExceptLeaf(gurl);
+ resource = new OutputResource(this, base, namer, NULL);
+ }
+ }
+ return resource;
+}
+
+void ResourceManager::set_filename_prefix(const StringPiece& file_prefix) {
+ file_prefix.CopyToString(&file_prefix_);
+}
+
+// Implements lazy initialization of resource_url_domain_rejections_,
+// necessitated by the fact that we can set_statistics before
+// Initialize(...) has been called and thus can't safely look
+// for the variable until first use.
+void ResourceManager::IncrementResourceUrlDomainRejections() {
+ if (resource_url_domain_rejections_ == NULL) {
+ if (statistics_ == NULL) {
+ return;
+ }
+ resource_url_domain_rejections_ =
+ statistics_->GetVariable(kResourceUrlDomainRejections);
+ }
+ resource_url_domain_rejections_->Add(1);
+}
+
+Resource* ResourceManager::CreateInputResource(
+ const GURL& base_gurl,
+ const StringPiece& input_url,
+ const RewriteOptions* rewrite_options,
+ MessageHandler* handler) {
+ UrlPartnership partnership(rewrite_options, base_gurl);
+ Resource* resource = NULL;
+ if (partnership.AddUrl(input_url, handler)) {
+ const GURL* input_gurl = partnership.FullPath(0);
+ resource = CreateInputResourceUnchecked(*input_gurl, rewrite_options,
+ handler);
+ } else {
+ handler->Message(kInfo, "%s: Invalid url relative to '%s'",
+ input_url.as_string().c_str(), base_gurl.spec().c_str());
+ IncrementResourceUrlDomainRejections();
+ resource = NULL;
+ }
+ return resource;
+}
+
+Resource* ResourceManager::CreateInputResourceAndReadIfCached(
+ const GURL& base_gurl, const StringPiece& input_url,
+ const RewriteOptions* rewrite_options,
+ MessageHandler* handler) {
+ Resource* input_resource = CreateInputResource(
+ base_gurl, input_url, rewrite_options, handler);
+ if (input_resource != NULL &&
+ (!input_resource->IsCacheable() ||
+ !ReadIfCached(input_resource, handler))) {
+ handler->Message(kInfo,
+ "%s: Couldn't fetch resource %s to rewrite.",
+ base_gurl.spec().c_str(), input_url.as_string().c_str());
+ delete input_resource;
+ input_resource = NULL;
+ }
+ return input_resource;
+}
+
+Resource* ResourceManager::CreateInputResourceFromOutputResource(
+ UrlSegmentEncoder* encoder,
+ OutputResource* output_resource,
+ const RewriteOptions* rewrite_options,
+ MessageHandler* handler) {
+ // Assumes output_resource has a url that's been checked by a lawyer. We
+ // should already have checked the signature on the encoded resource name and
+ // failed to create output_resource if it didn't match.
+ Resource* input_resource = NULL;
+ std::string input_name;
+ if (encoder->DecodeFromUrlSegment(output_resource->name(), &input_name)) {
+ GURL base_gurl(output_resource->resolved_base());
+ GURL input_gurl = base_gurl.Resolve(input_name);
+ input_resource = CreateInputResourceUnchecked(input_gurl,
+ rewrite_options, handler);
+ }
+ return input_resource;
+}
+
+Resource* ResourceManager::CreateInputResourceAbsolute(
+ const StringPiece& absolute_url, const RewriteOptions* rewrite_options,
+ MessageHandler* handler) {
+ std::string url_string(absolute_url.data(), absolute_url.size());
+ GURL url(url_string);
+ return CreateInputResourceUnchecked(url, rewrite_options, handler);
+}
+
+Resource* ResourceManager::CreateInputResourceUnchecked(
+ const GURL& url, const RewriteOptions* rewrite_options,
+ MessageHandler* handler) {
+ if (!url.is_valid()) {
+ // Note: Bad user-content can leave us here. But it's really hard
+ // to concatenate a valid protocol and domain onto an arbitrary string
+ // and end up with an invalid GURL.
+ handler->Message(kWarning, "%s: Invalid url",
+ url.possibly_invalid_spec().c_str());
+ return NULL;
+ }
+ std::string url_string = GoogleUrl::Spec(url);
+
+ Resource* resource = NULL;
+
+ if (url.SchemeIs("data")) {
+ resource = DataUrlInputResource::Make(url_string, this);
+ if (resource == NULL) {
+ // Note: Bad user-content can leave us here.
+ handler->Message(kWarning, "Badly formatted data url '%s'",
+ url_string.c_str());
+ }
+ } else if (url.SchemeIs("http")) {
+ // TODO(sligocki): Figure out if these are actually local, in
+ // which case we can do a local file read.
+
+ // Note: type may be NULL if url has an unexpected or malformed extension.
+ const ContentType* type = NameExtensionToContentType(url_string);
+ resource = new UrlInputResource(this, rewrite_options, type, url_string);
+ } else {
+ // Note: Bad user-content can leave us here.
+ handler->Message(kWarning, "Unsupported scheme '%s' for url '%s'",
+ url.scheme().c_str(), url_string.c_str());
+ }
+ return resource;
+}
+
+// TODO(jmarantz): remove writer/response_headers args from this function
+// and force caller to pull those directly from output_resource, as that will
+// save the effort of copying the headers.
+//
+// It will also simplify this routine quite a bit.
+bool ResourceManager::FetchOutputResource(
+ OutputResource* output_resource,
+ Writer* writer, MetaData* response_headers,
+ MessageHandler* handler, BlockingBehavior blocking) const {
+ if (output_resource == NULL) {
+ return false;
+ }
+
+ // TODO(jmarantz): we are making lots of copies of the data. We should
+ // retrieve the data from the cache without copying it.
+
+ // The http_cache is shared between multiple different classes in Instaweb.
+ // To avoid colliding hash keys, we use a class-specific prefix.
+ //
+ // TODO(jmarantz): consider formalizing this in the HTTPCache API and
+ // doing the StrCat inside.
+ bool ret = false;
+ StringPiece content;
+ MetaData* meta_data = output_resource->metadata();
+ if (output_resource->IsWritten()) {
+ ret = ((writer == NULL) ||
+ ((output_resource->value_.ExtractContents(&content)) &&
+ writer->Write(content, handler)));
+ } else if (output_resource->has_hash()) {
+ std::string url = output_resource->url();
+ // Check cache once without lock, then if that fails try again with lock.
+ // Note that it would be *correct* to lock up front and only check once.
+ // However, the common case here is that the resource is present (because
+ // this path mostly happens during resource fetch). We want to avoid
+ // unnecessarily serializing resource fetch on a lock.
+ for (int i = 0; !ret && i < 2; ++i) {
+ if ((http_cache_->Find(url, &output_resource->value_, meta_data, handler)
+ == HTTPCache::kFound) &&
+ ((writer == NULL) ||
+ output_resource->value_.ExtractContents(&content)) &&
+ ((writer == NULL) || writer->Write(content, handler))) {
+ output_resource->set_written(true);
+ ret = true;
+ } else if (ReadIfCached(output_resource, handler)) {
+ content = output_resource->contents();
+ http_cache_->Put(url, *meta_data, content, handler);
+ ret = ((writer == NULL) || writer->Write(content, handler));
+ }
+ // On the first iteration, obtain the lock if we don't have data.
+ if (!ret && i == 0 && !output_resource->LockForCreation(this, blocking)) {
+ // We didn't get the lock; we need to abandon ship. The caller should
+ // see this as a successful fetch for which IsWritten() remains false.
+ CHECK(!output_resource->IsWritten());
+ ret = true;
+ }
+ }
+ } else {
+ // TODO(jmaessen): This path should also re-try fetching the resource after
+ // obtaining the lock. However, in this case we need to look for the hash
+ // in the cache first, which duplicates logic from creation time and makes
+ // life generally complicated.
+ ret = !output_resource->LockForCreation(this, blocking);
+ }
+ if (ret && (response_headers != NULL) && (response_headers != meta_data)) {
+ response_headers->CopyFrom(*meta_data);
+ }
+ return ret;
+}
+
+bool ResourceManager::Write(HttpStatus::Code status_code,
+ const StringPiece& contents,
+ OutputResource* output,
+ int64 origin_expire_time_ms,
+ MessageHandler* handler) {
+ MetaData* meta_data = output->metadata();
+ SetDefaultHeaders(output->type(), meta_data);
+ meta_data->SetStatusAndReason(status_code);
+
+ scoped_ptr<OutputResource::OutputWriter> writer(output->BeginWrite(handler));
+ bool ret = (writer != NULL);
+ if (ret) {
+ ret = writer->Write(contents, handler);
+ ret &= output->EndWrite(writer.get(), handler);
+ http_cache_->Put(output->url(), &output->value_, handler);
+
+ if (!output->generated()) {
+ // Map the name of this resource to the fully expanded filename. The
+ // name of the output resource is usually a function of how it is
+ // constructed from input resources. For example, with combine_css,
+ // output->name() encodes all the component CSS filenames. The filename
+ // this maps to includes the hash of the content. Thus the two mappings
+ // have different lifetimes.
+ //
+ // The name->filename map expires when any of the origin files expire.
+ // When that occurs, fresh content must be read, and the output must
+ // be recomputed and re-hashed.
+ //
+ // However, the hashed output filename can live, essentially, forever.
+ // This is what we'll hash first as meta_data's default headers are
+ // to cache forever.
+
+ // Now we'll mutate meta_data to expire when the origin expires, and
+ // map the name to the hash.
+ int64 delta_ms = origin_expire_time_ms - http_cache_->timer()->NowMs();
+ int64 delta_sec = delta_ms / 1000;
+ if ((delta_sec > 0) || http_cache_->force_caching()) {
+ SimpleMetaData origin_meta_data;
+ SetDefaultHeaders(output->type(), &origin_meta_data);
+ std::string cache_control = StringPrintf(
+ "max-age=%ld",
+ static_cast<long>(delta_sec)); // NOLINT
+ origin_meta_data.RemoveAll(HttpAttributes::kCacheControl);
+ origin_meta_data.Add(HttpAttributes::kCacheControl, cache_control);
+ origin_meta_data.ComputeCaching();
+
+ std::string name_key = StrCat(kCacheKeyPrefix, output->name_key());
+ http_cache_->Put(name_key, origin_meta_data, output->hash_ext(),
+ handler);
+ }
+ }
+ } else {
+ // Note that we've already gotten a "could not open file" message;
+ // this just serves to explain why and suggest a remedy.
+ handler->Message(kInfo, "Could not create output resource"
+ " (bad filename prefix '%s'?)",
+ file_prefix_.c_str());
+ }
+ return ret;
+}
+
+void ResourceManager::ReadAsync(Resource* resource,
+ Resource::AsyncCallback* callback,
+ MessageHandler* handler) {
+ HTTPCache::FindResult result = HTTPCache::kNotFound;
+
+ // If the resource is not already loaded, and this type of resource (e.g.
+ // URL vs File vs Data) is cacheable, then try to load it.
+ if (resource->loaded()) {
+ result = HTTPCache::kFound;
+ } else if (resource->IsCacheable()) {
+ result = http_cache_->Find(resource->url(), &resource->value_,
+ resource->metadata(), handler);
+ }
+
+ switch (result) {
+ case HTTPCache::kFound:
+ callback->Done(true, resource);
+ break;
+ case HTTPCache::kRecentFetchFailedDoNotRefetch:
+ // TODO(jmarantz): in this path, should we try to fetch again
+ // sooner than 5 minutes? The issue is that in this path we are
+ // serving for the user, not for a rewrite. This could get
+ // frustrating, even if the software is functioning as intended,
+ // because a missing resource that is put in place by a site
+ // admin will not be checked again for 5 minutes.
+ //
+ // The "good" news is that if the admin is willing to crank up
+ // logging to 'info' then http_cache.cc will log the
+ // 'remembered' failure.
+ callback->Done(false, resource);
+ break;
+ case HTTPCache::kNotFound:
+ // If not, load it asynchronously.
+ resource->LoadAndCallback(callback, handler);
+ break;
+ }
+ // TODO(sligocki): Do we need to call DetermineContentType like below?
+}
+
+bool ResourceManager::ReadIfCached(Resource* resource,
+ MessageHandler* handler) const {
+ HTTPCache::FindResult result = HTTPCache::kNotFound;
+
+ // If the resource is not already loaded, and this type of resource (e.g.
+ // URL vs File vs Data) is cacheable, then try to load it.
+ if (resource->loaded()) {
+ result = HTTPCache::kFound;
+ } else if (resource->IsCacheable()) {
+ result = http_cache_->Find(resource->url(), &resource->value_,
+ resource->metadata(), handler);
+ }
+ if ((result == HTTPCache::kNotFound) && resource->Load(handler)) {
+ result = HTTPCache::kFound;
+ }
+ if (result == HTTPCache::kFound) {
+ resource->DetermineContentType();
+ return true;
+ }
+ return false;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/resource_manager_test.cc b/trunk/src/net/instaweb/rewriter/resource_manager_test.cc
new file mode 100644
index 0000000..074a8b0
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/resource_manager_test.cc
@@ -0,0 +1,261 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: abliss@google.com (Adam Bliss)
+
+// Unit-test the resource manager
+
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/output_resource.h"
+#include "net/instaweb/rewriter/public/resource_namer.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+#include "net/instaweb/rewriter/public/rewrite_filter.h"
+#include "net/instaweb/rewriter/resource_manager_testing_peer.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/meta_data.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/url_escaper.h"
+
+namespace net_instaweb {
+
+class VerifyContentsCallback : public Resource::AsyncCallback {
+ public:
+ explicit VerifyContentsCallback(const std::string& contents) :
+ contents_(contents), called_(false) {
+ }
+ virtual void Done(bool success, Resource* resource) {
+ EXPECT_EQ(0, contents_.compare(resource->contents().as_string()));
+ called_ = true;
+ }
+ void AssertCalled() {
+ EXPECT_TRUE(called_);
+ }
+ std::string contents_;
+ bool called_;
+};
+
+class ResourceManagerTest : public ResourceManagerTestBase {
+ protected:
+ ResourceManagerTest() { }
+
+ // Calls FetchOutputResource with different values of writer and
+ // response_headers, to test all branches. Expects the fetch to succeed all
+ // times, and finally returns the contents.
+ std::string FetchOutputResource(OutputResource* resource) {
+ EXPECT_TRUE(resource_manager_->FetchOutputResource(
+ resource, NULL, NULL, &message_handler_, ResourceManager::kMayBlock));
+ SimpleMetaData empty;
+ EXPECT_TRUE(resource_manager_->FetchOutputResource(
+ resource, NULL, &empty, &message_handler_, ResourceManager::kMayBlock));
+ std::string contents;
+ StringWriter writer(&contents);
+ EXPECT_TRUE(resource_manager_->FetchOutputResource(
+ resource, &writer, resource->metadata(),
+ &message_handler_, ResourceManager::kMayBlock));
+ return contents;
+ }
+
+ // Asserts that the given url starts with an appropriate prefix;
+ // then cuts off that prefix.
+ virtual void RemoveUrlPrefix(std::string* url) {
+ EXPECT_EQ(0, url->compare(0, url_prefix_.length(), url_prefix_));
+ url->erase(0, url_prefix_.length());
+ }
+
+
+ // Tests for the lifecycle and various flows of a named output resource.
+ void TestNamed() {
+ const char* filter_prefix = "fp";
+ const char* name = "name";
+ const char* contents = "contents";
+ // origin_expire_time_ms should be considerably longer than the various
+ // timeouts for resource locking, since we hit those timeouts in various
+ // places.
+ const int64 origin_expire_time_ms = 100000;
+ const ContentType* content_type = &kContentTypeText;
+ scoped_ptr<OutputResource> nor(
+ resource_manager_->CreateOutputResourceWithPath(
+ url_prefix_, filter_prefix, name, content_type, &message_handler_));
+ ASSERT_TRUE(nor.get() != NULL);
+ // Check name_key against url_prefix/fp.name
+ std::string name_key = nor->name_key();
+ RemoveUrlPrefix(&name_key);
+ EXPECT_EQ(nor->full_name().EncodeIdName(), name_key);
+ // Make sure the resource hasn't already been created (and lock it for
+ // creation).
+ EXPECT_FALSE(resource_manager_->FetchOutputResource(
+ nor.get(), NULL, NULL, &message_handler_,
+ ResourceManager::kNeverBlock));
+ EXPECT_FALSE(nor->IsWritten());
+ {
+ // Now show that another attempt to create the resource will fail.
+ // Here we attempt to create without the hash.
+ scoped_ptr<OutputResource> nor1(
+ resource_manager_->CreateOutputResourceWithPath(
+ url_prefix_, filter_prefix, name, content_type,
+ &message_handler_));
+ ASSERT_TRUE(nor1.get() != NULL);
+ // We'll succeed in fetching (meaning don't create the resource), but the
+ // resource won't be written.
+ EXPECT_TRUE(resource_manager_->FetchOutputResource(
+ nor1.get(), NULL, NULL, &message_handler_,
+ ResourceManager::kNeverBlock));
+ EXPECT_FALSE(nor1->IsWritten());
+ }
+ {
+ // Here we attempt to create the object with the hash and fail.
+ ResourceNamer namer;
+ namer.CopyFrom(nor->full_name());
+ namer.set_hash("0");
+ namer.set_ext("txt");
+ std::string name = StrCat(url_prefix_, namer.Encode());
+ scoped_ptr<OutputResource> nor1(
+ resource_manager_->CreateOutputResourceForFetch(name));
+ ASSERT_TRUE(nor1.get() != NULL);
+ // Again we'll succeed in fetching (meaning don't create), but the
+ // resource won't be written. Note that we do a non-blocking fetch here.
+ // An actual resource fetch does a blocking fetch that would end by
+ // stealing the creation lock; we don't want to steal the lock here.
+ EXPECT_TRUE(resource_manager_->FetchOutputResource(
+ nor1.get(), NULL, NULL, &message_handler_,
+ ResourceManager::kNeverBlock));
+ EXPECT_FALSE(nor1->IsWritten());
+ }
+ // Write some data
+ EXPECT_FALSE(ResourceManagerTestingPeer::HasHash(nor.get()));
+ EXPECT_FALSE(ResourceManagerTestingPeer::Generated(nor.get()));
+ EXPECT_TRUE(resource_manager_->Write(HttpStatus::kOK, contents, nor.get(),
+ origin_expire_time_ms,
+ &message_handler_));
+ EXPECT_TRUE(nor->IsWritten());
+ // Check that hash_ext() is correct.
+ ResourceNamer full_name;
+ EXPECT_TRUE(full_name.DecodeHashExt(nor->hash_ext()));
+ EXPECT_EQ("0", full_name.hash());
+ EXPECT_EQ("txt", full_name.ext());
+ // Retrieve the same NOR from the cache.
+ scoped_ptr<OutputResource> nor2(
+ resource_manager_->CreateOutputResourceWithPath(
+ url_prefix_, filter_prefix, name, &kContentTypeText,
+ &message_handler_));
+ ASSERT_TRUE(nor2.get() != NULL);
+ EXPECT_TRUE(ResourceManagerTestingPeer::HasHash(nor2.get()));
+ EXPECT_FALSE(ResourceManagerTestingPeer::Generated(nor2.get()));
+ EXPECT_FALSE(nor2->IsWritten());
+
+ // Fetch its contents and make sure they match
+ EXPECT_EQ(contents, FetchOutputResource(nor2.get()));
+
+ // Try asynchronously too
+ VerifyContentsCallback callback(contents);
+ resource_manager_->ReadAsync(nor2.get(), &callback, &message_handler_);
+ callback.AssertCalled();
+
+ // Grab the URL for later
+ EXPECT_TRUE(nor2->HasValidUrl());
+ std::string url = nor2->url();
+ EXPECT_LT(0, url.length());
+
+ // Now expire it from the HTTP cache. Since we don't know its hash, we
+ // cannot fetch it (even though the contents are still in the filesystem).
+ mock_timer()->advance_ms(2 * origin_expire_time_ms);
+ {
+ scoped_ptr<OutputResource> nor3(
+ resource_manager_->CreateOutputResourceWithPath(
+ url_prefix_, filter_prefix, name, &kContentTypeText,
+ &message_handler_));
+ EXPECT_FALSE(resource_manager_->FetchOutputResource(
+ nor3.get(), NULL, NULL, &message_handler_,
+ ResourceManager::kNeverBlock));
+ // Now nor3 has locked the resource for creation.
+ // We must destruct nor3 in order to unlock it again, since we
+ // have no intention of creating it.
+ }
+
+ RemoveUrlPrefix(&url);
+ ASSERT_TRUE(full_name.Decode(url));
+ EXPECT_EQ(content_type, full_name.ContentTypeFromExt());
+ EXPECT_EQ(filter_prefix, full_name.id());
+ EXPECT_EQ(name, full_name.name());
+
+ // But with the URL (which contains the hash), we can retrieve it
+ // from the http_cache.
+ // first cut off the "http://mysite{,.0,.1}/" from the front.
+ scoped_ptr<OutputResource> nor4(
+ resource_manager_->CreateOutputResourceForFetch(nor->url()));
+ EXPECT_EQ(nor->url(), nor4->url());
+ EXPECT_EQ(contents, FetchOutputResource(nor4.get()));
+
+ // If it's evicted from the http_cache, we can also retrieve it from the
+ // filesystem.
+ lru_cache_->Clear();
+ nor4.reset(resource_manager_->CreateOutputResourceForFetch(nor->url()));
+ EXPECT_EQ(nor->url(), nor4->url());
+ EXPECT_EQ(contents, FetchOutputResource(nor4.get()));
+ // This also works asynchronously.
+ lru_cache_->Clear();
+ VerifyContentsCallback callback2(contents);
+ resource_manager_->ReadAsync(nor4.get(), &callback2, &message_handler_);
+ callback2.AssertCalled();
+ }
+};
+
+TEST_F(ResourceManagerTest, TestNamed) {
+ TestNamed();
+}
+
+TEST_F(ResourceManagerTest, TestOutputInputUrl) {
+ std::string url = Encode("http://example.com/dir/123/",
+ "jm", "0", "orig", "js");
+ scoped_ptr<OutputResource> output_resource(
+ resource_manager_->CreateOutputResourceForFetch(url));
+ ASSERT_TRUE(output_resource.get());
+ scoped_ptr<Resource> input_resource(
+ resource_manager_->CreateInputResourceFromOutputResource(
+ resource_manager_->url_escaper(),
+ output_resource.get(),
+ &options_,
+ &message_handler_));
+ EXPECT_EQ("http://example.com/dir/123/orig", input_resource->url());
+}
+
+// TODO(jmaessen): re-enable after sharding works again.
+// class ResourceManagerShardedTest : public ResourceManagerTest {
+// protected:
+// ResourceManagerShardedTest() {
+// url_prefix_ ="http://mysite.%d/";
+// num_shards_ = 2;
+// }
+// virtual void RemoveUrlPrefix(std::string* url) {
+// // One of these two should match.
+// StringPiece prefix(url->data(), 16);
+// EXPECT_TRUE((prefix == "http://mysite.0/") ||
+// (prefix == "http://mysite.1/")) << *url;
+// // "%d" -> "0" (or "1") loses 1 char.
+// url->erase(0, url_prefix_.length() - 1);
+// }
+// };
+
+// TODO(jmaessen): re-enable after sharding works again.
+// TEST_F(ResourceManagerShardedTest, TestNamed) {
+// TestNamed();
+// }
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/resource_manager_test_base.cc b/trunk/src/net/instaweb/rewriter/resource_manager_test_base.cc
new file mode 100644
index 0000000..68cbd8e
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/resource_manager_test_base.cc
@@ -0,0 +1,93 @@
+// Copyright 2010 Google Inc. All Rights Reserved.
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+
+namespace net_instaweb {
+
+const char ResourceManagerTestBase::kTestData[] =
+ "/net/instaweb/rewriter/testdata/";
+
+// Test that a resource can be served from an new server that has not already
+// constructed it.
+void ResourceManagerTestBase::ServeResourceFromNewContext(
+ const std::string& resource_url,
+ RewriteOptions::Filter filter,
+ Hasher* hasher,
+ const StringPiece& expected_content) {
+
+ // New objects for the new server.
+ MemFileSystem other_file_system;
+ // other_lru_cache is owned by other_http_cache_.
+ LRUCache* other_lru_cache(new LRUCache(kCacheSize));
+ MockTimer* other_mock_timer = other_file_system.timer();
+ HTTPCache other_http_cache(other_lru_cache, other_mock_timer);
+ DomainLawyer other_domain_lawyer;
+ FileSystemLockManager other_lock_manager(
+ &other_file_system, other_mock_timer, &message_handler_);
+ WaitUrlAsyncFetcher wait_url_async_fetcher(&mock_url_fetcher_);
+ ResourceManager other_resource_manager(
+ file_prefix_, &other_file_system,
+ &filename_encoder_, &wait_url_async_fetcher, hasher,
+ &other_http_cache, &other_lock_manager);
+
+ SimpleStats stats;
+ RewriteDriver::Initialize(&stats);
+ other_resource_manager.set_statistics(&stats);
+
+ RewriteDriver other_rewrite_driver(&message_handler_, &other_file_system,
+ &wait_url_async_fetcher, other_options_);
+ other_rewrite_driver.SetResourceManager(&other_resource_manager);
+ other_options_.EnableFilter(filter);
+ other_rewrite_driver.AddFilters();
+
+ Variable* cached_resource_fetches =
+ stats.GetVariable(RewriteDriver::kResourceFetchesCached);
+ Variable* succeeded_filter_resource_fetches =
+ stats.GetVariable(RewriteDriver::kResourceFetchConstructSuccesses);
+ Variable* failed_filter_resource_fetches =
+ stats.GetVariable(RewriteDriver::kResourceFetchConstructFailures);
+
+ SimpleMetaData request_headers;
+ // TODO(sligocki): We should set default request headers.
+ SimpleMetaData response_headers;
+ std::string response_contents;
+ StringWriter response_writer(&response_contents);
+ DummyCallback callback(true);
+
+ // Check that we don't already have it in cache.
+ EXPECT_EQ(CacheInterface::kNotFound, other_http_cache.Query(resource_url));
+
+ // Initiate fetch.
+ EXPECT_EQ(true, other_rewrite_driver.FetchResource(
+ resource_url, request_headers, &response_headers, &response_writer,
+ &message_handler_, &callback));
+
+ // Content should not be set until we call the callback.
+ EXPECT_EQ(false, callback.done_);
+ EXPECT_EQ("", response_contents);
+
+ // After we call the callback, it should be correct.
+ wait_url_async_fetcher.CallCallbacks();
+ EXPECT_EQ(true, callback.done_);
+ EXPECT_EQ(expected_content, response_contents);
+ EXPECT_EQ(CacheInterface::kAvailable, other_http_cache.Query(resource_url));
+
+ // Check that stats say we took the construct resource path.
+ EXPECT_EQ(0, cached_resource_fetches->Get());
+ EXPECT_EQ(1, succeeded_filter_resource_fetches->Get());
+ EXPECT_EQ(0, failed_filter_resource_fetches->Get());
+}
+
+std::string ResourceManagerTestBase::Encode(
+ const StringPiece& path, const StringPiece& id, const StringPiece& hash,
+ const StringPiece& name, const StringPiece& ext) {
+ ResourceNamer namer;
+ namer.set_id(id);
+ namer.set_hash(hash);
+ namer.set_name(name);
+ namer.set_ext(ext);
+ return StrCat(path, namer.Encode());
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/resource_manager_testing_peer.h b/trunk/src/net/instaweb/rewriter/resource_manager_testing_peer.h
new file mode 100644
index 0000000..97d5011
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/resource_manager_testing_peer.h
@@ -0,0 +1,43 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: abliss@google.com (Adam Bliss)
+
+#ifndef NET_INSTAWEB_REWRITER_RESOURCE_MANAGER_TESTING_PEER_H_
+#define NET_INSTAWEB_REWRITER_RESOURCE_MANAGER_TESTING_PEER_H_
+
+#include "net/instaweb/rewriter/public/output_resource.h"
+
+namespace net_instaweb {
+
+class ResourceManagerTestingPeer {
+ public:
+ ResourceManagerTestingPeer() { }
+
+ static bool HasHash(const OutputResource* resource) {
+ return resource->has_hash();
+ }
+ static bool Generated(const OutputResource* resource) {
+ return resource->generated();
+ }
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(ResourceManagerTestingPeer);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_REWRITER_RESOURCE_MANAGER_TESTING_PEER_H_
diff --git a/trunk/src/net/instaweb/rewriter/resource_namer.cc b/trunk/src/net/instaweb/rewriter/resource_namer.cc
new file mode 100644
index 0000000..4903a23
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/resource_namer.cc
@@ -0,0 +1,186 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/resource_namer.h"
+
+#include <vector>
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/filename_encoder.h"
+#include "net/instaweb/util/public/string_hash.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// The format of all resource names is:
+//
+// ORIGINAL_NAME.ID.pagespeed.HASH.EXT
+//
+// "pagespeed" is what we'll call the system ID. Rationale:
+// 1. Any abbreviation of this will not be well known, e.g.
+// ps, mps (mod page speed), psa (page speed automatic)
+// and early reports from users indicate confusion over
+// the gibberish names in our resources.
+// 2. "pagespeed" is the family of products now, not just the
+// firebug plug in. Page Speed Automatic is the proper name for
+// the rewriting technology but it's longer, and "pagespeed" solves the
+// "WTF is this garbage in my URL" problem.
+// 3. "mod_pagespeed" is slightly longer if/when this technology
+// is ported to other servers then the "mod_" is less relevant.
+//
+// If you change this, or the structure of the encoded string,
+// you will also need to change:
+//
+// apache/install/system_test.sh
+//
+// Plus a few constants in _test.cc files.
+
+static const char kSystemId[] = "pagespeed";
+static const int kNumSegments = 5;
+static const char kSeparatorString[] = ".";
+static const char kSeparatorChar = kSeparatorString[0];
+
+bool TokenizeSegmentFromRight(StringPiece* src, std::string* dest) {
+ StringPiece::size_type pos = src->rfind(kSeparatorChar);
+ if (pos == StringPiece::npos) {
+ return false;
+ }
+ src->substr(pos + 1).CopyToString(dest);
+ *src = src->substr(0, pos);
+ return true;
+}
+
+} // namespace
+
+const int ResourceNamer::kOverhead = 4 + sizeof(kSystemId) - 1;
+
+bool ResourceNamer::Decode(const StringPiece& encoded_string) {
+ StringPiece src(encoded_string);
+ std::string system_id;
+ if (TokenizeSegmentFromRight(&src, &ext_) &&
+ TokenizeSegmentFromRight(&src, &hash_) &&
+ TokenizeSegmentFromRight(&src, &id_) &&
+ TokenizeSegmentFromRight(&src, &system_id) &&
+ (system_id == kSystemId)) {
+ src.CopyToString(&name_);
+ return true;
+ }
+ return LegacyDecode(encoded_string);
+}
+
+// TODO(jmarantz): validate that the 'id' is one of the filters that
+// were implemented as of Nov 2010. Also validate that the hash
+// code is a 32-char hex number.
+bool ResourceNamer::LegacyDecode(const StringPiece& encoded_string) {
+ bool ret = false;
+ // First check that this URL has a known extension type
+ if (NameExtensionToContentType(encoded_string) != NULL) {
+ std::vector<StringPiece> names;
+ SplitStringPieceToVector(encoded_string, kSeparatorString, &names, true);
+ if (names.size() == 4) {
+ names[1].CopyToString(&hash_);
+
+ // The legacy hash codes were all either 1-character (for tests) or
+ // 32 characters, all in hex.
+ if ((hash_.size() != 1) && (hash_.size() != 32)) {
+ return false;
+ }
+ for (int i = 0, n = hash_.size(); i < n; ++i) {
+ char ch = hash_[i];
+ if (!isdigit(ch)) {
+ ch = toupper(ch);
+ if ((ch < 'A') || (ch > 'F')) {
+ return false;
+ }
+ }
+ }
+
+ names[0].CopyToString(&id_);
+ names[2].CopyToString(&name_);
+ names[3].CopyToString(&ext_);
+ ret = true;
+ }
+ }
+ return ret;
+}
+
+// This is used for legacy compatibility as we transition to the grand new
+// world.
+std::string ResourceNamer::InternalEncode() const {
+ return StrCat(name_, kSeparatorString,
+ kSystemId, kSeparatorString,
+ id_, kSeparatorString,
+ StrCat(hash_, kSeparatorString, ext_));
+}
+
+// The current encoding assumes there are no dots in any of the components.
+// This restriction may be relaxed in the future, but check it aggressively
+// for now.
+std::string ResourceNamer::Encode() const {
+ CHECK_EQ(StringPiece::npos, id_.find(kSeparatorChar));
+ CHECK(!hash_.empty());
+ CHECK_EQ(StringPiece::npos, hash_.find(kSeparatorChar));
+ CHECK_EQ(StringPiece::npos, ext_.find(kSeparatorChar));
+ return InternalEncode();
+}
+
+std::string ResourceNamer::EncodeIdName() const {
+ CHECK(id_.find(kSeparatorChar) == StringPiece::npos);
+ return StrCat(id_, kSeparatorString, name_);
+}
+
+// Note: there is no need at this time to decode the name key.
+
+std::string ResourceNamer::EncodeHashExt() const {
+ CHECK_EQ(StringPiece::npos, hash_.find(kSeparatorChar));
+ CHECK_EQ(StringPiece::npos, ext_.find(kSeparatorChar));
+ return StrCat(hash_, kSeparatorString, ext_);
+}
+
+bool ResourceNamer::DecodeHashExt(const StringPiece& encoded_hash_ext) {
+ std::vector<StringPiece> names;
+ SplitStringPieceToVector(encoded_hash_ext, kSeparatorString, &names, true);
+ bool ret = (names.size() == 2);
+ if (ret) {
+ names[0].CopyToString(&hash_);
+ names[1].CopyToString(&ext_);
+ }
+ return ret;
+}
+
+size_t ResourceNamer::Hash() const {
+ size_t id_hash = HashString( id_.data(), id_.size());
+ size_t name_hash = HashString(name_.data(), name_.size());
+ size_t hash_hash = HashString(hash_.data(), hash_.size());
+ size_t ext_hash = HashString( ext_.data(), ext_.size());
+ return
+ JoinHash(JoinHash(JoinHash(id_hash, name_hash), hash_hash), ext_hash);
+}
+
+const ContentType* ResourceNamer::ContentTypeFromExt() const {
+ return NameExtensionToContentType(StrCat(".", ext_));
+}
+
+void ResourceNamer::CopyFrom(const ResourceNamer& other) {
+ other.id().CopyToString(&id_);
+ other.name().CopyToString(&name_);
+ other.hash().CopyToString(&hash_);
+ other.ext().CopyToString(&ext_);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/resource_namer_test.cc b/trunk/src/net/instaweb/rewriter/resource_namer_test.cc
new file mode 100644
index 0000000..74205ad
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/resource_namer_test.cc
@@ -0,0 +1,85 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/resource_namer.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace net_instaweb {
+
+namespace {
+
+class ResourceNamerTest : public testing::Test {
+ protected:
+ ResourceNamerTest() { }
+
+ ResourceNamer full_name_;
+};
+
+TEST_F(ResourceNamerTest, TestEncode) {
+ // Stand up a minimal resource manager that only has the
+ // resources we should need to encode.
+ full_name_.set_id("id");
+ full_name_.set_name("name.ext.as.many.as.I.like");
+ full_name_.set_hash("hash");
+ full_name_.set_ext("ext");
+ EXPECT_EQ(std::string("name.ext.as.many.as.I.like.pagespeed.id.hash.ext"),
+ full_name_.Encode());
+ EXPECT_EQ(std::string("id.name.ext.as.many.as.I.like"),
+ full_name_.EncodeIdName());
+ EXPECT_EQ(std::string("hash.ext"), full_name_.EncodeHashExt());
+}
+
+TEST_F(ResourceNamerTest, TestDecode) {
+ EXPECT_TRUE(full_name_.Decode(
+ "name.ext.as.many.as.I.like.pagespeed.id.hash.ext"));
+ EXPECT_EQ("id", full_name_.id());
+ EXPECT_EQ("name.ext.as.many.as.I.like", full_name_.name());
+ EXPECT_EQ("hash", full_name_.hash());
+ EXPECT_EQ("ext", full_name_.ext());
+}
+
+TEST_F(ResourceNamerTest, TestDecodeTooMany) {
+ EXPECT_TRUE(full_name_.Decode("name.extra_dot.pagespeed.id.hash.ext"));
+ EXPECT_FALSE(full_name_.DecodeHashExt("id.hash.ext"));
+}
+
+TEST_F(ResourceNamerTest, TestDecodeNotEnough) {
+ EXPECT_FALSE(full_name_.Decode("id.name.hash"));
+ EXPECT_FALSE(full_name_.DecodeHashExt("ext"));
+}
+
+TEST_F(ResourceNamerTest, TestDecodeHashExt) {
+ EXPECT_TRUE(full_name_.DecodeHashExt("hash.ext"));
+ EXPECT_EQ("", full_name_.id());
+ EXPECT_EQ("", full_name_.name());
+ EXPECT_EQ("hash", full_name_.hash());
+ EXPECT_EQ("ext", full_name_.ext());
+}
+
+TEST_F(ResourceNamerTest, TestLegacyDecode) {
+ EXPECT_TRUE(full_name_.Decode("id.0123456789abcdef0123456789ABCDEF.name.js"));
+ EXPECT_EQ("id", full_name_.id());
+ EXPECT_EQ("name", full_name_.name());
+ EXPECT_EQ("0123456789abcdef0123456789ABCDEF", full_name_.hash());
+ EXPECT_EQ("js", full_name_.ext());
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/resource_tag_scanner.cc b/trunk/src/net/instaweb/rewriter/resource_tag_scanner.cc
new file mode 100644
index 0000000..2f6cbe9
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/resource_tag_scanner.cc
@@ -0,0 +1,42 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/resource_tag_scanner.h"
+
+namespace net_instaweb {
+
+// Finds resource references.
+ResourceTagScanner::ResourceTagScanner(HtmlParse* html_parse)
+ : css_tag_scanner_(html_parse),
+ img_tag_scanner_(html_parse),
+ script_tag_scanner_(html_parse) {
+}
+
+HtmlElement::Attribute* ResourceTagScanner::ScanElement(HtmlElement* element) {
+ HtmlElement::Attribute* attr = NULL;
+ const char* media;
+ if (!css_tag_scanner_.ParseCssElement(element, &attr, &media)) {
+ attr = img_tag_scanner_.ParseImgElement(element);
+ if (attr == NULL) {
+ script_tag_scanner_.ParseScriptElement(element, &attr);
+ }
+ }
+ return attr;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/resource_tag_scanner_test.cc b/trunk/src/net/instaweb/rewriter/resource_tag_scanner_test.cc
new file mode 100644
index 0000000..e99c464
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/resource_tag_scanner_test.cc
@@ -0,0 +1,80 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/htmlparse/public/html_parse_test_base.h"
+#include "net/instaweb/rewriter/public/resource_tag_scanner.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace net_instaweb {
+
+class ResourceTagScannerTest : public HtmlParseTestBase {
+ protected:
+ ResourceTagScannerTest() {
+ }
+
+ virtual bool AddBody() const { return true; }
+
+ // Helper class to collect all external resources.
+ class ResourceCollector : public EmptyHtmlFilter {
+ public:
+ ResourceCollector(HtmlParse* html_parse, StringVector* resources)
+ : resources_(resources),
+ resource_tag_scanner_(html_parse) {
+ }
+
+ virtual void StartElement(HtmlElement* element) {
+ HtmlElement::Attribute* src =
+ resource_tag_scanner_.ScanElement(element);
+ if (src != NULL) {
+ resources_->push_back(src->value());
+ }
+ }
+
+ virtual const char* Name() const { return "ResourceCollector"; }
+
+ private:
+ StringVector* resources_;
+ ResourceTagScanner resource_tag_scanner_;
+
+ DISALLOW_COPY_AND_ASSIGN(ResourceCollector);
+ };
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(ResourceTagScannerTest);
+};
+
+TEST_F(ResourceTagScannerTest, FindTags) {
+ StringVector resources;
+ ResourceCollector collector(&html_parse_, &resources);
+ html_parse_.AddFilter(&collector);
+ ValidateNoChanges(
+ "simple_script",
+ "<script src='myscript.js'></script>\n"
+ "<img src=\"image.jpg\"/>\n"
+ "<link rel=\"stylesheet\" type=\"text/css\" href=\"nomedia.css\">\n"
+ "<link rel=stylesheet type=text/css href=media.css media=print>");
+ ASSERT_EQ(static_cast<size_t>(4), resources.size());
+ EXPECT_EQ(std::string("myscript.js"), resources[0]);
+ EXPECT_EQ(std::string("image.jpg"), resources[1]);
+ EXPECT_EQ(std::string("nomedia.css"), resources[2]);
+ EXPECT_EQ(std::string("media.css"), resources[3]);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/rewrite_driver.cc b/trunk/src/net/instaweb/rewriter/rewrite_driver.cc
new file mode 100644
index 0000000..d385207
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/rewrite_driver.cc
@@ -0,0 +1,499 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+
+#include <vector>
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/htmlparse/public/html_writer_filter.h"
+#include "net/instaweb/rewriter/public/add_head_filter.h"
+#include "net/instaweb/rewriter/public/add_instrumentation_filter.h"
+#include "net/instaweb/rewriter/public/base_tag_filter.h"
+#include "net/instaweb/rewriter/public/cache_extender.h"
+#include "net/instaweb/rewriter/public/collapse_whitespace_filter.h"
+#include "net/instaweb/rewriter/public/css_combine_filter.h"
+#include "net/instaweb/rewriter/public/css_filter.h"
+#include "net/instaweb/rewriter/public/css_inline_filter.h"
+#include "net/instaweb/rewriter/public/css_move_to_head_filter.h"
+#include "net/instaweb/rewriter/public/css_outline_filter.h"
+#include "net/instaweb/rewriter/public/elide_attributes_filter.h"
+#include "net/instaweb/rewriter/public/html_attribute_quote_removal.h"
+#include "net/instaweb/rewriter/public/img_rewrite_filter.h"
+#include "net/instaweb/rewriter/public/javascript_filter.h"
+#include "net/instaweb/rewriter/public/js_inline_filter.h"
+#include "net/instaweb/rewriter/public/js_outline_filter.h"
+#include "net/instaweb/rewriter/public/output_resource.h"
+#include "net/instaweb/rewriter/public/remove_comments_filter.h"
+#include "net/instaweb/rewriter/public/resource.h"
+#include "net/instaweb/rewriter/public/resource_namer.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/strip_scripts_filter.h"
+#include "net/instaweb/rewriter/public/url_left_trim_filter.h"
+#include "net/instaweb/util/public/stl_util.h"
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/filename_encoder.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/statistics.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+
+namespace net_instaweb {
+
+// RewriteFilter prefixes
+const char RewriteDriver::kCssCombinerId[] = "cc";
+const char RewriteDriver::kCssFilterId[] = "cf";
+const char RewriteDriver::kCacheExtenderId[] = "ce";
+const char RewriteDriver::kImageCompressionId[] = "ic";
+const char RewriteDriver::kJavascriptMinId[] = "jm";
+
+// TODO(jmarantz): Simplify the interface so we can just use
+// asynchronous fetchers, employing FakeUrlAsyncFetcher as needed
+// for running functional regression-tests where we don't mind blocking
+// behavior.
+RewriteDriver::RewriteDriver(MessageHandler* message_handler,
+ FileSystem* file_system,
+ UrlAsyncFetcher* url_async_fetcher,
+ const RewriteOptions& options)
+ : html_parse_(message_handler),
+ file_system_(file_system),
+ url_async_fetcher_(url_async_fetcher),
+ resource_manager_(NULL),
+ add_instrumentation_filter_(NULL),
+ cached_resource_fetches_(NULL),
+ succeeded_filter_resource_fetches_(NULL),
+ failed_filter_resource_fetches_(NULL),
+ options_(options) {
+}
+
+RewriteDriver::~RewriteDriver() {
+ STLDeleteElements(&filters_);
+}
+
+const char* RewriteDriver::kPassThroughRequestAttributes[3] = {
+ HttpAttributes::kIfModifiedSince,
+ HttpAttributes::kReferer,
+ HttpAttributes::kUserAgent
+};
+
+// names for Statistics variables.
+const char RewriteDriver::kResourceFetchesCached[] = "resource_fetches_cached";
+const char RewriteDriver::kResourceFetchConstructSuccesses[] =
+ "resource_fetch_construct_successes";
+const char RewriteDriver::kResourceFetchConstructFailures[] =
+ "resource_fetch_construct_failures";
+
+void RewriteDriver::Initialize(Statistics* statistics) {
+ if (statistics != NULL) {
+ statistics->AddVariable(kResourceFetchesCached);
+ statistics->AddVariable(kResourceFetchConstructSuccesses);
+ statistics->AddVariable(kResourceFetchConstructFailures);
+
+ // TODO(jmarantz): Make all of these work with null statistics so that
+ // they could mdo other required static initializations if desired
+ // without having to edit code to this method.
+ AddInstrumentationFilter::Initialize(statistics);
+ CacheExtender::Initialize(statistics);
+ CssCombineFilter::Initialize(statistics);
+ CssMoveToHeadFilter::Initialize(statistics);
+ ImgRewriteFilter::Initialize(statistics);
+ JavascriptFilter::Initialize(statistics);
+ ResourceManager::Initialize(statistics);
+ UrlLeftTrimFilter::Initialize(statistics);
+ }
+ CssFilter::Initialize(statistics);
+}
+
+void RewriteDriver::SetResourceManager(ResourceManager* resource_manager) {
+ resource_manager_ = resource_manager;
+ html_parse_.set_timer(resource_manager->timer());
+}
+
+// If flag starts with key (a string ending in "="), call m on the remainder of
+// flag (the piece after the "="). Always returns true if the key matched; m is
+// free to complain about invalid input using html_parse_->message_handler().
+bool RewriteDriver::ParseKeyString(const StringPiece& key, SetStringMethod m,
+ const std::string& flag) {
+ if (flag.rfind(key.data(), 0, key.size()) == 0) {
+ StringPiece sp(flag);
+ (this->*m)(flag.substr(key.size()));
+ return true;
+ } else {
+ return false;
+ }
+}
+
+// If flag starts with key (a string ending in "="), convert rest of flag after
+// the "=" to Int64, and call m on it. Always returns true if the key matched;
+// m is free to complain about invalid input using
+// html_parse_->message_handler() (failure to parse a number does so and never
+// calls m).
+bool RewriteDriver::ParseKeyInt64(const StringPiece& key, SetInt64Method m,
+ const std::string& flag) {
+ if (flag.rfind(key.data(), 0, key.size()) == 0) {
+ std::string str_value = flag.substr(key.size());
+ int64 value;
+ if (StringToInt64(str_value, &value)) {
+ (this->*m)(value);
+ } else {
+ html_parse_.message_handler()->Message(
+ kError, "'%s': ignoring value (should have been int64) after %s",
+ flag.c_str(), key.as_string().c_str());
+ }
+ return true;
+ } else {
+ return false;
+ }
+}
+
+void RewriteDriver::AddFilters() {
+ CHECK(html_writer_filter_ == NULL);
+
+ // Add the rewriting filters to the map unconditionally -- we may
+ // need the to process resource requests due to a query-specific
+ // 'rewriters' specification. We still use the passed-in options
+ // to determine whether they get added to the html parse filter chain.
+ RegisterRewriteFilter(new CssCombineFilter(this, kCssCombinerId));
+ RegisterRewriteFilter(new CssFilter(this, kCssFilterId));
+ RegisterRewriteFilter(new JavascriptFilter(this, kJavascriptMinId));
+ RegisterRewriteFilter(
+ new ImgRewriteFilter(
+ this,
+ options_.Enabled(RewriteOptions::kDebugLogImgTags),
+ options_.Enabled(RewriteOptions::kInsertImgDimensions),
+ kImageCompressionId,
+ options_.img_inline_max_bytes(),
+ options_.img_max_rewrites_at_once()));
+ RegisterRewriteFilter(new CacheExtender(this, kCacheExtenderId));
+
+ // This function defines the order that filters are run. We document
+ // in pagespeed.conf.template that the order specified in the conf
+ // file does not matter, but we give the filters there in the order
+ // they are actually applied, for the benefit of the understanding
+ // of the site owner. So if you change that here, change it in
+ // install/common/pagespeed.conf.template as well.
+
+ // Now process boolean options, which may include propagating non-boolean
+ // and boolean parameter settings to filters.
+ if (options_.Enabled(RewriteOptions::kAddHead) ||
+ options_.Enabled(RewriteOptions::kCombineHeads) ||
+ options_.Enabled(RewriteOptions::kAddBaseTag) ||
+ options_.Enabled(RewriteOptions::kMoveCssToHead) ||
+ options_.Enabled(RewriteOptions::kAddInstrumentation)) {
+ // Adds a filter that adds a 'head' section to html documents if
+ // none found prior to the body.
+ AddFilter(new AddHeadFilter(
+ &html_parse_, options_.Enabled(RewriteOptions::kCombineHeads)));
+ }
+ if (options_.Enabled(RewriteOptions::kAddBaseTag)) {
+ // Adds a filter that establishes a base tag for the HTML document.
+ // This is required when implementing a proxy server. The base
+ // tag used can be changed for every request with SetBaseUrl.
+ // Adding the base-tag filter will establish the AddHeadFilter
+ // if needed.
+ base_tag_filter_.reset(new BaseTagFilter(&html_parse_));
+ html_parse_.AddFilter(base_tag_filter_.get());
+ }
+ if (options_.Enabled(RewriteOptions::kStripScripts)) {
+ // Experimental filter that blindly scripts all strips from a page.
+ AddFilter(new StripScriptsFilter(&html_parse_));
+ }
+ if (options_.Enabled(RewriteOptions::kOutlineCss)) {
+ // Cut out inlined styles and make them into external resources.
+ // This can only be called once and requires a resource_manager to be set.
+ CHECK(resource_manager_ != NULL);
+ CssOutlineFilter* css_outline_filter = new CssOutlineFilter(this);
+ AddFilter(css_outline_filter);
+ }
+ if (options_.Enabled(RewriteOptions::kOutlineJavascript)) {
+ // Cut out inlined scripts and make them into external resources.
+ // This can only be called once and requires a resource_manager to be set.
+ CHECK(resource_manager_ != NULL);
+ JsOutlineFilter* js_outline_filter = new JsOutlineFilter(this);
+ AddFilter(js_outline_filter);
+ }
+ if (options_.Enabled(RewriteOptions::kMoveCssToHead)) {
+ // It's good to move CSS links to the head prior to running CSS combine,
+ // which only combines CSS links that are already in the head.
+ AddFilter(new CssMoveToHeadFilter(&html_parse_, statistics()));
+ }
+ if (options_.Enabled(RewriteOptions::kCombineCss)) {
+ // Combine external CSS resources after we've outlined them.
+ // CSS files in html document. This can only be called
+ // once and requires a resource_manager to be set.
+ EnableRewriteFilter(kCssCombinerId);
+ }
+ if (options_.Enabled(RewriteOptions::kRewriteCss)) {
+ EnableRewriteFilter(kCssFilterId);
+ }
+ if (options_.Enabled(RewriteOptions::kRewriteJavascript)) {
+ // Rewrite (minify etc.) JavaScript code to reduce time to first
+ // interaction.
+ EnableRewriteFilter(kJavascriptMinId);
+ }
+ if (options_.Enabled(RewriteOptions::kInlineCss)) {
+ // Inline small CSS files. Give CssCombineFilter and CSS minification a
+ // chance to run before we decide what counts as "small".
+ CHECK(resource_manager_ != NULL);
+ AddFilter(new CssInlineFilter(this));
+ }
+ if (options_.Enabled(RewriteOptions::kInlineJavascript)) {
+ // Inline small Javascript files. Give JS minification a chance to run
+ // before we decide what counts as "small".
+ CHECK(resource_manager_ != NULL);
+ AddFilter(new JsInlineFilter(this));
+ }
+ if (options_.Enabled(RewriteOptions::kRewriteImages)) {
+ EnableRewriteFilter(kImageCompressionId);
+ }
+ if (options_.Enabled(RewriteOptions::kRemoveComments)) {
+ AddFilter(new RemoveCommentsFilter(&html_parse_));
+ }
+ if (options_.Enabled(RewriteOptions::kCollapseWhitespace)) {
+ // Remove excess whitespace in HTML
+ AddFilter(new CollapseWhitespaceFilter(&html_parse_));
+ }
+ if (options_.Enabled(RewriteOptions::kElideAttributes)) {
+ // Remove HTML element attribute values where
+ // http://www.w3.org/TR/html4/loose.dtd says that the name is all
+ // that's necessary
+ AddFilter(new ElideAttributesFilter(&html_parse_));
+ }
+ if (options_.Enabled(RewriteOptions::kExtendCache)) {
+ // Extend the cache lifetime of resources.
+ EnableRewriteFilter(kCacheExtenderId);
+ }
+ if (options_.Enabled(RewriteOptions::kLeftTrimUrls)) {
+ // Trim extraneous prefixes from urls in attribute values.
+ // Happens before RemoveQuotes but after everything else. Note:
+ // we Must left trim urls BEFORE quote removal.
+ left_trim_filter_.reset(
+ new UrlLeftTrimFilter(&html_parse_,
+ resource_manager_->statistics()));
+ html_parse_.AddFilter(left_trim_filter_.get());
+ }
+ if (options_.Enabled(RewriteOptions::kRemoveQuotes)) {
+ // Remove extraneous quotes from html attributes. Does this save
+ // enough bytes to be worth it after compression? If we do it
+ // everywhere it seems to give a small savings.
+ AddFilter(new HtmlAttributeQuoteRemoval(&html_parse_));
+ }
+ if (options_.Enabled(RewriteOptions::kAddInstrumentation)) {
+ // Inject javascript to instrument loading-time.
+ add_instrumentation_filter_ =
+ new AddInstrumentationFilter(&html_parse_,
+ options_.beacon_url(),
+ resource_manager_->statistics());
+ AddFilter(add_instrumentation_filter_);
+ }
+ // NOTE(abliss): Adding a new filter? Does it export any statistics? If it
+ // doesn't, it probably should. If it does, be sure to add it to the
+ // Initialize() function above or it will break under Apache!
+}
+
+void RewriteDriver::SetBaseUrl(const StringPiece& base) {
+ if (base_tag_filter_ != NULL) {
+ base_tag_filter_->set_base_url(base);
+ }
+}
+
+void RewriteDriver::AddFilter(HtmlFilter* filter) {
+ filters_.push_back(filter);
+ html_parse_.AddFilter(filter);
+}
+
+void RewriteDriver::EnableRewriteFilter(const char* id) {
+ RewriteFilter* filter = resource_filter_map_[id];
+ CHECK(filter);
+ html_parse_.AddFilter(filter);
+}
+
+void RewriteDriver::RegisterRewriteFilter(RewriteFilter* filter) {
+ // Track resource_fetches if we care about statistics. Note that
+ // the statistics are owned by the resource manager, which generally
+ // should be set up prior to the rewrite_driver.
+ //
+ // TODO(sligocki): It'd be nice to get this into the constructor.
+ Statistics* stats = statistics();
+ if ((stats != NULL) && (cached_resource_fetches_ == NULL)) {
+ cached_resource_fetches_ = stats->GetVariable(kResourceFetchesCached);
+ succeeded_filter_resource_fetches_ =
+ stats->GetVariable(kResourceFetchConstructSuccesses);
+ failed_filter_resource_fetches_ =
+ stats->GetVariable(kResourceFetchConstructFailures);
+ }
+ resource_filter_map_[filter->id()] = filter;
+ filters_.push_back(filter);
+}
+
+void RewriteDriver::SetWriter(Writer* writer) {
+ if (html_writer_filter_ == NULL) {
+ html_writer_filter_.reset(new HtmlWriterFilter(&html_parse_));
+ html_parse_.AddFilter(html_writer_filter_.get());
+ }
+ html_writer_filter_->set_writer(writer);
+}
+
+Statistics* RewriteDriver::statistics() const {
+ return (resource_manager_ == NULL) ? NULL : resource_manager_->statistics();
+}
+
+namespace {
+
+// Wraps an async fetcher callback, in order to delete the output
+// resource.
+class ResourceDeleterCallback : public UrlAsyncFetcher::Callback {
+ public:
+ ResourceDeleterCallback(OutputResource* output_resource,
+ UrlAsyncFetcher::Callback* callback,
+ HTTPCache* http_cache,
+ MessageHandler* message_handler)
+ : output_resource_(output_resource),
+ callback_(callback),
+ http_cache_(http_cache),
+ message_handler_(message_handler) {
+ }
+
+ virtual void Done(bool status) {
+ callback_->Done(status);
+ // Filters should generally write their output to the OutputResource,
+ // in which case when they are done we can insert it into the cache.
+ // However, not all filters do this yet notably img_rewrite_filter,
+ // so check for ContentsValid().
+ if (status && output_resource_->ContentsValid()) {
+ http_cache_->Put(output_resource_->url(),
+ *output_resource_->metadata(),
+ output_resource_->contents(),
+ message_handler_);
+ }
+ delete this;
+ }
+
+ private:
+ scoped_ptr<OutputResource> output_resource_;
+ UrlAsyncFetcher::Callback* callback_;
+ HTTPCache* http_cache_;
+ MessageHandler* message_handler_;
+
+ DISALLOW_COPY_AND_ASSIGN(ResourceDeleterCallback);
+};
+
+} // namespace
+
+OutputResource* RewriteDriver::DecodeOutputResource(
+ const StringPiece& url,
+ RewriteFilter** filter) {
+ // Note that this does permission checking and parsing of the url, but doesn't
+ // actually fetch any data until we specifically ask it to.
+ OutputResource* output_resource =
+ resource_manager_->CreateOutputResourceForFetch(url);
+
+ // If the resource name was ill-formed or unrecognized, we reject the request
+ // so it can be passed along.
+ if (output_resource != NULL) {
+ // For now let's reject as mal-formed if the id string is not
+ // in the rewrite drivers.
+ // TODO(jmarantz): it might be better to 'handle' requests with known
+ // IDs even if that filter is not enabled, rather rejecting the request.
+ // TODO(jmarantz): consider query-specific rewrites. We may need to
+ // enable filters for this driver based on the referrer.
+ StringPiece id = output_resource->filter_prefix();
+ StringFilterMap::iterator p = resource_filter_map_.find(
+ std::string(id.data(), id.size()));
+
+ // OutlineFilter is special because it's not a RewriteFilter -- it's
+ // just an HtmlFilter, but it does encode rewritten resources that
+ // must be served from the cache.
+ //
+ // TODO(jmarantz): figure out a better way to refactor this.
+ // TODO(jmarantz): add a unit-test to show serving outline-filter resources.
+ if (p != resource_filter_map_.end()) {
+ *filter = p->second;
+ } else if (((id != CssOutlineFilter::kFilterId) &&
+ (id != JsOutlineFilter::kFilterId)) ||
+ (output_resource->type() == NULL)) {
+ delete output_resource;
+ output_resource = NULL;
+ }
+ }
+ return output_resource;
+}
+
+bool RewriteDriver::FetchResource(
+ const StringPiece& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* writer,
+ MessageHandler* message_handler,
+ UrlAsyncFetcher::Callback* callback) {
+ bool queued = false;
+ bool handled = false;
+
+ // Note that this does permission checking and parsing of the url, but doesn't
+ // actually fetch any data until we specifically ask it to.
+ RewriteFilter* filter = NULL;
+ OutputResource* output_resource = DecodeOutputResource(url, &filter);
+
+ if (output_resource != NULL) {
+ handled = true;
+ callback = new ResourceDeleterCallback(output_resource, callback,
+ resource_manager_->http_cache(),
+ message_handler);
+ // None of our resources ever change -- the hash of the content is embedded
+ // in the filename. This is why we serve them with very long cache
+ // lifetimes. However, when the user presses Reload, the browser may
+ // attempt to validate that the cached copy is still fresh by sending a GET
+ // with an If-Modified-Since header. If this header is present, we should
+ // return a 304 Not Modified, since any representation of the resource
+ // that's in the browser's cache must be correct.
+ CharStarVector values;
+ if (request_headers.Lookup(HttpAttributes::kIfModifiedSince, &values)) {
+ response_headers->SetStatusAndReason(HttpStatus::kNotModified);
+ callback->Done(true);
+ queued = true;
+ } else if (resource_manager_->FetchOutputResource(
+ output_resource, writer, response_headers, message_handler,
+ ResourceManager::kMayBlock)) {
+ callback->Done(true);
+ queued = true;
+ if (cached_resource_fetches_ != NULL) {
+ cached_resource_fetches_->Add(1);
+ }
+ } else if (filter != NULL) {
+ queued = filter->Fetch(output_resource, writer,
+ request_headers, response_headers,
+ message_handler, callback);
+ if (queued) {
+ if (succeeded_filter_resource_fetches_ != NULL) {
+ succeeded_filter_resource_fetches_->Add(1);
+ }
+ } else {
+ if (failed_filter_resource_fetches_ != NULL) {
+ failed_filter_resource_fetches_->Add(1);
+ }
+ }
+ }
+ }
+ if (!queued && handled) {
+ // If we got here, we were asked to decode a resource for which we have
+ // no filter.
+ callback->Done(false);
+ }
+ return handled;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/rewrite_driver_factory.cc b/trunk/src/net/instaweb/rewriter/rewrite_driver_factory.cc
new file mode 100644
index 0000000..708c492
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/rewrite_driver_factory.cc
@@ -0,0 +1,390 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/rewriter/public/rewrite_driver_factory.h"
+
+#include "base/logging.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+#include "net/instaweb/util/public/abstract_mutex.h"
+#include "net/instaweb/util/public/cache_url_async_fetcher.h"
+#include "net/instaweb/util/public/cache_url_fetcher.h"
+#include "net/instaweb/util/public/fake_url_async_fetcher.h"
+#include "net/instaweb/util/public/filename_encoder.h"
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/file_system_lock_manager.h"
+#include "net/instaweb/util/public/hasher.h"
+#include "net/instaweb/util/public/http_cache.h"
+#include "net/instaweb/util/public/http_dump_url_fetcher.h"
+#include "net/instaweb/util/public/http_dump_url_writer.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/named_lock_manager.h"
+#include "net/instaweb/util/public/statistics.h"
+#include "net/instaweb/util/public/stl_util.h"
+#include "net/instaweb/util/public/timer.h"
+
+namespace net_instaweb {
+
+const char kInstawebResource404Count[] = "resource_404_count";
+const char kInstawebSlurp404Count[] = "slurp_404_count";
+
+RewriteDriverFactory::RewriteDriverFactory()
+ : url_fetcher_(NULL),
+ url_async_fetcher_(NULL),
+ html_parse_(NULL),
+ filename_prefix_(""),
+ force_caching_(false),
+ slurp_read_only_(false),
+ slurp_print_urls_(false),
+ resource_404_count_(NULL),
+ slurp_404_count_(NULL) {
+}
+
+RewriteDriverFactory::~RewriteDriverFactory() {
+ ShutDown();
+}
+
+void RewriteDriverFactory::set_html_parse_message_handler(
+ MessageHandler* message_handler) {
+ html_parse_message_handler_.reset(message_handler);
+}
+
+void RewriteDriverFactory::set_message_handler(
+ MessageHandler* message_handler) {
+ message_handler_.reset(message_handler);
+}
+
+bool RewriteDriverFactory::FetchersComputed() const {
+ return (url_fetcher_ != NULL) || (url_async_fetcher_ != NULL);
+}
+
+void RewriteDriverFactory::set_slurp_directory(const StringPiece& dir) {
+ CHECK(!FetchersComputed())
+ << "Cannot call set_slurp_directory "
+ << " after ComputeUrl*Fetcher has been called";
+ dir.CopyToString(&slurp_directory_);
+}
+
+void RewriteDriverFactory::set_slurp_read_only(bool read_only) {
+ CHECK(!FetchersComputed())
+ << "Cannot call set_slurp_read_only "
+ << " after ComputeUrl*Fetcher has been called";
+ slurp_read_only_ = read_only;
+}
+
+void RewriteDriverFactory::set_slurp_print_urls(bool print_urls) {
+ CHECK(!FetchersComputed())
+ << "Cannot call set_slurp_print_urls "
+ << " after ComputeUrl*Fetcher has been called";
+ slurp_print_urls_ = print_urls;
+}
+
+void RewriteDriverFactory::set_file_system(FileSystem* file_system) {
+ file_system_.reset(file_system);
+}
+
+// TODO(jmarantz): Change this to set_base_url_fetcher
+void RewriteDriverFactory::set_base_url_fetcher(UrlFetcher* url_fetcher) {
+ CHECK(!FetchersComputed())
+ << "Cannot call set_base_url_fetcher "
+ << " after ComputeUrl*Fetcher has been called";
+ CHECK(base_url_async_fetcher_.get() == NULL)
+ << "Only call one of set_base_url_fetcher and set_base_url_async_fetcher";
+ base_url_fetcher_.reset(url_fetcher);
+}
+
+void RewriteDriverFactory::set_base_url_async_fetcher(
+ UrlAsyncFetcher* url_async_fetcher) {
+ CHECK(!FetchersComputed())
+ << "Cannot call set_base_url_fetcher "
+ << " after ComputeUrl*Fetcher has been called";
+ CHECK(base_url_fetcher_.get() == NULL)
+ << "Only call one of set_base_url_fetcher and set_base_url_async_fetcher";
+ base_url_async_fetcher_.reset(url_async_fetcher);
+}
+
+void RewriteDriverFactory::set_hasher(Hasher* hasher) {
+ hasher_.reset(hasher);
+}
+
+void RewriteDriverFactory::set_timer(Timer* timer) {
+ timer_.reset(timer);
+}
+
+void RewriteDriverFactory::set_filename_encoder(FilenameEncoder* e) {
+ filename_encoder_.reset(e);
+}
+
+MessageHandler* RewriteDriverFactory::html_parse_message_handler() {
+ if (html_parse_message_handler_ == NULL) {
+ html_parse_message_handler_.reset(DefaultHtmlParseMessageHandler());
+ }
+ return html_parse_message_handler_.get();
+}
+
+MessageHandler* RewriteDriverFactory::message_handler() {
+ if (message_handler_ == NULL) {
+ message_handler_.reset(DefaultMessageHandler());
+ }
+ return message_handler_.get();
+}
+
+FileSystem* RewriteDriverFactory::file_system() {
+ if (file_system_ == NULL) {
+ file_system_.reset(DefaultFileSystem());
+ }
+ return file_system_.get();
+}
+
+Timer* RewriteDriverFactory::timer() {
+ if (timer_ == NULL) {
+ timer_.reset(DefaultTimer());
+ }
+ return timer_.get();
+}
+
+Hasher* RewriteDriverFactory::hasher() {
+ if (hasher_ == NULL) {
+ hasher_.reset(NewHasher());
+ }
+ return hasher_.get();
+}
+
+FilenameEncoder* RewriteDriverFactory::filename_encoder() {
+ if (filename_encoder_ == NULL) {
+ filename_encoder_.reset(new FilenameEncoder);
+ }
+ return filename_encoder_.get();
+}
+
+NamedLockManager* RewriteDriverFactory::lock_manager() {
+ if (lock_manager_ == NULL) {
+ lock_manager_.reset(
+ new FileSystemLockManager(file_system(), timer(), message_handler()));
+ }
+ return lock_manager_.get();
+}
+
+bool RewriteDriverFactory::set_filename_prefix(StringPiece p) {
+ p.CopyToString(&filename_prefix_);
+ if (!file_system()->RecursivelyMakeDir(filename_prefix_, message_handler())) {
+ message_handler()->FatalError(
+ filename_prefix_.c_str(), 0,
+ "Directory does not exist and cannot be created");
+ return false;
+ }
+ return true;
+}
+
+StringPiece RewriteDriverFactory::filename_prefix() {
+ return filename_prefix_;
+}
+
+HTTPCache* RewriteDriverFactory::http_cache() {
+ if (http_cache_ == NULL) {
+ CacheInterface* cache = DefaultCacheInterface();
+ http_cache_.reset(new HTTPCache(cache, timer()));
+ http_cache_->set_force_caching(force_caching_);
+ }
+ return http_cache_.get();
+}
+
+ResourceManager* RewriteDriverFactory::ComputeResourceManager() {
+ if (resource_manager_ == NULL) {
+ CHECK(!filename_prefix_.empty())
+ << "Must specify --filename_prefix or call "
+ << "RewriteDriverFactory::set_filename_prefix.";
+ resource_manager_.reset(new ResourceManager(
+ filename_prefix_, file_system(), filename_encoder(),
+ ComputeUrlAsyncFetcher(), hasher(),
+ http_cache(), lock_manager()));
+ resource_manager_->set_store_outputs_in_file_system(
+ ShouldWriteResourcesToFileSystem());
+ }
+ return resource_manager_.get();
+}
+
+// TODO(jmaessen): Note that we *could* re-structure the
+// rewrite_driver freelist code as follows: Keep a
+// std::vector<RewriteDriver*> of all rewrite drivers. Have each
+// driver hold its index in the vector (as a number or iterator).
+// Keep index of first in use. To free, swap with first in use,
+// adjusting indexes, and increment first in use. To allocate,
+// decrement first in use and return that driver. If first in use was
+// 0, allocate a fresh driver and push it.
+
+RewriteDriver* RewriteDriverFactory::NewCustomRewriteDriver(
+ const RewriteOptions& options) {
+ RewriteDriver* rewrite_driver = new RewriteDriver(
+ message_handler(), file_system(), ComputeUrlAsyncFetcher(), options);
+ rewrite_driver->SetResourceManager(ComputeResourceManager());
+ AddPlatformSpecificRewritePasses(rewrite_driver);
+ rewrite_driver->AddFilters();
+ return rewrite_driver;
+}
+
+
+RewriteDriver* RewriteDriverFactory::NewRewriteDriver() {
+ ScopedMutex lock(rewrite_drivers_mutex());
+ RewriteDriver* rewrite_driver = NULL;
+ if (!available_rewrite_drivers_.empty()) {
+ rewrite_driver = available_rewrite_drivers_.back();
+ available_rewrite_drivers_.pop_back();
+ } else {
+ rewrite_driver = NewCustomRewriteDriver(options_);
+ }
+ active_rewrite_drivers_.insert(rewrite_driver);
+ return rewrite_driver;
+}
+
+void RewriteDriverFactory::ReleaseRewriteDriver(
+ RewriteDriver* rewrite_driver) {
+ ScopedMutex lock(rewrite_drivers_mutex());
+ int count = active_rewrite_drivers_.erase(rewrite_driver);
+ if (count != 1) {
+ LOG(ERROR) << "ReleaseRewriteDriver called with driver not in active set.";
+ } else {
+ available_rewrite_drivers_.push_back(rewrite_driver);
+ }
+}
+
+void RewriteDriverFactory::AddPlatformSpecificRewritePasses(
+ RewriteDriver* driver) {
+}
+
+UrlFetcher* RewriteDriverFactory::ComputeUrlFetcher() {
+ if (url_fetcher_ == NULL) {
+ // Run any hooks like setting up slurp directory.
+ FetcherSetupHooks();
+ if (slurp_directory_.empty()) {
+ if (base_url_fetcher_.get() == NULL) {
+ url_fetcher_ = DefaultUrlFetcher();
+ } else {
+ url_fetcher_ = base_url_fetcher_.get();
+ }
+ } else {
+ SetupSlurpDirectories();
+ }
+ }
+ return url_fetcher_;
+}
+
+UrlAsyncFetcher* RewriteDriverFactory::ComputeUrlAsyncFetcher() {
+ if (url_async_fetcher_ == NULL) {
+ // Run any hooks like setting up slurp directory.
+ FetcherSetupHooks();
+ if (slurp_directory_.empty()) {
+ if (base_url_async_fetcher_.get() == NULL) {
+ url_async_fetcher_ = DefaultAsyncUrlFetcher();
+ } else {
+ url_async_fetcher_ = base_url_async_fetcher_.get();
+ }
+ } else {
+ SetupSlurpDirectories();
+ }
+ }
+ return url_async_fetcher_;
+}
+
+void RewriteDriverFactory::SetupSlurpDirectories() {
+ CHECK(!FetchersComputed());
+ if (slurp_read_only_) {
+ CHECK(!FetchersComputed());
+ HttpDumpUrlFetcher* dump_fetcher = new HttpDumpUrlFetcher(
+ slurp_directory_, file_system(), timer());
+ dump_fetcher->set_print_urls(slurp_print_urls_);
+ url_fetcher_ = dump_fetcher;
+ } else {
+ // Check to see if the factory already had set_base_url_fetcher
+ // called on it. If so, then we'll want to use that fetcher
+ // as the mechanism for the dump-writer to retrieve missing
+ // content from the internet so it can be saved in the slurp
+ // directory.
+ url_fetcher_ = base_url_fetcher_.get();
+ if (url_fetcher_ == NULL) {
+ url_fetcher_ = DefaultUrlFetcher();
+ }
+ HttpDumpUrlWriter* dump_writer = new HttpDumpUrlWriter(
+ slurp_directory_, url_fetcher_, file_system(), timer());
+ dump_writer->set_print_urls(slurp_print_urls_);
+ url_fetcher_ = dump_writer;
+ }
+
+ // We do not use real async fetches when slurping.
+ url_async_fetcher_ = new FakeUrlAsyncFetcher(url_fetcher_);
+}
+
+void RewriteDriverFactory::FetcherSetupHooks() {
+}
+
+void RewriteDriverFactory::ShutDown() {
+ // Avoid double-destructing the url fetchers if they were not overridden
+ // programmatically
+ if ((url_async_fetcher_ != NULL) &&
+ (url_async_fetcher_ != base_url_async_fetcher_.get())) {
+ delete url_async_fetcher_;
+ }
+ url_async_fetcher_ = NULL;
+ if ((url_fetcher_ != NULL) && (url_fetcher_ != base_url_fetcher_.get())) {
+ delete url_fetcher_;
+ }
+ url_fetcher_ = NULL;
+ STLDeleteElements(&active_rewrite_drivers_);
+ STLDeleteElements(&available_rewrite_drivers_);
+
+ file_system_.reset(NULL);
+ hasher_.reset(NULL);
+ filename_encoder_.reset(NULL);
+ timer_.reset(NULL);
+ resource_manager_.reset(NULL);
+ html_parse_message_handler_.reset(NULL);
+ http_cache_.reset(NULL);
+ cache_fetcher_.reset(NULL);
+ cache_async_fetcher_.reset(NULL);
+}
+
+void RewriteDriverFactory::Initialize(Statistics* statistics) {
+ RewriteDriver::Initialize(statistics);
+ if (statistics != NULL) {
+ statistics->AddVariable(kInstawebResource404Count);
+ statistics->AddVariable(kInstawebSlurp404Count);
+ HTTPCache::Initialize(statistics);
+ }
+}
+
+void RewriteDriverFactory::Increment404Count() {
+ Statistics* statistics = resource_manager_->statistics();
+ if (statistics != NULL) {
+ if (resource_404_count_ == NULL) {
+ resource_404_count_ = statistics->GetVariable(kInstawebResource404Count);
+ }
+ resource_404_count_->Add(1);
+ }
+}
+
+void RewriteDriverFactory::IncrementSlurpCount() {
+ Statistics* statistics = resource_manager_->statistics();
+ if (statistics != NULL) {
+ if (slurp_404_count_ == NULL) {
+ slurp_404_count_ = statistics->GetVariable(kInstawebSlurp404Count);
+ }
+ slurp_404_count_->Add(1);
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/rewrite_driver_test.cc b/trunk/src/net/instaweb/rewriter/rewrite_driver_test.cc
new file mode 100644
index 0000000..7e2a405
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/rewrite_driver_test.cc
@@ -0,0 +1,67 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua D. Marantz)
+
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/output_resource.h"
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+
+namespace net_instaweb {
+
+class RewriteDriverTest : public ResourceManagerTestBase {
+ protected:
+ RewriteDriverTest() {}
+
+ bool CanDecodeUrl(const StringPiece& url) {
+ RewriteFilter* filter;
+ scoped_ptr<OutputResource> resource(
+ rewrite_driver_.DecodeOutputResource(url, &filter));
+ return (resource.get() != NULL);
+ }
+
+ DISALLOW_COPY_AND_ASSIGN(RewriteDriverTest);
+};
+
+TEST_F(RewriteDriverTest, NoChanges) {
+ ValidateNoChanges("no_changes",
+ "<head><script src=\"foo.js\"></script></head>"
+ "<body><form method=\"post\">"
+ "<input type=\"checkbox\" checked>"
+ "</form></body>");
+}
+
+TEST_F(RewriteDriverTest, TestLegacyUrl) {
+ rewrite_driver_.AddFilters();
+ EXPECT_FALSE(CanDecodeUrl("http://example.com/dir/123/jm.0.orig"))
+ << "not enough dots";
+ EXPECT_TRUE(CanDecodeUrl("http://example.com/dir/123/jm.0.orig.js"));
+ EXPECT_TRUE(CanDecodeUrl(
+ "http://x.com/dir/123/jm.0123456789abcdef0123456789ABCDEF.orig.js"));
+ EXPECT_FALSE(CanDecodeUrl("http://example.com/dir/123/xx.0.orig.js"))
+ << "invalid filter xx";
+ ASSERT_FALSE(CanDecodeUrl("http://example.com/dir/123/jm.z.orig.js"))
+ << "invalid hash code -- not hex";
+ ASSERT_FALSE(CanDecodeUrl("http://example.com/dir/123/jm.ab.orig.js"))
+ << "invalid hash code -- not 1 or 32 chars";
+ ASSERT_FALSE(CanDecodeUrl("http://example.com/dir/123/jm.0.orig.x"))
+ << "invalid extension";
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/rewrite_filter.cc b/trunk/src/net/instaweb/rewriter/rewrite_filter.cc
new file mode 100644
index 0000000..aa53d25
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/rewrite_filter.cc
@@ -0,0 +1,26 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/rewrite_filter.h"
+
+namespace net_instaweb {
+
+RewriteFilter::~RewriteFilter() {
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/rewrite_options.cc b/trunk/src/net/instaweb/rewriter/rewrite_options.cc
new file mode 100644
index 0000000..d28e1cb
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/rewrite_options.cc
@@ -0,0 +1,287 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "net/instaweb/rewriter/public/rewrite_options.h"
+
+#include <vector>
+#include "net/instaweb/util/public/message_handler.h"
+
+namespace net_instaweb {
+
+// TODO(jmarantz): consider merging this threshold with the image-inlining
+// threshold, which is currently defaulting at 2000, so we have a single
+// byte-count threshold, above which inlined resources get outlined, and
+// below which outlined resources get inlined.
+//
+// TODO(jmarantz): user-agent-specific selection of inline threshold so that
+// mobile phones are more prone to inlining.
+//
+// Further notes; jmaessen says:
+//
+// I suspect we do not want these bounds to match, and inlining for
+// images is a bit more complicated because base64 encoding inflates
+// the byte count of data: urls. This is a non-issue for other
+// resources (there may be some weirdness with iframes I haven't
+// thought about...).
+//
+// jmarantz says:
+//
+// One thing we could do, if we believe they should be conceptually
+// merged, is in img_rewrite_filter you could apply the
+// base64-bloat-factor before comparing against the threshold. Then
+// we could use one number if we like that idea.
+//
+
+// jmaessen: For the moment, there's a separate threshold for img inline.
+const int64 RewriteOptions::kDefaultCssInlineMaxBytes = 2048;
+const int64 RewriteOptions::kDefaultImgInlineMaxBytes = 2048;
+const int64 RewriteOptions::kDefaultJsInlineMaxBytes = 2048;
+const int64 RewriteOptions::kDefaultCssOutlineMinBytes = 3000;
+const int64 RewriteOptions::kDefaultJsOutlineMinBytes = 3000;
+
+// Limit on concurrent ongoing img rewrites.
+// TODO(jmaessen): Determine a sane default for this value.
+const int RewriteOptions::kDefaultImgMaxRewritesAtOnce = 8;
+
+// IE limits URL size overall to about 2k characters. See
+// http://support.microsoft.com/kb/208427/EN-US
+const int RewriteOptions::kMaxUrlSize = 2083;
+
+// See http://code.google.com/p/modpagespeed/issues/detail?id=9
+// Apache evidently limits each URL path segment (between /) to
+// about 256 characters. This is not a fundamental URL limitation
+// but is Apache specific. For the moment we will impose the
+// Apache limitation in general -- there is concern that even an
+// nginx server might fail if resources are then served through
+// an Apache proxy. Until then let's just set a reasonable limit.
+const int64 RewriteOptions::kMaxUrlSegmentSize = 250;
+
+const std::string RewriteOptions::kDefaultBeaconUrl =
+ "/mod_pagespeed_beacon?ets=";
+
+bool RewriteOptions::ParseRewriteLevel(
+ const StringPiece& in, RewriteLevel* out) {
+ bool ret = false;
+ if (in != NULL) {
+ if (strcasecmp(in.data(), "CoreFilters") == 0) {
+ *out = kCoreFilters;
+ ret = true;
+ } else if (strcasecmp(in.data(), "PassThrough") == 0) {
+ *out = kPassThrough;
+ ret = true;
+ } else if (strcasecmp(in.data(), "TestingCoreFilters") == 0) {
+ *out = kTestingCoreFilters;
+ ret = true;
+ } else if (strcasecmp(in.data(), "AllFilters") == 0) {
+ *out = kAllFilters;
+ ret = true;
+ }
+ }
+ return ret;
+}
+
+RewriteOptions::RewriteOptions()
+ : modified_(false),
+ level_(kPassThrough),
+ css_inline_max_bytes_(kDefaultCssInlineMaxBytes),
+ img_inline_max_bytes_(kDefaultImgInlineMaxBytes),
+ img_max_rewrites_at_once_(kDefaultImgMaxRewritesAtOnce),
+ js_inline_max_bytes_(kDefaultJsInlineMaxBytes),
+ css_outline_min_bytes_(kDefaultCssInlineMaxBytes),
+ js_outline_min_bytes_(kDefaultJsInlineMaxBytes),
+ num_shards_(0),
+ beacon_url_(kDefaultBeaconUrl),
+ max_url_segment_size_(kMaxUrlSegmentSize),
+ max_url_size_(kMaxUrlSize),
+ enabled_(true) {
+ // TODO: If we instantiate many RewriteOptions, this should become a
+ // public static method called once at startup.
+ SetUp();
+}
+
+RewriteOptions::~RewriteOptions() {
+}
+
+void RewriteOptions::SetUp() {
+ name_filter_map_["add_base_tag"] = kAddBaseTag;
+ name_filter_map_["add_head"] = kAddHead;
+ name_filter_map_["add_instrumentation"] = kAddInstrumentation;
+ name_filter_map_["collapse_whitespace"] = kCollapseWhitespace;
+ name_filter_map_["combine_css"] = kCombineCss;
+ name_filter_map_["combine_heads"] = kCombineHeads;
+ name_filter_map_["debug_log_img_tags"] = kDebugLogImgTags;
+ name_filter_map_["elide_attributes"] = kElideAttributes;
+ name_filter_map_["extend_cache"] = kExtendCache;
+ name_filter_map_["inline_css"] = kInlineCss;
+ name_filter_map_["inline_javascript"] = kInlineJavascript;
+ name_filter_map_["insert_img_dimensions"] = kInsertImgDimensions;
+ name_filter_map_["left_trim_urls"] = kLeftTrimUrls;
+ name_filter_map_["move_css_to_head"] = kMoveCssToHead;
+ name_filter_map_["outline_css"] = kOutlineCss;
+ name_filter_map_["outline_javascript"] = kOutlineJavascript;
+ name_filter_map_["remove_comments"] = kRemoveComments;
+ name_filter_map_["remove_quotes"] = kRemoveQuotes;
+ name_filter_map_["rewrite_css"] = kRewriteCss;
+ name_filter_map_["rewrite_images"] = kRewriteImages;
+ name_filter_map_["rewrite_javascript"] = kRewriteJavascript;
+ name_filter_map_["strip_scripts"] = kStripScripts;
+
+ // Create an empty set for the pass-through level.
+ level_filter_set_map_[kPassThrough];
+
+ // Core filter level includes the "core" filter set.
+ level_filter_set_map_[kCoreFilters].insert(kAddHead);
+ level_filter_set_map_[kCoreFilters].insert(kCombineCss);
+ // TODO(jmarantz): re-enable javascript and CSS minification in
+ // the core set after the reported bugs have been fixed. They
+ // can still be enabled individually.
+ // level_filter_set_map_[kCoreFilters].insert(kRewriteJavascript);
+ // level_filter_set_map_[kCoreFilters].insert(kRewriteCss);
+ level_filter_set_map_[kCoreFilters].insert(kInlineCss);
+ level_filter_set_map_[kCoreFilters].insert(kInlineJavascript);
+ level_filter_set_map_[kCoreFilters].insert(kRewriteImages);
+ level_filter_set_map_[kCoreFilters].insert(kInsertImgDimensions);
+ level_filter_set_map_[kCoreFilters].insert(kExtendCache);
+
+ // Copy CoreFilters set into TestingCoreFilters set ...
+ level_filter_set_map_[kTestingCoreFilters] =
+ level_filter_set_map_[kCoreFilters];
+ // ... and add possibly unsafe filters.
+ level_filter_set_map_[kTestingCoreFilters].insert(kRewriteJavascript);
+ level_filter_set_map_[kTestingCoreFilters].insert(kRewriteCss);
+
+ // Set complete set for all filters set.
+ for (int f = kFirstEnumFilter; f != kLastEnumFilter; ++f) {
+ level_filter_set_map_[kAllFilters].insert(static_cast<Filter>(f));
+ }
+}
+
+bool RewriteOptions::EnableFiltersByCommaSeparatedList(
+ const StringPiece& filters, MessageHandler* handler) {
+ return AddCommaSeparatedListToFilterSet(
+ filters, handler, &enabled_filters_);
+}
+
+bool RewriteOptions::DisableFiltersByCommaSeparatedList(
+ const StringPiece& filters, MessageHandler* handler) {
+ return AddCommaSeparatedListToFilterSet(
+ filters, handler, &disabled_filters_);
+}
+
+void RewriteOptions::EnableFilter(Filter filter) {
+ std::pair<FilterSet::iterator, bool> inserted =
+ enabled_filters_.insert(filter);
+ modified_ |= inserted.second;
+}
+
+void RewriteOptions::DisableFilter(Filter filter) {
+ std::pair<FilterSet::iterator, bool> inserted =
+ disabled_filters_.insert(filter);
+ modified_ |= inserted.second;
+}
+
+bool RewriteOptions::AddCommaSeparatedListToFilterSet(
+ const StringPiece& filters, MessageHandler* handler, FilterSet* set) {
+ std::vector<StringPiece> names;
+ SplitStringPieceToVector(filters, ",", &names, true);
+ bool ret = true;
+ for (int i = 0, n = names.size(); i < n; ++i) {
+ std::string option(names[i].data(), names[i].size());
+ NameToFilterMap::iterator p = name_filter_map_.find(option);
+ if (p == name_filter_map_.end()) {
+ handler->Message(kWarning, "Invalid filter name: %s", option.c_str());
+ ret = false;
+ } else {
+ std::pair<FilterSet::iterator, bool> inserted = set->insert(p->second);
+ modified_ |= inserted.second;
+ }
+ }
+ return ret;
+}
+
+bool RewriteOptions::Enabled(Filter filter) const {
+ if (disabled_filters_.find(filter) != disabled_filters_.end()) {
+ return false;
+ }
+
+ RewriteLevelToFilterSetMap::const_iterator it =
+ level_filter_set_map_.find(level_.value());
+ if (it != level_filter_set_map_.end()) {
+ const FilterSet& filters = it->second;
+ if (filters.find(filter) != filters.end()) {
+ return true;
+ }
+ }
+
+ return (enabled_filters_.find(filter) != enabled_filters_.end());
+}
+
+void RewriteOptions::Merge(const RewriteOptions& first,
+ const RewriteOptions& second) {
+ modified_ = first.modified_ || second.modified_;
+ enabled_filters_ = first.enabled_filters_;
+ disabled_filters_ = first.disabled_filters_;
+ for (FilterSet::const_iterator p = second.enabled_filters_.begin(),
+ e = second.enabled_filters_.end(); p != e; ++p) {
+ Filter filter = *p;
+ // Enabling in 'second' trumps Disabling in first.
+ disabled_filters_.erase(filter);
+ enabled_filters_.insert(filter);
+ }
+
+ for (FilterSet::const_iterator p = second.disabled_filters_.begin(),
+ e = second.disabled_filters_.end(); p != e; ++p) {
+ Filter filter = *p;
+ // Disabling in 'second' trumps enabling in anything.
+ disabled_filters_.insert(filter);
+ enabled_filters_.erase(filter);
+ }
+
+ // TODO(jmarantz): Use a virtual base class for Option so we can put
+ // this in a loop. Or something.
+ enabled_.Merge(first.enabled_, second.enabled_);
+ level_.Merge(first.level_, second.level_);
+ css_inline_max_bytes_.Merge(first.css_inline_max_bytes_,
+ second.css_inline_max_bytes_);
+ img_inline_max_bytes_.Merge(first.img_inline_max_bytes_,
+ second.img_inline_max_bytes_);
+ img_max_rewrites_at_once_.Merge(first.img_max_rewrites_at_once_,
+ second.img_max_rewrites_at_once_);
+ js_inline_max_bytes_.Merge(first.js_inline_max_bytes_,
+ second.js_inline_max_bytes_);
+ css_outline_min_bytes_.Merge(first.css_outline_min_bytes_,
+ second.css_outline_min_bytes_);
+ js_outline_min_bytes_.Merge(first.js_outline_min_bytes_,
+ second.js_outline_min_bytes_);
+ num_shards_.Merge(first.num_shards_,
+ second.num_shards_);
+ beacon_url_.Merge(first.beacon_url_,
+ second.beacon_url_);
+ max_url_segment_size_.Merge(first.max_url_segment_size_,
+ second.max_url_segment_size_);
+ max_url_size_.Merge(first.max_url_size_,
+ second.max_url_size_);
+
+ // Note that the domain-lawyer merge works one-at-a-time, which is easier
+ // to unit test. So we have to call it twice.
+ domain_lawyer_.Merge(first.domain_lawyer_);
+ domain_lawyer_.Merge(second.domain_lawyer_);
+
+ allow_resources_.CopyFrom(first.allow_resources_);
+ allow_resources_.AppendFrom(second.allow_resources_);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/rewrite_options_test.cc b/trunk/src/net/instaweb/rewriter/rewrite_options_test.cc
new file mode 100644
index 0000000..3d435fb
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/rewrite_options_test.cc
@@ -0,0 +1,314 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: bmcquade@google.com (Bryan McQuade)
+
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/null_message_handler.h"
+#include "net/instaweb/rewriter/public/rewrite_options.h"
+
+namespace {
+
+using net_instaweb::NullMessageHandler;
+using net_instaweb::RewriteOptions;
+
+class RewriteOptionsTest : public ::testing::Test {
+ protected:
+ typedef std::set<RewriteOptions::Filter> FilterSet;
+ void AssertNoneEnabled() {
+ FilterSet s;
+ AssertEnabled(s);
+ }
+
+ void AssertEnabled(const FilterSet& filters) {
+ for (RewriteOptions::Filter f = RewriteOptions::kFirstFilter;
+ f <= RewriteOptions::kLastFilter;
+ f = static_cast<RewriteOptions::Filter>(f + 1)) {
+ if (filters.find(f) != filters.end()) {
+ ASSERT_TRUE(options_.Enabled(f));
+ } else {
+ ASSERT_FALSE(options_.Enabled(f));
+ }
+ }
+ }
+
+ RewriteOptions options_;
+};
+
+TEST_F(RewriteOptionsTest, NoneEnabledByDefault) {
+ AssertNoneEnabled();
+}
+
+TEST_F(RewriteOptionsTest, InstrumentationDisabled) {
+ // Make sure the kCoreFilters enables some filters.
+ options_.SetRewriteLevel(RewriteOptions::kCoreFilters);
+ ASSERT_TRUE(options_.Enabled(RewriteOptions::kExtendCache));
+
+ // Now disable all filters and make sure none are enabled.
+ for (RewriteOptions::Filter f = RewriteOptions::kFirstFilter;
+ f <= RewriteOptions::kLastFilter;
+ f = static_cast<RewriteOptions::Filter>(f + 1)) {
+ options_.DisableFilter(f);
+ }
+ AssertNoneEnabled();
+}
+
+TEST_F(RewriteOptionsTest, DisableTrumpsEnable) {
+ for (RewriteOptions::Filter f = RewriteOptions::kFirstFilter;
+ f <= RewriteOptions::kLastFilter;
+ f = static_cast<RewriteOptions::Filter>(f + 1)) {
+ options_.DisableFilter(f);
+ options_.EnableFilter(f);
+ AssertNoneEnabled();
+ }
+}
+
+TEST_F(RewriteOptionsTest, CoreFilters) {
+ options_.SetRewriteLevel(RewriteOptions::kCoreFilters);
+ FilterSet s;
+ for (RewriteOptions::Filter f = RewriteOptions::kFirstFilter;
+ f <= RewriteOptions::kLastFilter;
+ f = static_cast<RewriteOptions::Filter>(f + 1)) {
+ if (options_.Enabled(f)) {
+ s.insert(f);
+ }
+ }
+
+ // Make sure that more than one filter is enabled in the core filter
+ // set.
+ ASSERT_GT(s.size(), 1);
+}
+
+TEST_F(RewriteOptionsTest, Enable) {
+ FilterSet s;
+ for (RewriteOptions::Filter f = RewriteOptions::kFirstFilter;
+ f <= RewriteOptions::kLastFilter;
+ f = static_cast<RewriteOptions::Filter>(f + 1)) {
+ s.insert(f);
+ options_.EnableFilter(f);
+ AssertEnabled(s);
+ }
+}
+
+TEST_F(RewriteOptionsTest, CommaSeparatedList) {
+ FilterSet s;
+ s.insert(RewriteOptions::kAddBaseTag);
+ s.insert(RewriteOptions::kLeftTrimUrls);
+ const char* kList = "add_base_tag,left_trim_urls";
+ NullMessageHandler handler;
+ ASSERT_TRUE(
+ options_.EnableFiltersByCommaSeparatedList(kList, &handler));
+ AssertEnabled(s);
+ ASSERT_TRUE(
+ options_.DisableFiltersByCommaSeparatedList(kList, &handler));
+ AssertNoneEnabled();
+}
+
+TEST_F(RewriteOptionsTest, ParseRewriteLevel) {
+ RewriteOptions::RewriteLevel level;
+ ASSERT_TRUE(RewriteOptions::ParseRewriteLevel("PassThrough", &level));
+ ASSERT_EQ(RewriteOptions::kPassThrough, level);
+
+ ASSERT_TRUE(RewriteOptions::ParseRewriteLevel("CoreFilters", &level));
+ ASSERT_EQ(RewriteOptions::kCoreFilters, level);
+
+ ASSERT_FALSE(RewriteOptions::ParseRewriteLevel(NULL, &level));
+ ASSERT_FALSE(RewriteOptions::ParseRewriteLevel("", &level));
+ ASSERT_FALSE(RewriteOptions::ParseRewriteLevel("Garbage", &level));
+}
+
+TEST_F(RewriteOptionsTest, MergeLevelsDefault) {
+ RewriteOptions one, two;
+ options_.Merge(one, two);
+ EXPECT_EQ(RewriteOptions::kPassThrough, options_.level());
+}
+
+TEST_F(RewriteOptionsTest, MergeLevelsOneCore) {
+ RewriteOptions one, two;
+ one.SetRewriteLevel(RewriteOptions::kCoreFilters);
+ options_.Merge(one, two);
+ EXPECT_EQ(RewriteOptions::kCoreFilters, options_.level());
+}
+
+TEST_F(RewriteOptionsTest, MergeLevelsOneCoreTwoPass) {
+ RewriteOptions one, two;
+ one.SetRewriteLevel(RewriteOptions::kCoreFilters);
+ two.SetRewriteLevel(RewriteOptions::kPassThrough); // overrides default
+ options_.Merge(one, two);
+ EXPECT_EQ(RewriteOptions::kPassThrough, options_.level());
+}
+
+TEST_F(RewriteOptionsTest, MergeLevelsOnePassTwoCore) {
+ RewriteOptions one, two;
+ one.SetRewriteLevel(RewriteOptions::kPassThrough); // overrides default
+ two.SetRewriteLevel(RewriteOptions::kCoreFilters); // overrides one
+ options_.Merge(one, two);
+ EXPECT_EQ(RewriteOptions::kCoreFilters, options_.level());
+}
+
+TEST_F(RewriteOptionsTest, MergeLevelsBothCore) {
+ RewriteOptions one, two;
+ one.SetRewriteLevel(RewriteOptions::kCoreFilters);
+ two.SetRewriteLevel(RewriteOptions::kCoreFilters);
+ options_.Merge(one, two);
+ EXPECT_EQ(RewriteOptions::kCoreFilters, options_.level());
+}
+
+TEST_F(RewriteOptionsTest, MergeFilterPassThrough) {
+ RewriteOptions one, two;
+ options_.Merge(one, two);
+ EXPECT_FALSE(options_.Enabled(RewriteOptions::kAddHead));
+}
+
+TEST_F(RewriteOptionsTest, MergeFilterEnaOne) {
+ RewriteOptions one, two;
+ one.EnableFilter(RewriteOptions::kAddHead);
+ options_.Merge(one, two);
+ EXPECT_TRUE(options_.Enabled(RewriteOptions::kAddHead));
+}
+
+TEST_F(RewriteOptionsTest, MergeFilterEnaTwo) {
+ RewriteOptions one, two;
+ two.EnableFilter(RewriteOptions::kAddHead);
+ options_.Merge(one, two);
+ EXPECT_TRUE(options_.Enabled(RewriteOptions::kAddHead));
+}
+
+TEST_F(RewriteOptionsTest, MergeFilterEnaOneDisTwo) {
+ RewriteOptions one, two;
+ one.EnableFilter(RewriteOptions::kAddHead);
+ two.DisableFilter(RewriteOptions::kAddHead);
+ options_.Merge(one, two);
+ EXPECT_FALSE(options_.Enabled(RewriteOptions::kAddHead));
+}
+
+TEST_F(RewriteOptionsTest, MergeFilterDisOneEnaTwo) {
+ RewriteOptions one, two;
+ one.DisableFilter(RewriteOptions::kAddHead);
+ two.EnableFilter(RewriteOptions::kAddHead);
+ options_.Merge(one, two);
+ EXPECT_TRUE(options_.Enabled(RewriteOptions::kAddHead));
+}
+
+TEST_F(RewriteOptionsTest, MergeCoreFilter) {
+ RewriteOptions one, two;
+ one.SetRewriteLevel(RewriteOptions::kCoreFilters);
+ options_.Merge(one, two);
+ EXPECT_TRUE(options_.Enabled(RewriteOptions::kExtendCache));
+}
+
+TEST_F(RewriteOptionsTest, MergeCoreFilterEnaOne) {
+ RewriteOptions one, two;
+ one.SetRewriteLevel(RewriteOptions::kCoreFilters);
+ one.EnableFilter(RewriteOptions::kExtendCache);
+ options_.Merge(one, two);
+ EXPECT_TRUE(options_.Enabled(RewriteOptions::kExtendCache));
+}
+
+TEST_F(RewriteOptionsTest, MergeCoreFilterEnaTwo) {
+ RewriteOptions one, two;
+ one.SetRewriteLevel(RewriteOptions::kCoreFilters);
+ two.EnableFilter(RewriteOptions::kExtendCache);
+ options_.Merge(one, two);
+ EXPECT_TRUE(options_.Enabled(RewriteOptions::kExtendCache));
+}
+
+TEST_F(RewriteOptionsTest, MergeCoreFilterEnaOneDisTwo) {
+ RewriteOptions one, two;
+ one.SetRewriteLevel(RewriteOptions::kCoreFilters);
+ one.EnableFilter(RewriteOptions::kExtendCache);
+ two.DisableFilter(RewriteOptions::kExtendCache);
+ options_.Merge(one, two);
+ EXPECT_FALSE(options_.Enabled(RewriteOptions::kExtendCache));
+}
+
+TEST_F(RewriteOptionsTest, MergeCoreFilterDisOne) {
+ RewriteOptions one, two;
+ one.SetRewriteLevel(RewriteOptions::kCoreFilters);
+ one.DisableFilter(RewriteOptions::kExtendCache);
+ options_.Merge(one, two);
+ EXPECT_FALSE(options_.Enabled(RewriteOptions::kExtendCache));
+}
+
+TEST_F(RewriteOptionsTest, MergeCoreFilterDisOneEnaTwo) {
+ RewriteOptions one, two;
+ one.SetRewriteLevel(RewriteOptions::kCoreFilters);
+ one.DisableFilter(RewriteOptions::kExtendCache);
+ two.EnableFilter(RewriteOptions::kExtendCache);
+ options_.Merge(one, two);
+ EXPECT_TRUE(options_.Enabled(RewriteOptions::kExtendCache));
+}
+
+TEST_F(RewriteOptionsTest, MergeThresholdDefault) {
+ RewriteOptions one, two;
+ options_.Merge(one, two);
+ EXPECT_EQ(RewriteOptions::kDefaultCssInlineMaxBytes,
+ options_.css_inline_max_bytes());
+}
+
+TEST_F(RewriteOptionsTest, MergeThresholdOne) {
+ RewriteOptions one, two;
+ one.set_css_inline_max_bytes(5);
+ options_.Merge(one, two);
+ EXPECT_EQ(5, options_.css_inline_max_bytes());
+}
+
+TEST_F(RewriteOptionsTest, MergeThresholdTwo) {
+ RewriteOptions one, two;
+ two.set_css_inline_max_bytes(6);
+ options_.Merge(one, two);
+ EXPECT_EQ(6, options_.css_inline_max_bytes());
+}
+
+TEST_F(RewriteOptionsTest, MergeThresholdOverride) {
+ RewriteOptions one, two;
+ one.set_css_inline_max_bytes(5);
+ two.set_css_inline_max_bytes(6);
+ options_.Merge(one, two);
+ EXPECT_EQ(6, options_.css_inline_max_bytes());
+}
+
+TEST_F(RewriteOptionsTest, Allow) {
+ options_.Allow("*.css");
+ EXPECT_TRUE(options_.IsAllowed("abcd.css"));
+ options_.Disallow("a*.css");
+ EXPECT_FALSE(options_.IsAllowed("abcd.css"));
+ options_.Allow("ab*.css");
+ EXPECT_TRUE(options_.IsAllowed("abcd.css"));
+ options_.Disallow("abc*.css");
+ EXPECT_FALSE(options_.IsAllowed("abcd.css"));
+}
+
+TEST_F(RewriteOptionsTest, MergeAllow) {
+ RewriteOptions one, two;
+ one.Allow("*.css");
+ EXPECT_TRUE(one.IsAllowed("abcd.css"));
+ one.Disallow("a*.css");
+ EXPECT_FALSE(one.IsAllowed("abcd.css"));
+
+ two.Allow("ab*.css");
+ EXPECT_TRUE(two.IsAllowed("abcd.css"));
+ two.Disallow("abc*.css");
+ EXPECT_FALSE(two.IsAllowed("abcd.css"));
+
+ options_.Merge(one, two);
+ EXPECT_FALSE(options_.IsAllowed("abcd.css"));
+ EXPECT_FALSE(options_.IsAllowed("abc.css"));
+ EXPECT_TRUE(options_.IsAllowed("ab.css"));
+ EXPECT_FALSE(options_.IsAllowed("a.css"));
+}
+
+} // namespace
diff --git a/trunk/src/net/instaweb/rewriter/rewrite_single_resource_filter.cc b/trunk/src/net/instaweb/rewriter/rewrite_single_resource_filter.cc
new file mode 100644
index 0000000..a6512da
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/rewrite_single_resource_filter.cc
@@ -0,0 +1,110 @@
+/*
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/rewriter/public/rewrite_single_resource_filter.h"
+
+#include "base/scoped_ptr.h"
+#include "net/instaweb/rewriter/public/output_resource.h"
+#include "net/instaweb/rewriter/public/resource.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/url_escaper.h"
+
+namespace net_instaweb {
+
+RewriteSingleResourceFilter::~RewriteSingleResourceFilter() {}
+
+// Manage rewriting an input resource after it has been fetched/loaded.
+class RewriteSingleResourceFilter::FetchCallback
+ : public Resource::AsyncCallback {
+ public:
+ FetchCallback(RewriteSingleResourceFilter* filter,
+ Resource* input_resource, OutputResource* output_resource,
+ MetaData* response_headers, Writer* response_writer,
+ MessageHandler* handler,
+ UrlAsyncFetcher::Callback* base_callback)
+ : filter_(filter),
+ input_resource_(input_resource),
+ output_resource_(output_resource),
+ response_headers_(response_headers),
+ response_writer_(response_writer),
+ handler_(handler),
+ base_callback_(base_callback) {}
+
+ virtual void Done(bool success, Resource* resource) {
+ CHECK_EQ(input_resource_.get(), resource);
+ if (success) {
+ // This checks HTTP status was 200 OK.
+ success = input_resource_->ContentsValid();
+ }
+ if (success) {
+ // Call the rewrite hook.
+ success = filter_->RewriteLoadedResource(input_resource_.get(),
+ output_resource_);
+ }
+ if (success) {
+ // Copy headers and content to HTTP response.
+ // TODO(sligocki): It might be worth streaming this.
+ response_headers_->CopyFrom(*output_resource_->metadata());
+ response_writer_->Write(output_resource_->contents(), handler_);
+ }
+ base_callback_->Done(success);
+ delete this;
+ }
+
+ private:
+ RewriteSingleResourceFilter* filter_;
+ scoped_ptr<Resource> input_resource_;
+ OutputResource* output_resource_;
+ MetaData* response_headers_;
+ Writer* response_writer_;
+ MessageHandler* handler_;
+ UrlAsyncFetcher::Callback* base_callback_;
+
+ DISALLOW_COPY_AND_ASSIGN(FetchCallback);
+};
+
+bool RewriteSingleResourceFilter::Fetch(
+ OutputResource* output_resource,
+ Writer* response_writer,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ MessageHandler* message_handler,
+ UrlAsyncFetcher::Callback* base_callback) {
+ bool ret = false;
+ Resource* input_resource =
+ resource_manager_->CreateInputResourceFromOutputResource(
+ resource_manager_->url_escaper(), output_resource,
+ driver_->options(), message_handler);
+ if (input_resource != NULL) {
+ // Callback takes ownership of input_resoruce.
+ FetchCallback* fetch_callback = new FetchCallback(
+ this, input_resource, output_resource,
+ response_headers, response_writer, message_handler, base_callback);
+ resource_manager_->ReadAsync(input_resource, fetch_callback,
+ message_handler);
+ ret = true;
+ } else {
+ std::string url;
+ output_resource->name().CopyToString(&url);
+ message_handler->Error(url.c_str(), 0, "Unable to decode resource string");
+ }
+ return ret;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/rewriter_test.cc b/trunk/src/net/instaweb/rewriter/rewriter_test.cc
new file mode 100644
index 0000000..425d0fb
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/rewriter_test.cc
@@ -0,0 +1,130 @@
+/*
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test some small filters.
+
+#include "net/instaweb/rewriter/public/resource_manager_test_base.h"
+
+#include "net/instaweb/rewriter/public/rewrite_driver.h"
+
+namespace net_instaweb {
+
+namespace {
+
+class RewriterTest : public ResourceManagerTestBase {};
+
+TEST_F(RewriterTest, AddHead) {
+ AddFilter(RewriteOptions::kAddHead);
+ ValidateExpected("add_head",
+ "<body><p>text</p></body>",
+ "<head/><body><p>text</p></body>");
+}
+
+TEST_F(RewriterTest, AddHeadNoBody) {
+ // Test for proper diagnostic (regression test for Issue 134)
+ AddFilter(RewriteOptions::kAddHead);
+ ValidateExpected("add_head_no_body",
+ "<p>text</p>",
+ "<p>text</p>");
+ EXPECT_EQ(0, message_handler_.SeriousMessages());
+}
+
+TEST_F(RewriterTest, MergeHead) {
+ AddFilter(RewriteOptions::kCombineHeads);
+ ValidateExpected("merge_2_heads",
+ "<head a><p>1</p></head>4<head b>2<link x>3</head><link y>end",
+ "<head a><p>1</p>2<link x>3</head>4<link y>end");
+ ValidateExpected("merge_3_heads",
+ "<head a><p>1</p></head>4<head b>2<link x>3</head><link y>"
+ "<body>b<head><link z></head>ye</body>",
+ "<head a><p>1</p>2<link x>3<link z></head>4<link y>"
+ "<body>bye</body>");
+}
+
+TEST_F(RewriterTest, BaseTagNoHead) {
+ AddFilter(RewriteOptions::kAddBaseTag);
+ rewrite_driver_.SetBaseUrl("http://base");
+ ValidateExpected("base_tag",
+ "<body><p>text</p></body>",
+ "<head><base href=\"http://base\"></head><body><p>text</p></body>");
+}
+
+TEST_F(RewriterTest, BaseTagExistingHead) {
+ AddFilter(RewriteOptions::kAddBaseTag);
+ rewrite_driver_.SetBaseUrl("http://base");
+ ValidateExpected("base_tag",
+ "<head><meta></head><body><p>text</p></body>",
+ "<head><base href=\"http://base\"><meta></head><body><p>text</p></body>");
+}
+
+TEST_F(RewriterTest, BaseTagExistingHeadAndNonHrefBase) {
+ AddFilter(RewriteOptions::kAddBaseTag);
+ rewrite_driver_.SetBaseUrl("http://base");
+ ValidateExpected("base_tag",
+ "<head><base x><meta></head><body></body>",
+ "<head><base href=\"http://base\"><base x><meta></head><body></body>");
+}
+
+TEST_F(RewriterTest, BaseTagExistingHeadAndHrefBase) {
+ AddFilter(RewriteOptions::kAddBaseTag);
+ rewrite_driver_.SetBaseUrl("http://base");
+ ValidateExpected("base_tag",
+ "<head><meta><base href=\"http://old\"></head><body></body>",
+ "<head><base href=\"http://base\"><meta></head><body></body>");
+}
+
+TEST_F(RewriterTest, FailGracefullyOnInvalidUrls) {
+ Hasher* hasher = &md5_hasher_;
+ resource_manager_->set_hasher(hasher);
+ AddFilter(RewriteOptions::kExtendCache);
+
+ const char kCssData[] = "a { color: red }";
+ InitMetaData("a.css", kContentTypeCss, kCssData, 100);
+
+ // Fetching the real rewritten resource name should work.
+ // TODO(sligocki): This will need to be regolded if naming format changes.
+ std::string hash = hasher->Hash(kCssData);
+ EXPECT_TRUE(
+ TryFetchResource(Encode("http://test.com/", "ce", hash, "a.css", "css")));
+
+ // Fetching variants should not cause system problems.
+ // Changing hash still works.
+ // Note: If any of these switch from true to false, that's probably fine.
+ // We'd just like to keep track of what causes errors and what doesn't.
+ EXPECT_TRUE(TryFetchResource(Encode("http://test.com/", "ce", "foobar",
+ "a.css", "css")));
+ EXPECT_TRUE(
+ TryFetchResource(Encode("http://test.com/", "ce", hash, "a.css",
+ "ext")));
+
+ // Changing other fields can lead to error.
+ std::string bad_url = Encode("http://test.com/", "xz", hash, "a.css", "css");
+
+ // Note that although we pass in a real value for 'request headers', we
+ // null out the response and writer as those should never be called.
+ SimpleMetaData request_headers;
+ FetchCallback callback;
+ bool fetched = rewrite_driver_.FetchResource(
+ bad_url, request_headers, NULL, NULL, &message_handler_, &callback);
+ EXPECT_FALSE(fetched);
+ EXPECT_FALSE(callback.done());
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/script_tag_scanner.cc b/trunk/src/net/instaweb/rewriter/script_tag_scanner.cc
new file mode 100644
index 0000000..1894504
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/script_tag_scanner.cc
@@ -0,0 +1,152 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/script_tag_scanner.h"
+
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/util/public/atom.h"
+#include "net/instaweb/util/public/atom.h"
+
+namespace net_instaweb {
+
+// The list is from HTML5, "4.3.1.1 Scripting languages"
+static const char* const javascript_mimetypes[] = {
+ "application/ecmascript",
+ "application/javascript",
+ "application/x-ecmascript",
+ "application/x-javascript",
+ "text/ecmascript",
+ "text/javascript",
+ "text/javascript1.0",
+ "text/javascript1.1",
+ "text/javascript1.2",
+ "text/javascript1.3",
+ "text/javascript1.4",
+ "text/javascript1.5",
+ "text/jscript",
+ "text/livescript",
+ "text/x-ecmascript",
+ "text/x-javascript"
+};
+
+ScriptTagScanner::ScriptTagScanner(HtmlParse* html_parse)
+ : s_async_(html_parse->Intern("async")),
+ s_defer_(html_parse->Intern("defer")),
+ s_event_(html_parse->Intern("event")),
+ s_for_(html_parse->Intern("for")),
+ s_language_(html_parse->Intern("language")),
+ s_script_(html_parse->Intern("script")),
+ s_src_(html_parse->Intern("src")),
+ s_type_(html_parse->Intern("type")) {
+ for (int i = 0; i < int(arraysize(javascript_mimetypes)); ++i) {
+ javascript_mimetypes_.insert(std::string(javascript_mimetypes[i]));
+ }
+}
+
+ScriptTagScanner::ScriptClassification ScriptTagScanner::ParseScriptElement(
+ HtmlElement* element, HtmlElement::Attribute** src) {
+ if (element->tag() != s_script_) {
+ return kNonScript;
+ }
+
+ *src = element->FindAttribute(s_src_);
+
+ // Figure out if we have JS or not.
+ // This is based on the 'type' and 'language' attributes, with
+ // type having precedence. For this determination, however,
+ // <script type> acts as if the type attribute is not there,
+ // which is different from <script type="">
+ ScriptClassification lang;
+ HtmlElement::Attribute* type_attr = element->FindAttribute(s_type_);
+ HtmlElement::Attribute* lang_attr = element->FindAttribute(s_language_);
+ if (type_attr != NULL && type_attr->value() != NULL) {
+ StringPiece type_str = type_attr->value();
+ if (type_str.empty() || IsJsMime(Normalized(type_str))) {
+ // An empty type string (but not whitespace-only!) is JS,
+ // So is one that's a known mimetype once lowercased and
+ // having its leading and trailing whitespace removed
+ lang = kJavaScript;
+ } else {
+ lang = kUnknownScript;
+ }
+ } else if (lang_attr != NULL && lang_attr->value() != NULL) {
+ // Without type= the ultra-deprecated language attribute determines things.
+ // empty or null one is ignored. The test is done case-insensitively,
+ // but leading/trailing whitespace matters.
+ // (Note: null check on ->value() above as it's passed to std::string)
+ std::string lang_str = lang_attr->value();
+ LowerString(&lang_str);
+ if (lang_str.empty() || IsJsMime(StrCat("text/", lang_str))) {
+ lang = kJavaScript;
+ } else {
+ lang = kUnknownScript;
+ }
+ } else {
+ // JS is the default if nothing is specified at all.
+ lang = kJavaScript;
+ }
+
+ return lang;
+}
+
+
+int ScriptTagScanner::ExecutionMode(const HtmlElement* element) const {
+ int flags = 0;
+
+ if (element->FindAttribute(s_async_) != NULL) {
+ flags |= kExecuteAsync;
+ }
+
+ if (element->FindAttribute(s_defer_) != NULL) {
+ flags |= kExecuteDefer;
+ }
+
+ // HTML5 notes that certain values of IE-proprietary 'for' and 'event'
+ // attributes are magic and are to be handled as if they're not there,
+ // while others will cause the script to not be run at all.
+ // Note: there is a disagreement between Chrome and Firefox on how
+ // empty ones are handled. We set kExecuteForEvent as it is the conservative
+ // value, requiring careful treatment by filters
+ const HtmlElement::Attribute* for_attr = element->FindAttribute(s_for_);
+ const HtmlElement::Attribute* event_attr = element->FindAttribute(s_event_);
+ if (for_attr != NULL && event_attr != NULL) {
+ if (Normalized(for_attr->value()) != "window") {
+ flags |= kExecuteForEvent;
+ }
+ std::string event_str = Normalized(event_attr->value());
+ if (event_str != "onload" && event_str != "onload()") {
+ flags |= kExecuteForEvent;
+ }
+ }
+
+ return flags;
+}
+
+std::string ScriptTagScanner::Normalized(const StringPiece& str) {
+ std::string normal_form;
+ TrimWhitespace(str, &normal_form);
+ LowerString(&normal_form);
+ return normal_form;
+}
+
+bool ScriptTagScanner::IsJsMime(const std::string& type_str) {
+ return javascript_mimetypes_.find(type_str) != javascript_mimetypes_.end();
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/script_tag_scanner_test.cc b/trunk/src/net/instaweb/rewriter/script_tag_scanner_test.cc
new file mode 100644
index 0000000..bdc4be3
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/script_tag_scanner_test.cc
@@ -0,0 +1,389 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: mdsteele@google.com (Matthew D. Steele)
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/empty_html_filter.h"
+#include "net/instaweb/htmlparse/public/html_parse_test_base.h"
+#include "net/instaweb/rewriter/public/script_tag_scanner.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace net_instaweb {
+
+class ScriptTagScannerTest : public HtmlParseTestBase {
+ protected:
+ ScriptTagScannerTest() : collector_(&html_parse_) {
+ html_parse_.AddFilter(&collector_);
+ }
+
+ virtual bool AddBody() const { return true; }
+
+ // Helper class to collect script information (language,
+ // and attributes)
+ class ScriptCollector : public EmptyHtmlFilter {
+ public:
+ ScriptCollector(HtmlParse* html_parse)
+ : script_tag_scanner_(html_parse) {
+ }
+
+ virtual void StartElement(HtmlElement* element) {
+ HtmlElement::Attribute* src;
+ ScriptInfo info;
+ info.classification =
+ script_tag_scanner_.ParseScriptElement(element, &src);
+ if (info.classification != ScriptTagScanner::kNonScript) {
+ if (src) {
+ info.url = src->value();
+ }
+ info.flags = script_tag_scanner_.ExecutionMode(element);
+ scripts_.push_back(info);
+ }
+ }
+
+ int Size() const { return static_cast<int>(scripts_.size()); }
+
+ const std::string& UrlAt(int pos) const {
+ return scripts_[pos].url;
+ }
+
+ ScriptTagScanner::ScriptClassification ClassificationAt(int pos) {
+ return scripts_[pos].classification;
+ }
+
+ int FlagsAt(int pos) {
+ return scripts_[pos].flags;
+ }
+
+ virtual const char* Name() const { return "ScriptCollector"; }
+
+ private:
+ struct ScriptInfo {
+ std::string url;
+ ScriptTagScanner::ScriptClassification classification;
+ int flags;
+ };
+
+ std::vector<ScriptInfo> scripts_;
+ ScriptTagScanner script_tag_scanner_;
+
+ DISALLOW_COPY_AND_ASSIGN(ScriptCollector);
+ };
+
+ struct TestSpec {
+ const char* attributes;
+ int expected_flags;
+ };
+
+ // Checks to make sure each of attributes inside <script> produces
+ // the appropriate flags. The array is expected to be 0-terminated
+ void TestFlags(const TestSpec* test_spec) {
+ std::string html;
+ int test;
+ for (test = 0; test_spec[test].attributes; ++test) {
+ html += "<script " + std::string(test_spec[test].attributes) +
+ "></script>";
+ }
+
+ ValidateNoChanges("from_test_spec", html);
+
+ ASSERT_EQ(test, collector_.Size());
+ for (test = 0; test_spec[test].attributes; ++test) {
+ LOG(INFO) << test_spec[test].attributes;
+ EXPECT_EQ(test_spec[test].expected_flags, collector_.FlagsAt(test));
+ }
+ }
+
+ std::string ScriptWithType(const std::string& type) {
+ return "<script type=\"" + type + "\"></script>";
+ }
+
+ std::string ScriptWithLang(const std::string& type) {
+ return "<script language=\"" + type + "\"></script>";
+ }
+
+ ScriptCollector collector_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(ScriptTagScannerTest);
+};
+
+// Note: kNonScript is covered by the length counts,
+// as it will not go into the collector
+
+TEST_F(ScriptTagScannerTest, NotFoundScriptTag) {
+ ValidateNoChanges("noscript", "<noscript>");
+ ASSERT_EQ(0, collector_.Size());
+}
+
+TEST_F(ScriptTagScannerTest, FindNoScriptTag) {
+ ValidateNoChanges("simple_script", "<script src=\"myscript.js\"></script>");
+ ASSERT_EQ(1, collector_.Size());
+ EXPECT_EQ(std::string("myscript.js"), collector_.UrlAt(0));
+ EXPECT_EQ(ScriptTagScanner::kJavaScript, collector_.ClassificationAt(0));
+}
+
+TEST_F(ScriptTagScannerTest, TypeNoVal) {
+ // type with no value - handle as JS
+ ValidateNoChanges("simple_script", "<script type></script>");
+ ASSERT_EQ(1, collector_.Size());
+ EXPECT_EQ(std::string(), collector_.UrlAt(0));
+ EXPECT_EQ(ScriptTagScanner::kJavaScript, collector_.ClassificationAt(0));
+}
+
+TEST_F(ScriptTagScannerTest, TypeEmpty) {
+ // type is empty - handle as JS
+ ValidateNoChanges("simple_script", "<script type=""></script>");
+ ASSERT_EQ(1, collector_.Size());
+ EXPECT_EQ(std::string(), collector_.UrlAt(0));
+ EXPECT_EQ(ScriptTagScanner::kJavaScript, collector_.ClassificationAt(0));
+}
+
+TEST_F(ScriptTagScannerTest, TypeNoValHaveLang) {
+ // type is no-value, but language is there --- it matters
+ ValidateNoChanges("simple_script", "<script type language=tcl></script>");
+ ASSERT_EQ(1, collector_.Size());
+ EXPECT_EQ(std::string(), collector_.UrlAt(0));
+ EXPECT_EQ(ScriptTagScanner::kUnknownScript, collector_.ClassificationAt(0));
+}
+
+TEST_F(ScriptTagScannerTest, TypeLangSubordinate) {
+ // make sure type beats language
+ ValidateNoChanges("simple_script",
+ "<script type=\"text/ecmascript\" language=tcl></script>");
+ ASSERT_EQ(1, collector_.Size());
+ EXPECT_EQ(std::string(), collector_.UrlAt(0));
+ EXPECT_EQ(ScriptTagScanner::kJavaScript, collector_.ClassificationAt(0));
+}
+
+TEST_F(ScriptTagScannerTest, LangNoVal) {
+ // lang no value - handle as JS
+ ValidateNoChanges("simple_script", "<script language></script>");
+ ASSERT_EQ(1, collector_.Size());
+ EXPECT_EQ(std::string(), collector_.UrlAt(0));
+ EXPECT_EQ(ScriptTagScanner::kJavaScript, collector_.ClassificationAt(0));
+}
+
+TEST_F(ScriptTagScannerTest, LangEmpty) {
+ // lang is empty - handle as JS
+ ValidateNoChanges("simple_script", "<script language=""></script>");
+ ASSERT_EQ(1, collector_.Size());
+ EXPECT_EQ(std::string(), collector_.UrlAt(0));
+ EXPECT_EQ(ScriptTagScanner::kJavaScript, collector_.ClassificationAt(0));
+}
+
+TEST_F(ScriptTagScannerTest, TypeScripts) {
+ // various type values. Nothing fancy done with them. List of types is from
+ // HTML5 + a few ones that are not
+ ValidateNoChanges("script types",
+ ScriptWithType("application/ecmascript") + // 0
+ ScriptWithType("application/javascript") +
+ ScriptWithType("application/x-ecmascript") +
+ ScriptWithType("application/x-javascript") +
+ ScriptWithType("text/ecmascript") + // 4
+ ScriptWithType("text/javascript") +
+ ScriptWithType("text/javascript1.0") +
+ ScriptWithType("text/javascript1.1") +
+ ScriptWithType("text/javascript1.2") +
+ ScriptWithType("text/javascript1.3") + // 9
+ ScriptWithType("text/javascript1.4") +
+ ScriptWithType("text/javascript1.5") +
+ ScriptWithType("text/jscript") +
+ ScriptWithType("text/livescript") +
+ ScriptWithType("text/x-ecmascript") + // 14
+ ScriptWithType("text/x-javascript") + // 15 -- last valid one
+ ScriptWithType("text/tcl") +
+ ScriptWithType("text/ecmascript4") +
+ ScriptWithType("text/javascript2.0") +
+ ScriptWithType(" ")); // 19 -- last invalid one
+
+ ASSERT_EQ(20, collector_.Size());
+ for (int i = 0; i <= 15; ++i) {
+ EXPECT_EQ(std::string(), collector_.UrlAt(i));
+ EXPECT_EQ(ScriptTagScanner::kJavaScript, collector_.ClassificationAt(i));
+ }
+
+ for (int i = 16; i <= 19; ++i) {
+ EXPECT_EQ(std::string(), collector_.UrlAt(i));
+ EXPECT_EQ(ScriptTagScanner::kUnknownScript, collector_.ClassificationAt(i));
+ }
+}
+
+TEST_F(ScriptTagScannerTest, TypeScriptsNormalize) {
+ // For type, we need to support removal of leading/trailing whitespace
+ // and case folding
+ ValidateNoChanges("script types",
+ ScriptWithType(" application/ecmascRipt") + // 0
+ ScriptWithType(" applicAtion/javascript ") +
+ ScriptWithType("application/x-ecmaScript ") +
+ ScriptWithType(" applicAtion/x-javascript") +
+ ScriptWithType("text/Ecmascript") + // 4
+ ScriptWithType(" text/jaVasCript ") +
+ ScriptWithType(" TEXt/javascript1.0\t") +
+ ScriptWithType(" text/javascript1.1") +
+ ScriptWithType(" teXt/javascripT1.2") +
+ ScriptWithType("\ttExt/javascRipt1.3 ") + // 9
+ ScriptWithType(" text/javascRipT1.4 ") +
+ ScriptWithType(" Text/javAscript1.5 ") +
+ ScriptWithType(" Text/jscrIpt") +
+ ScriptWithType(" text/lIvescript") +
+ ScriptWithType("teXt/x-ecmasCript ") + // 14
+ ScriptWithType("tExt/x-jaVascript ") + // 15 -- last valid one
+ ScriptWithType("Text/Tcl ") +
+ ScriptWithType(" text/Ecmascript4") +
+ ScriptWithType("tExt/javascript2.0")+
+ ScriptWithType("text/javasc ript")); // 19 -- last invalid one
+
+ ASSERT_EQ(20, collector_.Size());
+ for (int i = 0; i <= 15; ++i) {
+ EXPECT_EQ(std::string(), collector_.UrlAt(i));
+ EXPECT_EQ(ScriptTagScanner::kJavaScript, collector_.ClassificationAt(i));
+ }
+
+ for (int i = 16; i <= 19; ++i) {
+ EXPECT_EQ(std::string(), collector_.UrlAt(i));
+ EXPECT_EQ(ScriptTagScanner::kUnknownScript, collector_.ClassificationAt(i));
+ }
+}
+
+TEST_F(ScriptTagScannerTest, LangScripts) {
+ // for language attribute, we are supposed to test text/lang
+ // against the valid mimetypes list
+ ValidateNoChanges("script langs",
+ ScriptWithLang("ecmascript") +
+ ScriptWithLang("javascript") +
+ ScriptWithLang("javascript1.0") +
+ ScriptWithLang("javascript1.1") +
+ ScriptWithLang("javascript1.2") + // 4
+ ScriptWithLang("javascript1.3") +
+ ScriptWithLang("javascript1.4") +
+ ScriptWithLang("javascript1.5") +
+ ScriptWithLang("jscript") +
+ ScriptWithLang("livescript") + // 9
+ ScriptWithLang("x-ecmascript") +
+ ScriptWithLang("x-javascript") + // 11 -- last valid one
+ ScriptWithLang("tcl") +
+ ScriptWithLang("ecmascript4") +
+ ScriptWithLang("javascript2.0")); // 14 -- last invalid one
+
+ ASSERT_EQ(15, collector_.Size());
+ for (int i = 0; i <= 11; ++i) {
+ EXPECT_EQ(std::string(), collector_.UrlAt(i));
+ EXPECT_EQ(ScriptTagScanner::kJavaScript, collector_.ClassificationAt(i));
+ }
+
+ for (int i = 12; i <= 14; ++i) {
+ EXPECT_EQ(std::string(), collector_.UrlAt(i));
+ EXPECT_EQ(ScriptTagScanner::kUnknownScript, collector_.ClassificationAt(i));
+ }
+}
+
+TEST_F(ScriptTagScannerTest, LangScriptsNormalizeCase) {
+ // Case normalization is to be done for language="" as well.
+ ValidateNoChanges("script langs",
+ ScriptWithLang("ecmasCript") +
+ ScriptWithLang("javAscript") +
+ ScriptWithLang("javascript1.0") +
+ ScriptWithLang("javascRipt1.1") +
+ ScriptWithLang("javascripT1.2") + // 4
+ ScriptWithLang("javaScrIpt1.3") +
+ ScriptWithLang("jaVasCript1.4") +
+ ScriptWithLang("javaScriPt1.5") +
+ ScriptWithLang("jscRiPt") +
+ ScriptWithLang("livEscript") + // 9
+ ScriptWithLang("x-ecmaScript") +
+ ScriptWithLang("x-jaVascript") + // 11 -- last valid one
+ ScriptWithLang("tCl") +
+ ScriptWithLang("ecmasCript4") +
+ ScriptWithLang("jaVascript2.0")); // 14 -- last invalid one
+
+ ASSERT_EQ(15, collector_.Size());
+ for (int i = 0; i <= 11; ++i) {
+ EXPECT_EQ(std::string(), collector_.UrlAt(i));
+ EXPECT_EQ(ScriptTagScanner::kJavaScript, collector_.ClassificationAt(i));
+ }
+
+ for (int i = 12; i <= 14; ++i) {
+ EXPECT_EQ(std::string(), collector_.UrlAt(i));
+ EXPECT_EQ(ScriptTagScanner::kUnknownScript, collector_.ClassificationAt(i));
+ }
+}
+
+TEST_F(ScriptTagScannerTest, LangScriptsNormalizeWhitespace) {
+ // Whitespace, however, is not removed for language, unlike with type,
+ // so all of these are to fail
+ ValidateNoChanges("script langs",
+ ScriptWithLang(" ecmascript") +
+ ScriptWithLang("javascript\t") +
+ ScriptWithLang(" javascript1.0 ") +
+ ScriptWithLang(" javascript1.1") +
+ ScriptWithLang("javascript1.2 ") + // 4
+ ScriptWithLang(" javascript1.3") +
+ ScriptWithLang("javascript1.4 ") +
+ ScriptWithLang(" javascript1.5") +
+ ScriptWithLang("jscript ") +
+ ScriptWithLang("livescript ") + // 9
+ ScriptWithLang(" x-ecmascript") +
+ ScriptWithLang("x-javascript\t") +
+ ScriptWithLang(" tcl ") +
+ ScriptWithLang("ecmascript4 ") +
+ ScriptWithLang(" javascript2.0")); // 14 -- last invalid one
+
+ ASSERT_EQ(15, collector_.Size());
+ for (int i = 0; i <= 14; ++i) {
+ EXPECT_EQ(std::string(), collector_.UrlAt(i));
+ EXPECT_EQ(ScriptTagScanner::kUnknownScript, collector_.ClassificationAt(i));
+ }
+}
+
+TEST_F(ScriptTagScannerTest, ForEvent) {
+ TestSpec for_event_tests[] = {
+ { "for event", ScriptTagScanner::kExecuteForEvent },
+ { "for=\"\" event=\"\"", ScriptTagScanner::kExecuteForEvent },
+ { "for", ScriptTagScanner::kExecuteSync },
+ { "event", ScriptTagScanner::kExecuteSync },
+ { "for=\"a\" event=\"b\"", ScriptTagScanner::kExecuteForEvent },
+ { "for=\"window\" event=\"b\"", ScriptTagScanner::kExecuteForEvent },
+ { "for=\"window\" event=\"b\" async",
+ ScriptTagScanner::kExecuteForEvent | ScriptTagScanner::kExecuteAsync },
+ { "for=\"window\" event=\"onload\"", ScriptTagScanner::kExecuteSync },
+ { "for=\"window\" event=onload async", ScriptTagScanner::kExecuteAsync },
+ { "for=\"window\" event=\"onload()\"", ScriptTagScanner::kExecuteSync },
+ { "for=\"wiNdow \" event=\" onLoad \"", ScriptTagScanner::kExecuteSync },
+ { "for=\" windOw\" event=\"OnloAd() \"", ScriptTagScanner::kExecuteSync },
+ { 0, ScriptTagScanner::kExecuteSync }
+ };
+ TestFlags(for_event_tests);
+}
+
+TEST_F(ScriptTagScannerTest, AsyncDefer) {
+ TestSpec async_defer_tests[] = {
+ { "language=tcl async", ScriptTagScanner::kExecuteAsync },
+ { "async=\"irrelevant\"", ScriptTagScanner::kExecuteAsync },
+ { "defer", ScriptTagScanner::kExecuteDefer },
+ { "defer async",
+ ScriptTagScanner::kExecuteDefer | ScriptTagScanner::kExecuteAsync },
+ { "language=tcl async src=a", ScriptTagScanner::kExecuteAsync },
+ { "async=\"irrelevant\" src=a", ScriptTagScanner::kExecuteAsync },
+ { "defer src=a", ScriptTagScanner::kExecuteDefer },
+ { "defer async src=a",
+ ScriptTagScanner::kExecuteDefer | ScriptTagScanner::kExecuteAsync },
+ { 0, ScriptTagScanner::kExecuteSync }
+ };
+ TestFlags(async_defer_tests);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/strip_scripts_filter.cc b/trunk/src/net/instaweb/rewriter/strip_scripts_filter.cc
new file mode 100644
index 0000000..4e0a9ab
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/strip_scripts_filter.cc
@@ -0,0 +1,35 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/strip_scripts_filter.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+
+namespace net_instaweb {
+
+StripScriptsFilter::StripScriptsFilter(HtmlParse* html_parse)
+ : html_parse_(html_parse),
+ s_script_(html_parse->Intern("script")) {
+}
+
+void StripScriptsFilter::EndElement(HtmlElement* element) {
+ if (element->tag() == s_script_) {
+ html_parse_->DeleteElement(element);
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/strip_scripts_filter_test.cc b/trunk/src/net/instaweb/rewriter/strip_scripts_filter_test.cc
new file mode 100644
index 0000000..cd4e313
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/strip_scripts_filter_test.cc
@@ -0,0 +1,54 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_parse_test_base.h"
+#include "net/instaweb/rewriter/public/strip_scripts_filter.h"
+
+namespace net_instaweb {
+
+class StripScriptsFilterTest : public HtmlParseTestBase {
+ protected:
+ StripScriptsFilterTest()
+ : strip_scripts_filter_(&html_parse_) {
+ html_parse_.AddFilter(&strip_scripts_filter_);
+ }
+
+ virtual bool AddBody() const { return false; }
+
+ private:
+ StripScriptsFilter strip_scripts_filter_;
+
+ DISALLOW_COPY_AND_ASSIGN(StripScriptsFilterTest);
+};
+
+TEST_F(StripScriptsFilterTest, RemoveScriptSrc) {
+ ValidateExpected("remove_script_src",
+ "<head><script src='http://www.google.com/javascript"
+ "/ajax_apis.js'></script></head><body>Hello, world!</body>",
+ "<head></head><body>Hello, world!</body>");
+}
+
+TEST_F(StripScriptsFilterTest, RemoveScriptInline) {
+ ValidateExpected("remove_script_src",
+ "<head><script>alert('Alert, alert!')"
+ "</script></head><body>Hello, world!</body>",
+ "<head></head><body>Hello, world!</body>");
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/url_input_resource.cc b/trunk/src/net/instaweb/rewriter/url_input_resource.cc
new file mode 100644
index 0000000..a6f9c23
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/url_input_resource.cc
@@ -0,0 +1,247 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+// jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/url_input_resource.h"
+
+#include "base/basictypes.h"
+#include "net/instaweb/rewriter/public/domain_lawyer.h"
+#include "net/instaweb/rewriter/public/resource_manager.h"
+#include "net/instaweb/rewriter/public/rewrite_options.h"
+#include "net/instaweb/util/public/abstract_mutex.h"
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/hasher.h"
+#include "net/instaweb/util/public/http_value.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/timer.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+
+namespace net_instaweb {
+
+UrlInputResource::~UrlInputResource() {
+}
+
+// Shared fetch callback, used by both Load and LoadAndCallback
+class UrlResourceFetchCallback : public UrlAsyncFetcher::Callback {
+ public:
+ UrlResourceFetchCallback(ResourceManager* resource_manager,
+ const RewriteOptions* rewrite_options) :
+ resource_manager_(resource_manager),
+ rewrite_options_(rewrite_options),
+ message_handler_(NULL) { }
+ virtual ~UrlResourceFetchCallback() {}
+
+ void AddToCache(bool success) {
+ if (success) {
+ HTTPValue* value = http_value();
+ value->SetHeaders(*response_headers());
+ http_cache()->Put(url(), value, message_handler_);
+ } else {
+ http_cache()->RememberNotCacheable(url(), message_handler_);
+ }
+ }
+
+ bool Fetch(UrlAsyncFetcher* fetcher, MessageHandler* handler) {
+ // TODO(jmarantz): consider request_headers. E.g. will we ever
+ // get different resources depending on user-agent?
+ SimpleMetaData request_headers;
+ message_handler_ = handler;
+ std::string lock_name = StrCat(
+ resource_manager_->filename_prefix(),
+ resource_manager_->hasher()->Hash(url()),
+ ".lock");
+ int64 lock_timeout = fetcher->timeout_ms();
+ if (lock_timeout == UrlAsyncFetcher::kUnspecifiedTimeout) {
+ // Even if the fetcher never explicitly times out requests, they probably
+ // won't succeed after more than 2 minutes.
+ lock_timeout = 2 * Timer::kMinuteMs;
+ } else {
+ // Give a little slack for polling, writing the file, freeing the lock.
+ lock_timeout *= 2;
+ }
+ if (resource_manager_->file_system()->TryLockWithTimeout(
+ lock_name, lock_timeout, message_handler_).is_false()) {
+ message_handler_->Info(lock_name.c_str(), 0,
+ "Someone is already fetching %s ",
+ url().c_str());
+ // TODO(abliss): a per-unit-time statistic would be useful here.
+ if (should_yield()) {
+ DoneInternal(false);
+ delete this;
+ return false;
+ }
+ } else {
+ message_handler_->Info(lock_name.c_str(), 0,
+ "Locking %s for PID %ld",
+ url().c_str(), static_cast<long>(getpid()));
+ lock_name_ = lock_name;
+ }
+
+ std::string url_string = url(), origin_url;
+ bool ret = false;
+ const DomainLawyer* lawyer = rewrite_options_->domain_lawyer();
+ if (lawyer->MapOrigin(url_string, &origin_url)) {
+ if (origin_url != url_string) {
+ // If mapping the URL changes its host, then add a 'Host' header
+ // pointing to the origin URL's hostname.
+ GURL gurl = GoogleUrl::Create(url_string);
+ if (gurl.is_valid()) {
+ request_headers.Add(HttpAttributes::kHost, gurl.host().c_str());
+ }
+ }
+ ret = fetcher->StreamingFetch(
+ origin_url, request_headers, response_headers(), http_value(),
+ handler, this);
+ } else {
+ delete this;
+ }
+ return ret;
+ }
+
+ virtual void Done(bool success) {
+ AddToCache(success);
+ if (!lock_name_.empty()) {
+ message_handler_->Info(lock_name_.c_str(), 0,
+ "Unlocking %s for PID %ld with success=%s",
+ url().c_str(), static_cast<long>(getpid()),
+ success ? "true" : "false");
+ resource_manager_->file_system()->Unlock(lock_name_, message_handler_);
+ }
+ DoneInternal(success);
+ delete this;
+ }
+
+ // The two derived classes differ in how they provide the
+ // fields below. The Async callback gets them from the resource,
+ // which must be live at the time it is called. The ReadIfCached
+ // cannot rely on the resource still being alive when the callback
+ // is called, so it must keep them locally in the class.
+ virtual MetaData* response_headers() = 0;
+ virtual HTTPValue* http_value() = 0;
+ virtual std::string url() const = 0;
+ virtual HTTPCache* http_cache() = 0;
+ // If someone is already fetching this resource, should we yield to them and
+ // try again later? If so, return true. Otherwise, if we must fetch the
+ // resource regardless, return false.
+ // TODO(abliss): unit test this
+ virtual bool should_yield() = 0;
+
+ protected:
+ virtual void DoneInternal(bool success) {
+ }
+
+ ResourceManager* resource_manager_;
+ const RewriteOptions* rewrite_options_;
+ MessageHandler* message_handler_;
+
+ private:
+ std::string lock_name_;
+ DISALLOW_COPY_AND_ASSIGN(UrlResourceFetchCallback);
+};
+
+class UrlReadIfCachedCallback : public UrlResourceFetchCallback {
+ public:
+ UrlReadIfCachedCallback(const std::string& url, HTTPCache* http_cache,
+ ResourceManager* resource_manager,
+ const RewriteOptions* rewrite_options)
+ : UrlResourceFetchCallback(resource_manager, rewrite_options),
+ url_(url),
+ http_cache_(http_cache) {
+ }
+
+ // Indicate that it's OK for the callback to be executed on a different
+ // thread, as it only populates the cache, which is thread-safe.
+ virtual bool EnableThreaded() const { return true; }
+
+ virtual MetaData* response_headers() { return &response_headers_; }
+ virtual HTTPValue* http_value() { return &http_value_; }
+ virtual std::string url() const { return url_; }
+ virtual HTTPCache* http_cache() { return http_cache_; }
+ virtual bool should_yield() { return true; }
+
+ private:
+ std::string url_;
+ HTTPCache* http_cache_;
+ HTTPValue http_value_;
+ SimpleMetaData response_headers_;
+
+ DISALLOW_COPY_AND_ASSIGN(UrlReadIfCachedCallback);
+};
+
+bool UrlInputResource::Load(MessageHandler* handler) {
+ meta_data_.Clear();
+ value_.Clear();
+
+ HTTPCache* http_cache = resource_manager()->http_cache();
+ UrlReadIfCachedCallback* cb = new UrlReadIfCachedCallback(
+ url_, http_cache, resource_manager(), rewrite_options_);
+
+ // If the fetcher can satisfy the request instantly, then we
+ // can try to populate the resource from the cache.
+ bool data_available =
+ (cb->Fetch(resource_manager_->url_async_fetcher(), handler) &&
+ (http_cache->Find(url_, &value_, &meta_data_, handler) ==
+ HTTPCache::kFound));
+ return data_available;
+}
+
+
+class UrlReadAsyncFetchCallback : public UrlResourceFetchCallback {
+ public:
+ explicit UrlReadAsyncFetchCallback(Resource::AsyncCallback* callback,
+ UrlInputResource* resource)
+ : UrlResourceFetchCallback(resource->resource_manager(),
+ resource->rewrite_options()),
+ resource_(resource),
+ callback_(callback) {
+ }
+
+ virtual void DoneInternal(bool success) {
+ callback_->Done(success, resource_);
+ }
+
+ virtual MetaData* response_headers() { return &resource_->meta_data_; }
+ virtual HTTPValue* http_value() { return &resource_->value_; }
+ virtual std::string url() const { return resource_->url(); }
+ virtual HTTPCache* http_cache() {
+ return resource_->resource_manager()->http_cache();
+ }
+ virtual bool should_yield() { return false; }
+
+ private:
+ UrlInputResource* resource_;
+ Resource::AsyncCallback* callback_;
+
+ DISALLOW_COPY_AND_ASSIGN(UrlReadAsyncFetchCallback);
+};
+
+void UrlInputResource::LoadAndCallback(AsyncCallback* callback,
+ MessageHandler* message_handler) {
+ CHECK(callback != NULL) << "A callback must be supplied, or else it will "
+ "not be possible to determine when it's safe to delete the resource.";
+ if (loaded()) {
+ callback->Done(true, this);
+ } else {
+ UrlReadAsyncFetchCallback* cb =
+ new UrlReadAsyncFetchCallback(callback, this);
+ cb->Fetch(resource_manager_->url_async_fetcher(), message_handler);
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/url_left_trim_filter.cc b/trunk/src/net/instaweb/rewriter/url_left_trim_filter.cc
new file mode 100644
index 0000000..db4a454
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/url_left_trim_filter.cc
@@ -0,0 +1,137 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/rewriter/public/url_left_trim_filter.h"
+
+#include <vector>
+#include "net/instaweb/htmlparse/public/html_element.h"
+#include "net/instaweb/htmlparse/public/html_parse.h"
+#include "net/instaweb/util/public/statistics.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace {
+
+// names for Statistics variables.
+const char kUrlTrims[] = "url_trims";
+const char kUrlTrimSavedBytes[] = "url_trim_saved_bytes";
+
+} // namespace
+
+namespace net_instaweb {
+
+UrlLeftTrimFilter::UrlLeftTrimFilter(HtmlParse* html_parse,
+ Statistics* stats)
+ : html_parse_(html_parse),
+ s_base_(html_parse->Intern("base")),
+ s_href_(html_parse->Intern("href")),
+ s_src_(html_parse->Intern("src")),
+ trim_count_((stats == NULL) ? NULL : stats->GetVariable(kUrlTrims)),
+ trim_saved_bytes_(
+ (stats == NULL) ? NULL : stats->GetVariable(kUrlTrimSavedBytes)) {
+}
+
+void UrlLeftTrimFilter::Initialize(Statistics* statistics) {
+ statistics->AddVariable(kUrlTrims);
+ statistics->AddVariable(kUrlTrimSavedBytes);
+}
+
+void UrlLeftTrimFilter::StartElement(HtmlElement* element) {
+ // TODO(jmaessen): handle other places urls might lurk in html.
+ // But never rewrite the base tag; always include its full url.
+ if (element->tag() != s_base_) {
+ TrimAttribute(element->FindAttribute(s_href_));
+ TrimAttribute(element->FindAttribute(s_src_));
+ }
+}
+
+void UrlLeftTrimFilter::AddTrimming(const StringPiece& trimming) {
+ CHECK_LT(0u, trimming.size());
+ left_trim_strings_.push_back(trimming.as_string());
+}
+
+void UrlLeftTrimFilter::AddBaseUrl(const StringPiece& base) {
+ size_t colon_pos = base.find(':');
+ size_t host_start = 0;
+ if (colon_pos != base.npos) {
+ StringPiece protocol(base.data(), colon_pos+1);
+ AddTrimming(protocol);
+ host_start = colon_pos + 3;
+ } else {
+ colon_pos = -1;
+ }
+ size_t first_slash_pos = base.find('/', host_start);
+ if (first_slash_pos != base.npos) {
+ StringPiece host_name(base.data() + colon_pos + 1,
+ first_slash_pos - colon_pos - 1);
+ AddTrimming(host_name);
+ size_t last_slash_pos = base.rfind('/');
+ if (last_slash_pos != base.npos &&
+ last_slash_pos > first_slash_pos) {
+ // Note that we leave a case on the floor here: when base is the root of a
+ // domain (such as http://www.nytimes.com/ ) we can strip the leading /
+ // off rooted urls. We do not do so as the path / is a proper prefix of a
+ // protocol-stripped url such as //www.google.com/, and we don't want to
+ // transform the latter into the incorrect relative url /www.google.com/.
+ // If we simply require last_slash_pos >= first_slash_pos we include this
+ // case, and sites like nytimes break badly.
+ StringPiece base_dir(base.data() + first_slash_pos,
+ last_slash_pos-first_slash_pos+1);
+ AddTrimming(base_dir);
+ }
+ }
+}
+
+// Left trim all strings in left_trim_strings_ from url, in order.
+// StringPiece supports left and right trimming in place (the only
+// mutation it permits).
+bool UrlLeftTrimFilter::Trim(StringPiece* url) {
+ bool trimmed = false;
+ for (StringVector::iterator i = left_trim_strings_.begin();
+ i != left_trim_strings_.end(); ++i) {
+ // First condition below guarantees that we never completely
+ // remove a url, leaving it empty.
+ if (url->length() > i->length() && url->starts_with(*i)) {
+ url->remove_prefix(i->length());
+ trimmed = true;
+ }
+ }
+ return trimmed;
+}
+
+// Trim the value of the given attribute, if the attribute is non-NULL.
+void UrlLeftTrimFilter::TrimAttribute(HtmlElement::Attribute* attr) {
+ if (attr != NULL) {
+ StringPiece val(attr->value());
+ size_t orig_size = val.size();
+ if (Trim(&val)) {
+ size_t saved = orig_size - val.size();
+ const char* q = attr->quote();
+ html_parse_->InfoHere(
+ "trimmed %u %s=%s%s%s to %s%s%s.", static_cast<unsigned>(saved),
+ attr->name().c_str(), q, attr->value(), q,
+ q, val.as_string().c_str(), q);
+ attr->SetValue(val);
+ if (trim_count_ != NULL) {
+ trim_count_->Add(1);
+ trim_saved_bytes_->Add(orig_size - val.size());
+ }
+ }
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/url_left_trim_filter_test.cc b/trunk/src/net/instaweb/rewriter/url_left_trim_filter_test.cc
new file mode 100644
index 0000000..6d4d27d
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/url_left_trim_filter_test.cc
@@ -0,0 +1,122 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/rewriter/public/url_left_trim_filter.h"
+
+#include "base/basictypes.h"
+#include "net/instaweb/htmlparse/public/html_parse_test_base.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class UrlLeftTrimFilterTest : public HtmlParseTestBase {
+ protected:
+ UrlLeftTrimFilterTest()
+ : left_trim_filter_(&html_parse_, NULL) {
+ html_parse_.AddFilter(&left_trim_filter_);
+ }
+
+ void AddTrimming(const StringPiece& trimming) {
+ left_trim_filter_.AddTrimming(trimming);
+ }
+
+ void OneTrim(bool changed,
+ const StringPiece init, const StringPiece expected) {
+ StringPiece url(init);
+ EXPECT_EQ(changed, left_trim_filter_.Trim(&url));
+ EXPECT_EQ(expected, url);
+ }
+
+ virtual bool AddBody() const { return false; }
+
+ UrlLeftTrimFilter left_trim_filter_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(UrlLeftTrimFilterTest);
+};
+
+static const char kBase[] = "http://foo.bar/baz/";
+static const char kHttp[] = "http:";
+static const char kDomain[] = "//foo.bar";
+static const char kPath[] = "/baz/";
+
+TEST_F(UrlLeftTrimFilterTest, SimpleTrims) {
+ StringPiece spHttp(kHttp);
+ StringPiece spDomain(kDomain);
+ StringPiece spPath(kPath);
+ AddTrimming(spHttp);
+ AddTrimming(spDomain);
+ AddTrimming(spPath);
+ OneTrim(true, "http://www.google.com/", "//www.google.com/");
+ OneTrim(true, kBase, kPath);
+ OneTrim(true, "http://foo.bar/baz/quux", "quux");
+ OneTrim(true, "/baz/quux", "quux");
+ OneTrim(true, "//foo.bar/img/img1.jpg", "/img/img1.jpg");
+ OneTrim(false, "/img/img1.jpg", "/img/img1.jpg");
+ OneTrim(false, kHttp, kHttp);
+ OneTrim(true, "//foo.bar/baz/quux", "quux");
+}
+
+static const char kRootedBase[] = "http://foo.bar/";
+
+// Catch screw cases when a base url lies at the root of a domain.
+TEST_F(UrlLeftTrimFilterTest, RootedTrims) {
+ left_trim_filter_.AddBaseUrl(kRootedBase);
+ OneTrim(true, "http://www.google.com/", "//www.google.com/");
+ OneTrim(true, kBase, kPath);
+ OneTrim(false, "//www.google.com/", "//www.google.com/");
+ OneTrim(false, kPath, kPath);
+ OneTrim(false, "quux", "quux");
+}
+
+static const char kNone[] =
+ "<head><base href='ftp://what.the/heck/'/>"
+ "<link src='ftp://what.the/heck/'></head>"
+ "<body><a href='spdy://www.google.com/'>google</a>"
+ "<img src='file:///where/the/heck.jpg'/></body>";
+
+TEST_F(UrlLeftTrimFilterTest, NoChanges) {
+ left_trim_filter_.AddBaseUrl(kBase);
+ ValidateNoChanges("none forward", kNone);
+}
+
+static const char kSome[] =
+ "<head><base href='http://foo.bar/baz/'/>"
+ "<link src='http://foo.bar/baz/'></head>"
+ "<body><a href='http://www.google.com/'>google</a>"
+ "<img src='http://foo.bar/baz/nav.jpg'/>"
+ "<img src='http://foo.bar/img/img1.jpg'/>"
+ "<img src='/baz/img2.jpg'/>"
+ "<img src='//foo.bar/baz/widget.png'/></body>";
+
+static const char kSomeRewritten[] =
+ "<head><base href='http://foo.bar/baz/'/>"
+ "<link src='/baz/'></head>"
+ "<body><a href='//www.google.com/'>google</a>"
+ "<img src='nav.jpg'/>"
+ "<img src='/img/img1.jpg'/>"
+ "<img src='img2.jpg'/>"
+ "<img src='widget.png'/></body>";
+
+TEST_F(UrlLeftTrimFilterTest, SomeChanges) {
+ left_trim_filter_.AddBaseUrl(kBase);
+ ValidateExpected("some forward", kSome, kSomeRewritten);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/url_partnership.cc b/trunk/src/net/instaweb/rewriter/url_partnership.cc
new file mode 100644
index 0000000..24597a4
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/url_partnership.cc
@@ -0,0 +1,172 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/url_partnership.h"
+
+#include <algorithm> // for std::min
+#include <string>
+#include "net/instaweb/rewriter/public/domain_lawyer.h"
+#include "net/instaweb/rewriter/public/rewrite_options.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/stl_util.h"
+
+namespace net_instaweb {
+
+UrlPartnership::UrlPartnership(const RewriteOptions* rewrite_options,
+ const GURL& original_request)
+ : rewrite_options_(rewrite_options) {
+ if (original_request.is_valid()) {
+ original_origin_and_path_ = GoogleUrl::Create(
+ GoogleUrl::AllExceptLeaf(original_request));
+ }
+}
+
+UrlPartnership::~UrlPartnership() {
+ STLDeleteElements(&gurl_vector_);
+}
+
+// Adds a URL to a combination. If it can be legally added, consulting
+// the DomainLawyer, then true is returned. AddUrl cannot be called
+// after Resolve (CHECK failure).
+bool UrlPartnership::AddUrl(const StringPiece& untrimmed_resource_url,
+ MessageHandler* handler) {
+ std::string resource_url, mapped_domain_name;
+ bool ret = false;
+ TrimWhitespace(untrimmed_resource_url, &resource_url);
+
+ if (resource_url.empty()) {
+ handler->Message(kInfo, "Cannot rewrite empty URL relative to %s",
+ original_origin_and_path_.possibly_invalid_spec().c_str());
+ }
+ else if (!original_origin_and_path_.is_valid()) {
+ handler->Message(kInfo, "Cannot rewrite %s relative to invalid url %s",
+ resource_url.c_str(),
+ original_origin_and_path_.possibly_invalid_spec().c_str());
+ } else {
+ // First resolve the original request to ensure that it is allowed by the
+ // options.
+ GURL resolved_request = GoogleUrl::Resolve(original_origin_and_path_,
+ resource_url);
+ if (!resolved_request.is_valid()) {
+ handler->Message(
+ kInfo, "URL %s cannot be resolved relative to base URL %s",
+ resource_url.c_str(), original_origin_and_path_.spec().c_str());
+ } else if (!rewrite_options_->IsAllowed(
+ GoogleUrl::Spec(resolved_request))) {
+ handler->Message(kInfo,
+ "Rewriting URL %s is disallowed via configuration",
+ GoogleUrl::Spec(resolved_request).c_str());
+ } else if (rewrite_options_->domain_lawyer()->MapRequestToDomain(
+ original_origin_and_path_, resource_url, &mapped_domain_name,
+ &resolved_request, handler)) {
+ if (gurl_vector_.empty()) {
+ domain_.swap(mapped_domain_name);
+ domain_gurl_ = GoogleUrl::Create(domain_).Resolve(
+ GoogleUrl::Path(original_origin_and_path_));
+ ret = true;
+ } else {
+ ret = (domain_ == mapped_domain_name);
+ }
+
+ if (ret) {
+ gurl_vector_.push_back(new GURL(resolved_request));
+ int index = gurl_vector_.size() - 1;
+ IncrementalResolve(index);
+ }
+ }
+ }
+ return ret;
+}
+
+void UrlPartnership::RemoveLast() {
+ CHECK(!gurl_vector_.empty());
+ int last = gurl_vector_.size() - 1;
+ delete gurl_vector_[last];
+ gurl_vector_.resize(last);
+
+ // Re-resolve the entire partnership in the absense of the influence of the
+ // ex-partner, by re-adding the GURLs one at a time.
+ common_components_.clear();
+ for (int i = 0, n = gurl_vector_.size(); i < n; ++i) {
+ IncrementalResolve(i);
+ }
+}
+
+void UrlPartnership::IncrementalResolve(int index) {
+ CHECK_LE(0, index);
+ CHECK_LT(index, static_cast<int>(gurl_vector_.size()));
+
+ // When tokenizing a URL, we don't want to omit empty segments
+ // because we need to avoid aliasing "http://x" with "/http:/x".
+ bool omit_empty = false;
+ std::vector<StringPiece> components;
+
+ if (index == 0) {
+ std::string base = GoogleUrl::AllExceptLeaf(*gurl_vector_[0]);
+ SplitStringPieceToVector(base, "/", &components, omit_empty);
+ components.pop_back(); // base ends with "/"
+ CHECK_LE(3U, components.size()); // expect {"http:", "", "x"...}
+ for (size_t i = 0; i < components.size(); ++i) {
+ const StringPiece& sp = components[i];
+ common_components_.push_back(std::string(sp.data(), sp.size()));
+ }
+ } else {
+ // Split each string on / boundaries, then compare these path elements
+ // until one doesn't match, then shortening common_components.
+ std::string all_but_leaf = GoogleUrl::AllExceptLeaf(*gurl_vector_[index]);
+ SplitStringPieceToVector(all_but_leaf, "/", &components, omit_empty);
+ components.pop_back(); // base ends with "/"
+ CHECK_LE(3U, components.size()); // expect {"http:", "", "x"...}
+
+ if (components.size() < common_components_.size()) {
+ common_components_.resize(components.size());
+ }
+ for (size_t c = 0; c < common_components_.size(); ++c) {
+ if (common_components_[c] != components[c]) {
+ common_components_.resize(c);
+ break;
+ }
+ }
+ }
+}
+
+std::string UrlPartnership::ResolvedBase() const {
+ std::string ret;
+ if (!common_components_.empty()) {
+ for (size_t c = 0; c < common_components_.size(); ++c) {
+ const std::string& component = common_components_[c];
+ ret += component;
+ ret += "/"; // initial segment is "http" with no leading /
+ }
+ }
+ return ret;
+}
+
+// Returns the relative path of a particular URL that was added into
+// the partnership. This requires that Resolve() be called first.
+std::string UrlPartnership::RelativePath(int index) const {
+ std::string resolved_base = ResolvedBase();
+ std::string spec = gurl_vector_[index]->spec();
+ CHECK_GE(spec.size(), resolved_base.size());
+ CHECK_EQ(StringPiece(spec.data(), resolved_base.size()),
+ StringPiece(resolved_base));
+ return std::string(spec.data() + resolved_base.size(),
+ spec.size() - resolved_base.size());
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/rewriter/url_partnership_test.cc b/trunk/src/net/instaweb/rewriter/url_partnership_test.cc
new file mode 100644
index 0000000..a641431
--- /dev/null
+++ b/trunk/src/net/instaweb/rewriter/url_partnership_test.cc
@@ -0,0 +1,231 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/rewriter/public/url_partnership.h"
+
+#include <string>
+#include "net/instaweb/rewriter/public/domain_lawyer.h"
+#include "net/instaweb/rewriter/public/rewrite_options.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace {
+
+const char kOriginalRequest[] = "http://www.nytimes.com/index.html";
+const char kResourceUrl1[] = "r/styles/style.css?appearance=reader/writer?";
+const char kResourceUrl2[] = "r/styles/style2.css?appearance=reader";
+const char kResourceUrl3[] = "r/main.css";
+const char kCdnResourceUrl[] = "http://graphics8.nytimes.com/styles/style.css";
+
+// Resources 1-3 but specified absolutely
+const char kAbsoluteResourceUrl1[] =
+ "http://www.nytimes.com/r/styles/style.css?appearance=reader/writer?";
+const char kAbsoluteResourceUrl2[] =
+ "http://www.nytimes.com/r/styles/style2.css?appearance=reader";
+const char kAbsoluteResourceUrl3[] = "http://www.nytimes.com/r/main.css";
+
+} // namespace
+
+
+namespace net_instaweb {
+
+class UrlPartnershipTest : public testing::Test {
+ protected:
+ UrlPartnershipTest()
+ : domain_lawyer_(options_.domain_lawyer()),
+ partnership_(&options_, GURL(kOriginalRequest)),
+ styles_path_("http://www.nytimes.com/r/styles/"),
+ r_path_("http://www.nytimes.com/r/"),
+ style_url_("style.css?appearance=reader/writer?"),
+ style2_url_("style2.css?appearance=reader") {
+ }
+
+ // Add up to 3 URLs -- url2 and url3 are ignored if null.
+ bool AddUrls(const char* url1, const char* url2, const char* url3) {
+ bool ret = partnership_.AddUrl(url1, &message_handler_);
+ if (url2 != NULL) {
+ ret &= partnership_.AddUrl(url2, &message_handler_);
+ }
+ if (url3 != NULL) {
+ ret &= partnership_.AddUrl(url3, &message_handler_);
+ }
+ return ret;
+ }
+
+ // Gets the full path of an index as a std::string.
+ std::string FullPath(int index) {
+ const GURL* gurl = partnership_.FullPath(index);
+ std::string spec = gurl->spec();
+ return std::string(spec.data(), spec.size());
+ }
+
+ RewriteOptions options_;
+ DomainLawyer* domain_lawyer_;
+ UrlPartnership partnership_;
+ std::string styles_path_;
+ std::string r_path_;
+ std::string style_url_;
+ std::string style2_url_;
+ GoogleMessageHandler message_handler_;
+};
+
+TEST_F(UrlPartnershipTest, OneUrlFlow) {
+ ASSERT_TRUE(AddUrls(kResourceUrl1, NULL, NULL));
+ ASSERT_EQ(1, partnership_.num_urls());
+ EXPECT_EQ(styles_path_, partnership_.ResolvedBase());
+ EXPECT_EQ(style_url_, partnership_.RelativePath(0));
+ EXPECT_EQ(styles_path_ + style_url_, FullPath(0));
+}
+
+TEST_F(UrlPartnershipTest, OneUrlFlowAbsolute) {
+ ASSERT_TRUE(AddUrls(kAbsoluteResourceUrl1, NULL, NULL));
+ ASSERT_EQ(1, partnership_.num_urls());
+ EXPECT_EQ(styles_path_, partnership_.ResolvedBase());
+ EXPECT_EQ(style_url_, partnership_.RelativePath(0));
+ EXPECT_EQ(styles_path_ + style_url_, FullPath(0));
+}
+
+TEST_F(UrlPartnershipTest, TwoUrlFlowSamePath) {
+ AddUrls(kResourceUrl1, kResourceUrl2, NULL);
+ ASSERT_EQ(2, partnership_.num_urls());
+ EXPECT_EQ(styles_path_, partnership_.ResolvedBase());
+ EXPECT_EQ(style_url_, partnership_.RelativePath(0));
+ EXPECT_EQ(style2_url_, partnership_.RelativePath(1));
+}
+
+TEST_F(UrlPartnershipTest, TwoUrlFlowSamePathMixed) {
+ AddUrls(kAbsoluteResourceUrl1, kResourceUrl2, NULL);
+ ASSERT_EQ(2, partnership_.num_urls());
+ EXPECT_EQ(styles_path_, partnership_.ResolvedBase());
+ EXPECT_EQ(style_url_, partnership_.RelativePath(0));
+ EXPECT_EQ(style2_url_, partnership_.RelativePath(1));
+}
+
+TEST_F(UrlPartnershipTest, ThreeUrlFlowDifferentPaths) {
+ AddUrls(kResourceUrl1, kResourceUrl2, kResourceUrl3);
+ ASSERT_EQ(3, partnership_.num_urls());
+ EXPECT_EQ(r_path_, partnership_.ResolvedBase());
+ // We add 2 to the expected values of the 3 kResourceUrl* below to
+ // skip over the "r/".
+ EXPECT_EQ(std::string(kResourceUrl1 + 2), partnership_.RelativePath(0));
+ EXPECT_EQ(std::string(kResourceUrl2 + 2), partnership_.RelativePath(1));
+ EXPECT_EQ(std::string(kResourceUrl3 + 2), partnership_.RelativePath(2));
+}
+
+TEST_F(UrlPartnershipTest, ThreeUrlFlowDifferentPathsAbsolute) {
+ AddUrls(kAbsoluteResourceUrl1, kAbsoluteResourceUrl2, kAbsoluteResourceUrl3);
+ ASSERT_EQ(3, partnership_.num_urls());
+ EXPECT_EQ(r_path_, partnership_.ResolvedBase());
+ // We add 2 to the expected values of the 3 kResourceUrl* below to
+ // skip over the "r/".
+ EXPECT_EQ(std::string(kResourceUrl1 + 2), partnership_.RelativePath(0));
+ EXPECT_EQ(std::string(kResourceUrl2 + 2), partnership_.RelativePath(1));
+ EXPECT_EQ(std::string(kResourceUrl3 + 2), partnership_.RelativePath(2));
+}
+
+TEST_F(UrlPartnershipTest, ThreeUrlFlowDifferentPathsMixed) {
+ AddUrls(kAbsoluteResourceUrl1, kResourceUrl2, kAbsoluteResourceUrl3);
+ ASSERT_EQ(3, partnership_.num_urls());
+ EXPECT_EQ(r_path_, partnership_.ResolvedBase());
+ // We add 2 to the expected values of the 3 kResourceUrl* below to
+ // skip over the "r/".
+ EXPECT_EQ(std::string(kResourceUrl1 + 2), partnership_.RelativePath(0));
+ EXPECT_EQ(std::string(kResourceUrl2 + 2), partnership_.RelativePath(1));
+ EXPECT_EQ(std::string(kResourceUrl3 + 2), partnership_.RelativePath(2));
+}
+
+TEST_F(UrlPartnershipTest, ExternalDomainNotDeclared) {
+ EXPECT_FALSE(AddUrls(kCdnResourceUrl, NULL, NULL));
+}
+
+TEST_F(UrlPartnershipTest, ExternalDomainDeclared) {
+ domain_lawyer_->AddDomain("http://graphics8.nytimes.com", &message_handler_);
+ EXPECT_TRUE(partnership_.AddUrl(kCdnResourceUrl, &message_handler_));
+}
+
+TEST_F(UrlPartnershipTest, ExternalDomainDeclaredButNotMapped) {
+ // This test shows that while we can start partnerships from nytimes.com
+ // or graphics8.nytimes.com, we cannot combine those without a mapping.
+ domain_lawyer_->AddDomain("http://graphics8.nytimes.com", &message_handler_);
+ EXPECT_TRUE(partnership_.AddUrl(kCdnResourceUrl, &message_handler_));
+ EXPECT_FALSE(partnership_.AddUrl(kResourceUrl1, &message_handler_));
+}
+
+TEST_F(UrlPartnershipTest, AbsExternalDomainDeclaredButNotMapped) {
+ // This test shows that while we can start partnerships from nytimes.com
+ // or graphics8.nytimes.com, we cannot combine those without a mapping.
+ domain_lawyer_->AddDomain("http://graphics8.nytimes.com", &message_handler_);
+ EXPECT_TRUE(partnership_.AddUrl(kCdnResourceUrl, &message_handler_));
+ EXPECT_FALSE(partnership_.AddUrl(kAbsoluteResourceUrl1, &message_handler_));
+}
+
+TEST_F(UrlPartnershipTest, EmptyTail) {
+ EXPECT_FALSE(partnership_.AddUrl("", &message_handler_));
+ EXPECT_TRUE(partnership_.AddUrl("http://www.nytimes.com",
+ &message_handler_));
+ EXPECT_TRUE(partnership_.AddUrl("http://www.nytimes.com/",
+ &message_handler_));
+}
+
+TEST_F(UrlPartnershipTest, EmptyWithPartner) {
+ UrlPartnership p(&options_, GURL("http://www.google.com/styles/x.html"));
+ EXPECT_TRUE(p.AddUrl("/styles", &message_handler_));
+ EXPECT_FALSE(p.AddUrl("", &message_handler_));
+ EXPECT_TRUE(p.AddUrl("/", &message_handler_));
+ EXPECT_TRUE(p.AddUrl("..", &message_handler_));
+}
+
+TEST_F(UrlPartnershipTest, NeedsATrim) {
+ AddUrls(" http://www.nytimes.com/needs_a_trim.jpg ", NULL, NULL);
+ EXPECT_EQ(std::string("needs_a_trim.jpg"), partnership_.RelativePath(0));
+}
+
+TEST_F(UrlPartnershipTest, RemoveLast) {
+ AddUrls(kAbsoluteResourceUrl1, kAbsoluteResourceUrl2, kAbsoluteResourceUrl3);
+ EXPECT_EQ(r_path_, partnership_.ResolvedBase());
+ partnership_.RemoveLast();
+ EXPECT_EQ(styles_path_, partnership_.ResolvedBase());
+}
+
+TEST_F(UrlPartnershipTest, ResourcesFromMappedDomains) {
+ domain_lawyer_->AddRewriteDomainMapping(
+ "http://graphics8.nytimes.com", "http://www.nytimes.com",
+ &message_handler_);
+ domain_lawyer_->AddRewriteDomainMapping(
+ "http://graphics8.nytimes.com", "http://styles.com", &message_handler_);
+
+ // We can legally combine resources across multiple domains if they are
+ // all mapped together
+ ASSERT_TRUE(AddUrls(kCdnResourceUrl, kResourceUrl1,
+ "http://styles.com/external.css"));
+ EXPECT_EQ("http://graphics8.nytimes.com/", partnership_.ResolvedBase());
+}
+
+TEST_F(UrlPartnershipTest, AllowDisallow) {
+ // This test shows that while we can start partnerships from nytimes.com
+ // or graphics8.nytimes.com, we cannot combine those without a mapping.
+ domain_lawyer_->AddDomain("http://graphics8.nytimes.com", &message_handler_);
+ options_.Disallow("*/*.css");
+ options_.Allow("*/a*.css");
+ EXPECT_FALSE(partnership_.AddUrl("foo.css", &message_handler_));
+ EXPECT_TRUE(partnership_.AddUrl("afoo.css", &message_handler_));
+ EXPECT_TRUE(partnership_.AddUrl("foo.jpg", &message_handler_));
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/abstract_mutex.cc b/trunk/src/net/instaweb/util/abstract_mutex.cc
new file mode 100644
index 0000000..1f5d81b
--- /dev/null
+++ b/trunk/src/net/instaweb/util/abstract_mutex.cc
@@ -0,0 +1,26 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/abstract_mutex.h"
+
+namespace net_instaweb {
+
+AbstractMutex::~AbstractMutex() {
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/base64_test.cc b/trunk/src/net/instaweb/util/base64_test.cc
new file mode 100644
index 0000000..50c5033
--- /dev/null
+++ b/trunk/src/net/instaweb/util/base64_test.cc
@@ -0,0 +1,143 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the base64 encoder.
+
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "net/instaweb/util/public/base64_util.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/gtest.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/string_writer.h"
+
+namespace {
+
+const char chinese_data[] = "中华网,中华,中国,中文网,中国新闻,香港新闻,"
+ "国际新闻,中文新闻,新闻,港台新闻,两会,嫦娥一号";
+const int chinese_size = sizeof(chinese_data) - 1;
+
+// Also test some binary data, including embedded nulls, 2^7-1, 2^8-1
+const char binary_data[] = "\0\1\2\3\4\5\6\7\10\0\177\176\175\377\376";
+const int binary_size = sizeof(binary_data) - 1;
+
+class Codec {
+ public:
+ virtual ~Codec() {}
+ virtual void encode(const std::string& in, std::string* out) const = 0;
+ virtual bool decode(const std::string& in, std::string* out) const = 0;
+};
+
+class WebSafeBase64Codec : public Codec {
+ public:
+ virtual void encode(const std::string& in, std::string* out) const {
+ net_instaweb::Web64Encode(in, out);
+ }
+ virtual bool decode(const std::string& in, std::string* out) const {
+ return net_instaweb::Web64Decode(in, out);
+ }
+};
+
+class MimeBase64Codec : public Codec {
+ public:
+ virtual void encode(const std::string& in, std::string* out) const {
+ net_instaweb::Mime64Encode(in, out);
+ }
+ virtual bool decode(const std::string& in, std::string* out) const {
+ return net_instaweb::Mime64Decode(in, out);
+ }
+};
+
+} // namespace
+
+namespace net_instaweb {
+
+class Base64Test : public testing::Test {
+ protected:
+ Base64Test()
+ : chinese_(chinese_data, chinese_size),
+ binary_(binary_data, binary_size),
+ web64_codec_(),
+ mime64_codec_() {
+ }
+
+ void TestWeb64(const Codec &codec, const std::string& input) {
+ std::string encoded, decoded;
+ codec.encode(input, &encoded);
+ ASSERT_TRUE(codec.decode(encoded, &decoded));
+ EXPECT_EQ(input, decoded);
+ }
+
+ // Tests that attempts to decode a string that is not properly base64
+ // encoded will gracefully fail (Web64Decode returns false) rather than
+ // crash or produce invalid output. corrupt_char must be a character
+ // that is not in the base64 char-set.
+ //
+ // If the 'index' is specified as a negative number, it will be taken
+ // as an offset from the end of the string.
+ void TestCorrupt(const Codec &codec,
+ const std::string& input, char corrupt_char, int index) {
+ std::string encoded, decoded;
+ codec.encode(input, &encoded);
+ if (index < 0) {
+ index = encoded.size() + index;
+ }
+ encoded[index] = corrupt_char;
+ ASSERT_FALSE(codec.decode(encoded, &decoded));
+ }
+
+ std::string chinese_;
+ std::string binary_;
+ WebSafeBase64Codec web64_codec_;
+ MimeBase64Codec mime64_codec_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(Base64Test);
+};
+
+TEST_F(Base64Test, Chinese) {
+ TestWeb64(web64_codec_, chinese_);
+ TestWeb64(mime64_codec_, chinese_);
+}
+
+TEST_F(Base64Test, Binary) {
+ TestWeb64(web64_codec_, binary_);
+ TestWeb64(mime64_codec_, binary_);
+}
+
+TEST_F(Base64Test, CorruptFirst) {
+ TestCorrupt(web64_codec_, chinese_, '@', 0);
+ TestCorrupt(mime64_codec_, chinese_, '@', 0);
+}
+
+TEST_F(Base64Test, CorruptMiddle) {
+ TestCorrupt(web64_codec_, chinese_, ':', chinese_.size() / 2);
+ TestCorrupt(mime64_codec_, chinese_, ':', chinese_.size() / 2);
+}
+
+TEST_F(Base64Test, CorruptEnd) {
+ // I wanted to put the '/' as the last character, but it turns out
+ // that encoders may put '=' characters in to pad to a multiple of
+ // 4 bytes, and the decoder stops decoding when it gets to the first
+ // pad character, so changing "==" to "=/" has no effect.
+ TestCorrupt(web64_codec_, chinese_, '/', -4);
+ TestCorrupt(mime64_codec_, chinese_, '_', -4);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/cache_fetcher_test.cc b/trunk/src/net/instaweb/util/cache_fetcher_test.cc
new file mode 100644
index 0000000..a694cd4
--- /dev/null
+++ b/trunk/src/net/instaweb/util/cache_fetcher_test.cc
@@ -0,0 +1,25 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/cache_fetcher_test.h"
+
+namespace net_instaweb {
+
+const int CacheFetcherTest::kMaxSize = 10000;
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/cache_fetcher_test.h b/trunk/src/net/instaweb/util/cache_fetcher_test.h
new file mode 100644
index 0000000..97a1e73
--- /dev/null
+++ b/trunk/src/net/instaweb/util/cache_fetcher_test.h
@@ -0,0 +1,58 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test framework for caching fetchers. This is used by
+// both cache_url_fetcher_test.cc and cache_url_async_fetcher_test.cc.
+
+#ifndef NET_INSTAWEB_UTIL_CACHE_FETCHER_TEST_H_
+#define NET_INSTAWEB_UTIL_CACHE_FETCHER_TEST_H_
+
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "net/instaweb/util/fetcher_test.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/http_cache.h"
+#include "net/instaweb/util/public/lru_cache.h"
+#include "net/instaweb/util/public/mock_timer.h"
+
+namespace net_instaweb {
+
+class CacheFetcherTest : public FetcherTest {
+ protected:
+ static const int kMaxSize;
+
+ CacheFetcherTest()
+ : mock_timer_(0),
+ http_cache_(new LRUCache(kMaxSize), &mock_timer_) {
+ int64 start_time_ms;
+ bool parsed = MetaData::ParseTime(kStartDate, &start_time_ms);
+ CHECK(parsed);
+ mock_timer_.set_time_ms(start_time_ms);
+ }
+
+ MockTimer mock_timer_;
+ HTTPCache http_cache_;
+ GoogleMessageHandler message_handler_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(CacheFetcherTest);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_CACHE_FETCHER_TEST_H_
diff --git a/trunk/src/net/instaweb/util/cache_interface.cc b/trunk/src/net/instaweb/util/cache_interface.cc
new file mode 100644
index 0000000..16ca26d
--- /dev/null
+++ b/trunk/src/net/instaweb/util/cache_interface.cc
@@ -0,0 +1,26 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/cache_interface.h"
+
+namespace net_instaweb {
+
+CacheInterface::~CacheInterface() {
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/cache_url_async_fetcher.cc b/trunk/src/net/instaweb/util/cache_url_async_fetcher.cc
new file mode 100644
index 0000000..d04b345
--- /dev/null
+++ b/trunk/src/net/instaweb/util/cache_url_async_fetcher.cc
@@ -0,0 +1,112 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/cache_url_async_fetcher.h"
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/cache_url_fetcher.h"
+#include "net/instaweb/util/public/http_cache.h"
+#include "net/instaweb/util/public/http_value.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/string_writer.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// This version of the caching async fetcher callback uses the
+// caller-supplied response-header buffer, and also forwards the
+// content to the client before caching it.
+class ForwardingAsyncFetch : public CacheUrlFetcher::AsyncFetch {
+ public:
+ ForwardingAsyncFetch(const StringPiece& url, HTTPCache* cache,
+ MessageHandler* handler, Callback* callback,
+ Writer* writer, MetaData* response_headers,
+ bool force_caching)
+ : CacheUrlFetcher::AsyncFetch(url, cache, handler, force_caching),
+ callback_(callback),
+ client_writer_(writer),
+ response_headers_(response_headers) {
+ }
+
+ virtual void Done(bool success) {
+ // Copy the data to the client even with a failure; there may be useful
+ // error messages in the content.
+ StringPiece contents;
+ if (value_.ExtractContents(&contents)) {
+ client_writer_->Write(contents, message_handler_);
+ } else {
+ success = false;
+ }
+
+ // Update the cache before calling the client Done callback, which might
+ // delete the headers. Note that if the value is not cacheable, we will
+ // record in the cache that this entry is not cacheable, but we will still
+ // call the async fetcher callback with the value. This allows us to
+ // do resource-serving via CacheUrlAsyncFetcher, where we need to serve
+ // the resources even if they are not cacheable.
+ //
+ // We do not update the cache at all if the fetch failed.
+ if (success) {
+ UpdateCache();
+ }
+
+ callback_->Done(success);
+ delete this;
+ }
+
+ virtual MetaData* ResponseHeaders() { return response_headers_; }
+
+ private:
+ Callback* callback_;
+ Writer* client_writer_;
+ MetaData* response_headers_;
+
+ DISALLOW_COPY_AND_ASSIGN(ForwardingAsyncFetch);
+};
+
+} // namespace
+
+CacheUrlAsyncFetcher::~CacheUrlAsyncFetcher() {
+}
+
+bool CacheUrlAsyncFetcher::StreamingFetch(
+ const std::string& url, const MetaData& request_headers,
+ MetaData* response_headers, Writer* writer, MessageHandler* handler,
+ Callback* callback) {
+ HTTPValue value;
+ StringPiece contents;
+ bool ret = false;
+ if ((http_cache_->Find(url.c_str(), &value, response_headers, handler) ==
+ HTTPCache::kFound) &&
+ !CacheUrlFetcher::RememberNotCached(*response_headers) &&
+ value.ExtractContents(&contents)) {
+ bool success = writer->Write(contents, handler);
+ callback->Done(success);
+ ret = true;
+ } else {
+ response_headers->Clear();
+ ForwardingAsyncFetch* fetch = new ForwardingAsyncFetch(
+ url, http_cache_, handler, callback, writer, response_headers,
+ force_caching_);
+ fetch->Start(fetcher_, request_headers);
+ }
+ return ret;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/cache_url_async_fetcher_test.cc b/trunk/src/net/instaweb/util/cache_url_async_fetcher_test.cc
new file mode 100644
index 0000000..9dd66ca
--- /dev/null
+++ b/trunk/src/net/instaweb/util/cache_url_async_fetcher_test.cc
@@ -0,0 +1,105 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the async caching fetcher, using a mock fetcher, and an async
+// wrapper around that.
+
+#include "net/instaweb/util/public/cache_url_async_fetcher.h"
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/cache_fetcher_test.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+
+namespace net_instaweb {
+
+class CacheUrlAsyncFetcherTest : public CacheFetcherTest {
+ protected:
+ CacheUrlAsyncFetcherTest()
+ : cache_fetcher_(&http_cache_, &mock_async_fetcher_) {
+ }
+
+ virtual UrlAsyncFetcher* async_fetcher() { return &cache_fetcher_; }
+
+ CacheUrlAsyncFetcher cache_fetcher_;
+};
+
+TEST_F(CacheUrlAsyncFetcherTest, TestAsyncCache) {
+ // With the async cached fetching interface, we will expect even the
+ // initial request to succeed, once the callbacks are run.
+ bool callback_called1, callback_called2, callback_called3;
+ EXPECT_EQ(1, CountFetchesAsync(kGoodUrl, true, &callback_called1));
+ EXPECT_FALSE(callback_called1);
+
+ EXPECT_EQ(1, CountFetchesAsync(kGoodUrl, true, &callback_called2));
+ EXPECT_FALSE(callback_called1);
+ EXPECT_FALSE(callback_called2);
+
+ mock_async_fetcher_.CallCallbacks();
+ EXPECT_TRUE(callback_called1);
+ EXPECT_TRUE(callback_called2);
+
+ EXPECT_EQ(0, CountFetchesAsync(kGoodUrl, true, &callback_called3));
+ // No async fetcher callbacks were queued because the content
+ // was cached, so no need to call CallCallbacks() again here.
+ EXPECT_TRUE(callback_called3);
+}
+
+TEST_F(CacheUrlAsyncFetcherTest, TestAsyncNotCached) {
+ // With the async cached fetching interface, we will expect even the
+ // initial request to succeed, once the callbacks are run.
+ bool callback_called1, callback_called2, callback_called3;
+ EXPECT_EQ(1, CountFetchesAsync(kNotCachedUrl, true, &callback_called1));
+ EXPECT_FALSE(callback_called1);
+
+ EXPECT_EQ(1, CountFetchesAsync(kNotCachedUrl, true, &callback_called2));
+ EXPECT_FALSE(callback_called1);
+ EXPECT_FALSE(callback_called2);
+
+ mock_async_fetcher_.CallCallbacks();
+ EXPECT_TRUE(callback_called1);
+ EXPECT_TRUE(callback_called2);
+
+ // This is not a proper cache and does not distinguish between cacheable
+ // or non-cacheable URLs.
+ EXPECT_EQ(1, CountFetchesAsync(kNotCachedUrl, true, &callback_called3));
+ EXPECT_FALSE(callback_called3);
+ mock_async_fetcher_.CallCallbacks(); // Otherwise memory will be leaked
+ EXPECT_TRUE(callback_called3);
+}
+
+TEST_F(CacheUrlAsyncFetcherTest, TestCacheWithASyncFetcherFail) {
+ bool callback_called1, callback_called2, callback_called3;
+
+ EXPECT_EQ(1, CountFetchesAsync(kBadUrl, false, &callback_called1));
+ EXPECT_FALSE(callback_called1);
+
+ EXPECT_EQ(1, CountFetchesAsync(kBadUrl, false, &callback_called2));
+ EXPECT_FALSE(callback_called1);
+ EXPECT_FALSE(callback_called2);
+
+ mock_async_fetcher_.CallCallbacks();
+ EXPECT_TRUE(callback_called1);
+ EXPECT_TRUE(callback_called2);
+
+ EXPECT_EQ(1, CountFetchesAsync(kBadUrl, false, &callback_called3));
+ EXPECT_FALSE(callback_called3);
+ mock_async_fetcher_.CallCallbacks(); // Otherwise memory will be leaked
+ EXPECT_TRUE(callback_called3);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/cache_url_fetcher.cc b/trunk/src/net/instaweb/util/cache_url_fetcher.cc
new file mode 100644
index 0000000..45aa8bc
--- /dev/null
+++ b/trunk/src/net/instaweb/util/cache_url_fetcher.cc
@@ -0,0 +1,184 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/cache_url_fetcher.h"
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/http_cache.h"
+#include "net/instaweb/util/public/http_value.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/timer.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+
+namespace net_instaweb {
+
+namespace {
+
+const char kRememberNotCached[] = "X-Instaweb-Disable-cache";
+
+// The synchronous version of the caching fetch must supply a
+// response_headers buffer that will still be valid at when the fetch
+// completes and the callback executes.
+class AsyncFetchWithHeaders : public CacheUrlFetcher::AsyncFetch {
+ public:
+ AsyncFetchWithHeaders(const StringPiece& url, HTTPCache* cache,
+ MessageHandler* handler, bool force_caching)
+ : CacheUrlFetcher::AsyncFetch(url, cache, handler, force_caching) {
+ }
+
+ virtual MetaData* ResponseHeaders() {
+ return &response_headers_;
+ }
+ private:
+ SimpleMetaData response_headers_;
+
+ DISALLOW_COPY_AND_ASSIGN(AsyncFetchWithHeaders);
+};
+
+} // namespace
+
+CacheUrlFetcher::AsyncFetch::AsyncFetch(const StringPiece& url,
+ HTTPCache* cache,
+ MessageHandler* handler,
+ bool force_caching)
+ : message_handler_(handler),
+ url_(url.data(), url.size()),
+ http_cache_(cache),
+ force_caching_(force_caching) {
+}
+
+CacheUrlFetcher::AsyncFetch::~AsyncFetch() {
+}
+
+/*
+ * Note: this can be called from a different thread than the one where
+ * the request was made. We are depending on the caches being thread-safe
+ * if necessary.
+ */
+void CacheUrlFetcher::AsyncFetch::UpdateCache() {
+ // TODO(jmarantz): allow configuration of whether we ignore
+ // IsProxyCacheable, e.g. for content served from the same host
+ MetaData* response_headers = ResponseHeaders();
+ if ((http_cache_->Query(url_.c_str()) == CacheInterface::kNotFound)) {
+ if (force_caching_ || response_headers->IsProxyCacheable()) {
+ value_.SetHeaders(*response_headers);
+ http_cache_->Put(url_.c_str(), &value_, message_handler_);
+ } else {
+ // Leave value_ alone as we prep a cache entry to indicate that
+ // this url is not cacheable. This is because this code is
+ // shared with cache_url_async_fetcher.cc, which needs to
+ // actually pass through the real value and headers, even while
+ // remembering the non-cachability of the URL.
+ HTTPValue dummy_value;
+ SimpleMetaData remember_not_cached;
+
+ // We need to set the header status code to 'OK' to satisfy
+ // HTTPCache::IsCurrentlyValid. We rely on the detection of the
+ // header X-Instaweb-Disable-cache to avoid letting this escape into
+ // the wild. We may want to revisit if this proves problematic.
+ remember_not_cached.SetStatusAndReason(HttpStatus::kOK);
+ remember_not_cached.SetDate(http_cache_->timer()->NowMs());
+ remember_not_cached.Add("Cache-control", "max-age=300");
+ remember_not_cached.Add(kRememberNotCached, "1"); // value doesn't matter
+ dummy_value.Write("", message_handler_);
+ dummy_value.SetHeaders(remember_not_cached);
+ http_cache_->Put(url_.c_str(), &dummy_value, message_handler_);
+ }
+ }
+}
+
+void CacheUrlFetcher::AsyncFetch::Done(bool success) {
+ if (success) {
+ UpdateCache();
+ } else {
+ message_handler_->Info(url_.c_str(), 0, "Fetch failed, not caching.");
+ // TODO(jmarantz): cache that this request is not fetchable
+ }
+ delete this;
+}
+
+bool CacheUrlFetcher::AsyncFetch::EnableThreaded() const {
+ // Our cache implementations are thread-safe, so it's OK to update
+ // them asynchronously.
+ return true;
+}
+
+void CacheUrlFetcher::AsyncFetch::Start(
+ UrlAsyncFetcher* fetcher, const MetaData& request_headers) {
+ fetcher->StreamingFetch(url_, request_headers, ResponseHeaders(),
+ &value_, message_handler_, this);
+}
+
+CacheUrlFetcher::~CacheUrlFetcher() {
+}
+
+bool CacheUrlFetcher::RememberNotCached(const MetaData& headers) {
+ CharStarVector not_cached_values;
+ return headers.Lookup(kRememberNotCached, ¬_cached_values);
+}
+
+bool CacheUrlFetcher::StreamingFetchUrl(
+ const std::string& url, const MetaData& request_headers,
+ MetaData* response_headers, Writer* writer, MessageHandler* handler) {
+ bool ret = false;
+ HTTPValue value;
+ StringPiece contents;
+ ret = ((http_cache_->Find(url.c_str(), &value, response_headers, handler)
+ == HTTPCache::kFound) &&
+ value.ExtractContents(&contents));
+ if (ret) {
+ // If we have remembered that this value is not cachable, then mutate
+ // the reponse code and return false. Note that we must use an X-code
+ // in the headers, rather than the status code, so that HTTPCache will
+ // not reject the item on retrieval, spoiling our ability to remember
+ // fact that the item is uncacheable.
+ if (RememberNotCached(*response_headers)) {
+ response_headers->SetStatusAndReason(HttpStatus::kUnavailable);
+ ret = false;
+ } else {
+ ret = writer->Write(contents, handler);
+ }
+ } else if (sync_fetcher_ != NULL) {
+ // We need to hang onto a copy of the data so we can shove it
+ // into the cache, which lacks a streaming Put.
+ std::string content;
+ StringWriter string_writer(&content);
+ ret = sync_fetcher_->StreamingFetchUrl(
+ url, request_headers, response_headers, &string_writer, handler);
+ ret &= writer->Write(content, handler);
+ if (ret) {
+ if (force_caching_ || response_headers->IsProxyCacheable()) {
+ value.Clear();
+ value.SetHeaders(*response_headers);
+ value.Write(content, handler);
+ http_cache_->Put(url.c_str(), &value, handler);
+ }
+ } else {
+ // TODO(jmarantz): Consider caching that this request is not fetchable
+ }
+ } else {
+ AsyncFetch* fetch = new AsyncFetchWithHeaders(url, http_cache_, handler,
+ force_caching_);
+ fetch->Start(async_fetcher_, request_headers);
+ }
+ return ret;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/cache_url_fetcher_test.cc b/trunk/src/net/instaweb/util/cache_url_fetcher_test.cc
new file mode 100644
index 0000000..3274e76
--- /dev/null
+++ b/trunk/src/net/instaweb/util/cache_url_fetcher_test.cc
@@ -0,0 +1,99 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the caching fetcher, using a mock fetcher, and an async
+// wrapper around that.
+
+#include "net/instaweb/util/public/cache_url_fetcher.h"
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/cache_fetcher_test.h"
+
+namespace net_instaweb {
+
+class CacheUrlFetcherTest : public CacheFetcherTest {};
+
+TEST_F(CacheUrlFetcherTest, TestCachableWithSyncFetcher) {
+ CacheUrlFetcher cache_fetcher(&http_cache_, &mock_fetcher_);
+ EXPECT_EQ(1, CountFetchesSync(kGoodUrl, &cache_fetcher, true, true));
+ EXPECT_EQ(0, CountFetchesSync(kGoodUrl, &cache_fetcher, true, true));
+}
+
+TEST_F(CacheUrlFetcherTest, TestNonCachableWithSyncFetcher) {
+ // When a CacheUrlFetcher is implemented using a sync fetcher,
+ // then non-cacheable URLs will result in sync-fetch successes
+ // that are not cached.
+ // so we will not do the fetch the second time.
+ CacheUrlFetcher cache_fetcher(&http_cache_, &mock_fetcher_);
+ EXPECT_EQ(1, CountFetchesSync(kNotCachedUrl, &cache_fetcher, true, true));
+ EXPECT_EQ(1, CountFetchesSync(kNotCachedUrl, &cache_fetcher, true, true));
+}
+
+TEST_F(CacheUrlFetcherTest, TestCacheWithSyncFetcherFail) {
+ CacheUrlFetcher cache_fetcher(&http_cache_, &mock_fetcher_);
+ EXPECT_EQ(1, CountFetchesSync(kBadUrl, &cache_fetcher, false, true));
+ // For now, we don't cache failure, so we expect a new fetch
+ // on each request
+ EXPECT_EQ(1, CountFetchesSync(kBadUrl, &cache_fetcher, false, true));
+}
+
+TEST_F(CacheUrlFetcherTest, TestCachableWithASyncFetcher) {
+ CacheUrlFetcher cache_fetcher(&http_cache_, &mock_async_fetcher_);
+
+ // The first fetch from the cache fetcher with the async interface
+ // will not succeed (arg==false), and it will queue up an async fetch
+ EXPECT_EQ(1, CountFetchesSync(kGoodUrl, &cache_fetcher, false, false));
+
+ // The second one will behave exactly the same, because we have
+ // not let the async callback run yet.
+ EXPECT_EQ(1, CountFetchesSync(kGoodUrl, &cache_fetcher, false, false));
+
+ // Now we call the callbacks, and so subsequent requests to the
+ // the cache will succeed (arg==true) and will yield 0 new fetch
+ // requests.
+ mock_async_fetcher_.CallCallbacks();
+ EXPECT_EQ(0, CountFetchesSync(kGoodUrl, &cache_fetcher, true, false));
+}
+
+TEST_F(CacheUrlFetcherTest, TestNonCachableWithASyncFetcher) {
+ CacheUrlFetcher cache_fetcher(&http_cache_, &mock_async_fetcher_);
+
+ // The first fetch from the cache fetcher with the async interface
+ // will not succeed (arg==false), and it will queue up an async fetch
+ EXPECT_EQ(1, CountFetchesSync(kNotCachedUrl, &cache_fetcher, false, false));
+ EXPECT_EQ(1, CountFetchesSync(kNotCachedUrl, &cache_fetcher, false, false));
+ mock_async_fetcher_.CallCallbacks();
+
+ // When a CacheUrlFetcher is implemented using an async fetcher,
+ // then non-cacheable URLs will result in sync-fetch failures,
+ // but we will remember the failure so we don't have to do the
+ // second fetch.
+ EXPECT_EQ(0, CountFetchesSync(kNotCachedUrl, &cache_fetcher, false, false));
+}
+
+TEST_F(CacheUrlFetcherTest, TestCacheWithASyncFetcherFail) {
+ CacheUrlFetcher cache_fetcher(&http_cache_, &mock_async_fetcher_);
+
+ EXPECT_EQ(1, CountFetchesSync(kBadUrl, &cache_fetcher, false, false));
+ EXPECT_EQ(1, CountFetchesSync(kBadUrl, &cache_fetcher, false, false));
+ mock_async_fetcher_.CallCallbacks();
+ EXPECT_EQ(1, CountFetchesSync(kBadUrl, &cache_fetcher, false, false));
+ mock_async_fetcher_.CallCallbacks(); // Otherwise memory will be leaked
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/content_type.cc b/trunk/src/net/instaweb/util/content_type.cc
new file mode 100644
index 0000000..5716e69
--- /dev/null
+++ b/trunk/src/net/instaweb/util/content_type.cc
@@ -0,0 +1,134 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/public/content_type.h"
+
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+namespace {
+
+const ContentType kTypes[] = {
+ // Canonical types:
+ {"text/html", ".html", ContentType::kHtml}, // RFC 2854
+ {"application/xhtml+xml", ".xhtml", ContentType::kXhtml}, // RFC 3236
+ {"application/ce-html+xml", ".xhtml", ContentType::kCeHtml},
+
+ {"text/javascript", ".js", ContentType::kJavascript},
+ {"text/css", ".css", ContentType::kCss},
+ {"text/plain", ".txt", ContentType::kText},
+ {"text/xml", ".xml", ContentType::kXml}, // RFC 3023
+
+ {"image/png", ".png", ContentType::kPng},
+ {"image/gif", ".gif", ContentType::kGif},
+ {"image/jpeg", ".jpg", ContentType::kJpeg},
+
+ // Synonyms; Note that the canonical types are referenced by index
+ // in the named references declared below.
+ {"application/x-javascript", ".js", ContentType::kJavascript},
+ {"application/javascript", ".js", ContentType::kJavascript},
+ {"text/ecmascript", ".js", ContentType::kJavascript},
+ {"application/ecmascript", ".js", ContentType::kJavascript},
+ {"image/jpeg", ".jpeg", ContentType::kJpeg},
+ {"text/html", ".htm", ContentType::kHtml},
+ {"application/xml", ".xml", ContentType::kXml}, // RFC 3023
+};
+const int kNumTypes = arraysize(kTypes);
+
+} // namespace
+
+const ContentType& kContentTypeHtml = kTypes[0];
+const ContentType& kContentTypeXhtml = kTypes[1];
+const ContentType& kContentTypeCeHtml = kTypes[2];
+
+const ContentType& kContentTypeJavascript = kTypes[3];
+const ContentType& kContentTypeCss = kTypes[4];
+const ContentType& kContentTypeText = kTypes[5];
+const ContentType& kContentTypeXml = kTypes[6];
+
+const ContentType& kContentTypePng = kTypes[7];
+const ContentType& kContentTypeGif = kTypes[8];
+const ContentType& kContentTypeJpeg = kTypes[9];
+
+bool ContentType::IsHtmlLike() const {
+ switch (type_) {
+ case kHtml:
+ case kXhtml:
+ case kCeHtml:
+ return true;
+ default:
+ return false;
+ }
+}
+
+bool ContentType::IsXmlLike() const {
+ switch (type_) {
+ case kXhtml:
+ case kXml:
+ return true;
+ default:
+ return false;
+ }
+}
+
+const ContentType* NameExtensionToContentType(const StringPiece& name) {
+ // Get the name from the extension.
+ StringPiece::size_type ext_pos = name.rfind('.');
+ const ContentType* res = NULL;
+ if (ext_pos != StringPiece::npos) {
+ StringPiece ext = name.substr(ext_pos);
+ // TODO(jmarantz): convert to a map if the list gets large.
+ for (int i = 0; i < kNumTypes; ++i) {
+ if (StringCaseEqual(ext, kTypes[i].file_extension())) {
+ res = &kTypes[i];
+ break;
+ }
+ }
+ }
+ return res;
+}
+
+const ContentType* MimeTypeToContentType(const StringPiece& mime_type) {
+ // TODO(jmarantz): convert to a map if the list gets large.
+ const ContentType* res = NULL;
+
+ // The content-type can have a "; charset=...". We are not interested
+ // in that, for the purpose of our ContentType object.
+ //
+ // TODO(jmarantz): we should be grabbing the encoding, however, and
+ // saving it so that when we emit content-type headers for resources,
+ // they include the proper encoding.
+ StringPiece stripped_mime_type;
+ StringPiece::size_type semi_colon = mime_type.find(';');
+ if (semi_colon == StringPiece::npos) {
+ stripped_mime_type = mime_type;
+ } else {
+ stripped_mime_type = mime_type.substr(0, semi_colon);
+ }
+
+ for (int i = 0; i < kNumTypes; ++i) {
+ if (StringCaseEqual(stripped_mime_type, kTypes[i].mime_type())) {
+ res = &kTypes[i];
+ break;
+ }
+ }
+ return res;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/content_type_test.cc b/trunk/src/net/instaweb/util/content_type_test.cc
new file mode 100644
index 0000000..84cf563
--- /dev/null
+++ b/trunk/src/net/instaweb/util/content_type_test.cc
@@ -0,0 +1,72 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the string-splitter.
+
+#include "net/instaweb/util/public/content_type.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace net_instaweb {
+
+class ContentTypeTest : public testing::Test {
+ protected:
+ ContentType::Type ExtToType(const char* ext) {
+ return NameExtensionToContentType(ext)->type();
+ }
+ ContentType::Type MimeToType(const char* mime_type) {
+ return MimeTypeToContentType(mime_type)->type();
+ }
+};
+
+TEST_F(ContentTypeTest, TestUnknown) {
+ EXPECT_EQ(NULL, NameExtensionToContentType(".unknown"));
+ EXPECT_EQ(NULL, MimeTypeToContentType("unknown/unknown"));
+}
+
+TEST_F(ContentTypeTest, TestExtensions) {
+ EXPECT_EQ(ContentType::kHtml, ExtToType(".html"));
+ EXPECT_EQ(ContentType::kJavascript, ExtToType(".js"));
+ EXPECT_EQ(ContentType::kJpeg, ExtToType(".jpg"));
+ EXPECT_EQ(ContentType::kJpeg, ExtToType(".jpeg"));
+ EXPECT_EQ(ContentType::kCss, ExtToType(".css"));
+ EXPECT_EQ(ContentType::kText, ExtToType(".txt"));
+ EXPECT_EQ(ContentType::kXml, ExtToType(".xml"));
+ EXPECT_EQ(ContentType::kPng, ExtToType(".png"));
+ EXPECT_EQ(ContentType::kGif, ExtToType(".gif"));
+}
+
+TEST_F(ContentTypeTest, TestMimeType) {
+ EXPECT_EQ(ContentType::kHtml, MimeToType("text/html"));
+ EXPECT_EQ(ContentType::kHtml, MimeToType("text/html; charset=UTF-8"));
+ EXPECT_EQ(ContentType::kXhtml, MimeToType("application/xhtml+xml"));
+ EXPECT_EQ(ContentType::kCeHtml, MimeToType("application/ce-html+xml"));
+ EXPECT_EQ(ContentType::kJavascript, MimeToType("text/javascript"));
+ EXPECT_EQ(ContentType::kJavascript, MimeToType("application/x-javascript"));
+ EXPECT_EQ(ContentType::kJavascript, MimeToType("application/javascript"));
+ EXPECT_EQ(ContentType::kJavascript, MimeToType("text/ecmascript"));
+ EXPECT_EQ(ContentType::kJavascript, MimeToType("application/ecmascript"));
+ EXPECT_EQ(ContentType::kJpeg, MimeToType("image/jpeg"));
+ EXPECT_EQ(ContentType::kCss, MimeToType("text/css"));
+ EXPECT_EQ(ContentType::kText, MimeToType("text/plain"));
+ EXPECT_EQ(ContentType::kXml, MimeToType("application/xml"));
+ EXPECT_EQ(ContentType::kXml, MimeToType("text/xml"));
+ EXPECT_EQ(ContentType::kPng, MimeToType("image/png"));
+ EXPECT_EQ(ContentType::kGif, MimeToType("image/gif"));
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/data_url.cc b/trunk/src/net/instaweb/util/data_url.cc
new file mode 100644
index 0000000..fb977e1
--- /dev/null
+++ b/trunk/src/net/instaweb/util/data_url.cc
@@ -0,0 +1,116 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/util/public/data_url.h"
+
+#include "net/instaweb/util/public/base64_util.h"
+#include "net/instaweb/util/public/content_type.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+void DataUrl(const ContentType& content_type,
+ const Encoding encoding,
+ const StringPiece& content,
+ std::string* result) {
+ result->assign("data:");
+ result->append(content_type.mime_type());
+ switch (encoding) {
+ case BASE64: {
+ result->append(";base64,");
+ std::string encoded;
+ Mime64Encode(content, &encoded);
+ result->append(encoded);
+ break;
+ }
+// TODO(jmaessen): Figure out if we ever need non-BASE64 encodings,
+// and if so actually write them. Here are the stubs.
+// case UTF8:
+// result->append(";charset=\"utf-8\",");
+// // TODO(jmaessen): find %-encoding code to use here.
+// // jmarantz has one pending.
+// result.append(content);
+// case LATIN1:
+// result->append(";charset=\"\",");
+// // TODO(jmaessen): find %-encoding code to use here.
+// // Not the UTF-8 one!
+ default: {
+ // either UNKNOWN or PLAIN. No special encoding or alphabet. We're in a
+ // context where we don't want to fail, so we try to give sensible output
+ // if encoding is actually out of range; this gives some hope of graceful
+ // degradation of experience.
+ result->append(",");
+ content.AppendToString(result);
+ break;
+ }
+ }
+}
+
+bool ParseDataUrl(const StringPiece& url,
+ const ContentType** content_type,
+ Encoding* encoding,
+ StringPiece* encoded_content) {
+ const char kData[] = "data:";
+ const size_t kDataSize = sizeof(kData) - 1;
+ const char kBase64[] = ";base64";
+ const size_t kBase64Size = sizeof(kBase64) - 1;
+ // First invalidate all outputs.
+ *content_type = NULL;
+ *encoding = UNKNOWN;
+ encoded_content->clear();
+ size_t header_boundary = url.find(',');
+ if (header_boundary == url.npos || !url.starts_with(kData)) {
+ return false;
+ }
+ StringPiece header(url.data(), header_boundary);
+ size_t mime_boundary = header.find(';');
+ if (mime_boundary == url.npos) {
+ // no charset or base64 encoding.
+ mime_boundary = header_boundary;
+ *encoding = PLAIN;
+ } else if (header_boundary >= mime_boundary + kBase64Size) {
+ if (header.ends_with(kBase64)) {
+ *encoding = BASE64;
+ } else {
+ *encoding = PLAIN;
+ }
+ }
+ StringPiece mime_type(url.data() + kDataSize, mime_boundary - kDataSize);
+ *content_type = MimeTypeToContentType(mime_type);
+ encoded_content->set(url.data() + header_boundary + 1,
+ url.size() - header_boundary - 1);
+ return true;
+}
+
+bool DecodeDataUrlContent(Encoding encoding,
+ const StringPiece& encoded_content,
+ std::string* decoded_content) {
+ switch (encoding) {
+ case PLAIN:
+ // No change, just copy data.
+ encoded_content.CopyToString(decoded_content);
+ return true;
+ case BASE64:
+ return Mime64Decode(encoded_content, decoded_content);
+ default:
+ return false;
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/data_url_test.cc b/trunk/src/net/instaweb/util/data_url_test.cc
new file mode 100644
index 0000000..909514d
--- /dev/null
+++ b/trunk/src/net/instaweb/util/data_url_test.cc
@@ -0,0 +1,171 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/util/public/data_url.h"
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/content_type.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace net_instaweb {
+namespace {
+
+const std::string kAsciiData =
+ "A_Rather=Long,But-conventional?looking_string#with;some:odd,characters.";
+const std::string kAsciiDataBase64 =
+ "QV9SYXRoZXI9TG9uZyxCdXQtY29udmVudGlvbmFsP2xvb2tpbmdfc3RyaW5nI3dpdGg7c29"
+ "tZTpvZGQsY2hhcmFjdGVycy4=";
+
+// A string with embedded NULs; we must construct it carefully to avoid
+// truncation.
+const char kMixedDataChars[] =
+ "This string\ncontains\0lots of\tunusual\xe3~characters\xd7\xa5";
+const char kMixedDataBase64[] =
+ "VGhpcyBzdHJpbmcKY29udGFpbnMAbG90cyBvZgl1bnVzdWFs435jaGFyYWN0ZXJz16U=";
+
+const char kPlainPrefix[] = "data:text/plain,";
+const char kBase64Prefix[] = "data:text/plain;base64,";
+
+const char kGifPlainPrefix[] = "data:image/gif,";
+const char kGifBase64Prefix[] = "data:image/gif;base64,";
+
+class DataUrlTest : public testing::Test {
+ public:
+ DataUrlTest()
+ : mixed_data_(kMixedDataChars, sizeof(kMixedDataChars) - 1) { }
+
+ protected:
+ // Make a ContentType yield readable failure output. Needed this to fix bugs
+ // in the tests! (No actual program bugs found here...)
+ const char* Mime(const ContentType* type) {
+ if (type == NULL || type->mime_type() == NULL) {
+ return "NULL";
+ } else {
+ return type->mime_type();
+ }
+ }
+
+ void TestDecoding(const bool can_parse,
+ const bool can_decode,
+ const std::string& prefix,
+ const std::string& encoded,
+ const ContentType* type,
+ Encoding encoding,
+ const std::string& decoded) {
+ std::string url = prefix + encoded;
+ const ContentType* parsed_type;
+ Encoding parsed_encoding = UNKNOWN;
+ StringPiece parsed_encoded;
+ EXPECT_EQ(can_parse,
+ ParseDataUrl(url, &parsed_type,
+ &parsed_encoding, &parsed_encoded));
+ EXPECT_EQ(encoding, parsed_encoding);
+ EXPECT_EQ(type, parsed_type) << "type '" << Mime(type) <<
+ "' didn't match '" << Mime(parsed_type) << "'\n";
+ EXPECT_EQ(encoded, parsed_encoded);
+ std::string parsed_decoded;
+ EXPECT_EQ(can_decode,
+ DecodeDataUrlContent(encoding, parsed_encoded, &parsed_decoded));
+ EXPECT_EQ(decoded, parsed_decoded);
+ }
+
+ std::string mixed_data_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(DataUrlTest);
+};
+
+TEST_F(DataUrlTest, TestDataPlain) {
+ std::string url;
+ DataUrl(kContentTypeText, PLAIN, kAsciiData, &url);
+ EXPECT_EQ(kPlainPrefix + kAsciiData, url);
+}
+
+TEST_F(DataUrlTest, TestDataBase64) {
+ std::string url;
+ DataUrl(kContentTypeText, BASE64, kAsciiData, &url);
+ EXPECT_EQ(kBase64Prefix + kAsciiDataBase64, url);
+}
+
+TEST_F(DataUrlTest, TestData1Plain) {
+ std::string url;
+ DataUrl(kContentTypeGif, PLAIN, mixed_data_, &url);
+ EXPECT_EQ(kGifPlainPrefix + mixed_data_, url);
+}
+
+TEST_F(DataUrlTest, TestData1Base64) {
+ std::string url;
+ DataUrl(kContentTypeGif, BASE64, mixed_data_, &url);
+ EXPECT_EQ(StrCat(kGifBase64Prefix, kMixedDataBase64), url);
+}
+
+TEST_F(DataUrlTest, ParseDataPlain) {
+ TestDecoding(true, true, kPlainPrefix, kAsciiData,
+ &kContentTypeText, PLAIN, kAsciiData);
+}
+
+TEST_F(DataUrlTest, ParseDataBase64) {
+ TestDecoding(true, true, kBase64Prefix, kAsciiDataBase64,
+ &kContentTypeText, BASE64, kAsciiData);
+}
+
+TEST_F(DataUrlTest, ParseData1Plain) {
+ TestDecoding(true, true, kPlainPrefix, mixed_data_,
+ &kContentTypeText, PLAIN, mixed_data_);
+}
+
+TEST_F(DataUrlTest, ParseData1Base64) {
+ TestDecoding(true, true, kBase64Prefix, kMixedDataBase64,
+ &kContentTypeText, BASE64, mixed_data_);
+}
+
+TEST_F(DataUrlTest, ParseBadProtocol) {
+ TestDecoding(false, false, "http://www.google.com/", "",
+ NULL, UNKNOWN, "");
+}
+
+TEST_F(DataUrlTest, ParseNoComma) {
+ TestDecoding(false, false,
+ StrCat("data:text/plain;base64;", kMixedDataBase64), "",
+ NULL, UNKNOWN, "");
+}
+
+TEST_F(DataUrlTest, ParseNoMime) {
+ TestDecoding(true, true, "data:;base64,", kMixedDataBase64,
+ NULL, BASE64, mixed_data_);
+}
+
+TEST_F(DataUrlTest, ParseCorruptMime) {
+ TestDecoding(true, true, "data:#$!;base64,", kMixedDataBase64,
+ NULL, BASE64, mixed_data_);
+}
+
+TEST_F(DataUrlTest, ParseBadEncodingIsPlain) {
+ TestDecoding(true, true, "data:text/plain;mumbledypeg,", mixed_data_,
+ &kContentTypeText, PLAIN, mixed_data_);
+}
+
+TEST_F(DataUrlTest, ParseBadBase64) {
+ TestDecoding(true, false, kBase64Prefix, "@%#$%@#$%^@%%^%*%^&*",
+ &kContentTypeText, BASE64, "");
+}
+
+} // namespace
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/dummy_url_fetcher.cc b/trunk/src/net/instaweb/util/dummy_url_fetcher.cc
new file mode 100644
index 0000000..267c98d
--- /dev/null
+++ b/trunk/src/net/instaweb/util/dummy_url_fetcher.cc
@@ -0,0 +1,35 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/public/dummy_url_fetcher.h"
+
+#include "net/instaweb/util/public/message_handler.h"
+#include <string>
+
+namespace net_instaweb {
+
+bool DummyUrlFetcher::StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* fetched_content_writer,
+ MessageHandler* message_handler) {
+ message_handler->Message(kFatal, "DummyUrlFetcher used");
+ return false;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/fake_url_async_fetcher.cc b/trunk/src/net/instaweb/util/fake_url_async_fetcher.cc
new file mode 100644
index 0000000..b3558f2
--- /dev/null
+++ b/trunk/src/net/instaweb/util/fake_url_async_fetcher.cc
@@ -0,0 +1,26 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/fake_url_async_fetcher.h"
+
+namespace net_instaweb {
+
+FakeUrlAsyncFetcher::~FakeUrlAsyncFetcher() {
+}
+
+} // namespace instaweb
diff --git a/trunk/src/net/instaweb/util/fetcher_test.cc b/trunk/src/net/instaweb/util/fetcher_test.cc
new file mode 100644
index 0000000..6f7790f
--- /dev/null
+++ b/trunk/src/net/instaweb/util/fetcher_test.cc
@@ -0,0 +1,163 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/fetcher_test.h"
+
+namespace net_instaweb {
+
+const char FetcherTest::kStartDate[] = "Sun, 16 Dec 1979 02:27:45 GMT";
+const char FetcherTest::kHtmlContent[] = "<html><body>Nuts!</body></html>";
+const char FetcherTest::kErrorMessage[] = "Invalid URL";
+const char FetcherTest::kGoodUrl[] = "http://pi.com";
+const char FetcherTest::kNotCachedUrl[] = "http://not_cacheable.com";
+const char FetcherTest::kBadUrl[] = "http://this_url_will_fail.com";
+const char FetcherTest::kHeaderName[] = "header-name";
+const char FetcherTest::kHeaderValue[] = "header value";
+
+void FetcherTest::ValidateMockFetcherResponse(
+ bool success, bool check_error_message,
+ const std::string& content,
+ const MetaData& response_headers) {
+ if (success) {
+ EXPECT_EQ(std::string(kHtmlContent), content);
+ CharStarVector values;
+ EXPECT_TRUE(response_headers.Lookup(kHeaderName, &values));
+ EXPECT_EQ(1, values.size());
+ EXPECT_EQ(std::string(kHeaderValue), values[0]);
+ } else if (check_error_message) {
+ EXPECT_EQ(std::string(kErrorMessage), content);
+ }
+}
+
+int FetcherTest::CountFetchesSync(const StringPiece& url, bool expect_success,
+ bool check_error_message) {
+ CHECK(sync_fetcher() != NULL);
+ return CountFetchesSync(url, sync_fetcher(),
+ expect_success, check_error_message);
+}
+
+int FetcherTest::CountFetchesSync(
+ const StringPiece& url, UrlFetcher* fetcher,
+ bool expect_success, bool check_error_message) {
+ int starting_fetches = mock_fetcher_.num_fetches();
+ std::string content;
+ StringWriter content_writer(&content);
+ SimpleMetaData request_headers, response_headers;
+ bool success = fetcher->StreamingFetchUrl(
+ url.as_string(), request_headers, &response_headers, &content_writer,
+ &message_handler_);
+ EXPECT_EQ(expect_success, success);
+ ValidateMockFetcherResponse(success, check_error_message, content,
+ response_headers);
+ return mock_fetcher_.num_fetches() - starting_fetches;
+}
+
+int FetcherTest::CountFetchesAsync(const StringPiece& url, bool expect_success,
+ bool* callback_called) {
+ CHECK(async_fetcher() != NULL);
+ *callback_called = false;
+ int starting_fetches = mock_fetcher_.num_fetches();
+ SimpleMetaData request_headers;
+ CheckCallback* fetch = new CheckCallback(expect_success, callback_called);
+ async_fetcher()->StreamingFetch(
+ url.as_string(), request_headers, &fetch->response_headers_,
+ &fetch->content_writer_, &message_handler_, fetch);
+ return mock_fetcher_.num_fetches() - starting_fetches;
+}
+
+
+void FetcherTest::ValidateOutput(const std::string& content,
+ const MetaData& response_headers) {
+ // The detailed header parsing code is tested in
+ // simple_meta_data_test.cc. But let's check the rseponse code
+ // and the last header here, and make sure we got the content.
+ EXPECT_EQ(200, response_headers.status_code());
+ EXPECT_EQ(15, response_headers.NumAttributes());
+ EXPECT_EQ(std::string("X-Google-GFE-Response-Body-Transformations"),
+ std::string(response_headers.Name(14)));
+ EXPECT_EQ(std::string("gunzipped"),
+ std::string(response_headers.Value(14)));
+
+ // Verifies that after the headers, we see the content. Note that this
+ // currently assumes 'wget' style output. Wget takes care of any unzipping.
+ static const char start_of_doc[] = "<!doctype html>";
+ EXPECT_EQ(0, strncmp(start_of_doc, content.c_str(),
+ sizeof(start_of_doc) - 1));
+}
+
+
+// MockFetcher
+bool FetcherTest::MockFetcher::StreamingFetchUrl(
+ const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* writer,
+ MessageHandler* message_handler) {
+ bool ret = false;
+ if (url == kGoodUrl) {
+ ret = Populate("max-age=300", response_headers, writer,
+ message_handler);
+ } else if (url == kNotCachedUrl) {
+ ret = Populate("no-cache", response_headers, writer,
+ message_handler);
+ } else {
+ writer->Write(kErrorMessage, message_handler);
+ }
+ ++num_fetches_;
+ return ret;
+}
+
+bool FetcherTest::MockFetcher::Populate(const char* cache_control,
+ MetaData* response_headers,
+ Writer* writer,
+ MessageHandler* message_handler) {
+ response_headers->SetStatusAndReason(HttpStatus::kOK);
+ response_headers->Add(HttpAttributes::kCacheControl, cache_control);
+ response_headers->Add("Date", kStartDate);
+ response_headers->Add(kHeaderName, kHeaderValue);
+ response_headers->ComputeCaching();
+ response_headers->set_headers_complete(true);
+ writer->Write(kHtmlContent, message_handler);
+ return true;
+}
+
+
+// MockAsyncFetcher
+bool FetcherTest::MockAsyncFetcher::StreamingFetch(
+ const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* writer,
+ MessageHandler* handler,
+ Callback* callback) {
+ bool status = url_fetcher_->StreamingFetchUrl(
+ url, request_headers, response_headers, writer, handler);
+ deferred_callbacks_.push_back(std::make_pair(status, callback));
+ return false;
+}
+
+void FetcherTest::MockAsyncFetcher::CallCallbacks() {
+ for (int i = 0, n = deferred_callbacks_.size(); i < n; ++i) {
+ bool status = deferred_callbacks_[i].first;
+ Callback* callback = deferred_callbacks_[i].second;
+ callback->Done(status);
+ }
+ deferred_callbacks_.clear();
+}
+
+} // namespace net_isntaweb
diff --git a/trunk/src/net/instaweb/util/fetcher_test.h b/trunk/src/net/instaweb/util/fetcher_test.h
new file mode 100644
index 0000000..827410f
--- /dev/null
+++ b/trunk/src/net/instaweb/util/fetcher_test.h
@@ -0,0 +1,183 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test framework for wget fetcher
+
+#ifndef NET_INSTAWEB_UTIL_FETCHER_TEST_H_
+#define NET_INSTAWEB_UTIL_FETCHER_TEST_H_
+
+#include <algorithm>
+#include <utility> // for pair
+#include <vector>
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include <string>
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+#include "net/instaweb/util/public/url_fetcher.h"
+
+namespace net_instaweb {
+
+class FetcherTest : public testing::Test {
+ protected:
+ static const char kStartDate[];
+ static const char kHtmlContent[];
+ static const char kGoodUrl[];
+ static const char kNotCachedUrl[];
+ static const char kBadUrl[];
+ static const char kHeaderName[];
+ static const char kHeaderValue[];
+ static const char kErrorMessage[];
+
+ FetcherTest() : mock_async_fetcher_(&mock_fetcher_) {}
+
+ // Helpful classes for testing.
+
+ // This mock fetcher will only fetch kGoodUrl, returning kHtmlContent.
+ // If you ask for any other URL it will fail.
+ class MockFetcher : public UrlFetcher {
+ public:
+ MockFetcher() : num_fetches_(0) {}
+
+
+ virtual bool StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* message_handler);
+
+ int num_fetches() const { return num_fetches_; }
+
+ private:
+ bool Populate(const char* cache_control, MetaData* response_headers,
+ Writer* writer, MessageHandler* message_handler);
+
+ int num_fetches_;
+
+ DISALLOW_COPY_AND_ASSIGN(MockFetcher);
+ };
+
+ // This is a pseudo-asynchronous interface to MockFetcher. It performs
+ // fetches instantly, but defers calling the callback until the user
+ // calls CallCallbacks(). Then it will execute the deferred callbacks.
+ class MockAsyncFetcher : public UrlAsyncFetcher {
+ public:
+ explicit MockAsyncFetcher(UrlFetcher* url_fetcher)
+ : url_fetcher_(url_fetcher) {}
+
+ virtual bool StreamingFetch(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* handler,
+ Callback* callback);
+
+ void CallCallbacks();
+
+ private:
+ UrlFetcher* url_fetcher_;
+ std::vector<std::pair<bool, Callback*> > deferred_callbacks_;
+
+ DISALLOW_COPY_AND_ASSIGN(MockAsyncFetcher);
+ };
+
+ // Callback that just checks correct Done status and keeps track of whether
+ // it has been called yet or not.
+ class CheckCallback : public UrlAsyncFetcher::Callback {
+ public:
+ explicit CheckCallback(bool expect_success, bool* callback_called)
+ : expect_success_(expect_success),
+ content_writer_(&content_),
+ callback_called_(callback_called) {
+ }
+
+ virtual void Done(bool success) {
+ *callback_called_ = true;
+ CHECK_EQ(expect_success_, success);
+ ValidateMockFetcherResponse(success, true, content_, response_headers_);
+ delete this;
+ }
+
+ bool expect_success_;
+ SimpleMetaData response_headers_;
+ std::string content_;
+ StringWriter content_writer_;
+ bool* callback_called_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(CheckCallback);
+ };
+
+ static void ValidateMockFetcherResponse(bool success,
+ bool check_error_message,
+ const std::string& content,
+ const MetaData& response_headers);
+
+ // Do a URL fetch, and return the number of times the mock fetcher
+ // had to be run to perform the fetch.
+ // Note: You must override sync_fetcher() to return the correct fetcher.
+ int CountFetchesSync(const StringPiece& url, bool expect_success,
+ bool check_error_message);
+ // Use an explicit fetcher (you don't need to override sync_fetcher()).
+ int CountFetchesSync(const StringPiece& url, UrlFetcher* fetcher,
+ bool expect_success, bool check_error_message);
+
+ // Initiate an async URL fetch, and return the number of times the mock
+ // fetcher had to be run to perform the fetch.
+ // Note: You must override async_fetcher() to return the correct fetcher.
+ int CountFetchesAsync(const StringPiece& url, bool expect_success,
+ bool* callback_called);
+
+ // Override these to allow CountFetchesSync or Async respectively.
+ // These are not abstract (= 0) because they only need to be overridden by
+ // classes which want to use CountFetchersSync/Async without specifying the
+ // fetcher in each call.
+ virtual UrlFetcher* sync_fetcher() {
+ CHECK(false) << "sync_fetcher() must be overridden before use.";
+ return NULL;
+ };
+ virtual UrlAsyncFetcher* async_fetcher() {
+ CHECK(false) << "async_fetcher() must be overridden before use.";
+ return NULL;
+ };
+
+
+ std::string TestFilename() {
+ return (GTestSrcDir() +
+ "/net/instaweb/util/testdata/google.http");
+ }
+
+ // This validation code is hard-coded to the http request capture in
+ // testdata/google.http.
+ void ValidateOutput(const std::string& content,
+ const MetaData& response_headers);
+
+ GoogleMessageHandler message_handler_;
+ MockFetcher mock_fetcher_;
+ MockAsyncFetcher mock_async_fetcher_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(FetcherTest);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_FETCHER_TEST_H_
diff --git a/trunk/src/net/instaweb/util/file_cache.cc b/trunk/src/net/instaweb/util/file_cache.cc
new file mode 100644
index 0000000..6f3db80
--- /dev/null
+++ b/trunk/src/net/instaweb/util/file_cache.cc
@@ -0,0 +1,279 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: lsong@google.com (Libo Song)
+
+#include "net/instaweb/util/public/file_cache.h"
+
+#include <stdlib.h>
+#include <vector>
+#include <queue>
+#include "net/instaweb/util/public/base64_util.h"
+#include "net/instaweb/util/public/null_message_handler.h"
+#include "net/instaweb/util/public/shared_string.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+namespace { // For structs used only in Clean().
+
+class CacheFileInfo {
+ public:
+ CacheFileInfo(int64 size, int64 atime, const std::string& name)
+ : size_(size), atime_(atime), name_(name) {}
+ int64 size_;
+ int64 atime_;
+ std::string name_;
+ private:
+ DISALLOW_COPY_AND_ASSIGN(CacheFileInfo);
+};
+
+struct CompareByAtime {
+ public:
+ bool operator()(const CacheFileInfo* one,
+ const CacheFileInfo* two) const {
+ return one->atime_ < two->atime_;
+ }
+};
+
+} // namespace for structs used only in Clean().
+
+// Filenames for the next scheduled clean time and the lockfile. In
+// order to prevent these from colliding with actual cachefiles, they
+// contain characters that our filename encoder would escape.
+const char FileCache::kCleanTimeName[] = "!clean!time!";
+const char FileCache::kCleanLockName[] = "!clean!lock!";
+
+// TODO(abliss): remove policy from constructor; provide defaults here
+// and setters below.
+FileCache::FileCache(const std::string& path, FileSystem* file_system,
+ FilenameEncoder* filename_encoder,
+ CachePolicy* policy,
+ MessageHandler* handler)
+ : path_(path),
+ file_system_(file_system),
+ filename_encoder_(filename_encoder),
+ message_handler_(handler),
+ cache_policy_(policy),
+ // NOTE(abliss): We don't want all the caches racing for the
+ // lock at startup, so each one gets a random offset.
+ next_clean_ms_(policy->timer->NowMs()
+ + (random() % policy->clean_interval_ms)) {
+}
+
+FileCache::~FileCache() {
+}
+
+bool FileCache::Get(const std::string& key, SharedString* value) {
+ std::string filename;
+ bool ret = EncodeFilename(key, &filename);
+ if (ret) {
+ std::string* buffer = value->get();
+
+ // Suppress read errors. Note that we want to show Write errors,
+ // as they likely indicate a permissions or disk-space problem
+ // which is best not eaten. It's cheap enough to construct
+ // a NullMessageHandler on the stack when we want one.
+ NullMessageHandler null_handler;
+ ret = file_system_->ReadFile(filename.c_str(), buffer, &null_handler);
+ }
+ return ret;
+}
+
+void FileCache::Put(const std::string& key, SharedString* value) {
+ std::string filename;
+ if (EncodeFilename(key, &filename)) {
+ const std::string& buffer = **value;
+ std::string temp_filename;
+ if (file_system_->WriteTempFile(filename.c_str(), buffer,
+ &temp_filename, message_handler_)) {
+ file_system_->RenameFile(temp_filename.c_str(), filename.c_str(),
+ message_handler_);
+ }
+ }
+ CheckClean();
+}
+
+void FileCache::Delete(const std::string& key) {
+ std::string filename;
+ if (!EncodeFilename(key, &filename)) {
+ return;
+ }
+ file_system_->RemoveFile(filename.c_str(), message_handler_);
+ return;
+}
+
+bool FileCache::EncodeFilename(const std::string& key,
+ std::string* filename) {
+ std::string prefix = path_;
+ // TODO(abliss): unify and make explicit everyone's assumptions
+ // about trailing slashes.
+ EnsureEndsInSlash(&prefix);
+ filename_encoder_->Encode(prefix, key, filename);
+ return true;
+}
+
+CacheInterface::KeyState FileCache::Query(const std::string& key) {
+ std::string filename;
+ if (!EncodeFilename(key, &filename)) {
+ return CacheInterface::kNotFound;
+ }
+ NullMessageHandler null_handler;
+ if (file_system_->Exists(filename.c_str(), &null_handler).is_true()) {
+ return CacheInterface::kAvailable;
+ }
+ return CacheInterface::kNotFound;
+}
+
+bool FileCache::Clean(int64 target_size) {
+ StringVector files;
+ int64 file_size;
+ int64 file_atime;
+ int64 total_size = 0;
+ if (!file_system_->RecursiveDirSize(path_, &total_size, message_handler_)) {
+ return false;
+ }
+
+ // TODO(jmarantz): gcc 4.1 warns about double/int64 comparisons here,
+ // but this really should be factored into a settable member var.
+ if (total_size < ((target_size * 5) / 4)) {
+ message_handler_->Message(kInfo,
+ "File cache size is %ld; no cleanup needed.",
+ static_cast<long>(total_size));
+ return true;
+ }
+ message_handler_->Message(kInfo,
+ "File cache size is %ld; beginning cleanup.",
+ static_cast<long>(total_size));
+ bool everything_ok = true;
+ everything_ok &= file_system_->ListContents(path_, &files, message_handler_);
+
+ // We will now iterate over the entire directory and its children,
+ // keeping a heap of files to be deleted. Our goal is to delete the
+ // oldest set of files that sum to enough space to bring us below
+ // our target.
+ std::priority_queue<CacheFileInfo*, std::vector<CacheFileInfo*>,
+ CompareByAtime> heap;
+ int64 total_heap_size = 0;
+ // TODO(jmarantz): gcc 4.1 warns about double/int64 comparisons here,
+ // but this really should be factored into a settable member var.
+ int64 target_heap_size = total_size - ((target_size * 3 / 4));
+
+ std::string prefix = path_;
+ EnsureEndsInSlash(&prefix);
+ for (size_t i = 0; i < files.size(); i++) {
+ std::string file_name = files[i];
+ BoolOrError isDir = file_system_->IsDir(file_name.c_str(),
+ message_handler_);
+ if (isDir.is_error()) {
+ return false;
+ } else if (isDir.is_true()) {
+ // add files in this directory to the end of the vector, to be
+ // examined later.
+ everything_ok &= file_system_->ListContents(file_name, &files,
+ message_handler_);
+ } else {
+ everything_ok &= file_system_->Size(file_name, &file_size,
+ message_handler_);
+ everything_ok &= file_system_->Atime(file_name, &file_atime,
+ message_handler_);
+ // If our heap is still too small; add everything in.
+ // Otherwise, add the file in only if it's older than the newest
+ // thing in the heap.
+ if ((total_heap_size < target_heap_size) ||
+ (file_atime < heap.top()->atime_)) {
+ CacheFileInfo* info =
+ new CacheFileInfo(file_size, file_atime, file_name);
+ heap.push(info);
+ total_heap_size += file_size;
+ // Now remove new things from the heap which are not needed
+ // to keep the heap size over its target size.
+ while (total_heap_size - heap.top()->size_ > target_heap_size) {
+ total_heap_size -= heap.top()->size_;
+ delete heap.top();
+ heap.pop();
+ }
+ }
+ }
+ }
+ for (size_t i = heap.size(); i > 0; i--) {
+ everything_ok &= file_system_->RemoveFile(heap.top()->name_.c_str(),
+ message_handler_);
+ delete heap.top();
+ heap.pop();
+ }
+ message_handler_->Message(kInfo,
+ "File cache cleanup complete; freed %ld bytes\n",
+ static_cast<long>(total_heap_size));
+ return everything_ok;
+}
+
+bool FileCache::CheckClean() {
+ const int64 now_ms = cache_policy_->timer->NowMs();
+ if (now_ms < next_clean_ms_) {
+ return false;
+ }
+ std::string lock_name(path_);
+ EnsureEndsInSlash(&lock_name);
+ lock_name += kCleanLockName;
+ bool to_return = false;
+ if (file_system_->TryLock(lock_name, message_handler_).is_true()) {
+ std::string clean_time_name(path_);
+ EnsureEndsInSlash(&clean_time_name);
+ clean_time_name += kCleanTimeName;
+ std::string clean_time_str;
+ int64 clean_time_ms = 0;
+ int64 new_clean_time_ms = now_ms + cache_policy_->clean_interval_ms;
+ if (file_system_->ReadFile(clean_time_name.c_str(), &clean_time_str,
+ message_handler_)) {
+ StringToInt64(clean_time_str, &clean_time_ms);
+ }
+ // If the "clean time" written in the file is older than now, we
+ // clean.
+ if (clean_time_ms < now_ms) {
+ message_handler_->Message(kInfo,
+ "Checking file cache size against target %ld",
+ static_cast<long>(cache_policy_->target_size));
+ to_return = true;
+ }
+ // If the "clean time" is later than now plus one interval, something
+ // went wrong (like the system clock moving backwards or the file
+ // getting corrupt) so we clean and reset it.
+ if (clean_time_ms > new_clean_time_ms) {
+ message_handler_->Message(kError,
+ "Next scheduled file cache clean time %ld"
+ " is implausibly remote. Cleaning now.",
+ static_cast<long>(clean_time_ms));
+ to_return = true;
+ }
+ if (to_return) {
+ clean_time_str = Integer64ToString(new_clean_time_ms);
+ file_system_->WriteFile(clean_time_name.c_str(), clean_time_str,
+ message_handler_);
+ }
+ file_system_->Unlock(lock_name, message_handler_);
+ } else {
+ // TODO(abliss): add a way to break a stale lock
+ }
+ next_clean_ms_ = now_ms + cache_policy_->clean_interval_ms;
+ if (to_return) {
+ // TODO(abliss): add a thread here so we don't block the unlucky request.
+ to_return &= Clean(cache_policy_->target_size);
+ }
+ return to_return;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/file_cache_test.cc b/trunk/src/net/instaweb/util/file_cache_test.cc
new file mode 100644
index 0000000..cea4e5b
--- /dev/null
+++ b/trunk/src/net/instaweb/util/file_cache_test.cc
@@ -0,0 +1,144 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: lsong@google.com (Libo Song)
+
+// Unit-test the file cache
+
+#include "net/instaweb/util/public/file_cache.h"
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/filename_encoder.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/mem_file_system.h"
+#include "net/instaweb/util/public/mock_timer.h"
+#include "net/instaweb/util/public/shared_string.h"
+#include <string>
+
+namespace net_instaweb {
+
+class FileCacheTest : public testing::Test {
+ protected:
+ FileCacheTest()
+ : mock_timer_(0),
+ kCleanIntervalMs(Timer::kMinuteMs),
+ kTargetSize(409600),
+ cache_(GTestTempDir(), &file_system_, &filename_encoder_,
+ new FileCache::CachePolicy(
+ &mock_timer_, kCleanIntervalMs, kTargetSize),
+ &message_handler_) {
+ }
+
+ void CheckGet(const char* key, const std::string& expected_value) {
+ SharedString value_buffer;
+ EXPECT_TRUE(cache_.Get(key, &value_buffer));
+ EXPECT_EQ(expected_value, *value_buffer);
+ EXPECT_EQ(CacheInterface::kAvailable, cache_.Query(key));
+ }
+
+ void Put(const char* key, const char* value) {
+ SharedString put_buffer(value);
+ cache_.Put(key, &put_buffer);
+ }
+
+ void CheckNotFound(const char* key) {
+ SharedString value_buffer;
+ EXPECT_FALSE(cache_.Get(key, &value_buffer));
+ EXPECT_EQ(CacheInterface::kNotFound, cache_.Query(key));
+ }
+
+ protected:
+ MemFileSystem file_system_;
+ FilenameEncoder filename_encoder_;
+ MockTimer mock_timer_;
+ const int64 kCleanIntervalMs;
+ const int64 kTargetSize;
+ FileCache cache_;
+ GoogleMessageHandler message_handler_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(FileCacheTest);
+};
+
+// Simple flow of putting in an item, getting it, deleting it.
+TEST_F(FileCacheTest, PutGetDelete) {
+ Put("Name", "Value");
+ CheckGet("Name", "Value");
+ CheckNotFound("Another Name");
+
+ Put("Name", "NewValue");
+ CheckGet("Name", "NewValue");
+
+ cache_.Delete("Name");
+ SharedString value_buffer;
+ EXPECT_FALSE(cache_.Get("Name", &value_buffer));
+}
+
+// Throw a bunch of files into the cache and verify that they are
+// evicted sensibly.
+TEST_F(FileCacheTest, Clean) {
+ file_system_.Clear();
+ // Make some "directory" entries so that the mem_file_system recurses
+ // correctly.
+ std::string dir1 = GTestTempDir() + "/a/";
+ std::string dir2 = GTestTempDir() + "/b/";
+ EXPECT_TRUE(file_system_.MakeDir(dir1.c_str(), &message_handler_));
+ EXPECT_TRUE(file_system_.MakeDir(dir2.c_str(), &message_handler_));
+ // Commonly-used keys
+ const char* names1[] = {"a1", "a2", "a/3"};
+ const char* values1[] = {"a2", "a234", "a2345678"};
+ // Less common keys
+ const char* names2[] =
+ {"b/1", "b2", "b3", "b4", "b5", "b6", "b7", "b8", "b9"};
+ const char* values2[] = {"b2", "b234", "b2345678",
+ "b2", "b234", "b2345678",
+ "b2", "b234", "b2345678"};
+ for (int i = 0; i < 3; i++) {
+ Put(names1[i], values1[i]);
+ }
+ for (int i = 0; i < 9; i++) {
+ Put(names2[i], values2[i]);
+ }
+ int64 total_size = 0;
+ EXPECT_TRUE(
+ file_system_.RecursiveDirSize(GTestTempDir(), &total_size,
+ &message_handler_));
+ EXPECT_EQ((2 + 4 + 8) * 4, total_size);
+
+ // Clean should not remove anything if target is bigger than total size.
+ EXPECT_TRUE(cache_.Clean(total_size + 1));
+ for (int i = 0; i < 27; i++) {
+ // This pattern represents more common usage of the names1 files.
+ CheckGet(names1[i % 3], values1[i % 3]);
+ CheckGet(names2[i % 9], values2[i % 9]);
+ }
+
+ // TODO(jmarantz): gcc 4.1 warns about double/int64 comparisons here,
+ // but this really should be factored into a settable member var.
+ int64 target_size = (4 * total_size) / 5 - 1;
+ EXPECT_TRUE(cache_.Clean(target_size));
+ // Common files should stay
+ for (int i = 0; i < 3; i++) {
+ CheckGet(names1[i], values1[i]);
+ }
+ // Some of the less common files should be gone
+ for (int i = 0; i < 3; i++) {
+ CheckNotFound(names2[i]);
+ }
+}
+// TODO(abliss): add test for CheckClean
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/file_message_handler.cc b/trunk/src/net/instaweb/util/file_message_handler.cc
new file mode 100644
index 0000000..ba03b55
--- /dev/null
+++ b/trunk/src/net/instaweb/util/file_message_handler.cc
@@ -0,0 +1,52 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/file_message_handler.h"
+
+#include <stdio.h>
+#include <stdlib.h>
+
+namespace net_instaweb {
+
+FileMessageHandler::FileMessageHandler(FILE* file) : file_(file) {
+}
+
+void FileMessageHandler::MessageVImpl(MessageType type, const char* msg,
+ va_list args) {
+ fprintf(file_, "%s: ", MessageTypeToString(type));
+ vfprintf(file_, msg, args);
+ fputc('\n', file_);
+
+ if (type == kFatal) {
+ abort();
+ }
+}
+
+void FileMessageHandler::FileMessageVImpl(MessageType type,
+ const char* filename, int line,
+ const char *msg, va_list args) {
+ fprintf(file_, "%s: %s:%d: ", MessageTypeToString(type), filename, line);
+ vfprintf(file_, msg, args);
+ fputc('\n', file_);
+
+ if (type == kFatal) {
+ abort();
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/file_system.cc b/trunk/src/net/instaweb/util/file_system.cc
new file mode 100644
index 0000000..4d5d5c3
--- /dev/null
+++ b/trunk/src/net/instaweb/util/file_system.cc
@@ -0,0 +1,181 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/stack_buffer.h"
+
+namespace net_instaweb {
+
+FileSystem::~FileSystem() {
+}
+
+FileSystem::File::~File() {
+}
+
+FileSystem::InputFile::~InputFile() {
+}
+
+FileSystem::OutputFile::~OutputFile() {
+}
+
+bool FileSystem::ReadFile(const char* filename, std::string* buffer,
+ MessageHandler* message_handler) {
+ StringWriter writer(buffer);
+ return ReadFile(filename, &writer, message_handler);
+}
+
+bool FileSystem::ReadFile(const char* filename, Writer* writer,
+ MessageHandler* message_handler) {
+ InputFile* input_file = OpenInputFile(filename, message_handler);
+ bool ret = false;
+ if (input_file != NULL) {
+ char buf[kStackBufferSize];
+ int nread;
+ ret = true;
+ while (ret && ((nread = input_file->Read(
+ buf, sizeof(buf), message_handler)) > 0)) {
+ ret = writer->Write(StringPiece(buf, nread), message_handler);
+ }
+ ret &= (nread == 0);
+ ret &= Close(input_file, message_handler);
+ }
+ return ret;
+}
+
+bool FileSystem::WriteFile(const char* filename, const StringPiece& buffer,
+ MessageHandler* message_handler) {
+ OutputFile* output_file = OpenOutputFile(filename, message_handler);
+ bool ret = false;
+ if (output_file != NULL) {
+ ret = output_file->Write(buffer, message_handler);
+ ret &= output_file->SetWorldReadable(message_handler);
+ ret &= Close(output_file, message_handler);
+ }
+ return ret;
+}
+
+bool FileSystem::WriteTempFile(const StringPiece& prefix_name,
+ const StringPiece& buffer,
+ std::string* filename,
+ MessageHandler* message_handler) {
+ OutputFile* output_file = OpenTempFile(prefix_name, message_handler);
+ bool ok = (output_file != NULL);
+ if (ok) {
+ // Store filename early, since it's invalidated by Close.
+ *filename = output_file->filename();
+ ok = output_file->Write(buffer, message_handler);
+ // attempt Close even if write fails.
+ ok &= Close(output_file, message_handler);
+ }
+ if (!ok) {
+ // Clear filename so we end in a consistent state.
+ filename->clear();
+ }
+ return ok;
+}
+
+bool FileSystem::Close(File* file, MessageHandler* message_handler) {
+ bool ret = file->Close(message_handler);
+ delete file;
+ return ret;
+}
+
+
+bool FileSystem::RecursivelyMakeDir(const StringPiece& full_path_const,
+ MessageHandler* handler) {
+ bool ret = true;
+ std::string full_path = full_path_const.as_string();
+ EnsureEndsInSlash(&full_path);
+ std::string subpath;
+ subpath.reserve(full_path.size());
+ size_t old_pos = 0, new_pos;
+ // Note that we intentionally start searching at pos = 1 to avoid having
+ // subpath be "" on absolute paths.
+ while ((new_pos = full_path.find('/', old_pos + 1)) != std::string::npos) {
+ // Build up path, one segment at a time.
+ subpath.append(full_path.data() + old_pos, new_pos - old_pos);
+ if (Exists(subpath.c_str(), handler).is_false()) {
+ if (!MakeDir(subpath.c_str(), handler)) {
+ ret = false;
+ break;
+ }
+ } else if (IsDir(subpath.c_str(), handler).is_false()) {
+ handler->Message(kError, "Subpath '%s' of '%s' is a non-directory file.",
+ subpath.c_str(), full_path.c_str());
+ ret = false;
+ break;
+ }
+ old_pos = new_pos;
+ }
+ return ret;
+}
+
+bool FileSystem::RecursiveDirSize(const StringPiece& path, int64* size,
+ MessageHandler* handler) {
+ // TODO(abliss): replace this recursive algorithm with an iterator
+ // that keeps its own state. It can keep a tree of directory names
+ // to save memory, and simplify the implementation of file_cache.Clean.
+ const std::string path_string = path.as_string();
+ const char* path_str = path_string.c_str();
+ int64 file_size = 0;
+ StringVector files;
+ if (!ListContents(path_str, &files, handler)) {
+ return false;
+ }
+ const std::string prefix = path_string + "/";
+ for (int i = files.size() - 1; i >= 0; i--) {
+ const std::string file_name = files[i];
+ BoolOrError isDir = IsDir(file_name.c_str(), handler);
+ if (isDir.is_error()) {
+ return false;
+ } else if (isDir.is_false()) {
+ if (!Size(file_name, &file_size, handler)) {
+ return false;
+ }
+ *size += file_size;
+ } else {
+ // Recurse on directory
+ // TODO(abliss): Should guard against infinite loops here, in
+ // the case of a filesystem with cyclic symlinks.
+ if (!RecursiveDirSize(file_name, size, handler)) {
+ return false;
+ }
+ }
+ }
+ return true;
+}
+
+// Try to make directories to store file.
+void FileSystem::SetupFileDir(const StringPiece& filename,
+ MessageHandler* handler) {
+ size_t last_slash = filename.rfind('/');
+ if (last_slash != StringPiece::npos) {
+ StringPiece directory_name = filename.substr(0, last_slash);
+ if (!RecursivelyMakeDir(directory_name, handler)) {
+ // TODO(sligocki): Specify where dir creation failed?
+ handler->Message(kError, "Could not create directories for file %s",
+ filename.as_string().c_str());
+ }
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/file_system_lock_manager.cc b/trunk/src/net/instaweb/util/file_system_lock_manager.cc
new file mode 100644
index 0000000..737a597
--- /dev/null
+++ b/trunk/src/net/instaweb/util/file_system_lock_manager.cc
@@ -0,0 +1,84 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+#include "net/instaweb/util/public/file_system_lock_manager.h"
+
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/named_lock_manager.h"
+#include "net/instaweb/util/public/timer.h"
+#include "net/instaweb/util/public/timer_based_abstract_lock.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class FileSystemLock : public TimerBasedAbstractLock {
+ public:
+ virtual ~FileSystemLock() {
+ if (held_) {
+ Unlock();
+ }
+ }
+ virtual bool TryLock() {
+ bool result = false;
+ if (manager_->file_system()->
+ TryLock(name_, manager_->handler()).is_true()) {
+ held_ = result = true;
+ }
+ return result;
+ }
+ virtual bool TryLockStealOld(int64 timeout_ms) {
+ bool result = false;
+ if (manager_->file_system()->
+ TryLockWithTimeout(name_, timeout_ms, manager_->handler()).is_true()) {
+ held_ = result = true;
+ }
+ return result;
+ }
+ virtual void Unlock() {
+ held_ = !manager_->file_system()->Unlock(name_, manager_->handler());
+ }
+ protected:
+ virtual Timer* timer() const {
+ return manager_->timer();
+ }
+ private:
+ friend class FileSystemLockManager;
+
+ // ctor should only be called by CreateNamedLock below.
+ FileSystemLock(const StringPiece& name, FileSystemLockManager* manager)
+ : name_(name.data(), name.size()),
+ manager_(manager),
+ held_(false) { }
+
+ std::string name_;
+ FileSystemLockManager* manager_;
+ // The held_ field contains an approximation to whether the lock is locked or
+ // not. If we believe the lock to be locked, we will unlock on destruction.
+ // We therefore try to conservatively leave it "true" when we aren't sure.
+ bool held_;
+
+ DISALLOW_COPY_AND_ASSIGN(FileSystemLock);
+};
+
+FileSystemLockManager::~FileSystemLockManager() { }
+
+AbstractLock* FileSystemLockManager::CreateNamedLock(const StringPiece& name) {
+ return new FileSystemLock(name, this);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/file_system_test.cc b/trunk/src/net/instaweb/util/file_system_test.cc
new file mode 100644
index 0000000..9ade2af
--- /dev/null
+++ b/trunk/src/net/instaweb/util/file_system_test.cc
@@ -0,0 +1,305 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include <unistd.h>
+#include "base/logging.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/file_system_test.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include <string>
+#include "net/instaweb/util/public/gtest.h"
+
+namespace net_instaweb {
+
+FileSystemTest::FileSystemTest() { }
+FileSystemTest::~FileSystemTest() { }
+
+std::string FileSystemTest::WriteNewFile(const StringPiece& suffix,
+ const std::string& content) {
+ std::string filename = StrCat(test_tmpdir(), suffix);
+
+ // Make sure we don't read an old file.
+ DeleteRecursively(filename);
+ EXPECT_TRUE(file_system()->WriteFile(filename.c_str(), content, &handler_));
+
+ return filename;
+}
+
+// Check that a file has been read.
+void FileSystemTest::CheckRead(const std::string& filename,
+ const std::string& expected_contents) {
+ std::string buffer;
+ ASSERT_TRUE(file_system()->ReadFile(filename.c_str(), &buffer, &handler_));
+ EXPECT_EQ(buffer, expected_contents);
+}
+
+// Make sure we can no longer read the file by the old name. Note
+// that this will spew some error messages into the log file, and
+// we can add a null_message_handler implementation to
+// swallow them, if they become annoying.
+void FileSystemTest::CheckDoesNotExist(const std::string& filename) {
+ std::string read_buffer;
+ EXPECT_FALSE(file_system()->ReadFile(filename.c_str(), &read_buffer,
+ &handler_));
+ EXPECT_TRUE(file_system()->Exists(filename.c_str(), &handler_).is_false());
+}
+
+// Write a named file, then read it.
+void FileSystemTest::TestWriteRead() {
+ std::string filename = test_tmpdir() + "/write.txt";
+ std::string msg("Hello, world!");
+
+ DeleteRecursively(filename);
+ FileSystem::OutputFile* ofile = file_system()->OpenOutputFile(
+ filename.c_str(), &handler_);
+ ASSERT_TRUE(ofile != NULL);
+ EXPECT_TRUE(ofile->Write(msg, &handler_));
+ EXPECT_TRUE(file_system()->Close(ofile, &handler_));
+ CheckRead(filename, msg);
+}
+
+// Write a temp file, then read it.
+void FileSystemTest::TestTemp() {
+ std::string prefix = test_tmpdir() + "/temp_prefix";
+ FileSystem::OutputFile* ofile = file_system()->OpenTempFile(
+ prefix.c_str(), &handler_);
+ ASSERT_TRUE(ofile != NULL);
+ std::string filename(ofile->filename());
+ std::string msg("Hello, world!");
+ EXPECT_TRUE(ofile->Write(msg, &handler_));
+ EXPECT_TRUE(file_system()->Close(ofile, &handler_));
+
+ CheckRead(filename, msg);
+}
+
+// Write a temp file, rename it, then read it.
+void FileSystemTest::TestRename() {
+ std::string from_text = "Now is time time";
+ std::string to_file = test_tmpdir() + "/to.txt";
+ DeleteRecursively(to_file);
+
+ std::string from_file = WriteNewFile("/from.txt", from_text);
+ ASSERT_TRUE(file_system()->RenameFile(from_file.c_str(), to_file.c_str(),
+ &handler_));
+
+ CheckDoesNotExist(from_file);
+ CheckRead(to_file, from_text);
+}
+
+// Write a file and successfully delete it.
+void FileSystemTest::TestRemove() {
+ std::string filename = WriteNewFile("/remove.txt", "Goodbye, world!");
+ ASSERT_TRUE(file_system()->RemoveFile(filename.c_str(), &handler_));
+ CheckDoesNotExist(filename);
+}
+
+// Write a file and check that it exists.
+void FileSystemTest::TestExists() {
+ std::string filename = WriteNewFile("/exists.txt", "I'm here.");
+ ASSERT_TRUE(file_system()->Exists(filename.c_str(), &handler_).is_true());
+}
+
+// Create a file along with its directory which does not exist.
+void FileSystemTest::TestCreateFileInDir() {
+ std::string dir_name = test_tmpdir() + "/make_dir";
+ DeleteRecursively(dir_name);
+ std::string filename = dir_name + "/file-in-dir.txt";
+
+ FileSystem::OutputFile* file =
+ file_system()->OpenOutputFile(filename.c_str(), &handler_);
+ ASSERT_TRUE(file);
+ file_system()->Close(file, &handler_);
+}
+
+
+// Make a directory and check that files may be placed in it.
+void FileSystemTest::TestMakeDir() {
+ std::string dir_name = test_tmpdir() + "/make_dir";
+ DeleteRecursively(dir_name);
+ std::string filename = dir_name + "/file-in-dir.txt";
+
+ ASSERT_TRUE(file_system()->MakeDir(dir_name.c_str(), &handler_));
+ // ... but we can open a file after we've created the directory.
+ FileSystem::OutputFile* file =
+ file_system()->OpenOutputFile(filename.c_str(), &handler_);
+ ASSERT_TRUE(file);
+ file_system()->Close(file, &handler_);
+}
+
+// Make a directory and check that it is a directory.
+void FileSystemTest::TestIsDir() {
+ std::string dir_name = test_tmpdir() + "/this_is_a_dir";
+ DeleteRecursively(dir_name);
+
+ // Make sure we don't think the directory is there when it isn't ...
+ ASSERT_TRUE(file_system()->IsDir(dir_name.c_str(), &handler_).is_false());
+ ASSERT_TRUE(file_system()->MakeDir(dir_name.c_str(), &handler_));
+ // ... and that we do think it's there when it is.
+ ASSERT_TRUE(file_system()->IsDir(dir_name.c_str(), &handler_).is_true());
+
+ // Make sure that we don't think a regular file is a directory.
+ std::string filename = dir_name + "/this_is_a_file.txt";
+ std::string content = "I'm not a directory.";
+ ASSERT_TRUE(file_system()->WriteFile(filename.c_str(), content, &handler_));
+ ASSERT_TRUE(file_system()->IsDir(filename.c_str(), &handler_).is_false());
+}
+
+// Recursively make directories and check that it worked.
+void FileSystemTest::TestRecursivelyMakeDir() {
+ std::string base = test_tmpdir() + "/base";
+ std::string long_path = base + "/dir/of/a/really/deep/hierarchy";
+ DeleteRecursively(base);
+
+ // Make sure we don't think the directory is there when it isn't ...
+ ASSERT_TRUE(file_system()->IsDir(long_path.c_str(), &handler_).is_false());
+ ASSERT_TRUE(file_system()->RecursivelyMakeDir(long_path, &handler_));
+ // ... and that we do think it's there when it is.
+ ASSERT_TRUE(file_system()->IsDir(long_path.c_str(), &handler_).is_true());
+}
+
+// Check that we cannot create a directory we do not have permissions for.
+// Note: depends upon root dir not being writable.
+void FileSystemTest::TestRecursivelyMakeDir_NoPermission() {
+ std::string base = "/bogus-dir";
+ std::string path = base + "/no/permission/to/make/this/dir";
+
+ // Make sure the bogus bottom level directory is not there.
+ ASSERT_TRUE(file_system()->Exists(base.c_str(), &handler_).is_false());
+ // We do not have permission to create it.
+ ASSERT_FALSE(file_system()->RecursivelyMakeDir(path, &handler_));
+}
+
+// Check that we cannot create a directory below a file.
+void FileSystemTest::TestRecursivelyMakeDir_FileInPath() {
+ std::string base = test_tmpdir() + "/file-in-path";
+ std::string filename = base + "/this-is-a-file";
+ std::string bad_path = filename + "/some/more/path";
+ DeleteRecursively(base);
+ std::string content = "Your path must end here. You shall not pass!";
+
+ ASSERT_TRUE(file_system()->MakeDir(base.c_str(), &handler_));
+ ASSERT_TRUE(file_system()->WriteFile(filename.c_str(), content, &handler_));
+ ASSERT_FALSE(file_system()->RecursivelyMakeDir(bad_path, &handler_));
+}
+
+void FileSystemTest::TestListContents() {
+ std::string dir_name = test_tmpdir() + "/make_dir";
+ DeleteRecursively(dir_name);
+ std::string filename1 = dir_name + "/file-in-dir.txt";
+ std::string filename2 = dir_name + "/another-file-in-dir.txt";
+ std::string content = "Lorem ipsum dolor sit amet";
+
+ StringVector mylist;
+
+ ASSERT_TRUE(file_system()->MakeDir(dir_name.c_str(), &handler_));
+ ASSERT_TRUE(file_system()->WriteFile(filename1.c_str(),
+ content, &handler_));
+ ASSERT_TRUE(file_system()->WriteFile(filename2.c_str(),
+ content, &handler_));
+ EXPECT_TRUE(file_system()->ListContents(dir_name, &mylist, &handler_));
+ EXPECT_EQ(size_t(2), mylist.size());
+ // Make sure our filenames are in there
+ EXPECT_FALSE(filename1.compare(mylist.at(0))
+ && filename1.compare(mylist.at(1)));
+ EXPECT_FALSE(filename2.compare(mylist.at(0))
+ && filename2.compare(mylist.at(1)));
+}
+
+void FileSystemTest::TestAtime() {
+ std::string dir_name = test_tmpdir() + "/make_dir";
+ DeleteRecursively(dir_name);
+ std::string filename1 = "file-in-dir.txt";
+ std::string filename2 = "another-file-in-dir.txt";
+ std::string full_path1 = dir_name + "/" + filename1;
+ std::string full_path2 = dir_name + "/" + filename2;
+ std::string content = "Lorem ipsum dolor sit amet";
+ // We need to sleep a bit between accessing files so that the
+ // difference shows up in in atimes which are measured in seconds.
+ unsigned int sleep_micros = 1500000;
+
+ ASSERT_TRUE(file_system()->MakeDir(dir_name.c_str(), &handler_));
+ ASSERT_TRUE(file_system()->WriteFile(full_path1.c_str(),
+ content, &handler_));
+ ASSERT_TRUE(file_system()->WriteFile(full_path2.c_str(),
+ content, &handler_));
+
+ int64 atime1, atime2;
+ CheckRead(full_path1, content);
+ usleep(sleep_micros);
+ CheckRead(full_path2, content);
+ ASSERT_TRUE(file_system()->Atime(full_path1, &atime1, &handler_));
+ ASSERT_TRUE(file_system()->Atime(full_path2, &atime2, &handler_));
+ EXPECT_LT(atime1, atime2);
+
+ CheckRead(full_path2, content);
+ usleep(sleep_micros);
+ CheckRead(full_path1, content);
+ ASSERT_TRUE(file_system()->Atime(full_path1, &atime1, &handler_));
+ ASSERT_TRUE(file_system()->Atime(full_path2, &atime2, &handler_));
+ EXPECT_LT(atime2, atime1);
+
+}
+
+void FileSystemTest::TestSize() {
+ std::string dir_name = test_tmpdir() + "/make_dir";
+ DeleteRecursively(dir_name);
+ std::string dir_name2 = dir_name + "/make_dir2";
+ std::string filename1 = "file-in-dir.txt";
+ std::string filename2 = "another-file-in-dir.txt";
+ std::string full_path1 = dir_name2 + "/" + filename1;
+ std::string full_path2 = dir_name2 + "/" + filename2;
+ std::string content1 = "12345";
+ std::string content2 = "1234567890";
+ ASSERT_TRUE(file_system()->MakeDir(dir_name.c_str(), &handler_));
+ ASSERT_TRUE(file_system()->MakeDir(dir_name2.c_str(), &handler_));
+ ASSERT_TRUE(file_system()->WriteFile(full_path1.c_str(),
+ content1, &handler_));
+ ASSERT_TRUE(file_system()->WriteFile(full_path2.c_str(),
+ content2, &handler_));
+ int64 size;
+
+ EXPECT_TRUE(file_system()->Size(full_path1, &size, &handler_));
+ EXPECT_EQ(content1.size(), size_t(size));
+ EXPECT_TRUE(file_system()->Size(full_path2, &size, &handler_));
+ EXPECT_EQ(content2.size(), size_t(size));
+ size = 0;
+ EXPECT_TRUE(file_system()->RecursiveDirSize(dir_name2, &size, &handler_));
+ EXPECT_EQ(content1.size() + content2.size(), size_t(size));
+ size = 0;
+ EXPECT_TRUE(file_system()->RecursiveDirSize(dir_name, &size, &handler_));
+ EXPECT_EQ(content1.size() + content2.size(), size_t(size));
+}
+
+void FileSystemTest::TestLock() {
+ std::string dir_name = test_tmpdir() + "/make_dir";
+ DeleteRecursively(dir_name);
+ ASSERT_TRUE(file_system()->MakeDir(dir_name.c_str(), &handler_));
+ std::string lock_name = dir_name + "/lock";
+ // Acquire the lock
+ EXPECT_TRUE(file_system()->TryLock(lock_name, &handler_).is_true());
+ // Can't re-acquire the lock
+ EXPECT_TRUE(file_system()->TryLock(lock_name, &handler_).is_false());
+ // Release the lock
+ EXPECT_TRUE(file_system()->Unlock(lock_name, &handler_));
+ // Do it all again to make sure the release worked.
+ EXPECT_TRUE(file_system()->TryLock(lock_name, &handler_).is_true());
+ EXPECT_TRUE(file_system()->TryLock(lock_name, &handler_).is_false());
+ EXPECT_TRUE(file_system()->Unlock(lock_name, &handler_));
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/file_writer.cc b/trunk/src/net/instaweb/util/file_writer.cc
new file mode 100644
index 0000000..bd85aff
--- /dev/null
+++ b/trunk/src/net/instaweb/util/file_writer.cc
@@ -0,0 +1,34 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/file_writer.h"
+
+namespace net_instaweb {
+
+FileWriter::~FileWriter() {
+}
+
+bool FileWriter::Write(const StringPiece& str, MessageHandler* handler) {
+ return file_->Write(str, handler);
+}
+
+bool FileWriter::Flush(MessageHandler* message_handler) {
+ return file_->Flush(message_handler);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/filename_encoder.cc b/trunk/src/net/instaweb/util/filename_encoder.cc
new file mode 100644
index 0000000..0e77d0e
--- /dev/null
+++ b/trunk/src/net/instaweb/util/filename_encoder.cc
@@ -0,0 +1,42 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/public/filename_encoder.h"
+#include <vector>
+using net::UrlToFilenameEncoder;
+
+namespace net_instaweb {
+
+FilenameEncoder::~FilenameEncoder() {
+}
+
+void FilenameEncoder::Encode(const StringPiece& filename_prefix,
+ const StringPiece& filename_ending,
+ std::string* encoded_filename) {
+ UrlToFilenameEncoder::EncodeSegment(filename_prefix.as_string(),
+ filename_ending.as_string(),
+ '/', encoded_filename);
+}
+
+bool FilenameEncoder::Decode(const StringPiece& encoded_filename,
+ std::string* decoded_url) {
+ return UrlToFilenameEncoder::Decode(encoded_filename.as_string(), '/',
+ decoded_url);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/filename_encoder_test.cc b/trunk/src/net/instaweb/util/filename_encoder_test.cc
new file mode 100644
index 0000000..6647ea3
--- /dev/null
+++ b/trunk/src/net/instaweb/util/filename_encoder_test.cc
@@ -0,0 +1,148 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/public/filename_encoder.h"
+
+#include <string>
+#include <vector>
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace {
+// net::kMaximumSubdirectoryLength is defined in url_to_filename_encoder.cc, but
+// we cannot link it.
+const size_t kMaxLen = 128;
+} // namespace
+
+namespace net_instaweb {
+
+// Note -- the exact behavior of the encoder is tested in
+// gfe/tools/loadtesting/spdy_testing/url_to_filename_encoder_test.cc
+//
+// Here we just test that the names meet certain properties:
+// 1. The segments are small
+// 2. The URL can be recovered from the filename
+// 3. No invalid filename characters are present.
+class FilenameEncoderTest : public ::testing::Test {
+ public:
+ FilenameEncoderTest() { }
+
+ protected:
+ void CheckSegmentLength(const StringPiece& escaped_word) {
+ std::vector<StringPiece> components;
+ SplitStringPieceToVector(escaped_word, "/", &components, false);
+ for (size_t i = 0; i < components.size(); ++i) {
+ EXPECT_GE(kMaxLen, components[i].size());
+ }
+ }
+
+ void CheckValidChars(const StringPiece& escaped_word) {
+ // These characters are invalid in Windows. We will
+ // ignore / for this test, but add in '.
+ // See http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx
+ static const char kInvalidChars[] = "<>:\"\\|?*'";
+ for (size_t i = 0; i < escaped_word.size(); ++i) {
+ char c = escaped_word[i];
+ EXPECT_TRUE(NULL == strchr(kInvalidChars, c));
+ }
+ }
+
+ void Validate(const std::string& in_word) {
+ std::string escaped_word, decoded_url;
+ encoder.Encode("", in_word, &escaped_word);
+ CheckSegmentLength(escaped_word);
+ CheckValidChars(escaped_word);
+ EXPECT_TRUE(encoder.Decode(escaped_word, &decoded_url));
+ EXPECT_EQ(in_word, decoded_url);
+ }
+
+ FilenameEncoder encoder;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(FilenameEncoderTest);
+};
+
+TEST_F(FilenameEncoderTest, DoesNotEscapeAlphanum) {
+ Validate("");
+ Validate("abcdefg");
+ Validate("abcdefghijklmnopqrstuvwxyz");
+ Validate("ZYXWVUT");
+ Validate("ZYXWVUTSRQPONMLKJIHGFEDCBA");
+ Validate("01234567689");
+ Validate("/-_");
+ Validate("abcdefghijklmnopqrstuvwxyzZYXWVUTSRQPONMLKJIHGFEDCBA"
+ "01234567689/-_");
+}
+
+TEST_F(FilenameEncoderTest, DoesEscapeNonAlphanum) {
+ Validate(".");
+ Validate("`~!@#$%^&*()_=+[{]}\\|;:'\",<.>?");
+}
+
+TEST_F(FilenameEncoderTest, DoesEscapeCorrectly) {
+ Validate("index.html");
+ Validate("search?q=dogs&go=&form=QBLH&qs=n");
+ Validate("~joebob/my_neeto-website+with_stuff.asp?id=138&content=true");
+}
+
+TEST_F(FilenameEncoderTest, LongTail) {
+ static char long_word[] =
+ "~joebob/briggs/12345678901234567890123456789012345678901234567890"
+ "1234567890123456789012345678901234567890123456789012345678901234567890"
+ "1234567890123456789012345678901234567890123456789012345678901234567890"
+ "1234567890123456789012345678901234567890123456789012345678901234567890"
+ "1234567890123456789012345678901234567890123456789012345678901234567890"
+ "1234567890123456789012345678901234567890123456789012345678901234567890";
+ Validate(long_word);
+}
+
+TEST_F(FilenameEncoderTest, LongTailDots) {
+ // Here the '.' in the last path segment expands to x2E, making
+ // it hits 128 chars before the input segment gets that big.
+ static char long_word[] =
+ "~joebob/briggs/1234567.1234567.1234567.1234567.1234567."
+ "1234567.1234567.1234567.1234567.1234567.1234567.1234567."
+ "1234567.1234567.1234567.1234567.1234567.1234567.1234567."
+ "1234567.1234567.1234567.1234567.1234567.1234567.1234567."
+ "1234567.1234567.1234567.1234567.1234567.1234567.1234567."
+ "1234567.1234567.1234567.1234567.1234567.1234567.1234567.";
+ Validate(long_word);
+}
+
+TEST_F(FilenameEncoderTest, CornerCasesNearMaxLenNoEscape) {
+ // hit corner cases, +/- 4 characters from kMaxLen
+ for (int i = -4; i <= 4; ++i) {
+ std::string input;
+ input.append(i + kMaxLen, 'x');
+ Validate(input);
+ }
+}
+
+TEST_F(FilenameEncoderTest, CornerCasesNearMaxLenWithEscape) {
+ // hit corner cases, +/- 4 characters from kMaxLen. This time we
+ // leave off the last 'x' and put in a '.', which ensures that we
+ // are truncating with '/' *after* the expansion.
+ for (int i = -4; i <= 4; ++i) {
+ std::string input;
+ input.append(i + kMaxLen - 1, 'x');
+ input.append(1, '.'); // this will expand to 3 characters.
+ Validate(input);
+ }
+}
+
+} // namespace
diff --git a/trunk/src/net/instaweb/util/google_message_handler.cc b/trunk/src/net/instaweb/util/google_message_handler.cc
new file mode 100644
index 0000000..9940e84
--- /dev/null
+++ b/trunk/src/net/instaweb/util/google_message_handler.cc
@@ -0,0 +1,72 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "base/logging.h"
+
+namespace net_instaweb {
+
+void GoogleMessageHandler::MessageVImpl(MessageType type, const char* msg,
+ va_list args) {
+ switch (type) {
+ case kInfo:
+ LOG(INFO) << Format(msg, args);
+ break;
+ case kWarning:
+ LOG(WARNING) << Format(msg, args);
+ break;
+ case kError:
+ LOG(ERROR) << Format(msg, args);
+ break;
+ case kFatal:
+ LOG(FATAL) << Format(msg, args);
+ break;
+ }
+}
+
+void GoogleMessageHandler::FileMessageVImpl(MessageType type, const char* file,
+ int line, const char* msg,
+ va_list args) {
+ switch (type) {
+ case kInfo:
+ LOG(INFO) << file << ":" << line << ": " << Format(msg, args);
+ break;
+ case kWarning:
+ LOG(WARNING) << file << ":" << line << ": " << Format(msg, args);
+ break;
+ case kError:
+ LOG(ERROR) << file << ":" << line << ": " << Format(msg, args);
+ break;
+ case kFatal:
+ LOG(FATAL) << file << ":" << line << ": " << Format(msg, args);
+ break;
+ }
+}
+
+// TODO(sligocki): It'd be nice not to do so much string copying.
+std::string GoogleMessageHandler::Format(const char* msg, va_list args) {
+ std::string buffer;
+
+ // Ignore the name of this routine: it formats with vsnprintf.
+ // See base/stringprintf.cc.
+ StringAppendV(&buffer, msg, args);
+ return buffer;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/google_url.cc b/trunk/src/net/instaweb/util/google_url.cc
new file mode 100644
index 0000000..2070d2b
--- /dev/null
+++ b/trunk/src/net/instaweb/util/google_url.cc
@@ -0,0 +1,124 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+
+std::string GoogleUrl::AllExceptLeaf(const GURL& gurl) {
+ CHECK(gurl.is_valid());
+ std::string spec_str = gurl.spec();
+
+ // Find the last slash before the question-mark, if any. See
+ // http://en.wikipedia.org/wiki/URI_scheme -- the query-string
+ // syntax is not well-defined. But the query-separator is well-defined:
+ // it's a ? so I believe this implies that the first ? has to delimit
+ // the query string.
+ std::string::size_type question = spec_str.find('?');
+ std::string::size_type last_slash = spec_str.find_last_of('/', question);
+ CHECK(last_slash != std::string::npos);
+ return std::string(spec_str.data(), last_slash + 1);
+
+ // TODO(jmarantz): jmaessen suggests using GURL to do the above this
+ // way:
+ // void ResourceNamer::SetNonResourceFields(const GURL& origin) {
+ // // This code looks absolutely horrible, but appears to be the most
+ // // graceful means available to pull the information we need, intact, out
+ // // of origin.
+ // const std::string& spec = origin.possibly_invalid_spec();
+ // const url_parse::Parsed& parsed =
+ // origin.parsed_for_possibly_invalid_spec();
+ // url_parse::Component filename;
+ // url_parse::ExtractFileName(spec.data(), parsed.path, &filename);
+ // // By default, slice off trailing / character from path.
+ // size_t path_len = filename.begin - 1;
+ // if (spec[path_len] != kPathSeparatorChar) {
+ // // We have an incomplete path of some sort without a trailing /
+ // // so include all the characters in the path.
+ // path_len++;
+ // }
+ // host_and_path_.assign(spec.data(), 0, path_len);
+ // size_t filename_end = filename.begin + filename.len;
+ // size_t ext_begin = filename_end;
+ // size_t ext_len = 0;
+ // size_t filename_len = filename.len;
+ // if (filename_len > 0) {
+ // ext_begin = spec.rfind(kSeparatorChar, filename_end);
+ // if (ext_begin < static_cast<size_t>(filename.begin) ||
+ // ext_begin == spec.npos) {
+ // ext_begin = filename_end;
+ // } else {
+ // filename_len = ext_begin - filename.begin;
+ // ext_begin++;
+ // ext_len = filename_end - ext_begin;
+ // }
+ // }
+ // name_.assign(spec.data(), filename.begin, filename_len);
+ // ext_.assign(spec.data(), ext_begin, ext_len);
+ // query_params_.assign(spec.data() + filename_end,
+ // spec.size() - filename_end);
+ // }
+ //
+ // You only need about half these lines, since the above code also fishes
+ // around for an extension and splits off the query params.
+}
+
+std::string GoogleUrl::Leaf(const GURL& gurl) {
+ std::string spec_str = gurl.spec();
+ std::string::size_type question = spec_str.find('?');
+ std::string::size_type last_slash = spec_str.find_last_of('/', question);
+ CHECK(last_slash != std::string::npos);
+ return std::string(spec_str.data() + last_slash + 1,
+ spec_str.size() - (last_slash + 1));
+}
+
+// For "http://a.com/b/c/d?e=f/g returns "http://a.com" without trailing slash
+std::string GoogleUrl::Origin(const GURL& gurl) {
+ std::string spec = gurl.spec();
+ size_t origin_size = gurl.GetWithEmptyPath().spec().size();
+ CHECK_LT(0, static_cast<int>(origin_size));
+ --origin_size; // skip trailing slash
+ CHECK_LE(origin_size, spec.size());
+ CHECK_EQ('/', spec[origin_size]);
+ return std::string(spec.data(), origin_size);
+}
+
+// For "http://a.com/b/c/d?e=f/g returns "/b/c/d?e=f/g" including leading slash
+std::string GoogleUrl::PathAndLeaf(const GURL& gurl) {
+ std::string spec = gurl.spec();
+ size_t origin_size = gurl.GetWithEmptyPath().spec().size();
+ CHECK_LT(0, static_cast<int>(origin_size));
+ --origin_size; // include trailing slash
+ CHECK_LE(origin_size, spec.size());
+ CHECK_EQ('/', spec[origin_size]);
+ CHECK_LE(origin_size, spec.size());
+ return std::string(spec.data() + origin_size, spec.size() - origin_size);
+}
+
+// For "http://a.com/b/c/d?e=f/g returns "/b/c/d/" including leading and
+// trailing slashes.
+std::string GoogleUrl::PathSansLeaf(const GURL& gurl) {
+ std::string path_and_leaf = GoogleUrl::PathAndLeaf(gurl);
+ std::string leaf = GoogleUrl::Leaf(gurl);
+ CHECK_GE(path_and_leaf.size(), leaf.size());
+ return std::string(path_and_leaf.data(), path_and_leaf.size() - leaf.size());
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/google_url_test.cc b/trunk/src/net/instaweb/util/google_url_test.cc
new file mode 100644
index 0000000..b5341c2
--- /dev/null
+++ b/trunk/src/net/instaweb/util/google_url_test.cc
@@ -0,0 +1,82 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the string-splitter.
+
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace {
+
+const char kUrl[] = "http://a.com/b/c/d.ext?f=g/h";
+const char kUrlWithPort[] = "http://a.com:8080/b/c/d.ext?f=g/h";
+
+} // namespace
+
+namespace net_instaweb {
+
+class GoogleUrlTest : public testing::Test {
+ protected:
+ GoogleUrlTest() : gurl_(kUrl), gurl_with_port_(kUrlWithPort) {}
+
+ GURL gurl_;
+ GURL gurl_with_port_;
+};
+
+TEST_F(GoogleUrlTest, TestSpec) {
+ EXPECT_EQ(std::string(kUrl), GoogleUrl::Spec(gurl_));
+ EXPECT_EQ(std::string("http://a.com/b/c/"), GoogleUrl::AllExceptLeaf(gurl_));
+ EXPECT_EQ(std::string("d.ext?f=g/h"), GoogleUrl::Leaf(gurl_));
+ EXPECT_EQ(std::string("http://a.com"), GoogleUrl::Origin(gurl_));
+ EXPECT_EQ(std::string("/b/c/d.ext?f=g/h"), GoogleUrl::PathAndLeaf(gurl_));
+ EXPECT_EQ(std::string("/b/c/d.ext"), GoogleUrl::Path(gurl_));
+}
+
+TEST_F(GoogleUrlTest, TestSpecWithPort) {
+ EXPECT_EQ(std::string(kUrlWithPort), GoogleUrl::Spec(gurl_with_port_));
+ EXPECT_EQ(std::string("http://a.com:8080/b/c/"),
+ GoogleUrl::AllExceptLeaf(gurl_with_port_));
+ EXPECT_EQ(std::string("d.ext?f=g/h"), GoogleUrl::Leaf(gurl_with_port_));
+ EXPECT_EQ(std::string("http://a.com:8080"), GoogleUrl::Origin(gurl_with_port_));
+ EXPECT_EQ(std::string("/b/c/d.ext?f=g/h"),
+ GoogleUrl::PathAndLeaf(gurl_with_port_));
+ EXPECT_EQ(std::string("/b/c/d.ext"), GoogleUrl::Path(gurl_));
+ EXPECT_EQ(std::string("/b/c/"), GoogleUrl::PathSansLeaf(gurl_));
+}
+
+TEST_F(GoogleUrlTest, ResolveRelative) {
+ GURL base = GoogleUrl::Create(StringPiece("http://www.google.com"));
+ ASSERT_TRUE(base.is_valid());
+ GURL resolved = GoogleUrl::Resolve(base, "test.html");
+ ASSERT_TRUE(resolved.is_valid());
+ EXPECT_EQ(std::string("http://www.google.com/test.html"),
+ GoogleUrl::Spec(resolved));
+ EXPECT_EQ(std::string("/test.html"), GoogleUrl::Path(resolved));
+}
+
+TEST_F(GoogleUrlTest, ResolveAbsolute) {
+ GURL base = GoogleUrl::Create(StringPiece("http://www.google.com"));
+ ASSERT_TRUE(base.is_valid());
+ GURL resolved = GoogleUrl::Resolve(base, "http://www.google.com");
+ ASSERT_TRUE(resolved.is_valid());
+ EXPECT_EQ(std::string("http://www.google.com/"),
+ GoogleUrl::Spec(resolved));
+ EXPECT_EQ(std::string("/"), GoogleUrl::Path(resolved));
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/gtest.cc b/trunk/src/net/instaweb/util/gtest.cc
new file mode 100644
index 0000000..c7b90f9
--- /dev/null
+++ b/trunk/src/net/instaweb/util/gtest.cc
@@ -0,0 +1,62 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+//
+
+#include <unistd.h> // For getpid()
+
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/stack_buffer.h"
+
+namespace net_instaweb {
+
+
+std::string GTestSrcDir() {
+ // Climb up the directory hierarchy till we find "src".
+ // TODO(jmarantz): check to make sure we are not in a subdirectory of
+ // our top-level 'src' named src.
+
+ char cwd[kStackBufferSize];
+ CHECK(getcwd(cwd, sizeof(cwd)) != NULL);
+ std::vector<StringPiece> components;
+ SplitStringPieceToVector(cwd, "/", &components, true);
+ int level = components.size();
+ bool found = false;
+ for (int i = level - 1; i >= 0; --i) {
+ if (components[i] == "src") {
+ level = i + 1;
+ found = true;
+ break;
+ }
+ }
+
+ CHECK(found) << "Cannot find 'src' directory from cwd=" << cwd;
+ std::string src_dir;
+ for (int i = 0; i < level; ++i) {
+ src_dir += "/";
+ components[i].AppendToString(&src_dir);
+ }
+ return src_dir;
+}
+
+std::string GTestTempDir() {
+ return StringPrintf("/tmp/gtest.%d", getpid());
+}
+
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/gzip_inflater.cc b/trunk/src/net/instaweb/util/gzip_inflater.cc
new file mode 100644
index 0000000..5e2f0bf
--- /dev/null
+++ b/trunk/src/net/instaweb/util/gzip_inflater.cc
@@ -0,0 +1,180 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "net/instaweb/util/public/gzip_inflater.h"
+
+#include <stdlib.h>
+#include "base/logging.h"
+#ifdef USE_SYSTEM_ZLIB
+#include "zlib.h"
+#else
+#include "third_party/zlib/zlib.h"
+#endif
+
+namespace net_instaweb {
+
+GzipInflater::GzipInflater(InflateType type)
+ : zlib_(NULL),
+ finished_(false),
+ error_(false),
+ type_(type) {
+}
+
+GzipInflater::~GzipInflater() {
+ Free();
+}
+
+void GzipInflater::Free() {
+ if (zlib_ == NULL) {
+ // Already freed.
+ return;
+ }
+
+ int err = inflateEnd(zlib_);
+ if (err != Z_OK) {
+ error_ = true;
+ }
+
+ free(zlib_);
+
+ zlib_ = NULL;
+}
+
+bool GzipInflater::Init() {
+ if (zlib_ != NULL) {
+ return false;
+ }
+
+ zlib_ = static_cast<z_stream *>(malloc(sizeof(z_stream)));
+ if (zlib_ == NULL) {
+ return false;
+ }
+ memset(zlib_, 0, sizeof(z_stream));
+
+ // Tell zlib that the data is gzip-encoded or deflate-encode.
+ int window_size = 31; // window size of 15, plus 16 for gzip
+ if (type_ == kDeflate) {
+ window_size = -15; // window size of -15 for row deflate.
+ }
+ int err = inflateInit2(zlib_, window_size);
+
+ if (err != Z_OK) {
+ Free();
+ error_ = true;
+ return false;
+ }
+
+ return true;
+}
+
+bool GzipInflater::HasUnconsumedInput() const {
+ if (zlib_ == NULL) {
+ return false;
+ }
+
+ if (finished_ || error_) {
+ return false;
+ }
+
+ return zlib_->avail_in > 0;
+}
+
+bool GzipInflater::SetInput(const void *in, size_t in_size) {
+ if (zlib_ == NULL) {
+ return false;
+ }
+
+ if (HasUnconsumedInput()) {
+ return false;
+ }
+
+ if (finished_) {
+ return false;
+ }
+
+ if (error_) {
+ return false;
+ }
+
+ if (in == NULL || in_size == 0) {
+ return false;
+ }
+
+ // The zlib library won't modify the buffer, but it does not use
+ // const here, so we must const cast.
+ zlib_->next_in = const_cast<Bytef *>(reinterpret_cast<const Bytef *>(in));
+ zlib_->avail_in = static_cast<uInt>(in_size);
+
+ return true;
+}
+
+int GzipInflater::InflateBytes(char *buf, size_t buf_size) {
+ if (zlib_ == NULL) {
+ return -1;
+ }
+
+ if (!HasUnconsumedInput()) {
+ return -1;
+ }
+
+ if (finished_) {
+ return -1;
+ }
+
+ if (error_) {
+ return -1;
+ }
+
+ if (buf == NULL || buf_size == 0) {
+ return -1;
+ }
+
+ const size_t inflated_bytes_before = zlib_->total_out;
+
+ zlib_->next_out = reinterpret_cast<Bytef *>(buf);
+ zlib_->avail_out = static_cast<uInt>(buf_size);
+
+ int err = inflate(zlib_, Z_SYNC_FLUSH);
+
+ const size_t inflated_bytes = zlib_->total_out - inflated_bytes_before;
+
+ if (err == Z_STREAM_END) {
+ finished_ = true;
+ } else if (err == Z_OK) {
+ if (inflated_bytes < buf_size) {
+ // Sanity check that if we didn't fill the output buffer, it's
+ // because we consumed all of the input.
+ DCHECK(!HasUnconsumedInput());
+ }
+ } else if (err == Z_BUF_ERROR) {
+ // Sanity check that if we encountered this error, it's because we
+ // were unable to write any inflated bytes to the output
+ // buffer. zlib documentation says that this is a non-terminal
+ // error, so we do not set error_ to true here.
+ DCHECK_EQ(inflated_bytes, static_cast<size_t>(0));
+ } else {
+ error_ = true;
+ return -1;
+ }
+
+ return static_cast<int>(inflated_bytes);
+}
+
+void GzipInflater::ShutDown() {
+ Free();
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/gzip_inflater_test.cc b/trunk/src/net/instaweb/util/gzip_inflater_test.cc
new file mode 100644
index 0000000..7603a43
--- /dev/null
+++ b/trunk/src/net/instaweb/util/gzip_inflater_test.cc
@@ -0,0 +1,135 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <string>
+#include "net/instaweb/util/public/gzip_inflater.h"
+
+#include "testing/gtest/include/gtest/gtest.h"
+
+namespace net_instaweb {
+
+namespace {
+
+const char kBasic[] = "Hello\n";
+
+// The above string "Hello\n", gzip compressed.
+const unsigned char kCompressed[] = {
+ 0x1f, 0x8b, 0x08, 0x08, 0x38, 0x18, 0x2e, 0x4c, 0x00, 0x03, 0x63,
+ 0x6f, 0x6d, 0x70, 0x72, 0x65, 0x73, 0x73, 0x65, 0x64, 0x2e, 0x68,
+ 0x74, 0x6d, 0x6c, 0x00, 0xf3, 0x48, 0xcd, 0xc9, 0xc9, 0xe7, 0x02,
+ 0x00, 0x16, 0x35, 0x96, 0x31, 0x06, 0x00, 0x00, 0x00
+};
+
+// This is the word "test" deflated
+const char kTestDeflate[] ="\x2b\x49\x2d\x2e\x01";
+const char kTest[] = "test";
+
+const size_t kBufSize = 256;
+
+TEST(GzipInflaterTest, Simple) {
+ std::string buf;
+ buf.reserve(kBufSize);
+ GzipInflater inflater(GzipInflater::kGzip);
+ inflater.Init();
+ ASSERT_FALSE(inflater.HasUnconsumedInput());
+ ASSERT_TRUE(inflater.SetInput(kCompressed, sizeof(kCompressed)));
+ ASSERT_TRUE(inflater.HasUnconsumedInput());
+ int num_inflated_bytes = inflater.InflateBytes(&buf[0], kBufSize);
+ ASSERT_EQ(strlen(kBasic), static_cast<size_t>(num_inflated_bytes));
+ // null-terminate the buffer
+ buf[num_inflated_bytes] = '\0';
+ ASSERT_FALSE(inflater.HasUnconsumedInput());
+ ASSERT_TRUE(inflater.finished());
+ ASSERT_FALSE(inflater.error());
+ inflater.ShutDown();
+ ASSERT_STREQ(kBasic, buf.c_str());
+}
+
+TEST(GzipInflaterTest, OneByteAtATime) {
+ std::string buf;
+ buf.reserve(kBufSize);
+ GzipInflater inflater(GzipInflater::kGzip);
+ inflater.Init();
+ int num_inflated_bytes = 0;
+ ASSERT_FALSE(inflater.HasUnconsumedInput());
+ for (size_t input_offset = 0;
+ input_offset < sizeof(kCompressed);
+ ++input_offset) {
+ ASSERT_TRUE(inflater.SetInput(&kCompressed[input_offset], 1));
+ ASSERT_TRUE(inflater.HasUnconsumedInput());
+ num_inflated_bytes +=
+ inflater.InflateBytes(&buf[num_inflated_bytes],
+ kBufSize - num_inflated_bytes);
+ }
+ ASSERT_EQ(strlen(kBasic), static_cast<size_t>(num_inflated_bytes));
+ // null-terminate the buffer
+ buf[num_inflated_bytes] = '\0';
+ ASSERT_FALSE(inflater.HasUnconsumedInput());
+ ASSERT_TRUE(inflater.finished());
+ ASSERT_FALSE(inflater.error());
+ inflater.ShutDown();
+ ASSERT_STREQ(kBasic, buf.c_str());
+}
+
+TEST(GzipInflaterTest, SimpleDeflate) {
+ std::string buf;
+ buf.reserve(kBufSize);
+ GzipInflater inflater(GzipInflater::kDeflate);
+ inflater.Init();
+ ASSERT_FALSE(inflater.HasUnconsumedInput());
+ ASSERT_TRUE(inflater.SetInput(kTestDeflate, sizeof(kTestDeflate)));
+ ASSERT_TRUE(inflater.HasUnconsumedInput());
+ int num_inflated_bytes = inflater.InflateBytes(&buf[0], kBufSize);
+ ASSERT_EQ(strlen(kTest), static_cast<size_t>(num_inflated_bytes));
+ // null-terminate the buffer
+ buf[num_inflated_bytes] = '\0';
+ ASSERT_FALSE(inflater.HasUnconsumedInput());
+ ASSERT_TRUE(inflater.finished());
+ ASSERT_FALSE(inflater.error());
+ inflater.ShutDown();
+ ASSERT_STREQ(kTest, buf.c_str());
+}
+
+TEST(GzipInflaterTest, OneByteAtATimeDeflate) {
+ std::string buf;
+ buf.reserve(kBufSize);
+ GzipInflater inflater(GzipInflater::kDeflate);
+ inflater.Init();
+ int num_inflated_bytes = 0;
+ ASSERT_FALSE(inflater.HasUnconsumedInput());
+ for (size_t input_offset = 0;
+ input_offset < sizeof(kTestDeflate);
+ ++input_offset) {
+ ASSERT_TRUE(inflater.SetInput(&kTestDeflate[input_offset], 1));
+ ASSERT_TRUE(inflater.HasUnconsumedInput());
+ num_inflated_bytes +=
+ inflater.InflateBytes(&buf[num_inflated_bytes],
+ kBufSize - num_inflated_bytes);
+ }
+ ASSERT_EQ(strlen(kTest), static_cast<size_t>(num_inflated_bytes));
+ // null-terminate the buffer
+ buf[num_inflated_bytes] = '\0';
+ ASSERT_FALSE(inflater.HasUnconsumedInput());
+ ASSERT_TRUE(inflater.finished());
+ ASSERT_FALSE(inflater.error());
+ inflater.ShutDown();
+ ASSERT_STREQ(kTest, buf.c_str());
+}
+
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/hasher.cc b/trunk/src/net/instaweb/util/hasher.cc
new file mode 100644
index 0000000..5b5ef4c
--- /dev/null
+++ b/trunk/src/net/instaweb/util/hasher.cc
@@ -0,0 +1,26 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/public/hasher.h"
+
+namespace net_instaweb {
+
+Hasher::~Hasher() {
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/http_cache.cc b/trunk/src/net/instaweb/util/http_cache.cc
new file mode 100644
index 0000000..0cd2232
--- /dev/null
+++ b/trunk/src/net/instaweb/util/http_cache.cc
@@ -0,0 +1,174 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/http_cache.h"
+#include "net/instaweb/util/public/cache_interface.h"
+#include "net/instaweb/util/public/http_value.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/meta_data.h"
+#include "net/instaweb/util/public/shared_string.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/statistics.h"
+#include "net/instaweb/util/public/timer.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// Remember that a Fetch failed for 5 minutes.
+// TODO(jmarantz): consider allowing this to be configurable.
+//
+// TODO(jmarantz): We could handle cc-private a little differently:
+// in this case we could arguably remember it using the original cc-private ttl.
+const char kRememberNotFoundCacheControl[] = "max-age=300";
+const char kCacheTimeUs[] = "cache_time_us";
+const char kCacheHits[] = "cache_hits";
+const char kCacheMisses[] = "cache_misses";
+const char kCacheExpirations[] = "cache_expirations";
+const char kCacheInserts[] = "cache_inserts";
+
+} // namespace
+
+
+HTTPCache::~HTTPCache() {}
+
+bool HTTPCache::IsCurrentlyValid(const MetaData& headers, int64 now_ms) {
+ if (force_caching_) {
+ return true;
+ }
+ if (!headers.IsCacheable() || !headers.IsProxyCacheable()) {
+ // TODO(jmarantz): Should we have a separate 'force' bit that doesn't
+ // expired resources to be valid, but does ignore cache-control:private?
+ return false;
+ }
+ if (headers.CacheExpirationTimeMs() > now_ms) {
+ return true;
+ }
+ if (cache_expirations_ != NULL) {
+ cache_expirations_->Add(1);
+ }
+ return false;
+}
+
+HTTPCache::FindResult HTTPCache::Find(
+ const std::string& key, HTTPValue* value, MetaData* headers,
+ MessageHandler* handler) {
+ SharedString cache_buffer;
+
+ int64 start_us = timer_->NowUs();
+ int64 now_ms = start_us / 1000;
+ FindResult ret = kNotFound;
+
+ if ((cache_->Get(key, &cache_buffer) &&
+ value->Link(&cache_buffer, headers, handler))) {
+ if (IsCurrentlyValid(*headers, now_ms)) {
+ if (headers->status_code() == HttpStatus::kRememberNotFoundStatusCode) {
+ long remember_not_found_time_ms = headers->CacheExpirationTimeMs()
+ - now_ms;
+ handler->Info(key.c_str(), 0,
+ "HTTPCache: remembering not-found status for %ld seconds",
+ remember_not_found_time_ms / 1000);
+ ret = kRecentFetchFailedDoNotRefetch;
+ } else {
+ ret = kFound;
+ }
+ }
+ }
+
+ if (cache_time_us_ != NULL) {
+ long delta_us = timer_->NowUs() - start_us;
+ cache_time_us_->Add(delta_us);
+ if (ret == kFound) {
+ cache_hits_->Add(1);
+ } else {
+ cache_misses_->Add(1);
+ }
+ }
+
+ if (ret != kFound) {
+ headers->Clear();
+ value->Clear();
+ }
+ return ret;
+}
+
+void HTTPCache::RememberNotCacheable(const std::string& key,
+ MessageHandler* handler) {
+ SimpleMetaData headers;
+ headers.set_status_code(HttpStatus::kRememberNotFoundStatusCode);
+ headers.Add(HttpAttributes::kCacheControl, kRememberNotFoundCacheControl);
+ int64 now_ms = timer_->NowMs();
+ headers.UpdateDateHeader(HttpAttributes::kDate, now_ms);
+ headers.ComputeCaching();
+ headers.set_headers_complete(true);
+ Put(key, headers, "", handler);
+}
+
+void HTTPCache::Put(const std::string& key, HTTPValue* value,
+ MessageHandler* handler) {
+ SharedString* shared_string = value->share();
+ cache_->Put(key, shared_string);
+}
+
+void HTTPCache::Put(const std::string& key, const MetaData& headers,
+ const StringPiece& content,
+ MessageHandler* handler) {
+ int64 start_us = timer_->NowUs();
+ int64 now_ms = start_us / 1000;
+ if (!IsCurrentlyValid(headers, now_ms)) {
+ return;
+ }
+
+ HTTPValue value;
+ value.SetHeaders(headers);
+ value.Write(content, handler);
+ Put(key, &value, handler);
+ if (cache_time_us_ != NULL) {
+ int64 delta_us = timer_->NowUs() - start_us;
+ cache_time_us_->Add(delta_us);
+ cache_inserts_->Add(1);
+ }
+}
+
+CacheInterface::KeyState HTTPCache::Query(const std::string& key) {
+ return cache_->Query(key);
+}
+
+void HTTPCache::Delete(const std::string& key) {
+ return cache_->Delete(key);
+}
+
+void HTTPCache::Initialize(Statistics* statistics) {
+ statistics->AddVariable(kCacheTimeUs);
+ statistics->AddVariable(kCacheHits);
+ statistics->AddVariable(kCacheMisses);
+ statistics->AddVariable(kCacheExpirations);
+ statistics->AddVariable(kCacheInserts);
+}
+
+void HTTPCache::SetStatistics(Statistics* statistics) {
+ if (statistics != NULL) {
+ cache_time_us_ = statistics->GetVariable(kCacheTimeUs);
+ cache_hits_ = statistics->GetVariable(kCacheHits);
+ cache_misses_ = statistics->GetVariable(kCacheMisses);
+ cache_expirations_ = statistics->GetVariable(kCacheExpirations);
+ cache_inserts_ = statistics->GetVariable(kCacheInserts);
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/http_cache_test.cc b/trunk/src/net/instaweb/util/http_cache_test.cc
new file mode 100644
index 0000000..34719aa
--- /dev/null
+++ b/trunk/src/net/instaweb/util/http_cache_test.cc
@@ -0,0 +1,141 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the lru cache
+
+#include "net/instaweb/util/public/http_cache.h"
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "net/instaweb/util/public/cache_interface.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/http_value.h"
+#include "net/instaweb/util/public/lru_cache.h"
+#include "net/instaweb/util/public/mock_timer.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace {
+// Set the cache size large enough so nothing gets evicted during this test.
+const int kMaxSize = 10000;
+const char kStartDate[] = "Sun, 16 Dec 1979 02:27:45 GMT";
+}
+
+namespace net_instaweb {
+
+class HTTPCacheTest : public testing::Test {
+ protected:
+ static int64 ParseDate(const char* start_date) {
+ int64 time_ms;
+ MetaData::ParseTime(start_date, &time_ms);
+ return time_ms;
+ }
+
+ HTTPCacheTest() : mock_timer_(ParseDate(kStartDate)),
+ http_cache_(new LRUCache(kMaxSize), &mock_timer_) {
+ }
+
+ void InitHeaders(MetaData* headers, const char* cache_control) {
+ headers->Add("name", "value");
+ headers->Add("Date", kStartDate);
+ if (cache_control != NULL) {
+ headers->Add("Cache-control", cache_control);
+ }
+ headers->SetStatusAndReason(HttpStatus::kOK);
+ headers->ComputeCaching();
+ }
+
+ MockTimer mock_timer_;
+ HTTPCache http_cache_;
+ GoogleMessageHandler message_handler_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(HTTPCacheTest);
+};
+
+// Simple flow of putting in an item, getting it.
+TEST_F(HTTPCacheTest, PutGet) {
+ SimpleMetaData meta_data_in, meta_data_out;
+ InitHeaders(&meta_data_in, "max-age=300");
+ http_cache_.Put("mykey", meta_data_in, "content", &message_handler_);
+ EXPECT_EQ(CacheInterface::kAvailable, http_cache_.Query("mykey"));
+ HTTPValue value;
+ HTTPCache::FindResult found = http_cache_.Find(
+ "mykey", &value, &meta_data_out, &message_handler_);
+ ASSERT_EQ(HTTPCache::kFound, found);
+ ASSERT_TRUE(meta_data_out.headers_complete());
+ StringPiece contents;
+ ASSERT_TRUE(value.ExtractContents(&contents));
+ CharStarVector values;
+ ASSERT_TRUE(meta_data_out.Lookup("name", &values));
+ ASSERT_EQ(static_cast<size_t>(1), values.size());
+ EXPECT_EQ(std::string("value"), std::string(values[0]));
+ EXPECT_EQ("content", contents);
+
+ // Now advance time 301 seconds and the we should no longer
+ // be able to fetch this resource out of the cache.
+ mock_timer_.advance_ms(301 * 1000);
+ found = http_cache_.Find("mykey", &value, &meta_data_out, &message_handler_);
+ ASSERT_EQ(HTTPCache::kNotFound, found);
+ ASSERT_FALSE(meta_data_out.headers_complete());
+}
+
+// Verifies that the cache will 'remember' that a fetch should not be
+// cached for 5 minutes.
+TEST_F(HTTPCacheTest, RememberNotCacheable) {
+ SimpleMetaData meta_data_out;
+ http_cache_.RememberNotCacheable("mykey", &message_handler_);
+ HTTPValue value;
+ EXPECT_EQ(HTTPCache::kRecentFetchFailedDoNotRefetch,
+ http_cache_.Find("mykey", &value, &meta_data_out,
+ &message_handler_));
+
+ // Now advance time 301 seconds; the cache should allow us to try fetching
+ // again.
+ mock_timer_.advance_ms(301 * 1000);
+ EXPECT_EQ(HTTPCache::kNotFound,
+ http_cache_.Find("mykey", &value, &meta_data_out,
+ &message_handler_));
+}
+
+TEST_F(HTTPCacheTest, Uncacheable) {
+ SimpleMetaData meta_data_in, meta_data_out;
+ InitHeaders(&meta_data_in, NULL);
+ http_cache_.Put("mykey", meta_data_in, "content", &message_handler_);
+ EXPECT_EQ(CacheInterface::kNotFound, http_cache_.Query("mykey"));
+ HTTPValue value;
+ HTTPCache::FindResult found = http_cache_.Find(
+ "mykey", &value, &meta_data_out, &message_handler_);
+ ASSERT_EQ(HTTPCache::kNotFound, found);
+ ASSERT_FALSE(meta_data_out.headers_complete());
+}
+
+TEST_F(HTTPCacheTest, UncacheablePrivate) {
+ SimpleMetaData meta_data_in, meta_data_out;
+ InitHeaders(&meta_data_in, "private, max-age=300");
+ http_cache_.Put("mykey", meta_data_in, "content", &message_handler_);
+ EXPECT_EQ(CacheInterface::kNotFound, http_cache_.Query("mykey"));
+ HTTPValue value;
+ HTTPCache::FindResult found = http_cache_.Find(
+ "mykey", &value, &meta_data_out, &message_handler_);
+ ASSERT_EQ(HTTPCache::kNotFound, found);
+ ASSERT_FALSE(meta_data_out.headers_complete());
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/http_dump_url_async_writer.cc b/trunk/src/net/instaweb/util/http_dump_url_async_writer.cc
new file mode 100644
index 0000000..e873f27
--- /dev/null
+++ b/trunk/src/net/instaweb/util/http_dump_url_async_writer.cc
@@ -0,0 +1,150 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/public/http_dump_url_async_writer.h"
+
+#include "net/instaweb/util/public/file_writer.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/meta_data.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/string_writer.h"
+
+namespace net_instaweb {
+
+class HttpDumpUrlAsyncWriter::Fetch : UrlAsyncFetcher::Callback {
+ public:
+ Fetch(const std::string& url, const MetaData& request_headers,
+ MetaData* response_headers, Writer* response_writer,
+ MessageHandler* handler, Callback* callback,
+ const std::string& filename, UrlFetcher* dump_fetcher,
+ FileSystem* file_system)
+ : url_(url), response_headers_(response_headers),
+ response_writer_(response_writer), handler_(handler),
+ callback_(callback), filename_(filename), dump_fetcher_(dump_fetcher),
+ file_system_(file_system), string_writer_(&contents_) {
+ request_headers_.CopyFrom(request_headers);
+ }
+
+ // Like UrlAsyncFetcher::StreamingFetch, returns true if callback has been
+ // called already.
+ bool StartFetch(const bool accept_gzip, UrlAsyncFetcher* base_fetcher) {
+ // In general we will want to always ask the origin for gzipped output,
+ // but we are leaving in variable so this could be overridden by the
+ // instantiator of the DumpUrlWriter.
+ compress_headers_.CopyFrom(request_headers_);
+ if (accept_gzip) {
+ compress_headers_.RemoveAll(HttpAttributes::kAcceptEncoding);
+ compress_headers_.Add(HttpAttributes::kAcceptEncoding,
+ HttpAttributes::kGzip);
+ }
+
+ return base_fetcher->StreamingFetch(url_, compress_headers_,
+ &compressed_response_, &string_writer_,
+ handler_, this);
+ }
+
+ // Finishes the Fetch when called back.
+ void Done(bool success) {
+ compressed_response_.RemoveAll(HttpAttributes::kContentLength);
+ compressed_response_.Add(HttpAttributes::kContentLength,
+ IntegerToString(contents_.size()).c_str());
+ compressed_response_.ComputeCaching();
+
+ // Do not write an empty file if the fetch failed.
+ if (success) {
+ FileSystem::OutputFile* file = file_system_->OpenTempFile(
+ filename_ + ".temp", handler_);
+ if (file != NULL) {
+ handler_->Message(kInfo, "Storing %s as %s", url_.c_str(),
+ filename_.c_str());
+ std::string temp_filename = file->filename();
+ FileWriter file_writer(file);
+ success = compressed_response_.Write(&file_writer, handler_) &&
+ file->Write(contents_, handler_);
+ success &= file_system_->Close(file, handler_);
+ success &= file_system_->RenameFile(temp_filename.c_str(),
+ filename_.c_str(),
+ handler_);
+ } else {
+ success = false;
+ }
+ }
+
+ // We are not going to be able to read the response from the file
+ // system so we better pass the error message through.
+ if (!success) {
+ response_headers_->CopyFrom(compressed_response_);
+ response_writer_->Write(contents_, handler_);
+ } else {
+ // Let dump fetcher fetch the actual response so that it can decompress.
+ success = dump_fetcher_->StreamingFetchUrl(
+ url_, request_headers_, response_headers_, response_writer_,
+ handler_);
+ }
+
+ callback_->Done(success);
+ delete this;
+ }
+
+ private:
+ const std::string url_;
+ SimpleMetaData request_headers_;
+ MetaData* response_headers_;
+ Writer* response_writer_;
+ MessageHandler* handler_;
+ Callback* callback_;
+
+ const std::string filename_;
+ UrlFetcher* dump_fetcher_;
+ FileSystem* file_system_;
+
+ std::string contents_;
+ StringWriter string_writer_;
+ SimpleMetaData compress_headers_;
+ SimpleMetaData compressed_response_;
+
+ DISALLOW_COPY_AND_ASSIGN(Fetch);
+};
+
+HttpDumpUrlAsyncWriter::~HttpDumpUrlAsyncWriter() {
+}
+
+bool HttpDumpUrlAsyncWriter::StreamingFetch(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* handler,
+ Callback* callback) {
+ std::string filename;
+ dump_fetcher_.GetFilename(GURL(url), &filename, handler);
+
+ if (file_system_->Exists(filename.c_str(), handler).is_true()) {
+ bool success = dump_fetcher_.StreamingFetchUrl(
+ url, request_headers, response_headers, response_writer, handler);
+ callback->Done(success);
+ return true;
+ } else {
+ Fetch* fetch = new Fetch(url, request_headers, response_headers,
+ response_writer, handler, callback, filename,
+ &dump_fetcher_, file_system_);
+ return fetch->StartFetch(accept_gzip_, base_fetcher_);
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/http_dump_url_async_writer_test.cc b/trunk/src/net/instaweb/util/http_dump_url_async_writer_test.cc
new file mode 100644
index 0000000..7027dca
--- /dev/null
+++ b/trunk/src/net/instaweb/util/http_dump_url_async_writer_test.cc
@@ -0,0 +1,111 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/public/http_dump_url_async_writer.h"
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/fetcher_test.h"
+#include "net/instaweb/util/public/mock_timer.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/stdio_file_system.h"
+
+namespace net_instaweb {
+
+// TODO(sligocki): Merge with CacheUrlAsyncFetcherTest and refactor into
+// FetcherTestBase.
+class HttpDumpUrlAsyncWriterTest : public FetcherTest {
+ protected:
+ HttpDumpUrlAsyncWriterTest()
+ : root_dir_(GTestTempDir() + "/http_dump_url_async_writer_test/"),
+ mock_timer_(0),
+ dump_fetcher_(root_dir_, &mock_async_fetcher_, &file_system_,
+ &mock_timer_) {
+ }
+
+ UrlAsyncFetcher* async_fetcher() { return &dump_fetcher_; }
+
+ std::string root_dir_;
+ StdioFileSystem file_system_;
+ MockTimer mock_timer_;
+ HttpDumpUrlAsyncWriter dump_fetcher_;
+};
+
+TEST_F(HttpDumpUrlAsyncWriterTest, TestCacheable) {
+ // With the async cached fetching interface, we will expect even the
+ // initial request to succeed, once the callbacks are run.
+ bool callback_called1, callback_called2, callback_called3;
+ EXPECT_EQ(1, CountFetchesAsync(kGoodUrl, true, &callback_called1));
+ EXPECT_FALSE(callback_called1);
+
+ EXPECT_EQ(1, CountFetchesAsync(kGoodUrl, true, &callback_called2));
+ EXPECT_FALSE(callback_called1);
+ EXPECT_FALSE(callback_called2);
+
+ mock_async_fetcher_.CallCallbacks();
+ EXPECT_TRUE(callback_called1);
+ EXPECT_TRUE(callback_called2);
+
+ EXPECT_EQ(0, CountFetchesAsync(kGoodUrl, true, &callback_called3));
+ // No async fetcher callbacks were queued because the content
+ // was cached, so no need to call CallCallbacks() again here.
+ EXPECT_TRUE(callback_called3);
+}
+
+TEST_F(HttpDumpUrlAsyncWriterTest, TestNotCacheable) {
+ // With the async cached fetching interface, we will expect even the
+ // initial request to succeed, once the callbacks are run.
+ bool callback_called1, callback_called2, callback_called3;
+ EXPECT_EQ(1, CountFetchesAsync(kNotCachedUrl, true, &callback_called1));
+ EXPECT_FALSE(callback_called1);
+
+ EXPECT_EQ(1, CountFetchesAsync(kNotCachedUrl, true, &callback_called2));
+ EXPECT_FALSE(callback_called1);
+ EXPECT_FALSE(callback_called2);
+
+ mock_async_fetcher_.CallCallbacks();
+ EXPECT_TRUE(callback_called1);
+ EXPECT_TRUE(callback_called2);
+
+ // This is not a proper cache and does not distinguish between cacheable
+ // or non-cacheable URLs.
+ EXPECT_EQ(0, CountFetchesAsync(kNotCachedUrl, true, &callback_called3));
+ EXPECT_TRUE(callback_called3);
+}
+
+TEST_F(HttpDumpUrlAsyncWriterTest, TestCacheWithASyncFetcherFail) {
+ bool callback_called1, callback_called2, callback_called3;
+
+ EXPECT_EQ(1, CountFetchesAsync(kBadUrl, false, &callback_called1));
+ EXPECT_FALSE(callback_called1);
+
+ EXPECT_EQ(1, CountFetchesAsync(kBadUrl, false, &callback_called2));
+ EXPECT_FALSE(callback_called1);
+ EXPECT_FALSE(callback_called2);
+
+ mock_async_fetcher_.CallCallbacks();
+ EXPECT_TRUE(callback_called1);
+ EXPECT_TRUE(callback_called2);
+
+ EXPECT_EQ(1, CountFetchesAsync(kBadUrl, false, &callback_called3));
+ EXPECT_FALSE(callback_called3);
+ mock_async_fetcher_.CallCallbacks(); // Otherwise memory will be leaked
+ EXPECT_TRUE(callback_called3);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/http_dump_url_fetcher.cc b/trunk/src/net/instaweb/util/http_dump_url_fetcher.cc
new file mode 100644
index 0000000..6700954
--- /dev/null
+++ b/trunk/src/net/instaweb/util/http_dump_url_fetcher.cc
@@ -0,0 +1,300 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/public/http_dump_url_fetcher.h"
+
+#include <stdio.h>
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/filename_encoder.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/gzip_inflater.h"
+#include "net/instaweb/util/public/http_response_parser.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/meta_data.h"
+#include "net/instaweb/util/public/null_message_handler.h"
+#include <string>
+#include "net/instaweb/util/public/timer.h"
+#include "net/instaweb/util/public/writer.h"
+#include "net/instaweb/util/stack_buffer.h"
+
+namespace net_instaweb {
+
+namespace {
+
+static const char kErrorHtml[] =
+ "<html><head><title>HttpDumpUrlFetcher Error</title></head>"
+ "<body><h1>HttpDumpUrlFetcher Error</h1></body></html>";
+
+void ApplyTimeDelta(const char* attr, int64 delta_ms, MetaData* headers) {
+ int64 time_ms;
+ if (headers->ParseDateHeader(attr, &time_ms) && (time_ms > delta_ms)) {
+ headers->UpdateDateHeader(attr, time_ms + delta_ms);
+ }
+}
+
+// The slurped files we read off the disk will contain a Date header from the
+// time we did the slurp. They may have an Expires header shortly after that.
+// As part of the dump-fetching process, we will want to correct the Date
+// header based on the current time, and also update the Expires header by
+// the same delta.
+void CorrectDateHeaders(int64 now_ms, MetaData* headers) {
+ int64 date_ms;
+ if (headers->ParseDateHeader(HttpAttributes::kDate, &date_ms) &&
+ (date_ms < now_ms)) {
+ int64 delta_ms = now_ms - date_ms;
+ headers->UpdateDateHeader(HttpAttributes::kDate, now_ms);
+ ApplyTimeDelta(HttpAttributes::kExpires, delta_ms, headers);
+ ApplyTimeDelta(HttpAttributes::kLastModified, delta_ms, headers);
+ }
+}
+
+} // namespace
+
+const char HttpDumpUrlFetcher::kGzipContentLengthAttribute[] =
+ "X-Instaweb-Gzip-Content-Length";
+
+HttpDumpUrlFetcher::HttpDumpUrlFetcher(const StringPiece& root_dir,
+ FileSystem* file_system,
+ Timer* timer)
+ : root_dir_(root_dir.data(), root_dir.size()),
+ file_system_(file_system),
+ timer_(timer),
+ error_body_(kErrorHtml) {
+ EnsureEndsInSlash(&root_dir_);
+}
+
+HttpDumpUrlFetcher::~HttpDumpUrlFetcher() {
+}
+
+bool HttpDumpUrlFetcher::GetFilenameFromUrl(const StringPiece& root_dir,
+ const GURL& gurl,
+ std::string* filename,
+ MessageHandler* handler) {
+ bool ret = false;
+ if (!EndsInSlash(root_dir)) {
+ handler->Message(kError,
+ "GetFilenameFromUrl: root_dir must end in slash, was %s",
+ root_dir.as_string().c_str());
+ } else if (!gurl.is_valid()) {
+ handler->Message(kError, "GetFilenameFromUrl: gurl is invalid");
+ } else {
+ ret = true;
+
+ // Seperate the url into domain and path. Note: we ignore scheme, username,
+ // password, port and ref (stuff after '#').
+ // TODO(sligocki): Perhaps we should include these (except ref).
+ std::string domain = gurl.host();
+ std::string path = gurl.path();
+
+ // Add other bits of url used by latency lab.
+ if (!gurl.query().empty()) { // Part after '?' in url.
+ path.append(1, '?');
+ path.append(gurl.query());
+ }
+
+ FilenameEncoder encoder;
+ const std::string prefix = StrCat(root_dir, domain);
+ encoder.Encode(prefix, path, filename); // Writes encoded filename.
+ }
+ return ret;
+}
+
+bool HttpDumpUrlFetcher::GetFilenamePrefixFromUrl(const StringPiece& root_dir,
+ const GURL& url,
+ std::string* filename,
+ MessageHandler* handler) {
+ handler->Check(EndsInSlash(StringPiece(url.spec())),
+ "Prefix url must end in '/', was %s", url.spec().c_str());
+ bool ret = GetFilenameFromUrl(root_dir, url, filename, handler);
+ if (ret) {
+ size_t last_slash = filename->find_last_of('/');
+ CHECK(last_slash != std::string::npos);
+ filename->resize(last_slash + 1);
+ }
+ return ret;
+}
+
+void HttpDumpUrlFetcher::RespondError(MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* handler) {
+ response_headers->SetStatusAndReason(HttpStatus::kNotFound);
+ response_headers->Add(HttpAttributes::kContentType, "text/html");
+ response_headers->ComputeCaching();
+ response_headers->set_headers_complete(true);
+ response_writer->Write(error_body_, handler);
+}
+
+// Passes Http contents through to another writer, optionally
+// gunzipping if want_gzip is set (and content is gzipped).
+class HttpResponseWriter : public Writer {
+ public:
+ HttpResponseWriter(const StringPiece& url, bool want_gzip, Writer* writer,
+ MetaData* response)
+ : url_(url.data(), url.size()),
+ content_length_(0),
+ gzip_content_length_(0),
+ want_gzip_(want_gzip),
+ first_write_(true),
+ writer_(writer),
+ response_(response) {
+ }
+
+ virtual bool Write(const StringPiece& str, MessageHandler* handler) {
+ bool ret = true;
+
+ // We don't store the request headers with the slurped file. So if
+ // we slurp with a gzipped encoding, but the requester wants to see
+ // cleartext, then we will convert inline in the Writer. Determine
+ // that the first time Write() is called.
+ if (first_write_) {
+ first_write_ = false;
+ CHECK(response_->headers_complete());
+ CharStarVector v;
+ if (!want_gzip_ && response_->IsGzipped()) {
+ inflater_.reset(new GzipInflater(GzipInflater::kGzip));
+ CHECK(inflater_->Init());
+ response_->RemoveAll(HttpAttributes::kContentEncoding);
+ }
+ }
+ if (inflater_.get() != NULL) {
+ CHECK(!inflater_->HasUnconsumedInput());
+ CHECK(inflater_->SetInput(str.data(), str.size()));
+ gzip_content_length_ += str.size();
+ while (inflater_->HasUnconsumedInput() && ret) {
+ char buf[kStackBufferSize];
+ int bytes = inflater_->InflateBytes(buf, sizeof(buf));
+ if (bytes == 0) {
+ handler->Error(url_.c_str(), 0,
+ "zlib reported unconsumed data but yielded 0 bytes");
+ ret = false;
+ } else {
+ if (inflater_->error()) {
+ handler->Error(url_.c_str(), 0, "zlib inflate error");
+ ret = false;
+ } else {
+ ret = writer_->Write(StringPiece(buf, bytes), handler);
+ content_length_ += bytes;
+ }
+ }
+ }
+ } else {
+ ret = writer_->Write(str, handler);
+ content_length_ += str.size();
+ }
+ return ret;
+ }
+
+ bool Flush(MessageHandler* handler) {
+ return writer_->Flush(handler);
+ }
+
+ int content_length() const { return content_length_; }
+ int gzip_content_length() const { return gzip_content_length_; }
+
+ private:
+ std::string url_;
+ int content_length_;
+ int gzip_content_length_;
+ bool want_gzip_;
+ bool first_write_;
+ Writer* writer_;
+ MetaData* response_;
+ scoped_ptr<GzipInflater> inflater_;
+
+ DISALLOW_COPY_AND_ASSIGN(HttpResponseWriter);
+};
+
+bool HttpDumpUrlFetcher::StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* handler) {
+ bool ret = false;
+ std::string filename;
+ GURL gurl(url);
+ if (gurl.is_valid() && gurl.IsStandard() &&
+ GetFilenameFromUrl(root_dir_, gurl, &filename, handler)) {
+ NullMessageHandler null_handler;
+ // Pass in NullMessageHandler so that we don't get errors for file not found
+ FileSystem::InputFile* file =
+ file_system_->OpenInputFile(filename.c_str(), &null_handler);
+ if (file != NULL) {
+ CharStarVector v;
+ // TODO(jmarantz): handle 'deflate'.
+ bool want_gzip = request_headers.AcceptsGzip();
+ HttpResponseWriter writer(url, want_gzip, response_writer,
+ response_headers);
+ HttpResponseParser response(response_headers, &writer, handler);
+ if (response.ParseFile(file)) {
+ handler->Message(kInfo, "HttpDumpUrlFetcher: Fetched %s as %s",
+ url.c_str(), filename.c_str());
+ if (!response_headers->headers_complete()) {
+ // Fill in some default headers and body. Note that if we have
+ // a file, then we will return true, even if the file is corrupt.
+ RespondError(response_headers, response_writer, handler);
+ } else {
+ // Update 'date' and 'Expires' headers, if found.
+ //
+ // TODO(jmarantz): make this conditional based on a flag.
+ int64 now_ms = timer_->NowMs();
+ CorrectDateHeaders(now_ms, response_headers);
+ response_headers->RemoveAll(HttpAttributes::kContentLength);
+ response_headers->Add(HttpAttributes::kContentLength, IntegerToString(
+ writer.content_length()).c_str());
+ }
+ if (writer.gzip_content_length() != 0) {
+ response_headers->Add(kGzipContentLengthAttribute, IntegerToString(
+ writer.gzip_content_length()).c_str());
+ }
+ response_headers->ComputeCaching();
+ ret = true;
+ } else {
+ handler->Message(kWarning,
+ "HttpDumpUrlFetcher: Failed to parse %s for %s",
+ filename.c_str(), url.c_str());
+ }
+ file_system_->Close(file, handler);
+ } else {
+ handler->Message(kInfo,
+ "HttpDumpUrlFetcher: Failed to find file %s for %s",
+ filename.c_str(), url.c_str());
+ }
+ } else {
+ handler->Message(kError,
+ "HttpDumpUrlFetcher: Requested invalid URL %s",
+ url.c_str());
+ }
+
+ if ((urls_.get() != NULL) && urls_->insert(url).second) {
+ fprintf(stdout, "url: %s\n", url.c_str());
+ }
+
+ return ret;
+}
+
+void HttpDumpUrlFetcher::set_print_urls(bool on) {
+ if (on) {
+ urls_.reset(new StringSet);
+ } else {
+ urls_.reset(NULL);
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/http_dump_url_fetcher_test.cc b/trunk/src/net/instaweb/util/http_dump_url_fetcher_test.cc
new file mode 100644
index 0000000..1b44f02
--- /dev/null
+++ b/trunk/src/net/instaweb/util/http_dump_url_fetcher_test.cc
@@ -0,0 +1,98 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the http dump fetcher, using a mock fetcher. Note that
+// the HTTP Dump Fetcher is, in essence, a caching fetcher except that:
+// 1. It ignores caching headers completely
+// 2. It uses file-based storage with no expectation of ever evicting
+// anything.
+//
+// TODO(jmarantz): consider making this class a special case of the
+// combination of HTTPCache, FileCache, and HttpDumpUrlFetcher.
+
+#include "net/instaweb/util/public/http_dump_url_fetcher.h"
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/mock_timer.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/stdio_file_system.h"
+#include <string>
+#include "net/instaweb/util/public/string_writer.h"
+
+namespace net_instaweb {
+
+class HttpDumpUrlFetcherTest : public testing::Test {
+ public:
+ HttpDumpUrlFetcherTest()
+ : content_writer_(&content_),
+ mock_timer_(0),
+ http_dump_fetcher_(
+ GTestSrcDir() + "/net/instaweb/util/testdata",
+ &file_system_,
+ &mock_timer_) {
+ }
+
+ protected:
+ StdioFileSystem file_system_;
+ std::string content_;
+ StringWriter content_writer_;
+ MockTimer mock_timer_;
+ HttpDumpUrlFetcher http_dump_fetcher_;
+ GoogleMessageHandler message_handler_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(HttpDumpUrlFetcherTest);
+};
+
+TEST_F(HttpDumpUrlFetcherTest, TestReadWithGzip) {
+ SimpleMetaData request, response;
+ request.Add(HttpAttributes::kAcceptEncoding, HttpAttributes::kGzip);
+ ASSERT_TRUE(http_dump_fetcher_.StreamingFetchUrl(
+ "http://www.google.com", request, &response, &content_writer_,
+ &message_handler_));
+ CharStarVector v;
+ ASSERT_TRUE(response.Lookup(HttpAttributes::kContentEncoding, &v));
+ ASSERT_EQ(1, v.size());
+ CHECK_EQ(std::string(HttpAttributes::kGzip), v[0]);
+ CHECK_EQ(5513, content_.size());
+ v.clear();
+ ASSERT_TRUE(response.Lookup(HttpAttributes::kContentLength, &v));
+ ASSERT_EQ(1, v.size());
+ CHECK_EQ(std::string("5513"), v[0]);
+}
+
+TEST_F(HttpDumpUrlFetcherTest, TestReadUncompressedFromGzippedDump) {
+ SimpleMetaData request, response;
+ ASSERT_TRUE(http_dump_fetcher_.StreamingFetchUrl(
+ "http://www.google.com", request, &response, &content_writer_,
+ &message_handler_));
+ CharStarVector v;
+ if (response.Lookup(HttpAttributes::kContentEncoding, &v)) {
+ ASSERT_EQ(1, v.size());
+ CHECK_NE(std::string(HttpAttributes::kGzip), v[0]);
+ }
+ CHECK_EQ(14450, content_.size());
+ v.clear();
+ ASSERT_TRUE(response.Lookup(HttpAttributes::kContentLength, &v));
+ ASSERT_EQ(1, v.size());
+ CHECK_EQ(std::string("14450"), v[0]);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/http_dump_url_writer.cc b/trunk/src/net/instaweb/util/http_dump_url_writer.cc
new file mode 100644
index 0000000..5fffaee
--- /dev/null
+++ b/trunk/src/net/instaweb/util/http_dump_url_writer.cc
@@ -0,0 +1,134 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/http_dump_url_writer.h"
+#include "net/instaweb/util/public/file_writer.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/gzip_inflater.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/meta_data.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/stack_buffer.h"
+
+namespace net_instaweb {
+
+HttpDumpUrlWriter::~HttpDumpUrlWriter() {
+}
+
+bool HttpDumpUrlWriter::StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* handler) {
+ bool ret = true;
+ std::string filename;
+
+ if (!dump_fetcher_.GetFilename(GURL(url), &filename, handler)) {
+ handler->Message(kError, "Invalid url: %s", url.c_str());
+ ret = false;
+ } else if (!file_system_->Exists(filename.c_str(), handler).is_true()) {
+ // Do the Fetch first, before opening the output file, so that if the
+ // fetch fails, do not make an empty file.
+ //
+ // TODO(jmarantz): Re-integrate the use of SplitWriter. We'll have
+ // to do a lazy-open of the OutputFile* in a custom writer, though, to
+ // avoid opening up a zero-size file when the URL fetch fails.
+ std::string contents;
+ StringWriter string_writer(&contents);
+ // TODO(sligocki): Have this actually stream to response_writer.
+
+ // In general we will want to always ask the origin for gzipped output,
+ // but we are leaving in variable so this could be overridden by the
+ // instantiator of the DumpUrlWriter.
+ SimpleMetaData compress_headers, compressed_response;
+ compress_headers.CopyFrom(request_headers);
+ if (accept_gzip_) {
+ compress_headers.RemoveAll(HttpAttributes::kAcceptEncoding);
+ compress_headers.Add(HttpAttributes::kAcceptEncoding,
+ HttpAttributes::kGzip);
+ }
+
+ ret = base_fetcher_->StreamingFetchUrl(url, compress_headers,
+ &compressed_response, &string_writer,
+ handler);
+ compressed_response.RemoveAll(HttpAttributes::kContentLength);
+ compressed_response.Add(HttpAttributes::kContentLength,
+ IntegerToString(contents.size()).c_str());
+ compressed_response.ComputeCaching();
+
+ // Do not write an empty file if the fetch failed.
+ if (ret) {
+ // Check to see if a response marked as gzipped are really unzippable.
+ if (compressed_response.IsGzipped()) {
+ GzipInflater inflater(GzipInflater::kGzip);
+ inflater.Init();
+ if (contents.data() == NULL || contents.size() == 0) {
+ // CHECK below would fail on these.
+ compressed_response.RemoveAll(HttpAttributes::kContentEncoding);
+ } else {
+ CHECK(inflater.SetInput(contents.data(), contents.size()));
+ while (inflater.HasUnconsumedInput()) {
+ char buf[kStackBufferSize];
+ if ((inflater.InflateBytes(buf, sizeof(buf)) == 0) ||
+ inflater.error()) {
+ compressed_response.RemoveAll(HttpAttributes::kContentEncoding);
+ break;
+ }
+ }
+ }
+ }
+
+ FileSystem::OutputFile* file = file_system_->OpenTempFile(
+ filename + ".temp", handler);
+ if (file != NULL) {
+ handler->Message(kInfo, "Storing %s as %s", url.c_str(),
+ filename.c_str());
+ std::string temp_filename = file->filename();
+ FileWriter file_writer(file);
+ ret = compressed_response.Write(&file_writer, handler) &&
+ file->Write(contents, handler);
+ ret &= file_system_->Close(file, handler);
+ ret &= file_system_->RenameFile(temp_filename.c_str(), filename.c_str(),
+ handler);
+ } else {
+ ret = false;
+ }
+ }
+
+ // We are not going to be able to read the response from the file
+ // system so we better pass the error message through.
+ if (!ret) {
+ response_headers->CopyFrom(compressed_response);
+ if (!response_headers->headers_complete()) {
+ response_headers->SetStatusAndReason(HttpStatus::kNotFound);
+ response_headers->ComputeCaching();
+ response_headers->set_headers_complete(true);
+ }
+ response_writer->Write(contents, handler);
+ }
+ }
+
+ // Always use the HttpDumpUrlFetcher, even if we are reading the file
+ // ourselves. Thus the problem of inflating gzipped requests for requesters
+ // that want cleartext only is solved only in that file.
+ return ret && dump_fetcher_.StreamingFetchUrl(
+ url, request_headers, response_headers, response_writer, handler);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/http_dump_url_writer_test.cc b/trunk/src/net/instaweb/util/http_dump_url_writer_test.cc
new file mode 100644
index 0000000..954aaf3
--- /dev/null
+++ b/trunk/src/net/instaweb/util/http_dump_url_writer_test.cc
@@ -0,0 +1,88 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the http dump fetcher, using a mock fetcher. Note that
+// the HTTP Dump Fetcher is, in essence, a caching fetcher except that:
+// 1. It ignores caching headers completely
+// 2. It uses file-based storage with no expectation of ever evicting
+// anything.
+
+#include "net/instaweb/util/public/http_dump_url_writer.h"
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/fetcher_test.h"
+#include "net/instaweb/util/public/google_url.h"
+#include "net/instaweb/util/public/mock_timer.h"
+#include "net/instaweb/util/public/stdio_file_system.h"
+
+namespace net_instaweb {
+
+class HttpDumpUrlWriterTest : public FetcherTest {
+ protected:
+ HttpDumpUrlWriterTest()
+ : mock_timer_(0),
+ http_dump_writer_(GTestTempDir() + "/http_dump/", &mock_fetcher_,
+ &file_system_, &mock_timer_) {
+ }
+
+ virtual UrlFetcher* sync_fetcher() { return &http_dump_writer_; }
+
+ virtual void SetUp() {
+ RemoveFileIfPresent(kGoodUrl);
+ RemoveFileIfPresent(kNotCachedUrl);
+ RemoveFileIfPresent(kBadUrl);
+ }
+
+ void RemoveFileIfPresent(const char* url) {
+ GURL gurl(url);
+ std::string path;
+ HttpDumpUrlFetcher::GetFilenameFromUrl(GTestTempDir() + "/http_dump/",
+ gurl, &path, &message_handler_);
+ file_system_.RemoveFile(path.c_str(), &message_handler_);
+ }
+
+ StdioFileSystem file_system_;
+ MockTimer mock_timer_;
+ HttpDumpUrlWriter http_dump_writer_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(HttpDumpUrlWriterTest);
+};
+
+TEST_F(HttpDumpUrlWriterTest, TestCachableWithSyncFetcher) {
+ EXPECT_EQ(1, CountFetchesSync(kGoodUrl, true, true));
+ EXPECT_EQ(0, CountFetchesSync(kGoodUrl, true, true));
+}
+
+TEST_F(HttpDumpUrlWriterTest, TestNonCachableWithSyncFetcher) {
+ // When a HttpDumpUrlFetcher is implemented using a sync fetcher,
+ // then non-cacheable URLs will result in sync-fetch successes.
+ // As we are ignoring caching headers, we don't expect the
+ // underlying fetcher to be called here either.
+ EXPECT_EQ(1, CountFetchesSync(kNotCachedUrl, true, true));
+ EXPECT_EQ(0, CountFetchesSync(kNotCachedUrl, true, true));
+}
+
+TEST_F(HttpDumpUrlWriterTest, TestCacheWithSyncFetcherFail) {
+ EXPECT_EQ(1, CountFetchesSync(kBadUrl, false, true));
+ // For now, we don't cache failure, so we expect a new fetch
+ // on each request
+ EXPECT_EQ(1, CountFetchesSync(kBadUrl, false, true));
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/http_response_parser.cc b/trunk/src/net/instaweb/util/http_response_parser.cc
new file mode 100644
index 0000000..9c61230
--- /dev/null
+++ b/trunk/src/net/instaweb/util/http_response_parser.cc
@@ -0,0 +1,63 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/public/http_response_parser.h"
+
+#include <stdio.h>
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/meta_data.h"
+#include "net/instaweb/util/public/writer.h"
+#include "net/instaweb/util/stack_buffer.h"
+
+namespace net_instaweb {
+
+bool HttpResponseParser::ParseFile(FileSystem::InputFile* file) {
+ char buf[kStackBufferSize];
+ int nread;
+ while (ok_ && ((nread = file->Read(buf, sizeof(buf), handler_)) > 0)) {
+ ParseChunk(StringPiece(buf, nread));
+ }
+ return ok_;
+}
+
+bool HttpResponseParser::Parse(FILE* stream) {
+ char buf[kStackBufferSize];
+ int nread;
+ while (ok_ && ((nread = fread(buf, 1, sizeof(buf), stream)) > 0)) {
+ ParseChunk(StringPiece(buf, nread));
+ }
+ return ok_;
+}
+
+bool HttpResponseParser::ParseChunk(const StringPiece& data) {
+ if (reading_headers_) {
+ int consumed = response_headers_->ParseChunk(data, handler_);
+ if (response_headers_->headers_complete()) {
+ // In this chunk we may have picked up some of the body.
+ // Before we move to the next buffer, send it to the output
+ // stream.
+ ok_ = writer_->Write(data.substr(consumed), handler_);
+ reading_headers_ = false;
+ }
+ } else {
+ ok_ = writer_->Write(data, handler_);
+ }
+ return ok_;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/http_value.cc b/trunk/src/net/instaweb/util/http_value.cc
new file mode 100644
index 0000000..99bcb4b
--- /dev/null
+++ b/trunk/src/net/instaweb/util/http_value.cc
@@ -0,0 +1,202 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/http_value.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+
+namespace {
+
+// The headers and body are both encoded into one Shared String, which can then
+// be efficiently held in an in-memory cache, or passed around as an HTTPValue
+// object. The class supports both setting the headers first and then the body,
+// and vice versa. Both the headers and body are variable length, and to avoid
+// having to re-shuffle memory, we encode which is first in the buffer as the
+// first byte. The next four bytes encode the size.
+const char kHeadersFirst = 'h';
+const char kBodyFirst = 'b';
+
+const int kStorageTypeOverhead = 1;
+const int kStorageSizeOverhead = 4;
+const unsigned int kStorageOverhead =
+ kStorageTypeOverhead + kStorageSizeOverhead;
+
+} // namespace
+
+namespace net_instaweb {
+
+void HTTPValue::CopyOnWrite() {
+ if (!storage_.unique()) {
+ SharedString new_storage(*storage_);
+ storage_ = new_storage;
+ }
+}
+
+void HTTPValue::Clear() {
+ CopyOnWrite();
+ storage_->clear();
+}
+
+void HTTPValue::SetHeaders(const MetaData& headers) {
+ CopyOnWrite();
+ std::string headers_string = headers.ToString();
+ if (storage_->empty()) {
+ storage_->append(&kHeadersFirst, 1);
+ SetSizeOfFirstChunk(headers_string.size());
+ } else {
+ CHECK(type_identifier() == kBodyFirst);
+ // Using 'unsigned int' to facilitate bit-shifting in
+ // SizeOfFirstChunk and SetSizeOfFirstChunk, and I don't
+ // want to worry about sign extension.
+ unsigned int size = SizeOfFirstChunk();
+ CHECK(storage_->size() == (kStorageOverhead + size));
+ }
+ storage_->append(headers_string);
+}
+
+bool HTTPValue::Write(const StringPiece& str, MessageHandler* handler) {
+ CopyOnWrite();
+ if (storage_->empty()) {
+ storage_->append(&kBodyFirst, 1);
+ storage_->append(" ", 4);
+ SetSizeOfFirstChunk(str.size());
+ } else if (type_identifier() == kBodyFirst) {
+ CHECK(storage_->size() >= kStorageOverhead);
+ unsigned int string_size = SizeOfFirstChunk();
+ CHECK(string_size == storage_->size() - kStorageOverhead);
+ SetSizeOfFirstChunk(str.size() + string_size);
+ } else {
+ CHECK(type_identifier() == kHeadersFirst);
+ }
+ storage_->append(str.data(), str.size());
+ return true;
+}
+
+bool HTTPValue::Flush(MessageHandler* handler) {
+ return true;
+}
+
+// Encode the size of the first chunk, which is either the headers or body,
+// depending on the order they are called. Rather than trying to assume any
+// particular alignment for casting between char* and int*, we just manually
+// encode one byte at a time.
+void HTTPValue::SetSizeOfFirstChunk(unsigned int size) {
+ CHECK(!storage_->empty()) << "type encoding should already be in first byte";
+ unsigned char size_buffer[4];
+ size_buffer[0] = size & 0xff;
+ size_buffer[1] = (size >> 8) & 0xff;
+ size_buffer[2] = (size >> 16) & 0xff;
+ size_buffer[3] = (size >> 24) & 0xff;
+ if (storage_->size() < kStorageOverhead) {
+ // Ensure the buffer is exactly 5 bytes so we can overwrite
+ // bytes 1-4 (the type code in byte 0).
+ storage_->append(" ", kStorageOverhead - storage_->size());
+ }
+ memcpy(&((*storage_)[1]), &size_buffer, sizeof(size_buffer));
+}
+
+// Decodes the size of the first chunk, which is either the headers or body,
+// depending on the order they are called. Rather than trying to assume any
+// particular alignment for casting between char* and int*, we just manually
+// decode one byte at a time.
+unsigned int HTTPValue::SizeOfFirstChunk() const {
+ CHECK(storage_->size() >= kStorageOverhead);
+ const unsigned char *size_buffer =
+ reinterpret_cast<const unsigned char*>(storage_->data() + 1);
+ unsigned int size = size_buffer[0];
+ size |= size_buffer[1] << 8;
+ size |= size_buffer[2] << 16;
+ size |= size_buffer[3] << 24;
+ return size;
+}
+
+// Note that we avoid CHECK, and instead return false on error. So if
+// our cache gets corrupted (say) on disk, we just consider it an
+// invalid entry rather than aborting the server.
+bool HTTPValue::ExtractHeaders(MetaData* headers, MessageHandler* handler)
+ const {
+ bool ret = false;
+ headers->Clear();
+ if (storage_->size() >= kStorageOverhead) {
+ char type_id = type_identifier();
+ const char* start = storage_->data() + kStorageOverhead;
+ unsigned int size = SizeOfFirstChunk();
+ if (size <= storage_->size() - kStorageOverhead) {
+ if (type_id == kBodyFirst) {
+ start += size;
+ size = storage_->size() - size - kStorageOverhead;
+ } else {
+ ret = (type_id == kHeadersFirst);
+ }
+ unsigned int num_consumed =
+ headers->ParseChunk(StringPiece(start, size), handler);
+ ret = (num_consumed == size);
+ ret &= (headers->headers_complete());
+ }
+ }
+ return ret;
+}
+
+// Note that we avoid CHECK, and instead return false on error. So if
+// our cache gets corrupted (say) on disk, we just consider it an
+// invalid entry rather than aborting the server.
+bool HTTPValue::ExtractContents(StringPiece* val) const {
+ bool ret = false;
+ if (storage_->size() >= kStorageOverhead) {
+ char type_id = type_identifier();
+ const char* start = storage_->data() + kStorageOverhead;
+ unsigned int size = SizeOfFirstChunk();
+ if (size <= storage_->size() - kStorageOverhead) {
+ if (type_id == kHeadersFirst) {
+ start += size;
+ size = storage_->size() - size - kStorageOverhead;
+ ret = true;
+ } else {
+ ret = (type_id == kBodyFirst);
+ }
+ *val = StringPiece(start, size);
+ }
+ }
+ return ret;
+}
+
+bool HTTPValue::Link(SharedString* src, MetaData* headers,
+ MessageHandler* handler) {
+ bool ok = false;
+ if ((*src)->size() >= kStorageOverhead) {
+ // The simplest way to ensure that src is well formed is to save the
+ // existing storage_ in a temp, assign the storage, and make sure
+ // Headers and Contents return true. The drawback is that the headers
+ // parsing is arguably a little heavyweight. We could consider encoding
+ // the headers in an easier-to-extract form, so we don't have to give up
+ // the integrity checks.
+ SharedString temp(storage_);
+ storage_ = *src;
+
+ // TODO(jmarantz): this could be a lot lighter weight, but we are going
+ // to be sure at this point that both the headers and the contents are
+ // valid. It would be nice to have an HTML headers parser that didn't
+ // actually create new temp copies of all the names/values.
+ ok = ExtractHeaders(headers, handler);
+ if (!ok) {
+ storage_ = temp;
+ }
+ }
+ return ok;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/http_value_test.cc b/trunk/src/net/instaweb/util/http_value_test.cc
new file mode 100644
index 0000000..e7620f5
--- /dev/null
+++ b/trunk/src/net/instaweb/util/http_value_test.cc
@@ -0,0 +1,191 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the lru cache
+
+#include "net/instaweb/util/public/http_value.h"
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/shared_string.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+
+namespace {
+const int kMaxSize = 100;
+}
+
+namespace net_instaweb {
+
+class HTTPValueTest : public testing::Test {
+ protected:
+ HTTPValueTest() { }
+
+ void FillMetaData(MetaData* meta_data) {
+ meta_data->SetStatusAndReason(HttpStatus::kOK);
+ meta_data->set_major_version(1);
+ meta_data->set_minor_version(0);
+ meta_data->set_reason_phrase("OK");
+ meta_data->Add("Cache-control", "max-age=300");
+ }
+
+ void CheckMetaData(const MetaData& meta_data) {
+ SimpleMetaData expected;
+ FillMetaData(&expected);
+ EXPECT_EQ(expected.ToString(), meta_data.ToString());
+ }
+
+ GoogleMessageHandler message_handler_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(HTTPValueTest);
+};
+
+TEST_F(HTTPValueTest, Empty) {
+ HTTPValue value;
+ EXPECT_TRUE(value.Empty());
+}
+
+TEST_F(HTTPValueTest, HeadersFirst) {
+ HTTPValue value;
+ SimpleMetaData headers, check_headers;
+ FillMetaData(&headers);
+ value.SetHeaders(headers);
+ value.Write("body", &message_handler_);
+ StringPiece body;
+ ASSERT_TRUE(value.ExtractContents(&body));
+ EXPECT_EQ("body", body.as_string());
+ ASSERT_TRUE(value.ExtractHeaders(&check_headers, &message_handler_));
+ CheckMetaData(check_headers);
+}
+
+TEST_F(HTTPValueTest, ContentsFirst) {
+ HTTPValue value;
+ SimpleMetaData headers, check_headers;
+ FillMetaData(&headers);
+ value.Write("body", &message_handler_);
+ value.SetHeaders(headers);
+ StringPiece body;
+ ASSERT_TRUE(value.ExtractContents(&body));
+ EXPECT_EQ("body", body.as_string());
+ ASSERT_TRUE(value.ExtractHeaders(&check_headers, &message_handler_));
+ CheckMetaData(check_headers);
+}
+
+TEST_F(HTTPValueTest, EmptyContentsFirst) {
+ HTTPValue value;
+ SimpleMetaData headers, check_headers;
+ FillMetaData(&headers);
+ value.Write("", &message_handler_);
+ value.SetHeaders(headers);
+ StringPiece body;
+ ASSERT_TRUE(value.ExtractContents(&body));
+ EXPECT_EQ("", body.as_string());
+ ASSERT_TRUE(value.ExtractHeaders(&check_headers, &message_handler_));
+ CheckMetaData(check_headers);
+}
+
+TEST_F(HTTPValueTest, TestCopyOnWrite) {
+ HTTPValue v1;
+ v1.Write("Hello", &message_handler_);
+ StringPiece v1_contents, v2_contents, v3_contents;
+ ASSERT_TRUE(v1.ExtractContents(&v1_contents));
+ EXPECT_TRUE(v1.unique());
+
+ // Now check that copy-construction shares the buffer.
+ HTTPValue v2(v1);
+ EXPECT_FALSE(v1.unique());
+ EXPECT_FALSE(v2.unique());
+ ASSERT_TRUE(v2.ExtractContents(&v2_contents));
+ EXPECT_EQ(v1_contents, v2_contents);
+ EXPECT_EQ(v1_contents.data(), v2_contents.data()); // buffer sharing
+
+ // Also the assignment operator should induce sharing.
+ HTTPValue v3 = v1;
+ EXPECT_FALSE(v3.unique());
+ ASSERT_TRUE(v3.ExtractContents(&v3_contents));
+ EXPECT_EQ(v1_contents, v3_contents);
+ EXPECT_EQ(v1_contents.data(), v3_contents.data()); // buffer sharing
+
+ // Now write something into v1. Due to copy-on-write semantics, v2 and
+ // will v3 not see it.
+ v1.Write(", World!", &message_handler_);
+ ASSERT_TRUE(v1.ExtractContents(&v1_contents));
+ ASSERT_TRUE(v2.ExtractContents(&v2_contents));
+ ASSERT_TRUE(v3.ExtractContents(&v3_contents));
+ EXPECT_EQ("Hello, World!", v1_contents);
+ EXPECT_NE(v1_contents, v2_contents);
+ EXPECT_NE(v1_contents.data(), v2_contents.data()); // no buffer sharing
+ EXPECT_NE(v1_contents, v3_contents);
+ EXPECT_NE(v1_contents.data(), v3_contents.data()); // no buffer sharing
+
+ // But v2 and v3 will remain connected to one another
+ EXPECT_EQ(v2_contents, v3_contents);
+ EXPECT_EQ(v2_contents.data(), v3_contents.data()); // buffer sharing
+}
+
+TEST_F(HTTPValueTest, TestShare) {
+ SharedString storage;
+
+ {
+ HTTPValue value;
+ SimpleMetaData headers, check_headers;
+ FillMetaData(&headers);
+ value.SetHeaders(headers);
+ value.Write("body", &message_handler_);
+ storage = *value.share();
+ }
+
+ {
+ HTTPValue value;
+ SimpleMetaData check_headers;
+ ASSERT_TRUE(value.Link(&storage, &check_headers, &message_handler_));
+ StringPiece body;
+ ASSERT_TRUE(value.ExtractContents(&body));
+ EXPECT_EQ("body", body.as_string());
+ CheckMetaData(check_headers);
+ }
+}
+
+TEST_F(HTTPValueTest, LinkEmpty) {
+ SharedString storage;
+ HTTPValue value;
+ SimpleMetaData headers;
+ ASSERT_FALSE(value.Link(&storage, &headers, &message_handler_));
+}
+
+TEST_F(HTTPValueTest, LinkCorrupt) {
+ SharedString storage("h");
+ HTTPValue value;
+ SimpleMetaData headers;
+ ASSERT_FALSE(value.Link(&storage, &headers, &message_handler_));
+ storage->append("9999");
+ ASSERT_FALSE(value.Link(&storage, &headers, &message_handler_));
+ storage->append("xyz");
+ ASSERT_FALSE(value.Link(&storage, &headers, &message_handler_));
+ *storage = "b";
+ ASSERT_FALSE(value.Link(&storage, &headers, &message_handler_));
+ storage->append("9999");
+ ASSERT_FALSE(value.Link(&storage, &headers, &message_handler_));
+ storage->append("xyz");
+ ASSERT_FALSE(value.Link(&storage, &headers, &message_handler_));
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/lru_cache.cc b/trunk/src/net/instaweb/util/lru_cache.cc
new file mode 100644
index 0000000..5b6d156
--- /dev/null
+++ b/trunk/src/net/instaweb/util/lru_cache.cc
@@ -0,0 +1,208 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/lru_cache.h"
+
+#include "base/logging.h"
+#include "net/instaweb/util/public/shared_string.h"
+
+namespace net_instaweb {
+
+LRUCache::~LRUCache() {
+ Clear();
+}
+
+// Freshen a key-value pair by putting it in the front of the
+// LRU list. Returns a ListNode to insert into the map so we
+// can do fast lookup.
+LRUCache::ListNode LRUCache::Freshen(KeyValuePair* key_value) {
+ lru_ordered_list_.push_front(key_value);
+ return lru_ordered_list_.begin();
+}
+
+bool LRUCache::Get(const std::string& key, SharedString* value) {
+ Map::iterator p = map_.find(key);
+ bool ret = false;
+ if (p != map_.end()) {
+ ret = true;
+ ListNode cell = p->second;
+ KeyValuePair* key_value = *cell;
+ lru_ordered_list_.erase(cell);
+ p->second = Freshen(key_value);
+ *value = key_value->second;
+ ++num_hits_;
+ } else {
+ ++num_misses_;
+ }
+ return ret;
+}
+
+void LRUCache::Put(const std::string& key, SharedString* new_value) {
+ // Just do one map operation, calling the awkward 'insert' which returns
+ // a pair. The bool indicates whether a new value was inserted, and the
+ // iterator provides access to the element, whether it's new or old.
+ //
+ // If the key is already in the map, this will give us access to the value
+ // cell, and the uninitialized cell will not be used.
+ ListNode cell;
+ std::pair<Map::iterator, bool> iter_found =
+ map_.insert(Map::value_type(key, cell));
+ bool found = !iter_found.second;
+ Map::iterator map_iter = iter_found.first;
+ bool need_to_insert = true;
+ if (found) {
+ cell = map_iter->second;
+ KeyValuePair* key_value = *cell;
+
+ // Protect the element that we are rewriting by erasing
+ // it from the entry_list prior to calling EvictIfNecessary,
+ // which can't find it if it isn't in the list.
+ lru_ordered_list_.erase(cell);
+ if (**new_value == *(key_value->second)) {
+ map_iter->second = Freshen(key_value);
+ need_to_insert = false;
+ // TODO(jmarantz): count number of re-inserts of existing value?
+ } else {
+ ++num_deletes_;
+ current_bytes_in_cache_ -= entry_size(key_value);
+ delete key_value;
+ }
+ }
+
+ if (need_to_insert) {
+ // At this point, if we were doing a replacement, then the value
+ // is removed from the list, so we can treat replacements and new
+ // insertions the same way. In both cases, the new key is in the map
+ // as a result of the call to map_.insert above.
+
+ if (EvictIfNecessary(key.size() + (*new_value)->size())) {
+ // The new value fits. Put it in the LRU-list.
+ KeyValuePair* kvp = new KeyValuePair(&map_iter->first, *new_value);
+ map_iter->second = Freshen(kvp);
+ ++num_inserts_;
+ } else {
+ // The new value was too big to fit. Remove it from the map.
+ // it's already removed from the list. We have failed. We
+ // could potentially log this somewhere or keep a stat.
+ map_.erase(map_iter);
+ }
+ }
+}
+
+// Evicts enough items from the cache to allow an object of the
+// specified bytes-size to be inserted. If successful, we assumes that
+// the item will be inserted and current_bytes_in_cache_ is adjusted
+// accordingly.
+bool LRUCache::EvictIfNecessary(size_t bytes_needed) {
+ bool ret = false;
+ if (bytes_needed < max_bytes_in_cache_) {
+ while (bytes_needed + current_bytes_in_cache_ > max_bytes_in_cache_) {
+ KeyValuePair* key_value = lru_ordered_list_.back();
+ lru_ordered_list_.pop_back();
+ current_bytes_in_cache_ -= entry_size(key_value);
+ map_.erase(*key_value->first);
+ CHECK(current_bytes_in_cache_ >= 0);
+ delete key_value;
+ ++num_evictions_;
+ }
+ current_bytes_in_cache_ += bytes_needed;
+ ret = true;
+ }
+ return ret;
+}
+
+void LRUCache::Delete(const std::string& key) {
+ Map::iterator p = map_.find(key);
+ if (p != map_.end()) {
+ ListNode cell = p->second;
+ KeyValuePair* key_value = *cell;
+ lru_ordered_list_.erase(cell);
+ current_bytes_in_cache_ -= entry_size(key_value);
+ map_.erase(p);
+ delete key_value;
+ ++num_deletes_;
+ } else {
+ // TODO(jmarantz): count number of misses on a 'delete' request?
+ }
+}
+
+void LRUCache::SanityCheck() {
+ CHECK(map_.size() == lru_ordered_list_.size());
+ size_t count = 0;
+ size_t bytes_used = 0;
+
+ // Walk forward through the list, making sure the map and list elements
+ // point to each other correctly.
+ for (ListNode cell = lru_ordered_list_.begin(), e = lru_ordered_list_.end();
+ cell != e; ++cell, ++count) {
+ KeyValuePair* key_value = *cell;
+ Map::iterator map_iter = map_.find(*key_value->first);
+ CHECK(map_iter != map_.end());
+ CHECK(&map_iter->first == key_value->first);
+ CHECK(map_iter->second == cell);
+ bytes_used += entry_size(key_value);
+ }
+ CHECK(count == map_.size());
+ CHECK(current_bytes_in_cache_ == bytes_used);
+ CHECK(current_bytes_in_cache_ <= max_bytes_in_cache_);
+
+ // Walk backward through the list, making sure it's coherent as well.
+ count = 0;
+ for (EntryList::reverse_iterator cell = lru_ordered_list_.rbegin(),
+ e = lru_ordered_list_.rend(); cell != e; ++cell, ++count) {
+ }
+ CHECK(count == map_.size());
+}
+
+CacheInterface::KeyState LRUCache::Query(const std::string& key) {
+ Map::iterator p = map_.find(key);
+ KeyState state = kNotFound;
+ if (p != map_.end()) {
+ state = kAvailable;
+ }
+ return state;
+}
+
+// TODO(jmarantz): consider accounting for overhead for list cells, map
+// cells, string objects, etc. Currently we are only accounting for the
+// actual characters in the key and value.
+int LRUCache::entry_size(KeyValuePair* kvp) const {
+ return kvp->first->size() + kvp->second->size();
+}
+
+void LRUCache::Clear() {
+ current_bytes_in_cache_ = 0;
+
+ for (ListNode p = lru_ordered_list_.begin(), e = lru_ordered_list_.end();
+ p != e; ++p) {
+ KeyValuePair* key_value = *p;
+ delete key_value;
+ }
+ lru_ordered_list_.clear();
+ map_.clear();
+}
+
+void LRUCache::ClearStats() {
+ num_evictions_ = 0;
+ num_hits_ = 0;
+ num_misses_ = 0;
+ num_inserts_ = 0;
+ num_deletes_ = 0;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/lru_cache_test.cc b/trunk/src/net/instaweb/util/lru_cache_test.cc
new file mode 100644
index 0000000..2076bbf
--- /dev/null
+++ b/trunk/src/net/instaweb/util/lru_cache_test.cc
@@ -0,0 +1,159 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the lru cache
+
+#include "net/instaweb/util/public/lru_cache.h"
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/shared_string.h"
+
+namespace {
+const size_t kMaxSize = 100;
+}
+
+namespace net_instaweb {
+
+class LRUCacheTest : public testing::Test {
+ protected:
+ LRUCacheTest()
+ : cache_(kMaxSize) {
+ }
+
+ void CheckGet(const char* key, const std::string& expected_value) {
+ SharedString value_buffer;
+ ASSERT_TRUE(cache_.Get(key, &value_buffer));
+ EXPECT_EQ(expected_value, *value_buffer);
+ EXPECT_EQ(CacheInterface::kAvailable, cache_.Query(key));
+ cache_.SanityCheck();
+ }
+
+ void CheckPut(const char* key, const char* value) {
+ SharedString put_buffer(value);
+ cache_.Put(key, &put_buffer);
+ cache_.SanityCheck();
+ }
+
+ void CheckNotFound(const char* key) {
+ SharedString value_buffer;
+ ASSERT_FALSE(cache_.Get(key, &value_buffer));
+ EXPECT_EQ(CacheInterface::kNotFound, cache_.Query(key));
+ cache_.SanityCheck();
+ }
+
+ LRUCache cache_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(LRUCacheTest);
+};
+
+// Simple flow of putting in an item, getting it, deleting it.
+TEST_F(LRUCacheTest, PutGetDelete) {
+ EXPECT_EQ(static_cast<size_t>(0), cache_.size_bytes());
+ EXPECT_EQ(static_cast<size_t>(0), cache_.num_elements());
+ CheckPut("Name", "Value");
+ CheckGet("Name", "Value");
+ EXPECT_EQ(static_cast<size_t>(9), cache_.size_bytes()); // "Name" + "Value"
+ EXPECT_EQ(static_cast<size_t>(1), cache_.num_elements());
+ CheckNotFound("Another Name");
+
+ CheckPut("Name", "NewValue");
+ CheckGet("Name", "NewValue");
+ EXPECT_EQ(static_cast<size_t>(12),
+ cache_.size_bytes()); // "Name" + "NewValue"
+ EXPECT_EQ(static_cast<size_t>(1), cache_.num_elements());
+
+ cache_.Delete("Name");
+ cache_.SanityCheck();
+ SharedString value_buffer;
+ EXPECT_FALSE(cache_.Get("Name", &value_buffer));
+ EXPECT_EQ(static_cast<size_t>(0), cache_.size_bytes());
+ EXPECT_EQ(static_cast<size_t>(0), cache_.num_elements());
+}
+
+// Test eviction. We happen to know that the cache does not account for
+// STL overhead -- it's just counting key/value size. Exploit that to
+// understand when objects fall off the end.
+TEST_F(LRUCacheTest, LeastRecentlyUsed) {
+ // Fill the cache.
+ std::string keys[10], values[10];
+ const char key_pattern[] = "name%d";
+ const char value_pattern[] = "valu%d";
+ const int key_plus_value_size = 10; // strlen("name7") + strlen("valu7")
+ const size_t num_elements = kMaxSize / key_plus_value_size;
+ for (int i = 0; i < 10; ++i) {
+ SStringPrintf(&keys[i], key_pattern, i);
+ SStringPrintf(&values[i], value_pattern, i);
+ CheckPut(keys[i].c_str(), values[i].c_str());
+ }
+ EXPECT_EQ(kMaxSize, cache_.size_bytes());
+ EXPECT_EQ(num_elements, cache_.num_elements());
+
+ // Ensure we can see those.
+ for (int i = 0; i < 10; ++i) {
+ CheckGet(keys[i].c_str(), values[i].c_str());
+ }
+
+ // Now if we insert a new entry totaling 10 bytes, that should work,
+ // but we will lose name0 due to LRU semantics. We should still have name1,
+ // and by Get-ing name1 it we will make it the MRU.
+ CheckPut("nameA", "valuA");
+ CheckGet("nameA", "valuA");
+ CheckNotFound("name0");
+ CheckGet("name1", "valu1");
+
+ // So now when we put in nameB,valuB we will lose name2 but keep name1,
+ // which got bumped up to the MRU when we checked it above.
+ CheckPut("nameB", "valuB");
+ CheckGet("nameB", "valuB");
+ CheckGet("name1", "valu1");
+ CheckNotFound("name2");
+
+ // Now insert something 1 byte too big, spelling out "value" this time.
+ // We will now lose name3 and name4. We should still have name5-name9,
+ // plus name1, nameA, and nameB.
+ CheckPut("nameC", "valueC");
+ CheckNotFound("name3");
+ CheckNotFound("name4");
+ CheckGet("nameA", "valuA");
+ CheckGet("nameB", "valuB");
+ CheckGet("nameC", "valueC");
+ CheckGet("name1", "valu1");
+ for (int i = 5; i < 10; ++i) {
+ CheckGet(keys[i].c_str(), values[i].c_str());
+ }
+
+ // Now the oldest item is "nameA". Freshen it by re-inserting it, tickling
+ // the code-path in lru_cache.cc that special-cases handling of re-inserting
+ // the same value.
+ CheckPut("nameA", "valuA");
+ CheckPut("nameD", "valuD");
+ // nameB should be evicted, the others should be retained.
+ CheckNotFound("nameB");
+ CheckGet("nameA", "valuA");
+ CheckGet("nameC", "valueC");
+ CheckGet("name1", "valu1");
+ for (int i = 5; i < 10; ++i) {
+ CheckGet(keys[i].c_str(), values[i].c_str());
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/md5_hasher.cc b/trunk/src/net/instaweb/util/md5_hasher.cc
new file mode 100644
index 0000000..1309cf7
--- /dev/null
+++ b/trunk/src/net/instaweb/util/md5_hasher.cc
@@ -0,0 +1,55 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Authors: sligocki@google.com (Shawn Ligocki),
+// lsong@google.com (Libo Song)
+
+#include "net/instaweb/util/public/md5_hasher.h"
+
+#include "net/instaweb/util/public/base64_util.h"
+
+#include "base/md5.h"
+
+namespace net_instaweb {
+
+namespace {
+
+const int kMD5NumBytes = sizeof(MD5Digest);
+
+} // namespace
+
+// Hash size is size after Base64 encoding, which expands by 4/3. We round down,
+// this should not matter unless someone really wants that extra few bits.
+const int MD5Hasher::kMaxHashSize = kMD5NumBytes * 4 / 3;
+
+MD5Hasher::~MD5Hasher() {
+}
+
+std::string MD5Hasher::Hash(const StringPiece& content) const {
+ // Note: It may seem more efficient to initialize the MD5Context
+ // in a constructor so it can be re-used. But a quick inspection
+ // of src/third_party/chromium/src/base/md5.cc indicates that
+ // the cost of MD5Init is very tiny compared to the cost of
+ // MD5Update, so it's better to stay thread-safe.
+ MD5Digest digest;
+ MD5Sum(content.data(), content.size(), &digest);
+ // Note: digest.a is an unsigned char[16] so it's not null-terminated.
+ StringPiece raw_hash(reinterpret_cast<char*>(digest.a), sizeof(digest.a));
+ std::string out;
+ Web64Encode(raw_hash, &out);
+ out.resize(hash_size_);
+ return out;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/md5_hasher_test.cc b/trunk/src/net/instaweb/util/md5_hasher_test.cc
new file mode 100644
index 0000000..7144ab3
--- /dev/null
+++ b/trunk/src/net/instaweb/util/md5_hasher_test.cc
@@ -0,0 +1,50 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Authors: sligocki@google.com (Shawn Ligocki),
+
+#include "net/instaweb/util/public/md5_hasher.h"
+
+#include "net/instaweb/util/public/gtest.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// See http://goto/gunitprimer for an introduction to gUnit.
+
+class MD5HasherTest : public ::testing::Test {};
+
+TEST_F(MD5HasherTest, CorrectHashSize) {
+ for (int i = MD5Hasher::kMaxHashSize; i >= 0; --i) {
+ MD5Hasher hasher(i);
+ EXPECT_EQ(i, hasher.HashSizeInChars());
+ EXPECT_EQ(i, hasher.Hash("foobar").size());
+ // Large string.
+ EXPECT_EQ(i, hasher.Hash(std::string(5000, 'z')).size());
+ }
+}
+
+TEST_F(MD5HasherTest, HashesDiffer) {
+ MD5Hasher hasher;
+
+ // Basic sanity tests. More thorough tests belong in the base implementation.
+ EXPECT_NE(hasher.Hash("foo"), hasher.Hash("bar"));
+ EXPECT_NE(hasher.Hash(std::string(5000, 'z')),
+ hasher.Hash(std::string(5001, 'z')));
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/mem_file_system.cc b/trunk/src/net/instaweb/util/mem_file_system.cc
new file mode 100644
index 0000000..a000849
--- /dev/null
+++ b/trunk/src/net/instaweb/util/mem_file_system.cc
@@ -0,0 +1,278 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: abliss@google.com (Adam Bliss)
+
+#include "net/instaweb/util/public/mem_file_system.h"
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/errno.h>
+#include <sys/stat.h>
+
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/mock_timer.h"
+#include <string>
+#include "net/instaweb/util/public/timer.h"
+
+namespace net_instaweb {
+
+class MemInputFile : public FileSystem::InputFile {
+ public:
+ MemInputFile(const StringPiece& filename, const std::string& contents)
+ : contents_(contents),
+ filename_(filename.data(), filename.size()),
+ offset_(0) {
+ }
+
+ virtual bool Close(MessageHandler* message_handler) {
+ offset_ = contents_.length();
+ return true;
+ }
+
+ virtual const char* filename() { return filename_.c_str(); }
+
+ virtual int Read(char* buf, int size, MessageHandler* message_handler) {
+ if (size + offset_ > static_cast<int>(contents_.length())) {
+ size = contents_.length() - offset_;
+ }
+ memcpy(buf, contents_.c_str() + offset_, size);
+ offset_ += size;
+ return size;
+ }
+
+ private:
+ const std::string contents_;
+ const std::string filename_;
+ int offset_;
+
+ DISALLOW_COPY_AND_ASSIGN(MemInputFile);
+};
+
+
+class MemOutputFile : public FileSystem::OutputFile {
+ public:
+ MemOutputFile(const StringPiece& filename, std::string* contents)
+ : contents_(contents), filename_(filename.data(), filename.size()) {
+ contents_->clear();
+ }
+
+ virtual bool Close(MessageHandler* message_handler) {
+ Flush(message_handler);
+ return true;
+ }
+
+ virtual const char* filename() { return filename_.c_str(); }
+
+ virtual bool Flush(MessageHandler* message_handler) {
+ contents_->append(written_);
+ written_.clear();
+ return true;
+ }
+
+ virtual bool SetWorldReadable(MessageHandler* message_handler) {
+ return true;
+ }
+
+ virtual bool Write(const StringPiece& buf, MessageHandler* handler) {
+ buf.AppendToString(&written_);
+ return true;
+ }
+
+ private:
+ std::string* contents_;
+ const std::string filename_;
+ std::string written_;
+
+ DISALLOW_COPY_AND_ASSIGN(MemOutputFile);
+};
+
+MemFileSystem::~MemFileSystem() {
+}
+
+int64 MemFileSystem::CurrentTimeAndAdvance() {
+ int64 now_us = timer_.NowUs();
+ int64 now_s = now_us / Timer::kSecondUs;
+ timer_.advance_us(Timer::kSecondUs);
+ return now_s;
+}
+
+void MemFileSystem::Clear() {
+ string_map_.clear();
+}
+
+BoolOrError MemFileSystem::Exists(const char* path, MessageHandler* handler) {
+ StringMap::const_iterator iter = string_map_.find(path);
+ return BoolOrError(iter != string_map_.end());
+}
+
+BoolOrError MemFileSystem::IsDir(const char* path, MessageHandler* handler) {
+ return Exists(path, handler).is_true()
+ ? BoolOrError(EndsInSlash(path)) : BoolOrError();
+}
+
+bool MemFileSystem::MakeDir(const char* path, MessageHandler* handler) {
+ // We store directories as empty files with trailing slashes.
+ std::string path_string = path;
+ EnsureEndsInSlash(&path_string);
+ string_map_[path_string] = "";
+ atime_map_[path_string] = CurrentTimeAndAdvance();
+ return true;
+}
+
+FileSystem::InputFile* MemFileSystem::OpenInputFile(
+ const char* filename, MessageHandler* message_handler) {
+ if (!enabled_) {
+ return NULL;
+ }
+
+ StringMap::const_iterator iter = string_map_.find(filename);
+ if (iter == string_map_.end()) {
+ message_handler->Error(filename, 0, "opening input file: %s",
+ "file not found");
+ return NULL;
+ } else {
+ atime_map_[filename] = CurrentTimeAndAdvance();
+ return new MemInputFile(filename, iter->second);
+ }
+}
+
+FileSystem::OutputFile* MemFileSystem::OpenOutputFileHelper(
+ const char* filename, MessageHandler* message_handler) {
+ atime_map_[filename] = CurrentTimeAndAdvance();
+ return new MemOutputFile(filename, &(string_map_[filename]));
+}
+
+FileSystem::OutputFile* MemFileSystem::OpenTempFileHelper(
+ const StringPiece& prefix, MessageHandler* message_handler) {
+ std::string filename = StringPrintf("tmpfile%d", temp_file_index_++);
+ atime_map_[filename] = CurrentTimeAndAdvance();
+ return new MemOutputFile(filename, &string_map_[filename]);
+}
+
+bool MemFileSystem::RecursivelyMakeDir(const StringPiece& full_path_const,
+ MessageHandler* handler) {
+ // This is called to make sure that files can be written under the
+ // named directory. We don't have directories and files can be
+ // written anywhere, so just return true.
+ return true;
+}
+
+bool MemFileSystem::RemoveFile(const char* filename,
+ MessageHandler* handler) {
+ atime_map_.erase(filename);
+ return (string_map_.erase(filename) == 1);
+}
+
+bool MemFileSystem::RenameFileHelper(const char* old_file,
+ const char* new_file,
+ MessageHandler* handler) {
+ atime_map_[new_file] = CurrentTimeAndAdvance();
+ if (strcmp(old_file, new_file) == 0) {
+ handler->Error(old_file, 0, "Cannot move a file to itself");
+ return false;
+ }
+
+ StringMap::iterator iter = string_map_.find(old_file);
+ if (iter == string_map_.end()) {
+ handler->Error(old_file, 0, "File not found");
+ return false;
+ }
+
+ string_map_[new_file] = iter->second;
+ string_map_.erase(iter);
+ return true;
+}
+
+bool MemFileSystem::ListContents(const StringPiece& dir, StringVector* files,
+ MessageHandler* handler) {
+ std::string prefix = dir.as_string();
+ EnsureEndsInSlash(&prefix);
+ const size_t prefix_length = prefix.size();
+ // We don't have directories, so we just list everything in the
+ // filesystem that matches the prefix and doesn't have another
+ // internal slash.
+ for (StringMap::iterator it = string_map_.begin(), end = string_map_.end();
+ it != end;
+ it++) {
+ const std::string& path = (*it).first;
+ if ((0 == path.compare(0, prefix_length, prefix)) &&
+ path.length() > prefix_length) {
+ const size_t next_slash = path.find("/", prefix_length + 1);
+ // Only want to list files without another slash, unless that
+ // slash is the last char in the filename.
+ if ((next_slash == std::string::npos)
+ || (next_slash == path.length() - 1)) {
+ files->push_back(path);
+ }
+ }
+ }
+ return true;
+}
+
+bool MemFileSystem::Atime(const StringPiece& path, int64* timestamp_sec,
+ MessageHandler* handler) {
+ *timestamp_sec = atime_map_[path.as_string()];
+ return true;
+}
+
+bool MemFileSystem::Size(const StringPiece& path, int64* size,
+ MessageHandler* handler) {
+ const std::string path_string = path.as_string();
+ const char* path_str = path_string.c_str();
+ if (Exists(path_str, handler).is_true()) {
+ *size = string_map_[path_string].size();
+ return true;
+ } else {
+ return false;
+ }
+}
+
+BoolOrError MemFileSystem::TryLock(const StringPiece& lock_name,
+ MessageHandler* handler) {
+ // Not actually threadsafe! This is just for tests.
+ if (lock_map_.count(lock_name.as_string()) != 0) {
+ return BoolOrError(false);
+ } else {
+ lock_map_[lock_name.as_string()] = timer_.NowMs();
+ return BoolOrError(true);
+ }
+}
+
+BoolOrError MemFileSystem::TryLockWithTimeout(const StringPiece& lock_name,
+ int64 timeout_ms,
+ MessageHandler* handler) {
+ // As above, not actually threadsafe (and quick-and-dirty rather than
+ // efficient; efficiency requires map::find).
+ std::string name = lock_name.as_string();
+ int64 now = timer_.NowMs();
+ if (lock_map_.count(name) != 0 &&
+ now <= lock_map_[name] + timeout_ms) {
+ return BoolOrError(false);
+ } else {
+ lock_map_[name] = timer_.NowMs();
+ return BoolOrError(true);
+ }
+}
+
+bool MemFileSystem::Unlock(const StringPiece& lock_name,
+ MessageHandler* handler) {
+ return (lock_map_.erase(lock_name.as_string()) == 1);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/mem_file_system_test.cc b/trunk/src/net/instaweb/util/mem_file_system_test.cc
new file mode 100644
index 0000000..256d46f
--- /dev/null
+++ b/trunk/src/net/instaweb/util/mem_file_system_test.cc
@@ -0,0 +1,149 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: abliss@google.com (Adam Bliss)
+
+// Unit-test the in-memory filesystem
+
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/file_system_test.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/mem_file_system.h"
+#include "net/instaweb/util/public/null_message_handler.h"
+#include <string>
+
+namespace net_instaweb {
+
+class MemFileSystemTest : public FileSystemTest {
+ protected:
+ MemFileSystemTest() {}
+ virtual void DeleteRecursively(const StringPiece& filename) {
+ mem_file_system_.Clear();
+ }
+ virtual FileSystem* file_system() {
+ return &mem_file_system_;
+ }
+ virtual std::string test_tmpdir() {
+ return GTestTempDir();
+ }
+ private:
+ MemFileSystem mem_file_system_;
+
+ DISALLOW_COPY_AND_ASSIGN(MemFileSystemTest);
+};
+
+// Write a named file, then read it.
+TEST_F(MemFileSystemTest, TestWriteRead) {
+ TestWriteRead();
+}
+
+// Write a temp file, then read it.
+TEST_F(MemFileSystemTest, TestTemp) {
+ TestTemp();
+}
+
+// Write a temp file, rename it, then read it.
+TEST_F(MemFileSystemTest, TestRename) {
+ TestRename();
+}
+
+// Write a file and successfully delete it.
+TEST_F(MemFileSystemTest, TestRemove) {
+ TestRemove();
+}
+
+// Write a file and check that it exists.
+TEST_F(MemFileSystemTest, TestExists) {
+ TestExists();
+}
+
+// Create a file along with its directory which does not exist.
+TEST_F(MemFileSystemTest, TestCreateFileInDir) {
+ TestCreateFileInDir();
+}
+
+
+// Make a directory and check that files may be placed in it.
+TEST_F(MemFileSystemTest, TestMakeDir) {
+ TestMakeDir();
+}
+
+TEST_F(MemFileSystemTest, TestSize) {
+ // Since we don't have directories, we need to do a slightly
+ // different size test.
+ std::string filename1 = "file-in-dir.txt";
+ std::string filename2 = "another-file-in-dir.txt";
+ std::string content1 = "12345";
+ std::string content2 = "1234567890";
+ ASSERT_TRUE(file_system()->WriteFile(filename1.c_str(),
+ content1, &handler_));
+ ASSERT_TRUE(file_system()->WriteFile(filename2.c_str(),
+ content2, &handler_));
+ int64 size;
+
+ EXPECT_TRUE(file_system()->Size(filename1, &size, &handler_));
+ EXPECT_EQ(5, size);
+ EXPECT_TRUE(file_system()->Size(filename2, &size, &handler_));
+ EXPECT_EQ(10, size);
+}
+
+TEST_F(MemFileSystemTest, TestListContents) {
+ TestListContents();
+}
+
+TEST_F(MemFileSystemTest, TestAtime) {
+ // Slightly modified version of TestAtime, without the sleeps
+ std::string dir_name = test_tmpdir() + "/make_dir";
+ DeleteRecursively(dir_name);
+ std::string filename1 = "file-in-dir.txt";
+ std::string filename2 = "another-file-in-dir.txt";
+ std::string full_path1 = dir_name + "/" + filename1;
+ std::string full_path2 = dir_name + "/" + filename2;
+ std::string content = "Lorem ipsum dolor sit amet";
+
+ ASSERT_TRUE(file_system()->MakeDir(dir_name.c_str(), &handler_));
+ ASSERT_TRUE(file_system()->WriteFile(full_path1.c_str(),
+ content, &handler_));
+ ASSERT_TRUE(file_system()->WriteFile(full_path2.c_str(),
+ content, &handler_));
+
+ int64 atime1, atime2;
+ CheckRead(full_path1, content);
+ CheckRead(full_path2, content);
+ ASSERT_TRUE(file_system()->Atime(full_path1, &atime1, &handler_));
+ ASSERT_TRUE(file_system()->Atime(full_path2, &atime2, &handler_));
+ EXPECT_LT(atime1, atime2);
+
+ CheckRead(full_path2, content);
+ CheckRead(full_path1, content);
+ ASSERT_TRUE(file_system()->Atime(full_path1, &atime1, &handler_));
+ ASSERT_TRUE(file_system()->Atime(full_path2, &atime2, &handler_));
+ EXPECT_LT(atime2, atime1);
+}
+
+TEST_F(MemFileSystemTest, TestLock) {
+ TestLock();
+}
+
+// Since this filesystem doesn't support directories, we skip these tests:
+// TestIsDir
+// TestRecursivelyMakeDir
+// TestRecursivelyMakeDir_NoPermission
+// TestRecursivelyMakeDir_FileInPath
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/message_handler.cc b/trunk/src/net/instaweb/util/message_handler.cc
new file mode 100644
index 0000000..1882f04
--- /dev/null
+++ b/trunk/src/net/instaweb/util/message_handler.cc
@@ -0,0 +1,125 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/message_handler.h"
+
+#include "base/logging.h"
+
+namespace net_instaweb {
+
+MessageHandler::MessageHandler() : min_message_type_(kInfo) {
+}
+
+MessageHandler::~MessageHandler() {
+}
+
+const char* MessageHandler::MessageTypeToString(const MessageType type) const {
+ const char* type_string = NULL;
+
+ // Don't include a 'default:' clause so that the compiler can tell us when
+ // we are missing an enum value. Intead use a null check for 'type_string' to
+ // indicate a data corruption that avoids hitting any of the cases.
+ switch (type) {
+ case kInfo:
+ type_string = "Info";
+ break;
+ case kWarning:
+ type_string = "Warning";
+ break;
+ case kError:
+ type_string = "Error";
+ break;
+ case kFatal:
+ type_string = "Fatal";
+ break;
+ }
+ CHECK(type_string != NULL) << "INVALID MessageType!";
+ return type_string;
+}
+
+void MessageHandler::Message(MessageType type, const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ MessageV(type, msg, args);
+ va_end(args);
+}
+
+void MessageHandler::MessageV(MessageType type, const char* msg, va_list args) {
+ if (type >= min_message_type_) {
+ MessageVImpl(type, msg, args);
+ }
+}
+
+void MessageHandler::FileMessage(MessageType type, const char* file, int line,
+ const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ FileMessageV(type, file, line, msg, args);
+ va_end(args);
+}
+
+void MessageHandler::FileMessageV(MessageType type, const char* filename,
+ int line, const char* msg, va_list args) {
+ if (type >= min_message_type_) {
+ FileMessageVImpl(type, filename, line, msg, args);
+ }
+}
+
+void MessageHandler::Check(bool condition, const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ CheckV(condition, msg, args);
+ va_end(args);
+}
+
+void MessageHandler::CheckV(bool condition, const char* msg, va_list args) {
+ if (!condition) {
+ MessageV(kFatal, msg, args);
+ }
+}
+
+void MessageHandler::Info(const char* file, int line, const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ InfoV(file, line, msg, args);
+ va_end(args);
+}
+
+void MessageHandler::Warning(const char* file, int line, const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ WarningV(file, line, msg, args);
+ va_end(args);
+}
+
+void MessageHandler::Error(const char* file, int line, const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ ErrorV(file, line, msg, args);
+ va_end(args);
+}
+
+void MessageHandler::FatalError(
+ const char* file, int line, const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ FatalErrorV(file, line, msg, args);
+ va_end(args);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/message_handler_test.cc b/trunk/src/net/instaweb/util/message_handler_test.cc
new file mode 100644
index 0000000..160114f
--- /dev/null
+++ b/trunk/src/net/instaweb/util/message_handler_test.cc
@@ -0,0 +1,85 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: bmcquade@google.com (Bryan McQuade)
+
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+namespace {
+
+class TestMessageHandler : public net_instaweb::MessageHandler {
+ public:
+ typedef std::vector<std::string> MessageVector;
+
+ const MessageVector& messages() { return messages_; }
+
+ protected:
+ virtual void MessageVImpl(MessageType type, const char* msg,
+ va_list args) {
+ std::string message;
+ StringAppendF(&message, "%s: ", MessageTypeToString(type));
+ StringAppendV(&message, msg, args);
+ messages_.push_back(message);
+ }
+
+ virtual void FileMessageVImpl(MessageType type, const char* filename,
+ int line, const char* msg, va_list args) {
+ std::string message;
+ StringAppendF(&message, "%s: %s: %d: ", MessageTypeToString(type),
+ filename, line);
+ StringAppendV(&message, msg, args);
+ messages_.push_back(message);
+ }
+
+ private:
+ MessageVector messages_;
+};
+
+class MessageHandlerTest : public testing::Test {
+ protected:
+ const TestMessageHandler::MessageVector& messages() {
+ return handler_.messages();
+ }
+
+ TestMessageHandler handler_;
+};
+
+
+TEST_F(MessageHandlerTest, Simple) {
+ handler_.Message(kWarning, "here is a message");
+ handler_.Info("filename.cc", 1, "here is another message");
+ ASSERT_EQ(2U, messages().size());
+ ASSERT_EQ(messages()[0], "Warning: here is a message");
+ ASSERT_EQ(messages()[1], "Info: filename.cc: 1: here is another message");
+}
+
+TEST_F(MessageHandlerTest, MinMessageType) {
+ handler_.set_min_message_type(kError);
+ handler_.Info("filename.cc", 1, "here is a message");
+ handler_.Warning("filename.cc", 1, "here is a message");
+ ASSERT_EQ(0U, messages().size());
+ handler_.Error("filename.cc", 1, "here is another message");
+ ASSERT_EQ(1U, messages().size());
+ ASSERT_EQ(messages()[0], "Error: filename.cc: 1: here is another message");
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/meta_data.cc b/trunk/src/net/instaweb/util/meta_data.cc
new file mode 100644
index 0000000..92ae961
--- /dev/null
+++ b/trunk/src/net/instaweb/util/meta_data.cc
@@ -0,0 +1,177 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/meta_data.h"
+
+#include <stdio.h>
+#include "net/instaweb/util/public/time_util.h"
+#include "pagespeed/core/resource_util.h"
+
+namespace net_instaweb {
+
+const char HttpAttributes::kAcceptEncoding[] = "Accept-Encoding";
+const char HttpAttributes::kCacheControl[] = "Cache-Control";
+const char HttpAttributes::kContentEncoding[] = "Content-Encoding";
+const char HttpAttributes::kContentLength[] = "Content-Length";
+const char HttpAttributes::kContentType[] = "Content-Type";
+const char HttpAttributes::kDate[] = "Date";
+const char HttpAttributes::kDeflate[] = "deflate";
+const char HttpAttributes::kEtag[] = "Etag";
+const char HttpAttributes::kExpires[] = "Expires";
+const char HttpAttributes::kGzip[] = "gzip";
+const char HttpAttributes::kHost[] = "Host";
+const char HttpAttributes::kIfModifiedSince[] = "If-Modified-Since";
+const char HttpAttributes::kLastModified[] = "Last-Modified";
+const char HttpAttributes::kLocation[] = "Location";
+const char HttpAttributes::kNoCache[] = "max-age=0, no-cache, no-store";
+const char HttpAttributes::kReferer[] = "Referer"; // sic
+const char HttpAttributes::kServer[] = "Server";
+const char HttpAttributes::kSetCookie[] = "Set-Cookie";
+const char HttpAttributes::kTransferEncoding[] = "Transfer-Encoding";
+const char HttpAttributes::kUserAgent[] = "User-Agent";
+const char HttpAttributes::kVary[] = "Vary";
+
+MetaData::~MetaData() {
+}
+
+void MetaData::CopyFrom(const MetaData& other) {
+ set_major_version(other.major_version());
+ set_minor_version(other.minor_version());
+ set_status_code(other.status_code());
+ set_reason_phrase(other.reason_phrase());
+ set_headers_complete(other.headers_complete());
+ for (int i = 0; i < other.NumAttributes(); ++i) {
+ Add(other.Name(i), other.Value(i));
+ }
+ ComputeCaching();
+}
+
+const char* HttpStatus::GetReasonPhrase(HttpStatus::Code rc) {
+ switch (rc) {
+ case HttpStatus::kContinue : return "Continue";
+ case HttpStatus::kSwitchingProtocols : return "Switching Protocols";
+
+ case HttpStatus::kOK : return "OK";
+ case HttpStatus::kCreated : return "Created";
+ case HttpStatus::kAccepted : return "Accepted";
+ case HttpStatus::kNonAuthoritative :
+ return "Non-Authoritative Information";
+ case HttpStatus::kNoContent : return "No Content";
+ case HttpStatus::kResetContent : return "Reset Content";
+ case HttpStatus::kPartialContent : return "Partial Content";
+
+ // 300 range: redirects
+ case HttpStatus::kMultipleChoices : return "Multiple Choices";
+ case HttpStatus::kMovedPermanently : return "Moved Permanently";
+ case HttpStatus::kFound : return "Found";
+ case HttpStatus::kSeeOther : return "See Other";
+ case HttpStatus::kNotModified : return "Not Modified";
+ case HttpStatus::kUseProxy : return "Use Proxy";
+ case HttpStatus::kTemporaryRedirect : return "OK";
+
+ // 400 range: client errors
+ case HttpStatus::kBadRequest : return "Bad Request";
+ case HttpStatus::kUnauthorized : return "Unauthorized";
+ case HttpStatus::kPaymentRequired : return "Payment Required";
+ case HttpStatus::kForbidden : return "Forbidden";
+ case HttpStatus::kNotFound : return "Not Found";
+ case HttpStatus::kMethodNotAllowed : return "Method Not Allowed";
+ case HttpStatus::kNotAcceptable : return "Not Acceptable";
+ case HttpStatus::kProxyAuthRequired :
+ return "Proxy Authentication Required";
+ case HttpStatus::kRequestTimeout : return "Request Time-out";
+ case HttpStatus::kConflict : return "Conflict";
+ case HttpStatus::kGone : return "Gone";
+ case HttpStatus::kLengthRequired : return "Length Required";
+ case HttpStatus::kPreconditionFailed : return "Precondition Failed";
+ case HttpStatus::kEntityTooLarge :
+ return "Request Entity Too Large";
+ case HttpStatus::kUriTooLong : return "Request-URI Too Large";
+ case HttpStatus::kUnsupportedMediaType : return "Unsupported Media Type";
+ case HttpStatus::kRangeNotSatisfiable :
+ return "Requested range not satisfiable";
+ case HttpStatus::kExpectationFailed : return "Expectation Failed";
+
+ // 500 range: server errors
+ case HttpStatus::kInternalServerError : return "Internal Server Error";
+ case HttpStatus::kNotImplemented : return "Not Implemented";
+ case HttpStatus::kBadGateway : return "Bad Gateway";
+ case HttpStatus::kUnavailable : return "Service Unavailable";
+ case HttpStatus::kGatewayTimeout : return "Gateway Time-out";
+
+ default:
+ // We don't have a name for this response code, so we'll just
+ // take the blame
+ return "Internal Server Error";
+ }
+ return "";
+}
+
+void MetaData::SetStatusAndReason(HttpStatus::Code code) {
+ set_status_code(code);
+ set_reason_phrase(HttpStatus::GetReasonPhrase(code));
+}
+
+bool MetaData::ParseTime(const char* time_str, int64* time_ms) {
+ return pagespeed::resource_util::ParseTimeValuedHeader(time_str, time_ms);
+}
+
+bool MetaData::IsGzipped() const {
+ CHECK(headers_complete());
+ CharStarVector v;
+ return (Lookup(HttpAttributes::kContentEncoding, &v) && (v.size() == 1) &&
+ (strcmp(v[0], HttpAttributes::kGzip) == 0));
+}
+
+bool MetaData::AcceptsGzip() const {
+ CharStarVector v;
+ if (Lookup(HttpAttributes::kAcceptEncoding, &v)) {
+ for (int i = 0, nv = v.size(); i < nv; ++i) {
+ std::vector<StringPiece> encodings;
+ SplitStringPieceToVector(v[i], ",", &encodings, true);
+ for (int j = 0, nencodings = encodings.size(); j < nencodings; ++j) {
+ if (strcasecmp(encodings[j].as_string().c_str(),
+ HttpAttributes::kGzip) == 0) {
+ return true;
+ }
+ }
+ }
+ }
+ return false;
+}
+
+bool MetaData::ParseDateHeader(const char* attr, int64* date_ms) const {
+ CharStarVector values;
+ return (Lookup(attr, &values) &&
+ (values.size() == 1) &&
+ ConvertStringToTime(values[0], date_ms));
+}
+
+void MetaData::UpdateDateHeader(const char* attr, int64 date_ms) {
+ RemoveAll(attr);
+ std::string buf;
+ if (ConvertTimeToString(date_ms, &buf)) {
+ Add(attr, buf.c_str());
+ }
+}
+
+void MetaData::DebugPrint() const {
+ fprintf(stderr, "%s\n", ToString().c_str());
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/mock_hasher.cc b/trunk/src/net/instaweb/util/mock_hasher.cc
new file mode 100644
index 0000000..bbd9b97
--- /dev/null
+++ b/trunk/src/net/instaweb/util/mock_hasher.cc
@@ -0,0 +1,26 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/mock_hasher.h"
+
+namespace net_instaweb {
+
+MockHasher::~MockHasher() {
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/mock_message_handler.cc b/trunk/src/net/instaweb/util/mock_message_handler.cc
new file mode 100644
index 0000000..fdca5ef
--- /dev/null
+++ b/trunk/src/net/instaweb/util/mock_message_handler.cc
@@ -0,0 +1,62 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: morlovich@google.com (Maksim Orlovich)
+
+#include "net/instaweb/util/public/mock_message_handler.h"
+
+namespace net_instaweb {
+
+MockMessageHandler::~MockMessageHandler() {
+}
+
+void MockMessageHandler::MessageVImpl(MessageType type,
+ const char* msg,
+ va_list args) {
+ GoogleMessageHandler::MessageVImpl(type, msg, args);
+ ++message_counts_[type];
+}
+
+void MockMessageHandler::FileMessageVImpl(MessageType type,
+ const char* filename, int line,
+ const char* msg, va_list args) {
+ GoogleMessageHandler::FileMessageVImpl(type, filename, line, msg, args);
+ ++message_counts_[type];
+}
+
+int MockMessageHandler::MessagesOfType(MessageType type) const {
+ MessageCountMap::const_iterator i = message_counts_.find(type);
+ if (i != message_counts_.end()) {
+ return i->second;
+ } else {
+ return 0;
+ }
+}
+
+int MockMessageHandler::TotalMessages() const {
+ int total = 0;
+ for (MessageCountMap::const_iterator i = message_counts_.begin();
+ i != message_counts_.end(); ++i) {
+ total += i->second;
+ }
+ return total;
+}
+
+int MockMessageHandler::SeriousMessages() const {
+ return TotalMessages() - MessagesOfType(kInfo);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/mock_message_handler_test.cc b/trunk/src/net/instaweb/util/mock_message_handler_test.cc
new file mode 100644
index 0000000..9c36daf
--- /dev/null
+++ b/trunk/src/net/instaweb/util/mock_message_handler_test.cc
@@ -0,0 +1,70 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: morlovich@google.com (Maksim Orlovich)
+
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/mock_message_handler.h"
+
+namespace net_instaweb {
+
+namespace {
+
+class MockMessageHandlerTest : public testing::Test {
+ protected:
+ void CheckCounts(int expectInfo, int expectWarn, int expectError,
+ int expectFatal) {
+ EXPECT_EQ(expectInfo, handler_.MessagesOfType(kInfo));
+ EXPECT_EQ(expectWarn, handler_.MessagesOfType(kWarning));
+ EXPECT_EQ(expectError, handler_.MessagesOfType(kError));
+ EXPECT_EQ(expectFatal, handler_.MessagesOfType(kFatal));
+ }
+
+ MockMessageHandler handler_;
+};
+
+
+TEST_F(MockMessageHandlerTest, Simple) {
+ EXPECT_EQ(0, handler_.TotalMessages());
+ EXPECT_EQ(0, handler_.SeriousMessages());
+
+ handler_.Message(kInfo, "test info message");
+ EXPECT_EQ(1, handler_.TotalMessages());
+ EXPECT_EQ(0, handler_.SeriousMessages());
+ CheckCounts(1, 0, 0, 0);
+
+ handler_.Message(kWarning, "text warning message");
+ EXPECT_EQ(2, handler_.TotalMessages());
+ EXPECT_EQ(1, handler_.SeriousMessages());
+ CheckCounts(1, 1, 0, 0);
+
+ handler_.Message(kError, "text Error message");
+ EXPECT_EQ(3, handler_.TotalMessages());
+ EXPECT_EQ(2, handler_.SeriousMessages());
+ CheckCounts(1, 1, 1, 0);
+
+ // We can't actually test fatal, as it aborts
+ // TODO(morlovich) mock the fatal behavior so the test does not crash
+
+ handler_.Message(kInfo, "another test info message");
+ EXPECT_EQ(4, handler_.TotalMessages());
+ EXPECT_EQ(2, handler_.SeriousMessages());
+ CheckCounts(2, 1, 1, 0);
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/mock_timer.cc b/trunk/src/net/instaweb/util/mock_timer.cc
new file mode 100644
index 0000000..47a85a8
--- /dev/null
+++ b/trunk/src/net/instaweb/util/mock_timer.cc
@@ -0,0 +1,28 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/mock_timer.h"
+
+namespace net_instaweb {
+
+const int64 MockTimer::kApr_5_2010_ms = 1270493486000LL;
+
+MockTimer::~MockTimer() {
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/mock_url_fetcher.cc b/trunk/src/net/instaweb/util/mock_url_fetcher.cc
new file mode 100644
index 0000000..b3aeed5
--- /dev/null
+++ b/trunk/src/net/instaweb/util/mock_url_fetcher.cc
@@ -0,0 +1,79 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/mock_url_fetcher.h"
+#include "net/instaweb/util/public/stl_util.h"
+#include "net/instaweb/util/public/writer.h"
+
+namespace net_instaweb {
+
+MockUrlFetcher::~MockUrlFetcher() {
+ Clear();
+}
+
+void MockUrlFetcher::SetResponse(const StringPiece& url,
+ const MetaData& response_header,
+ const StringPiece& response_body) {
+ std::string url_string = url.as_string();
+ // Delete any old response.
+ ResponseMap::iterator iter = response_map_.find(url_string);
+ if (iter != response_map_.end()) {
+ delete iter->second;
+ response_map_.erase(iter);
+ }
+
+ // Add new response.
+ HttpResponse* response = new HttpResponse(response_header, response_body);
+ response_map_.insert(ResponseMap::value_type(url_string, response));
+}
+
+void MockUrlFetcher::Clear() {
+ STLDeleteContainerPairSecondPointers(response_map_.begin(),
+ response_map_.end());
+ response_map_.clear();
+}
+
+bool MockUrlFetcher::StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* message_handler) {
+ bool ret = false;
+ if (enabled_) {
+ ResponseMap::iterator iter = response_map_.find(url);
+ if (iter != response_map_.end()) {
+ const HttpResponse* response = iter->second;
+ response_headers->CopyFrom(response->header());
+ response_writer->Write(response->body(), message_handler);
+ ret = true;
+ } else {
+ // This is used in tests and we do not expect the test to request a
+ // resource that we don't have. So fail if we do.
+ //
+ // If you want a 404 response, you must explicitly use SetResponse.
+ if (fail_on_unexpected_) {
+ EXPECT_TRUE(false) << "Requested unset url " << url;
+ }
+ }
+ }
+ return ret;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/mock_url_fetcher_test.cc b/trunk/src/net/instaweb/util/mock_url_fetcher_test.cc
new file mode 100644
index 0000000..e8d52db
--- /dev/null
+++ b/trunk/src/net/instaweb/util/mock_url_fetcher_test.cc
@@ -0,0 +1,116 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/public/mock_url_fetcher.h"
+
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/gtest.h"
+#include <string>
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// See http://goto/gunitprimer for an introduction to gUnit.
+
+class MockUrlFetcherTest : public ::testing::Test {
+ protected:
+ MockUrlFetcherTest() {
+ fetcher_.set_fail_on_unexpected(false);
+ }
+
+ void TestResponse(const std::string& url, const MetaData& expected_header,
+ const std::string& expected_body) {
+ const SimpleMetaData dummy_header;
+ SimpleMetaData response_header;
+ std::string response_body;
+ StringWriter response_writer(&response_body);
+ GoogleMessageHandler handler;
+
+ EXPECT_TRUE(fetcher_.StreamingFetchUrl(url, dummy_header, &response_header,
+ &response_writer, &handler));
+ EXPECT_EQ(expected_header.ToString(), response_header.ToString());
+ EXPECT_EQ(expected_body, response_body);
+ }
+
+ void TestFetchFail(const std::string& url) {
+ const SimpleMetaData dummy_header;
+ SimpleMetaData response_header;
+ std::string response_body;
+ StringWriter response_writer(&response_body);
+ GoogleMessageHandler handler;
+
+ EXPECT_FALSE(fetcher_.StreamingFetchUrl(url, dummy_header, &response_header,
+ &response_writer, &handler));
+ }
+
+ MockUrlFetcher fetcher_;
+};
+
+TEST_F(MockUrlFetcherTest, GetsCorrectMappedResponse) {
+ const char url1[] = "http://www.example.com/successs.html";
+ SimpleMetaData header1;
+ header1.set_first_line(1, 1, 200, "OK");
+ const char body1[] = "This website loaded :)";
+
+ const char url2[] = "http://www.example.com/failure.html";
+ SimpleMetaData header2;
+ header2.set_first_line(1, 1, 404, "Not Found");
+ const char body2[] = "File Not Found :(";
+
+ // We can't fetch the URLs before they're set.
+ // Note: this does not crash because we are using NullMessageHandler.
+ TestFetchFail(url1);
+ TestFetchFail(url2);
+
+ // Set the responses.
+ fetcher_.SetResponse(url1, header1, body1);
+ fetcher_.SetResponse(url2, header2, body2);
+
+ // Now we can fetch the correct URLs
+ TestResponse(url1, header1, body1);
+ TestResponse(url2, header2, body2);
+
+ // Check that we can fetch the same URL multiple times.
+ TestResponse(url1, header1, body1);
+
+
+ // Check that fetches fail after disabling the fetcher_.
+ fetcher_.Disable();
+ TestFetchFail(url1);
+ // And then work again when re-enabled.
+ fetcher_.Enable();
+ TestResponse(url1, header1, body1);
+
+
+ // Change the response. Test both editability and memory management.
+ fetcher_.SetResponse(url1, header2, body2);
+
+ // Check that we get the new response.
+ TestResponse(url1, header2, body2);
+
+ // Change it back.
+ fetcher_.SetResponse(url1, header1, body1);
+ TestResponse(url1, header1, body1);
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/named_lock_manager.cc b/trunk/src/net/instaweb/util/named_lock_manager.cc
new file mode 100644
index 0000000..f39dc95
--- /dev/null
+++ b/trunk/src/net/instaweb/util/named_lock_manager.cc
@@ -0,0 +1,27 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/util/public/named_lock_manager.h"
+
+namespace net_instaweb {
+
+AbstractLock::~AbstractLock() { }
+
+NamedLockManager::~NamedLockManager() { }
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/null_message_handler.cc b/trunk/src/net/instaweb/util/null_message_handler.cc
new file mode 100644
index 0000000..9db5a2e
--- /dev/null
+++ b/trunk/src/net/instaweb/util/null_message_handler.cc
@@ -0,0 +1,35 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: lsong@google.com (Libo Song)
+
+#include "net/instaweb/util/public/null_message_handler.h"
+
+namespace net_instaweb {
+
+NullMessageHandler::~NullMessageHandler() {
+}
+
+void NullMessageHandler::MessageVImpl(MessageType type, const char* msg,
+ va_list args) {
+}
+
+void NullMessageHandler::FileMessageVImpl(MessageType type, const char* file,
+ int line, const char* msg,
+ va_list args) {
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/null_writer.cc b/trunk/src/net/instaweb/util/null_writer.cc
new file mode 100644
index 0000000..e407ab9
--- /dev/null
+++ b/trunk/src/net/instaweb/util/null_writer.cc
@@ -0,0 +1,34 @@
+/*
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/null_writer.h"
+
+namespace net_instaweb {
+
+NullWriter::~NullWriter() {
+}
+
+bool NullWriter::Write(const StringPiece& str, MessageHandler* handler) {
+ return true;
+}
+
+bool NullWriter::Flush(MessageHandler* handler) {
+ return true;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/pthread_mutex.cc b/trunk/src/net/instaweb/util/pthread_mutex.cc
new file mode 100644
index 0000000..e97d204
--- /dev/null
+++ b/trunk/src/net/instaweb/util/pthread_mutex.cc
@@ -0,0 +1,39 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/pthread_mutex.h"
+
+namespace net_instaweb {
+
+PthreadMutex::PthreadMutex() {
+ pthread_mutex_init(&mutex_, NULL);
+}
+
+PthreadMutex::~PthreadMutex() {
+ pthread_mutex_destroy(&mutex_);
+}
+
+void PthreadMutex::Lock() {
+ pthread_mutex_lock(&mutex_);
+}
+
+void PthreadMutex::Unlock() {
+ pthread_mutex_unlock(&mutex_);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/public/abstract_mutex.h b/trunk/src/net/instaweb/util/public/abstract_mutex.h
new file mode 100644
index 0000000..7086069
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/abstract_mutex.h
@@ -0,0 +1,52 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_ABSTRACT_MUTEX_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_ABSTRACT_MUTEX_H_
+
+#include "base/basictypes.h"
+
+namespace net_instaweb {
+
+// Abstract interface for implementing a mutex.
+class AbstractMutex {
+ public:
+ virtual ~AbstractMutex();
+ virtual void Lock() = 0;
+ virtual void Unlock() = 0;
+};
+
+// Helper class for lexically scoped mutexing.
+class ScopedMutex {
+ public:
+ explicit ScopedMutex(AbstractMutex* mutex) : mutex_(mutex) {
+ mutex_->Lock();
+ }
+
+ ~ScopedMutex() {
+ mutex_->Unlock();
+ }
+ private:
+ AbstractMutex* mutex_;
+
+ DISALLOW_COPY_AND_ASSIGN(ScopedMutex);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_ABSTRACT_MUTEX_H_
diff --git a/trunk/src/net/instaweb/util/public/atom.h b/trunk/src/net/instaweb/util/public/atom.h
new file mode 100644
index 0000000..8887ee4
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/atom.h
@@ -0,0 +1,89 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_ATOM_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_ATOM_H_
+
+#include <stdlib.h>
+#include <set>
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+struct StringCompareInsensitive;
+struct StringCompareSensitive;
+template<class SymbolCompare> class SymbolTable;
+
+// Atoms are idempotent representations of strings, created
+// via a symbol table.
+class Atom {
+ public:
+ Atom(const Atom& src) : str_(src.str_) {}
+ Atom() : str_("") {}
+ ~Atom() {} // atoms are memory-managed by SymbolTables.
+
+ Atom& operator=(const Atom& src) {
+ if (&src != this) {
+ str_ = src.str_;
+ }
+ return *this;
+ }
+
+ // string-like accessors.
+ const char* c_str() const { return str_; }
+ int size() const { return strlen(str_); }
+
+ // This is comparing the underlying char* pointers. It is invalid
+ // to compare Atoms from different symbol tables.
+ bool operator==(const Atom& sym) const {
+ return str_ == sym.str_;
+ }
+
+ // This is comparing the underlying char* pointers. It is invalid
+ // to compare Atoms from different symbol tables.
+ bool operator!=(const Atom& sym) const {
+ return str_ != sym.str_;
+ }
+
+ // SymbolTable is a friend of Symbol because SymbolTable is the
+ // only class that has the right to construct a new Atom from
+ // a char*.
+ friend class SymbolTable<CharStarCompareInsensitive>;
+ friend class SymbolTable<CharStarCompareSensitive>;
+
+ private:
+ explicit Atom(const char* str) : str_(str) {}
+ const char* str_;
+};
+
+// Once interned, Atoms are very cheap to put in a set, using
+// pointer-comparison.
+struct AtomCompare {
+ bool operator()(const Atom& a1, const Atom& a2) const {
+ return a1.c_str() < a2.c_str(); // compares pointers
+ }
+};
+
+// A set of atoms can be constructed very efficiently. Note that
+// iteration over this set will *not* be in alphabetical order.
+typedef std::set<Atom, AtomCompare> AtomSet;
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_ATOM_H_
diff --git a/trunk/src/net/instaweb/util/public/base64_util.h b/trunk/src/net/instaweb/util/public/base64_util.h
new file mode 100644
index 0000000..f7eb0e9
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/base64_util.h
@@ -0,0 +1,54 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_BASE64_UTIL_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_BASE64_UTIL_H_
+
+
+#include <string>
+
+#include "base/string_piece.h"
+#include "third_party/base64/base64.h"
+
+namespace net_instaweb {
+
+typedef base::StringPiece StringPiece;
+
+inline void Web64Encode(const StringPiece& in, std::string* out) {
+ *out = web64_encode(reinterpret_cast<const unsigned char*>(in.data()),
+ in.size());
+}
+
+inline bool Web64Decode(const StringPiece& in, std::string* out) {
+ bool ret = web64_decode(in.as_string(), out);
+ return ret;
+}
+
+inline void Mime64Encode(const StringPiece& in, std::string* out) {
+ *out = base64_encode(reinterpret_cast<const unsigned char*>(in.data()),
+ in.size());
+}
+
+inline bool Mime64Decode(const StringPiece& in, std::string* out) {
+ bool ret = base64_decode(in.as_string(), out);
+ return ret;
+}
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_BASE64_UTIL_H_
diff --git a/trunk/src/net/instaweb/util/public/cache_interface.h b/trunk/src/net/instaweb/util/public/cache_interface.h
new file mode 100644
index 0000000..62604a6
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/cache_interface.h
@@ -0,0 +1,57 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_CACHE_INTERFACE_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_CACHE_INTERFACE_H_
+
+// TODO(sligocki): We shouldn't need to include this in the .h, but it was
+// breaking someone somewhere, look into later.
+#include "net/instaweb/util/public/shared_string.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class MessageHandler;
+class SharedString;
+
+// Abstract interface for a cache.
+class CacheInterface {
+ public:
+ enum KeyState {
+ kAvailable, // Requested key is available for serving
+ kInTransit, // Requested key is being written, but is not readable
+ kNotFound // Requested key needs to be written
+ };
+
+ virtual ~CacheInterface();
+
+ // Gets an object from the cache, returning false on a cache miss
+ virtual bool Get(const std::string& key, SharedString* value) = 0;
+
+ // Puts a value into the cache. The value that is passed in is not modified,
+ // but the SharedString is passed by non-const pointer because its reference
+ // count is bumped.
+ virtual void Put(const std::string& key, SharedString* value) = 0;
+ virtual void Delete(const std::string& key) = 0;
+ virtual KeyState Query(const std::string& key) = 0;
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_CACHE_INTERFACE_H_
diff --git a/trunk/src/net/instaweb/util/public/cache_url_async_fetcher.h b/trunk/src/net/instaweb/util/public/cache_url_async_fetcher.h
new file mode 100644
index 0000000..59fa32c
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/cache_url_async_fetcher.h
@@ -0,0 +1,75 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_CACHE_URL_ASYNC_FETCHER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_CACHE_URL_ASYNC_FETCHER_H_
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/http_cache.h"
+#include <string>
+#include "net/instaweb/util/public/url_async_fetcher.h"
+
+namespace net_instaweb {
+
+class MessageHandler;
+class UrlAsyncFetcher;
+
+// Composes an asynchronous URL fetcher with an http cache, to
+// generate an asynchronous caching URL fetcher.
+//
+// This fetcher will call the callback immediately for entries in the
+// cache. When entries are not in the cache, it will initiate an
+// asynchronous 'get' and store the result in the cache, as well as
+// calling the passed-in callback.
+//
+// See also CacheUrlFetcher, which will returns results only for
+// URLs still in the cache.
+class CacheUrlAsyncFetcher : public UrlAsyncFetcher {
+ public:
+ CacheUrlAsyncFetcher(HTTPCache* cache, UrlAsyncFetcher* fetcher)
+ : http_cache_(cache),
+ fetcher_(fetcher),
+ force_caching_(false) {
+ }
+ virtual ~CacheUrlAsyncFetcher();
+
+ virtual bool StreamingFetch(
+ const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* fetched_content_writer,
+ MessageHandler* message_handler,
+ Callback* callback);
+
+ void set_force_caching(bool force) {
+ force_caching_ = force;
+ http_cache_->set_force_caching(force);
+ }
+
+ private:
+ HTTPCache* http_cache_;
+ UrlAsyncFetcher* fetcher_;
+ bool force_caching_;
+
+ DISALLOW_COPY_AND_ASSIGN(CacheUrlAsyncFetcher);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_CACHE_URL_ASYNC_FETCHER_H_
diff --git a/trunk/src/net/instaweb/util/public/cache_url_fetcher.h b/trunk/src/net/instaweb/util/public/cache_url_fetcher.h
new file mode 100644
index 0000000..edd9059
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/cache_url_fetcher.h
@@ -0,0 +1,133 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_CACHE_URL_FETCHER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_CACHE_URL_FETCHER_H_
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/http_cache.h"
+#include "net/instaweb/util/public/http_value.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+#include "net/instaweb/util/public/url_fetcher.h"
+
+namespace net_instaweb {
+
+class MessageHandler;
+class UrlAsyncFetcher;
+
+// Composes a URL fetcher with an http cache, to generate a caching
+// URL fetcher.
+//
+// This fetcher will return true and provide an immediate result for
+// entries in the cache. When entries are not in the cache, it will
+// initiate an asynchronous 'get' and store the result in the cache.
+//
+// When the supplied url fetcher indicates that the entry is not
+// cacheable, then we cache the fact that the resource is not cacheable
+// for 5 minutes, and during that time we will not request the element
+// again, but instead will return false from StreamingUrlFetch. This
+// will allow us to quickly punt on rewrites for non-cacheable items.
+//
+// TODO(jmarantz): allow experimental adjustment of the timeout
+// period, and whether or not we will retain resources marked as
+// cache-control:private.
+//
+// See also CacheUrlAsyncFetcher, which will yield its results asynchronously
+// for elements not in the cache, and immediately for results that are.
+class CacheUrlFetcher : public UrlFetcher {
+ public:
+ CacheUrlFetcher(HTTPCache* cache, UrlFetcher* fetcher)
+ : http_cache_(cache),
+ sync_fetcher_(fetcher),
+ async_fetcher_(NULL),
+ force_caching_(false) {
+ }
+ CacheUrlFetcher(HTTPCache* cache, UrlAsyncFetcher* fetcher)
+ : http_cache_(cache),
+ sync_fetcher_(NULL),
+ async_fetcher_(fetcher),
+ force_caching_(false) {
+ }
+ virtual ~CacheUrlFetcher();
+
+ virtual bool StreamingFetchUrl(
+ const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* fetched_content_writer,
+ MessageHandler* message_handler);
+
+ // Helper class to hold state for a single asynchronous fetch. When
+ // the fetch is complete, we'll put the resource in the cache.
+ //
+ // This class is declared here to facilitate code-sharing with
+ // CacheAsyncUrlFetcher.
+ class AsyncFetch : public UrlAsyncFetcher::Callback {
+ public:
+ AsyncFetch(const StringPiece& url, HTTPCache* cache,
+ MessageHandler* handler, bool force_caching);
+ virtual ~AsyncFetch();
+
+ virtual void Done(bool success);
+ void Start(UrlAsyncFetcher* fetcher, const MetaData& request_headers);
+
+ // This hook allows the CacheUrlAsyncFetcher to capture the headers for
+ // its client, while still enabling this class to cache them.
+ virtual MetaData* ResponseHeaders() = 0;
+
+ void UpdateCache();
+
+ virtual bool EnableThreaded() const;
+
+ protected:
+ HTTPValue value_;
+ MessageHandler* message_handler_;
+
+ private:
+ std::string url_;
+ HTTPCache* http_cache_;
+ bool force_caching_;
+ };
+
+ void set_force_caching(bool force) {
+ force_caching_ = force;
+ http_cache_->set_force_caching(force);
+ }
+
+ // Determines whether the HTTP response headers indicate a response that
+ // we have cached to indicate that the response is not cacheable. This
+ // is in this header file so the functionality can be shared between
+ // CacheUrlAsyncFetcher and CacheUrlFetcher.
+ static bool RememberNotCached(const MetaData& headers);
+
+ private:
+ HTTPCache* http_cache_;
+ UrlFetcher* sync_fetcher_;
+ UrlAsyncFetcher* async_fetcher_;
+ bool force_caching_;
+
+ DISALLOW_COPY_AND_ASSIGN(CacheUrlFetcher);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_CACHE_URL_FETCHER_H_
diff --git a/trunk/src/net/instaweb/util/public/content_type.h b/trunk/src/net/instaweb/util/public/content_type.h
new file mode 100644
index 0000000..0b9c9b2
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/content_type.h
@@ -0,0 +1,84 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+//
+// A collection of content-types and their attributes.
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_CONTENT_TYPE_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_CONTENT_TYPE_H_
+
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+struct ContentType {
+ public:
+ enum Type {
+ kHtml,
+ kXhtml,
+ kCeHtml, // See http://en.wikipedia.org/wiki/CE-HTML
+ kJavascript,
+ kCss,
+ kText,
+ kXml,
+ kPng,
+ kGif,
+ kJpeg,
+ };
+
+ const char* mime_type() const { return mime_type_; }
+ const char* file_extension() const { return file_extension_; }
+ Type type() const { return type_; }
+
+ // Return true iff this content type is HTML, or XHTML, or some other such
+ // thing (e.g. CE-HTML) that we can rewrite.
+ bool IsHtmlLike() const;
+
+ // Return true iff this content type is XML of some kind (either XHTML or
+ // some other XML).
+ bool IsXmlLike() const;
+
+ // These fields should be private; we leave them public only so we can use
+ // struct literals in content_type.cc. Other code should use the above
+ // accessor methods instead of accessing these fields directly.
+ const char* mime_type_;
+ const char* file_extension_; // includes ".", e.g. ".ext"
+ Type type_;
+};
+
+// HTML-like (i.e. rewritable) text:
+extern const ContentType& kContentTypeHtml;
+extern const ContentType& kContentTypeXhtml;
+extern const ContentType& kContentTypeCeHtml;
+// Other text:
+extern const ContentType& kContentTypeJavascript;
+extern const ContentType& kContentTypeCss;
+extern const ContentType& kContentTypeText;
+extern const ContentType& kContentTypeXml;
+// Images:
+extern const ContentType& kContentTypePng;
+extern const ContentType& kContentTypeGif;
+extern const ContentType& kContentTypeJpeg;
+
+// Given a name (file or url), see if it has the canonical extension
+// corresponding to a particular content type.
+const ContentType* NameExtensionToContentType(const StringPiece& name);
+const ContentType* MimeTypeToContentType(const StringPiece& mime_type);
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_CONTENT_TYPE_H_
diff --git a/trunk/src/net/instaweb/util/public/data_url.h b/trunk/src/net/instaweb/util/public/data_url.h
new file mode 100644
index 0000000..1f29f3c
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/data_url.h
@@ -0,0 +1,64 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_DATA_URL_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_DATA_URL_H_
+
+#include "net/instaweb/util/public/content_type.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+enum Encoding {
+ UNKNOWN, // Used only for output of ParseDataUrl.
+ BASE64,
+// LATIN1, // TODO(jmaessen): implement non-BASE64 encodings.
+// UTF8,
+ PLAIN
+};
+
+// Create a data: url from the given content-type and content. See:
+// http://en.wikipedia.org/wiki/Data_URI_scheme
+//
+// The ENCODING indicates how to encode the content; for binary data
+// this is UTF8, for ascii / Latin1 it's LATIN1. If you have ascii
+// without high bits or NULs, use LATIN1. If you have alphanumeric data,
+// use PLAIN (which doesn't encode at all).
+//
+// Note in particular that IE<=7 does not support this, so it makes us
+// UserAgent-dependent. It also pretty much requires outgoing content to be
+// compressed as we tend to base64-encode the content.
+void DataUrl(const ContentType& content_type, const Encoding encoding,
+ const StringPiece& content, std::string* result);
+
+// Dismantle a data: url into its component pieces, but do not decode the
+// content. Note that encoded_content will be a substring of the input url and
+// shares its lifetime. Invalidates all outputs if url does not parse.
+bool ParseDataUrl(const StringPiece& url,
+ const ContentType** content_type,
+ Encoding* encoding,
+ StringPiece* encoded_content);
+
+bool DecodeDataUrlContent(Encoding encoding,
+ const StringPiece& encoded_content,
+ std::string* decoded_content);
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_DATA_URL_H_
diff --git a/trunk/src/net/instaweb/util/public/dense_hash_set.h b/trunk/src/net/instaweb/util/public/dense_hash_set.h
new file mode 100644
index 0000000..a7cd887
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/dense_hash_set.h
@@ -0,0 +1,24 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_DENSE_HASH_SET_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_DENSE_HASH_SET_H_
+
+#include "google/dense_hash_set"
+using namespace google;
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_DENSE_HASH_SET_H_
diff --git a/trunk/src/net/instaweb/util/public/dummy_url_fetcher.h b/trunk/src/net/instaweb/util/public/dummy_url_fetcher.h
new file mode 100644
index 0000000..f7fc61b
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/dummy_url_fetcher.h
@@ -0,0 +1,41 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+//
+// Dummy implementation that aborts if used (useful for tests).
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_DUMMY_URL_FETCHER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_DUMMY_URL_FETCHER_H_
+
+#include "net/instaweb/util/public/url_fetcher.h"
+
+namespace net_instaweb {
+
+class MessageHandler;
+
+class DummyUrlFetcher : public UrlFetcher {
+ public:
+ virtual bool StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* fetched_content_writer,
+ MessageHandler* message_handler);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_DUMMY_URL_FETCHER_H_
diff --git a/trunk/src/net/instaweb/util/public/fake_url_async_fetcher.h b/trunk/src/net/instaweb/util/public/fake_url_async_fetcher.h
new file mode 100644
index 0000000..cad09a0
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/fake_url_async_fetcher.h
@@ -0,0 +1,64 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+//
+// UrlFetcher is an interface for asynchronously fetching urls. The
+// caller must supply a callback to be called when the fetch is complete.
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_FAKE_URL_ASYNC_FETCHER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_FAKE_URL_ASYNC_FETCHER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+#include "net/instaweb/util/public/url_fetcher.h"
+
+namespace net_instaweb {
+
+// Constructs an async fetcher using a synchronous fetcher, blocking
+// on a fetch and then the 'done' callback directly. It's also
+// possible to construct a real async interface using a synchronous
+// fetcher in a thread, but this does not do that: it blocks.
+//
+// This is intended for functional regression tests only.
+class FakeUrlAsyncFetcher : public UrlAsyncFetcher {
+ public:
+ explicit FakeUrlAsyncFetcher(UrlFetcher* url_fetcher)
+ : url_fetcher_(url_fetcher) {
+ }
+ virtual ~FakeUrlAsyncFetcher();
+
+ virtual bool StreamingFetch(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* writer,
+ MessageHandler* handler,
+ Callback* callback) {
+ bool ret = url_fetcher_->StreamingFetchUrl(
+ url, request_headers, response_headers, writer, handler);
+ callback->Done(ret);
+ return true;
+ }
+
+ private:
+ UrlFetcher* url_fetcher_;
+
+ DISALLOW_COPY_AND_ASSIGN(FakeUrlAsyncFetcher);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_FAKE_URL_ASYNC_FETCHER_H_
diff --git a/trunk/src/net/instaweb/util/public/file_cache.h b/trunk/src/net/instaweb/util/public/file_cache.h
new file mode 100644
index 0000000..4f472b3
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/file_cache.h
@@ -0,0 +1,86 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: lsong@google.com (Libo Song)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_FILE_CACHE_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_FILE_CACHE_H_
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/cache_interface.h"
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/filename_encoder.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include <string>
+#include "net/instaweb/util/public/timer.h"
+
+namespace net_instaweb {
+
+class Timer;
+
+// Simple C++ implementation of file cache.
+class FileCache : public CacheInterface {
+ public:
+
+ struct CachePolicy {
+ CachePolicy(Timer* timer, int64 clean_interval_ms, int64 target_size)
+ : timer(timer), clean_interval_ms(clean_interval_ms),
+ target_size(target_size) {}
+ const Timer* timer;
+ const int64 clean_interval_ms;
+ const int64 target_size;
+ private:
+ DISALLOW_COPY_AND_ASSIGN(CachePolicy);
+ };
+
+ FileCache(const std::string& path, FileSystem* file_system,
+ FilenameEncoder* filename_encoder, CachePolicy* policy,
+ MessageHandler* handler);
+ virtual ~FileCache();
+
+ virtual bool Get(const std::string& key, SharedString* value);
+ virtual void Put(const std::string& key, SharedString* value);
+ virtual void Delete(const std::string& key);
+ virtual KeyState Query(const std::string& key);
+ // Attempts to clean the cache. Returns false if we failed and the
+ // cache still needs to be cleaned. Returns true if everything's
+ // fine. This may take a while. It's OK for others to write and
+ // read from the cache while this is going on, but try to avoid
+ // Cleaning from two threads at the same time.
+ bool Clean(int64 target_size);
+ // Check to see if it's time to clean the cache, and if so start
+ // cleaning. Return true if we cleaned, false if we didn't.
+ bool CheckClean();
+ private:
+ bool EncodeFilename(const std::string& key, std::string* filename);
+
+ std::string path_;
+ FileSystem* file_system_;
+ FilenameEncoder* filename_encoder_;
+ MessageHandler* message_handler_;
+ const scoped_ptr<CachePolicy> cache_policy_;
+ int64 next_clean_ms_;
+ // The file where we keep the next scheduled cleanup time in seconds.
+ static const char kCleanTimeName[];
+ // The name of the global mutex protecting reads and writes to that file.
+ static const char kCleanLockName[];
+ DISALLOW_COPY_AND_ASSIGN(FileCache);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_FILE_CACHE_H_
diff --git a/trunk/src/net/instaweb/util/public/file_message_handler.h b/trunk/src/net/instaweb/util/public/file_message_handler.h
new file mode 100644
index 0000000..56534ce
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/file_message_handler.h
@@ -0,0 +1,48 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_FILE_MESSAGE_HANDLER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_FILE_MESSAGE_HANDLER_H_
+
+#include <stdio.h>
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/message_handler.h"
+
+namespace net_instaweb {
+
+// Message handler implementation for directing all error and
+// warning messages to a file.
+class FileMessageHandler : public MessageHandler {
+ public:
+ explicit FileMessageHandler(FILE* file);
+
+ protected:
+ virtual void MessageVImpl(MessageType type, const char* msg, va_list args);
+
+ virtual void FileMessageVImpl(MessageType type, const char* filename,
+ int line, const char* msg, va_list args);
+
+ private:
+ FILE* file_;
+
+ DISALLOW_COPY_AND_ASSIGN(FileMessageHandler);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_FILE_MESSAGE_HANDLER_H_
diff --git a/trunk/src/net/instaweb/util/public/file_system.h b/trunk/src/net/instaweb/util/public/file_system.h
new file mode 100644
index 0000000..ea91235
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/file_system.h
@@ -0,0 +1,263 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_FILE_SYSTEM_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_FILE_SYSTEM_H_
+
+#include "base/basictypes.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+// Three-way return type for distinguishing Errors from boolean answer.
+//
+// This is physically just an enum, but is wrapped in a class to prevent
+// accidental usage in an if- or ternary-condition without explicitly indicating
+// whether you are looking for true, false, or error.
+class BoolOrError {
+ enum Choice {
+ kIsFalse,
+ kIsTrue,
+ kIsError
+ };
+
+ public:
+ BoolOrError() : choice_(kIsError) { }
+ explicit BoolOrError(bool t_or_f) : choice_(t_or_f ? kIsTrue : kIsFalse) { }
+
+ // Intended to be passed by value; explicitly support copy & assign
+ BoolOrError(const BoolOrError& src) : choice_(src.choice_) { }
+ BoolOrError& operator=(const BoolOrError& src) {
+ if (&src != this) {
+ choice_ = src.choice_;
+ }
+ return *this;
+ }
+
+ bool is_false() const { return choice_ == kIsFalse; }
+ bool is_true() const { return choice_ == kIsTrue; }
+ bool is_error() const { return choice_ == kIsError; }
+ void set_error() { choice_ = kIsError; }
+ void set(bool t_or_f) { choice_ = t_or_f ? kIsTrue : kIsFalse; }
+
+ private:
+ Choice choice_;
+};
+
+class MessageHandler;
+class Writer;
+
+// Provides abstract file system interface. This isolation layer helps us:
+// - write unit tests that don't test the physical filesystem via a
+// MemFileSystem.
+// - Eases integration with Apache, which has its own file system interface,
+// and this class can help serve as the glue.
+// - provides a speculative conduit to a database so we can store resources
+// in a place where multiple Apache servers can see them.
+class FileSystem {
+ public:
+ virtual ~FileSystem();
+
+ class File {
+ public:
+ virtual ~File();
+
+ // Gets the name of the file.
+ virtual const char* filename() = 0;
+
+ protected:
+ // Use public interface provided by FileSystem::Close.
+ friend class FileSystem;
+ virtual bool Close(MessageHandler* handler) = 0;
+ };
+
+ class InputFile : public File {
+ public:
+ // TODO(sligocki): Perhaps this should be renamed to avoid confusing
+ // that it returns a bool to indicate success like all other Read methods
+ // in our codebase.
+ virtual int Read(char* buf, int size, MessageHandler* handler) = 0;
+
+ protected:
+ friend class FileSystem;
+ virtual ~InputFile();
+ };
+
+ class OutputFile : public File {
+ public:
+ // Note: Write is not atomic. If Write fails, there is no indication of how
+ // much data has already been written to the file.
+ //
+ // TODO(sligocki): Would we like a version that returns the amound written?
+ // If so, it should be named so that it is clear it is returning int.
+ virtual bool Write(const StringPiece& buf, MessageHandler* handler) = 0;
+ virtual bool Flush(MessageHandler* handler) = 0;
+ virtual bool SetWorldReadable(MessageHandler* handler) = 0;
+
+ protected:
+ friend class FileSystem;
+ virtual ~OutputFile();
+ };
+
+ // High level support to read/write entire files in one shot.
+ virtual bool ReadFile(const char* filename,
+ std::string* buffer,
+ MessageHandler* handler);
+ virtual bool ReadFile(const char* filename,
+ Writer* writer,
+ MessageHandler* handler);
+ virtual bool WriteFile(const char* filename,
+ const StringPiece& buffer,
+ MessageHandler* handler);
+ // Writes given data to a temp file in one shot, storing the filename
+ // in filename on success. Returns false and clears filename on failure.
+ virtual bool WriteTempFile(const StringPiece& prefix_name,
+ const StringPiece& buffer,
+ std::string* filename,
+ MessageHandler* handler);
+
+ virtual InputFile* OpenInputFile(const char* filename,
+ MessageHandler* handler) = 0;
+ // Automatically creates sub-directories to filename.
+ OutputFile* OpenOutputFile(const char* filename,
+ MessageHandler* handler) {
+ SetupFileDir(filename, handler);
+ return OpenOutputFileHelper(filename, handler);
+ }
+ // Opens a temporary file to write, with the specified prefix.
+ // If successful, the filename can be obtained from File::filename().
+ // Automatically creates sub-directories to filename.
+ //
+ // NULL is returned on failure.
+ OutputFile* OpenTempFile(const StringPiece& prefix_name,
+ MessageHandler* handler) {
+ SetupFileDir(prefix_name, handler);
+ return OpenTempFileHelper(prefix_name, handler);
+ }
+
+ // Closes the File and cleans up memory.
+ virtual bool Close(File* file, MessageHandler* handler);
+
+
+ // Like POSIX 'rm'.
+ virtual bool RemoveFile(const char* filename, MessageHandler* handler) = 0;
+
+ // Like POSIX 'mv', except it automatically creates sub-directories for
+ // new_filename.
+ bool RenameFile(const char* old_filename, const char* new_filename,
+ MessageHandler* handler) {
+ SetupFileDir(new_filename, handler);
+ return RenameFileHelper(old_filename, new_filename, handler);
+ }
+
+ // Like POSIX 'mkdir', makes a directory only if parent directory exists.
+ // Fails if directory_name already exists or parent directory doesn't exist.
+ virtual bool MakeDir(const char* directory_path, MessageHandler* handler) = 0;
+
+ // Like POSIX 'test -e', checks if path exists (is a file, directory, etc.).
+ virtual BoolOrError Exists(const char* path, MessageHandler* handler) = 0;
+
+ // Like POSIX 'test -d', checks if path exists and refers to a directory.
+ virtual BoolOrError IsDir(const char* path, MessageHandler* handler) = 0;
+
+ // Like POSIX 'mkdir -p', makes all directories up to this one recursively.
+ // Fails if we do not have permission to make any directory in chain.
+ virtual bool RecursivelyMakeDir(const StringPiece& directory_path,
+ MessageHandler* handler);
+
+ // Like POSIX 'ls -a', lists all files and directories under the given
+ // directory (but omits "." and ".."). Full paths (not just filenames) will
+ // be pushed onto the back of the supplied vector (without clearing it).
+ // Returns true on success (even if the dir was empty), false on error (even
+ // if some files were pushed onto the vector). This is generally not
+ // threadsafe! Use a mutex.
+ virtual bool ListContents(const StringPiece& dir, StringVector* files,
+ MessageHandler* handler) = 0;
+
+ // Stores in *timestamp_sec the timestamp (in seconds since the
+ // epoch) of the last time the file was accessed (through one of our
+ // Read methods, or by someone else accessing the filesystem
+ // directly). Returns true on success, false on failure.
+ // TODO(abliss): replace this with a single Stat() function.
+ virtual bool Atime(const StringPiece& path, int64* timestamp_sec,
+ MessageHandler* handler) = 0;
+
+ // Given a directory, recursively computes the total size of all its
+ // files and directories, and increments *size by the sum total. We
+ // assume no circular links. Returns true on success, false on
+ // failure. If the files are modified while we traverse, we are not
+ // guaranteed to represent their final state.
+ // The path name should NOT end in a "/".
+ // TODO(abliss): unify all slash-ending assumptions
+ virtual bool RecursiveDirSize(const StringPiece& path, int64* size,
+ MessageHandler* handler);
+
+ // Given a file, computes its size in bytes and store it in *size. Returns
+ // true on success, false on failure. Behavior is undefined if path refers to
+ // a directory.
+ // TODO(abliss): replace this with a single Stat() function.
+ virtual bool Size(const StringPiece& path, int64* size,
+ MessageHandler* handler) = 0;
+
+ // Attempts to obtain a global (cross-process, cross-thread) lock of the given
+ // name (which should be a valid filename, not otherwise used, in an extant
+ // directory). If someone else has this lock, returns False immediately. If
+ // anything goes wrong, returns Error. On success, returns True: then you
+ // must call Unlock when you are done.
+ virtual BoolOrError TryLock(const StringPiece& lock_name,
+ MessageHandler* handler) = 0;
+
+ // Like TryLock, but may attempt to break the lock if it appears to be staler
+ // than the given number of milliseconds. (The default implementation never
+ // actually breaks locks.) If you obtain a lock through this method, there
+ // are no hard guarantees that nobody else has it too.
+ // <blink> If you use this function, your lock becomes "best-effort". </blink>
+ virtual BoolOrError TryLockWithTimeout(const StringPiece& lock_name,
+ int64 timeout_millis,
+ MessageHandler* handler) {
+ return TryLock(lock_name, handler);
+ }
+
+ // Attempts to release a lock previously obtained through TryLock. If your
+ // thread did not prevously obtain the lock, the behavior is undefined.
+ // Returns true if we successfully release the lock. Returns false if we were
+ // unable to release the lock (e.g. somebody came along and write-protected
+ // the lockfile). You might try again, or start using a different lock name.
+ virtual bool Unlock(const StringPiece& lock_name,
+ MessageHandler* handler) = 0;
+
+ protected:
+ // These interfaces must be defined by implementers of FileSystem.
+ // They may assume the directory already exists.
+ virtual OutputFile* OpenOutputFileHelper(const char* filename,
+ MessageHandler* handler) = 0;
+ virtual OutputFile* OpenTempFileHelper(const StringPiece& filename,
+ MessageHandler* handler) = 0;
+ virtual bool RenameFileHelper(const char* old_filename,
+ const char* new_filename,
+ MessageHandler* handler) = 0;
+
+ private:
+ // RecursiveMakeDir the directory needed for filename.
+ void SetupFileDir(const StringPiece& filename, MessageHandler* handler);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_FILE_SYSTEM_H_
diff --git a/trunk/src/net/instaweb/util/public/file_system_lock_manager.h b/trunk/src/net/instaweb/util/public/file_system_lock_manager.h
new file mode 100644
index 0000000..a9c2746
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/file_system_lock_manager.h
@@ -0,0 +1,75 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_FILE_SYSTEM_LOCK_MANAGER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_FILE_SYSTEM_LOCK_MANAGER_H_
+
+#include "net/instaweb/util/public/named_lock_manager.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class FileSystem;
+class Timer;
+class MessageHandler;
+
+// Use the locking routines in FileSystem to implement named locks. Requires a
+// Timer as well because the FileSystem locks are non-blocking and we must use
+// spin+sleep. A MessageHandler is used to report file system errors during
+// lock creation and cleanup.
+class FileSystemLockManager : public NamedLockManager {
+ public:
+ // Note: a FileSystemLockManager must outlive
+ // any and all locks that it creates.
+ // It does not assume ownership of the passed-in
+ // constructor arguments.
+ FileSystemLockManager(FileSystem* file_system,
+ Timer* timer,
+ MessageHandler* handler)
+ : file_system_(file_system),
+ timer_(timer),
+ handler_(handler) { }
+ virtual ~FileSystemLockManager();
+
+ // Multiple lock objects with the same name will manage the same underlying
+ // lock. Lock names must be legal file names according to file_system.
+ //
+ // A lock created by CreateNamedLock will be Unlocked when it is destructed if
+ // the AbstractLock object appears to still be locked at destruction time.
+ // This attempts to ensure that the file system is not littered with the
+ // remnants of dead locks. A given AbstractLock object should Lock and Unlock
+ // in matched pairs; DO NOT use separate AbstractLock objects created with the
+ // same name to perform a Lock and the corresponding Unlock.
+ virtual AbstractLock* CreateNamedLock(const StringPiece& name);
+
+ // Simple accessors for constructor arguments
+ FileSystem* file_system() const { return file_system_; }
+ Timer* timer() const { return timer_; }
+ MessageHandler* handler() const { return handler_; }
+
+ private:
+ FileSystem* file_system_;
+ Timer* timer_;
+ MessageHandler* handler_;
+
+ DISALLOW_COPY_AND_ASSIGN(FileSystemLockManager);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_FILE_SYSTEM_LOCK_MANAGER_H_
diff --git a/trunk/src/net/instaweb/util/public/file_system_test.h b/trunk/src/net/instaweb/util/public/file_system_test.h
new file mode 100644
index 0000000..3d8ea82
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/file_system_test.h
@@ -0,0 +1,82 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: abliss@google.com (Adam Bliss)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_FILE_SYSTEM_TEST_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_FILE_SYSTEM_TEST_H_
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/gtest.h"
+#include <string>
+
+namespace net_instaweb {
+
+// Base class for testing a FileSystem implementation. Subclasses
+// must implement DeleteRecursively and GetFileSystem, then should
+// create their own tests calling each of our Test* methods.
+class FileSystemTest : public testing::Test {
+ protected:
+ FileSystemTest();
+ virtual ~FileSystemTest();
+
+ void CheckDoesNotExist(const std::string& filename);
+
+ void CheckRead(const std::string& filename,
+ const std::string& expected_contents);
+
+ // Delete (at least) the named file or directory and everything
+ // underneath it. The test is permitted to delete more things (up
+ // to and including the entire file system).
+ virtual void DeleteRecursively(const StringPiece& filename) = 0;
+
+ // Provide a pointer to your favorite filesystem implementation.
+ virtual FileSystem* file_system() = 0;
+
+ // Provide a temporary directory for tests to put files in.
+ virtual std::string test_tmpdir() = 0;
+
+ std::string WriteNewFile(const StringPiece& suffix,
+ const std::string& content);
+
+ void TestWriteRead();
+ void TestTemp();
+ void TestRename();
+ void TestRemove();
+ void TestExists();
+ void TestCreateFileInDir();
+ void TestMakeDir();
+ void TestIsDir();
+ void TestRecursivelyMakeDir();
+ void TestRecursivelyMakeDir_NoPermission();
+ void TestRecursivelyMakeDir_FileInPath();
+ void TestListContents();
+ void TestAtime();
+ void TestSize();
+ void TestLock();
+
+ GoogleMessageHandler handler_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(FileSystemTest);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_FILE_SYSTEM_TEST_H_
diff --git a/trunk/src/net/instaweb/util/public/file_writer.h b/trunk/src/net/instaweb/util/public/file_writer.h
new file mode 100644
index 0000000..adac186
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/file_writer.h
@@ -0,0 +1,43 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_FILE_WRITER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_FILE_WRITER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/writer.h"
+
+namespace net_instaweb {
+
+// Writer implementation for directing HTML output to a file.
+class FileWriter : public Writer {
+ public:
+ explicit FileWriter(FileSystem::OutputFile* f) : file_(f) { }
+ virtual ~FileWriter();
+ virtual bool Write(const StringPiece& str, MessageHandler* message_handler);
+ virtual bool Flush(MessageHandler* message_handler);
+ private:
+ FileSystem::OutputFile* file_;
+
+ DISALLOW_COPY_AND_ASSIGN(FileWriter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_FILE_WRITER_H_
diff --git a/trunk/src/net/instaweb/util/public/filename_encoder.h b/trunk/src/net/instaweb/util/public/filename_encoder.h
new file mode 100644
index 0000000..742efb7
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/filename_encoder.h
@@ -0,0 +1,41 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_FILENAME_ENCODER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_FILENAME_ENCODER_H_
+
+#include "net/instaweb/util/public/string_util.h"
+#include "net/tools/dump_cache/url_to_filename_encoder.h"
+namespace net_instaweb {
+
+class FilenameEncoder {
+ public:
+ FilenameEncoder() {}
+ ~FilenameEncoder();
+
+ void Encode(const StringPiece& filename_prefix,
+ const StringPiece& filename_ending,
+ std::string* encoded_filename);
+
+ bool Decode(const StringPiece& encoded_filename,
+ std::string* decoded_url);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_FILENAME_ENCODER_H_
diff --git a/trunk/src/net/instaweb/util/public/google_message_handler.h b/trunk/src/net/instaweb/util/public/google_message_handler.h
new file mode 100644
index 0000000..7a15333
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/google_message_handler.h
@@ -0,0 +1,48 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_GOOGLE_MESSAGE_HANDLER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_GOOGLE_MESSAGE_HANDLER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include <string>
+
+namespace net_instaweb {
+
+// Implementation of an HTML parser message handler that uses Google
+// logging to emit messsages.
+class GoogleMessageHandler : public MessageHandler {
+ public:
+ GoogleMessageHandler() { }
+
+ protected:
+ virtual void MessageVImpl(MessageType type, const char* msg, va_list args);
+
+ virtual void FileMessageVImpl(MessageType type, const char* filename,
+ int line, const char* msg, va_list args);
+
+ private:
+ std::string Format(const char* msg, va_list args);
+
+ DISALLOW_COPY_AND_ASSIGN(GoogleMessageHandler);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_GOOGLE_MESSAGE_HANDLER_H_
diff --git a/trunk/src/net/instaweb/util/public/google_url.h b/trunk/src/net/instaweb/util/public/google_url.h
new file mode 100644
index 0000000..0c8c5e7
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/google_url.h
@@ -0,0 +1,83 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_GOOGLE_URL_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_GOOGLE_URL_H_
+
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+
+#include "googleurl/src/gurl.h"
+
+namespace net_instaweb {
+
+namespace GoogleUrl {
+
+// Helper functions around GURL to make it a little easier to use
+inline std::string Spec(const GURL& gurl) { return gurl.spec(); }
+
+// Makes a GURL object from a StringPiece.
+inline GURL Create(const StringPiece& sp) { return GURL(sp.as_string()); }
+
+// Makes a GURL object from a string.
+inline GURL Create(const std::string& str) {
+ return GURL(str);
+}
+
+// Resolves a GURL object and a new path into a new GURL.
+inline GURL Resolve(const GURL& gurl, const StringPiece& sp) {
+ return gurl.Resolve(sp.as_string()); }
+
+// Resolves a GURL object and a new path into a new GURL.
+inline GURL Resolve(const GURL& gurl, const std::string& str) {
+ return gurl.Resolve(str);
+}
+
+// Resolves a GURL object and a new path into a new GURL.
+inline GURL Resolve(const GURL& gurl, const char* str) {
+ return gurl.Resolve(str);
+}
+
+// For "http://a.com/b/c/d?e=f/g returns "http://a/b/c/",
+// including trailing slash.
+std::string AllExceptLeaf(const GURL& gurl);
+
+// For "http://a.com/b/c/d?e=f/g returns "d?e=f/g", omitting leading slash.
+std::string Leaf(const GURL& gurl);
+
+// For "http://a.com/b/c/d?e=f/g returns "http://a.com" without trailing slash
+std::string Origin(const GURL& gurl);
+
+// For "http://a.com/b/c/d?E=f/g returns "/b/c/d?e=f/g" including leading slash
+std::string PathAndLeaf(const GURL& gurl);
+
+// For "http://a.com/b/c/d?E=f/g returns "/b/c/d" including leading slash
+inline std::string Path(const GURL& gurl) {
+ return gurl.path();
+}
+
+// For "http://a.com/b/c/d?e=f/g returns "/b/c/d/" including leading and
+// trailing slashes.
+std::string PathSansLeaf(const GURL& gurl);
+
+} // namespace GoogleUrl
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_GOOGLE_URL_H_
diff --git a/trunk/src/net/instaweb/util/public/gtest.h b/trunk/src/net/instaweb/util/public/gtest.h
new file mode 100644
index 0000000..c7c2c20
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/gtest.h
@@ -0,0 +1,35 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+//
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_GTEST_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_GTEST_H_
+
+#include <vector>
+
+#include <string>
+#include "gtest/gtest.h"
+
+namespace net_instaweb {
+
+std::string GTestSrcDir();
+std::string GTestTempDir();
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_GTEST_H_
diff --git a/trunk/src/net/instaweb/util/public/gzip_inflater.h b/trunk/src/net/instaweb/util/public/gzip_inflater.h
new file mode 100644
index 0000000..00a302c
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/gzip_inflater.h
@@ -0,0 +1,77 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_GZIP_INFLATER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_GZIP_INFLATER_H_
+
+#include <string.h>
+#include "base/basictypes.h"
+
+typedef struct z_stream_s z_stream;
+
+namespace net_instaweb {
+
+class GzipInflater {
+ public:
+ enum InflateType {kGzip, kDeflate};
+
+ explicit GzipInflater(InflateType type);
+ ~GzipInflater();
+
+ // Should be called once, before inflating any data.
+ bool Init();
+
+ // Should be called once, after inflating is finished.
+ void ShutDown();
+
+ // Does the inflater still have input that has not yet been
+ // consumed? If true, the caller should call InflateBytes(). If
+ // false, the gzip inflater is ready for additional input.
+ bool HasUnconsumedInput() const;
+
+ // Pass a gzip-compressed buffer to the gzip inflater. The gzip
+ // inflater will inflate the buffer via InflateBytes(). SetInput
+ // should not be called if HasUnconsumedInput() is true, and the
+ // buffer passed into SetInput should not be modified by the caller
+ // until HasUnconsumedInput() returns false.
+ bool SetInput(const void *in, size_t in_size);
+
+ // Decompress the input passed in via SetInput. Should be called
+ // until HasUnconsumedInput returns false. Returns the number of
+ // bytes inflated, or -1 if an error was encountered while
+ // inflating.
+ int InflateBytes(char *buf, size_t buf_size);
+
+ // Has the entire input been inflated?
+ bool finished() const { return finished_; }
+
+ // Was an error encountered during inflating?
+ bool error() const { return error_; }
+
+ private:
+ void Free();
+
+ z_stream *zlib_;
+ bool finished_;
+ bool error_;
+ InflateType type_;
+
+ DISALLOW_COPY_AND_ASSIGN(GzipInflater);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_GZIP_INFLATER_H_
diff --git a/trunk/src/net/instaweb/util/public/hasher.h b/trunk/src/net/instaweb/util/public/hasher.h
new file mode 100644
index 0000000..ef54129
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/hasher.h
@@ -0,0 +1,47 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+//
+// Interface for a hash function.
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_HASHER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_HASHER_H_
+
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class Hasher {
+ public:
+ Hasher() { }
+ virtual ~Hasher();
+
+ // Interface to compute a hash of a single string. This
+ // operation is thread-safe.
+ virtual std::string Hash(const StringPiece& content) const = 0;
+
+ // Return string length of hashes produced by this hasher.
+ virtual int HashSizeInChars() const = 0;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(Hasher);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_HASHER_H_
diff --git a/trunk/src/net/instaweb/util/public/http_cache.h b/trunk/src/net/instaweb/util/public/http_cache.h
new file mode 100644
index 0000000..d9a2964
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/http_cache.h
@@ -0,0 +1,124 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_HTTP_CACHE_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_HTTP_CACHE_H_
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/cache_interface.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class CacheInterface;
+class HTTPValue;
+class MessageHandler;
+class MetaData;
+class Statistics;
+class Timer;
+class Variable;
+
+// Implements HTTP caching semantics, including cache expiration and
+// retention of the originally served cache headers.
+class HTTPCache {
+ public:
+ // Takes over ownership of the cache.
+ HTTPCache(CacheInterface* cache, Timer* timer)
+ : cache_(cache),
+ timer_(timer),
+ force_caching_(false),
+ cache_time_us_(NULL),
+ cache_hits_(NULL),
+ cache_misses_(NULL),
+ cache_expirations_(NULL),
+ cache_inserts_(NULL) {
+ }
+
+ ~HTTPCache();
+
+ // When a lookup is done in the HTTP Cache, it returns one of these values.
+ // 2 of these are obvious, one is used to help avoid frequently re-fetching
+ // the same content that failed to fetch, or was fetched but was not cacheable.
+ //
+ // TODO(jmarantz): consider merging these 3 into the 3 status codes defined
+ // CacheInterface, making 4 distinct codes. That would be a little clearer,
+ // but would require that all callers of Find handle kInTransit which no
+ // cache implementations currently generate.
+ enum FindResult {
+ kFound,
+ kRecentFetchFailedDoNotRefetch,
+ kNotFound
+ };
+
+ FindResult Find(const std::string& key, HTTPValue* value, MetaData* headers,
+ MessageHandler* handler);
+
+ // Note that Put takes a non-const pointer for HTTPValue so it can
+ // bump the reference count.
+ void Put(const std::string& key, HTTPValue* value, MessageHandler* handler);
+ void Put(const std::string& key, const MetaData& headers,
+ const StringPiece& content, MessageHandler* handler);
+
+ CacheInterface::KeyState Query(const std::string& key);
+ void Delete(const std::string& key);
+
+ void set_force_caching(bool force) { force_caching_ = force; }
+ bool force_caching() const { return force_caching_; }
+ Timer* timer() const { return timer_; }
+
+ // Initializes statistics for the cache (time, hits, misses, expirations)
+ void SetStatistics(Statistics* stats);
+
+ // Tell the HTTP Cache to remember that a particular key is not cacheable.
+ // This may be due to the associated URL failing Fetch, or it may be because
+ // the URL was fetched but was marked with Cache-Control 'nocache' or
+ // Cache-Control 'private'. In any case we would like to avoid DOSing
+ // the origin server or spinning our own wheels trying to re-fetch this
+ // resource.
+ //
+ // The not-cacheable setting will be 'remembered' for 5 minutes -- currently
+ // hard-coded in http_cache.cc.
+ //
+ // TODO(jmarantz): if fetch failed, maybe we should try back soon,
+ // but if it is Cache-Control: private, we can probably assume that
+ // it still will be in 5 minutes.
+ void RememberNotCacheable(const std::string& key, MessageHandler * handler);
+
+ // Initialize statistics variables for the cache
+ static void Initialize(Statistics* statistics);
+
+ private:
+ bool IsCurrentlyValid(const MetaData& headers, int64 now_ms);
+
+ scoped_ptr<CacheInterface> cache_;
+ Timer* timer_;
+ bool force_caching_;
+ Variable* cache_time_us_;
+ Variable* cache_hits_;
+ Variable* cache_misses_;
+ Variable* cache_expirations_;
+ Variable* cache_inserts_;
+
+ DISALLOW_COPY_AND_ASSIGN(HTTPCache);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_HTTP_CACHE_H_
diff --git a/trunk/src/net/instaweb/util/public/http_dump_url_async_writer.h b/trunk/src/net/instaweb/util/public/http_dump_url_async_writer.h
new file mode 100644
index 0000000..575c35e
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/http_dump_url_async_writer.h
@@ -0,0 +1,78 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_HTTP_DUMP_URL_ASYNC_WRITER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_HTTP_DUMP_URL_ASYNC_WRITER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/http_dump_url_fetcher.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+
+namespace net_instaweb {
+
+class FileSystem;
+
+// HttpDumpWriter checks to see whether the HTTP dump is available on the
+// filesystem. If not, it fetches it from another fetcher (e.g. one that
+// uses the network) and writes it to the filesystem so that HttpDumpFetcher
+// can find it.
+class HttpDumpUrlAsyncWriter : public UrlAsyncFetcher {
+ public:
+ HttpDumpUrlAsyncWriter(const StringPiece& root_dir,
+ UrlAsyncFetcher* base_fetcher,
+ FileSystem* file_system,
+ Timer* timer)
+ : dump_fetcher_(root_dir, file_system, timer),
+ base_fetcher_(base_fetcher),
+ file_system_(file_system),
+ accept_gzip_(true) {
+ root_dir.CopyToString(&root_dir_);
+ }
+ virtual ~HttpDumpUrlAsyncWriter();
+
+ // This is a synchronous/blocking implementation.
+ virtual bool StreamingFetch(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* message_handler,
+ Callback* callback);
+
+ // Controls whether we will request and save gzipped content to the
+ // file system. Note that http_dump_url_fetcher will inflate on
+ // read if its caller does not want gzipped output.
+ void set_accept_gzip(bool x) { accept_gzip_ = x; }
+
+ private:
+ // Helper class to manage individual fetchs.
+ class Fetch;
+
+ HttpDumpUrlFetcher dump_fetcher_;
+ // Used to fetch urls that aren't in the dump yet.
+ UrlAsyncFetcher* base_fetcher_;
+ std::string root_dir_; // Root directory of the HTTP dumps.
+ FileSystem* file_system_;
+ bool accept_gzip_;
+
+ DISALLOW_COPY_AND_ASSIGN(HttpDumpUrlAsyncWriter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_HTTP_DUMP_URL_ASYNC_WRITER_H_
diff --git a/trunk/src/net/instaweb/util/public/http_dump_url_fetcher.h b/trunk/src/net/instaweb/util/public/http_dump_url_fetcher.h
new file mode 100644
index 0000000..24c8c4b
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/http_dump_url_fetcher.h
@@ -0,0 +1,113 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_HTTP_DUMP_URL_FETCHER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_HTTP_DUMP_URL_FETCHER_H_
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/file_system.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/url_fetcher.h"
+
+class GURL;
+
+namespace net_instaweb {
+
+class Timer;
+
+// TODO(sligocki): Can we forward declare these somehow?
+// class FileSystem;
+// class FileSystem::InputFile;
+
+// HttpDumpFetcher fetches raw HTTP dumps from the filesystem.
+// These dumps could be compressed or chunked, the fetcher does not
+// decompress or de-chunk them.
+class HttpDumpUrlFetcher : public UrlFetcher {
+ public:
+
+ // When the slurped data is gzipped, but request headers are made
+ // that don't include 'gzip' in an Accept-Encodings header, then
+ // this fetcher inflates the gzipped output as it streams. It
+ // also captures the original gzipped size in this attribute in
+ // the response headers.
+ static const char kGzipContentLengthAttribute[];
+
+ HttpDumpUrlFetcher(const StringPiece& root_dir, FileSystem* file_system,
+ Timer* timer);
+ virtual ~HttpDumpUrlFetcher();
+
+ // Converts URL into filename the way that Latency Lab does.
+ // Note: root_dir_ must be standardized to have a / at end already.
+ static bool GetFilenameFromUrl(const StringPiece& root_dir,
+ const GURL& url,
+ std::string* filename,
+ MessageHandler* message_handler);
+
+ // Non-static version that uses the fetcher's root dir.
+ bool GetFilename(const GURL& url,
+ std::string* filename,
+ MessageHandler* message_handler) {
+ return GetFilenameFromUrl(root_dir_, url, filename, message_handler);
+ }
+
+ // Converts URL into filename prefix the way that Latency Lab does.
+ // Note: root_dir_ must be standardized to have a / at end already.
+ static bool GetFilenamePrefixFromUrl(const StringPiece& root_dir,
+ const GURL& url,
+ std::string* filename,
+ MessageHandler* message_handler);
+
+ // This is a synchronous/blocking implementation.
+ virtual bool StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* fetched_content_writer,
+ MessageHandler* message_handler);
+
+ // Parse file into response_headers and response_writer as if it were bytes
+ // off the wire.
+ bool ParseFile(FileSystem::InputFile* file,
+ MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* handler);
+
+ // Helper function to return a generic error response.
+ void RespondError(MetaData* response_headers, Writer* response_writer,
+ MessageHandler* handler);
+
+ // Print URLs each time they are fetched.
+ void set_print_urls(bool on);
+
+ private:
+ std::string root_dir_; // Root directory of the HTTP dumps.
+ FileSystem* file_system_;
+ Timer* timer_;
+
+ // Response to use if something goes wrong.
+ std::string error_body_;
+
+ scoped_ptr<StringSet> urls_;
+
+ DISALLOW_COPY_AND_ASSIGN(HttpDumpUrlFetcher);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_HTTP_DUMP_URL_FETCHER_H_
diff --git a/trunk/src/net/instaweb/util/public/http_dump_url_writer.h b/trunk/src/net/instaweb/util/public/http_dump_url_writer.h
new file mode 100644
index 0000000..819e320
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/http_dump_url_writer.h
@@ -0,0 +1,74 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_HTTP_DUMP_URL_WRITER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_HTTP_DUMP_URL_WRITER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/http_dump_url_fetcher.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/url_fetcher.h"
+
+namespace net_instaweb {
+
+class FileSystem;
+
+// HttpDumpWriter checks to see whether the HTTP dump is available on the
+// filesystem. If not, it fetches it from another fetcher (e.g. one that
+// uses the network) and writes it to the filesystem so that HttpDumpFetcher
+// can find it.
+class HttpDumpUrlWriter : public UrlFetcher {
+ public:
+ HttpDumpUrlWriter(const StringPiece& root_dir, UrlFetcher* base_fetcher,
+ FileSystem* file_system, Timer* timer)
+ : dump_fetcher_(root_dir, file_system, timer),
+ base_fetcher_(base_fetcher),
+ file_system_(file_system),
+ accept_gzip_(true) {
+ root_dir.CopyToString(&root_dir_);
+ }
+ virtual ~HttpDumpUrlWriter();
+
+ // This is a synchronous/blocking implementation.
+ virtual bool StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* message_handler);
+
+ // Controls whether we will request and save gzipped content to the
+ // file system. Note that http_dump_url_fetcher will inflate on
+ // read if its caller does not want gzipped output.
+ void set_accept_gzip(bool x) { accept_gzip_ = x; }
+
+ // Print URLs each time they are fetched.
+ void set_print_urls(bool on) { dump_fetcher_.set_print_urls(on); }
+
+ private:
+ HttpDumpUrlFetcher dump_fetcher_;
+ UrlFetcher* base_fetcher_; // Used to fetch urls that aren't in the dump yet.
+ std::string root_dir_; // Root directory of the HTTP dumps.
+ FileSystem* file_system_;
+ bool accept_gzip_;
+
+ DISALLOW_COPY_AND_ASSIGN(HttpDumpUrlWriter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_HTTP_DUMP_URL_WRITER_H_
diff --git a/trunk/src/net/instaweb/util/public/http_response_parser.h b/trunk/src/net/instaweb/util/public/http_response_parser.h
new file mode 100644
index 0000000..c8313ac
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/http_response_parser.h
@@ -0,0 +1,71 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_HTTP_RESPONSE_PARSER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_HTTP_RESPONSE_PARSER_H_
+
+#include "base/basictypes.h"
+// TODO(sligocki): Find a way to forward declare FileSystem::InputFile.
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class MessageHandler;
+class MetaData;
+class Writer;
+
+// Helper class to fascilitate parsing a raw streaming HTTP response including
+// headers and body.
+class HttpResponseParser {
+ public:
+ HttpResponseParser(MetaData* response_headers, Writer* writer,
+ MessageHandler* handler)
+ : reading_headers_(true),
+ ok_(true),
+ response_headers_(response_headers),
+ writer_(writer),
+ handler_(handler) {
+ }
+
+ // Parse complete HTTP response from a file.
+ bool ParseFile(FileSystem::InputFile* file);
+
+ // Parse complete HTTP response from a FILE stream.
+ // TODO(sligocki): We need a Readable abstraction (like Writer)
+ bool Parse(FILE* stream);
+
+ // Read a chunk of HTTP response, populating response_headers and call
+ // writer on output body, returning true if the status is ok.
+ bool ParseChunk(const StringPiece& data);
+
+ bool ok() const { return ok_; }
+
+ private:
+ bool reading_headers_;
+ bool ok_;
+ MetaData* response_headers_;
+ Writer* writer_;
+ MessageHandler* handler_;
+
+ DISALLOW_COPY_AND_ASSIGN(HttpResponseParser);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_HTTP_RESPONSE_PARSER_H_
diff --git a/trunk/src/net/instaweb/util/public/http_value.h b/trunk/src/net/instaweb/util/public/http_value.h
new file mode 100644
index 0000000..22679bb
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/http_value.h
@@ -0,0 +1,103 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_HTTP_VALUE_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_HTTP_VALUE_H_
+
+#include "net/instaweb/util/public/shared_string.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/writer.h"
+
+namespace net_instaweb {
+
+class MetaData;
+
+// Provides shared, ref-counted, copy-on-write storage for HTTP
+// contents, to aid sharing between active fetches and filters, and
+// the cache, which from which data may be evicted at any time.
+class HTTPValue : public Writer {
+ public:
+ HTTPValue() { }
+
+ // Clears the value (both headers and content)
+ void Clear();
+
+ // Is this HTTPValue empty
+ bool Empty() const { return storage_->empty(); }
+
+ // Sets the HTTP headers for this value. This method may only
+ // be called once and must be called before or after all of the
+ // contents are set (using the streaming interface Write).
+ //
+ // If Clear() is called, then SetHeaders() can be called once again.
+ void SetHeaders(const MetaData& headers);
+
+ // Writes contents into the HTTPValue object. Write can be called
+ // multiple times to append more data, and can be called before
+ // or after SetHeaders. However, SetHeaders cannot be interleaved
+ // in between calls to Write.
+ virtual bool Write(const StringPiece& str, MessageHandler* handler);
+ virtual bool Flush(MessageHandler* handler);
+
+ // Retrieves the headers, returning false if empty.
+ bool ExtractHeaders(MetaData* headers, MessageHandler* handler) const;
+
+ // Retrieves the contents, returning false if empty. Note that the
+ // contents are only guaranteed valid as long as the HTTPValue
+ // object is in scope.
+ bool ExtractContents(StringPiece* str) const;
+
+ // Explicit support for copy-construction and assignment operators
+ HTTPValue(const HTTPValue& src) : storage_(src.storage_) { }
+ HTTPValue& operator=(const HTTPValue& src) {
+ if (&src != this) {
+ storage_ = src.storage_;
+ }
+ return *this;
+ }
+
+ // Tests whether this reference is the only active one to the string object.
+ bool unique() const { return storage_.unique(); }
+
+ // Assigns the storage of an HTTPValue based on the provided storage. This
+ // can be used for a cache Get. Returns false if the string is not
+ // well-formed.
+ //
+ // Extracts the headers into the provided MetaData buffer.
+ bool Link(SharedString* src, MetaData* headers, MessageHandler* handler);
+
+ // Access the shared string, for insertion into a cache via Put.
+ SharedString* share() { return &storage_; }
+
+ size_t size() const { return storage_->size(); }
+
+ private:
+ char type_identifier() const { return (*storage_.get())[0]; }
+ unsigned int SizeOfFirstChunk() const;
+ void SetSizeOfFirstChunk(unsigned int size);
+
+ // Disconnects this HTTPValue from other HTTPValues that may share the
+ // underlying storage, allowing a new buffer.
+ void CopyOnWrite();
+
+ SharedString storage_;
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_HTTP_VALUE_H_
diff --git a/trunk/src/net/instaweb/util/public/lru_cache.h b/trunk/src/net/instaweb/util/public/lru_cache.h
new file mode 100644
index 0000000..94aa359
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/lru_cache.h
@@ -0,0 +1,110 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_LRU_CACHE_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_LRU_CACHE_H_
+
+#include <list>
+#include <map>
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/cache_interface.h"
+#include <string>
+
+namespace net_instaweb {
+
+// Simple C++ implementation of an in-memory least-recently used (LRU)
+// cache. This implementation is not thread-safe, and must be
+// combined with a mutex to make it so.
+//
+// The purpose of this implementation is as a default implementation,
+// or an local shadow for memcached.
+//
+// Also of note: the Get interface allows for streaming. To get into
+// a std::string, use a StringWriter.
+//
+// TODO(jmarantz): The Put interface does not currently stream, but this
+// should be added.
+class LRUCache : public CacheInterface {
+ public:
+ explicit LRUCache(size_t max_size)
+ : max_bytes_in_cache_(max_size),
+ current_bytes_in_cache_(0) {
+ ClearStats();
+ }
+ virtual ~LRUCache();
+
+ virtual bool Get(const std::string& key, SharedString* value);
+
+ // Puts an object into the cache, sharing the bytes.
+ //
+ // TODO(jmarantz): currently if the caller mutates the
+ // SharedString after having called Put, it will actually
+ // modify the value in the cache. We should change
+ // SharedString to Copy-On-Write semantics.
+ virtual void Put(const std::string& key, SharedString* new_value);
+ virtual void Delete(const std::string& key);
+ virtual KeyState Query(const std::string& key);
+
+ // Total size in bytes of keys and values stored.
+ size_t size_bytes() const { return current_bytes_in_cache_; }
+
+ // Number of elements stored
+ size_t num_elements() const { return map_.size(); }
+
+ size_t num_evictions() const { return num_evictions_; }
+ size_t num_hits() const { return num_hits_; }
+ size_t num_misses() const { return num_misses_; }
+ size_t num_inserts() const { return num_inserts_; }
+ size_t num_deletes() const { return num_deletes_; }
+
+ // Sanity check the cache data structures.
+ void SanityCheck();
+
+ // Clear the entire cache. Used primarily for testing. Note that this
+ // will not clear the stats, however it will update current_bytes_in_cache_.
+ void Clear();
+
+ // Clear the stats -- note that this will not clear the content.
+ void ClearStats();
+
+ private:
+ typedef std::pair<const std::string*, SharedString> KeyValuePair;
+ typedef std::list<KeyValuePair*> EntryList;
+ // STL guarantees lifetime of list itererators as long as the node is in list.
+ typedef EntryList::iterator ListNode;
+ typedef std::map<std::string, ListNode> Map;
+ inline int entry_size(KeyValuePair* kvp) const;
+ inline ListNode Freshen(KeyValuePair* key_value);
+ bool EvictIfNecessary(size_t bytes_needed);
+
+ size_t max_bytes_in_cache_;
+ size_t current_bytes_in_cache_;
+ size_t num_evictions_;
+ size_t num_hits_;
+ size_t num_misses_;
+ size_t num_inserts_;
+ size_t num_deletes_;
+ EntryList lru_ordered_list_;
+ Map map_;
+
+ DISALLOW_COPY_AND_ASSIGN(LRUCache);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_LRU_CACHE_H_
diff --git a/trunk/src/net/instaweb/util/public/md5_hasher.h b/trunk/src/net/instaweb/util/public/md5_hasher.h
new file mode 100644
index 0000000..48c660c
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/md5_hasher.h
@@ -0,0 +1,51 @@
+// Copyright 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Authors: sligocki@google.com (Shawn Ligocki),
+// lsong@google.com (Libo Song)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_MD5_HASHER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_MD5_HASHER_H_
+
+#include <algorithm> // for std::min
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/hasher.h"
+
+namespace net_instaweb {
+
+class MD5Hasher : public Hasher {
+ public:
+ static const int kMaxHashSize;
+ static const int kDefaultHashSize = 10;
+
+ MD5Hasher() : hash_size_(kDefaultHashSize) {}
+ MD5Hasher(int hash_size) {
+ CHECK(hash_size >= 0);
+ hash_size_ = std::min(hash_size, kMaxHashSize);
+ }
+ virtual ~MD5Hasher();
+
+ virtual std::string Hash(const StringPiece& content) const;
+
+ virtual int HashSizeInChars() const { return hash_size_; }
+
+ private:
+ int hash_size_;
+
+ DISALLOW_COPY_AND_ASSIGN(MD5Hasher);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_MD5_HASHER_H_
diff --git a/trunk/src/net/instaweb/util/public/mem_file_system.h b/trunk/src/net/instaweb/util/public/mem_file_system.h
new file mode 100644
index 0000000..fbc2cf2
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/mem_file_system.h
@@ -0,0 +1,101 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: abliss@google.com (Adam Bliss)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_MEM_FILE_SYSTEM_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_MEM_FILE_SYSTEM_H_
+
+#include <stdlib.h>
+#include <map>
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/file_system.h"
+#include "net/instaweb/util/public/mock_timer.h"
+
+namespace net_instaweb {
+
+// An in-memory implementation of the FileSystem interface, for use in
+// unit tests. Does not fully support directories. Not particularly efficient.
+// Not threadsafe.
+// TODO(abliss): add an ability to block writes for arbitrarily long, to
+// enable testing resilience to concurrency problems with real filesystems.
+class MemFileSystem : public FileSystem {
+ public:
+ MemFileSystem() : enabled_(true), timer_(0), temp_file_index_(0) {}
+ virtual ~MemFileSystem();
+
+ // We offer a "simulated atime" in which the clock ticks forward one
+ // second every time you read or write a file.
+ virtual bool Atime(const StringPiece& path, int64* timestamp_sec,
+ MessageHandler* handler);
+ virtual BoolOrError Exists(const char* path, MessageHandler* handler);
+ virtual BoolOrError IsDir(const char* path, MessageHandler* handler);
+ virtual bool ListContents(const StringPiece& dir, StringVector* files,
+ MessageHandler* handler);
+ virtual bool MakeDir(const char* directory_path, MessageHandler* handler);
+ virtual InputFile* OpenInputFile(const char* filename,
+ MessageHandler* message_handler);
+ virtual OutputFile* OpenOutputFileHelper(const char* filename,
+ MessageHandler* message_handler);
+ virtual OutputFile* OpenTempFileHelper(const StringPiece& prefix_name,
+ MessageHandler* message_handle);
+ virtual bool RecursivelyMakeDir(const StringPiece& directory_path,
+ MessageHandler* handler);
+ virtual bool RemoveFile(const char* filename, MessageHandler* handler);
+ virtual bool RenameFileHelper(const char* old_file, const char* new_file,
+ MessageHandler* handler);
+ virtual bool Size(const StringPiece& path, int64* size,
+ MessageHandler* handler);
+ virtual BoolOrError TryLock(const StringPiece& lock_name,
+ MessageHandler* handler);
+ virtual BoolOrError TryLockWithTimeout(const StringPiece& lock_name,
+ int64 timeout_ms,
+ MessageHandler* handler);
+ virtual bool Unlock(const StringPiece& lock_name,
+ MessageHandler* handler);
+
+ // Empties out the entire filesystem. Should not be called while files
+ // are open.
+ void Clear();
+
+ // Test-specific functionality to disable and re-enable the filesystem.
+ void Disable() { enabled_ = false; }
+ void Enable() { enabled_ = true; }
+
+ // Accessor for timer. Timer is owned by mem_file_system.
+ MockTimer* timer() { return &timer_; }
+
+ private:
+ inline int64 CurrentTimeAndAdvance();
+ bool enabled_; // When disabled, OpenInputFile returns NULL.
+ typedef std::map<std::string, std::string> StringMap;
+ StringMap string_map_;
+ MockTimer timer_;
+ // atime_map_ holds times (in s) that files were last opened/modified. Each
+ // time we do such an operation, timer() advances by 1s (so all ATimes are
+ // distinct).
+ std::map<std::string, int64> atime_map_;
+ int temp_file_index_;
+ // lock_map_ holds times that locks were established (in ms).
+ // locking and unlocking don't advance time.
+ std::map<std::string, int64> lock_map_;
+
+ DISALLOW_COPY_AND_ASSIGN(MemFileSystem);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_MEM_FILE_SYSTEM_H_
diff --git a/trunk/src/net/instaweb/util/public/message_handler.h b/trunk/src/net/instaweb/util/public/message_handler.h
new file mode 100644
index 0000000..580e57d
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/message_handler.h
@@ -0,0 +1,103 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_MESSAGE_HANDLER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_MESSAGE_HANDLER_H_
+
+#include <stdarg.h>
+
+#include "net/instaweb/util/public/printf_format.h"
+
+namespace net_instaweb {
+
+enum MessageType {
+ kInfo,
+ kWarning,
+ kError,
+ kFatal
+};
+
+class MessageHandler {
+ public:
+ MessageHandler();
+ virtual ~MessageHandler();
+
+ // String representation for MessageType.
+ const char* MessageTypeToString(const MessageType type) const;
+
+ // Specify the minimum message type. Lower message types will not be
+ // logged.
+ void set_min_message_type(MessageType min) { min_message_type_ = min; }
+
+ // Log an info, warning, error or fatal error message.
+ void Message(MessageType type, const char* msg, ...)
+ INSTAWEB_PRINTF_FORMAT(3, 4);
+ void MessageV(MessageType type, const char* msg, va_list args);
+
+ // Log a message with a filename and line number attached.
+ void FileMessage(MessageType type, const char* filename, int line,
+ const char* msg, ...) INSTAWEB_PRINTF_FORMAT(5, 6);
+ void FileMessageV(MessageType type, const char* filename, int line,
+ const char* msg, va_list args);
+
+
+ // Conditional errors.
+ void Check(bool condition, const char* msg, ...) INSTAWEB_PRINTF_FORMAT(3, 4);
+ void CheckV(bool condition, const char* msg, va_list args);
+
+
+ // Convenience functions for FileMessage for backwards compatibility.
+ // TODO(sligocki): Rename these to InfoAt, ... so that Info, ... can be used
+ // for general Messages.
+ void Info(const char* filename, int line, const char* msg, ...)
+ INSTAWEB_PRINTF_FORMAT(4, 5);
+ void Warning(const char* filename, int line, const char* msg, ...)
+ INSTAWEB_PRINTF_FORMAT(4, 5);
+ void Error(const char* filename, int line, const char* msg, ...)
+ INSTAWEB_PRINTF_FORMAT(4, 5);
+ void FatalError(const char* filename, int line, const char* msg, ...)
+ INSTAWEB_PRINTF_FORMAT(4, 5);
+
+ void InfoV(const char* filename, int line, const char* msg, va_list args) {
+ FileMessageV(kInfo, filename, line, msg, args);
+ }
+ void WarningV(const char* filename, int line, const char* msg, va_list a) {
+ FileMessageV(kWarning, filename, line, msg, a);
+ }
+ void ErrorV(const char* filename, int line, const char* msg, va_list args) {
+ FileMessageV(kError, filename, line, msg, args);
+ }
+ void FatalErrorV(const char* fname, int line, const char* msg, va_list a) {
+ FileMessageV(kFatal, fname, line, msg, a);
+ }
+
+ protected:
+ virtual void MessageVImpl(MessageType type, const char* msg,
+ va_list args) = 0;
+ virtual void FileMessageVImpl(MessageType type, const char* filename,
+ int line, const char* msg, va_list args) = 0;
+
+ private:
+ // The minimum message type to log at. Any messages below this level
+ // will not be logged.
+ MessageType min_message_type_;
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_MESSAGE_HANDLER_H_
diff --git a/trunk/src/net/instaweb/util/public/meta_data.h b/trunk/src/net/instaweb/util/public/meta_data.h
new file mode 100644
index 0000000..edf4e97
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/meta_data.h
@@ -0,0 +1,232 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+//
+// Meta-data associated with a rewriting resource. This is
+// primarily a key-value store, but additionally we want to
+// get easy access to the cache expiration time.
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_META_DATA_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_META_DATA_H_
+
+#include <vector>
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class MessageHandler;
+class Writer;
+
+// Global constants for common HTML attribues names and values.
+//
+// TODO(jmarantz): proactively change all the occurences of the static strings
+// to use these shared constants.
+struct HttpAttributes {
+ static const char kAcceptEncoding[];
+ static const char kCacheControl[];
+ static const char kContentEncoding[];
+ static const char kContentLength[];
+ static const char kContentType[];
+ static const char kDate[];
+ static const char kDeflate[];
+ static const char kEtag[];
+ static const char kExpires[];
+ static const char kGzip[];
+ static const char kHost[];
+ static const char kIfModifiedSince[];
+ static const char kLastModified[];
+ static const char kLocation[];
+ static const char kNoCache[];
+ static const char kReferer[]; // sic
+ static const char kServer[];
+ static const char kSetCookie[];
+ static const char kTransferEncoding[];
+ static const char kUserAgent[];
+ static const char kVary[];
+};
+
+namespace HttpStatus {
+// Http status codes.
+// Grokked from http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
+enum Code {
+ kContinue = 100,
+ kSwitchingProtocols = 101,
+
+ kOK = 200,
+ kCreated = 201,
+ kAccepted = 202,
+ kNonAuthoritative = 203,
+ kNoContent = 204,
+ kResetContent = 205,
+ kPartialContent = 206,
+
+ kMultipleChoices = 300,
+ kMovedPermanently = 301,
+ kFound = 302,
+ kSeeOther = 303,
+ kNotModified = 304,
+ kUseProxy = 305,
+ kSwitchProxy = 306, // In old spec; no longer used.
+ kTemporaryRedirect = 307,
+
+ kBadRequest = 400,
+ kUnauthorized = 401,
+ kPaymentRequired = 402,
+ kForbidden = 403,
+ kNotFound = 404,
+ kMethodNotAllowed = 405,
+ kNotAcceptable = 406,
+ kProxyAuthRequired = 407,
+ kRequestTimeout = 408,
+ kConflict = 409,
+ kGone = 410,
+ kLengthRequired = 411,
+ kPreconditionFailed = 412,
+ kEntityTooLarge = 413,
+ kUriTooLong = 414,
+ kUnsupportedMediaType = 415,
+ kRangeNotSatisfiable = 416,
+ kExpectationFailed = 417,
+
+ kInternalServerError = 500,
+ kNotImplemented = 501,
+ kBadGateway = 502,
+ kUnavailable = 503,
+ kGatewayTimeout = 504,
+ kHttpVersionNotSupported = 505,
+
+ // Instaweb-specific response codes: these are intentionally chosen to be
+ // outside the normal HTTP range, but we consider these response codes
+ // to be 'cacheable' in our own cache.
+ kRememberNotFoundStatusCode = 10001,
+};
+
+// Transform a status code into the equivalent reason phrase.
+const char* GetReasonPhrase(Code rc);
+
+} // namespace HttpStatus
+
+// Container for required meta-data. General HTTP headers can be added
+// here as name/value pairs, and caching information can then be derived.
+//
+// TODO(jmarantz): consider rename to HTTPHeader.
+// TODO(sligocki): This represents an HTTP response header. We need a request
+// header class as well.
+// TODO(sligocki): Stop using char* in interface.
+class MetaData {
+ public:
+ MetaData() {}
+ virtual ~MetaData();
+
+ void CopyFrom(const MetaData& other);
+
+ // Reset headers to initial state.
+ virtual void Clear() = 0;
+
+ // Raw access for random access to attribute name/value pairs.
+ virtual int NumAttributes() const = 0;
+ virtual const char* Name(int index) const = 0;
+ virtual const char* Value(int index) const = 0;
+
+ // Get the attribute values associated with this name. Returns
+ // false if the attribute is not found. If it was found, then
+ // the values vector is filled in.
+ virtual bool Lookup(const char* name, CharStarVector* values) const = 0;
+
+ // Add a new header.
+ virtual void Add(const StringPiece& name, const StringPiece& value) = 0;
+
+ // Remove all headers by name.
+ virtual void RemoveAll(const char* name) = 0;
+
+ // Serialize HTTP response header to a stream.
+ virtual bool Write(Writer* writer, MessageHandler* handler) const = 0;
+ // Serialize just the headers (not the version and response code line).
+ virtual bool WriteHeaders(Writer* writer, MessageHandler* handler) const = 0;
+
+ // Parse a chunk of HTTP response header. Returns number of bytes consumed.
+ virtual int ParseChunk(const StringPiece& text, MessageHandler* handler) = 0;
+
+ // Compute caching information. The current time is used to compute
+ // the absolute time when a cache resource will expire. The timestamp
+ // is in milliseconds since 1970. It is an error to call any of the
+ // accessors before ComputeCaching is called.
+ virtual void ComputeCaching() = 0;
+ virtual bool IsCacheable() const = 0;
+ virtual bool IsProxyCacheable() const = 0;
+ virtual int64 CacheExpirationTimeMs() const = 0;
+ virtual void SetDate(int64 date_ms) = 0;
+ virtual void SetLastModified(int64 last_modified_ms) = 0;
+
+ virtual bool headers_complete() const = 0;
+ virtual void set_headers_complete(bool x) = 0;
+
+ virtual int major_version() const = 0;
+ virtual int minor_version() const = 0;
+ virtual int status_code() const = 0;
+ virtual const char* reason_phrase() const = 0;
+ virtual int64 timestamp_ms() const = 0;
+ virtual bool has_timestamp_ms() const = 0;
+
+ virtual void set_major_version(int major_version) = 0;
+ virtual void set_minor_version(int minor_version) = 0;
+
+ // Sets the status code and reason_phrase based on an internal table.
+ void SetStatusAndReason(HttpStatus::Code code);
+
+ virtual void set_status_code(int status_code) = 0;
+ virtual void set_reason_phrase(const StringPiece& reason_phrase) = 0;
+ // Set whole first line.
+ virtual void set_first_line(int major_version,
+ int minor_version,
+ int status_code,
+ const StringPiece& reason_phrase) {
+ set_major_version(major_version);
+ set_minor_version(minor_version);
+ set_status_code(status_code);
+ set_reason_phrase(reason_phrase);
+ }
+
+ virtual std::string ToString() const = 0;
+ void DebugPrint() const;
+
+ // Parses an arbitrary string into milliseconds since 1970
+ static bool ParseTime(const char* time_str, int64* time_ms);
+
+ // Determines whether a response header is marked as gzipped.
+ bool IsGzipped() const;
+
+ // Determines whether a request header accepts gzipped content.
+ bool AcceptsGzip() const;
+
+ // Parses a date header such as HttpAttributes::kDate or
+ // HttpAttributes::kExpires, returning the timestamp as
+ // number of milliseconds since 1970.
+ bool ParseDateHeader(const char* attr, int64* date_ms) const;
+
+ // Updates a date header using time specified as a number of milliseconds
+ // since 1970.
+ void UpdateDateHeader(const char* attr, int64 date_ms);
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(MetaData);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_META_DATA_H_
diff --git a/trunk/src/net/instaweb/util/public/mock_hasher.h b/trunk/src/net/instaweb/util/public/mock_hasher.h
new file mode 100644
index 0000000..440053f
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/mock_hasher.h
@@ -0,0 +1,41 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_MOCK_HASHER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_MOCK_HASHER_H_
+
+#include "net/instaweb/util/public/hasher.h"
+#include <string>
+
+namespace net_instaweb {
+
+class MockHasher : public Hasher {
+ public:
+ MockHasher() {}
+ virtual ~MockHasher();
+
+ virtual std::string Hash(const StringPiece& content) const { return "0"; }
+ virtual int HashSizeInChars() const { return 1; }
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(MockHasher);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_MOCK_HASHER_H_
diff --git a/trunk/src/net/instaweb/util/public/mock_message_handler.h b/trunk/src/net/instaweb/util/public/mock_message_handler.h
new file mode 100644
index 0000000..60c0160
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/mock_message_handler.h
@@ -0,0 +1,58 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: morlovich@google.com (Maksim Orlovich)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_MOCK_MESSAGE_HANDLER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_MOCK_MESSAGE_HANDLER_H_
+
+#include <map>
+
+#include "net/instaweb/util/public/google_message_handler.h"
+
+namespace net_instaweb {
+
+// A version of GoogleMessageHandler to use in testcases that keeps
+// track of number of messages output, to validate diagnostics
+class MockMessageHandler : public GoogleMessageHandler {
+ public:
+ MockMessageHandler() {}
+ virtual ~MockMessageHandler();
+
+ // Returns number of messages of given type issued
+ int MessagesOfType(MessageType type) const;
+
+ // Returns total number of messages issued
+ int TotalMessages() const;
+
+ // Returns number of messages of severity higher than info
+ int SeriousMessages() const;
+
+ protected:
+ virtual void MessageVImpl(MessageType type, const char* msg, va_list args);
+
+ virtual void FileMessageVImpl(MessageType type, const char* filename,
+ int line, const char* msg, va_list args);
+
+ private:
+ typedef std::map<MessageType, int> MessageCountMap;
+ MessageCountMap message_counts_;
+ DISALLOW_COPY_AND_ASSIGN(MockMessageHandler);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_MOCK_HASHER_H_
diff --git a/trunk/src/net/instaweb/util/public/mock_timer.h b/trunk/src/net/instaweb/util/public/mock_timer.h
new file mode 100644
index 0000000..4bf093f
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/mock_timer.h
@@ -0,0 +1,53 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_MOCK_TIMER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_MOCK_TIMER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/timer.h"
+
+namespace net_instaweb {
+
+class MockTimer : public Timer {
+ public:
+ // A useful recent time-constant for testing.
+ static const int64 kApr_5_2010_ms;
+
+ explicit MockTimer(int64 time_ms) : time_us_(1000 * time_ms) {}
+ virtual ~MockTimer();
+
+ void set_time_us(int64 time_us) { time_us_ = time_us; }
+ void set_time_ms(int64 time_ms) { set_time_us(1000 * time_ms); }
+ void advance_us(int64 delta_us) { time_us_ += delta_us; }
+ void advance_ms(int64 delta_ms) { advance_us(1000 * delta_ms); }
+
+ // Returns number of microseconds since 1970.
+ virtual int64 NowUs() const { return time_us_; }
+ virtual void SleepUs(int64 us) { advance_us(us); }
+ virtual void SleepMs(int64 ms) { advance_us(1000 * ms); }
+
+ private:
+ int64 time_us_;
+
+ DISALLOW_COPY_AND_ASSIGN(MockTimer);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_MOCK_TIMER_H_
diff --git a/trunk/src/net/instaweb/util/public/mock_url_fetcher.h b/trunk/src/net/instaweb/util/public/mock_url_fetcher.h
new file mode 100644
index 0000000..7335c25
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/mock_url_fetcher.h
@@ -0,0 +1,86 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_MOCK_URL_FETCHER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_MOCK_URL_FETCHER_H_
+
+#include <map>
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/url_fetcher.h"
+
+namespace net_instaweb {
+
+// Simple UrlFetcher meant for tests, you can set responses for individual URLs.
+class MockUrlFetcher : public UrlFetcher {
+ public:
+ MockUrlFetcher() : enabled_(true), fail_on_unexpected_(true) {}
+ virtual ~MockUrlFetcher();
+
+ void SetResponse(const StringPiece& url, const MetaData& response_header,
+ const StringPiece& response_body);
+
+ // Fetching unset URLs will cause EXPECT failures as well as return false.
+ virtual bool StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* message_handler);
+
+ // Clear all set responses.
+ void Clear();
+
+ // When disabled, fetcher will fail (but not crash) for all requests.
+ // Use to simulate temporarily not having access to resources, for example.
+ void Disable() { enabled_ = false; }
+ void Enable() { enabled_ = true; }
+
+ // Set to false if you don't want the fetcher to EXPECT fail on unfound URL.
+ // Useful in MockUrlFetcher unittest :)
+ void set_fail_on_unexpected(bool x) { fail_on_unexpected_ = x; }
+
+ private:
+ class HttpResponse {
+ public:
+ HttpResponse(const MetaData& in_header, const StringPiece& in_body)
+ : body_(in_body.data(), in_body.size()) {
+ header_.CopyFrom(in_header);
+ }
+
+ const SimpleMetaData& header() const { return header_; }
+ const std::string& body() const { return body_; }
+
+ private:
+ SimpleMetaData header_;
+ std::string body_;
+
+ DISALLOW_COPY_AND_ASSIGN(HttpResponse);
+ };
+ typedef std::map<const std::string, const HttpResponse*> ResponseMap;
+
+ ResponseMap response_map_;
+ bool enabled_;
+ bool fail_on_unexpected_; // Should we EXPECT if unexpected url called?
+
+ DISALLOW_COPY_AND_ASSIGN(MockUrlFetcher);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_MOCK_URL_FETCHER_H_
diff --git a/trunk/src/net/instaweb/util/public/named_lock_manager.h b/trunk/src/net/instaweb/util/public/named_lock_manager.h
new file mode 100644
index 0000000..c33f529
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/named_lock_manager.h
@@ -0,0 +1,62 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_NAMED_LOCK_MANAGER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_NAMED_LOCK_MANAGER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/abstract_mutex.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class AbstractLock : public AbstractMutex {
+ public:
+ virtual ~AbstractLock();
+ // If lock is held, return false, otherwise lock and return true
+ virtual bool TryLock() = 0;
+ // Wait bounded amount of time to take lock, otherwise return false.
+ virtual bool LockTimedWait(int64 wait_ms) = 0;
+
+ // ...StealOld versions of locking routines steal the lock if its current
+ // holder has locked it for more than timeout_ms. *WARNING* If you use
+ // any ...StealOld methods, your lock becomes "best-effort" and there may
+ // be multiple workers in a section! *WARNING*
+
+ // LockStealOld will block until the lock is unlocked or times out.
+ virtual void LockStealOld(int64 timeout_ms) = 0;
+ // Locks if lock is unlocked or held longer than timeout_ms.
+ virtual bool TryLockStealOld(int64 timeout_ms) = 0;
+ // LockTimedWaitStealOld will block until unlocked, the lock has been held for
+ // timeout_ms, or the caller has waited for wait_ms. Thus wait_ms is
+ // effectively bounded by timeout_ms.
+ virtual bool LockTimedWaitStealOld(int64 wait_ms, int64 timeout_ms) = 0;
+};
+
+// A named_lock_manager provides global locks named by strings (with the same
+// naming limitations in general as file names). They provide a fairly rich
+// api, with blocking and try versions and various timeout / steal behaviors.
+class NamedLockManager {
+ public:
+ virtual ~NamedLockManager();
+ virtual AbstractLock* CreateNamedLock(const StringPiece& name) = 0;
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_NAMED_LOCK_MANAGER_H_
diff --git a/trunk/src/net/instaweb/util/public/null_message_handler.h b/trunk/src/net/instaweb/util/public/null_message_handler.h
new file mode 100644
index 0000000..68c1e6e
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/null_message_handler.h
@@ -0,0 +1,45 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: lsong@google.com (Libo Song)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_NULL_MESSAGE_HANDLER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_NULL_MESSAGE_HANDLER_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/message_handler.h"
+
+namespace net_instaweb {
+
+// Implementation of a message handler that does nothing.
+class NullMessageHandler : public MessageHandler {
+ public:
+ NullMessageHandler() {}
+ virtual ~NullMessageHandler();
+
+ protected:
+ virtual void MessageVImpl(MessageType type, const char* msg, va_list args);
+
+ virtual void FileMessageVImpl(MessageType type, const char* filename,
+ int line, const char* msg, va_list args);
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(NullMessageHandler);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_NULL_MESSAGE_HANDLER_H_
diff --git a/trunk/src/net/instaweb/util/public/null_writer.h b/trunk/src/net/instaweb/util/public/null_writer.h
new file mode 100644
index 0000000..7c11c82
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/null_writer.h
@@ -0,0 +1,41 @@
+/*
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_NULL_WRITER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_NULL_WRITER_H_
+
+#include "net/instaweb/util/public/writer.h"
+
+namespace net_instaweb {
+
+// A writer that silently eats the bytes. This can be used, for
+// example, with writers is designed to cascade to another one, such
+// as CountingWriter. If you just want to count the bytes and don't
+// want to store them, you can pass a NullWriter to a CountingWriter's
+// constructor.
+class NullWriter : public Writer {
+ public:
+ explicit NullWriter() { }
+ virtual ~NullWriter();
+ virtual bool Write(const StringPiece& str, MessageHandler* handler);
+ virtual bool Flush(MessageHandler* handler);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_NULL_WRITER_H_
diff --git a/trunk/src/net/instaweb/util/public/printf_format.h b/trunk/src/net/instaweb/util/public/printf_format.h
new file mode 100644
index 0000000..d1c350b
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/printf_format.h
@@ -0,0 +1,38 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: bmcquade@google.com (Bryan McQuade)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_PRINTF_FORMAT_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_PRINTF_FORMAT_H_
+
+#ifdef __GNUC__
+
+// Tell the compiler a function is using a printf-style format string.
+// |format_param| is the one-based index of the format string parameter;
+// |dots_param| is the one-based index of the "..." parameter.
+// For v*printf functions (which take a va_list), pass 0 for dots_param.
+// (This is undocumented but matches what the system C headers do.)
+#define INSTAWEB_PRINTF_FORMAT(format_param, dots_param) \
+ __attribute__((format(printf, format_param, dots_param)))
+
+#else // Not GCC
+
+#define INSTAWEB_PRINTF_FORMAT(x, y)
+
+#endif
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_PRINTF_FORMAT_H_
diff --git a/trunk/src/net/instaweb/util/public/pthread_mutex.h b/trunk/src/net/instaweb/util/public/pthread_mutex.h
new file mode 100644
index 0000000..3825474
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/pthread_mutex.h
@@ -0,0 +1,43 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_PTHREAD_MUTEX_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_PTHREAD_MUTEX_H_
+
+#include <pthread.h>
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/abstract_mutex.h"
+
+namespace net_instaweb {
+
+// Implementation of AbstractMutex for Pthread mutexes.
+class PthreadMutex : public AbstractMutex {
+ public:
+ PthreadMutex();
+ virtual ~PthreadMutex();
+ virtual void Lock();
+ virtual void Unlock();
+ private:
+ pthread_mutex_t mutex_;
+
+ DISALLOW_COPY_AND_ASSIGN(PthreadMutex);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_PTHREAD_MUTEX_H_
diff --git a/trunk/src/net/instaweb/util/public/query_params.h b/trunk/src/net/instaweb/util/public/query_params.h
new file mode 100644
index 0000000..87a0c33
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/query_params.h
@@ -0,0 +1,85 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_QUERY_PARAMS_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_QUERY_PARAMS_H_
+
+#include <map>
+#include <vector>
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+// Parses and rewrites URL query parameters.
+class QueryParams {
+ public:
+ QueryParams() {}
+ ~QueryParams() { Clear(); }
+
+ // Parse a query param string, e.g. x=0&y=1&z=2. We expect the "?"
+ // to be extracted (e.g. this string is the output of GURL::query().
+ void Parse(const StringPiece& query_string);
+
+ // Find the values associated with a variable. Note that you may
+ // specify a variable multiple times in a query-string, e.g.
+ // a=0&a=1&a=2
+ bool Lookup(const char* name, CharStarVector* values) const;
+
+ // Remove all variables by name.
+ void RemoveAll(const char* name);
+
+ // Remove all variables.
+ void Clear();
+
+ // Raw access for random access to variable name/value pairs.
+ int size() const { return variable_vector_.size(); }
+ const char* name(int index) const { return variable_vector_[index].first; }
+
+ // Note that the value can be NULL, indicating that the variables
+ // was not followed by a '='. So given "a=0&b&c=", the values will
+ // be {"0", NULL, ""}.
+ const char* value(int index) const { return variable_vector_[index].second; }
+
+ std::string ToString() const;
+
+ // Add a new variable.
+ void Add(const StringPiece& name, const StringPiece& value);
+
+ private:
+ // We are keeping two structures, conceptually map<String,vector<String>> and
+ // vector<pair<String,String>>, so we can do associative lookups and
+ // also order-preserving iteration and random access.
+ //
+ // To avoid duplicating the strings, we will have the map own the
+ // Names (keys) in a std::string, and the string-pair-vector own the
+ // value as an explicitly newed char*. The risk of using a std::string
+ // to hold the value is that the pointers will not survive a resize.
+ typedef std::pair<const char*, char*> StringPair; // owns the value
+ typedef std::map<std::string, CharStarVector> VariableMap;
+ typedef std::vector<StringPair> VariableVector;
+
+ VariableMap variable_map_;
+ VariableVector variable_vector_;
+
+ DISALLOW_COPY_AND_ASSIGN(QueryParams);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_QUERY_PARAMS_H_
diff --git a/trunk/src/net/instaweb/util/public/rolling_hash.h b/trunk/src/net/instaweb/util/public/rolling_hash.h
new file mode 100644
index 0000000..c245ba3
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/rolling_hash.h
@@ -0,0 +1,69 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_CYCLIC_HASH_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_CYCLIC_HASH_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+// Rolling hash for char buffers based on a polynomial lookup table.
+// See http://en.wikipedia.org/wiki/Rolling_hash
+
+// Per character hash values. Exported for use in NextRollingHash.
+extern const uint64 kRollingHashCharTable[256];
+
+// Compute the rolling hash of buf[start : start + n - 1]
+uint64 RollingHash(const char* buf, size_t start, size_t n);
+
+// Given the rolling hash prev of buf[start - 1 : start + n - 2], efficiently
+// compute the hash of buf[start : start + n - 1]. Note that this indexes
+// buf[start - 1], so we can't just use a StringPiece here. We eschew
+// StringPiece in any case, because of efficiency.
+//
+// Note that to get efficient operation here for fixed n (eg when we're doing
+// something like Rabin-Karp string matching), we must inline the computation of
+// shift amounts and then hoist them as loop invariants. That is why this
+// function (intended for use in an inner loop) is inlined.
+inline uint64 NextRollingHash(
+ const char* buf, size_t start, size_t n, uint64 prev) {
+ // In a reasonable loop, the following two tests should be eliminated based on
+ // contextual information, if our compiler is optimizing enough.
+ CHECK_LT(static_cast<size_t>(0), start);
+ uint64 start_hash =
+ kRollingHashCharTable[static_cast<uint8>(buf[start - 1])];
+ uint64 end_hash =
+ kRollingHashCharTable[static_cast<uint8>(buf[start - 1 + n])];
+ uint64 prev_rot1 = (prev << 1) | (prev >> 63); // rotate left 1
+ uint64 start_hash_rotn;
+ // Corner case: shift by >= 64 bits is not defined in C. gcc had better
+ // constant-fold this to a rotate! (It appears to.) We inline in large part
+ // to ensure the truthiness of this fact.
+ size_t shift = n % 64;
+ if (shift == 0) {
+ start_hash_rotn = start_hash;
+ } else {
+ // rotate left by shift (equiv to rotating left n times).
+ start_hash_rotn = (start_hash << shift) | (start_hash >> (64 - shift));
+ }
+ return (start_hash_rotn ^ prev_rot1 ^ end_hash);
+}
+} // net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_ROLLING_HASH_H_
diff --git a/trunk/src/net/instaweb/util/public/shared_string.h b/trunk/src/net/instaweb/util/public/shared_string.h
new file mode 100644
index 0000000..9eb41fb
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/shared_string.h
@@ -0,0 +1,77 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+//
+// Implements a ref-counted string class, with full sharing. This
+// class does *not* implement copy-on-write semantics, however, it
+// does support a unique() method for determining, prior to writing,
+// whether other references exist. Thus it is feasible to implement
+// copy-on-write as a layer over this class.
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_SHARED_STRING_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_SHARED_STRING_H_
+
+#include "base/basictypes.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+#include "base/ref_counted.h"
+
+namespace net_instaweb {
+
+
+class RefCountedString : public base::RefCountedThreadSafe<RefCountedString> {
+ public:
+ RefCountedString() { }
+ explicit RefCountedString(const StringPiece& str)
+ : string_(str.data(), str.size()) {
+ }
+ const std::string& value() const { return string_; }
+ std::string& value() { return string_; }
+ size_t size() const { return string_.size(); }
+ const char* data() const { return string_.data(); }
+
+ private:
+ friend class base::RefCountedThreadSafe<RefCountedString>;
+ ~RefCountedString() { }
+
+ std::string string_;
+
+ DISALLOW_COPY_AND_ASSIGN(RefCountedString);
+};
+
+class SharedString : public scoped_refptr<RefCountedString> {
+ public:
+ SharedString() : scoped_refptr<RefCountedString>(new RefCountedString) {
+ }
+
+ explicit SharedString(const StringPiece& str)
+ : scoped_refptr<RefCountedString>(new RefCountedString(str)) {
+ }
+ std::string& operator*() { return ptr_->value(); }
+ std::string* get() { return &(ptr_->value()); }
+ const std::string* get() const { return &(ptr_->value()); }
+ std::string* operator->() { return &(ptr_->value()); }
+ const std::string* operator->() const { return &(ptr_->value()); }
+ bool unique() const { return ptr_->HasOneRef(); }
+ std::string::size_type size() const { return ptr_->value().size(); }
+};
+
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_SHARED_STRING_H_
diff --git a/trunk/src/net/instaweb/util/public/simple_meta_data.h b/trunk/src/net/instaweb/util/public/simple_meta_data.h
new file mode 100644
index 0000000..0909fa8
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/simple_meta_data.h
@@ -0,0 +1,140 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_SIMPLE_META_DATA_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_SIMPLE_META_DATA_H_
+
+#include <stdlib.h>
+#include <map>
+#include <vector>
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/meta_data.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+// Very basic implementation of HTTP headers.
+//
+// TODO(jmarantz): implement caching rules properly.
+class SimpleMetaData : public MetaData {
+ public:
+ SimpleMetaData();
+ virtual ~SimpleMetaData();
+
+ virtual void Clear();
+
+ // Raw access for random access to attribute name/value pairs.
+ virtual int NumAttributes() const;
+ virtual const char* Name(int index) const;
+ virtual const char* Value(int index) const;
+ virtual bool Lookup(const char* name, CharStarVector* values) const;
+
+ // Add a new header.
+ virtual void Add(const StringPiece& name, const StringPiece& value);
+
+ // Remove all headers by name.
+ virtual void RemoveAll(const char* name);
+
+ // Serialize HTTP response header to a stream.
+ virtual bool Write(Writer* writer, MessageHandler* message_handler) const;
+ // Serialize just the headers (not the version and response code line).
+ virtual bool WriteHeaders(Writer* writer, MessageHandler* handler) const;
+
+ // Parse a chunk of HTTP response header. Returns number of bytes consumed.
+ virtual int ParseChunk(const StringPiece& text, MessageHandler* handler);
+
+ // Compute caching information. The current time is used to compute
+ // the absolute time when a cache resource will expire. The timestamp
+ // is in milliseconds since 1970. It is an error to call any of the
+ // accessors before ComputeCaching is called.
+ virtual void ComputeCaching();
+ virtual bool IsCacheable() const;
+ virtual bool IsProxyCacheable() const;
+ virtual int64 CacheExpirationTimeMs() const;
+ virtual void SetDate(int64 date_ms);
+ virtual void SetLastModified(int64 last_modified_ms);
+
+ virtual bool headers_complete() const { return headers_complete_; }
+ virtual void set_headers_complete(bool x) { headers_complete_ = x; }
+
+ virtual int major_version() const { return major_version_; }
+ virtual int minor_version() const { return minor_version_; }
+ virtual int status_code() const { return status_code_; }
+ virtual const char* reason_phrase() const {
+ return reason_phrase_.c_str();
+ }
+ virtual int64 timestamp_ms() const { return timestamp_ms_; }
+ virtual bool has_timestamp_ms() const;
+
+ virtual void set_major_version(const int major_version) {
+ major_version_ = major_version;
+ }
+ virtual void set_minor_version(const int minor_version) {
+ minor_version_ = minor_version;
+ }
+ virtual void set_status_code(const int code) { status_code_ = code; }
+ virtual void set_reason_phrase(const StringPiece& reason_phrase) {
+ reason_phrase.CopyToString(&reason_phrase_);
+ }
+
+ virtual std::string ToString() const;
+
+ private:
+ bool GrabLastToken(const std::string& input, std::string* output);
+
+ friend class SimpleMetaDataTest;
+
+ // We are keeping two structures, conseptually map<String,vector<String>> and
+ // vector<pair<String,String>>, so we can do associative lookups and
+ // also order-preserving iteration and random access.
+ //
+ // To avoid duplicating the strings, we will have the map own the
+ // Names (keys) in a std::string, and the string-pair-vector own the
+ // value as an explicitly newed char*. The risk of using a std::string
+ // to hold the value is that the pointers will not survive a resize.
+ typedef std::pair<const char*, char*> StringPair; // owns the value
+ typedef std::map<std::string, CharStarVector,
+ StringCompareInsensitive> AttributeMap;
+ typedef std::vector<StringPair> AttributeVector;
+
+ AttributeMap attribute_map_;
+ AttributeVector attribute_vector_;
+
+ bool parsing_http_;
+ bool parsing_value_;
+ bool headers_complete_;
+ bool cache_fields_dirty_;
+ bool is_cacheable_; // accurate only if !cache_fields_dirty_
+ bool is_proxy_cacheable_; // accurate only if !cache_fields_dirty_
+ int64 expiration_time_ms_; // accurate only if !cache_fields_dirty_
+ int64 timestamp_ms_; // accurate only if !cache_fields_dirty_
+ std::string parse_name_;
+ std::string parse_value_;
+
+ int major_version_;
+ int minor_version_;
+ int status_code_;
+ std::string reason_phrase_;
+
+ DISALLOW_COPY_AND_ASSIGN(SimpleMetaData);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_SIMPLE_META_DATA_H_
diff --git a/trunk/src/net/instaweb/util/public/simple_stats.h b/trunk/src/net/instaweb/util/public/simple_stats.h
new file mode 100644
index 0000000..48e6dfe
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/simple_stats.h
@@ -0,0 +1,57 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_SIMPLE_STATS_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_SIMPLE_STATS_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/statistics_template.h"
+
+namespace net_instaweb {
+
+class SimpleStats;
+class SimpleStatsVariable : public Variable {
+ public:
+ SimpleStatsVariable() : value_(0) {}
+ virtual ~SimpleStatsVariable();
+ virtual int Get() const { return value_; }
+ virtual void Set(int value) { value_ = value; }
+
+ private:
+ int value_;
+
+ DISALLOW_COPY_AND_ASSIGN(SimpleStatsVariable);
+};
+
+// Simple name/value pair statistics implementation.
+class SimpleStats : public StatisticsTemplate<SimpleStatsVariable> {
+ public:
+ static const int kNotFound;
+
+ SimpleStats() { }
+ virtual ~SimpleStats();
+
+ protected:
+ virtual SimpleStatsVariable* NewVariable(const StringPiece& name, int index);
+
+ DISALLOW_COPY_AND_ASSIGN(SimpleStats);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_SIMPLE_STATS_H_
diff --git a/trunk/src/net/instaweb/util/public/statistics.h b/trunk/src/net/instaweb/util/public/statistics.h
new file mode 100644
index 0000000..bd2a643
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/statistics.h
@@ -0,0 +1,60 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_STATISTICS_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_STATISTICS_H_
+
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class Variable {
+ public:
+ virtual ~Variable();
+ // TODO(sligocki): int -> int64
+ virtual int Get() const = 0;
+ virtual void Set(int delta) = 0;
+
+ virtual void Add(int delta) { Set(delta + Get()); }
+ void Clear() { Set(0); }
+};
+
+// Helps build a statistics that can be exported as a CSV file.
+class Statistics {
+ public:
+ virtual ~Statistics();
+
+ // Add a new variable, or returns an existing one of that name.
+ // The Variable* is owned by the Statistics class -- it should
+ // not be deleted by the caller.
+ virtual Variable* AddVariable(const StringPiece& name) = 0;
+
+ // Find a variable from a name, returning NULL if not found.
+ virtual Variable* FindVariable(const StringPiece& name) const = 0;
+
+ // Find a variable from a name, aborting if not found.
+ virtual Variable* GetVariable(const StringPiece& name) const {
+ Variable* var = FindVariable(name);
+ CHECK(var != NULL) << "Variable not found: " << name;
+ return var;
+ }
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_STATISTICS_H_
diff --git a/trunk/src/net/instaweb/util/public/statistics_template.h b/trunk/src/net/instaweb/util/public/statistics_template.h
new file mode 100644
index 0000000..bcd848d
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/statistics_template.h
@@ -0,0 +1,90 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_STATISTICS_TEMPLATE_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_STATISTICS_TEMPLATE_H_
+
+#include <map>
+#include <vector>
+#include "net/instaweb/util/public/statistics.h"
+#include "net/instaweb/util/public/stl_util.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+// This class makes it easier to define new Statistics implementations
+// by providing a templatized implementation of variable registration and
+// management.
+template<class Var> class StatisticsTemplate : public Statistics {
+ public:
+ StatisticsTemplate() {}
+ virtual ~StatisticsTemplate() {
+ STLDeleteContainerPointers(variables_.begin(), variables_.end());
+ }
+
+ // Add a new variable, or returns an existing one of that name.
+ // The Variable* is owned by the Statistics class -- it should
+ // not be deleted by the caller.
+ virtual Variable* AddVariable(const StringPiece& name) {
+ return AddVariableInternal(name);
+ }
+
+ // Find a variable from a name, returning NULL if not found.
+ virtual Variable* FindVariable(const StringPiece& name) const {
+ return FindVariableInternal(name);
+ }
+
+ protected:
+ // Adds a variable, returning as the template class Var type, rather
+ // than its base class, as AddVariable does.
+ virtual Var* AddVariableInternal(const StringPiece& name) {
+ Var* var = FindVariableInternal(name);
+ if (var == NULL) {
+ var = NewVariable(name, variables_.size());
+ variables_.push_back(var);
+ variable_map_[std::string(name.data(), name.size())] = var;
+ }
+ return var;
+ }
+
+ // Finds a variable, returning as the template class Var type, rather
+ // than its base class, as FindVariable does.
+ virtual Var* FindVariableInternal(const StringPiece& name) const {
+ typename VarMap::const_iterator p = variable_map_.find(name.as_string());
+ Var* var = NULL;
+ if (p != variable_map_.end()) {
+ var = p->second;
+ }
+ return var;
+ }
+
+ virtual Var* NewVariable(const StringPiece& name, int index) = 0;
+
+ protected:
+ typedef std::vector<Var*> VarVector;
+ typedef std::map<std::string, Var*> VarMap;
+ VarVector variables_;
+ VarMap variable_map_;
+
+ DISALLOW_COPY_AND_ASSIGN(StatisticsTemplate);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_STATISTICS_TEMPLATE_H_
diff --git a/trunk/src/net/instaweb/util/public/statistics_work_bound.h b/trunk/src/net/instaweb/util/public/statistics_work_bound.h
new file mode 100644
index 0000000..7e50c97
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/statistics_work_bound.h
@@ -0,0 +1,49 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_STATISTICS_WORK_BOUND_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_STATISTICS_WORK_BOUND_H_
+
+#include "net/instaweb/util/public/work_bound.h"
+
+namespace net_instaweb {
+
+class Variable;
+
+// A WorkBound implementation in terms of statistics. This is a bit of a hack
+// that gets things implemented quickly (especially given the complexity of
+// multiprocess shared-memory infrastructure, which we only want to roll once).
+// Note in particular that we handle a NULL variable gracefully by imposing no
+// bound at all.
+class StatisticsWorkBound : public WorkBound {
+ public:
+ // Note that ownership of variable remains with the creating Statistics
+ // object. If the bound is 0, the bound is actually infinite.
+ StatisticsWorkBound(Variable* variable, int bound);
+ virtual ~StatisticsWorkBound();
+
+ virtual bool TryToWork();
+ virtual void WorkComplete();
+ private:
+ Variable* variable_;
+ int bound_;
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_STATISTICS_WORK_BOUND_H_
diff --git a/trunk/src/net/instaweb/util/public/stdio_file_system.h b/trunk/src/net/instaweb/util/public/stdio_file_system.h
new file mode 100644
index 0000000..7ff8aec
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/stdio_file_system.h
@@ -0,0 +1,65 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_STDIO_FILE_SYSTEM_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_STDIO_FILE_SYSTEM_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/file_system.h"
+
+namespace net_instaweb {
+
+class StdioFileSystem : public FileSystem {
+ public:
+ StdioFileSystem() {}
+ virtual ~StdioFileSystem();
+
+ virtual bool Atime(const StringPiece& path, int64* timestamp_sec,
+ MessageHandler* handler);
+ virtual InputFile* OpenInputFile(const char* filename,
+ MessageHandler* message_handler);
+ virtual OutputFile* OpenOutputFileHelper(const char* filename,
+ MessageHandler* message_handler);
+ virtual OutputFile* OpenTempFileHelper(const StringPiece& prefix_name,
+ MessageHandler* message_handle);
+
+ virtual bool RemoveFile(const char* filename, MessageHandler* handler);
+ virtual bool RenameFileHelper(const char* old_file, const char* new_file,
+ MessageHandler* handler);
+ virtual bool MakeDir(const char* directory_path, MessageHandler* handler);
+ virtual BoolOrError Exists(const char* path, MessageHandler* handler);
+ virtual BoolOrError IsDir(const char* path, MessageHandler* handler);
+ virtual bool ListContents(const StringPiece& dir, StringVector* files,
+ MessageHandler* handler);
+ virtual bool Size(const StringPiece& path, int64* size,
+ MessageHandler* handler);
+ virtual BoolOrError TryLock(const StringPiece& lock_name,
+ MessageHandler* handler);
+ virtual bool Unlock(const StringPiece& lock_name,
+ MessageHandler* handler);
+ InputFile* Stdin();
+ OutputFile* Stdout();
+ OutputFile* Stderr();
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(StdioFileSystem);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_STDIO_FILE_SYSTEM_H_
diff --git a/trunk/src/net/instaweb/util/public/stl_util.h b/trunk/src/net/instaweb/util/public/stl_util.h
new file mode 100644
index 0000000..b09c180
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/stl_util.h
@@ -0,0 +1,24 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_STL_UTIL_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_STL_UTIL_H_
+
+#include "base/stl_util-inl.h"
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_STL_UTIL_H_
diff --git a/trunk/src/net/instaweb/util/public/string_hash.h b/trunk/src/net/instaweb/util/public/string_hash.h
new file mode 100644
index 0000000..da86af8
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/string_hash.h
@@ -0,0 +1,52 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_STRING_HASH_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_STRING_HASH_H_
+
+#include <stddef.h>
+
+namespace net_instaweb {
+
+inline size_t HashString(const char* s, size_t len) {
+ // TODO(jmarantz): The portability layers defined in
+ // third_party/chromium/src/base/hash_tables.h
+ // third_party/protobuf2/src/src/google/protobuf/stubs/hash.h
+ // do not make it obvious how to hash a string directly -- they only
+ // make it possible to instantiate hash_set<string> and hash_map<string,...>.
+ // A reasonable hashing function looks available in apr_hashfunc_default,
+ // but for build isolation reasons we don't want to reference that here.
+ //
+ // For now, define our own implemention, copied from code in
+ // third_party/chromium/src/base/hash_tables.h.
+ size_t result = 0;
+ for (size_t i = 0; i < len; ++i, ++s) {
+ result = (result * 131) + *s;
+ }
+ return result;
+}
+
+// Combine two hash values in a reasonable way. Here to avoid
+// excessive mysticism in the remainder of the code.
+inline size_t JoinHash(size_t a, size_t b) {
+ return (a + 56) * 137 + b * 151; // Uses different prime multipliers.
+}
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_STRING_HASH_H_
diff --git a/trunk/src/net/instaweb/util/public/string_util.h b/trunk/src/net/instaweb/util/public/string_util.h
new file mode 100644
index 0000000..bf8874f
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/string_util.h
@@ -0,0 +1,170 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_STRING_UTIL_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_STRING_UTIL_H_
+
+
+#include <set>
+#include <vector>
+#include <string>
+
+#include <stdlib.h>
+#include "base/string_number_conversions.h"
+#include "base/string_piece.h"
+#include "base/string_util.h"
+
+namespace net_instaweb {
+
+typedef base::StringPiece StringPiece;
+
+inline std::string IntegerToString(int i) {
+ return base::IntToString(i);
+}
+
+inline std::string Integer64ToString(int64 i) {
+ return base::Int64ToString(i);
+}
+
+inline bool StringToInt(const char* in, int* out) {
+ // TODO(bmcquade): Use char*-based StringToInt once we sync the
+ // Chromium repository.
+ std::string str(in);
+ return base::StringToInt(str, out);
+}
+
+inline bool StringToInt64(const char* in, int64* out) {
+ // TODO(bmcquade): Use char*-based StringToInt64 once we sync the
+ // Chromium repository.
+ std::string str(in);
+ return base::StringToInt64(str, out);
+}
+
+inline bool StringToInt(const std::string& in, int* out) {
+ return base::StringToInt(in, out);
+}
+
+inline bool StringToInt64(const std::string& in, int64* out) {
+ return base::StringToInt64(in, out);
+}
+
+class EmptyString {
+ public:
+ static const StringPiece kEmptyString;
+};
+
+// TODO(jmarantz): use overloading instead of default args and get
+// rid of this statically constructed global object.
+std::string StrCat(const StringPiece& a, const StringPiece& b,
+ const StringPiece& c = EmptyString::kEmptyString,
+ const StringPiece& d = EmptyString::kEmptyString,
+ const StringPiece& e = EmptyString::kEmptyString,
+ const StringPiece& f = EmptyString::kEmptyString,
+ const StringPiece& g = EmptyString::kEmptyString,
+ const StringPiece& h = EmptyString::kEmptyString);
+
+void SplitStringPieceToVector(const StringPiece& sp, const char* separator,
+ std::vector<StringPiece>* components,
+ bool omit_empty_strings);
+
+void BackslashEscape(const StringPiece& src,
+ const StringPiece& to_escape,
+ std::string* dest);
+
+bool HasPrefixString(const StringPiece& str, const StringPiece& prefix);
+
+void LowerString(std::string* str);
+
+inline bool OnlyWhitespace(const std::string& str) {
+ return ContainsOnlyWhitespaceASCII(str);
+}
+
+inline char* strdup(const char* str) {
+ return base::strdup(str);
+}
+
+inline int strcasecmp(const char* s1, const char* s2) {
+ return base::strcasecmp(s1, s2);
+}
+
+inline int strncasecmp(const char* s1, const char* s2, size_t count) {
+ return base::strncasecmp(s1, s2, count);
+}
+
+inline void TrimWhitespace(const StringPiece& in, std::string* output) {
+ static const char whitespace[] = " \r\n\t";
+ TrimString(std::string(in.data(), in.size()), whitespace, output);
+}
+
+// Accumulates a decimal value from 'c' into *value.
+// Returns false and leaves *value unchanged if c is not a decimal digit.
+bool AccumulateDecimalValue(char c, int* value);
+
+// Accumulates a hex value from 'c' into *value
+// Returns false and leaves *value unchanged if c is not a hex digit.
+bool AccumulateHexValue(char c, int* value);
+
+// Return true iff the two strings are equal, ignoring case.
+bool StringCaseEqual(const StringPiece& s1, const StringPiece& s2);
+// Return true iff str starts with prefix, ignoring case.
+bool StringCaseStartsWith(const StringPiece& str, const StringPiece& prefix);
+// Return true iff str ends with suffix, ignoring case.
+bool StringCaseEndsWith(const StringPiece& str, const StringPiece& suffix);
+
+struct CharStarCompareInsensitive {
+ bool operator()(const char* s1, const char* s2) const {
+ return strcasecmp(s1, s2) < 0;
+ };
+};
+
+struct CharStarCompareSensitive {
+ bool operator()(const char* s1, const char* s2) const {
+ return strcmp(s1, s2) < 0;
+ }
+};
+
+struct StringCompareInsensitive {
+ bool operator()(const std::string& s1, const std::string& s2) const {
+ return strcasecmp(s1.c_str(), s2.c_str()) < 0;
+ };
+};
+
+typedef std::vector<const char*> CharStarVector;
+typedef std::vector<std::string> StringVector;
+typedef std::set<std::string> StringSet;
+
+// Does a path end in slash?
+inline bool EndsInSlash(const StringPiece& path) {
+ return path.ends_with("/");
+}
+
+// Make sure directory's path ends in '/'.
+inline void EnsureEndsInSlash(std::string* dir) {
+ if (!EndsInSlash(*dir)) {
+ dir->append("/");
+ }
+}
+
+// Given a string such as: a b "c d" e 'f g'
+// Parse it into a vector: ["a", "b", "c d", "e", "f g"]
+void ParseShellLikeString(const StringPiece& input,
+ std::vector<std::string>* output);
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_STRING_UTIL_H_
diff --git a/trunk/src/net/instaweb/util/public/string_writer.h b/trunk/src/net/instaweb/util/public/string_writer.h
new file mode 100644
index 0000000..43d93c7
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/string_writer.h
@@ -0,0 +1,43 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_STRING_WRITER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_STRING_WRITER_H_
+
+#include "base/basictypes.h"
+#include <string>
+#include "net/instaweb/util/public/writer.h"
+
+namespace net_instaweb {
+
+// Writer implementation for directing HTML output to a string.
+class StringWriter : public Writer {
+ public:
+ explicit StringWriter(std::string* str) : string_(str) { }
+ virtual ~StringWriter();
+ virtual bool Write(const StringPiece& str, MessageHandler* message_handler);
+ virtual bool Flush(MessageHandler* message_handler);
+ private:
+ std::string* string_;
+
+ DISALLOW_COPY_AND_ASSIGN(StringWriter);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_STRING_WRITER_H_
diff --git a/trunk/src/net/instaweb/util/public/symbol_table.h b/trunk/src/net/instaweb/util/public/symbol_table.h
new file mode 100644
index 0000000..bfec527
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/symbol_table.h
@@ -0,0 +1,100 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_SYMBOL_TABLE_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_SYMBOL_TABLE_H_
+
+#include <stdlib.h>
+#include <set>
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/atom.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+// Implements a generic symbol table, allowing for case-sensitive
+// and case insensitive versions. The elements of SymbolTables are
+// Atoms. Atoms are created by Interning strings.
+//
+// Atoms are cheap and are passed around by value, not by reference or
+// pointer. Atoms can be compared to one another via ==. A const char*
+// can be extracted from an Atom via ==.
+//
+// Atoms are memory-managed by the symbol table from which they came.
+// When the symbol table is destroyed, so are all the Atoms that
+// were interned in it.
+//
+// Care should be taken not to attempt to compare Atoms created from
+// multiple symbol tables.
+//
+// TODO(jmarantz): Symbol tables are not currently thread-safe. We
+// should consider whether it's worth making them thread-safe, or
+// whether it's better to use separate symbol tables in each thread.
+template<class SymbolCompare> class SymbolTable {
+ public:
+ SymbolTable() { }
+ ~SymbolTable() {
+ while (!string_set_.empty()) {
+ // Note: This should perform OK for rb-trees, but will perform
+ // poorly if a hash-table is used.
+ typename SymbolSet::const_iterator p = string_set_.begin();
+ const char* str = *p;
+ string_set_.erase(p);
+ free(const_cast<char*>(str));
+ }
+ }
+
+ Atom Intern(const char* src) {
+ Atom atom(src);
+ typename SymbolSet::const_iterator iter = string_set_.find(src);
+ if (iter == string_set_.end()) {
+ char* str = strdup(src);
+ string_set_.insert(str);
+ return Atom(str);
+ }
+ return Atom(*iter);
+ }
+
+ inline Atom Intern(const std::string& src) {
+ return Intern(src.c_str());
+ }
+
+ private:
+ typedef std::set<const char*, SymbolCompare> SymbolSet;
+ SymbolSet string_set_;
+
+ DISALLOW_COPY_AND_ASSIGN(SymbolTable);
+};
+
+class SymbolTableInsensitive : public SymbolTable<CharStarCompareInsensitive> {
+ public:
+ SymbolTableInsensitive() { }
+ private:
+ DISALLOW_COPY_AND_ASSIGN(SymbolTableInsensitive);
+};
+class SymbolTableSensitive : public SymbolTable<CharStarCompareSensitive> {
+ public:
+ SymbolTableSensitive() { }
+ private:
+ DISALLOW_COPY_AND_ASSIGN(SymbolTableSensitive);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_SYMBOL_TABLE_H_
diff --git a/trunk/src/net/instaweb/util/public/threadsafe_cache.h b/trunk/src/net/instaweb/util/public/threadsafe_cache.h
new file mode 100644
index 0000000..52f2bccd
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/threadsafe_cache.h
@@ -0,0 +1,57 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_THREADSAFE_CACHE_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_THREADSAFE_CACHE_H_
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/cache_interface.h"
+#include <string>
+
+namespace net_instaweb {
+
+class MessageHandler;
+class Writer;
+class AbstractMutex;
+
+// Composes a cache with a Mutex to form a threadsafe cache.
+class ThreadsafeCache : public CacheInterface {
+ public:
+ // Takes ownership of the cache that's passed in.
+ ThreadsafeCache(CacheInterface* cache, AbstractMutex* mutex)
+ : cache_(cache),
+ mutex_(mutex) {
+ }
+ virtual ~ThreadsafeCache();
+
+ virtual bool Get(const std::string& key, SharedString* value);
+ virtual void Put(const std::string& key, SharedString* value);
+ virtual void Delete(const std::string& key);
+ virtual KeyState Query(const std::string& key);
+
+ private:
+ scoped_ptr<CacheInterface> cache_;
+ AbstractMutex* mutex_;
+
+ DISALLOW_COPY_AND_ASSIGN(ThreadsafeCache);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_THREADSAFE_CACHE_H_
diff --git a/trunk/src/net/instaweb/util/public/time_util.h b/trunk/src/net/instaweb/util/public/time_util.h
new file mode 100644
index 0000000..088ba05
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/time_util.h
@@ -0,0 +1,37 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_TIME_UTIL_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_TIME_UTIL_H_
+
+#include "base/basictypes.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+// Converts time, in milliseconds, to a string. Returns false on failure.
+bool ConvertTimeToString(int64 time_ms, std::string* time_string);
+
+// Converts time in string format, to the number of milliseconds since 1970.
+// Returns false on failure.
+bool ConvertStringToTime(const StringPiece& time_string, int64 *time_ms);
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_TIME_UTIL_H_
diff --git a/trunk/src/net/instaweb/util/public/timer.h b/trunk/src/net/instaweb/util/public/timer.h
new file mode 100644
index 0000000..4a6b48b
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/timer.h
@@ -0,0 +1,55 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_TIMER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_TIMER_H_
+
+#include "base/basictypes.h"
+
+namespace net_instaweb {
+
+// Timer interface, made virtual so it can be mocked for tests.
+class Timer {
+ public:
+ static const int64 kSecondUs;
+ static const int64 kSecondMs;
+ static const int64 kMinuteMs;
+ static const int64 kHourMs;
+ static const int64 kDayMs;
+ static const int64 kWeekMs;
+ static const int64 kMonthMs;
+ static const int64 kYearMs;
+
+ virtual ~Timer();
+
+ // Returns number of milliseconds since 1970.
+ virtual int64 NowMs() const;
+
+ // Returns number of microseconds since 1970.
+ virtual int64 NowUs() const = 0;
+
+ // Sleep for given number of milliseconds.
+ virtual void SleepMs(int64 ms);
+
+ // Sleep for given number of microseconds.
+ virtual void SleepUs(int64 us) = 0;
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_TIMER_H_
diff --git a/trunk/src/net/instaweb/util/public/timer_based_abstract_lock.h b/trunk/src/net/instaweb/util/public/timer_based_abstract_lock.h
new file mode 100644
index 0000000..83e2d9f
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/timer_based_abstract_lock.h
@@ -0,0 +1,54 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_TIMER_BASED_ABSTRACT_LOCK_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_TIMER_BASED_ABSTRACT_LOCK_H_
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/named_lock_manager.h"
+
+namespace net_instaweb {
+
+class Timer;
+
+// A TimerBasedAbstractLock implements a Lock by spinning on a TryLock, using a
+// Timer to perform exponential sleep backoff. Note that this means that it may
+// not obtain a long-held lock in a timely fashion after it has been unlocked.
+class TimerBasedAbstractLock : public AbstractLock {
+ public:
+ virtual ~TimerBasedAbstractLock();
+ virtual void Lock();
+ virtual bool LockTimedWait(int64 wait_ms);
+
+ virtual void LockStealOld(int64 timeout_ms);
+ virtual bool LockTimedWaitStealOld(int64 wait_ms, int64 timeout_ms);
+
+ protected:
+ virtual Timer* timer() const = 0;
+
+ private:
+ typedef bool (TimerBasedAbstractLock::*TryLockMethod)(int64 timeout_ms);
+ bool TryLockIgnoreTimeout(int64 timeout_ignored);
+ bool BusySpin(TryLockMethod try_lock, int64 timeout_ms);
+ void Spin(TryLockMethod try_lock, int64 timeout_ms);
+ bool SpinFor(TryLockMethod try_lock, int64 timeout_ms, int64 wait_ms);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_TIMER_BASED_ABSTRACT_LOCK_H_
diff --git a/trunk/src/net/instaweb/util/public/url_async_fetcher.h b/trunk/src/net/instaweb/util/public/url_async_fetcher.h
new file mode 100644
index 0000000..0a41f41
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/url_async_fetcher.h
@@ -0,0 +1,69 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+//
+// UrlFetcher is an interface for asynchronously fetching urls. The
+// caller must supply a callback to be called when the fetch is complete.
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_URL_ASYNC_FETCHER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_URL_ASYNC_FETCHER_H_
+
+#include "base/basictypes.h"
+#include <string>
+
+namespace net_instaweb {
+
+class MessageHandler;
+class MetaData;
+class Writer;
+
+class UrlAsyncFetcher {
+ public:
+ static const int64 kUnspecifiedTimeout;
+ struct Callback {
+ virtual ~Callback();
+ virtual void Done(bool success) = 0;
+
+ // Set to true if it's OK to call the callback from a different
+ // thread. The base class implementation returns false.
+ virtual bool EnableThreaded() const;
+ };
+
+ virtual ~UrlAsyncFetcher();
+
+ // Fetch a URL, streaming the output to fetched_content_writer, and
+ // returning the headers. request_headers is optional -- it can be NULL.
+ // response_headers and fetched_content_writer must be valid until
+ // the call to Done().
+ //
+ // This function returns true if the request was immediately satisfied.
+ // In either case, the callback will be called with the completion status,
+ // so it's safe to ignore the return value.
+ virtual bool StreamingFetch(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* message_handler,
+ Callback* callback) = 0;
+ // Returns a maximum time that we will allow fetches to take, or
+ // kUnspecifiedTimeout (the default) if we don't promise to timeout fetches.
+ virtual int64 timeout_ms() { return kUnspecifiedTimeout; }
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_URL_ASYNC_FETCHER_H_
diff --git a/trunk/src/net/instaweb/util/public/url_escaper.h b/trunk/src/net/instaweb/util/public/url_escaper.h
new file mode 100644
index 0000000..44525c6
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/url_escaper.h
@@ -0,0 +1,86 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_URL_ESCAPER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_URL_ESCAPER_H_
+
+#include "net/instaweb/util/public/url_segment_encoder.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+/*
+common url format
+
+ http://www.foo.bar/z1234/b_c.d?e=f&g=h
+
+this suggests we should have short encodings for a-zA-Z0-9.:/?&
+
+We would like the above URL to be reasonably legible if possible.
+However it's also nice if it's short.
+
+One annoyance is that we are using '.' to delimit the 4 fields of an
+instaweb-generated URL. That can probably be changed to use ^, which
+is a legal URL but is rarely used in URLs. This would enable us to
+leave . alone. But that's probably a moderately painful change
+involving a fair amount of regolding.
+
+In the meantime we can replace . with ^ in this encoder so the they
+don't change size. So the transform table is:
+
+a-zA-Z0-9_=+-&? unchanged
+% %%
+/ %_
+\ %-
+http:// %h
+.com %c
+.css %s
+.edu %e
+.gif %g
+.html %t
+.jpeg %k
+.jpg %j
+.js %l
+.net %n
+.png %p
+www. %w
+. ^
+^ %^
+
+everything else %XX where xx are hex digits using capital latters
+
+
+The intent of this class to to help encode arbitrary URLs (really, any
+stream of 8-byte characters, but optimized for URLs) so that it can be
+used in one 'segment' of a new URL. This means we will not output
+. or / but will instead escape those.
+*/
+
+class UrlEscaper : public UrlSegmentEncoder {
+ public:
+ virtual ~UrlEscaper();
+ virtual void EncodeToUrlSegment(
+ const StringPiece& in, std::string* url_segment);
+ virtual bool DecodeFromUrlSegment(
+ const StringPiece& url_segment, std::string* out);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_URL_ESCAPER_H_
diff --git a/trunk/src/net/instaweb/util/public/url_fetcher.h b/trunk/src/net/instaweb/util/public/url_fetcher.h
new file mode 100644
index 0000000..3bd720c
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/url_fetcher.h
@@ -0,0 +1,61 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+//
+// UrlFetcher is an interface for fetching urls.
+//
+// TODO(jmarantz): Consider asynchronous fetches. This may not require
+// a change in interface; we would simply always return 'false' if the
+// url contents is not already cached. We may want to consider a richer
+// return-value enum to distinguish illegal ULRs from invalid ones, from
+// ones where the fetch is in-progress. Or maybe the caller doesn't care.
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_URL_FETCHER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_URL_FETCHER_H_
+
+#include <string>
+
+namespace net_instaweb {
+
+class MessageHandler;
+class MetaData;
+class Writer;
+
+class UrlFetcher {
+ public:
+ virtual ~UrlFetcher();
+
+ // Fetch a URL, streaming the output to fetched_content_writer, and
+ // returning the headers. Returns true if the fetch was successful.
+ virtual bool StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* message_handler) = 0;
+
+ // Convenience method for fetching URL into a string, with no headers in
+ // our out. This is primarily for upward compatibility.
+ //
+ // TODO(jmarantz): change callers to use StreamingFetchUrl and remove this.
+ bool FetchUrl(const std::string& url,
+ std::string* content,
+ MessageHandler* message_handler);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_URL_FETCHER_H_
diff --git a/trunk/src/net/instaweb/util/public/url_multipart_encoder.h b/trunk/src/net/instaweb/util/public/url_multipart_encoder.h
new file mode 100644
index 0000000..e33a301
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/url_multipart_encoder.h
@@ -0,0 +1,80 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_URL_MULTIPART_ENCODER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_URL_MULTIPART_ENCODER_H_
+
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class MessageHandler;
+
+// Encodes a multiple strings into a single string so that it
+// can be decoded. This is not restricted to URLs but is optimized
+// for them in its choice of escape characters. '+' is used to
+// separate the parts, and any parts that include '+' are prefixed
+// by a '='. '=' is converted to '==' -- it's a pretty lightweight
+// encoding, and any other character restrictions will have to be
+// applied to the output of this class.
+//
+// TODO(jmarantz): One possibly improvement is to bake this
+// functionality into UrlEscaper, changing its interface to accept
+// arbitrary numbers of pieces in & out. However, that would change
+// an interface that's used in multiple places, so this is left as
+// a TODO.
+class UrlMultipartEncoder {
+ public:
+ UrlMultipartEncoder() {}
+
+ // Removes all the URLs from the encoding.
+ void clear() { urls_.clear(); }
+
+ // Adds a new URL to the encoding. Actually there are no
+ // character-set restrictions imposed by this method.
+ void AddUrl(const StringPiece& url) {
+ urls_.push_back(std::string(url.data(), url.size()));
+ }
+
+ // Encode the URLs added to this class into a single string.
+ std::string Encode() const;
+
+ // Decodde an encoding produced by Encode() above to populate
+ // this class.
+ bool Decode(const StringPiece& encoding, MessageHandler* handler);
+
+ // Returns the number of URLs stored (either by Decode or by
+ // AddUrl.
+ int num_urls() const { return urls_.size(); }
+
+ // Returns the url at the index.
+ const std::string& url(int index) const { return urls_[index]; }
+
+ // Removes the last URL.
+ void pop_back() { urls_.pop_back(); }
+
+ private:
+ StringVector urls_;
+
+ DISALLOW_COPY_AND_ASSIGN(UrlMultipartEncoder);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_URL_MULTIPART_ENCODER_H_
diff --git a/trunk/src/net/instaweb/util/public/url_segment_encoder.h b/trunk/src/net/instaweb/util/public/url_segment_encoder.h
new file mode 100644
index 0000000..ab38f73
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/url_segment_encoder.h
@@ -0,0 +1,48 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_URL_SEGMENT_ENCODER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_URL_SEGMENT_ENCODER_H_
+
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+// Abstract class that describes encoding of url segments by rewriters.
+// Most instances of this will want to delegate to a UrlEscaper, which
+// is itself an instance.
+class UrlSegmentEncoder {
+ public:
+ virtual ~UrlSegmentEncoder();
+ // *Append* encoding of url segment "in" to "url_segment".
+ virtual void EncodeToUrlSegment(const StringPiece& in,
+ std::string* url_segment) = 0;
+ // Decode url segment from "url_segment", *appending* to "out"; should consume
+ // entire StringPiece. Return false on decode failure.
+ virtual bool DecodeFromUrlSegment(const StringPiece& url_segment,
+ std::string* out) = 0;
+ protected:
+ UrlSegmentEncoder() { }
+ private:
+ DISALLOW_COPY_AND_ASSIGN(UrlSegmentEncoder);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_URL_SEGMENT_ENCODER_H_
diff --git a/trunk/src/net/instaweb/util/public/user_agent.h b/trunk/src/net/instaweb/util/public/user_agent.h
new file mode 100644
index 0000000..16bb53f
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/user_agent.h
@@ -0,0 +1,41 @@
+// Copyright 2010 Google Inc. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_USER_AGENT_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_USER_AGENT_H_
+
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class UserAgent {
+ public:
+ UserAgent();
+ void set_user_agent(const char* user_agent);
+
+ bool IsIe() const;
+ bool IsIe6() const;
+ bool IsIe7() const;
+ bool IsIe6or7() const {
+ return IsIe6() || IsIe7();
+ };
+
+ private:
+ std::string user_agent_;
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_USER_AGENT_H_
diff --git a/trunk/src/net/instaweb/util/public/wait_url_async_fetcher.h b/trunk/src/net/instaweb/util/public/wait_url_async_fetcher.h
new file mode 100644
index 0000000..e162701
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/wait_url_async_fetcher.h
@@ -0,0 +1,60 @@
+/*
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_WAIT_URL_ASYNC_FETCHER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_WAIT_URL_ASYNC_FETCHER_H_
+
+#include <vector>
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/url_async_fetcher.h"
+
+namespace net_instaweb {
+
+class UrlFetcher;
+
+// Fake UrlAsyncFetcher which waits to call underlying blocking fetcher until
+// you explicitly call CallCallbacks().
+class WaitUrlAsyncFetcher : public UrlAsyncFetcher {
+ public:
+ WaitUrlAsyncFetcher(UrlFetcher* url_fetcher) : url_fetcher_(url_fetcher) {}
+ virtual ~WaitUrlAsyncFetcher();
+
+ // Initiate fetches that will finish when CallCallbacks is called.
+ virtual bool StreamingFetch(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* message_handler,
+ Callback* callback);
+
+ // Call all callbacks from previously initiated fetches.
+ void CallCallbacks();
+
+ private:
+ class DelayedFetch;
+
+ UrlFetcher* url_fetcher_;
+ std::vector<DelayedFetch*> delayed_fetches_;
+
+ DISALLOW_COPY_AND_ASSIGN(WaitUrlAsyncFetcher);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_WAIT_URL_ASYNC_FETCHER_H_
diff --git a/trunk/src/net/instaweb/util/public/wget_url_fetcher.h b/trunk/src/net/instaweb/util/public/wget_url_fetcher.h
new file mode 100644
index 0000000..53ef679
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/wget_url_fetcher.h
@@ -0,0 +1,52 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_WGET_URL_FETCHER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_WGET_URL_FETCHER_H_
+
+#include <stdio.h>
+#include "base/basictypes.h"
+#include <string>
+#include "net/instaweb/util/public/url_fetcher.h"
+
+namespace net_instaweb {
+
+// Runs 'wget' via popen for blocking URL fetches.
+class WgetUrlFetcher : public UrlFetcher {
+ public:
+ WgetUrlFetcher() { }
+ virtual ~WgetUrlFetcher();
+
+ // TODO(sligocki): Allow protocol version number (e.g. HTTP/1.1)
+ // and request type (e.g. GET, POST, etc.) to be specified.
+ virtual bool StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* writer,
+ MessageHandler* message_handler);
+
+ // Default user agent to use.
+ static const char kDefaultUserAgent[];
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(WgetUrlFetcher);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_WGET_URL_FETCHER_H_
diff --git a/trunk/src/net/instaweb/util/public/wildcard.h b/trunk/src/net/instaweb/util/public/wildcard.h
new file mode 100644
index 0000000..e1f2652
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/wildcard.h
@@ -0,0 +1,60 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_WILDCARD_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_WILDCARD_H_
+
+#include <vector>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class Wildcard {
+ public:
+ static const char kMatchAny; // *
+ static const char kMatchOne; // ?
+
+ // Create a wildcard object with the specification using * and ?
+ // as wildcards. There is currently no way to quote * or ?.
+ explicit Wildcard(const StringPiece& wildcard_spec);
+
+ // Determines whether a string matches the wildcard.
+ bool Match(const StringPiece& str) { return MatchHelper(0, str); }
+
+ // Determines whether this wildcard is just a simple name, lacking
+ // any wildcard characters.
+ bool IsSimple() const;
+
+ // Returns the original wildcard specification.
+ const StringPiece spec() const { return storage_; }
+
+ // Makes a duplicate copy of the wildcard object.
+ Wildcard* Duplicate() const;
+
+ private:
+ bool MatchHelper(int piece_index, const StringPiece& str);
+ void InitFromStorage();
+
+ std::string storage_;
+ std::vector<StringPiece> pieces_;
+ DISALLOW_COPY_AND_ASSIGN(Wildcard);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_WILDCARD_H_
diff --git a/trunk/src/net/instaweb/util/public/wildcard_group.h b/trunk/src/net/instaweb/util/public/wildcard_group.h
new file mode 100644
index 0000000..9ac948f
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/wildcard_group.h
@@ -0,0 +1,77 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_SELECTOR_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_SELECTOR_H_
+
+#include <vector>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class Wildcard;
+
+// This forms the basis of a wildcard selection mechanism, allowing
+// a user to issue sequence of commands like:
+//
+// 1. allow *.cc
+// 2. allow *.h
+// 3. disallow a*.h
+// 4. allow ab*.h
+// 5. disallow c*.cc
+//
+// This sequence would yield the following results whnen
+// Match("x.cc") --> true due to rule #1
+// Match("c.cc") --> false due to rule #5 which overrides rule #1
+// Match("y.h") --> true due to rule #2
+// Match("a.h") --> false due to rule #3 which overrides rule #2
+// Match("ab.h") --> true due to rule #4 which overrides rule #3
+// So order matters.
+class WildcardGroup {
+ public:
+ WildcardGroup() {}
+ ~WildcardGroup();
+
+ // Determines whether a string matches the wildcard group.
+ bool Match(const StringPiece& str) const;
+
+ // Add an expression to Allow, potentially overriding previous calls to
+ // Disallow.
+ void Allow(const StringPiece& wildcard);
+
+ // Add an expression to Disallow, potentially overriding previous calls to
+ // Allow.
+ void Disallow(const StringPiece& wildcard);
+
+ void CopyFrom(const WildcardGroup& src);
+ void AppendFrom(const WildcardGroup& src);
+
+ private:
+ bool MatchHelper(int piece_index, const StringPiece& str);
+
+ // To avoid having to new another structure we use two parallel
+ // vectors. Note that vector<bool> is special-case implemented
+ // in STL to be bit-packed.
+ std::vector<Wildcard*> wildcards_;
+ std::vector<bool> allow_; // parallel array (actually a bitvector)
+ DISALLOW_COPY_AND_ASSIGN(WildcardGroup);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_SELECTOR_H_
diff --git a/trunk/src/net/instaweb/util/public/work_bound.h b/trunk/src/net/instaweb/util/public/work_bound.h
new file mode 100644
index 0000000..29f875e
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/work_bound.h
@@ -0,0 +1,36 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_WORK_BOUND_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_WORK_BOUND_H_
+
+namespace net_instaweb {
+
+// A WorkBound represents permission to do work bounded by some upper bound.
+// Roughly speaking we can represent this as a bounded shared counter, but
+// how we realize the counter implementation must vary from system to system.
+class WorkBound {
+ public:
+ virtual ~WorkBound();
+ virtual bool TryToWork() = 0;
+ virtual void WorkComplete() = 0;
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_WORK_BOUND_H_
diff --git a/trunk/src/net/instaweb/util/public/write_through_cache.h b/trunk/src/net/instaweb/util/public/write_through_cache.h
new file mode 100644
index 0000000..0122c36
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/write_through_cache.h
@@ -0,0 +1,69 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_WRITE_THROUGH_CACHE_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_WRITE_THROUGH_CACHE_H_
+
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/cache_interface.h"
+#include <string>
+
+namespace net_instaweb {
+
+class MessageHandler;
+class Writer;
+class AbstractMutex;
+
+// Composes two caches to form a write-through cache.
+class WriteThroughCache : public CacheInterface {
+ public:
+ static const size_t kUnlimited;
+
+ // Takes ownership of both caches passed in.
+ WriteThroughCache(CacheInterface* cache1, CacheInterface* cache2)
+ : cache1_(cache1),
+ cache2_(cache2),
+ cache1_size_limit_(kUnlimited) {
+ }
+ virtual ~WriteThroughCache();
+
+ virtual bool Get(const std::string& key, SharedString* value);
+ virtual void Put(const std::string& key, SharedString* value);
+ virtual void Delete(const std::string& key);
+ virtual KeyState Query(const std::string& key);
+
+ // By default, all data goes into both cache1 and cache2. But
+ // if you only want to put small items in cache1, you can set the
+ // size limit. Note that both the key and value will count
+ // torward the size.
+ void set_cache1_limit(size_t limit) { cache1_size_limit_ = limit; }
+
+ private:
+ void PutInCache1(const std::string& key, SharedString* value);
+
+ scoped_ptr<CacheInterface> cache1_;
+ scoped_ptr<CacheInterface> cache2_;
+ size_t cache1_size_limit_;
+
+ DISALLOW_COPY_AND_ASSIGN(WriteThroughCache);
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_WRITE_THROUGH_CACHE_H_
diff --git a/trunk/src/net/instaweb/util/public/writer.h b/trunk/src/net/instaweb/util/public/writer.h
new file mode 100644
index 0000000..01ba5ad
--- /dev/null
+++ b/trunk/src/net/instaweb/util/public/writer.h
@@ -0,0 +1,41 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#ifndef NET_INSTAWEB_UTIL_PUBLIC_WRITER_H_
+#define NET_INSTAWEB_UTIL_PUBLIC_WRITER_H_
+
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class MessageHandler;
+
+// Interface for writing bytes to an output stream.
+class Writer {
+ public:
+ virtual ~Writer();
+
+ virtual bool Write(const StringPiece& str, MessageHandler* handler) = 0;
+
+ virtual bool Flush(MessageHandler* message_handler) = 0;
+};
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_PUBLIC_WRITER_H_
diff --git a/trunk/src/net/instaweb/util/query_params.cc b/trunk/src/net/instaweb/util/query_params.cc
new file mode 100644
index 0000000..09aed4b
--- /dev/null
+++ b/trunk/src/net/instaweb/util/query_params.cc
@@ -0,0 +1,111 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/query_params.h"
+
+#include <stdio.h>
+#include "base/logging.h"
+#include "net/instaweb/util/public/message_handler.h"
+
+namespace net_instaweb {
+
+// TODO(jmarantz): Refactor much of this code with simple_meta_data.cc. We'd
+// have to templatize to indicate whether we wanted case sensitivity in our
+// variables.
+
+void QueryParams::Clear() {
+ for (int i = 0, n = variable_vector_.size(); i < n; ++i) {
+ delete [] variable_vector_[i].second;
+ }
+ variable_map_.clear();
+ variable_vector_.clear();
+}
+
+bool QueryParams::Lookup(const char* name, CharStarVector* values) const {
+ VariableMap::const_iterator p = variable_map_.find(name);
+ bool ret = false;
+ if (p != variable_map_.end()) {
+ ret = true;
+ *values = p->second;
+ }
+ return ret;
+}
+
+void QueryParams::Add(const StringPiece& name, const StringPiece& value) {
+ CharStarVector dummy_values;
+ std::string name_buf(name.data(), name.size());
+ std::pair<VariableMap::iterator, bool> iter_inserted = variable_map_.insert(
+ VariableMap::value_type(name_buf.c_str(), dummy_values));
+ VariableMap::iterator iter = iter_inserted.first;
+ CharStarVector& values = iter->second;
+ char* value_copy = NULL;
+ if (value.data() != NULL) {
+ int value_size = value.size();
+ value_copy = new char[value_size + 1];
+ memcpy(value_copy, value.data(), value_size);
+ value_copy[value_size] = '\0';
+ }
+ values.push_back(value_copy);
+ variable_vector_.push_back(StringPair(iter->first.c_str(), value_copy));
+}
+
+void QueryParams::RemoveAll(const char* var_name) {
+ VariableVector temp_vector; // Temp variable for new vector.
+ temp_vector.reserve(variable_vector_.size());
+ for (int i = 0; i < size(); ++i) {
+ if (strcasecmp(name(i), var_name) != 0) {
+ temp_vector.push_back(variable_vector_[i]);
+ } else {
+ delete [] variable_vector_[i].second;
+ }
+ }
+ variable_vector_.swap(temp_vector);
+
+ // Note: we have to erase from the map second, because map owns the name.
+ variable_map_.erase(var_name);
+}
+
+void QueryParams::Parse(const StringPiece& text) {
+ CHECK_EQ(0, size());
+ std::vector<StringPiece> components;
+ SplitStringPieceToVector(text, "&", &components, true);
+ for (int i = 0, n = components.size(); i < n; ++i) {
+ StringPiece::size_type pos = components[i].find('=');
+ if (pos != StringPiece::npos) {
+ Add(components[i].substr(0, pos), components[i].substr(pos + 1));
+ } else {
+ Add(components[i], StringPiece(NULL, 0));
+ }
+ }
+}
+
+std::string QueryParams::ToString() const {
+ std::string str;
+ const char* prefix="";
+ for (int i = 0; i < size(); ++i) {
+ if (value(i) == NULL) {
+ str += StrCat(prefix, name(i));
+ } else {
+ str += StrCat(prefix, name(i), "=", value(i));
+ }
+ prefix = "&";
+ }
+ return str;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/query_params_test.cc b/trunk/src/net/instaweb/util/query_params_test.cc
new file mode 100644
index 0000000..e589936
--- /dev/null
+++ b/trunk/src/net/instaweb/util/query_params_test.cc
@@ -0,0 +1,101 @@
+// Copyright 2010 and onwards Google Inc.
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test SimpleUrlData, in particular it's HTTP header parser.
+
+#include "net/instaweb/util/public/query_params.h"
+#include <algorithm>
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/stdio_file_system.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/string_writer.h"
+
+namespace {
+
+const char kQueryString[] = "a=1&b&c=2&d=&a=3";
+
+} // namespace
+
+namespace net_instaweb {
+
+class QueryParamsTest : public testing::Test {
+ protected:
+ QueryParamsTest() { }
+
+ virtual void SetUp() {
+ query_params_.Parse(kQueryString);
+ }
+
+ QueryParams query_params_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(QueryParamsTest);
+};
+
+TEST_F(QueryParamsTest, TestParse) {
+ ASSERT_EQ(5, query_params_.size());
+ EXPECT_EQ(std::string("a"), query_params_.name(0));
+ EXPECT_EQ(std::string("1"), query_params_.value(0));
+ EXPECT_EQ(std::string("b"), query_params_.name(1));
+ EXPECT_EQ(NULL, query_params_.value(1));
+ EXPECT_EQ(std::string("c"), query_params_.name(2));
+ EXPECT_EQ(std::string("2"), query_params_.value(2));
+ EXPECT_EQ(std::string("d"), query_params_.name(3));
+ EXPECT_EQ(std::string(""), query_params_.value(3));
+ EXPECT_EQ(std::string("a"), query_params_.name(4));
+ EXPECT_EQ(std::string("3"), query_params_.value(4));
+ EXPECT_EQ(kQueryString, query_params_.ToString());
+}
+
+TEST_F(QueryParamsTest, TestLookup) {
+ CharStarVector v;
+ ASSERT_TRUE(query_params_.Lookup("a", &v));
+ ASSERT_EQ(2, v.size());
+ EXPECT_EQ(std::string("1"), v[0]);
+ EXPECT_EQ(std::string("3"), v[1]);
+ ASSERT_TRUE(query_params_.Lookup("b", &v));
+ ASSERT_EQ(1, v.size());
+ EXPECT_EQ(NULL, v[0]);
+ ASSERT_TRUE(query_params_.Lookup("c", &v));
+ ASSERT_EQ(1, v.size());
+ EXPECT_EQ(std::string("2"), v[0]);
+ ASSERT_TRUE(query_params_.Lookup("d", &v));
+ ASSERT_EQ(1, v.size());
+ EXPECT_EQ(std::string(""), v[0]);
+}
+
+TEST_F(QueryParamsTest, TestRemove) {
+ CharStarVector v;
+ query_params_.RemoveAll("a");
+ EXPECT_EQ("b&c=2&d=", query_params_.ToString());
+ EXPECT_EQ(3, query_params_.size());
+ query_params_.RemoveAll("b");
+ EXPECT_EQ("c=2&d=", query_params_.ToString());
+ EXPECT_EQ(2, query_params_.size());
+ query_params_.RemoveAll("c");
+ EXPECT_EQ("d=", query_params_.ToString());
+ EXPECT_EQ(1, query_params_.size());
+ query_params_.RemoveAll("d");
+ EXPECT_EQ("", query_params_.ToString());
+ EXPECT_EQ(0, query_params_.size());
+}
+
+TEST_F(QueryParamsTest, TestClear) {
+ query_params_.Clear();
+ EXPECT_EQ("", query_params_.ToString());
+ EXPECT_EQ(0, query_params_.size());
+}
+
+TEST_F(QueryParamsTest, TestAEqualsBEquals1) {
+ query_params_.Clear();
+ query_params_.Parse("a=b=1");
+ ASSERT_EQ(1, query_params_.size());
+ ASSERT_EQ(std::string("a"), query_params_.name(0));
+ ASSERT_EQ(std::string("b=1"), query_params_.value(0));
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/ref_counted.cc b/trunk/src/net/instaweb/util/ref_counted.cc
new file mode 100644
index 0000000..d82e900
--- /dev/null
+++ b/trunk/src/net/instaweb/util/ref_counted.cc
@@ -0,0 +1,26 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// This file works around an apparent omission in the Chromium
+// libraries. The Chromium source tree includes classes for reference
+// counting, but the implementation files are not included in the
+// Chromium library. The easiest way to ensure they are compiled into
+// Instaweb is to include them in a new .cc file:
+
+#include "base/ref_counted.cc"
+#include "base/thread_collision_warner.cc"
diff --git a/trunk/src/net/instaweb/util/rolling_hash.cc b/trunk/src/net/instaweb/util/rolling_hash.cc
new file mode 100644
index 0000000..01ec29c
--- /dev/null
+++ b/trunk/src/net/instaweb/util/rolling_hash.cc
@@ -0,0 +1,293 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/util/public/rolling_hash.h"
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+uint64 RollingHash(const char* buf, size_t start, size_t n) {
+ CHECK_LE(static_cast<size_t>(0), start);
+ buf += start;
+ uint64 hash = 0;
+ for (size_t i = 0; i < n; ++i) {
+ hash = (hash << 1) | (hash >> 63); // Rotate left 1
+ hash ^= kRollingHashCharTable[static_cast<uint8>(buf[i])];
+ }
+ return hash;
+}
+
+const uint64 kRollingHashCharTable[256] = {
+ 0xf0dcd27e0762b251ULL,
+ 0xb621e668da0b3d53ULL,
+ 0xce8c72d316e9293cULL,
+ 0xe958017c3359bad5ULL,
+ 0x6b11ca3935e9bb14ULL,
+ 0x64f67431b8b0bcb4ULL,
+ 0x8c4dd71ad988d23eULL,
+ 0x24936d1c08aff78dULL,
+ 0x69ecc7f3052e0396ULL,
+ 0xcdc8f08d999754c5ULL,
+ 0x39ab6e32568685cdULL,
+ 0x4e4f289b8b96e626ULL,
+ 0xa3c3cb4a32546b3dULL,
+ 0xe929810fec372e4dULL,
+ 0xfd94f0e82a05ac37ULL,
+ 0x3b71322f12f04f66ULL,
+ 0x8ad767cc6b92f018ULL,
+ 0xb4fff19004481ff1ULL,
+ 0xc9a1422e55e9d8bbULL,
+ 0x016fccc64de16d17ULL,
+ 0xd27154733cd80e3eULL,
+ 0x3f837bd6c3588413ULL,
+ 0x2c493e9f8c70ee0eULL,
+ 0xb45526d0cdcae876ULL,
+ 0xfa8778e134e91383ULL,
+ 0x0e8f821f8c783f69ULL,
+ 0xc28fd9493d9268e9ULL,
+ 0xf5b90b302376dc19ULL,
+ 0x12d5edf71944c0b9ULL,
+ 0x3de65234604d9f95ULL,
+ 0x6e0b7823a5ab650fULL,
+ 0xf30d1b84d74f542cULL,
+ 0xd59ea7a1a04daa1eULL,
+ 0xc322bb145da0eaeeULL,
+ 0x9f55e20e76bc821cULL,
+ 0x84eb8f72c858db2cULL,
+ 0x0727b30ff9b3080fULL,
+ 0x40cb8bf1f3e4114fULL,
+ 0x379d10f1f0166fc2ULL,
+ 0x8766af09de794470ULL,
+ 0x4cebe022f58b47f0ULL,
+ 0x75d232d0c4a94ebbULL,
+ 0xc63ba1fbd098199cULL,
+ 0xa0d947c29f383a9eULL,
+ 0x3bfd354a0404e6deULL,
+ 0x9d3164f90bd471a3ULL,
+ 0x357f031c2b1cee0eULL,
+ 0x47bb2d71880becd1ULL,
+ 0xc048dc946fd9dd85ULL,
+ 0xa21e8226ed9b2db9ULL,
+ 0xad06169fbe3a05a9ULL,
+ 0x0a7b91d58f44e29bULL,
+ 0x36bdd30c057a61baULL,
+ 0xcb78135eb2c3b107ULL,
+ 0xa65907ee1f4209ddULL,
+ 0x0962bb3ccf9816f2ULL,
+ 0xceb546dda951601eULL,
+ 0x5e5769344a1f8f21ULL,
+ 0x1fe188e97a0685eeULL,
+ 0x59d0d4289c5edacbULL,
+ 0xaa212857bb62f723ULL,
+ 0xa527e223de884eceULL,
+ 0x9c9a15c9aa56eba4ULL,
+ 0x9b16d95901aee267ULL,
+ 0xf4c5b630931f11e9ULL,
+ 0x3df802e7e85a6750ULL,
+ 0x4968f654cce17653ULL,
+ 0xfc3bc0ad02761567ULL,
+ 0xff18ec7e401c445dULL,
+ 0x2f844385f9c31b7aULL,
+ 0x88d750f43c6989edULL,
+ 0x4bb020be66f231f9ULL,
+ 0x12502d671fa7f487ULL,
+ 0x1926420bae67fb5aULL,
+ 0x2acb699d19d6c4d4ULL,
+ 0x5cdb79d0349158eaULL,
+ 0x67e93d1bf4815342ULL,
+ 0x43dc874f7024e36dULL,
+ 0xb92097281fed49b6ULL,
+ 0x62ed4f7a0a6958a9ULL,
+ 0x574c6b02ae6f3626ULL,
+ 0xd49b443e8932f23dULL,
+ 0x7dc50d1a701aab7aULL,
+ 0x48621174dff0fd92ULL,
+ 0x5746557ae580c35eULL,
+ 0x4bc7f97a8ebc2480ULL,
+ 0x07e620aad2f7b646ULL,
+ 0x1285636d57e3f612ULL,
+ 0x9f6cd306e5514c5cULL,
+ 0x0de00632f6f709d7ULL,
+ 0x7aa301698adee68bULL,
+ 0x2df10184f4cfd4ecULL,
+ 0x5f5b744cf78e8048ULL,
+ 0x88ebe0a2615fe4b3ULL,
+ 0xcb6f29a29401f98fULL,
+ 0x9c306f274e66d287ULL,
+ 0x88f632944f2ccdd9ULL,
+ 0x63bcf4a925db4311ULL,
+ 0xc6d2f5e9c0a50b72ULL,
+ 0xd6a731846a17be49ULL,
+ 0x2fd613caa70c235eULL,
+ 0x534de67031253facULL,
+ 0x4c685e47e9592796ULL,
+ 0x9b153e27865bd258ULL,
+ 0xd614462b61589f7dULL,
+ 0x1532cb94e28d8cbfULL,
+ 0x91497f579632a631ULL,
+ 0x26e3a246cc96fb61ULL,
+ 0xb28d962661bf129bULL,
+ 0x85da5d60a6ae5aa3ULL,
+ 0x0a9a1f6cb7c3e6a0ULL,
+ 0xae039586bd8c7f11ULL,
+ 0x0bfd1a8454e77964ULL,
+ 0xd6fa6b85a423d107ULL,
+ 0x015c3ab5134f69fcULL,
+ 0xdab1025b8bed7c18ULL,
+ 0xdf404b3edca12395ULL,
+ 0xad3d2d77444e1887ULL,
+ 0x92654b67bda1e990ULL,
+ 0xc412fe06d4e29f1eULL,
+ 0x2d382f434b99af46ULL,
+ 0x8cb666ada86575a1ULL,
+ 0x833d4b153f5f5405ULL,
+ 0x6007f6c51629e797ULL,
+ 0xd8a1f922cba896e3ULL,
+ 0x2c9b5a1a249f9e47ULL,
+ 0xd899781c79cb063bULL,
+ 0x911f5c7513ea3c91ULL,
+ 0x0c77ae198ca79978ULL,
+ 0x20ba7b1e0c97d74cULL,
+ 0xb3711eccdd549521ULL,
+ 0x0a94dc19e59543dfULL,
+ 0x3b348e9c6b0e36a6ULL,
+ 0x9627b15951a7a6b8ULL,
+ 0x9193dc839dbe7049ULL,
+ 0xba707aa6a2add40eULL,
+ 0x1777251ac57b2a2aULL,
+ 0xca6c2edba50c4c4fULL,
+ 0xa6c43cd31526ff60ULL,
+ 0x310d05dff737a640ULL,
+ 0xa8663ab4709bdd52ULL,
+ 0xd05f9d24885f3b94ULL,
+ 0x8c863ea5aafb221dULL,
+ 0x21ef263381f8cd63ULL,
+ 0x4b6fb4a8bc5458aaULL,
+ 0xc1d84b559be971b0ULL,
+ 0x210fb1bab7f15072ULL,
+ 0xe66ee09a51a55963ULL,
+ 0x39c21db635c08debULL,
+ 0xf05b979841675d0dULL,
+ 0x2978a9f732d96470ULL,
+ 0x4dce978a19c3a1f1ULL,
+ 0x5c0e92ab7319cd99ULL,
+ 0x743b286678d9e686ULL,
+ 0xdaf46993e90a5e81ULL,
+ 0x96dc2e0adcbb0d16ULL,
+ 0xd95690fe868663d8ULL,
+ 0x66ef0231433f2b39ULL,
+ 0x30774974792d96ccULL,
+ 0x44ffb0e25c47c44eULL,
+ 0x4cd9b89be5889713ULL,
+ 0xe62e5a6014f07cf3ULL,
+ 0xb56ac85a8af4dea0ULL,
+ 0xde92b05ba2e1b34aULL,
+ 0x23fd1311f823b78cULL,
+ 0xff80ac0f287ec43cULL,
+ 0x5d90a3866dd95fc0ULL,
+ 0xdec51083823f593fULL,
+ 0x86ad4349ac174faeULL,
+ 0xaedff28454b16a41ULL,
+ 0x3d8bc08e3f2e0c9dULL,
+ 0xf86319f232a3c729ULL,
+ 0xcd5e5e0f94263499ULL,
+ 0x8d7ba52980b19b79ULL,
+ 0xa67e8a3d335c81d2ULL,
+ 0x65d09ac6aa0fdab2ULL,
+ 0xc1795b7f5b644a20ULL,
+ 0xec83e7447c32d0dcULL,
+ 0x6f90245af445f4aeULL,
+ 0x56d70cadcc4a28fcULL,
+ 0xaf264eb82bcc4f90ULL,
+ 0xa16010fce3634affULL,
+ 0x8ea336b9c2c1e45bULL,
+ 0x7680ddb84af244bbULL,
+ 0xf97371315d450666ULL,
+ 0x8166dfa721896adcULL,
+ 0x3648e8d5b8d995d8ULL,
+ 0xa63975a605b31cf2ULL,
+ 0x35e60de4a3359ad4ULL,
+ 0x9b81705b2e4be07bULL,
+ 0xb9ae701a8eb85593ULL,
+ 0xaedecc0d138f3115ULL,
+ 0x8a1fdbb92e3423c1ULL,
+ 0xb0c96dee77615860ULL,
+ 0x629b7c06bf44e634ULL,
+ 0x696eb82aa746b1e1ULL,
+ 0x02efce1165256d4fULL,
+ 0x69d7a6b7150117a6ULL,
+ 0x9dcc2d6e896e5681ULL,
+ 0xcd815845d154cfbeULL,
+ 0x28ea53acb6da26f0ULL,
+ 0x7100bc4d21fe52dfULL,
+ 0x653e558ec969142fULL,
+ 0x83d730de45a0f5d4ULL,
+ 0x3831dd5c5647e4b1ULL,
+ 0x0f4072bf5123e9d5ULL,
+ 0xa642dce46ec285eeULL,
+ 0xae995373b5193c28ULL,
+ 0xf0f3dafb8a611288ULL,
+ 0xbc482a9e9b15c5abULL,
+ 0x41668ebcdc7cc0f4ULL,
+ 0xcdaaef0cd5519603ULL,
+ 0xebc0b08ad6d7c0e5ULL,
+ 0xb35073399de97c08ULL,
+ 0x97e4241d5f265b52ULL,
+ 0x2e52578c54e99676ULL,
+ 0x288a75c28fad917bULL,
+ 0x466e1dd37d710926ULL,
+ 0x25fd42c4845ce5fcULL,
+ 0xe0a1137fa3234b76ULL,
+ 0xfbfc21548eb90370ULL,
+ 0x8c4cf202e1e875feULL,
+ 0x9b7cd1e8a5aaa1a1ULL,
+ 0x2dec3dc9615ca561ULL,
+ 0xb4458b07977eea11ULL,
+ 0x75cea1e7a8560366ULL,
+ 0x109bf4b3b2e39a46ULL,
+ 0xdf8ed90441ba266bULL,
+ 0xd1856901e5f8bac7ULL,
+ 0x7a8b4ab2edd81a62ULL,
+ 0xb1ae8556c90ae5cbULL,
+ 0x42d15186c967771fULL,
+ 0xa07cdf5a38a872a9ULL,
+ 0x986954db9832acebULL,
+ 0xf130e68aa4fce616ULL,
+ 0xcc0caef3d0c76c52ULL,
+ 0x2c6a3fe49a7d500eULL,
+ 0x076cb3ad48ed1b85ULL,
+ 0xe6f8d0aecbb14027ULL,
+ 0x2cbe5b65d4e5c490ULL,
+ 0x9dac9ec2da0d12a7ULL,
+ 0x14ab4abbd07e5ab0ULL,
+ 0xae99ca6cb4cd1e03ULL,
+ 0x3bcd346450954fbaULL,
+ 0x2430fee89dd27a62ULL,
+ 0x7dc551166f0c94f8ULL,
+ 0x87947cbebe2a5031ULL,
+ 0x67003a8dfe2fd1b0ULL,
+ 0xfed080735857b507ULL,
+ 0x5097a42c664d9bf9ULL,
+ 0x9389590f671cf117ULL,
+ 0x9c5781b4f071956dULL,
+ 0x9c16eac9bd72017aULL,
+ 0x45186d635717d743ULL,
+ 0x3f375208bc7ce161ULL,
+ 0x768cd5d30f885c47ULL };
+} // net_instaweb
diff --git a/trunk/src/net/instaweb/util/rolling_hash_test.cc b/trunk/src/net/instaweb/util/rolling_hash_test.cc
new file mode 100644
index 0000000..7f63669
--- /dev/null
+++ b/trunk/src/net/instaweb/util/rolling_hash_test.cc
@@ -0,0 +1,111 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/util/public/rolling_hash.h"
+
+#include "net/instaweb/util/public/dense_hash_set.h"
+#include "net/instaweb/util/public/gtest.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+namespace {
+
+const char kTestString[] =
+ "The quick brown fox jumps over the lazy dog.\n"
+ "Now is the time for ALL good men to come to the aid of their party.\r\n"
+ "@$%^@#$%#^%^987293 458798\x8f\xfa\xce\t";
+const size_t kTestStringSize = sizeof(kTestString) - 1;
+
+TEST(RollingHashTest, EmptyString) {
+ EXPECT_EQ(0, RollingHash("", 0, 0));
+ EXPECT_EQ(0, RollingHash(NULL, 0, 0));
+}
+
+TEST(RollingHashTest, SingleChar) {
+ EXPECT_EQ(kRollingHashCharTable[' '], RollingHash(" ", 0, 1));
+}
+
+TEST(RollingHashTest, SingleRoll) {
+ static const char kCSpace[] = "C ";
+ uint64 h0 = RollingHash(kCSpace, 0, 1);
+ EXPECT_EQ(kRollingHashCharTable['C'], h0);
+ EXPECT_EQ(kRollingHashCharTable[' '], NextRollingHash(kCSpace, 1, 1, h0));
+}
+
+TEST(RollingHashTest, RollShakedown) {
+ for (size_t i = 1; i < kTestStringSize; ++i) {
+ uint64 hash = RollingHash(kTestString, 0, i);
+ for (size_t j = 1; j < kTestStringSize - i; ++j) {
+ hash = NextRollingHash(kTestString, j, i, hash);
+ EXPECT_EQ(RollingHash(kTestString, j, i), hash);
+ }
+ }
+}
+
+// Prove that there are no trivial 1-, 2-, or 3-gram overlaps.
+// Note that the open-vcdiff rolling hash cannot pass this test
+// as it only has 23 bits!
+TEST(RollingHashTest, NGrams) {
+ // Using dense_hash_set is MUCH faster (6x) than std::set.
+ // That keeps this test "small".
+ dense_hash_set<uint64> grams;
+ grams.set_empty_key(0ULL);
+ char buf[3];
+ uint64 hash;
+ bool gramOverlap = false;
+ for (int i = 0; i < 256; i++) {
+ buf[0] = i;
+ hash = RollingHash(buf, 0, 1);
+ ASSERT_NE(0, hash);
+ if (!grams.insert(hash).second) {
+ gramOverlap = true;
+ printf("Gram overlap including %2x", i);
+ }
+ }
+ for (int i = 0; i < 256; i++) {
+ buf[0] = i;
+ for (int j = 0; j < 256; j++) {
+ buf[1] = j;
+ hash = RollingHash(buf, 0, 2);
+ ASSERT_NE(0, hash);
+ if (!grams.insert(hash).second) {
+ gramOverlap = true;
+ printf("Gram overlap including %2x %2x", i, j);
+ }
+ }
+ }
+ for (int i = 0; i < 256; i++) {
+ buf[0] = i;
+ for (int j = 0; j < 256; j++) {
+ buf[1] = j;
+ for (int k = 0; k < 256; k++) {
+ buf[2] = k;
+ hash = RollingHash(buf, 0, 3);
+ ASSERT_NE(0, hash);
+ if (!grams.insert(hash).second) {
+ gramOverlap = true;
+ printf("Gram overlap including %2x %2x %2x\n", i, j, k);
+ }
+ }
+ }
+ }
+ EXPECT_FALSE(gramOverlap);
+}
+
+} // namespace
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/simple_meta_data.cc b/trunk/src/net/instaweb/util/simple_meta_data.cc
new file mode 100644
index 0000000..d1e25b5
--- /dev/null
+++ b/trunk/src/net/instaweb/util/simple_meta_data.cc
@@ -0,0 +1,380 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/simple_meta_data.h"
+
+#include <stdio.h>
+#include "base/logging.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/timer.h"
+#include "net/instaweb/util/public/time_util.h"
+#include "net/instaweb/util/public/writer.h"
+#include "pagespeed/core/resource_util.h"
+
+namespace {
+
+const int64 TIME_UNINITIALIZED = -1;
+const int64 kImplicitCacheTtlMs = 5 * net_instaweb::Timer::kMinuteMs;
+
+}
+
+namespace net_instaweb {
+
+SimpleMetaData::SimpleMetaData() {
+ Clear();
+}
+
+SimpleMetaData::~SimpleMetaData() {
+ Clear();
+}
+
+void SimpleMetaData::Clear() {
+ for (int i = 0, n = attribute_vector_.size(); i < n; ++i) {
+ delete [] attribute_vector_[i].second;
+ }
+ attribute_map_.clear();
+ attribute_vector_.clear();
+
+ parsing_http_ = false;
+ parsing_value_ = false;
+ headers_complete_ = false;
+ cache_fields_dirty_ = false;
+ is_cacheable_ = false;
+ is_proxy_cacheable_ = false; // accurate only if !cache_fields_dirty_
+ expiration_time_ms_= TIME_UNINITIALIZED;
+ timestamp_ms_= TIME_UNINITIALIZED;
+ parse_name_.clear();
+ parse_value_.clear();
+
+ major_version_ = 0;
+ minor_version_ = 0;
+ status_code_ = 0;
+ reason_phrase_.clear();
+}
+
+int SimpleMetaData::NumAttributes() const {
+ return attribute_vector_.size();
+}
+
+const char* SimpleMetaData::Name(int index) const {
+ return attribute_vector_[index].first;
+}
+
+const char* SimpleMetaData::Value(int index) const {
+ return attribute_vector_[index].second;
+}
+
+bool SimpleMetaData::Lookup(const char* name, CharStarVector* values) const {
+ AttributeMap::const_iterator p = attribute_map_.find(name);
+ bool ret = false;
+ if (p != attribute_map_.end()) {
+ ret = true;
+ *values = p->second;
+ }
+ return ret;
+}
+
+void SimpleMetaData::Add(const StringPiece& name, const StringPiece& value) {
+ // TODO(jmarantz): Parse comma-separated values. bmcquade sez:
+ // you probably want to normalize these by splitting on commas and
+ // adding a separate k,v pair for each comma-separated value. then
+ // it becomes very easy to do things like search for individual
+ // Content-Type tokens. Otherwise the client has to assume that
+ // every single value could be comma-separated and they have to
+ // parse it as such. the list of header names that are not safe to
+ // comma-split is at
+ // http://src.chromium.org/viewvc/chrome/trunk/src/net/http/http_util.cc
+ // (search for IsNonCoalescingHeader)
+
+ CharStarVector dummy_values;
+ std::pair<AttributeMap::iterator, bool> iter_inserted =
+ attribute_map_.insert(AttributeMap::value_type(name.as_string(),
+ dummy_values));
+ AttributeMap::iterator iter = iter_inserted.first;
+ CharStarVector& values = iter->second;
+ int value_buf_size = value.size() + 1;
+ char* value_copy = new char[value_buf_size];
+ memcpy(value_copy, value.data(), value_buf_size);
+ values.push_back(value_copy);
+ attribute_vector_.push_back(StringPair(iter->first.c_str(), value_copy));
+ cache_fields_dirty_ = true;
+}
+
+void SimpleMetaData::RemoveAll(const char* name) {
+ AttributeVector temp_vector; // Temp variable for new vector.
+ temp_vector.reserve(attribute_vector_.size());
+ for (int i = 0; i < NumAttributes(); ++i) {
+ if (strcasecmp(Name(i), name) != 0) {
+ temp_vector.push_back(attribute_vector_[i]);
+ } else {
+ delete [] attribute_vector_[i].second;
+ }
+ }
+ attribute_vector_.swap(temp_vector);
+
+ // Note: we have to erase from the map second, because map owns the name.
+ attribute_map_.erase(name);
+}
+
+// Serialize meta-data to a stream.
+bool SimpleMetaData::Write(Writer* writer, MessageHandler* handler) const {
+ bool ret = true;
+ char buf[100];
+ snprintf(buf, sizeof(buf), "HTTP/%d.%d %d ",
+ major_version_, minor_version_, status_code_);
+ ret &= writer->Write(buf, handler);
+ ret &= writer->Write(reason_phrase_, handler);
+ ret &= writer->Write("\r\n", handler);
+ ret &= WriteHeaders(writer, handler);
+ return ret;
+}
+
+bool SimpleMetaData::WriteHeaders(Writer* writer,
+ MessageHandler* handler) const {
+ bool ret = true;
+ for (int i = 0, n = attribute_vector_.size(); ret && (i < n); ++i) {
+ const StringPair& attribute = attribute_vector_[i];
+ ret &= writer->Write(attribute.first, handler);
+ ret &= writer->Write(": ", handler);
+ ret &= writer->Write(attribute.second, handler);
+ ret &= writer->Write("\r\n", handler);
+ }
+ ret &= writer->Write("\r\n", handler);
+ return ret;
+}
+
+// TODO(jmaessen): http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2
+// I bet we're doing this wrong:
+// Header fields can be extended over multiple lines by preceding each extra
+// line with at least one SP or HT.
+int SimpleMetaData::ParseChunk(const StringPiece& text,
+ MessageHandler* handler) {
+ CHECK(!headers_complete_);
+ int num_consumed = 0;
+ int num_bytes = text.size();
+
+ for (; num_consumed < num_bytes; ++num_consumed) {
+ char c = text[num_consumed];
+ if ((c == '/') && (parse_name_ == "HTTP")) {
+ if (major_version_ != 0) {
+ handler->Message(kError, "Multiple HTTP Lines");
+ } else {
+ parsing_http_ = true;
+ parsing_value_ = true;
+ }
+ } else if (!parsing_value_ && (c == ':')) {
+ parsing_value_ = true;
+ } else if (c == '\r') {
+ // Just ignore CRs for now, and break up headers on newlines for
+ // simplicity. It's not clear to me if it's important that we
+ // reject headers that lack the CR in front of the LF.
+ } else if (c == '\n') {
+ if (parse_name_.empty()) {
+ // blank line. This marks the end of the headers.
+ ++num_consumed;
+ headers_complete_ = true;
+ ComputeCaching();
+ break;
+ }
+ if (parsing_http_) {
+ // Parsing "1.0 200 OK\r", using sscanf for the integers, and
+ // private method GrabLastToken for the "OK".
+ if ((sscanf(parse_value_.c_str(), "%d.%d %d ", // NOLINT
+ &major_version_, &minor_version_, &status_code_) != 3) ||
+ !GrabLastToken(parse_value_, &reason_phrase_)) {
+ // TODO(jmarantz): capture the filename/url, track the line numbers.
+ handler->Message(kError, "Invalid HTML headers: %s",
+ parse_value_.c_str());
+ }
+ parsing_http_ = false;
+ } else {
+ Add(parse_name_.c_str(), parse_value_.c_str());
+ }
+ parsing_value_ = false;
+ parse_name_.clear();
+ parse_value_.clear();
+ } else if (parsing_value_) {
+ // Skip leading whitespace
+ if (!parse_value_.empty() || !isspace(c)) {
+ parse_value_ += c;
+ }
+ } else {
+ parse_name_ += c;
+ }
+ }
+ return num_consumed;
+}
+
+// Specific information about cache. This is all embodied in the
+// headers but is centrally parsed so we can try to get it right.
+bool SimpleMetaData::IsCacheable() const {
+ // We do not compute caching from accessors so that the
+ // accessors can be easier to call from multiple threads
+ // without mutexing.
+ CHECK(!cache_fields_dirty_);
+ return is_cacheable_;
+}
+
+bool SimpleMetaData::IsProxyCacheable() const {
+ CHECK(!cache_fields_dirty_);
+ return is_proxy_cacheable_;
+}
+
+// Returns the ms-since-1970 absolute time when this resource
+// should be expired out of caches.
+int64 SimpleMetaData::CacheExpirationTimeMs() const {
+ CHECK(!cache_fields_dirty_);
+ return expiration_time_ms_;
+}
+
+void SimpleMetaData::SetDate(int64 date_ms) {
+ std::string time_string;
+ if (ConvertTimeToString(date_ms, &time_string)) {
+ Add("Date", time_string.c_str());
+ }
+}
+
+void SimpleMetaData::SetLastModified(int64 last_modified_ms) {
+ std::string time_string;
+ if (ConvertTimeToString(last_modified_ms, &time_string)) {
+ Add(HttpAttributes::kLastModified, time_string.c_str());
+ }
+}
+
+void SimpleMetaData::ComputeCaching() {
+ pagespeed::Resource resource;
+ for (int i = 0, n = NumAttributes(); i < n; ++i) {
+ resource.AddResponseHeader(Name(i), Value(i));
+ }
+ resource.SetResponseStatusCode(status_code_);
+
+ CharStarVector values;
+ int64 date;
+ // Compute the timestamp if we can find it
+ if (Lookup("Date", &values) && (values.size() == 1) &&
+ ConvertStringToTime(values[0], &date)) {
+ timestamp_ms_ = date;
+ }
+
+ // TODO(jmarantz): Should we consider as cacheable a resource
+ // that simply has no cacheable hints at all? For now, let's
+ // make that assumption. We should review this policy with bmcquade,
+ // souders, etc, but first let's try to measure some value with this
+ // optimistic intrepretation.
+ //
+ // TODO(jmarantz): get from bmcquade a comprehensive ways in which these
+ // policies will differ for Instaweb vs Pagespeed.
+ bool explicit_no_cache =
+ pagespeed::resource_util::HasExplicitNoCacheDirective(resource);
+ bool likely_static =
+ pagespeed::resource_util::IsLikelyStaticResource(resource);
+
+ // status_cacheable implies that either the resource content was
+ // cacheable, or the status code indicated some other aspect of
+ // our system that we want to remember in the cache, such as
+ // that fact that a fetch failed for a resource, and we don't want
+ // to try again until some time has passed.
+ bool status_cacheable =
+ ((status_code_ == HttpStatus::kRememberNotFoundStatusCode) ||
+ pagespeed::resource_util::IsCacheableResourceStatusCode(status_code_));
+ int64 freshness_lifetime_ms;
+ bool explicit_cacheable =
+ pagespeed::resource_util::GetFreshnessLifetimeMillis(
+ resource, &freshness_lifetime_ms) && has_timestamp_ms();
+ is_cacheable_ = (!explicit_no_cache &&
+ (explicit_cacheable || likely_static) &&
+ status_cacheable);
+
+ if (is_cacheable_) {
+ if (explicit_cacheable) {
+ // TODO(jmarantz): check "Age" resource and use that to reduce
+ // the expiration_time_ms_. This is, says, bmcquade@google.com,
+ // typically use to indicate how long a resource has been sitting
+ // in a proxy-cache.
+ expiration_time_ms_ = timestamp_ms_ + freshness_lifetime_ms;
+ } else {
+ // implicitly cached items stay alive in our system for 5 minutes
+ // TODO(jmarantz): consider making this a flag, or getting some
+ // other heuristic value from the PageSpeed libraries.
+ expiration_time_ms_ = timestamp_ms_ + kImplicitCacheTtlMs;
+ }
+
+ // Assume it's proxy cacheable. Then iterate over all the headers
+ // with key HttpAttributes::kCacheControl, and all the comma-separated
+ // values within those values, and look for 'private'.
+ is_proxy_cacheable_ = true;
+ values.clear();
+ if (Lookup(HttpAttributes::kCacheControl, &values)) {
+ for (int i = 0, n = values.size(); i < n; ++i) {
+ const char* cache_control = values[i];
+ pagespeed::resource_util::DirectiveMap directive_map;
+ if (pagespeed::resource_util::GetHeaderDirectives(
+ cache_control, &directive_map)) {
+ pagespeed::resource_util::DirectiveMap::iterator p =
+ directive_map.find("private");
+ if (p != directive_map.end()) {
+ is_proxy_cacheable_ = false;
+ break;
+ }
+ }
+ }
+ }
+ } else {
+ expiration_time_ms_ = 0;
+ is_proxy_cacheable_ = false;
+ }
+ cache_fields_dirty_ = false;
+}
+
+bool SimpleMetaData::has_timestamp_ms() const {
+ return timestamp_ms_ != TIME_UNINITIALIZED;
+}
+
+std::string SimpleMetaData::ToString() const {
+ std::string str;
+ StringWriter writer(&str);
+ Write(&writer, NULL);
+ return str;
+}
+
+// Grabs the last non-whitespace token from 'input' and puts it in 'output'.
+bool SimpleMetaData::GrabLastToken(const std::string& input,
+ std::string* output) {
+ bool ret = false;
+ // Safely grab the response code string from the end of parse_value_.
+ int last_token_char = -1;
+ for (int i = input.size() - 1; i >= 0; --i) {
+ char c = input[i];
+ if (isspace(c)) {
+ if (last_token_char >= 0) {
+ // We found the whole token.
+ const char* token_start = input.c_str() + i + 1;
+ int token_len = last_token_char - i;
+ output->append(token_start, token_len);
+ ret = true;
+ break;
+ }
+ } else if (last_token_char == -1) {
+ last_token_char = i;
+ }
+ }
+ return ret;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/simple_stats.cc b/trunk/src/net/instaweb/util/simple_stats.cc
new file mode 100644
index 0000000..19e0b66
--- /dev/null
+++ b/trunk/src/net/instaweb/util/simple_stats.cc
@@ -0,0 +1,34 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/simple_stats.h"
+
+namespace net_instaweb {
+
+SimpleStatsVariable::~SimpleStatsVariable() {
+}
+
+SimpleStats::~SimpleStats() {
+}
+
+SimpleStatsVariable* SimpleStats::NewVariable(
+ const StringPiece& name, int index) {
+ return new SimpleStatsVariable;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/stack_buffer.h b/trunk/src/net/instaweb/util/stack_buffer.h
new file mode 100644
index 0000000..8013b93
--- /dev/null
+++ b/trunk/src/net/instaweb/util/stack_buffer.h
@@ -0,0 +1,32 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Constant for allocating stack buffers.
+
+#ifndef NET_INSTAWEB_UTIL_STACK_BUFFER_H_
+#define NET_INSTAWEB_UTIL_STACK_BUFFER_H_
+
+namespace net_instaweb {
+
+// Size of stack buffer for read-blocks. This can't be too big or it will blow
+// the stack, which may be set small in multi-threaded environments.
+const int kStackBufferSize = 10000;
+
+} // namespace net_instaweb
+
+#endif // NET_INSTAWEB_UTIL_STACK_BUFFER_H_
diff --git a/trunk/src/net/instaweb/util/statistics.cc b/trunk/src/net/instaweb/util/statistics.cc
new file mode 100644
index 0000000..73ee39c
--- /dev/null
+++ b/trunk/src/net/instaweb/util/statistics.cc
@@ -0,0 +1,29 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/statistics.h"
+
+namespace net_instaweb {
+
+Variable::~Variable() {
+}
+
+Statistics::~Statistics() {
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/statistics_work_bound.cc b/trunk/src/net/instaweb/util/statistics_work_bound.cc
new file mode 100644
index 0000000..7acc770
--- /dev/null
+++ b/trunk/src/net/instaweb/util/statistics_work_bound.cc
@@ -0,0 +1,51 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/util/public/statistics_work_bound.h"
+
+#include "net/instaweb/util/public/statistics.h"
+
+namespace net_instaweb {
+
+StatisticsWorkBound::StatisticsWorkBound(Variable* variable, int bound)
+ : variable_((bound == 0) ? NULL : variable), bound_(bound) { }
+StatisticsWorkBound::~StatisticsWorkBound() { }
+
+bool StatisticsWorkBound::TryToWork() {
+ bool ok = true;
+ if (variable_ != NULL) {
+ // We conservatively increment, then test, and decrement on failure. This
+ // guarantees that two incrementors don't both get through when we're within
+ // 1 of the bound, at the cost of occasionally rejecting them both.
+ variable_->Add(1);
+ ok = (variable_->Get() <= bound_);
+ if (!ok) {
+ variable_->Add(-1);
+ }
+ }
+ return ok;
+}
+
+void StatisticsWorkBound::WorkComplete() {
+ if (variable_ != NULL) {
+ variable_->Add(-1);
+ }
+}
+
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/statistics_work_bound_test.cc b/trunk/src/net/instaweb/util/statistics_work_bound_test.cc
new file mode 100644
index 0000000..56bab8a
--- /dev/null
+++ b/trunk/src/net/instaweb/util/statistics_work_bound_test.cc
@@ -0,0 +1,122 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+// Unit-test the statistics work bound.
+
+#include "net/instaweb/util/public/statistics_work_bound.h"
+
+#include "base/scoped_ptr.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/simple_stats.h"
+
+namespace net_instaweb {
+
+namespace {
+
+class StatisticsWorkBoundTest : public testing::Test {
+ public:
+ StatisticsWorkBoundTest()
+ : stats_(),
+ var1_(stats_.AddVariable("var1")),
+ var2_(stats_.AddVariable("var2")) { }
+
+ protected:
+ SimpleStats stats_;
+ Variable* var1_;
+ Variable* var2_;
+
+ StatisticsWorkBound* MakeBound(Variable* var, int bound) {
+ StatisticsWorkBound* result = new StatisticsWorkBound(var, bound);
+ CHECK(NULL != result);
+ return result;
+ }
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(StatisticsWorkBoundTest);
+};
+
+// Test with a bound of two.
+TEST_F(StatisticsWorkBoundTest, TestTwoBound) {
+ // We allocate two objects backed by the same statistic,
+ // to ensure that they share a common count. This is what
+ // happens in a multi-process setting.
+ scoped_ptr<StatisticsWorkBound> bound1(MakeBound(var1_, 2));
+ scoped_ptr<StatisticsWorkBound> bound2(MakeBound(var1_, 2));
+ // Repeat twice to ensure that we actually made it back to 0.
+ for (int i = 0; i < 2; i++) {
+ // Start with no workers.
+ EXPECT_TRUE(bound1->TryToWork());
+ // One worker here.
+ EXPECT_TRUE(bound1->TryToWork());
+ EXPECT_FALSE(bound1->TryToWork());
+ EXPECT_FALSE(bound2->TryToWork());
+ bound1->WorkComplete();
+ // One worker here.
+ EXPECT_TRUE(bound2->TryToWork());
+ EXPECT_FALSE(bound1->TryToWork());
+ EXPECT_FALSE(bound2->TryToWork());
+ bound1->WorkComplete();
+ // Back to one worker.
+ EXPECT_TRUE(bound2->TryToWork());
+ EXPECT_FALSE(bound1->TryToWork());
+ EXPECT_FALSE(bound2->TryToWork());
+ bound2->WorkComplete();
+ // Back to one worker.
+ bound2->WorkComplete();
+ // Back to none.
+ }
+}
+
+// Test that a bound of 0 allows large # of tries.
+TEST_F(StatisticsWorkBoundTest, TestZeroBound) {
+ scoped_ptr<StatisticsWorkBound> bound1(MakeBound(var1_, 0));
+ scoped_ptr<StatisticsWorkBound> bound2(MakeBound(var1_, 0));
+ for (int i = 0; i < 1000; ++i) {
+ EXPECT_TRUE(bound1->TryToWork());
+ EXPECT_TRUE(bound2->TryToWork());
+ }
+}
+
+// Test that absent variable allows large # of tries.
+TEST_F(StatisticsWorkBoundTest, TestNullVar) {
+ scoped_ptr<StatisticsWorkBound> bound1(MakeBound(NULL, 2));
+ scoped_ptr<StatisticsWorkBound> bound2(MakeBound(NULL, 2));
+ for (int i = 0; i < 1000; ++i) {
+ EXPECT_TRUE(bound1->TryToWork());
+ EXPECT_TRUE(bound2->TryToWork());
+ }
+}
+
+// Test that differently-named bounds are distinct
+TEST_F(StatisticsWorkBoundTest, TestDistinctVar) {
+ scoped_ptr<StatisticsWorkBound> bound1(MakeBound(var1_, 2));
+ scoped_ptr<StatisticsWorkBound> bound2(MakeBound(var2_, 2));
+ EXPECT_TRUE(bound1->TryToWork());
+ EXPECT_TRUE(bound1->TryToWork());
+ EXPECT_FALSE(bound1->TryToWork());
+ EXPECT_TRUE(bound2->TryToWork());
+ EXPECT_TRUE(bound2->TryToWork());
+ EXPECT_FALSE(bound2->TryToWork());
+ bound1->WorkComplete();
+ EXPECT_FALSE(bound2->TryToWork());
+ EXPECT_TRUE(bound1->TryToWork());
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/stdio_file_system.cc b/trunk/src/net/instaweb/util/stdio_file_system.cc
new file mode 100644
index 0000000..f9abd9c
--- /dev/null
+++ b/trunk/src/net/instaweb/util/stdio_file_system.cc
@@ -0,0 +1,379 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/stdio_file_system.h"
+
+#include <dirent.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/errno.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include <string>
+
+namespace net_instaweb {
+
+// Helper class to factor out common implementation details between Input and
+// Output files, in lieu of multiple inheritance.
+class StdioFileHelper {
+ public:
+ StdioFileHelper(FILE* f, const StringPiece& filename)
+ : file_(f),
+ line_(1) {
+ filename.CopyToString(&filename_);
+ }
+
+ ~StdioFileHelper() {
+ CHECK(file_ == NULL);
+ }
+
+ void CountNewlines(const char* buf, int size) {
+ for (int i = 0; i < size; ++i, ++buf) {
+ line_ += (*buf == '\n');
+ }
+ }
+
+ void ReportError(MessageHandler* message_handler, const char* format) {
+ message_handler->Error(filename_.c_str(), line_, format, strerror(errno));
+ }
+
+ bool Close(MessageHandler* message_handler) {
+ bool ret = true;
+ if (file_ != stdout && file_ != stderr && file_ != stdin) {
+ if (fclose(file_) != 0) {
+ ReportError(message_handler, "closing file: %s");
+ ret = false;
+ }
+ }
+ file_ = NULL;
+ return ret;
+ }
+
+ FILE* file_;
+ std::string filename_;
+ int line_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(StdioFileHelper);
+};
+
+class StdioInputFile : public FileSystem::InputFile {
+ public:
+ StdioInputFile(FILE* f, const StringPiece& filename)
+ : file_helper_(f, filename) {
+ }
+
+ virtual int Read(char* buf, int size, MessageHandler* message_handler) {
+ int ret = fread(buf, 1, size, file_helper_.file_);
+ file_helper_.CountNewlines(buf, ret);
+ if ((ret == 0) && (ferror(file_helper_.file_) != 0)) {
+ file_helper_.ReportError(message_handler, "reading file: %s");
+ }
+ return ret;
+ }
+
+ virtual bool Close(MessageHandler* message_handler) {
+ return file_helper_.Close(message_handler);
+ }
+
+ virtual const char* filename() { return file_helper_.filename_.c_str(); }
+
+ private:
+ StdioFileHelper file_helper_;
+
+ DISALLOW_COPY_AND_ASSIGN(StdioInputFile);
+};
+
+class StdioOutputFile : public FileSystem::OutputFile {
+ public:
+ StdioOutputFile(FILE* f, const StringPiece& filename)
+ : file_helper_(f, filename) {
+ }
+
+ virtual bool Write(const StringPiece& buf, MessageHandler* handler) {
+ size_t bytes_written =
+ fwrite(buf.data(), 1, buf.size(), file_helper_.file_);
+ file_helper_.CountNewlines(buf.data(), bytes_written);
+ bool ret = (bytes_written == buf.size());
+ if (!ret) {
+ file_helper_.ReportError(handler, "writing file: %s");
+ }
+ return ret;
+ }
+
+ virtual bool Flush(MessageHandler* message_handler) {
+ bool ret = true;
+ if (fflush(file_helper_.file_) != 0) {
+ file_helper_.ReportError(message_handler, "flushing file: %s");
+ ret = false;
+ }
+ return ret;
+ }
+
+ virtual bool Close(MessageHandler* message_handler) {
+ return file_helper_.Close(message_handler);
+ }
+
+ virtual const char* filename() { return file_helper_.filename_.c_str(); }
+
+ virtual bool SetWorldReadable(MessageHandler* message_handler) {
+ bool ret = true;
+ int fd = fileno(file_helper_.file_);
+ int status = fchmod(fd, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);
+ if (status != 0) {
+ ret = false;
+ file_helper_.ReportError(message_handler, "setting world-readble: %s");
+ }
+ return ret;
+ }
+
+ private:
+ StdioFileHelper file_helper_;
+
+ DISALLOW_COPY_AND_ASSIGN(StdioOutputFile);
+};
+
+StdioFileSystem::~StdioFileSystem() {
+}
+
+FileSystem::InputFile* StdioFileSystem::OpenInputFile(
+ const char* filename, MessageHandler* message_handler) {
+ FileSystem::InputFile* input_file = NULL;
+ FILE* f = fopen(filename, "r");
+ if (f == NULL) {
+ message_handler->Error(filename, 0, "opening input file: %s",
+ strerror(errno));
+ } else {
+ input_file = new StdioInputFile(f, filename);
+ }
+ return input_file;
+}
+
+
+FileSystem::OutputFile* StdioFileSystem::OpenOutputFileHelper(
+ const char* filename, MessageHandler* message_handler) {
+ FileSystem::OutputFile* output_file = NULL;
+ if (strcmp(filename, "-") == 0) {
+ output_file = new StdioOutputFile(stdout, "<stdout>");
+ } else {
+ FILE* f = fopen(filename, "w");
+ if (f == NULL) {
+ message_handler->Error(filename, 0,
+ "opening output file: %s", strerror(errno));
+ } else {
+ output_file = new StdioOutputFile(f, filename);
+ }
+ }
+ return output_file;
+}
+
+FileSystem::OutputFile* StdioFileSystem::OpenTempFileHelper(
+ const StringPiece& prefix, MessageHandler* message_handler) {
+ // TODO(jmarantz): As jmaessen points out, mkstemp warns "Don't use
+ // this function, use tmpfile(3) instead. It is better defined and
+ // more portable." However, tmpfile does not allow a location to be
+ // specified. I'm not 100% sure if that's going to be work well for
+ // us. More importantly, our usage scenario is that we will be
+ // closing the file and renaming it to a permanent name. tmpfiles
+ // automatically are deleted when they are closed.
+ int prefix_len = prefix.length();
+ static char mkstemp_hook[] = "XXXXXX";
+ char* template_name = new char[prefix_len + sizeof(mkstemp_hook)];
+ memcpy(template_name, prefix.data(), prefix_len);
+ memcpy(template_name + prefix_len, mkstemp_hook, sizeof(mkstemp_hook));
+ int fd = mkstemp(template_name);
+ OutputFile* output_file = NULL;
+ if (fd < 0) {
+ message_handler->Error(template_name, 0,
+ "opening temp file: %s", strerror(errno));
+ } else {
+ FILE* f = fdopen(fd, "w");
+ if (f == NULL) {
+ close(fd);
+ message_handler->Error(template_name, 0,
+ "re-opening temp file: %s", strerror(errno));
+ } else {
+ output_file = new StdioOutputFile(f, template_name);
+ }
+ }
+ delete [] template_name;
+ return output_file;
+}
+
+
+bool StdioFileSystem::RemoveFile(const char* filename,
+ MessageHandler* handler) {
+ bool ret = (remove(filename) == 0);
+ if (!ret) {
+ handler->Message(kError, "Failed to delete file %s: %s",
+ filename, strerror(errno));
+ }
+ return ret;
+}
+
+bool StdioFileSystem::RenameFileHelper(const char* old_file,
+ const char* new_file,
+ MessageHandler* handler) {
+ bool ret = (rename(old_file, new_file) == 0);
+ if (!ret) {
+ handler->Message(kError, "Failed to rename file %s to %s: %s",
+ old_file, new_file, strerror(errno));
+ }
+ return ret;
+}
+
+bool StdioFileSystem::MakeDir(const char* path, MessageHandler* handler) {
+ // Mode 0777 makes the file use standard umask permissions.
+ bool ret = (mkdir(path, 0777) == 0);
+ if (!ret) {
+ handler->Message(kError, "Failed to make directory %s: %s",
+ path, strerror(errno));
+ }
+ return ret;
+}
+
+BoolOrError StdioFileSystem::Exists(const char* path, MessageHandler* handler) {
+ struct stat statbuf;
+ BoolOrError ret(stat(path, &statbuf) == 0);
+ if (ret.is_false() && errno != ENOENT) { // Not error if file doesn't exist.
+ handler->Message(kError, "Failed to stat %s: %s",
+ path, strerror(errno));
+ ret.set_error();
+ }
+ return ret;
+}
+
+BoolOrError StdioFileSystem::IsDir(const char* path, MessageHandler* handler) {
+ struct stat statbuf;
+ BoolOrError ret(false);
+ if (stat(path, &statbuf) == 0) {
+ ret.set(S_ISDIR(statbuf.st_mode));
+ } else if (errno != ENOENT) { // Not an error if file doesn't exist.
+ handler->Message(kError, "Failed to stat %s: %s",
+ path, strerror(errno));
+ ret.set_error();
+ }
+ return ret;
+}
+
+bool StdioFileSystem::ListContents(const StringPiece& dir, StringVector* files,
+ MessageHandler* handler) {
+ std::string dir_string = dir.as_string();
+ EnsureEndsInSlash(&dir_string);
+ const char* dirname = dir_string.c_str();
+ DIR* mydir = opendir(dirname);
+ if (mydir == NULL) {
+ handler->Error(dirname, 0, "Failed to opendir: %s", strerror(errno));
+ return false;
+ } else {
+ dirent* entry = NULL;
+ dirent buffer;
+ while (readdir_r(mydir, &buffer, &entry) == 0 && entry != NULL) {
+ if ((strcmp(entry->d_name, ".") != 0) &&
+ (strcmp(entry->d_name, "..") != 0)) {
+ files->push_back(dir_string + entry->d_name);
+ }
+ }
+ if (closedir(mydir) != 0) {
+ handler->Error(dirname, 0, "Failed to closedir: %s", strerror(errno));
+ return false;
+ }
+ return true;
+ }
+}
+
+bool StdioFileSystem::Atime(const StringPiece& path, int64* timestamp_sec,
+ MessageHandler* handler) {
+ // TODO(abliss): there are some situations where this doesn't work
+ // -- e.g. if the filesystem is mounted noatime. We should try to
+ // detect that and provide a workaround.
+ const std::string path_string = path.as_string();
+ const char* path_str = path_string.c_str();
+ struct stat statbuf;
+ if (stat(path_str, &statbuf) == 0) {
+ *timestamp_sec = statbuf.st_atime;
+ return true;
+ } else {
+ handler->Message(kError, "Failed to stat %s: %s",
+ path_str, strerror(errno));
+ return false;
+ }
+}
+
+bool StdioFileSystem::Size(const StringPiece& path, int64* size,
+ MessageHandler* handler) {
+ const std::string path_string = path.as_string();
+ const char* path_str = path_string.c_str();
+ struct stat statbuf;
+ if (stat(path_str, &statbuf) == 0) {
+ *size = statbuf.st_size;
+ return true;
+ } else {
+ handler->Message(kError, "Failed to stat %s: %s",
+ path_str, strerror(errno));
+ return false;
+ }
+}
+
+BoolOrError StdioFileSystem::TryLock(const StringPiece& lock_name,
+ MessageHandler* handler) {
+ const std::string lock_string = lock_name.as_string();
+ const char* lock_str = lock_string.c_str();
+ // POSIX mkdir is widely believed to be atomic, although I have
+ // found no reliable documentation of this fact.
+ if (mkdir(lock_str, 0777) == 0) {
+ return BoolOrError(true);
+ } else if (errno == EEXIST) {
+ return BoolOrError(false);
+ } else {
+ handler->Message(kError, "Failed to mkdir %s: %s",
+ lock_str, strerror(errno));
+ return BoolOrError();
+ }
+}
+
+bool StdioFileSystem::Unlock(const StringPiece& lock_name,
+ MessageHandler* handler) {
+ const std::string lock_string = lock_name.as_string();
+ const char* lock_str = lock_string.c_str();
+ if (rmdir(lock_str) == 0) {
+ return true;
+ } else {
+ handler->Message(kError, "Failed to rmdir %s: %s",
+ lock_str, strerror(errno));
+ return false;
+ }
+}
+
+FileSystem::InputFile* StdioFileSystem::Stdin() {
+ return new StdioInputFile(stdin, "stdin");
+}
+
+FileSystem::OutputFile* StdioFileSystem::Stdout() {
+ return new StdioOutputFile(stdout, "stdout");
+}
+
+FileSystem::OutputFile* StdioFileSystem::Stderr() {
+ return new StdioOutputFile(stderr, "stderr");
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/string_convert.cc b/trunk/src/net/instaweb/util/string_convert.cc
new file mode 100644
index 0000000..2316518
--- /dev/null
+++ b/trunk/src/net/instaweb/util/string_convert.cc
@@ -0,0 +1,49 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+bool AccumulateDecimalValue(char c, int* value) {
+ bool ret = true;
+ if ((c >= '0') && (c <= '9')) {
+ *value *= 10;
+ *value += c - '0';
+ } else {
+ ret = false;
+ }
+ return ret;
+}
+
+bool AccumulateHexValue(char c, int* value) {
+ int digit = 0;
+ if ((c >= '0') && (c <= '9')) {
+ digit = c - '0';
+ } else if ((c >= 'a') && (c <= 'f')) {
+ digit = 10 + c - 'a';
+ } else if ((c >= 'A') && (c <= 'F')) {
+ digit = 10 + c - 'A';
+ } else {
+ return false;
+ }
+ *value = *value * 16 + digit;
+ return true;
+}
+
+} // namespace
diff --git a/trunk/src/net/instaweb/util/string_util.cc b/trunk/src/net/instaweb/util/string_util.cc
new file mode 100644
index 0000000..990321b
--- /dev/null
+++ b/trunk/src/net/instaweb/util/string_util.cc
@@ -0,0 +1,152 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/string_util.h"
+#include <vector>
+
+namespace net_instaweb {
+
+void SplitStringPieceToVector(const StringPiece& sp, const char* separator,
+ std::vector<StringPiece>* components,
+ bool omit_empty_strings) {
+ size_t prev_pos = 0;
+ size_t pos = 0;
+ StringPiece sep(separator);
+ while ((pos = sp.find(sep, pos)) != StringPiece::npos) {
+ if (!omit_empty_strings || (pos > prev_pos)) {
+ components->push_back(sp.substr(prev_pos, pos - prev_pos));
+ }
+ ++pos;
+ prev_pos = pos;
+ }
+ if (!omit_empty_strings || (prev_pos < sp.size())) {
+ components->push_back(sp.substr(prev_pos, prev_pos - sp.size()));
+ }
+}
+
+std::string StrCat(const StringPiece& a,
+ const StringPiece& b,
+ const StringPiece& c,
+ const StringPiece& d,
+ const StringPiece& e,
+ const StringPiece& f,
+ const StringPiece& g,
+ const StringPiece& h) {
+ std::string res;
+ res.reserve(a.size() + b.size() + c.size() + d.size() + e.size() + f.size() +
+ g.size() + h.size());
+ a.AppendToString(&res);
+ b.AppendToString(&res);
+ c.AppendToString(&res);
+ d.AppendToString(&res);
+ e.AppendToString(&res);
+ f.AppendToString(&res);
+ g.AppendToString(&res);
+ h.AppendToString(&res);
+ return res;
+}
+
+void BackslashEscape(const StringPiece& src,
+ const StringPiece& to_escape,
+ std::string* dest) {
+ dest->reserve(dest->size() + src.size());
+ for (const char *p = src.data(), *end = src.data() + src.size();
+ p != end; ++p) {
+ if (to_escape.find(*p) != StringPiece::npos) {
+ dest->push_back('\\');
+ }
+ dest->push_back(*p);
+ }
+}
+
+bool StringCaseEqual(const StringPiece& s1, const StringPiece& s2) {
+ return (s1.size() == s2.size() &&
+ 0 == strncasecmp(s1.data(), s2.data(), s1.size()));
+}
+
+bool StringCaseStartsWith(const StringPiece& str, const StringPiece& prefix) {
+ return (str.size() >= prefix.size() &&
+ 0 == strncasecmp(str.data(), prefix.data(), prefix.size()));
+}
+
+bool StringCaseEndsWith(const StringPiece& str, const StringPiece& suffix) {
+ return (str.size() >= suffix.size() &&
+ 0 == strncasecmp(str.data() + str.size() - suffix.size(),
+ suffix.data(), suffix.size()));
+}
+
+void ParseShellLikeString(const StringPiece& input,
+ std::vector<std::string>* output) {
+ output->clear();
+ for (size_t index = 0; index < input.size();) {
+ const char ch = input[index];
+ // If we see a quoted section, treat it as a single item even if there are
+ // spaces in it.
+ if (ch == '"' || ch == '\'') {
+ const char quote = ch;
+ ++index; // skip open quote
+ output->push_back("");
+ std::string& part = output->back();
+ for (; index < input.size() && input[index] != quote; ++index) {
+ if (input[index] == '\\') {
+ ++index; // skip backslash
+ if (index >= input.size()) {
+ break;
+ }
+ }
+ part.push_back(input[index]);
+ }
+ ++index; // skip close quote
+ }
+ // Without quotes, items are whitespace-separated.
+ else if (!isspace(ch)) {
+ output->push_back("");
+ std::string& part = output->back();
+ for (;index < input.size() && !isspace(input[index]); ++index) {
+ part.push_back(input[index]);
+ }
+ }
+ // Ignore whitespace (outside of quotes).
+ else {
+ ++index;
+ }
+ }
+}
+
+// From src/third_party/protobuf2/src/src/google/protobuf/stubs/strutil.h
+// but we don't need any other aspect of protobufs so we don't want to
+// incur the link cost.
+void LowerString(std::string* s) {
+ std::string::iterator end = s->end();
+ for (std::string::iterator i = s->begin(); i != end; ++i) {
+ // tolower() changes based on locale. We don't want this!
+ if ('A' <= *i && *i <= 'Z') *i += 'a' - 'A';
+ }
+}
+
+// From src/third_party/protobuf2/src/src/google/protobuf/stubs/strutil.h
+// but we don't need any other aspect of protobufs so we don't want to
+// incur the link cost.
+bool HasPrefixString(const StringPiece& str, const StringPiece& prefix) {
+ return ((str.size() >= prefix.size()) &&
+ (str.substr(0, prefix.size()) == prefix));
+}
+
+const StringPiece EmptyString::kEmptyString;
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/string_util_test.cc b/trunk/src/net/instaweb/util/string_util_test.cc
new file mode 100644
index 0000000..9bafc83
--- /dev/null
+++ b/trunk/src/net/instaweb/util/string_util_test.cc
@@ -0,0 +1,260 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the string-splitter.
+
+#include <vector>
+
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace net_instaweb {
+
+class IntegerToStringToIntTest : public testing::Test {
+ protected:
+ void ValidateIntegerToString(int i, std::string s) {
+ EXPECT_EQ(s, IntegerToString(i));
+ ValidateInteger64ToString(static_cast<int64>(i), s);
+ }
+
+ void ValidateStringToInt(std::string s, int i) {
+ int i2;
+ EXPECT_TRUE(StringToInt(s, &i2));
+ EXPECT_EQ(i, i2);
+ ValidateStringToInt64(s, static_cast<int64>(i));
+ }
+
+ void InvalidStringToInt(std::string s) {
+ int i;
+ EXPECT_FALSE(StringToInt(s, &i));
+ InvalidStringToInt64(s);
+ }
+
+ void ValidateIntegerToStringToInt(int i) {
+ ValidateStringToInt(IntegerToString(i), i);
+ }
+
+ // Second verse, same as the first, a little more bits...
+ void ValidateInteger64ToString(int64 i, std::string s) {
+ EXPECT_EQ(s, Integer64ToString(i));
+ }
+
+ void ValidateStringToInt64(std::string s, int64 i) {
+ int64 i2;
+ EXPECT_TRUE(StringToInt64(s, &i2));
+ EXPECT_EQ(i, i2);
+ }
+
+ void InvalidStringToInt64(std::string s) {
+ int64 i;
+ EXPECT_FALSE(StringToInt64(s, &i));
+ }
+
+ void ValidateInteger64ToStringToInt64(int64 i) {
+ ValidateStringToInt64(Integer64ToString(i), i);
+ }
+};
+
+TEST_F(IntegerToStringToIntTest, TestIntegerToString) {
+ ValidateIntegerToString(0, "0");
+ ValidateIntegerToString(1, "1");
+ ValidateIntegerToString(10, "10");
+ ValidateIntegerToString(-5, "-5");
+ ValidateIntegerToString(123456789, "123456789");
+ ValidateIntegerToString(-123456789, "-123456789");
+ ValidateInteger64ToString(99123456789LL, "99123456789");
+ ValidateInteger64ToString(-99123456789LL, "-99123456789");
+}
+
+TEST_F(IntegerToStringToIntTest, TestStringToInt) {
+ ValidateStringToInt("0", 0);
+ ValidateStringToInt("1", 1);
+ ValidateStringToInt("10", 10);
+ ValidateStringToInt("-5", -5);
+ ValidateStringToInt("+5", 5);
+ ValidateStringToInt("123456789", 123456789);
+ ValidateStringToInt("-123456789", -123456789);
+ ValidateStringToInt("00000", 0);
+ ValidateStringToInt("010", 10);
+ ValidateStringToInt("-0000005", -5);
+ ValidateStringToInt("-00089", -89);
+ ValidateStringToInt64("-99123456789", -99123456789LL);
+}
+
+TEST_F(IntegerToStringToIntTest, TestInvalidString) {
+ InvalidStringToInt("");
+ InvalidStringToInt("-");
+ InvalidStringToInt("+");
+ InvalidStringToInt("--1");
+ InvalidStringToInt("++1");
+ InvalidStringToInt("1-");
+ InvalidStringToInt("1+");
+ InvalidStringToInt("1 000");
+ InvalidStringToInt("a");
+ InvalidStringToInt("1e2");
+ InvalidStringToInt("10^3");
+ InvalidStringToInt("1+3");
+ InvalidStringToInt("0x6A7");
+}
+
+TEST_F(IntegerToStringToIntTest, TestIntegerToStringToInt) {
+ int n = 1;
+ for (int i = 0; i < 1000; ++i) {
+ ValidateIntegerToStringToInt(n);
+ n *= -3; // This will overflow, that's fine, we just want a range of ints.
+ }
+ int64 n64 = 1LL;
+ for (int i = 0; i < 1000; ++i) {
+ ValidateInteger64ToStringToInt64(n64);
+ n64 *= -3; // This will overflow, that's fine, we just want a range of ints
+ }
+}
+
+class SplitStringTest : public testing::Test {
+};
+
+TEST_F(SplitStringTest, TestSplitNoOmitTrailing) {
+ std::vector<StringPiece> components;
+ SplitStringPieceToVector(".a.b..c.", ".", &components, false);
+ ASSERT_EQ(static_cast<size_t>(6), components.size());
+ ASSERT_EQ("", components[0]);
+ ASSERT_EQ("a", components[1]);
+ ASSERT_EQ("b", components[2]);
+ ASSERT_EQ("", components[3]);
+ ASSERT_EQ("c", components[4]);
+ ASSERT_EQ("", components[5]);
+}
+
+TEST_F(SplitStringTest, TestSplitNoOmitNoTrailing) {
+ std::vector<StringPiece> components;
+ SplitStringPieceToVector(".a.b..c", ".", &components, false);
+ ASSERT_EQ(static_cast<size_t>(5), components.size());
+ ASSERT_EQ("", components[0]);
+ ASSERT_EQ("a", components[1]);
+ ASSERT_EQ("b", components[2]);
+ ASSERT_EQ("", components[3]);
+ ASSERT_EQ("c", components[4]);
+}
+
+TEST_F(SplitStringTest, TestSplitNoOmitEmpty) {
+ std::vector<StringPiece> components;
+ SplitStringPieceToVector("", ".", &components, false);
+ ASSERT_EQ(static_cast<size_t>(1), components.size());
+ ASSERT_EQ("", components[0]);
+}
+
+TEST_F(SplitStringTest, TestSplitNoOmitOneDot) {
+ std::vector<StringPiece> components;
+ SplitStringPieceToVector(".", ".", &components, false);
+ ASSERT_EQ(static_cast<size_t>(2), components.size());
+ ASSERT_EQ("", components[0]);
+ ASSERT_EQ("", components[1]);
+}
+
+TEST_F(SplitStringTest, TestSplitOmitTrailing) {
+ std::vector<StringPiece> components;
+ SplitStringPieceToVector(".a.b..c.", ".", &components, true);
+ ASSERT_EQ(static_cast<size_t>(3), components.size());
+ ASSERT_EQ("a", components[0]);
+ ASSERT_EQ("b", components[1]);
+ ASSERT_EQ("c", components[2]);
+}
+
+TEST_F(SplitStringTest, TestSplitOmitNoTrailing) {
+ std::vector<StringPiece> components;
+ SplitStringPieceToVector(".a.b..c", ".", &components, true);
+ ASSERT_EQ(static_cast<size_t>(3), components.size());
+ ASSERT_EQ("a", components[0]);
+ ASSERT_EQ("b", components[1]);
+ ASSERT_EQ("c", components[2]);
+}
+
+TEST_F(SplitStringTest, TestSplitOmitEmpty) {
+ std::vector<StringPiece> components;
+ SplitStringPieceToVector("", ".", &components, true);
+ ASSERT_EQ(static_cast<size_t>(0), components.size());
+}
+
+TEST_F(SplitStringTest, TestSplitOmitOneDot) {
+ std::vector<StringPiece> components;
+ SplitStringPieceToVector(".", ".", &components, true);
+ ASSERT_EQ(static_cast<size_t>(0), components.size());
+}
+
+TEST(StringCaseTest, TestStringCaseEqual) {
+ EXPECT_FALSE(StringCaseEqual("foobar", "fobar"));
+ EXPECT_TRUE(StringCaseEqual("foobar", "foobar"));
+ EXPECT_TRUE(StringCaseEqual("foobar", "FOOBAR"));
+ EXPECT_TRUE(StringCaseEqual("FOOBAR", "foobar"));
+ EXPECT_TRUE(StringCaseEqual("fOoBaR", "FoObAr"));
+}
+
+TEST(StringCaseTest, TestStringCaseStartsWith) {
+ EXPECT_FALSE(StringCaseStartsWith("foobar", "fob"));
+ EXPECT_TRUE(StringCaseStartsWith("foobar", "foobar"));
+ EXPECT_TRUE(StringCaseStartsWith("foobar", "foo"));
+ EXPECT_TRUE(StringCaseStartsWith("foobar", "FOO"));
+ EXPECT_TRUE(StringCaseStartsWith("FOOBAR", "foo"));
+ EXPECT_TRUE(StringCaseStartsWith("fOoBaR", "FoO"));
+ EXPECT_FALSE(StringCaseStartsWith("zzz", "zzzz"));
+}
+
+TEST(StringCaseTest, TestStringCaseEndsWith) {
+ EXPECT_FALSE(StringCaseEndsWith("foobar", "baar"));
+ EXPECT_TRUE(StringCaseEndsWith("foobar", "foobar"));
+ EXPECT_TRUE(StringCaseEndsWith("foobar", "bar"));
+ EXPECT_TRUE(StringCaseEndsWith("foobar", "BAR"));
+ EXPECT_TRUE(StringCaseEndsWith("FOOBAR", "bar"));
+ EXPECT_TRUE(StringCaseEndsWith("fOoBaR", "bAr"));
+ EXPECT_FALSE(StringCaseEndsWith("zzz", "zzzz"));
+}
+
+TEST(ParseShellLikeStringTest, TestParse) {
+ std::vector<std::string> parts;
+ ParseShellLikeString("a b \"c d\" e 'f g'", &parts);
+ ASSERT_EQ(5, parts.size());
+ EXPECT_EQ("a", parts[0]);
+ EXPECT_EQ("b", parts[1]);
+ EXPECT_EQ("c d", parts[2]);
+ EXPECT_EQ("e", parts[3]);
+ EXPECT_EQ("f g", parts[4]);
+}
+
+TEST(ParseShellLikeStringTest, Backslash) {
+ std::vector<std::string> parts;
+ ParseShellLikeString(" \"a\\\"b\" 'c\\'d' ", &parts);
+ ASSERT_EQ(2, parts.size());
+ EXPECT_EQ("a\"b", parts[0]);
+ EXPECT_EQ("c'd", parts[1]);
+}
+
+TEST(ParseShellLikeStringTest, UnclosedQuote) {
+ std::vector<std::string> parts;
+ ParseShellLikeString("'a b", &parts);
+ ASSERT_EQ(1, parts.size());
+ EXPECT_EQ("a b", parts[0]);
+}
+
+TEST(ParseShellLikeStringTest, UnclosedQuoteAndBackslash) {
+ std::vector<std::string> parts;
+ ParseShellLikeString("'a b\\", &parts);
+ ASSERT_EQ(1, parts.size());
+ EXPECT_EQ("a b", parts[0]);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/string_writer.cc b/trunk/src/net/instaweb/util/string_writer.cc
new file mode 100644
index 0000000..b6b5d66
--- /dev/null
+++ b/trunk/src/net/instaweb/util/string_writer.cc
@@ -0,0 +1,35 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/string_writer.h"
+
+namespace net_instaweb {
+
+StringWriter::~StringWriter() {
+}
+
+bool StringWriter::Write(const StringPiece& str, MessageHandler* handler) {
+ string_->append(str.data(), str.size());
+ return true;
+}
+
+bool StringWriter::Flush(MessageHandler* message_handler) {
+ return true;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/symbol_table_test.cc b/trunk/src/net/instaweb/util/symbol_table_test.cc
new file mode 100644
index 0000000..d604ad9
--- /dev/null
+++ b/trunk/src/net/instaweb/util/symbol_table_test.cc
@@ -0,0 +1,73 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the symbol table
+
+#include "net/instaweb/util/public/symbol_table.h"
+#include "base/logging.h"
+#include "net/instaweb/util/public/gtest.h"
+#include <string>
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+class SymbolTableTest : public testing::Test {
+};
+
+TEST_F(SymbolTableTest, TestInternSensitive) {
+ SymbolTableSensitive symbol_table;
+ std::string s1("hello");
+ std::string s2("hello");
+ std::string s3("goodbye");
+ std::string s4("Goodbye");
+ EXPECT_NE(s1.c_str(), s2.c_str());
+ Atom a1 = symbol_table.Intern(s1);
+ Atom a2 = symbol_table.Intern(s2);
+ Atom a3 = symbol_table.Intern(s3);
+ Atom a4 = symbol_table.Intern(s4);
+ EXPECT_TRUE(a1 == a2);
+ EXPECT_EQ(a1.c_str(), a2.c_str());
+ EXPECT_FALSE(a1 == a3);
+ EXPECT_NE(a1.c_str(), a3.c_str());
+ EXPECT_FALSE(a3 == a4);
+
+ EXPECT_EQ(s1, a1.c_str());
+ EXPECT_EQ(s2, a2.c_str());
+ EXPECT_EQ(s3, a3.c_str());
+ EXPECT_EQ(s4, a4.c_str());
+}
+
+TEST_F(SymbolTableTest, TestInternInsensitive) {
+ SymbolTableInsensitive symbol_table;
+ std::string s1("hello");
+ std::string s2("Hello");
+ std::string s3("goodbye");
+ Atom a1 = symbol_table.Intern(s1);
+ Atom a2 = symbol_table.Intern(s2);
+ Atom a3 = symbol_table.Intern(s3);
+ EXPECT_TRUE(a1 == a2);
+ EXPECT_EQ(a1.c_str(), a2.c_str());
+ EXPECT_FALSE(a1 == a3);
+ EXPECT_NE(a1.c_str(), a3.c_str());
+
+ EXPECT_EQ(0, strcasecmp(s1.c_str(), a1.c_str()));
+ EXPECT_EQ(0, strcasecmp(s2.c_str(), a2.c_str()));
+ EXPECT_EQ(0, strcasecmp(s3.c_str(), a3.c_str()));
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/threadsafe_cache.cc b/trunk/src/net/instaweb/util/threadsafe_cache.cc
new file mode 100644
index 0000000..b131fe9
--- /dev/null
+++ b/trunk/src/net/instaweb/util/threadsafe_cache.cc
@@ -0,0 +1,47 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/threadsafe_cache.h"
+#include "net/instaweb/util/public/abstract_mutex.h"
+
+namespace net_instaweb {
+
+ThreadsafeCache::~ThreadsafeCache() {
+}
+
+bool ThreadsafeCache::Get(const std::string& key, SharedString* value) {
+ ScopedMutex mutex(mutex_);
+ return cache_->Get(key, value);
+}
+
+void ThreadsafeCache::Put(const std::string& key, SharedString* value) {
+ ScopedMutex mutex(mutex_);
+ cache_->Put(key, value);
+}
+
+void ThreadsafeCache::Delete(const std::string& key) {
+ ScopedMutex mutex(mutex_);
+ cache_->Delete(key);
+}
+
+CacheInterface::KeyState ThreadsafeCache::Query(const std::string& key) {
+ ScopedMutex mutex(mutex_);
+ return cache_->Query(key);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/time_util.cc b/trunk/src/net/instaweb/util/time_util.cc
new file mode 100644
index 0000000..c16f3ca
--- /dev/null
+++ b/trunk/src/net/instaweb/util/time_util.cc
@@ -0,0 +1,66 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/time_util.h"
+#include <time.h>
+
+#include "pagespeed/core/resource_util.h"
+
+namespace net_instaweb {
+
+bool ConvertTimeToString(int64 time_ms, std::string* time_string) {
+ time_t time_sec = time_ms / 1000;
+ struct tm time_buf;
+ struct tm* time_info = gmtime_r(&time_sec, &time_buf);
+ if ((time_info == NULL) ||
+ (time_buf.tm_wday < 0) ||
+ (time_buf.tm_wday > 6) ||
+ (time_buf.tm_mon < 0) ||
+ (time_buf.tm_mon > 11)) {
+ return false;
+ }
+
+ static const char* kWeekDay[] = {
+ "Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"};
+ static const char* kMonth[] = {
+ "Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov",
+ "Dec"};
+
+ // RFC 822 says to format like this:
+ // Thu Nov 18 02:15:22 2010 GMT
+ // See http://www.faqs.org/rfcs/rfc822.html
+ //
+ // But redbot.org likes this:
+ // Wed, 24 Nov 2010 21:14:12 GMT
+ *time_string = StringPrintf("%s, %02d %s %4d %02d:%02d:%02d GMT",
+ kWeekDay[time_buf.tm_wday],
+ time_buf.tm_mday,
+ kMonth[time_buf.tm_mon],
+ 1900 + time_buf.tm_year,
+ time_buf.tm_hour,
+ time_buf.tm_min,
+ time_buf.tm_sec);
+ return true;
+}
+
+bool ConvertStringToTime(const StringPiece& time_string, int64 *time_ms) {
+ std::string buf(time_string.data(), time_string.size());
+ return pagespeed::resource_util::ParseTimeValuedHeader(buf.c_str(), time_ms);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/time_util_test.cc b/trunk/src/net/instaweb/util/time_util_test.cc
new file mode 100644
index 0000000..8b6244c
--- /dev/null
+++ b/trunk/src/net/instaweb/util/time_util_test.cc
@@ -0,0 +1,62 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/time_util.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace {
+
+const char kApr5[] = "Mon, 05 Apr 2010 18:49:46 GMT";
+
+// The time-conversion functions are only accurate to the second,
+// and we will not be able to test for identity transforms if we
+// are not using a mulitple of 1000.
+const int64 kTimestampMs = 718981 * 1000;
+
+} // namespace
+
+namespace net_instaweb {
+
+class TimeUtilTest : public testing::Test {
+ protected:
+ std::string GetTimeString(int64 time_ms) {
+ std::string out;
+ EXPECT_TRUE(ConvertTimeToString(time_ms, &out));
+ return out;
+ }
+
+ int64 GetTimeValue(const std::string& time_str) {
+ int64 val;
+ CHECK(ConvertStringToTime(time_str, &val)) << "Time conversion failed";
+ return val;
+ }
+
+ std::string out_;
+};
+
+TEST_F(TimeUtilTest, Test1970) {
+ EXPECT_EQ("Thu, 01 Jan 1970 00:00:00 GMT", GetTimeString(0));
+ EXPECT_EQ(1270493386000LL, GetTimeValue(kApr5));
+}
+
+TEST_F(TimeUtilTest, TestIdentity) {
+ EXPECT_EQ(kTimestampMs, GetTimeValue(GetTimeString(kTimestampMs)));
+ EXPECT_EQ(kApr5, GetTimeString(GetTimeValue(kApr5)));
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/timer.cc b/trunk/src/net/instaweb/util/timer.cc
new file mode 100644
index 0000000..a0e48a8
--- /dev/null
+++ b/trunk/src/net/instaweb/util/timer.cc
@@ -0,0 +1,43 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/timer.h"
+
+namespace net_instaweb {
+
+const int64 Timer::kSecondUs = 1000 * 1000;
+const int64 Timer::kSecondMs = 1000;
+const int64 Timer::kMinuteMs = 60 * Timer::kSecondMs;
+const int64 Timer::kHourMs = 60 * Timer::kMinuteMs;
+const int64 Timer::kDayMs = 24 * Timer::kHourMs;
+const int64 Timer::kWeekMs = 7 * Timer::kDayMs;
+const int64 Timer::kMonthMs = 31 * Timer::kDayMs;
+const int64 Timer::kYearMs = 365 * Timer::kDayMs;
+
+Timer::~Timer() {
+}
+
+int64 Timer::NowMs() const {
+ return NowUs() / 1000;
+}
+
+void Timer::SleepMs(int64 ms) {
+ SleepUs(ms * 1000);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/timer_based_abstract_lock.cc b/trunk/src/net/instaweb/util/timer_based_abstract_lock.cc
new file mode 100644
index 0000000..e360fe1
--- /dev/null
+++ b/trunk/src/net/instaweb/util/timer_based_abstract_lock.cc
@@ -0,0 +1,126 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/util/public/timer_based_abstract_lock.h"
+
+#include "net/instaweb/util/public/timer.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// Number of times we busy spin before we start to sleep.
+int kBusySpinIterations = 100;
+
+// We back off exponentially, with a constant of 1.5.
+int64 Backoff(int64 interval_ms) {
+ return (1 + interval_ms + (interval_ms >> 1));
+}
+
+int64 IntervalWithEnd(Timer* timer, int64 interval_ms, int64 end_time_ms) {
+ int64 now_ms = timer->NowMs();
+ int64 remaining = end_time_ms - now_ms;
+ interval_ms = Backoff(interval_ms);
+ if (remaining > interval_ms) {
+ return interval_ms;
+ } else {
+ return remaining;
+ }
+}
+
+} // namespace
+
+TimerBasedAbstractLock::~TimerBasedAbstractLock() { }
+
+void TimerBasedAbstractLock::Lock() {
+ if (TryLock() ||
+ BusySpin(&TimerBasedAbstractLock::TryLockIgnoreTimeout, 0)) {
+ return;
+ }
+ Spin(&TimerBasedAbstractLock::TryLockIgnoreTimeout, 0);
+}
+
+bool TimerBasedAbstractLock::LockTimedWait(int64 wait_ms) {
+ return (TryLock() ||
+ SpinFor(&TimerBasedAbstractLock::TryLockIgnoreTimeout, 0, wait_ms));
+}
+
+void TimerBasedAbstractLock::LockStealOld(int64 timeout_ms) {
+ if (LockTimedWaitStealOld(timeout_ms, timeout_ms)) {
+ return;
+ }
+ // Contention! Restart spin, but don't decay again in future.
+ Spin(&TimerBasedAbstractLock::TryLockStealOld, timeout_ms);
+}
+
+bool TimerBasedAbstractLock::LockTimedWaitStealOld(
+ int64 wait_ms, int64 timeout_ms) {
+ return
+ (TryLock() ||
+ SpinFor(&TimerBasedAbstractLock::TryLockStealOld, timeout_ms, wait_ms));
+}
+
+// For uniformity, we implement spinning without regard to whether the
+// underlying lock primitive can time out or not. This wrapper method is just
+// TryLock with a bogus timeout parameter for uniformity.
+bool TimerBasedAbstractLock::TryLockIgnoreTimeout(int64 timeout_ignored) {
+ return TryLock();
+}
+
+// Actively attempt to take lock without pausing.
+bool TimerBasedAbstractLock::BusySpin(TryLockMethod try_lock,
+ int64 timeout_ms) {
+ for (int i = 0; i < kBusySpinIterations; i++) {
+ if ((this->*try_lock)(timeout_ms)) {
+ return true;
+ }
+ }
+ return false;
+}
+
+// Attempt to take lock, backing off forever.
+void TimerBasedAbstractLock::Spin(TryLockMethod try_lock, int64 timeout_ms) {
+ Timer* the_timer = timer();
+ int64 interval_ms = 1;
+ while (!(this->*try_lock)(timeout_ms)) {
+ the_timer->SleepMs(interval_ms);
+ interval_ms = Backoff(interval_ms);
+ // interval_ms can't realistically overflow in geologic time.
+ }
+}
+
+// Attempt to take lock until end_time_ms.
+bool TimerBasedAbstractLock::SpinFor(TryLockMethod try_lock, int64 timeout_ms,
+ int64 wait_ms) {
+ int64 end_time_ms = timer()->NowMs() + wait_ms;
+ if (BusySpin(try_lock, timeout_ms)) {
+ return true;
+ }
+ Timer* the_timer = timer();
+ int64 interval_ms = IntervalWithEnd(the_timer, 0, end_time_ms);
+ bool result;
+ // Ugh. Here we're spinning until result is true (we got the lock), or we run
+ // out of time.
+ while (!((result = (this->*try_lock)(timeout_ms))) && interval_ms > 0) {
+ the_timer->SleepMs(interval_ms);
+ interval_ms = IntervalWithEnd(the_timer, interval_ms, end_time_ms);
+ }
+ return result;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/url_async_fetcher.cc b/trunk/src/net/instaweb/util/url_async_fetcher.cc
new file mode 100644
index 0000000..4ed8674
--- /dev/null
+++ b/trunk/src/net/instaweb/util/url_async_fetcher.cc
@@ -0,0 +1,39 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/url_async_fetcher.h"
+#include "net/instaweb/util/public/meta_data.h"
+#include "net/instaweb/util/public/string_writer.h"
+
+namespace net_instaweb {
+
+const int64 UrlAsyncFetcher::kUnspecifiedTimeout = 0;
+
+UrlAsyncFetcher::~UrlAsyncFetcher() {
+}
+
+UrlAsyncFetcher::Callback::~Callback() {
+}
+
+bool UrlAsyncFetcher::Callback::EnableThreaded() const {
+ // Most fetcher callbacks are not prepared to be called from a different
+ // thread.
+ return false;
+}
+
+} // namespace instaweb
diff --git a/trunk/src/net/instaweb/util/url_escaper.cc b/trunk/src/net/instaweb/util/url_escaper.cc
new file mode 100644
index 0000000..6b75592
--- /dev/null
+++ b/trunk/src/net/instaweb/util/url_escaper.cc
@@ -0,0 +1,149 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/url_escaper.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// Firefox converts ^ to a % sequence.
+// Apache rejects requests with % sequences it does not understand.
+// So limit the pass-through characters as follows, and use ',' as
+// an escaper.
+//
+// Unfortunately this makes longer filenames because ',' is also used
+// in the filenam encoder.
+//
+// TODO(jmarantz): Pass through '.', and exploit '/' as a legal character
+// in URLs. This requires redefining the constraints of a 'segment', which
+// currently excludes both '.' and '/' due to rules enforced primarily
+// in net/instaweb/rewriter/resource_manager.cc, but are distributed a bit
+// more widely.
+const char kPassThroughChars[] = "._=+-";
+
+// Checks for 'search' at start of 'src'. If found, appends
+// 'replacement' into 'out', and advances the start-point in 'src'
+// past the search string, returning true.
+bool ReplaceSubstring(const StringPiece& search, const char* replacement,
+ StringPiece* src, std::string* out) {
+ bool ret = false;
+ if ((src->size() >= search.size()) &&
+ (memcmp(src->data(), search.data(), search.size()) == 0)) {
+ out->append(replacement);
+ *src = src->substr(search.size());
+ ret = true;
+ }
+ return ret;
+}
+
+} // namespace
+
+UrlEscaper::~UrlEscaper() { }
+
+void UrlEscaper::EncodeToUrlSegment(const StringPiece& in,
+ std::string* url_segment) {
+ for (StringPiece src = in; src.size() != 0; ) {
+ // We need to check for common prefixes that begin with pass-through
+ // characters before doing the isalnum check.
+ if (!ReplaceSubstring("http://", ",h", &src, url_segment) &&
+ !ReplaceSubstring(".pagespeed.", ",M", &src, url_segment)) {
+ char c = src[0];
+ if (isalnum(c) || (strchr(kPassThroughChars, c) != NULL)) {
+ url_segment->append(1, c);
+ src = src.substr(1);
+ } else if (
+ // TODO(jmarantz): put these in a static table and generate
+ // an FSM so we don't have so much lookahed scanning, and we
+ // don't have to work hard to keep the encoder and decoder
+ // in sync.
+ !ReplaceSubstring("^", ",u", &src, url_segment) &&
+ !ReplaceSubstring("%", ",P", &src, url_segment) &&
+ !ReplaceSubstring("/", ",_", &src, url_segment) &&
+ !ReplaceSubstring("\\", ",-", &src, url_segment) &&
+ !ReplaceSubstring(",", ",,", &src, url_segment) &&
+ !ReplaceSubstring("?", ",q", &src, url_segment) &&
+ !ReplaceSubstring("&", ",a", &src, url_segment)) {
+ url_segment->append(StringPrintf(",%02X",
+ static_cast<unsigned char>(c)));
+ src = src.substr(1);
+ }
+ }
+ }
+}
+
+bool UrlEscaper::DecodeFromUrlSegment(const StringPiece& url_segment,
+ std::string* out) {
+ int remaining = url_segment.size();
+ for (const char* p = url_segment.data(); remaining != 0; ++p, --remaining) {
+ char c = *p;
+ if (isalnum(c) || (strchr(kPassThroughChars, c) != NULL)) {
+ out->append(&c, 1);
+ } else if ((c != ',') || (remaining < 2)) {
+ return false; // unknown char or trailing ,; this is an invalid encoding.
+ } else {
+ ++p;
+ --remaining;
+ switch (*p) {
+ case '_': *out += "/"; break;
+ case '-': *out += "\\"; break;
+ case ',': *out += ","; break;
+ case 'a': *out += "&"; break;
+ case 'M': *out += ".pagespeed."; break;
+ case 'P': *out += "%"; break;
+ case 'q': *out += "?"; break;
+ case 'u': *out += "^"; break;
+
+ // The following legacy encodings are no longer made. However
+ // we should continue to decode what we previously encoded in
+ // November 2010 to avoid (for example) breaking image search.
+ case 'c': *out += ".com"; break;
+ case 'e': *out += ".edu"; break;
+ case 'g': *out += ".gif"; break;
+ case 'h': *out += "http://"; break;
+ case 'j': *out += ".jpg"; break;
+ case 'k': *out += ".jpeg"; break;
+ case 'l': *out += ".js"; break;
+ case 'n': *out += ".net"; break;
+ case 'o': *out += "."; break;
+ case 'p': *out += ".png"; break;
+ case 's': *out += ".css"; break;
+ case 't': *out += ".html"; break;
+ case 'w': *out += "www."; break;
+
+ default:
+ if (remaining < 2) {
+ return false;
+ }
+ --remaining;
+ int char_val = 0;
+ if (AccumulateHexValue(*p++, &char_val) &&
+ AccumulateHexValue(*p, &char_val)) {
+ out->append(1, static_cast<char>(char_val));
+ } else {
+ return false;
+ }
+ break;
+ }
+ }
+ }
+ return true;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/url_escaper_test.cc b/trunk/src/net/instaweb/util/url_escaper_test.cc
new file mode 100644
index 0000000..f314c5a
--- /dev/null
+++ b/trunk/src/net/instaweb/util/url_escaper_test.cc
@@ -0,0 +1,131 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/url_escaper.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace {
+
+// We pass through a few special characters unchanged, and we
+// accept those characters, plus ',', as acceptable in the encoded
+// URLs.
+static const char kAcceptableSpecialChars[] = ",._+-=";
+static const char kPassThruChars[] = "._+-=";
+
+} // namespace
+
+namespace net_instaweb {
+
+class UrlEscaperTest : public testing::Test {
+ protected:
+ void CheckEncoding(const StringPiece& url) {
+ std::string encoded, decoded;
+ escaper_.EncodeToUrlSegment(url, &encoded);
+
+ // Make sure there are only alphanumerics and _+-=%
+ for (size_t i = 0; i < encoded.size(); ++i) {
+ char c = encoded[i];
+ EXPECT_TRUE(isalnum(c) || (strchr(kAcceptableSpecialChars, c) != NULL));
+ }
+
+ EXPECT_TRUE(escaper_.DecodeFromUrlSegment(encoded, &decoded));
+ EXPECT_EQ(url, decoded);
+ }
+
+ // Some basic text should be completely unchanged upon encode/decode.
+ void CheckUnchanged(const StringPiece& url) {
+ std::string encoded, decoded;
+ escaper_.EncodeToUrlSegment(url, &encoded);
+ EXPECT_EQ(url, encoded);
+ EXPECT_TRUE(escaper_.DecodeFromUrlSegment(encoded, &decoded));
+ EXPECT_EQ(url, decoded);
+ }
+
+ std::string Decode(const StringPiece& encoding) {
+ std::string decoded;
+ EXPECT_TRUE(escaper_.DecodeFromUrlSegment(encoding, &decoded));
+ return decoded;
+ }
+
+ std::string Encode(const StringPiece& url) {
+ std::string encoded;
+ escaper_.EncodeToUrlSegment(url, &encoded);
+ return encoded;
+ }
+
+ UrlEscaper escaper_;
+};
+
+TEST_F(UrlEscaperTest, TestUrls) {
+ CheckEncoding("http://www.google.com");
+ // Test encoding of % and lack of leading http:// (beware of double encoding):
+ CheckEncoding("//web.mit.edu/foo.cgi?bar%baz");
+ CheckEncoding("http://x.com/images/hacks.js.pagespeed.jm.GSLMcHP-fl.js");
+ CheckEncoding("http://www.foo.bar/z1234/b_c.d?e=f&g=h");
+ CheckEncoding("http://china.com/\u591a\u5e74\u7ecf\u5178\u5361\u7247\u673a");
+ CheckEncoding("http://中国 汪 世 孟");
+ CheckEncoding("/static/f.1.js?v=120");
+ CheckEncoding("!@#$%^&*()_+=-[]{}?><,./");
+}
+
+TEST_F(UrlEscaperTest, TestUnchanged) {
+ CheckUnchanged("abcdefghijklmnopqrstuvwxyz");
+ CheckUnchanged("ABCDEFGHIJKLMNOPQRSTUVWXYZ");
+ CheckUnchanged("0123456789");
+ CheckUnchanged("=+-_");
+ CheckUnchanged(kPassThruChars);
+}
+
+TEST_F(UrlEscaperTest, LegacyDecode) {
+ EXPECT_EQ("a.css", Decode("a,s"));
+ EXPECT_EQ("b.jpg", Decode("b,j"));
+ EXPECT_EQ("c.png", Decode("c,p"));
+ EXPECT_EQ("d.gif", Decode("d,g"));
+ EXPECT_EQ("e.jpeg", Decode("e,k"));
+ EXPECT_EQ("f.js", Decode("f,l"));
+ EXPECT_EQ("g.anything", Decode("g,oanything"));
+ EXPECT_EQ("http://www.myhost.com", Decode(",h,wmyhost,c"));
+}
+
+TEST_F(UrlEscaperTest, TestEncoding) {
+ // Special case encoding a common sequence that would be long and
+ // ugly to escape char-by-char. We used to encode more than this
+ // (e.g. .com -> ,c) but now that we can allow '.' in encoded names,
+ // we favor legibility over compactness and have dropped the encoding
+ // of ".com" and others. However http:// requires three characters
+ // to be decoded so we'll encode it in one piece.
+ EXPECT_EQ(",h", Encode("http://"));
+
+ // These common characters get special-case encodings.
+ EXPECT_EQ(",u", Encode("^"));
+ EXPECT_EQ(",P", Encode("%"));
+ EXPECT_EQ(",_", Encode("/"));
+ EXPECT_EQ(",-", Encode("\\"));
+ EXPECT_EQ(",,", Encode(","));
+ EXPECT_EQ(",q", Encode("?"));
+ EXPECT_EQ(",a", Encode("&"));
+ EXPECT_EQ(",M", Encode(".pagespeed."));
+ EXPECT_EQ(",hx.com,_images,_hacks.js,Mjm.GSLMcHP-fl.js",
+ Encode("http://x.com/images/hacks.js.pagespeed.jm.GSLMcHP-fl.js"));
+
+ // Other characters are simply hexified.
+ EXPECT_EQ(",3A", Encode(":"));
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/url_fetcher.cc b/trunk/src/net/instaweb/util/url_fetcher.cc
new file mode 100644
index 0000000..5156bde
--- /dev/null
+++ b/trunk/src/net/instaweb/util/url_fetcher.cc
@@ -0,0 +1,36 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/public/url_fetcher.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/string_writer.h"
+
+namespace net_instaweb {
+
+UrlFetcher::~UrlFetcher() {
+}
+
+bool UrlFetcher::FetchUrl(const std::string& url, std::string* content,
+ MessageHandler* message_handler) {
+ StringWriter writer(content);
+ SimpleMetaData request_headers, response_headers;
+ return StreamingFetchUrl(url, request_headers, &response_headers, &writer,
+ message_handler);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/url_multipart_encoder.cc b/trunk/src/net/instaweb/util/url_multipart_encoder.cc
new file mode 100644
index 0000000..3aa057f
--- /dev/null
+++ b/trunk/src/net/instaweb/util/url_multipart_encoder.cc
@@ -0,0 +1,97 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/url_multipart_encoder.h"
+#include "net/instaweb/util/public/message_handler.h"
+
+namespace {
+
+// Ultimately these will be encoded by the URL Escaper so we want to
+// stay within legal URL space so we don't blow up. We'll have to
+// see how we like this aesthetically. We want to stay within legal
+// filename space as well so the filenames don't look too ugly.
+
+const char kEscape = '='; // Nice if this is filename-legal
+const char kEscapedEscape[] = "==";
+const char kSeparator = '+';
+const char kEscapedSeparator[] = "=+";
+
+} // namespace
+
+namespace net_instaweb {
+
+std::string UrlMultipartEncoder::Encode() const {
+ std::string encoding;
+ for (int i = 0, n = urls_.size(); i <n; ++i) {
+ if (i != 0) {
+ encoding += kSeparator;
+ }
+ const std::string& url = urls_[i];
+ for (int c = 0, nc = url.size(); c < nc; ++c) {
+ char ch = url[c];
+ if (ch == kEscape) {
+ encoding += kEscapedEscape;
+ } else if (ch == kSeparator) {
+ encoding += kEscapedSeparator;
+ } else {
+ encoding += ch;
+ }
+ }
+ }
+ return encoding;
+}
+
+bool UrlMultipartEncoder::Decode(const StringPiece& encoding,
+ MessageHandler* handler) {
+ urls_.clear();
+ std::string url;
+ bool append_last = false;
+ for (int c = 0, nc = encoding.size(); c < nc; ++c) {
+ char ch = encoding[c];
+ if (ch == kSeparator) {
+ urls_.push_back(url);
+ url.clear();
+ append_last = true;
+ // ensure that a "a+b+" results in 3 urls with the last one empty.
+ } else {
+ if (ch == kEscape) {
+ ++c;
+ if (c == nc) {
+ handler->Message(kError,
+ "Invalid encoding: escape at end of string %s",
+ encoding.as_string().c_str());
+ return false;
+ }
+ ch = encoding[c];
+ if ((ch != kEscape) && (ch != kSeparator)) {
+ handler->Message(kError,
+ "Invalid character `%c', after escape `%c' in %s",
+ ch, kEscape, encoding.as_string().c_str());
+ return false;
+ }
+ }
+ url += ch;
+ }
+ }
+ if (append_last || !url.empty()) {
+ urls_.push_back(url);
+ }
+ return true;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/url_multipart_encoder_test.cc b/trunk/src/net/instaweb/util/url_multipart_encoder_test.cc
new file mode 100644
index 0000000..22c7d84
--- /dev/null
+++ b/trunk/src/net/instaweb/util/url_multipart_encoder_test.cc
@@ -0,0 +1,65 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/url_escaper.h"
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/url_multipart_encoder.h"
+
+namespace net_instaweb {
+
+class UrlMultipartEncoderTest : public testing::Test {
+ protected:
+ UrlMultipartEncoder encoder_;
+ GoogleMessageHandler handler_;
+};
+
+TEST_F(UrlMultipartEncoderTest, EscapeSeparatorsAndEscapes) {
+ encoder_.AddUrl("abc");
+ encoder_.AddUrl("def");
+ encoder_.AddUrl("a=b+c"); // escape and separate characters
+ std::string encoding = encoder_.Encode();
+ encoder_.clear();
+ ASSERT_TRUE(encoder_.Decode(encoding, &handler_));
+ ASSERT_EQ(3, encoder_.num_urls());
+ EXPECT_EQ(std::string("abc"), encoder_.url(0));
+ EXPECT_EQ(std::string("def"), encoder_.url(1));
+ EXPECT_EQ(std::string("a=b+c"), encoder_.url(2));
+}
+
+TEST_F(UrlMultipartEncoderTest, Empty) {
+ ASSERT_TRUE(encoder_.Decode("", &handler_));
+ EXPECT_EQ(0, encoder_.num_urls());
+}
+
+TEST_F(UrlMultipartEncoderTest, LastIsEmpty) {
+ ASSERT_TRUE(encoder_.Decode("a+b+", &handler_));
+ ASSERT_EQ(3, encoder_.num_urls());
+ EXPECT_EQ(std::string("a"), encoder_.url(0));
+ EXPECT_EQ(std::string("b"), encoder_.url(1));
+ EXPECT_EQ(std::string(""), encoder_.url(2));
+}
+
+TEST_F(UrlMultipartEncoderTest, One) {
+ ASSERT_TRUE(encoder_.Decode("a", &handler_));
+ ASSERT_EQ(1, encoder_.num_urls());
+ EXPECT_EQ(std::string("a"), encoder_.url(0));
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/url_segment_encoder.cc b/trunk/src/net/instaweb/util/url_segment_encoder.cc
new file mode 100644
index 0000000..ba490c7
--- /dev/null
+++ b/trunk/src/net/instaweb/util/url_segment_encoder.cc
@@ -0,0 +1,24 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/util/public/url_segment_encoder.h"
+
+namespace net_instaweb {
+// This file exists *only* to declare the virtual destructor implementation.
+UrlSegmentEncoder::~UrlSegmentEncoder() { }
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/user_agent.cc b/trunk/src/net/instaweb/util/user_agent.cc
new file mode 100644
index 0000000..354ffee
--- /dev/null
+++ b/trunk/src/net/instaweb/util/user_agent.cc
@@ -0,0 +1,42 @@
+// Copyright 2010 Google Inc. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "net/instaweb/util/public/user_agent.h"
+
+namespace net_instaweb {
+
+UserAgent::UserAgent() {
+}
+
+void UserAgent::set_user_agent(const char* user_agent) {
+ if (user_agent) {
+ user_agent_.assign(user_agent);
+ } else {
+ user_agent_.clear();
+ }
+}
+
+bool UserAgent::IsIe() const {
+ return user_agent_.find(" MSIE ") != std::string::npos;
+}
+
+bool UserAgent::IsIe6() const {
+ return user_agent_.find(" MSIE 6.") != std::string::npos;
+}
+
+bool UserAgent::IsIe7() const {
+ return user_agent_.find(" MSIE 7.") != std::string::npos;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/user_agent_test.cc b/trunk/src/net/instaweb/util/user_agent_test.cc
new file mode 100644
index 0000000..a969fec
--- /dev/null
+++ b/trunk/src/net/instaweb/util/user_agent_test.cc
@@ -0,0 +1,86 @@
+// Copyright 2010 Google Inc. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "net/instaweb/util/public/user_agent.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace {
+
+// User Agent strings are from http://www.useragentstring.com/.
+// IE: http://www.useragentstring.com/pages/Internet%20Explorer/
+// FireFox: http://www.useragentstring.com/pages/Firefox/
+// Chrome: http://www.useragentstring.com/pages/Chrome/
+// And there are many more.
+
+const char kIe6UserAgent[] =
+ "Mozilla/5.0 (Windows; U; MSIE 6.0; Windows NT 5.1; SV1;"
+ " .NET CLR 2.0.50727)";
+const char kIe7UserAgent[] =
+ "Mozilla/5.0 (Windows; U; MSIE 7.0; Windows NT 6.0; en-US)";
+const char kIe8UserAgent[] =
+ "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64;"
+ " Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729;"
+ " .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2;"
+ " .NET4.0C; .NET4.0E; FDM)";
+const char kIe9UserAgent[] =
+ "Mozilla/5.0 (Windows; U; MSIE 9.0; WIndows NT 9.0; en-US))";
+const char kFirefoxUserAgent[] =
+ "Mozilla/5.0 (X11; U; Linux x86_64; zh-CN; rv:1.9.2.10) "
+ "Gecko/20100922 Ubuntu/10.10 (maverick) Firefox/3.6.10";
+const char kChromeUserAgent[] =
+ "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) "
+ "AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.A.B.C Safari/525.13";
+
+} // namespace
+
+namespace net_instaweb {
+
+class UserAgentTest : public testing::Test {
+ protected:
+ void SetUserAgent(const char* agent) {
+ user_agent_.set_user_agent(agent);
+ }
+ UserAgent user_agent_;
+};
+
+TEST_F(UserAgentTest, IsIeTest) {
+ SetUserAgent(kIe6UserAgent);
+ EXPECT_TRUE(user_agent_.IsIe());
+ EXPECT_TRUE(user_agent_.IsIe6());
+ EXPECT_FALSE(user_agent_.IsIe7());
+ EXPECT_TRUE(user_agent_.IsIe6or7());
+
+ SetUserAgent(kIe7UserAgent);
+ EXPECT_TRUE(user_agent_.IsIe());
+ EXPECT_TRUE(user_agent_.IsIe7());
+ EXPECT_FALSE(user_agent_.IsIe6());
+ EXPECT_TRUE(user_agent_.IsIe6or7());
+
+ SetUserAgent(kIe8UserAgent);
+ EXPECT_TRUE(user_agent_.IsIe());
+ EXPECT_FALSE(user_agent_.IsIe6());
+ EXPECT_FALSE(user_agent_.IsIe7());
+ EXPECT_FALSE(user_agent_.IsIe6or7());
+
+
+}
+
+TEST_F(UserAgentTest, IsNotIeTest) {
+ SetUserAgent(kFirefoxUserAgent);
+ EXPECT_FALSE(user_agent_.IsIe());
+ EXPECT_FALSE(user_agent_.IsIe6());
+ EXPECT_FALSE(user_agent_.IsIe6or7());
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/wait_url_async_fetcher.cc b/trunk/src/net/instaweb/util/wait_url_async_fetcher.cc
new file mode 100644
index 0000000..91e2424
--- /dev/null
+++ b/trunk/src/net/instaweb/util/wait_url_async_fetcher.cc
@@ -0,0 +1,81 @@
+/*
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/public/wait_url_async_fetcher.h"
+
+#include "base/basictypes.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/stl_util.h"
+#include "net/instaweb/util/public/url_fetcher.h"
+
+namespace net_instaweb {
+
+class WaitUrlAsyncFetcher::DelayedFetch {
+ public:
+ DelayedFetch(UrlFetcher* base_fetcher,
+ const std::string& url, const MetaData& request_headers,
+ MetaData* response_headers, Writer* response_writer,
+ MessageHandler* handler, Callback* callback)
+ : base_fetcher_(base_fetcher), url_(url),
+ response_headers_(response_headers), response_writer_(response_writer),
+ handler_(handler), callback_(callback) {
+ request_headers_.CopyFrom(request_headers);
+ }
+
+ void FetchNow() {
+ bool status = base_fetcher_->StreamingFetchUrl(
+ url_, request_headers_, response_headers_, response_writer_, handler_);
+ callback_->Done(status);
+ }
+
+ private:
+ UrlFetcher* base_fetcher_;
+ std::string url_;
+ SimpleMetaData request_headers_;
+ MetaData* response_headers_;
+ Writer* response_writer_;
+ MessageHandler* handler_;
+ Callback* callback_;
+
+ DISALLOW_COPY_AND_ASSIGN(DelayedFetch);
+};
+
+WaitUrlAsyncFetcher::~WaitUrlAsyncFetcher() {}
+
+bool WaitUrlAsyncFetcher::StreamingFetch(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* response_writer,
+ MessageHandler* handler,
+ Callback* callback) {
+ // Don't call the blocking fetcher until CallCallbacks.
+ delayed_fetches_.push_back(new DelayedFetch(
+ url_fetcher_, url, request_headers, response_headers, response_writer,
+ handler, callback));
+ return false;
+}
+
+void WaitUrlAsyncFetcher::CallCallbacks() {
+ for (int i = 0, n = delayed_fetches_.size(); i < n; ++i) {
+ delayed_fetches_[i]->FetchNow();
+ }
+ STLDeleteElements(&delayed_fetches_);
+ delayed_fetches_.clear();
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/wait_url_async_fetcher_test.cc b/trunk/src/net/instaweb/util/wait_url_async_fetcher_test.cc
new file mode 100644
index 0000000..e82000e
--- /dev/null
+++ b/trunk/src/net/instaweb/util/wait_url_async_fetcher_test.cc
@@ -0,0 +1,85 @@
+/*
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "net/instaweb/util/public/wait_url_async_fetcher.h"
+
+#include "net/instaweb/util/public/google_message_handler.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/string_writer.h"
+#include "net/instaweb/util/public/mock_url_fetcher.h"
+
+namespace net_instaweb {
+
+namespace {
+
+// Callback that just checks that it was called with success.
+class DummyCallback : public UrlAsyncFetcher::Callback {
+ public:
+ DummyCallback() : done_(false) {}
+ virtual ~DummyCallback() {
+ EXPECT_TRUE(done_);
+ }
+ virtual void Done(bool success) {
+ EXPECT_FALSE(done_) << "Already Done; perhaps you reused without Reset()";
+ done_ = true;
+ EXPECT_TRUE(success);
+ }
+ bool done_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(DummyCallback);
+};
+
+class WaitUrlAsyncFetcherTest : public ::testing::Test {};
+
+TEST_F(WaitUrlAsyncFetcherTest, FetcherWaits) {
+ MockUrlFetcher base_fetcher;
+ WaitUrlAsyncFetcher wait_fetcher(&base_fetcher);
+
+ const char url[] = "http://www.example.com/";
+ SimpleMetaData header;
+ header.set_first_line(1, 1, 200, "OK");
+ const char body[] = "Contents.";
+
+ base_fetcher.SetResponse(url, header, body);
+
+ const SimpleMetaData request_headers;
+ SimpleMetaData response_headers;
+ std::string response_body;
+ StringWriter response_writer(&response_body);
+ GoogleMessageHandler handler;
+ DummyCallback callback;
+
+ EXPECT_FALSE(wait_fetcher.StreamingFetch(url, request_headers,
+ &response_headers, &response_writer,
+ &handler, &callback));
+
+ // Nothing gets set ...
+ EXPECT_EQ(false, callback.done_);
+ EXPECT_EQ("", response_body);
+
+ // ... until we CallCallbacks.
+ wait_fetcher.CallCallbacks();
+ EXPECT_EQ(true, callback.done_);
+ EXPECT_EQ(body, response_body);
+
+}
+
+} // namespace
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/wget_url_fetcher.cc b/trunk/src/net/instaweb/util/wget_url_fetcher.cc
new file mode 100644
index 0000000..db33428
--- /dev/null
+++ b/trunk/src/net/instaweb/util/wget_url_fetcher.cc
@@ -0,0 +1,113 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/wget_url_fetcher.h"
+
+#include <errno.h>
+#include "net/instaweb/util/public/http_response_parser.h"
+#include "net/instaweb/util/public/message_handler.h"
+#include "net/instaweb/util/public/simple_meta_data.h"
+#include "net/instaweb/util/public/writer.h"
+
+namespace {
+
+// It turns out to be harder to quote in bash with single-quote
+// than double-quote. From man sh:
+//
+// Single Quotes
+// Enclosing characters in single quotes preserves the literal meaning of
+// all the characters (except single quotes, making it impossible to put
+// single-quotes in a single-quoted string).
+//
+// Double Quotes
+// Enclosing characters within double quotes preserves the literal meaning
+// of all characters except dollarsign ($), backquote (‘), and backslash
+// (\). The backslash inside double quotes is historically weird, and
+// serves to quote only the following characters:
+// $ ‘ " \ <newline>.
+// Otherwise it remains literal.
+//
+// So we put double-quotes around most strings, after first escaping
+// any of these characters:
+const char kEscapeChars[] = "\"$`\\";
+}
+
+namespace net_instaweb {
+
+// Default user agent to a Chrome user agent, so that we get real website.
+const char WgetUrlFetcher::kDefaultUserAgent[] =
+ "Mozilla/5.0 (X11; U; Linux x86_64; en-US) "
+ "AppleWebKit/534.0 (KHTML, like Gecko) Chrome/6.0.408.1 Safari/534.0";
+
+WgetUrlFetcher::~WgetUrlFetcher() {
+}
+
+bool WgetUrlFetcher::StreamingFetchUrl(const std::string& url,
+ const MetaData& request_headers,
+ MetaData* response_headers,
+ Writer* writer,
+ MessageHandler* handler) {
+ std::string cmd("/usr/bin/wget --save-headers -q -O -"), escaped_url;
+
+ // Use default user-agent if none is set in headers.
+ CharStarVector values;
+ request_headers.Lookup("user-agent", &values);
+ if (values.empty()) {
+ cmd += StrCat(" --user-agent=\"", kDefaultUserAgent, "\"");
+ }
+
+ for (int i = 0, n = request_headers.NumAttributes(); i < n; ++i) {
+ std::string escaped_name, escaped_value;
+
+ BackslashEscape(request_headers.Name(i), kEscapeChars, &escaped_name);
+ BackslashEscape(request_headers.Value(i), kEscapeChars, &escaped_value);
+ cmd += StrCat(" --header=\"", escaped_name, ": ", escaped_value, "\"");
+ }
+
+ BackslashEscape(url, kEscapeChars, &escaped_url);
+ cmd += StrCat(" \"", escaped_url, "\"");
+ handler->Message(kInfo, "wget %s\n", url.c_str());
+ FILE* wget_stdout = popen(cmd.c_str(), "r");
+
+ bool ret = false;
+ if (wget_stdout == NULL) {
+ handler->Message(kError, "Wget popen failed on url %s: %s",
+ url.c_str(), strerror(errno));
+ } else {
+ HttpResponseParser parser(response_headers, writer, handler);
+ ret = parser.Parse(wget_stdout);
+ int exit_status = pclose(wget_stdout);
+ if (exit_status != 0) {
+ // The wget failed. wget does not always (ever?) write appropriate
+ // headers when it fails, so invent some.
+ if (response_headers->status_code() == 0) {
+ response_headers->set_first_line(1, 1, HttpStatus::kBadRequest,
+ "Wget Failed");
+ response_headers->ComputeCaching();
+ response_headers->set_headers_complete(true);
+ writer->Write("wget failed: ", handler);
+ writer->Write(url, handler);
+ writer->Write("<br>\nExit Status: ", handler);
+ writer->Write(IntegerToString(exit_status), handler);
+ }
+ }
+ }
+ return ret;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/wildcard.cc b/trunk/src/net/instaweb/util/wildcard.cc
new file mode 100644
index 0000000..0c0f776
--- /dev/null
+++ b/trunk/src/net/instaweb/util/wildcard.cc
@@ -0,0 +1,150 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/wildcard.h"
+#include "net/instaweb/util/public/string_util.h"
+
+namespace net_instaweb {
+
+const char Wildcard::kMatchAny = '*';
+const char Wildcard::kMatchOne = '?';
+
+Wildcard::Wildcard(const StringPiece& wildcard_spec)
+ : storage_(wildcard_spec.data(), wildcard_spec.size()) {
+ InitFromStorage();
+}
+
+void Wildcard::InitFromStorage() {
+ // Pre-scan the wildcard spec into an array of StringPieces. We will
+ // copy the original string spec into storage_ but that will only be
+ // backing-store for the StringPieces.
+ //
+ // E.g. we will transform "a?c*def**?xyz?" into
+ // {"a", "?", "c", "*", "def", "*", "?", "xyz", "?"}
+ // Note that multiple consecutive '*' will be collapsed into one.
+ //
+ // TODO(jmarantz): jmaessen suggests moving ? ahead of * as it will
+ // reduce code complexity and the cost of backtracking.
+ int num_pieces = 0;
+ char prev_c = '\0';
+ for (int i = 0, n = storage_.size(); i < n; ++i) {
+ char c = storage_[i];
+ bool add_to_previous = (num_pieces != 0);
+ bool skip = false;
+ if (add_to_previous) {
+ if (c == kMatchAny) {
+ add_to_previous = false;
+ skip = (prev_c == kMatchAny); // multiple * in a row means nothing
+ } else if (c == kMatchOne) {
+ add_to_previous = false;
+ } else {
+ add_to_previous = ((prev_c != kMatchAny) && (prev_c != kMatchOne));
+ }
+ }
+ if (add_to_previous) {
+ pieces_[num_pieces - 1] = StringPiece(pieces_[num_pieces - 1].data(),
+ pieces_[num_pieces - 1].size() + 1);
+ } else if (!skip) {
+ pieces_.push_back(StringPiece(storage_.data() + i, 1));
+ ++num_pieces;
+ }
+ prev_c = c;
+ }
+}
+
+bool Wildcard::MatchHelper(int piece_index, const StringPiece& str) {
+ bool prev_was_any = false;
+ size_t num_skips_after_any = 0;
+ size_t str_index = 0;
+
+ // Walk through the pieces parsed out in the constructor, matching them
+ // against str. Our algorithm will walk linealy through str, although we
+ // may need to backtrack if there are multiple matches for the segment
+ // following a *.
+ for (int num_pieces = pieces_.size(); piece_index < num_pieces; ++piece_index) {
+ StringPiece piece = pieces_[piece_index];
+ if (piece[0] == kMatchAny) {
+ prev_was_any = true;
+ } else if (piece[0] == kMatchOne) {
+ if (prev_was_any) {
+ ++num_skips_after_any;
+ } else {
+ ++str_index;
+ if (str_index > str.size()) {
+ return false;
+ }
+ }
+ } else if (prev_was_any) {
+ str_index += num_skips_after_any;
+ if (str_index > str.size()) {
+ return false;
+ }
+
+ // Now we have the unenviable task of figuring out how many
+ // characters of 'str' to swallow. Consider this complexity.
+ // CHECK(Wildcard("*abcd?").Match("abcabcdabcdabcdabcde"));
+ // If we greedily match the "a" in the wildcard against any of
+ // the first 4 'a's in the string then we are screwed -- we
+ // won't find the d. Even if we match against the first or
+ // second "abcd" we will get a failure because we will have
+ // string left, but no more pattern.
+ //
+ // There are probably more efficient ways to do this, such as in
+ // http://code.google.com/p/re2/, but we will, for short-term
+ // expediency, use recursion to search all the possible matches
+ // for the current piece in str.
+ while ((str_index = str.find(piece, str_index)) != StringPiece::npos) {
+ if (MatchHelper(piece_index + 1,
+ str.substr(str_index + piece.size()))) {
+ return true;
+ }
+ ++str_index;
+ }
+ return false;
+ } else if ((str.size() - str_index) < piece.size()) {
+ return false;
+ } else if (str.substr(str_index, piece.size()) == piece) {
+ str_index += piece.size();
+ } else {
+ return false;
+ }
+ }
+ if (prev_was_any) {
+ return (str.size() >= num_skips_after_any);
+ }
+ return (str_index == str.size());
+}
+
+bool Wildcard::IsSimple() const {
+ if (pieces_.size() == 0) {
+ return true;
+ }
+ if (pieces_.size() != 1) {
+ return false;
+ }
+ StringPiece piece = pieces_[0];
+ CHECK(!piece.empty());
+ char ch = piece[0];
+ return ((ch != kMatchAny) && (ch != kMatchOne));
+}
+
+Wildcard* Wildcard::Duplicate() const {
+ return new Wildcard(storage_);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/wildcard_group.cc b/trunk/src/net/instaweb/util/wildcard_group.cc
new file mode 100644
index 0000000..359d380
--- /dev/null
+++ b/trunk/src/net/instaweb/util/wildcard_group.cc
@@ -0,0 +1,70 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/wildcard_group.h"
+#include "net/instaweb/util/public/stl_util.h"
+#include "net/instaweb/util/public/wildcard.h"
+
+namespace net_instaweb {
+
+WildcardGroup::~WildcardGroup() {
+ STLDeleteElements(&wildcards_);
+}
+
+void WildcardGroup::Allow(const StringPiece& expr) {
+ Wildcard* wildcard = new Wildcard(expr);
+ wildcards_.push_back(wildcard);
+ allow_.push_back(true);
+}
+
+void WildcardGroup::Disallow(const StringPiece& expr) {
+ Wildcard* wildcard = new Wildcard(expr);
+ wildcards_.push_back(wildcard);
+ allow_.push_back(false);
+}
+
+bool WildcardGroup::Match(const StringPiece& str) const {
+ bool allow = true;
+ CHECK_EQ(wildcards_.size(), allow_.size());
+ for (int i = 0, n = wildcards_.size(); i < n; ++i) {
+ // Do not bother to execute the wildcard match if a match would
+ // not change the current 'allow' status. E.g. once we have found
+ // an 'allow' match, we can ignore all subsequent 'allow' tests
+ // until a rule disallows the match.
+ if ((allow != allow_[i]) && wildcards_[i]->Match(str)) {
+ allow = !allow;
+ }
+ }
+ return allow;
+}
+
+void WildcardGroup::CopyFrom(const WildcardGroup& src) {
+ wildcards_.clear();
+ allow_.clear();
+ AppendFrom(src);
+}
+
+void WildcardGroup::AppendFrom(const WildcardGroup& src) {
+ CHECK_EQ(src.wildcards_.size(), src.allow_.size());
+ for (int i = 0, n = src.wildcards_.size(); i < n; ++i) {
+ wildcards_.push_back(src.wildcards_[i]->Duplicate());
+ allow_.push_back(src.allow_[i]);
+ }
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/wildcard_group_test.cc b/trunk/src/net/instaweb/util/wildcard_group_test.cc
new file mode 100644
index 0000000..6a42194
--- /dev/null
+++ b/trunk/src/net/instaweb/util/wildcard_group_test.cc
@@ -0,0 +1,55 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/wildcard_group.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace net_instaweb {
+
+class WildcardGroupTest : public testing::Test {
+ protected:
+ virtual void SetUp() {
+ group_.Allow("*.cc");
+ group_.Allow("*.h");
+ group_.Disallow("a*.h");
+ group_.Allow("ab*.h");
+ group_.Disallow("c*.cc");
+ }
+
+ void TestGroup(const WildcardGroup& group) {
+ EXPECT_TRUE(group.Match("x.cc"));
+ EXPECT_FALSE(group.Match("c.cc"));
+ EXPECT_TRUE(group.Match("y.h"));
+ EXPECT_FALSE(group.Match("a.h"));
+ EXPECT_TRUE(group.Match("ab.h"));
+ }
+
+ WildcardGroup group_;
+};
+
+TEST_F(WildcardGroupTest, Sequence) {
+ TestGroup(group_);
+}
+
+TEST_F(WildcardGroupTest, CopySequence) {
+ WildcardGroup copy;
+ copy.CopyFrom(group_);
+ TestGroup(copy);
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/wildcard_test.cc b/trunk/src/net/instaweb/util/wildcard_test.cc
new file mode 100644
index 0000000..484daec
--- /dev/null
+++ b/trunk/src/net/instaweb/util/wildcard_test.cc
@@ -0,0 +1,104 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/wildcard.h"
+#include "net/instaweb/util/public/gtest.h"
+
+namespace net_instaweb {
+
+class WildcardTest : public testing::Test {
+};
+
+TEST_F(WildcardTest, Identity) {
+ EXPECT_TRUE(Wildcard("Hello").Match("Hello"));
+}
+
+TEST_F(WildcardTest, IdentityExtra) {
+ EXPECT_FALSE(Wildcard("Hello").Match("xHello"));
+ EXPECT_FALSE(Wildcard("Hello").Match("HelloxX"));
+}
+
+TEST_F(WildcardTest, OneStar) {
+ EXPECT_TRUE(Wildcard("H*o").Match("Hello"));
+}
+
+TEST_F(WildcardTest, OneQuestion) {
+ EXPECT_TRUE(Wildcard("H?llo").Match("Hello"));
+}
+
+TEST_F(WildcardTest, TwoQuestionSplit) {
+ EXPECT_TRUE(Wildcard("H?l?o").Match("Hello"));
+}
+
+TEST_F(WildcardTest, ThreeQuestionAdjacent) {
+ EXPECT_TRUE(Wildcard("H???o").Match("Hello"));
+}
+
+TEST_F(WildcardTest, SimpleMismatch) {
+ EXPECT_FALSE(Wildcard("Hello").Match("Goodbye"));
+}
+
+TEST_F(WildcardTest, GreedyTrap1) {
+ EXPECT_TRUE(Wildcard("*abcd").Match("abcabcabcabcabcd"));
+}
+
+TEST_F(WildcardTest, GreedyTrap2) {
+ EXPECT_FALSE(Wildcard("*abcd?").Match("abcabcabcabcabcd"));
+ EXPECT_TRUE(Wildcard("*abcd*").Match("abcabcabcabcabcd"));
+}
+
+TEST_F(WildcardTest, GreedyTrap3) {
+ EXPECT_TRUE(Wildcard("*abcd?").Match("abcabcabcabcabcdabcde"));
+}
+
+TEST_F(WildcardTest, StarAtBeginning) {
+ EXPECT_TRUE(Wildcard("*Hello").Match("Hello"));
+ EXPECT_TRUE(Wildcard("*ello").Match("Hello"));
+}
+
+TEST_F(WildcardTest, StarAtEnd) {
+ EXPECT_TRUE(Wildcard("Hello*").Match("Hello"));
+ EXPECT_TRUE(Wildcard("Hell*").Match("Hello"));
+}
+
+TEST_F(WildcardTest, QuestionAtBeginning) {
+ EXPECT_FALSE(Wildcard("?Hello").Match("Hello"));
+ EXPECT_TRUE(Wildcard("?ello").Match("Hello"));
+}
+
+TEST_F(WildcardTest, QuestionAtEnd) {
+ EXPECT_FALSE(Wildcard("Hello?").Match("Hello"));
+ EXPECT_TRUE(Wildcard("Hell?").Match("Hello"));
+}
+
+TEST_F(WildcardTest, Empty) {
+ EXPECT_TRUE(Wildcard("").Match(""));
+ EXPECT_FALSE(Wildcard("").Match("x"));
+ EXPECT_TRUE(Wildcard("*").Match(""));
+ EXPECT_FALSE(Wildcard("?").Match(""));
+}
+
+TEST_F(WildcardTest, Simple) {
+ EXPECT_FALSE(Wildcard("H*o").IsSimple());
+ EXPECT_TRUE(Wildcard("Hello").IsSimple());
+ EXPECT_TRUE(Wildcard("").IsSimple());
+ EXPECT_FALSE(Wildcard("*").IsSimple());
+ EXPECT_FALSE(Wildcard("?").IsSimple());
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/work_bound.cc b/trunk/src/net/instaweb/util/work_bound.cc
new file mode 100644
index 0000000..2cbd866
--- /dev/null
+++ b/trunk/src/net/instaweb/util/work_bound.cc
@@ -0,0 +1,25 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmaessen@google.com (Jan Maessen)
+
+#include "net/instaweb/util/public/work_bound.h"
+
+namespace net_instaweb {
+
+WorkBound::~WorkBound() { }
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/write_through_cache.cc b/trunk/src/net/instaweb/util/write_through_cache.cc
new file mode 100644
index 0000000..28dcee2
--- /dev/null
+++ b/trunk/src/net/instaweb/util/write_through_cache.cc
@@ -0,0 +1,73 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/write_through_cache.h"
+#include "net/instaweb/util/public/shared_string.h"
+
+namespace net_instaweb {
+
+const size_t WriteThroughCache::kUnlimited = static_cast<size_t>(-1);
+
+WriteThroughCache::~WriteThroughCache() {
+}
+
+void WriteThroughCache::PutInCache1(const std::string& key,
+ SharedString* value) {
+ if ((cache1_size_limit_ == kUnlimited) ||
+ (key.size() + value->size() < cache1_size_limit_)) {
+ cache1_->Put(key, value);
+ }
+}
+
+bool WriteThroughCache::Get(const std::string& key, SharedString* value) {
+ bool ret = cache1_->Get(key, value);
+ if (!ret) {
+ ret = cache2_->Get(key, value);
+ if (ret) {
+ PutInCache1(key, value);
+ }
+ }
+ return ret;
+}
+
+void WriteThroughCache::Put(const std::string& key, SharedString* value) {
+ PutInCache1(key, value);
+ cache2_->Put(key, value);
+}
+
+void WriteThroughCache::Delete(const std::string& key) {
+ cache1_->Delete(key);
+ cache2_->Delete(key);
+}
+
+CacheInterface::KeyState WriteThroughCache::Query(const std::string& key) {
+ // There are 3 possible states, kAvailable, kInTransit, and kNotFound.
+ // We want to 'promote' the state obtained from cache1. But we don't want
+ // to demote. Specifically, if cache1 says kInTransit, and cache2 says
+ // kNotFound, we'd want to return kInTransit.
+ CacheInterface::KeyState state = cache1_->Query(key);
+ if (state != kAvailable) {
+ CacheInterface::KeyState state2 = cache2_->Query(key);
+ if (state2 != kNotFound) {
+ state = state2;
+ }
+ }
+ return state;
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/write_through_cache_test.cc b/trunk/src/net/instaweb/util/write_through_cache_test.cc
new file mode 100644
index 0000000..a6cd28d
--- /dev/null
+++ b/trunk/src/net/instaweb/util/write_through_cache_test.cc
@@ -0,0 +1,126 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+// Unit-test the write-through cache
+
+#include "net/instaweb/util/public/write_through_cache.h"
+
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "net/instaweb/util/public/gtest.h"
+#include "net/instaweb/util/public/lru_cache.h"
+#include "net/instaweb/util/public/shared_string.h"
+#include <string>
+
+namespace net_instaweb {
+
+class WriteThroughCacheTest : public testing::Test {
+ protected:
+ WriteThroughCacheTest()
+ : small_cache_(new LRUCache(15)),
+ big_cache_(new LRUCache(1000)),
+ write_through_cache_(small_cache_, big_cache_) {
+ }
+
+ void CheckGet(CacheInterface* cache, const char* key,
+ const std::string& expected_value) {
+ SharedString value_buffer;
+ EXPECT_TRUE(cache->Get(key, &value_buffer));
+ EXPECT_EQ(expected_value, *value_buffer);
+ EXPECT_EQ(CacheInterface::kAvailable, cache->Query(key));
+ }
+
+ void Put(const char* key, const char* value) {
+ SharedString put_buffer(value);
+ write_through_cache_.Put(key, &put_buffer);
+ }
+
+ void CheckNotFound(CacheInterface* cache, const char* key) {
+ SharedString value_buffer;
+ EXPECT_FALSE(cache->Get(key, &value_buffer));
+ EXPECT_EQ(CacheInterface::kNotFound, cache->Query(key));
+ }
+
+ LRUCache* small_cache_;
+ LRUCache* big_cache_;
+ WriteThroughCache write_through_cache_;
+
+ private:
+ DISALLOW_COPY_AND_ASSIGN(WriteThroughCacheTest);
+};
+
+// Simple flow of putting in an item, getting it, deleting it.
+TEST_F(WriteThroughCacheTest, PutGetDelete) {
+ // First, put some small data into the write-through. It should
+ // be available in both caches.
+ Put("Name", "Value");
+ CheckGet(&write_through_cache_, "Name", "Value");
+ CheckGet(small_cache_, "Name", "Value");
+ CheckGet(big_cache_, "Name", "Value");
+
+ CheckNotFound(&write_through_cache_, "Another Name");
+
+ // next, put another value in. This will evict the first item
+ // out of the small cache,
+ Put("Name2", "NewValue");
+ CheckGet(&write_through_cache_, "Name2", "NewValue");
+ CheckGet(small_cache_, "Name2", "NewValue");
+ CheckGet(big_cache_, "Name2", "NewValue");
+
+ // The first item will still be available in the write-through,
+ // and in the big-cache, but will have been evicted from the
+ // small cache.
+ CheckNotFound(small_cache_, "Name");
+ CheckGet(big_cache_, "Name", "Value");
+ CheckNotFound(small_cache_, "Name");
+
+ CheckGet(&write_through_cache_, "Name", "Value");
+
+ // But now, once we've gotten in out of the write-through cache,
+ // the small cache will have the value "freshened."
+ CheckGet(small_cache_, "Name", "Value");
+
+ write_through_cache_.Delete("Name2");
+ CheckNotFound(&write_through_cache_, "Name2");
+ CheckNotFound(small_cache_, "Name2");
+ CheckNotFound(big_cache_, "Name2");
+}
+
+// Check size-limits for the small cache
+TEST_F(WriteThroughCacheTest, SizeLimit) {
+ write_through_cache_.set_cache1_limit(10);
+
+ // This one will fit.
+ Put("Name", "Value");
+ CheckGet(&write_through_cache_, "Name", "Value");
+ CheckGet(small_cache_, "Name", "Value");
+ CheckGet(big_cache_, "Name", "Value");
+
+ // This one will not.
+ Put("Name2", "TooBig");
+ CheckGet(&write_through_cache_, "Name2", "TooBig");
+ CheckNotFound(small_cache_, "Name2");
+ CheckGet(big_cache_, "Name2", "TooBig");
+
+ // However "Name" is still in both caches.
+ CheckGet(small_cache_, "Name", "Value");
+ CheckGet(&write_through_cache_, "Name", "Value");
+ CheckGet(big_cache_, "Name", "Value");
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/net/instaweb/util/writer.cc b/trunk/src/net/instaweb/util/writer.cc
new file mode 100644
index 0000000..7a8cda9
--- /dev/null
+++ b/trunk/src/net/instaweb/util/writer.cc
@@ -0,0 +1,26 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: jmarantz@google.com (Joshua Marantz)
+
+#include "net/instaweb/util/public/writer.h"
+
+namespace net_instaweb {
+
+Writer::~Writer() {
+}
+
+} // namespace net_instaweb
diff --git a/trunk/src/third_party/base64/README.google b/trunk/src/third_party/base64/README.google
new file mode 100644
index 0000000..b0519f0
--- /dev/null
+++ b/trunk/src/third_party/base64/README.google
@@ -0,0 +1,36 @@
+Modified version of base64.
+
+URL:http://www.adp-gmbh.ch/cpp/common/base64.html
+
+ base64.cpp and base64.h
+
+ Copyright (C) 2004-2008 René Nyffenegger
+
+ This source code is provided 'as-is', without any express or implied
+ warranty. In no event will the author be held liable for any damages
+ arising from the use of this software.
+
+ Permission is granted to anyone to use this software for any purpose,
+ including commercial applications, and to alter it and redistribute it
+ freely, subject to the following restrictions:
+
+ 1. The origin of this source code must not be misrepresented; you must not
+ claim that you wrote the original source code. If you use this source code
+ in a product, an acknowledgment in the product documentation would be
+ appreciated but is not required.
+
+ 2. Altered source versions must be plainly marked as such, and must not be
+ misrepresented as being the original source code.
+
+ 3. This notice may not be removed or altered from any source distribution.
+
+ René Nyffenegger rene.nyffenegger@adp-gmbh.ch
+
+Modified by jmarantz@google.com to:
+ - Add a static map and an initializer function
+ - add a web-safe variant that uses '-' and '_' in lieu of '+' and '/'
+ - change the 'decode' interface to return false if the input contains
+ characters not in the expected char-set.
+ - removed is_base64 helper function and instead rely on the char_maps.
+ - use a static constant for the padding character '='
+
diff --git a/trunk/src/third_party/base64/base64.cc b/trunk/src/third_party/base64/base64.cc
new file mode 100644
index 0000000..6f02fe3
--- /dev/null
+++ b/trunk/src/third_party/base64/base64.cc
@@ -0,0 +1,199 @@
+/*
+ From: http://www.adp-gmbh.ch/cpp/common/base64.html
+
+ base64.cpp and base64.h
+
+ Copyright (C) 2004-2008 René Nyffenegger
+
+ This source code is provided 'as-is', without any express or implied
+ warranty. In no event will the author be held liable for any damages
+ arising from the use of this software.
+
+ Permission is granted to anyone to use this software for any purpose,
+ including commercial applications, and to alter it and redistribute it
+ freely, subject to the following restrictions:
+
+ 1. The origin of this source code must not be misrepresented; you must not
+ claim that you wrote the original source code. If you use this source code
+ in a product, an acknowledgment in the product documentation would be
+ appreciated but is not required.
+
+ 2. Altered source versions must be plainly marked as such, and must not be
+ misrepresented as being the original source code.
+
+ 3. This notice may not be removed or altered from any source distribution.
+
+ René Nyffenegger rene.nyffenegger@adp-gmbh.ch
+
+ This file was modified by jmarantz@google.com to:
+ - Add a static map and an initializer function
+ - add a web-safe variant that uses '-' and '_' in lieu of '+' and '/'
+ - change the 'decode' interface to return false if the input contains
+ characters not in the expected char-set.
+ - removed is_base64 helper function and instead rely on the char_maps.
+ - use a static constant for the padding character '='
+*/
+
+#include "base64.h"
+#include <assert.h>
+
+static const char base64_chars[] =
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "abcdefghijklmnopqrstuvwxyz"
+ "0123456789+/";
+
+static const char web64_chars[] =
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "abcdefghijklmnopqrstuvwxyz"
+ "0123456789-_";
+
+static unsigned int base64_char_map[256], web64_char_map[256];
+static bool initialized = false;
+static const unsigned int kInvalidChar = 0xffffffff;
+static const char kPadChar = '=';
+
+// Be sure to call this function before calling any other functions
+// in this file when using multiple threads.
+void base64_init() {
+ if (!initialized) {
+ initialized = true;
+ for (int i = 0; i < 256; ++i) {
+ base64_char_map[i] = kInvalidChar;
+ web64_char_map[i] = kInvalidChar;
+ }
+ for (int i = 0; i < sizeof(base64_chars) - 1; ++i) {
+ base64_char_map[base64_chars[i]] = i;
+ }
+ for (int i = 0; i < sizeof(web64_chars) - 1; ++i) {
+ web64_char_map[web64_chars[i]] = i;
+ }
+ }
+}
+
+static inline std::string encode(
+ const char* char_set, unsigned char const* bytes_to_encode,
+ unsigned int in_len) {
+ base64_init();
+ std::string ret;
+ int i = 0;
+ int j = 0;
+ unsigned char char_array_3[3];
+ unsigned char char_array_4[4];
+
+ while (in_len--) {
+ char_array_3[i++] = *(bytes_to_encode++);
+ if (i == 3) {
+ char_array_4[0] = (char_array_3[0] & 0xfc) >> 2;
+ char_array_4[1] = ((char_array_3[0] & 0x03) << 4) +
+ ((char_array_3[1] & 0xf0) >> 4);
+ char_array_4[2] = ((char_array_3[1] & 0x0f) << 2) +
+ ((char_array_3[2] & 0xc0) >> 6);
+ char_array_4[3] = char_array_3[2] & 0x3f;
+
+ for (i = 0; i < 4 ; i++)
+ ret += char_set[char_array_4[i]];
+ i = 0;
+ }
+ }
+
+ if (i) {
+ for (j = i; j < 3; j++) {
+ char_array_3[j] = '\0';
+ }
+
+ char_array_4[0] = (char_array_3[0] & 0xfc) >> 2;
+ char_array_4[1] = ((char_array_3[0] & 0x03) << 4) +
+ ((char_array_3[1] & 0xf0) >> 4);
+ char_array_4[2] = ((char_array_3[1] & 0x0f) << 2) +
+ ((char_array_3[2] & 0xc0) >> 6);
+ char_array_4[3] = char_array_3[2] & 0x3f;
+
+ for (j = 0; (j < i + 1); j++)
+ ret += char_set[char_array_4[j]];
+
+ while ((i++ < 3)) {
+ ret += kPadChar;
+ }
+ }
+
+ return ret;
+}
+
+static inline bool decode(
+ unsigned int* char_map, const std::string& encoded_string,
+ std::string* output) {
+ base64_init();
+ int in_len = encoded_string.size();
+ int i = 0;
+ int j = 0;
+ int in_ = 0;
+ unsigned char char_array_4[4], char_array_3[3];
+ std::string ret;
+
+ while (in_len-- && (encoded_string[in_] != kPadChar)) {
+ char_array_4[i++] = encoded_string[in_];
+ in_++;
+ if (i == 4) {
+ for (i = 0; i <4; i++) {
+ unsigned int char_index = char_map[char_array_4[i]];
+ if (char_index == kInvalidChar) {
+ return false;
+ }
+ char_array_4[i] = char_index;
+ }
+
+ char_array_3[0] = (char_array_4[0] << 2) +
+ ((char_array_4[1] & 0x30) >> 4);
+ char_array_3[1] = ((char_array_4[1] & 0xf) << 4) +
+ ((char_array_4[2] & 0x3c) >> 2);
+ char_array_3[2] = ((char_array_4[2] & 0x3) << 6) + char_array_4[3];
+
+ for (i = 0; (i < 3); i++)
+ *output += char_array_3[i];
+ i = 0;
+ }
+ }
+
+ if (i) {
+ for (j = i; j < 4; j++)
+ char_array_4[j] = 0;
+
+ for (j = 0; j < i; j++) {
+ unsigned int char_index = char_map[char_array_4[j]];
+ if (char_index == kInvalidChar) {
+ return false;
+ }
+ char_array_4[j] = char_index;
+ }
+
+ char_array_3[0] = (char_array_4[0] << 2) + ((char_array_4[1] & 0x30) >> 4);
+ char_array_3[1] = ((char_array_4[1] & 0xf) << 4) +
+ ((char_array_4[2] & 0x3c) >> 2);
+ char_array_3[2] = ((char_array_4[2] & 0x3) << 6) + char_array_4[3];
+
+ for (j = 0; (j < i - 1); j++) {
+ *output += char_array_3[j];
+ }
+ }
+
+ return true;
+}
+
+
+std::string base64_encode(unsigned char const* bytes_to_encode,
+ unsigned int in_len) {
+ return encode(base64_chars, bytes_to_encode, in_len);
+}
+
+std::string web64_encode(unsigned char const* bytes_to_encode,
+ unsigned int in_len) {
+ return encode(web64_chars, bytes_to_encode, in_len);
+}
+
+bool base64_decode(const std::string& encoded_string, std::string* output) {
+ return decode(base64_char_map, encoded_string, output);
+}
+
+bool web64_decode(const std::string& encoded_string, std::string* output) {
+ return decode(web64_char_map, encoded_string, output);
+}
diff --git a/trunk/src/third_party/base64/base64.gyp b/trunk/src/third_party/base64/base64.gyp
new file mode 100644
index 0000000..7623f74
--- /dev/null
+++ b/trunk/src/third_party/base64/base64.gyp
@@ -0,0 +1,29 @@
+# Copyright 2009 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'targets': [
+ {
+ 'target_name': 'base64',
+ 'type': '<(library)',
+ 'sources': [
+ 'base64.cc',
+ ],
+ 'include_dirs': [
+ '.',
+ ],
+ }
+ ],
+}
+
diff --git a/trunk/src/third_party/base64/base64.h b/trunk/src/third_party/base64/base64.h
new file mode 100644
index 0000000..8d32b1f
--- /dev/null
+++ b/trunk/src/third_party/base64/base64.h
@@ -0,0 +1,51 @@
+/*
+ From: http://www.adp-gmbh.ch/cpp/common/base64.html
+
+ base64.cpp and base64.h
+
+ Copyright (C) 2004-2008 René Nyffenegger
+
+ This source code is provided 'as-is', without any express or implied
+ warranty. In no event will the author be held liable for any damages
+ arising from the use of this software.
+
+ Permission is granted to anyone to use this software for any purpose,
+ including commercial applications, and to alter it and redistribute it
+ freely, subject to the following restrictions:
+
+ 1. The origin of this source code must not be misrepresented; you must not
+ claim that you wrote the original source code. If you use this source code
+ in a product, an acknowledgment in the product documentation would be
+ appreciated but is not required.
+
+ 2. Altered source versions must be plainly marked as such, and must not be
+ misrepresented as being the original source code.
+
+ 3. This notice may not be removed or altered from any source distribution.
+
+ René Nyffenegger rene.nyffenegger@adp-gmbh.ch
+
+ This file was modified by jmarantz@google.com to:
+ - add a web-safe variant that uses '-' and '_' in lieu of '+' and '/'
+ - change the 'decode' interface to return false if the input contains
+ characters not in the expected char-set.
+*/
+
+#ifndef THIRD_PARTY_BASE64_BASE64_H_
+#define THIRD_PARTY_BASE64_BASE64_H_
+
+#include <string>
+
+// Initialize the base64 library. Call this before other methods if
+// multi-threaded access is required.
+void base64_init();
+
+// Encode/deocde strings in common base64 encoding
+std::string base64_encode(const unsigned char* data, unsigned int len);
+bool base64_decode(const std::string& s, std::string* output);
+
+// Encode/deocde strings in a base64 encoding for URL values.
+std::string web64_encode(const unsigned char* data, unsigned int len);
+bool web64_decode(const std::string& s, std::string* output);
+
+#endif // THIRD_PARTY_BASE64_BASE64_H_
diff --git a/trunk/src/third_party/chromium/src/net/tools/dump_cache.gyp b/trunk/src/third_party/chromium/src/net/tools/dump_cache.gyp
new file mode 100644
index 0000000..6c6bb05
--- /dev/null
+++ b/trunk/src/third_party/chromium/src/net/tools/dump_cache.gyp
@@ -0,0 +1,43 @@
+# Copyright 2010 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'variables': {
+ # chromium_code indicates that the code is not
+ # third-party code and should be subjected to strict compiler
+ # warnings/errors in order to catch programming mistakes.
+ 'chromium_code': 1,
+ },
+
+ 'targets': [
+ {
+ 'target_name': 'url_to_filename_encoder',
+ 'type': '<(library)',
+ 'dependencies': [
+ '<(DEPTH)/base/base.gyp:base',
+ ],
+ 'include_dirs': [
+ 'dump_cache',
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ 'dump_cache',
+ ],
+ },
+ 'sources': [
+ 'dump_cache/url_to_filename_encoder.cc',
+ ],
+ },
+ ],
+}
diff --git a/trunk/src/third_party/css_parser/css_parser.gyp b/trunk/src/third_party/css_parser/css_parser.gyp
new file mode 100644
index 0000000..57be7dc
--- /dev/null
+++ b/trunk/src/third_party/css_parser/css_parser.gyp
@@ -0,0 +1,144 @@
+# Copyright 2009 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'variables': {
+ # chromium_code indicates that the code is not
+ # third-party code and should be subjected to strict compiler
+ # warnings/errors in order to catch programming mistakes.
+ 'chromium_code': 1,
+ 'css_parser_root': 'src',
+ },
+
+ 'targets': [
+ {
+ 'target_name': 'css_parser',
+ 'type': '<(library)',
+ 'dependencies': [
+ '<(DEPTH)/base/base.gyp:base',
+ '<(DEPTH)/third_party/gflags/gflags.gyp:gflags',
+ '<(DEPTH)/third_party/google-sparsehash/google-sparsehash.gyp:include',
+ ],
+ 'export_dependent_settings': [
+ '<(DEPTH)/third_party/google-sparsehash/google-sparsehash.gyp:include',
+ ],
+ 'include_dirs': [
+ '<(css_parser_root)',
+ '<(DEPTH)',
+ ],
+ 'cflags': ['-funsigned-char', '-Wno-sign-compare', '-Wno-return-type'],
+ 'sources': [
+ '<(css_parser_root)/string_using.h',
+ '<(css_parser_root)/webutil/css/identifier.cc',
+ '<(css_parser_root)/webutil/css/identifier.h',
+ '<(css_parser_root)/webutil/css/parser.cc',
+ '<(css_parser_root)/webutil/css/parser.h',
+ '<(css_parser_root)/webutil/css/property.cc',
+ '<(css_parser_root)/webutil/css/property.h',
+ '<(css_parser_root)/webutil/css/selector.cc',
+ '<(css_parser_root)/webutil/css/selector.h',
+ '<(css_parser_root)/webutil/css/string.h',
+ '<(css_parser_root)/webutil/css/string_util.cc',
+ '<(css_parser_root)/webutil/css/string_util.h',
+ '<(css_parser_root)/webutil/css/tostring.cc',
+ '<(css_parser_root)/webutil/css/util.cc',
+ '<(css_parser_root)/webutil/css/util.h',
+ '<(css_parser_root)/webutil/css/value.cc',
+ '<(css_parser_root)/webutil/css/value.h',
+ '<(css_parser_root)/webutil/css/valuevalidator.cc',
+ '<(css_parser_root)/webutil/css/valuevalidator.h',
+
+ #'<(css_parser_root)/webutil/css/parse_arg.cc',
+ # Tests
+ #'<(css_parser_root)/webutil/css/gtest_main.cc',
+ #'<(css_parser_root)/webutil/css/identifier_test.cc',
+ #'<(css_parser_root)/webutil/css/parser_unittest.cc',
+ #'<(css_parser_root)/webutil/css/property_test.cc',
+ #'<(css_parser_root)/webutil/css/tostring_test.cc',
+ #'<(css_parser_root)/webutil/css/util_test.cc',
+ #'<(css_parser_root)/webutil/css/valuevalidator_test.cc',
+
+ '<(css_parser_root)/webutil/html/htmlcolor.cc',
+ '<(css_parser_root)/webutil/html/htmlcolor.h',
+ '<(css_parser_root)/webutil/html/htmltagenum.cc',
+ '<(css_parser_root)/webutil/html/htmltagenum.h',
+ '<(css_parser_root)/webutil/html/htmltagindex.cc',
+ '<(css_parser_root)/webutil/html/htmltagindex.h',
+
+ # UnicodeText
+ '<(css_parser_root)/util/utf8/internal/unicodetext.cc',
+ '<(css_parser_root)/util/utf8/internal/unilib.cc',
+ '<(css_parser_root)/util/utf8/public/config.h',
+ '<(css_parser_root)/util/utf8/public/unicodetext.h',
+ '<(css_parser_root)/util/utf8/public/unilib.h',
+
+ # libutf
+ #'<(css_parser_root)/third_party/utf/Make.Linux-x86_64',
+ #'<(css_parser_root)/third_party/utf/Makefile',
+ #'<(css_parser_root)/third_party/utf/NOTICE',
+ #'<(css_parser_root)/third_party/utf/README',
+ '<(css_parser_root)/third_party/utf/rune.c',
+ '<(css_parser_root)/third_party/utf/runestrcat.c',
+ '<(css_parser_root)/third_party/utf/runestrchr.c',
+ '<(css_parser_root)/third_party/utf/runestrcmp.c',
+ '<(css_parser_root)/third_party/utf/runestrcpy.c',
+ '<(css_parser_root)/third_party/utf/runestrecpy.c',
+ '<(css_parser_root)/third_party/utf/runestrlen.c',
+ '<(css_parser_root)/third_party/utf/runestrncat.c',
+ '<(css_parser_root)/third_party/utf/runestrncmp.c',
+ '<(css_parser_root)/third_party/utf/runestrncpy.c',
+ '<(css_parser_root)/third_party/utf/runestrrchr.c',
+ '<(css_parser_root)/third_party/utf/runestrstr.c',
+ '<(css_parser_root)/third_party/utf/runetype.c',
+ # TODO(sligocki): What is the correct format for this?
+ # runetypebody-5.0.0.c should not be compiled by itself, only #included.
+ #'<(css_parser_root)/third_party/utf/runetypebody-5.0.0.c',
+ '<(css_parser_root)/third_party/utf/utf.h',
+ '<(css_parser_root)/third_party/utf/utfdef.h',
+ '<(css_parser_root)/third_party/utf/utfecpy.c',
+ '<(css_parser_root)/third_party/utf/utflen.c',
+ '<(css_parser_root)/third_party/utf/utfnlen.c',
+ '<(css_parser_root)/third_party/utf/utfrrune.c',
+ '<(css_parser_root)/third_party/utf/utfrune.c',
+ '<(css_parser_root)/third_party/utf/utfutf.c',
+
+ # Supporting interfaces.
+ '<(css_parser_root)/base/commandlineflags.h',
+ '<(css_parser_root)/base/googleinit.h',
+ '<(css_parser_root)/base/macros.h',
+ '<(css_parser_root)/base/paranoid.h',
+ '<(css_parser_root)/base/stringprintf.h',
+ '<(css_parser_root)/string_using.h',
+ '<(css_parser_root)/strings/ascii_ctype.cc',
+ '<(css_parser_root)/strings/ascii_ctype.h',
+ '<(css_parser_root)/strings/escaping.h',
+ '<(css_parser_root)/strings/join.h',
+ '<(css_parser_root)/strings/memutil.h',
+ '<(css_parser_root)/strings/stringpiece.h',
+ '<(css_parser_root)/strings/stringpiece_utils.cc',
+ '<(css_parser_root)/strings/stringpiece_utils.h',
+ '<(css_parser_root)/strings/stringprintf.h',
+ '<(css_parser_root)/strings/strutil.h',
+ #'<(css_parser_root)/testing/base/public/googletest.h',
+ #'<(css_parser_root)/testing/base/public/gunit.h',
+ '<(css_parser_root)/testing/production_stub/public/gunit_prod.h',
+ '<(css_parser_root)/util/gtl/dense_hash_map.h',
+ '<(css_parser_root)/util/gtl/map-util.h',
+ '<(css_parser_root)/util/gtl/singleton.h',
+ '<(css_parser_root)/util/gtl/stl_util-inl.h',
+ '<(css_parser_root)/util/hash/hash.h',
+ ],
+ },
+ ],
+}
diff --git a/trunk/src/third_party/css_parser/src/Makefile b/trunk/src/third_party/css_parser/src/Makefile
new file mode 100644
index 0000000..f1d7099
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/Makefile
@@ -0,0 +1,110 @@
+CXX = g++
+COMPILE_OPTS = -g -Wall -Werror -Wno-sign-compare -Wno-return-type \
+ -funsigned-char -fdiagnostics-show-option
+# TODO(sligocki): -Wno-error=sign-compare -Wno-error=return-type
+# TODO(sligocki): Why is LOG(FATAL) not considered a return?
+# TODO(sligocki): Why isn't -fdiagnostics-show-option working?
+#GPERF = false # Must explicitly set GPERF
+#GPERF_OPTS = --ignore-case -L C++ -S 1 -t -C -c -E -G -o
+
+SRC_DIR = /home/sligocki/src/css_parsing
+PAGE_SPEED_DIR = $(SRC_DIR)/pagespeed/src
+CHROMIUM_DIR = $(PAGE_SPEED_DIR)/third_party/chromium/src
+GFLAGS_ROOT = $(SRC_DIR)/gflags-1.3/install
+GTEST_ROOT = $(SRC_DIR)/gtest-1.5.0/install
+GHASH_DIR = $(SRC_DIR)/sparsehash-1.8.1/install/include
+
+INCLUDES = -I. -I$(SRC_DIR) -I$(CHROMIUM_DIR) -I$(GFLAGS_ROOT)/include -I$(GHASH_DIR)
+
+PAGE_SPEED_LIBS = -L$(PAGE_SPEED_DIR)/out/Release/obj.target/base -lbase -lrt
+GFLAGS_LIBS = $(GFLAGS_ROOT)/lib/libgflags.a
+GTEST_LIBS = $(GTEST_ROOT)/lib/libgtest.a
+
+LIBS = $(PAGE_SPEED_LIBS) $(GFLAGS_LIBS) $(GTEST_LIBS)
+
+OBJ_DIR = genfiles
+
+
+# Generated files are included, you do not need to build them.
+#CSS_GPERFS = $(wildcard webutil/css/*.gperf)
+#CSS_GPERF_CC = $(CSS_GPERFS:%.gperf=$(OBJ_DIR)/%.cc) # Generated files.
+
+CSS_HDRS = $(wildcard webutil/css/*.h)
+CSS_SRCS = webutil/css/parser.cc webutil/css/identifier.cc webutil/css/property.cc webutil/css/selector.cc webutil/css/string_util.cc webutil/css/tostring.cc webutil/css/util.cc webutil/css/value.cc webutil/css/valuevalidator.cc $(CSS_GPERF_CC)
+CSS_OBJS = $(CSS_SRCS:%.cc=$(OBJ_DIR)/%.o)
+
+CSS_TEST_SRCS = $(wildcard webutil/css/*test.cc)
+CSS_TEST_OBJS = $(CSS_TEST_SRCS:%.cc=$(OBJ_DIR)/%.o)
+CSS_TESTS = $(CSS_TEST_SRCS:%.cc=$(OBJ_DIR)/%)
+
+HTML_HDRS = $(wildcard webutil/html/*.h)
+HTML_SRCS = $(wildcard webutil/html/*.cc)
+HTML_OBJS = $(HTML_SRCS:%.cc=$(OBJ_DIR)/%.o)
+
+UNICODETEXT_HDRS = $(wildcard util/utf8/public/*.h)
+UNICODETEXT_SRCS = $(wildcard util/utf8/internal/*.cc)
+UNICODETEXT_OBJS = $(UNICODETEXT_SRCS:%.cc=$(OBJ_DIR)/%.o)
+
+OTHER_SRCS = strings/ascii_ctype.cc strings/stringpiece_utils.cc
+OTHER_OBJS = $(OTHER_SRCS:%.cc=$(OBJ_DIR)/%.o) third_party/utf/libutf.a
+
+OBJS = $(CSS_OBJS) $(HTML_OBJS) $(UNICODETEXT_OBJS) $(OTHER_OBJS)
+
+all: parse_args run_tests
+
+PARSE_ARGS_MAIN = $(OBJ_DIR)/webutil/css/parse_arg.o
+parse_args: $(PARSE_ARGS_MAIN) $(OBJS)
+ $(CXX) $^ $(LIBS) -o $@
+
+run_tests: $(CSS_TESTS)
+ STATUS=0; \
+ for x in $^; do \
+ echo -e "\n** Running $$x **"; \
+ ./$$x || STATUS=$$?; \
+ done; \
+ exit $$STATUS # So that build fails if any test fails
+
+TEST_MAIN = $(OBJ_DIR)/webutil/css/gtest_main.o
+%test: %test.o $(TEST_MAIN) $(OBJS)
+ $(CXX) $^ $(LIBS) -o $@
+
+$(OBJ_DIR)/%.o: %.cc # TODO(sligocki): Add dep on headers.
+ mkdir -p `dirname $@`
+ $(CXX) $(COMPILE_OPTS) $< -c $(INCLUDES) -o $@
+
+
+#$(OBJ_DIR)/webutil/css/property.cc: $(OBJ_DIR)/%.cc: %.gperf $(CSS_HDRS)
+# mkdir -p `dirname $@`
+# $(GPERF) $(GPERF_OPTS) -ZPropertyMapper $< > $@
+#
+#$(OBJ_DIR)/webutil/css/identifier.cc: $(OBJ_DIR)/%.cc: %.gperf $(CSS_HDRS)
+# mkdir -p `dirname $@`
+# $(GPERF) $(GPERF_OPTS) -ZIdentifierMapper $< > $@
+
+
+third_party/utf/libutf.a: $(wildcard third_party/utf/*.cc) $(wildcard third_party/utf/*.h)
+ make -C third_party/utf CC=$(CXX)
+
+
+clean:
+ chmod +w -R .
+ rm -f $(OBJ_DIR) css
+
+
+# Prepare files by adding license and other details needed for open-source.
+prepare_files:
+# Prepare webutil/html files (first by adding #include "string_using.h")
+ mkdir -p webutil/html; \
+ for f in `ls original/webutil/html/`; do \
+ cp apache-license.txt webutil/html/$$f; \
+ sed 's@#include <string>@\0\n#include "string_using.h"@' \
+ < original/webutil/html/$$f \
+ >> webutil/html/$$f; \
+ rm -f original/webutil/html/$$f; \
+ done
+# The rest of the files just need to have licenses added.
+ for f in `cd original/ && find . -type f`; do \
+ mkdir -p `dirname $$f`; \
+ cat apache-license.txt original/$$f > $$f; \
+ done
+ rm -rf original/
\ No newline at end of file
diff --git a/trunk/src/third_party/css_parser/src/README b/trunk/src/third_party/css_parser/src/README
new file mode 100644
index 0000000..1eb4590
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/README
@@ -0,0 +1,75 @@
+==================== Dependencies ==========================
+
+CSS parser depends upon:
+ * gflags: http://code.google.com/p/google-gflags/
+ * googletest: http://code.google.com/p/googletest/
+ * google-sparsehash: http://code.google.com/p/google-sparsehash/
+ * page-speed: http://code.google.com/p/page-speed/
+
+It also uses
+ * gperf: http://www.gnu.org/software/gperf/
+to generate .cc files. However, the generated .cc files are included in this
+distribution for convenience, so that gperf is not needed to build.
+
+To prepare gflags:
+
+$ cd $SRC_DIR
+$ wget http://google-gflags.googlecode.com/files/gflags-1.3.tar.gz
+$ tar -xzf gflags-1.3.tar.gz
+$ cd gflags-1.3
+$ ./configure --prefix=$PWD/install
+$ make
+$ make install
+
+To prepare googletest:
+
+$ cd $SRC_DIR
+$ wget http://googletest.googlecode.com/files/gtest-1.5.0.tar.gz
+$ tar -xzf gtest-1.5.0.tar.gz
+$ cd gtext-1.5.0
+$ ./configure --prefix=$PWD/install
+$ make
+$ make install
+
+To prepare google-sparsehash:
+
+$ cd $SRC_DIR
+$ wget http://google-sparsehash.googlecode.com/files/sparsehash-1.8.1.tar.gz
+$ tar -xzf sparsehash-1.8.1.tar.gz
+$ cd sparsehash-1.8.1
+$ ./configure --prefix=$PWD/install
+$ make
+$ make install
+
+To prepare pagespeed:
+
+$ cd $SRC_DIR
+$ mkdir pagespeed
+$ cd pagespeed
+$ gclient config http://page-speed.googlecode.com/svn/lib/trunk/src
+$ gclient update --force
+$ cd src
+$ BUILDTYPE=Release make -j4
+
+You will need gclient installed, see pagespeed documentation:
+ http://code.google.com/p/page-speed/wiki/HowToBuildNativeLibraries
+
+
+==================== Building ==============================
+
+Once the dependencies have been built as described above (in some directory
+$SRC_DIR) build UnicodeText:
+
+$ make SRC_DIR=$SRC_DIR
+
+This builds the libraries and a simple program parse_args you can use
+to test. Ex:
+
+$ ./parse_args "body,p,h1 { font-size:70%; color:#000000; background-color:#f1f1ed;} this.is¬#valid"
+Parsing:
+body,p,h1 { font-size:70%; color:#000000; background-color:#f1f1ed;} this.is¬#valid
+
+Stylesheet:
+/* AUTHOR */
+
+body,p,h1 {font-size: 70%;color: #000000;background-color: #f1f1ed}
diff --git a/trunk/src/third_party/css_parser/src/apache-license.txt b/trunk/src/third_party/css_parser/src/apache-license.txt
new file mode 100644
index 0000000..d74e930
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/apache-license.txt
@@ -0,0 +1,16 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
diff --git a/trunk/src/third_party/css_parser/src/base/commandlineflags.h b/trunk/src/third_party/css_parser/src/base/commandlineflags.h
new file mode 100644
index 0000000..fe8d85e
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/base/commandlineflags.h
@@ -0,0 +1,22 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef BASE_COMMANDLINEFLAGS_H_
+#define BASE_COMMANDLINEFLAGS_H_
+
+//#include "third_party/gflags/src/gflags/gflags.h"
+#include "gflags/gflags.h"
+
+#endif // BASE_COMMANDLINEFLAGS_H_
diff --git a/trunk/src/third_party/css_parser/src/base/googleinit.h b/trunk/src/third_party/css_parser/src/base/googleinit.h
new file mode 100644
index 0000000..5f49ea2
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/base/googleinit.h
@@ -0,0 +1,52 @@
+
+
+// Copyright (c) 2005, Google Inc.
+// All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions are
+// met:
+//
+// * Redistributions of source code must retain the above copyright
+// notice, this list of conditions and the following disclaimer.
+// * Redistributions in binary form must reproduce the above
+// copyright notice, this list of conditions and the following disclaimer
+// in the documentation and/or other materials provided with the
+// distribution.
+// * Neither the name of Google Inc. nor the names of its
+// contributors may be used to endorse or promote products derived from
+// this software without specific prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+// ---
+
+#ifndef _GOOGLEINIT_H
+#define _GOOGLEINIT_H
+
+class GoogleInitializer {
+ public:
+ typedef void (*void_function)(void);
+ GoogleInitializer(const char* name, void_function f) {
+ f();
+ }
+};
+
+#define REGISTER_MODULE_INITIALIZER(name, body) \
+ namespace { \
+ static void google_init_module_##name () { body; } \
+ GoogleInitializer google_initializer_module_##name(#name, \
+ google_init_module_##name); \
+ }
+
+#endif /* _GOOGLEINIT_H */
diff --git a/trunk/src/third_party/css_parser/src/base/macros.h b/trunk/src/third_party/css_parser/src/base/macros.h
new file mode 100644
index 0000000..aed0b77
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/base/macros.h
@@ -0,0 +1,23 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef BASE_MACROS_H_
+#define BASE_MACROS_H_
+
+//#include "third_party/chromium/src/base/basictypes.h"
+#include "base/basictypes.h"
+// For DISALLOW_COPY_AND_ASSIGN.
+
+#endif // BASE_MACROS_H_
diff --git a/trunk/src/third_party/css_parser/src/base/paranoid.h b/trunk/src/third_party/css_parser/src/base/paranoid.h
new file mode 100644
index 0000000..571ec0f
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/base/paranoid.h
@@ -0,0 +1,22 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef BASE_PARANOID_H_
+#define BASE_PARANOID_H_
+
+inline bool SanitizeBool(bool b) { return b; }
+
+#endif // BASE_PARANOID_H_
diff --git a/trunk/src/third_party/css_parser/src/base/stringprintf.h b/trunk/src/third_party/css_parser/src/base/stringprintf.h
new file mode 100644
index 0000000..e01096f
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/base/stringprintf.h
@@ -0,0 +1,23 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef BASE_STRINGPRINTF_H_
+#define BASE_STRINGPRINTF_H_
+
+//#include "third_party/chromium/src/base/string_util.h"
+#include "base/string_util.h"
+
+#endif // BASE_STRINGPRINTF_H_
diff --git a/trunk/src/third_party/css_parser/src/string_using.h b/trunk/src/third_party/css_parser/src/string_using.h
new file mode 100644
index 0000000..f460dbc
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/string_using.h
@@ -0,0 +1,9 @@
+// Copyright 2010 Google Inc. All Rights Reserved.
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef WEBUTIL_CSS_OPEN_SOURCE_STRING_USING_H_
+#define WEBUTIL_CSS_OPEN_SOURCE_STRING_USING_H_
+
+using std::string;
+
+#endif // WEBUTIL_CSS_OPEN_SOURCE_STRING_USING_H_
diff --git a/trunk/src/third_party/css_parser/src/strings/ascii_ctype.cc b/trunk/src/third_party/css_parser/src/strings/ascii_ctype.cc
new file mode 100644
index 0000000..3b4229f
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/strings/ascii_ctype.cc
@@ -0,0 +1,103 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2007 Google Inc. All Rights Reserved.
+// Author: dpeng@google.com (Daniel Peng)
+
+#include "strings/ascii_ctype.h"
+
+// # Table generated by this Python code (bit 0x02 is currently unused):
+// def Hex2(n):
+// return '0x' + hex(n/16)[2:] + hex(n%16)[2:]
+// def IsPunct(ch):
+// return (ord(ch) >= 32 and ord(ch) < 127 and
+// not ch.isspace() and not ch.isalnum())
+// def IsBlank(ch):
+// return ch in ' \t'
+// def IsCntrl(ch):
+// return ord(ch) < 32 or ord(ch) == 127
+// def IsXDigit(ch):
+// return ch.isdigit() or ch.lower() in 'abcdef'
+// for i in range(128):
+// ch = chr(i)
+// mask = ((ch.isalpha() and 0x01 or 0) |
+// (ch.isalnum() and 0x04 or 0) |
+// (ch.isspace() and 0x08 or 0) |
+// (IsPunct(ch) and 0x10 or 0) |
+// (IsBlank(ch) and 0x20 or 0) |
+// (IsCntrl(ch) and 0x40 or 0) |
+// (IsXDigit(ch) and 0x80 or 0))
+// print Hex2(mask) + ',',
+// if i % 16 == 7:
+// print ' //', Hex2(i & 0x78)
+// elif i % 16 == 15:
+// print
+const uint8 kAsciiPropertyBits[256] = {
+ 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, // 0x00
+ 0x40, 0x68, 0x48, 0x48, 0x48, 0x48, 0x40, 0x40,
+ 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, // 0x10
+ 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40,
+ 0x28, 0x10, 0x10, 0x10, 0x10, 0x10, 0x10, 0x10, // 0x20
+ 0x10, 0x10, 0x10, 0x10, 0x10, 0x10, 0x10, 0x10,
+ 0x84, 0x84, 0x84, 0x84, 0x84, 0x84, 0x84, 0x84, // 0x30
+ 0x84, 0x84, 0x10, 0x10, 0x10, 0x10, 0x10, 0x10,
+ 0x10, 0x85, 0x85, 0x85, 0x85, 0x85, 0x85, 0x05, // 0x40
+ 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05,
+ 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, // 0x50
+ 0x05, 0x05, 0x05, 0x10, 0x10, 0x10, 0x10, 0x10,
+ 0x10, 0x85, 0x85, 0x85, 0x85, 0x85, 0x85, 0x05, // 0x60
+ 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05,
+ 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, // 0x70
+ 0x05, 0x05, 0x05, 0x10, 0x10, 0x10, 0x10, 0x40,
+};
+
+const char kAsciiToLower[256] = {
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
+ 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
+ 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
+ 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
+ 64, 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o',
+ 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', 91, 92, 93, 94, 95,
+ 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,
+ 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127,
+ 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143,
+ 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159,
+ 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175,
+ 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191,
+ 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207,
+ 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223,
+ 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239,
+ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255
+};
+
+const char kAsciiToUpper[256] = {
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
+ 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
+ 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
+ 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
+ 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79,
+ 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95,
+ 96, 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O',
+ 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', 123, 124, 125, 126, 127,
+ 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143,
+ 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159,
+ 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175,
+ 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191,
+ 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207,
+ 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223,
+ 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239,
+ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255
+};
diff --git a/trunk/src/third_party/css_parser/src/strings/ascii_ctype.h b/trunk/src/third_party/css_parser/src/strings/ascii_ctype.h
new file mode 100644
index 0000000..20ab628
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/strings/ascii_ctype.h
@@ -0,0 +1,112 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2007 Google Inc. All Rights Reserved.
+// Author: dpeng@google.com (Daniel Peng)
+
+#ifndef STRINGS_ASCII_CTYPE_H__
+#define STRINGS_ASCII_CTYPE_H__
+
+#include "base/basictypes.h"
+
+// ----------------------------------------------------------------------
+// ascii_isalpha()
+// ascii_isdigit()
+// ascii_isalnum()
+// ascii_isspace()
+// ascii_ispunct()
+// ascii_isblank()
+// ascii_iscntrl()
+// ascii_isxdigit()
+// ascii_isprint()
+// ascii_isgraph()
+// ascii_isupper()
+// ascii_islower()
+// ascii_tolower()
+// ascii_toupper()
+// The ctype.h versions of these routines are slow with some
+// compilers and/or architectures, perhaps because of locale
+// issues. These versions work for ascii only: they return
+// false for everything above \x7f (which means they return
+// false for any byte from any non-ascii UTF8 character).
+//
+// The individual bits do not have names because the array definition
+// is already tightly coupled to this, and names would make it harder
+// to read and debug.
+//
+// This is an example of the benchmark times from the unittest:
+// $ strings/ascii_ctype_test --benchmarks=all --heap_check=
+// Benchmark Time(ns) CPU(ns) Iterations
+// --------------------------------------------------
+// BM_Identity 121 120 5785985 2027.0 MB/s
+// BM_isalpha 1603 1597 511027 152.9 MB/s
+// BM_ascii_isalpha 223 224 3111595 1088.5 MB/s
+// BM_isdigit 181 183 3825722 1336.4 MB/s
+// BM_ascii_isdigit 236 239 2929312 1023.3 MB/s
+// BM_isalnum 1623 1615 460596 151.2 MB/s
+// BM_ascii_isalnum 253 255 2745518 959.1 MB/s
+// BM_isspace 1264 1258 555639 194.1 MB/s
+// BM_ascii_isspace 253 255 2745507 959.1 MB/s
+// BM_ispunct 1324 1317 555639 185.3 MB/s
+// BM_ascii_ispunct 252 255 2745507 959.1 MB/s
+// BM_isblank 1433 1426 511027 171.2 MB/s
+// BM_ascii_isblank 253 254 2745518 960.5 MB/s
+// BM_iscntrl 1643 1634 530383 149.4 MB/s
+// BM_ascii_iscntrl 252 255 2745518 959.1 MB/s
+// BM_isxdigit 1826 1817 414265 134.3 MB/s
+// BM_ascii_isxdigit 258 260 2692712 939.3 MB/s
+// BM_isprint 1677 1669 419224 146.2 MB/s
+// BM_ascii_isprint 237 239 2929312 1021.8 MB/s
+// BM_isgraph 1436 1429 507324 170.9 MB/s
+// BM_ascii_isgraph 237 239 2929312 1021.8 MB/s
+// BM_isupper 1550 1544 463647 158.1 MB/s
+// BM_ascii_isupper 237 239 2929312 1021.8 MB/s
+// BM_islower 1301 1294 538544 188.7 MB/s
+// BM_ascii_islower 237 239 2929312 1023.3 MB/s
+// BM_isascii 182 181 3846746 1345.7 MB/s
+// BM_ascii_isascii 209 211 3318039 1159.1 MB/s
+// BM_tolower 1743 1764 397786 138.4 MB/s
+// BM_ascii_tolower 210 211 3318039 1155.8 MB/s
+// BM_toupper 1742 1764 397788 138.4 MB/s
+// BM_ascii_toupper 212 211 3302401 1156.9 MB/s
+//
+// ----------------------------------------------------------------------
+
+#define kApb kAsciiPropertyBits
+extern const uint8 kAsciiPropertyBits[256];
+static inline bool ascii_isalpha(unsigned char c) { return kApb[c] & 0x01; }
+static inline bool ascii_isalnum(unsigned char c) { return kApb[c] & 0x04; }
+static inline bool ascii_isspace(unsigned char c) { return kApb[c] & 0x08; }
+static inline bool ascii_ispunct(unsigned char c) { return kApb[c] & 0x10; }
+static inline bool ascii_isblank(unsigned char c) { return kApb[c] & 0x20; }
+static inline bool ascii_iscntrl(unsigned char c) { return kApb[c] & 0x40; }
+static inline bool ascii_isxdigit(unsigned char c) { return kApb[c] & 0x80; }
+static inline bool ascii_isdigit(unsigned char c) { return c >= '0' && c <= '9'; }
+static inline bool ascii_isprint(unsigned char c) { return c >= 32 && c < 127; }
+static inline bool ascii_isgraph(unsigned char c) { return c > 32 && c < 127; }
+static inline bool ascii_isupper(unsigned char c) { return c >= 'A' && c <= 'Z'; }
+static inline bool ascii_islower(unsigned char c) { return c >= 'a' && c <= 'z'; }
+static inline bool ascii_isascii(unsigned char c) {
+ return static_cast<signed char>(c) >= 0;
+}
+#undef kApb
+
+extern const char kAsciiToLower[256];
+static inline char ascii_tolower(unsigned char c) { return kAsciiToLower[c]; }
+extern const char kAsciiToUpper[256];
+static inline char ascii_toupper(unsigned char c) { return kAsciiToUpper[c]; }
+
+#endif // STRINGS_ASCII_CTYPE_H__
diff --git a/trunk/src/third_party/css_parser/src/strings/escaping.h b/trunk/src/third_party/css_parser/src/strings/escaping.h
new file mode 100644
index 0000000..8f84ca7
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/strings/escaping.h
@@ -0,0 +1,32 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef STRINGS_ESCAPING_H_
+#define STRINGS_ESCAPING_H_
+
+#include "base/logging.h"
+
+inline int hex_digit_to_int(char c) {
+ /* Assume ASCII. */
+ DCHECK('0' == 0x30 && 'A' == 0x41 && 'a' == 0x61);
+ DCHECK(isxdigit(c));
+ int x = static_cast<unsigned char>(c);
+ if (x > '9') {
+ x += 9;
+ }
+ return x & 0xf;
+}
+
+#endif // STRINGS_ESCAPING_H_
diff --git a/trunk/src/third_party/css_parser/src/strings/join.h b/trunk/src/third_party/css_parser/src/strings/join.h
new file mode 100644
index 0000000..30f61b2
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/strings/join.h
@@ -0,0 +1,37 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef STRINGS_JOIN_H_
+#define STRINGS_JOIN_H_
+
+#include <string>
+#include <vector>
+
+// TODO(sligocki): Add this to Chromium string_util.h
+inline void JoinStrings(const std::vector<std::string>& parts,
+ const char* delim, std::string* result) {
+ if (parts.size() != 0) {
+ result->append(parts[0]);
+ std::vector<std::string>::const_iterator iter = parts.begin();
+ ++iter;
+
+ for (; iter != parts.end(); ++iter) {
+ result->append(delim);
+ result->append(*iter);
+ }
+ }
+}
+
+#endif // STRINGS_JOIN_H_
diff --git a/trunk/src/third_party/css_parser/src/strings/memutil.h b/trunk/src/third_party/css_parser/src/strings/memutil.h
new file mode 100644
index 0000000..f9a48a6
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/strings/memutil.h
@@ -0,0 +1,42 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef MEMUTIL_H_
+#define MEMUTIL_H_
+
+// The ""'s catch people who don't pass in a literal for "str"
+#define strliterallen(str) (sizeof("" str "")-1)
+
+static int memcasecmp(const char *s1, const char *s2, size_t len) {
+ const unsigned char *us1 = reinterpret_cast<const unsigned char *>(s1);
+ const unsigned char *us2 = reinterpret_cast<const unsigned char *>(s2);
+
+ for ( size_t i = 0; i < len; i++ ) {
+ const int diff = tolower(us1[i]) - tolower(us2[i]);
+ if (diff != 0) return diff;
+ }
+ return 0;
+}
+
+#define memcaseis(str, len, literal) \
+ ( (((len) == strliterallen(literal)) \
+ && memcasecmp(str, literal, strliterallen(literal)) == 0) )
+
+#define memis(str, len, literal) \
+ ( (((len) == strliterallen(literal)) \
+ && memcmp(str, literal, strliterallen(literal)) == 0) )
+
+
+#endif // MEMUTIL_H_
diff --git a/trunk/src/third_party/css_parser/src/strings/stringpiece.h b/trunk/src/third_party/css_parser/src/strings/stringpiece.h
new file mode 100644
index 0000000..7b56772
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/strings/stringpiece.h
@@ -0,0 +1,24 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef STRINGS_STRINGPIECE_H_
+#define STRINGS_STRINGPIECE_H_
+
+//#include "third_party/chromium/src/base/string_piece.h"
+#include "base/string_piece.h"
+
+using base::StringPiece;
+
+#endif // STRINGS_STRINGPIECE_H_
diff --git a/trunk/src/third_party/css_parser/src/strings/stringpiece_utils.cc b/trunk/src/third_party/css_parser/src/strings/stringpiece_utils.cc
new file mode 100644
index 0000000..4c987c9
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/strings/stringpiece_utils.cc
@@ -0,0 +1,83 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "strings/stringpiece_utils.h"
+
+#include <vector>
+#include "strings/ascii_ctype.h"
+
+namespace StringPieceUtils {
+
+int RemoveLeadingWhitespace(StringPiece* text) {
+ int count = 0;
+ const char* ptr = text->data();
+ while (count < text->size() && ascii_isspace(*ptr)) {
+ count++;
+ ptr++;
+ }
+ text->remove_prefix(count);
+ return count;
+}
+
+int RemoveTrailingWhitespace(StringPiece* text) {
+ int count = 0;
+ const char* ptr = text->data() + text->size() - 1;
+ while (count < text->size() && ascii_isspace(*ptr)) {
+ ++count;
+ --ptr;
+ }
+ text->remove_suffix(count);
+ return count;
+}
+
+int RemoveWhitespaceContext(StringPiece* text) {
+ // use RemoveLeadingWhitespace() and RemoveTrailingWhitespace() to do the job
+ return (RemoveLeadingWhitespace(text) + RemoveTrailingWhitespace(text));
+}
+
+
+template <typename DelimType>
+static void SplitInternal(const StringPiece& full, DelimType delim,
+ std::vector<StringPiece>* result) {
+ StringPiece::size_type begin_index, end_index;
+ begin_index = full.find_first_not_of(delim);
+ while (begin_index != StringPiece::npos) {
+ end_index = full.find_first_of(delim, begin_index);
+ if (end_index == StringPiece::npos) {
+ result->push_back(StringPiece(full.data() + begin_index,
+ full.size() - begin_index));
+ return;
+ }
+ result->push_back(StringPiece(full.data() + begin_index,
+ end_index - begin_index));
+ begin_index = full.find_first_not_of(delim, end_index);
+ }
+}
+
+
+// the code below is similar to SplitStringUsing in strings/strutil.cc but
+// this one returns a vector of pieces in the original string thus eliminating
+// the allocation/copy for each string in the result vector.
+/* static */
+void Split(const StringPiece& full, const char* delim,
+ std::vector<StringPiece>* result) {
+ if (delim[0] != '\0' && delim[1] == '\0') {
+ SplitInternal(full, delim[0], result);
+ } else {
+ SplitInternal(full, delim, result);
+ }
+}
+
+} // namespace StringPieceUtils
diff --git a/trunk/src/third_party/css_parser/src/strings/stringpiece_utils.h b/trunk/src/third_party/css_parser/src/strings/stringpiece_utils.h
new file mode 100644
index 0000000..a4576c0
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/strings/stringpiece_utils.h
@@ -0,0 +1,42 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2003 and onwards Google Inc.
+//
+// Utility functions for operating on StringPieces
+// Collected here for convenience
+
+#ifndef STRINGS_STRINGPIECE_UTILS_H_
+#define STRINGS_STRINGPIECE_UTILS_H_
+
+#include <vector>
+#include "strings/stringpiece.h"
+
+namespace StringPieceUtils {
+
+// removes leading and trailing ascii_isspace() chars. Returns
+// number of chars removed
+int RemoveWhitespaceContext(StringPiece* text);
+
+// similar to SplitStringUsing (see strings/split.h).
+// this one returns a vector of pieces in the original string thus eliminating
+// the allocation/copy for each string in the result vector.
+void Split(const StringPiece& full, const char* delim,
+ std::vector<StringPiece>* pieces);
+
+} // namespace StringPieceUtils
+
+#endif // STRINGS_STRINGPIECE_UTILS_H_
diff --git a/trunk/src/third_party/css_parser/src/strings/stringprintf.h b/trunk/src/third_party/css_parser/src/strings/stringprintf.h
new file mode 100644
index 0000000..208d338
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/strings/stringprintf.h
@@ -0,0 +1,22 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef STRINGS_STRINGPRINTF_H_
+#define STRINGS_STRINGPRINTF_H_
+
+//#include "third_party/chromium/src/base/string_util.h"
+#include "base/string_util.h"
+
+#endif // STRINGS_STRINGPRINTF_H_
diff --git a/trunk/src/third_party/css_parser/src/strings/strutil.h b/trunk/src/third_party/css_parser/src/strings/strutil.h
new file mode 100644
index 0000000..5598843
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/strings/strutil.h
@@ -0,0 +1,34 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef STRINGS_STRUTIL_H_
+#define STRINGS_STRUTIL_H_
+
+//#include "third_party/chromium/src/base/string_util.h"
+#include "base/string_util.h"
+
+// Like the CONSIDER macro above except that it supports enums from another
+// class -- e.g., if: enum Status { VERIFIED, NOT_VERIFIED, WHITE_LISTED }
+// is in class Foo and you are using it in another class, use:
+// switch (val) {
+// CONSIDER_IN_CLASS(Foo, VERIFIED);
+// CONSIDER_IN_CLASS(Foo, NOT_VERIFIED);
+// CONSIDER_IN_CLASS(Foo, WHITE_LISTED);
+// default: return "UNKNOWN value";
+// }
+// Only the enum string will be returned (i.e., without the "Foo::" prefix).
+#define CONSIDER_IN_CLASS(cls,val) case cls::val: return #val
+
+#endif // WEBUTIL_CSS_OPEN_SOURCE_STRINGS_STRUTIL_H_
diff --git a/trunk/src/third_party/css_parser/src/testing/base/public/googletest.h b/trunk/src/third_party/css_parser/src/testing/base/public/googletest.h
new file mode 100644
index 0000000..d09072c
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/testing/base/public/googletest.h
@@ -0,0 +1,15 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
diff --git a/trunk/src/third_party/css_parser/src/testing/base/public/gunit.h b/trunk/src/third_party/css_parser/src/testing/base/public/gunit.h
new file mode 100644
index 0000000..a9ad0b0
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/testing/base/public/gunit.h
@@ -0,0 +1,21 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef WEBUTIL_CSS_OPEN_SOURCE_TESTING_BASE_PUBLIC_GUNIT_H_
+#define WEBUTIL_CSS_OPEN_SOURCE_TESTING_BASE_PUBLIC_GUNIT_H_
+
+#include "testing/gtest/include/gtest/gtest.h"
+
+#endif // WEBUTIL_CSS_OPEN_SOURCE_TESTING_BASE_PUBLIC_GUNIT_H_
diff --git a/trunk/src/third_party/css_parser/src/testing/production_stub/public/gunit_prod.h b/trunk/src/third_party/css_parser/src/testing/production_stub/public/gunit_prod.h
new file mode 100644
index 0000000..d27ec4c
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/testing/production_stub/public/gunit_prod.h
@@ -0,0 +1,21 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef WEBUTIL_CSS_OPEN_SOURCE_TESTING_PRODUCTION_STUB_PUBLIC_GUNIT_PROD_H_
+#define WEBUTIL_CSS_OPEN_SOURCE_TESTING_PRODUCTION_STUB_PUBLIC_GUNIT_PROD_H_
+
+#include "testing/gtest/include/gtest/gtest_prod.h"
+
+#endif // WEBUTIL_CSS_OPEN_SOURCE_TESTING_PRODUCTION_STUB_PUBLIC_GUNIT_PROD_H_
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/Make.Linux-x86_64 b/trunk/src/third_party/css_parser/src/third_party/utf/Make.Linux-x86_64
new file mode 100644
index 0000000..d5ecd0f
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/Make.Linux-x86_64
@@ -0,0 +1,7 @@
+CC=gcc
+CFLAGS+=-Wall -Wno-missing-braces -Wno-parentheses -Wno-switch -fPIC -O2 -g -c -I. -I../..
+O=o
+AR=ar
+ARFLAGS=rvc
+NAN=nan64.$O # default, can be overriden by Make.$(SYSNAME)
+NAN=nan64.$O
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/Makefile b/trunk/src/third_party/css_parser/src/third_party/utf/Makefile
new file mode 100644
index 0000000..2823aaa
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/Makefile
@@ -0,0 +1,116 @@
+
+# this works in gnu make
+SYSNAME:=${shell uname}
+OBJTYPE:=${shell uname -m | sed 's;i.86;386;; s;/.*;;'}
+
+# this works in bsd make
+SYSNAME!=uname
+OBJTYPE!=uname -m | sed 's;i.86;386;; s;/.*;;'
+
+# the gnu rules will mess up bsd but not vice versa,
+# hence the gnu rules come first.
+
+include Make.$(SYSNAME)-$(OBJTYPE)
+
+PREFIX=/usr/local
+
+NUKEFILES=
+
+TGZFILES=
+
+LIB=libutf.a
+VERSION=2.0
+PORTPLACE=devel/libutf
+NAME=libutf
+
+OFILES=\
+ rune.$O\
+ runestrcat.$O\
+ runestrchr.$O\
+ runestrcmp.$O\
+ runestrcpy.$O\
+ runestrdup.$O\
+ runestrlen.$O\
+ runestrecpy.$O\
+ runestrncat.$O\
+ runestrncmp.$O\
+ runestrncpy.$O\
+ runestrrchr.$O\
+ runestrstr.$O\
+ runetype.$O\
+ utfecpy.$O\
+ utflen.$O\
+ utfnlen.$O\
+ utfrrune.$O\
+ utfrune.$O\
+ utfutf.$O\
+
+HFILES=\
+ utf.h\
+
+all: $(LIB)
+
+install: $(LIB)
+ test -d $(PREFIX)/man/man3 || mkdir $(PREFIX)/man/man3
+ install -m 0644 isalpharune.3 $(PREFIX)/man/man3/isalpharune.3
+ install -m 0644 utf.7 $(PREFIX)/man/man7/utf.7
+ install -m 0644 rune.3 $(PREFIX)/man/man3/rune.3
+ install -m 0644 runestrcat.3 $(PREFIX)/man/man3/runestrcat.3
+ install -m 0644 utf.h $(PREFIX)/include/utf.h
+ install -m 0644 $(LIB) $(PREFIX)/lib/$(LIB)
+
+# for now - we probably should connect this with the Google makefile so
+# we use the same compiler at all times...
+CC = /home/build/buildtools/production/gcc-2.95.3/bin/i686-google-linux-gcc
+
+$(LIB): $(OFILES)
+ $(AR) $(ARFLAGS) $(LIB) $(OFILES)
+
+NUKEFILES+=$(LIB)
+.c.$O:
+ $(CC) $(CFLAGS) -I$(PREFIX)/include $*.c
+
+%.$O: %.c
+ $(CC) $(CFLAGS) -I$(PREFIX)/include $*.c
+
+
+$(OFILES): $(HFILES)
+
+tgz:
+ rm -rf $(NAME)-$(VERSION)
+ mkdir $(NAME)-$(VERSION)
+ cp Makefile Make.* README LICENSE NOTICE *.[ch137] rpm.spec bundle.ports $(TGZFILES) $(NAME)-$(VERSION)
+ tar cf - $(NAME)-$(VERSION) | gzip >$(NAME)-$(VERSION).tgz
+ rm -rf $(NAME)-$(VERSION)
+
+clean:
+ rm -f $(OFILES)
+
+nuke:
+ rm -f $(OFILES) *.tgz *.rpm $(NUKEFILES)
+
+rpm:
+ make tgz
+ cp $(NAME)-$(VERSION).tgz /usr/src/RPM/SOURCES
+ rpm -ba rpm.spec
+ cp /usr/src/RPM/SRPMS/$(NAME)-$(VERSION)-1.src.rpm .
+ cp /usr/src/RPM/RPMS/i586/$(NAME)-$(VERSION)-1.i586.rpm .
+ scp *.rpm rsc@amsterdam.lcs.mit.edu:public_html/software
+
+PORTDIR=/usr/ports/$(PORTPLACE)
+
+ports:
+ make tgz
+ rm -rf $(PORTDIR)
+ mkdir $(PORTDIR)
+ cp $(NAME)-$(VERSION).tgz /usr/ports/distfiles
+ cat bundle.ports | (cd $(PORTDIR) && awk '$$1=="---" && $$3=="---" { ofile=$$2; next} {if(ofile) print >ofile}')
+ (cd $(PORTDIR); make makesum)
+ (cd $(PORTDIR); make)
+ (cd $(PORTDIR); /usr/local/bin/portlint)
+ rm -rf $(PORTDIR)/work
+ shar `find $(PORTDIR)` > ports.shar
+ (cd $(PORTDIR); tar cf - *) | gzip >$(NAME)-$(VERSION)-ports.tgz
+ scp *.tgz rsc@amsterdam.lcs.mit.edu:public_html/software
+
+.phony: all clean nuke install tgz rpm ports
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/NOTICE b/trunk/src/third_party/css_parser/src/third_party/utf/NOTICE
new file mode 100644
index 0000000..ad76cd5
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/NOTICE
@@ -0,0 +1,13 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 1998-2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/README b/trunk/src/third_party/css_parser/src/third_party/utf/README
new file mode 100644
index 0000000..ad76cd5
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/README
@@ -0,0 +1,13 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 1998-2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/rune.c b/trunk/src/third_party/css_parser/src/third_party/utf/rune.c
new file mode 100644
index 0000000..5fb31cd
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/rune.c
@@ -0,0 +1,350 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+enum
+{
+ Bit1 = 7,
+ Bitx = 6,
+ Bit2 = 5,
+ Bit3 = 4,
+ Bit4 = 3,
+ Bit5 = 2,
+
+ T1 = ((1<<(Bit1+1))-1) ^ 0xFF, /* 0000 0000 */
+ Tx = ((1<<(Bitx+1))-1) ^ 0xFF, /* 1000 0000 */
+ T2 = ((1<<(Bit2+1))-1) ^ 0xFF, /* 1100 0000 */
+ T3 = ((1<<(Bit3+1))-1) ^ 0xFF, /* 1110 0000 */
+ T4 = ((1<<(Bit4+1))-1) ^ 0xFF, /* 1111 0000 */
+ T5 = ((1<<(Bit5+1))-1) ^ 0xFF, /* 1111 1000 */
+
+ Rune1 = (1<<(Bit1+0*Bitx))-1, /* 0000 0000 0111 1111 */
+ Rune2 = (1<<(Bit2+1*Bitx))-1, /* 0000 0111 1111 1111 */
+ Rune3 = (1<<(Bit3+2*Bitx))-1, /* 1111 1111 1111 1111 */
+ Rune4 = (1<<(Bit4+3*Bitx))-1,
+ /* 0001 1111 1111 1111 1111 1111 */
+
+ Maskx = (1<<Bitx)-1, /* 0011 1111 */
+ Testx = Maskx ^ 0xFF, /* 1100 0000 */
+
+ Bad = Runeerror,
+};
+
+/*
+ * Modified by Wei-Hwa Huang, Google Inc., on 2004-09-24
+ * This is a slower but "safe" version of the old chartorune
+ * that works on strings that are not necessarily null-terminated.
+ *
+ * If you know for sure that your string is null-terminated,
+ * chartorune will be a bit faster.
+ *
+ * It is guaranteed not to attempt to access "length"
+ * past the incoming pointer. This is to avoid
+ * possible access violations. If the string appears to be
+ * well-formed but incomplete (i.e., to get the whole Rune
+ * we'd need to read past str+length) then we'll set the Rune
+ * to Bad and return 0.
+ *
+ * Note that if we have decoding problems for other
+ * reasons, we return 1 instead of 0.
+ */
+int
+charntorune(Rune *rune, const char *str, int length)
+{
+ int c, c1, c2, c3;
+ long l;
+
+ /* When we're not allowed to read anything */
+ if(length <= 0) {
+ goto badlen;
+ }
+
+ /*
+ * one character sequence (7-bit value)
+ * 00000-0007F => T1
+ */
+ c = *(uchar*)str;
+ if(c < Tx) {
+ *rune = c;
+ return 1;
+ }
+
+ // If we can't read more than one character we must stop
+ if(length <= 1) {
+ goto badlen;
+ }
+
+ /*
+ * two character sequence (11-bit value)
+ * 0080-07FF => T2 Tx
+ */
+ c1 = *(uchar*)(str+1) ^ Tx;
+ if(c1 & Testx)
+ goto bad;
+ if(c < T3) {
+ if(c < T2)
+ goto bad;
+ l = ((c << Bitx) | c1) & Rune2;
+ if(l <= Rune1)
+ goto bad;
+ *rune = l;
+ return 2;
+ }
+
+ // If we can't read more than two characters we must stop
+ if(length <= 2) {
+ goto badlen;
+ }
+
+ /*
+ * three character sequence (16-bit value)
+ * 0800-FFFF => T3 Tx Tx
+ */
+ c2 = *(uchar*)(str+2) ^ Tx;
+ if(c2 & Testx)
+ goto bad;
+ if(c < T4) {
+ l = ((((c << Bitx) | c1) << Bitx) | c2) & Rune3;
+ if(l <= Rune2)
+ goto bad;
+ *rune = l;
+ return 3;
+ }
+
+ if (length <= 3)
+ goto badlen;
+
+ /*
+ * four character sequence (21-bit value)
+ * 10000-1FFFFF => T4 Tx Tx Tx
+ */
+ c3 = *(uchar*)(str+3) ^ Tx;
+ if (c3 & Testx)
+ goto bad;
+ if (c < T5) {
+ l = ((((((c << Bitx) | c1) << Bitx) | c2) << Bitx) | c3) & Rune4;
+ if (l <= Rune3)
+ goto bad;
+ *rune = l;
+ return 4;
+ }
+
+ // Support for 5-byte or longer UTF-8 would go here, but
+ // since we don't have that, we'll just fall through to bad.
+
+ /*
+ * bad decoding
+ */
+bad:
+ *rune = Bad;
+ return 1;
+badlen:
+ *rune = Bad;
+ return 0;
+
+}
+
+
+/*
+ * This is the older "unsafe" version, which works fine on
+ * null-terminated strings.
+ */
+int
+chartorune(Rune *rune, const char *str)
+{
+ int c, c1, c2, c3;
+ long l;
+
+ /*
+ * one character sequence
+ * 00000-0007F => T1
+ */
+ c = *(uchar*)str;
+ if(c < Tx) {
+ *rune = c;
+ return 1;
+ }
+
+ /*
+ * two character sequence
+ * 0080-07FF => T2 Tx
+ */
+ c1 = *(uchar*)(str+1) ^ Tx;
+ if(c1 & Testx)
+ goto bad;
+ if(c < T3) {
+ if(c < T2)
+ goto bad;
+ l = ((c << Bitx) | c1) & Rune2;
+ if(l <= Rune1)
+ goto bad;
+ *rune = l;
+ return 2;
+ }
+
+ /*
+ * three character sequence
+ * 0800-FFFF => T3 Tx Tx
+ */
+ c2 = *(uchar*)(str+2) ^ Tx;
+ if(c2 & Testx)
+ goto bad;
+ if(c < T4) {
+ l = ((((c << Bitx) | c1) << Bitx) | c2) & Rune3;
+ if(l <= Rune2)
+ goto bad;
+ *rune = l;
+ return 3;
+ }
+
+ /*
+ * four character sequence (21-bit value)
+ * 10000-1FFFFF => T4 Tx Tx Tx
+ */
+ c3 = *(uchar*)(str+3) ^ Tx;
+ if (c3 & Testx)
+ goto bad;
+ if (c < T5) {
+ l = ((((((c << Bitx) | c1) << Bitx) | c2) << Bitx) | c3) & Rune4;
+ if (l <= Rune3)
+ goto bad;
+ *rune = l;
+ return 4;
+ }
+
+ /*
+ * Support for 5-byte or longer UTF-8 would go here, but
+ * since we don't have that, we'll just fall through to bad.
+ */
+
+ /*
+ * bad decoding
+ */
+bad:
+ *rune = Bad;
+ return 1;
+}
+
+int
+isvalidcharntorune(const char* str, int length, Rune* rune, int* consumed) {
+ *consumed = charntorune(rune, str, length);
+ return *rune != Runeerror || *consumed == 3;
+}
+
+int
+runetochar(char *str, const Rune *rune)
+{
+ /* Runes are signed, so convert to unsigned for range check. */
+ unsigned long c;
+
+ /*
+ * one character sequence
+ * 00000-0007F => 00-7F
+ */
+ c = *rune;
+ if(c <= Rune1) {
+ str[0] = c;
+ return 1;
+ }
+
+ /*
+ * two character sequence
+ * 0080-07FF => T2 Tx
+ */
+ if(c <= Rune2) {
+ str[0] = T2 | (c >> 1*Bitx);
+ str[1] = Tx | (c & Maskx);
+ return 2;
+ }
+
+ /*
+ * If the Rune is out of range, convert it to the error rune.
+ * Do this test here because the error rune encodes to three bytes.
+ * Doing it earlier would duplicate work, since an out of range
+ * Rune wouldn't have fit in one or two bytes.
+ */
+ if (c > Runemax)
+ c = Runeerror;
+
+ /*
+ * three character sequence
+ * 0800-FFFF => T3 Tx Tx
+ */
+ if (c <= Rune3) {
+ str[0] = T3 | (c >> 2*Bitx);
+ str[1] = Tx | ((c >> 1*Bitx) & Maskx);
+ str[2] = Tx | (c & Maskx);
+ return 3;
+ }
+
+ /*
+ * four character sequence (21-bit value)
+ * 10000-1FFFFF => T4 Tx Tx Tx
+ */
+ str[0] = T4 | (c >> 3*Bitx);
+ str[1] = Tx | ((c >> 2*Bitx) & Maskx);
+ str[2] = Tx | ((c >> 1*Bitx) & Maskx);
+ str[3] = Tx | (c & Maskx);
+ return 4;
+}
+
+int
+runelen(Rune rune)
+{
+ char str[10];
+
+ return runetochar(str, &rune);
+}
+
+int
+runenlen(const Rune *r, int nrune)
+{
+ int nb, c;
+
+ nb = 0;
+ while(nrune--) {
+ c = *r++;
+ if (c <= Rune1)
+ nb++;
+ else if (c <= Rune2)
+ nb += 2;
+ else if (c <= Rune3)
+ nb += 3;
+ else /* assert(c <= Rune4) */
+ nb += 4;
+ }
+ return nb;
+}
+
+int
+fullrune(const char *str, int n)
+{
+ if (n > 0) {
+ int c = *(uchar*)str;
+ if (c < Tx)
+ return 1;
+ if (n > 1) {
+ if (c < T3)
+ return 1;
+ if (n > 2) {
+ if (c < T4 || n > 3)
+ return 1;
+ }
+ }
+ }
+ return 0;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/runestrcat.c b/trunk/src/third_party/css_parser/src/third_party/utf/runestrcat.c
new file mode 100644
index 0000000..d68f595
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/runestrcat.c
@@ -0,0 +1,25 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+Rune*
+runestrcat(Rune *s1, const Rune *s2)
+{
+
+ runestrcpy((Rune*)runestrchr(s1, 0), s2);
+ return s1;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/runestrchr.c b/trunk/src/third_party/css_parser/src/third_party/utf/runestrchr.c
new file mode 100644
index 0000000..d5bd97d
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/runestrchr.c
@@ -0,0 +1,36 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+const
+Rune*
+runestrchr(const Rune *s, Rune c)
+{
+ Rune c0 = c;
+ Rune c1;
+
+ if(c == 0) {
+ while(*s++)
+ ;
+ return s-1;
+ }
+
+ while((c1 = *s++) != 0)
+ if(c1 == c0)
+ return s-1;
+ return 0;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/runestrcmp.c b/trunk/src/third_party/css_parser/src/third_party/utf/runestrcmp.c
new file mode 100644
index 0000000..ac9d0e5
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/runestrcmp.c
@@ -0,0 +1,35 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+int
+runestrcmp(const Rune *s1, const Rune *s2)
+{
+ Rune c1, c2;
+
+ for(;;) {
+ c1 = *s1++;
+ c2 = *s2++;
+ if(c1 != c2) {
+ if(c1 > c2)
+ return 1;
+ return -1;
+ }
+ if(c1 == 0)
+ return 0;
+ }
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/runestrcpy.c b/trunk/src/third_party/css_parser/src/third_party/utf/runestrcpy.c
new file mode 100644
index 0000000..eb41413
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/runestrcpy.c
@@ -0,0 +1,28 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+Rune*
+runestrcpy(Rune *s1, const Rune *s2)
+{
+ Rune *os1;
+
+ os1 = s1;
+ while((*s1++ = *s2++) != 0)
+ ;
+ return os1;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/runestrdup.c b/trunk/src/third_party/css_parser/src/third_party/utf/runestrdup.c
new file mode 100644
index 0000000..2b15035
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/runestrdup.c
@@ -0,0 +1,30 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include <stdlib.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+Rune*
+runestrdup(const Rune *s)
+{
+ Rune *ns;
+
+ ns = (Rune*)malloc(sizeof(Rune)*(runestrlen(s) + 1));
+ if(ns == 0)
+ return 0;
+
+ return runestrcpy(ns, s);
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/runestrecpy.c b/trunk/src/third_party/css_parser/src/third_party/utf/runestrecpy.c
new file mode 100644
index 0000000..b56abe7
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/runestrecpy.c
@@ -0,0 +1,32 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+Rune*
+runestrecpy(Rune *s1, Rune *es1, const Rune *s2)
+{
+ if(s1 >= es1)
+ return s1;
+
+ while((*s1++ = *s2++) != 0){
+ if(s1 == es1){
+ *--s1 = '\0';
+ break;
+ }
+ }
+ return s1;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/runestrlen.c b/trunk/src/third_party/css_parser/src/third_party/utf/runestrlen.c
new file mode 100644
index 0000000..e019881
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/runestrlen.c
@@ -0,0 +1,24 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+long
+runestrlen(const Rune *s)
+{
+
+ return runestrchr(s, 0) - s;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/runestrncat.c b/trunk/src/third_party/css_parser/src/third_party/utf/runestrncat.c
new file mode 100644
index 0000000..bca42f4
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/runestrncat.c
@@ -0,0 +1,32 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+Rune*
+runestrncat(Rune *s1, const Rune *s2, long n)
+{
+ Rune *os1;
+
+ os1 = s1;
+ s1 = (Rune*)runestrchr(s1, 0);
+ while((*s1++ = *s2++) != 0)
+ if(--n < 0) {
+ s1[-1] = 0;
+ break;
+ }
+ return os1;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/runestrncmp.c b/trunk/src/third_party/css_parser/src/third_party/utf/runestrncmp.c
new file mode 100644
index 0000000..02925b1
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/runestrncmp.c
@@ -0,0 +1,37 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+int
+runestrncmp(const Rune *s1, const Rune *s2, long n)
+{
+ Rune c1, c2;
+
+ while(n > 0) {
+ c1 = *s1++;
+ c2 = *s2++;
+ n--;
+ if(c1 != c2) {
+ if(c1 > c2)
+ return 1;
+ return -1;
+ }
+ if(c1 == 0)
+ break;
+ }
+ return 0;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/runestrncpy.c b/trunk/src/third_party/css_parser/src/third_party/utf/runestrncpy.c
new file mode 100644
index 0000000..95ea9dd
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/runestrncpy.c
@@ -0,0 +1,33 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+Rune*
+runestrncpy(Rune *s1, const Rune *s2, long n)
+{
+ int i;
+ Rune *os1;
+
+ os1 = s1;
+ for(i = 0; i < n; i++)
+ if((*s1++ = *s2++) == 0) {
+ while(++i < n)
+ *s1++ = 0;
+ return os1;
+ }
+ return os1;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/runestrrchr.c b/trunk/src/third_party/css_parser/src/third_party/utf/runestrrchr.c
new file mode 100644
index 0000000..7349f23
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/runestrrchr.c
@@ -0,0 +1,31 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+const
+Rune*
+runestrrchr(const Rune *s, Rune c)
+{
+ const Rune *r;
+
+ if(c == 0)
+ return runestrchr(s, 0);
+ r = 0;
+ while((s = runestrchr(s, c)) != 0)
+ r = s++;
+ return r;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/runestrstr.c b/trunk/src/third_party/css_parser/src/third_party/utf/runestrstr.c
new file mode 100644
index 0000000..1d53f9c
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/runestrstr.c
@@ -0,0 +1,45 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+/*
+ * Return pointer to first occurrence of s2 in s1,
+ * 0 if none
+ */
+const
+Rune*
+runestrstr(const Rune *s1, const Rune *s2)
+{
+ const Rune *p, *pa, *pb;
+ int c0, c;
+
+ c0 = *s2;
+ if(c0 == 0)
+ return s1;
+ s2++;
+ for(p=runestrchr(s1, c0); p; p=runestrchr(p+1, c0)) {
+ pa = p;
+ for(pb=s2;; pb++) {
+ c = *pb;
+ if(c == 0)
+ return p;
+ if(c != *++pa)
+ break;
+ }
+ }
+ return 0;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/runetype.c b/trunk/src/third_party/css_parser/src/third_party/utf/runetype.c
new file mode 100644
index 0000000..da49d39
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/runetype.c
@@ -0,0 +1,70 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+static
+Rune*
+rbsearch(Rune c, Rune *t, int n, int ne)
+{
+ Rune *p;
+ int m;
+
+ while(n > 1) {
+ m = n >> 1;
+ p = t + m*ne;
+ if(c >= p[0]) {
+ t = p;
+ n = n-m;
+ } else
+ n = m;
+ }
+ if(n && c >= t[0])
+ return t;
+ return 0;
+}
+
+/*
+ * The "ideographic" property is hard to extract from UnicodeData.txt,
+ * so it is hard coded here.
+ *
+ * It is defined in the Unicode PropList.txt file, for example
+ * PropList-3.0.0.txt. Unlike the UnicodeData.txt file, the format of
+ * PropList changes between versions. This property appears relatively static;
+ * it is the same in version 4.0.1, except that version defines some >16 bit
+ * chars as ideographic as well: 20000..2a6d6, and 2f800..2Fa1d.
+ */
+static Rune __isideographicr[] = {
+ 0x3006, 0x3007, /* 3006 not in Unicode 2, in 2.1 */
+ 0x3021, 0x3029,
+ 0x3038, 0x303a, /* not in Unicode 2 or 2.1 */
+ 0x3400, 0x4db5, /* not in Unicode 2 or 2.1 */
+ 0x4e00, 0x9fbb, /* 0x9FA6..0x9FBB added for 4.1.0? */
+ 0xf900, 0xfa2d,
+ 0x20000, 0x2A6D6,
+ 0x2F800, 0x2FA1D,
+};
+
+int
+isideographicrune(Rune c)
+{
+ Rune *p;
+
+ p = rbsearch(c, __isideographicr, nelem(__isideographicr)/2, 2);
+ if(p && c >= p[0] && c <= p[1])
+ return 1;
+ return 0;
+}
+
+#include "third_party/utf/runetypebody-5.0.0.c"
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/runetypebody-5.0.0.c b/trunk/src/third_party/css_parser/src/third_party/utf/runetypebody-5.0.0.c
new file mode 100644
index 0000000..67a645d
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/runetypebody-5.0.0.c
@@ -0,0 +1,1361 @@
+/* generated automatically by mkrunetype.c from UnicodeData-5.0.0.txt */
+
+static Rune __isspacer[] = {
+ 0x0009, 0x000d,
+ 0x0020, 0x0020,
+ 0x0085, 0x0085,
+ 0x00a0, 0x00a0,
+ 0x1680, 0x1680,
+ 0x180e, 0x180e,
+ 0x2000, 0x200a,
+ 0x2028, 0x2029,
+ 0x202f, 0x202f,
+ 0x205f, 0x205f,
+ 0x3000, 0x3000,
+ 0xfeff, 0xfeff,
+};
+
+int
+isspacerune(Rune c)
+{
+ Rune *p;
+
+ p = rbsearch(c, __isspacer, nelem(__isspacer)/2, 2);
+ if(p && c >= p[0] && c <= p[1])
+ return 1;
+ return 0;
+}
+
+static Rune __isdigitr[] = {
+ 0x0030, 0x0039,
+ 0x0660, 0x0669,
+ 0x06f0, 0x06f9,
+ 0x07c0, 0x07c9,
+ 0x0966, 0x096f,
+ 0x09e6, 0x09ef,
+ 0x0a66, 0x0a6f,
+ 0x0ae6, 0x0aef,
+ 0x0b66, 0x0b6f,
+ 0x0be6, 0x0bef,
+ 0x0c66, 0x0c6f,
+ 0x0ce6, 0x0cef,
+ 0x0d66, 0x0d6f,
+ 0x0e50, 0x0e59,
+ 0x0ed0, 0x0ed9,
+ 0x0f20, 0x0f29,
+ 0x1040, 0x1049,
+ 0x17e0, 0x17e9,
+ 0x1810, 0x1819,
+ 0x1946, 0x194f,
+ 0x19d0, 0x19d9,
+ 0x1b50, 0x1b59,
+ 0xff10, 0xff19,
+ 0x104a0, 0x104a9,
+ 0x1d7ce, 0x1d7ff,
+};
+
+int
+isdigitrune(Rune c)
+{
+ Rune *p;
+
+ p = rbsearch(c, __isdigitr, nelem(__isdigitr)/2, 2);
+ if(p && c >= p[0] && c <= p[1])
+ return 1;
+ return 0;
+}
+
+static Rune __isalphar[] = {
+ 0x0041, 0x005a,
+ 0x0061, 0x007a,
+ 0x00c0, 0x00d6,
+ 0x00d8, 0x00f6,
+ 0x00f8, 0x02c1,
+ 0x02c6, 0x02d1,
+ 0x02e0, 0x02e4,
+ 0x037a, 0x037d,
+ 0x0388, 0x038a,
+ 0x038e, 0x03a1,
+ 0x03a3, 0x03ce,
+ 0x03d0, 0x03f5,
+ 0x03f7, 0x0481,
+ 0x048a, 0x0513,
+ 0x0531, 0x0556,
+ 0x0561, 0x0587,
+ 0x05d0, 0x05ea,
+ 0x05f0, 0x05f2,
+ 0x0621, 0x063a,
+ 0x0640, 0x064a,
+ 0x066e, 0x066f,
+ 0x0671, 0x06d3,
+ 0x06e5, 0x06e6,
+ 0x06ee, 0x06ef,
+ 0x06fa, 0x06fc,
+ 0x0712, 0x072f,
+ 0x074d, 0x076d,
+ 0x0780, 0x07a5,
+ 0x07ca, 0x07ea,
+ 0x07f4, 0x07f5,
+ 0x0904, 0x0939,
+ 0x0958, 0x0961,
+ 0x097b, 0x097f,
+ 0x0985, 0x098c,
+ 0x098f, 0x0990,
+ 0x0993, 0x09a8,
+ 0x09aa, 0x09b0,
+ 0x09b6, 0x09b9,
+ 0x09dc, 0x09dd,
+ 0x09df, 0x09e1,
+ 0x09f0, 0x09f1,
+ 0x0a05, 0x0a0a,
+ 0x0a0f, 0x0a10,
+ 0x0a13, 0x0a28,
+ 0x0a2a, 0x0a30,
+ 0x0a32, 0x0a33,
+ 0x0a35, 0x0a36,
+ 0x0a38, 0x0a39,
+ 0x0a59, 0x0a5c,
+ 0x0a72, 0x0a74,
+ 0x0a85, 0x0a8d,
+ 0x0a8f, 0x0a91,
+ 0x0a93, 0x0aa8,
+ 0x0aaa, 0x0ab0,
+ 0x0ab2, 0x0ab3,
+ 0x0ab5, 0x0ab9,
+ 0x0ae0, 0x0ae1,
+ 0x0b05, 0x0b0c,
+ 0x0b0f, 0x0b10,
+ 0x0b13, 0x0b28,
+ 0x0b2a, 0x0b30,
+ 0x0b32, 0x0b33,
+ 0x0b35, 0x0b39,
+ 0x0b5c, 0x0b5d,
+ 0x0b5f, 0x0b61,
+ 0x0b85, 0x0b8a,
+ 0x0b8e, 0x0b90,
+ 0x0b92, 0x0b95,
+ 0x0b99, 0x0b9a,
+ 0x0b9e, 0x0b9f,
+ 0x0ba3, 0x0ba4,
+ 0x0ba8, 0x0baa,
+ 0x0bae, 0x0bb9,
+ 0x0c05, 0x0c0c,
+ 0x0c0e, 0x0c10,
+ 0x0c12, 0x0c28,
+ 0x0c2a, 0x0c33,
+ 0x0c35, 0x0c39,
+ 0x0c60, 0x0c61,
+ 0x0c85, 0x0c8c,
+ 0x0c8e, 0x0c90,
+ 0x0c92, 0x0ca8,
+ 0x0caa, 0x0cb3,
+ 0x0cb5, 0x0cb9,
+ 0x0ce0, 0x0ce1,
+ 0x0d05, 0x0d0c,
+ 0x0d0e, 0x0d10,
+ 0x0d12, 0x0d28,
+ 0x0d2a, 0x0d39,
+ 0x0d60, 0x0d61,
+ 0x0d85, 0x0d96,
+ 0x0d9a, 0x0db1,
+ 0x0db3, 0x0dbb,
+ 0x0dc0, 0x0dc6,
+ 0x0e01, 0x0e30,
+ 0x0e32, 0x0e33,
+ 0x0e40, 0x0e46,
+ 0x0e81, 0x0e82,
+ 0x0e87, 0x0e88,
+ 0x0e94, 0x0e97,
+ 0x0e99, 0x0e9f,
+ 0x0ea1, 0x0ea3,
+ 0x0eaa, 0x0eab,
+ 0x0ead, 0x0eb0,
+ 0x0eb2, 0x0eb3,
+ 0x0ec0, 0x0ec4,
+ 0x0edc, 0x0edd,
+ 0x0f40, 0x0f47,
+ 0x0f49, 0x0f6a,
+ 0x0f88, 0x0f8b,
+ 0x1000, 0x1021,
+ 0x1023, 0x1027,
+ 0x1029, 0x102a,
+ 0x1050, 0x1055,
+ 0x10a0, 0x10c5,
+ 0x10d0, 0x10fa,
+ 0x1100, 0x1159,
+ 0x115f, 0x11a2,
+ 0x11a8, 0x11f9,
+ 0x1200, 0x1248,
+ 0x124a, 0x124d,
+ 0x1250, 0x1256,
+ 0x125a, 0x125d,
+ 0x1260, 0x1288,
+ 0x128a, 0x128d,
+ 0x1290, 0x12b0,
+ 0x12b2, 0x12b5,
+ 0x12b8, 0x12be,
+ 0x12c2, 0x12c5,
+ 0x12c8, 0x12d6,
+ 0x12d8, 0x1310,
+ 0x1312, 0x1315,
+ 0x1318, 0x135a,
+ 0x1380, 0x138f,
+ 0x13a0, 0x13f4,
+ 0x1401, 0x166c,
+ 0x166f, 0x1676,
+ 0x1681, 0x169a,
+ 0x16a0, 0x16ea,
+ 0x1700, 0x170c,
+ 0x170e, 0x1711,
+ 0x1720, 0x1731,
+ 0x1740, 0x1751,
+ 0x1760, 0x176c,
+ 0x176e, 0x1770,
+ 0x1780, 0x17b3,
+ 0x1820, 0x1877,
+ 0x1880, 0x18a8,
+ 0x1900, 0x191c,
+ 0x1950, 0x196d,
+ 0x1970, 0x1974,
+ 0x1980, 0x19a9,
+ 0x19c1, 0x19c7,
+ 0x1a00, 0x1a16,
+ 0x1b05, 0x1b33,
+ 0x1b45, 0x1b4b,
+ 0x1d00, 0x1dbf,
+ 0x1e00, 0x1e9b,
+ 0x1ea0, 0x1ef9,
+ 0x1f00, 0x1f15,
+ 0x1f18, 0x1f1d,
+ 0x1f20, 0x1f45,
+ 0x1f48, 0x1f4d,
+ 0x1f50, 0x1f57,
+ 0x1f5f, 0x1f7d,
+ 0x1f80, 0x1fb4,
+ 0x1fb6, 0x1fbc,
+ 0x1fc2, 0x1fc4,
+ 0x1fc6, 0x1fcc,
+ 0x1fd0, 0x1fd3,
+ 0x1fd6, 0x1fdb,
+ 0x1fe0, 0x1fec,
+ 0x1ff2, 0x1ff4,
+ 0x1ff6, 0x1ffc,
+ 0x2090, 0x2094,
+ 0x210a, 0x2113,
+ 0x2119, 0x211d,
+ 0x212a, 0x212d,
+ 0x212f, 0x2139,
+ 0x213c, 0x213f,
+ 0x2145, 0x2149,
+ 0x2183, 0x2184,
+ 0x2c00, 0x2c2e,
+ 0x2c30, 0x2c5e,
+ 0x2c60, 0x2c6c,
+ 0x2c74, 0x2c77,
+ 0x2c80, 0x2ce4,
+ 0x2d00, 0x2d25,
+ 0x2d30, 0x2d65,
+ 0x2d80, 0x2d96,
+ 0x2da0, 0x2da6,
+ 0x2da8, 0x2dae,
+ 0x2db0, 0x2db6,
+ 0x2db8, 0x2dbe,
+ 0x2dc0, 0x2dc6,
+ 0x2dc8, 0x2dce,
+ 0x2dd0, 0x2dd6,
+ 0x2dd8, 0x2dde,
+ 0x3005, 0x3006,
+ 0x3031, 0x3035,
+ 0x303b, 0x303c,
+ 0x3041, 0x3096,
+ 0x309d, 0x309f,
+ 0x30a1, 0x30fa,
+ 0x30fc, 0x30ff,
+ 0x3105, 0x312c,
+ 0x3131, 0x318e,
+ 0x31a0, 0x31b7,
+ 0x31f0, 0x31ff,
+ 0x3400, 0x4db5,
+ 0x4e00, 0x9fbb,
+ 0xa000, 0xa48c,
+ 0xa717, 0xa71a,
+ 0xa800, 0xa801,
+ 0xa803, 0xa805,
+ 0xa807, 0xa80a,
+ 0xa80c, 0xa822,
+ 0xa840, 0xa873,
+ 0xac00, 0xd7a3,
+ 0xf900, 0xfa2d,
+ 0xfa30, 0xfa6a,
+ 0xfa70, 0xfad9,
+ 0xfb00, 0xfb06,
+ 0xfb13, 0xfb17,
+ 0xfb1f, 0xfb28,
+ 0xfb2a, 0xfb36,
+ 0xfb38, 0xfb3c,
+ 0xfb40, 0xfb41,
+ 0xfb43, 0xfb44,
+ 0xfb46, 0xfbb1,
+ 0xfbd3, 0xfd3d,
+ 0xfd50, 0xfd8f,
+ 0xfd92, 0xfdc7,
+ 0xfdf0, 0xfdfb,
+ 0xfe70, 0xfe74,
+ 0xfe76, 0xfefc,
+ 0xff21, 0xff3a,
+ 0xff41, 0xff5a,
+ 0xff66, 0xffbe,
+ 0xffc2, 0xffc7,
+ 0xffca, 0xffcf,
+ 0xffd2, 0xffd7,
+ 0xffda, 0xffdc,
+ 0x10000, 0x1000b,
+ 0x1000d, 0x10026,
+ 0x10028, 0x1003a,
+ 0x1003c, 0x1003d,
+ 0x1003f, 0x1004d,
+ 0x10050, 0x1005d,
+ 0x10080, 0x100fa,
+ 0x10300, 0x1031e,
+ 0x10330, 0x10340,
+ 0x10342, 0x10349,
+ 0x10380, 0x1039d,
+ 0x103a0, 0x103c3,
+ 0x103c8, 0x103cf,
+ 0x10400, 0x1049d,
+ 0x10800, 0x10805,
+ 0x1080a, 0x10835,
+ 0x10837, 0x10838,
+ 0x10900, 0x10915,
+ 0x10a10, 0x10a13,
+ 0x10a15, 0x10a17,
+ 0x10a19, 0x10a33,
+ 0x12000, 0x1236e,
+ 0x1d400, 0x1d454,
+ 0x1d456, 0x1d49c,
+ 0x1d49e, 0x1d49f,
+ 0x1d4a5, 0x1d4a6,
+ 0x1d4a9, 0x1d4ac,
+ 0x1d4ae, 0x1d4b9,
+ 0x1d4bd, 0x1d4c3,
+ 0x1d4c5, 0x1d505,
+ 0x1d507, 0x1d50a,
+ 0x1d50d, 0x1d514,
+ 0x1d516, 0x1d51c,
+ 0x1d51e, 0x1d539,
+ 0x1d53b, 0x1d53e,
+ 0x1d540, 0x1d544,
+ 0x1d54a, 0x1d550,
+ 0x1d552, 0x1d6a5,
+ 0x1d6a8, 0x1d6c0,
+ 0x1d6c2, 0x1d6da,
+ 0x1d6dc, 0x1d6fa,
+ 0x1d6fc, 0x1d714,
+ 0x1d716, 0x1d734,
+ 0x1d736, 0x1d74e,
+ 0x1d750, 0x1d76e,
+ 0x1d770, 0x1d788,
+ 0x1d78a, 0x1d7a8,
+ 0x1d7aa, 0x1d7c2,
+ 0x1d7c4, 0x1d7cb,
+ 0x20000, 0x2a6d6,
+ 0x2f800, 0x2fa1d,
+};
+
+static Rune __isalphas[] = {
+ 0x00aa,
+ 0x00b5,
+ 0x00ba,
+ 0x02ee,
+ 0x0386,
+ 0x038c,
+ 0x0559,
+ 0x06d5,
+ 0x06ff,
+ 0x0710,
+ 0x07b1,
+ 0x07fa,
+ 0x093d,
+ 0x0950,
+ 0x09b2,
+ 0x09bd,
+ 0x09ce,
+ 0x0a5e,
+ 0x0abd,
+ 0x0ad0,
+ 0x0b3d,
+ 0x0b71,
+ 0x0b83,
+ 0x0b9c,
+ 0x0cbd,
+ 0x0cde,
+ 0x0dbd,
+ 0x0e84,
+ 0x0e8a,
+ 0x0e8d,
+ 0x0ea5,
+ 0x0ea7,
+ 0x0ebd,
+ 0x0ec6,
+ 0x0f00,
+ 0x10fc,
+ 0x1258,
+ 0x12c0,
+ 0x17d7,
+ 0x17dc,
+ 0x1f59,
+ 0x1f5b,
+ 0x1f5d,
+ 0x1fbe,
+ 0x2071,
+ 0x207f,
+ 0x2102,
+ 0x2107,
+ 0x2115,
+ 0x2124,
+ 0x2126,
+ 0x2128,
+ 0x214e,
+ 0x2d6f,
+ 0xfb1d,
+ 0xfb3e,
+ 0x10808,
+ 0x1083c,
+ 0x1083f,
+ 0x10a00,
+ 0x1d4a2,
+ 0x1d4bb,
+ 0x1d546,
+};
+
+int
+isalpharune(Rune c)
+{
+ Rune *p;
+
+ p = rbsearch(c, __isalphar, nelem(__isalphar)/2, 2);
+ if(p && c >= p[0] && c <= p[1])
+ return 1;
+ p = rbsearch(c, __isalphas, nelem(__isalphas), 1);
+ if(p && c == p[0])
+ return 1;
+ return 0;
+}
+
+static Rune __isupperr[] = {
+ 0x0041, 0x005a,
+ 0x00c0, 0x00d6,
+ 0x00d8, 0x00de,
+ 0x0178, 0x0179,
+ 0x0181, 0x0182,
+ 0x0186, 0x0187,
+ 0x0189, 0x018b,
+ 0x018e, 0x0191,
+ 0x0193, 0x0194,
+ 0x0196, 0x0198,
+ 0x019c, 0x019d,
+ 0x019f, 0x01a0,
+ 0x01a6, 0x01a7,
+ 0x01ae, 0x01af,
+ 0x01b1, 0x01b3,
+ 0x01b7, 0x01b8,
+ 0x01f6, 0x01f8,
+ 0x023a, 0x023b,
+ 0x023d, 0x023e,
+ 0x0243, 0x0246,
+ 0x0388, 0x038a,
+ 0x038e, 0x038f,
+ 0x0391, 0x03a1,
+ 0x03a3, 0x03ab,
+ 0x03d2, 0x03d4,
+ 0x03f9, 0x03fa,
+ 0x03fd, 0x042f,
+ 0x04c0, 0x04c1,
+ 0x0531, 0x0556,
+ 0x10a0, 0x10c5,
+ 0x1f08, 0x1f0f,
+ 0x1f18, 0x1f1d,
+ 0x1f28, 0x1f2f,
+ 0x1f38, 0x1f3f,
+ 0x1f48, 0x1f4d,
+ 0x1f68, 0x1f6f,
+ 0x1f88, 0x1f8f,
+ 0x1f98, 0x1f9f,
+ 0x1fa8, 0x1faf,
+ 0x1fb8, 0x1fbc,
+ 0x1fc8, 0x1fcc,
+ 0x1fd8, 0x1fdb,
+ 0x1fe8, 0x1fec,
+ 0x1ff8, 0x1ffc,
+ 0x210b, 0x210d,
+ 0x2110, 0x2112,
+ 0x2119, 0x211d,
+ 0x212a, 0x212d,
+ 0x2130, 0x2133,
+ 0x213e, 0x213f,
+ 0x2160, 0x216f,
+ 0x24b6, 0x24cf,
+ 0x2c00, 0x2c2e,
+ 0x2c62, 0x2c64,
+ 0xff21, 0xff3a,
+ 0x10400, 0x10427,
+ 0x1d400, 0x1d419,
+ 0x1d434, 0x1d44d,
+ 0x1d468, 0x1d481,
+ 0x1d49e, 0x1d49f,
+ 0x1d4a5, 0x1d4a6,
+ 0x1d4a9, 0x1d4ac,
+ 0x1d4ae, 0x1d4b5,
+ 0x1d4d0, 0x1d4e9,
+ 0x1d504, 0x1d505,
+ 0x1d507, 0x1d50a,
+ 0x1d50d, 0x1d514,
+ 0x1d516, 0x1d51c,
+ 0x1d538, 0x1d539,
+ 0x1d53b, 0x1d53e,
+ 0x1d540, 0x1d544,
+ 0x1d54a, 0x1d550,
+ 0x1d56c, 0x1d585,
+ 0x1d5a0, 0x1d5b9,
+ 0x1d5d4, 0x1d5ed,
+ 0x1d608, 0x1d621,
+ 0x1d63c, 0x1d655,
+ 0x1d670, 0x1d689,
+ 0x1d6a8, 0x1d6c0,
+ 0x1d6e2, 0x1d6fa,
+ 0x1d71c, 0x1d734,
+ 0x1d756, 0x1d76e,
+ 0x1d790, 0x1d7a8,
+};
+
+static Rune __isupperp[] = {
+ 0x0100, 0x0136,
+ 0x0139, 0x0147,
+ 0x014a, 0x0176,
+ 0x017b, 0x017d,
+ 0x01a2, 0x01a4,
+ 0x01cd, 0x01db,
+ 0x01de, 0x01ee,
+ 0x01fa, 0x0232,
+ 0x0248, 0x024e,
+ 0x03d8, 0x03ee,
+ 0x0460, 0x0480,
+ 0x048a, 0x04be,
+ 0x04c3, 0x04cd,
+ 0x04d0, 0x0512,
+ 0x1e00, 0x1e94,
+ 0x1ea0, 0x1ef8,
+ 0x1f59, 0x1f5f,
+ 0x2124, 0x2128,
+ 0x2c67, 0x2c6b,
+ 0x2c80, 0x2ce2,
+};
+
+static Rune __isuppers[] = {
+ 0x0184,
+ 0x01a9,
+ 0x01ac,
+ 0x01b5,
+ 0x01bc,
+ 0x01c4,
+ 0x01c7,
+ 0x01ca,
+ 0x01f1,
+ 0x01f4,
+ 0x0241,
+ 0x0386,
+ 0x038c,
+ 0x03f4,
+ 0x03f7,
+ 0x2102,
+ 0x2107,
+ 0x2115,
+ 0x2145,
+ 0x2183,
+ 0x2c60,
+ 0x2c75,
+ 0x1d49c,
+ 0x1d4a2,
+ 0x1d546,
+ 0x1d7ca,
+};
+
+int
+isupperrune(Rune c)
+{
+ Rune *p;
+
+ p = rbsearch(c, __isupperr, nelem(__isupperr)/2, 2);
+ if(p && c >= p[0] && c <= p[1])
+ return 1;
+ p = rbsearch(c, __isupperp, nelem(__isupperp)/2, 2);
+ if(p && c >= p[0] && c <= p[1] && !((c - p[0]) & 1))
+ return 1;
+ p = rbsearch(c, __isuppers, nelem(__isuppers), 1);
+ if(p && c == p[0])
+ return 1;
+ return 0;
+}
+
+static Rune __islowerr[] = {
+ 0x0061, 0x007a,
+ 0x00df, 0x00f6,
+ 0x00f8, 0x00ff,
+ 0x0137, 0x0138,
+ 0x0148, 0x0149,
+ 0x017e, 0x0180,
+ 0x018c, 0x018d,
+ 0x0199, 0x019b,
+ 0x01aa, 0x01ab,
+ 0x01b9, 0x01ba,
+ 0x01bd, 0x01bf,
+ 0x01dc, 0x01dd,
+ 0x01ef, 0x01f0,
+ 0x0233, 0x0239,
+ 0x023f, 0x0240,
+ 0x024f, 0x0293,
+ 0x0295, 0x02af,
+ 0x037b, 0x037d,
+ 0x03ac, 0x03ce,
+ 0x03d0, 0x03d1,
+ 0x03d5, 0x03d7,
+ 0x03ef, 0x03f3,
+ 0x03fb, 0x03fc,
+ 0x0430, 0x045f,
+ 0x04ce, 0x04cf,
+ 0x0561, 0x0587,
+ 0x1d00, 0x1d2b,
+ 0x1d62, 0x1d77,
+ 0x1d79, 0x1d9a,
+ 0x1e95, 0x1e9b,
+ 0x1f00, 0x1f07,
+ 0x1f10, 0x1f15,
+ 0x1f20, 0x1f27,
+ 0x1f30, 0x1f37,
+ 0x1f40, 0x1f45,
+ 0x1f50, 0x1f57,
+ 0x1f60, 0x1f67,
+ 0x1f70, 0x1f7d,
+ 0x1f80, 0x1f87,
+ 0x1f90, 0x1f97,
+ 0x1fa0, 0x1fa7,
+ 0x1fb0, 0x1fb4,
+ 0x1fb6, 0x1fb7,
+ 0x1fc2, 0x1fc4,
+ 0x1fc6, 0x1fc7,
+ 0x1fd0, 0x1fd3,
+ 0x1fd6, 0x1fd7,
+ 0x1fe0, 0x1fe7,
+ 0x1ff2, 0x1ff4,
+ 0x1ff6, 0x1ff7,
+ 0x210e, 0x210f,
+ 0x213c, 0x213d,
+ 0x2146, 0x2149,
+ 0x2170, 0x217f,
+ 0x24d0, 0x24e9,
+ 0x2c30, 0x2c5e,
+ 0x2c65, 0x2c66,
+ 0x2c76, 0x2c77,
+ 0x2ce3, 0x2ce4,
+ 0x2d00, 0x2d25,
+ 0xfb00, 0xfb06,
+ 0xfb13, 0xfb17,
+ 0xff41, 0xff5a,
+ 0x10428, 0x1044f,
+ 0x1d41a, 0x1d433,
+ 0x1d44e, 0x1d454,
+ 0x1d456, 0x1d467,
+ 0x1d482, 0x1d49b,
+ 0x1d4b6, 0x1d4b9,
+ 0x1d4bd, 0x1d4c3,
+ 0x1d4c5, 0x1d4cf,
+ 0x1d4ea, 0x1d503,
+ 0x1d51e, 0x1d537,
+ 0x1d552, 0x1d56b,
+ 0x1d586, 0x1d59f,
+ 0x1d5ba, 0x1d5d3,
+ 0x1d5ee, 0x1d607,
+ 0x1d622, 0x1d63b,
+ 0x1d656, 0x1d66f,
+ 0x1d68a, 0x1d6a5,
+ 0x1d6c2, 0x1d6da,
+ 0x1d6dc, 0x1d6e1,
+ 0x1d6fc, 0x1d714,
+ 0x1d716, 0x1d71b,
+ 0x1d736, 0x1d74e,
+ 0x1d750, 0x1d755,
+ 0x1d770, 0x1d788,
+ 0x1d78a, 0x1d78f,
+ 0x1d7aa, 0x1d7c2,
+ 0x1d7c4, 0x1d7c9,
+};
+
+static Rune __islowerp[] = {
+ 0x0101, 0x0135,
+ 0x013a, 0x0146,
+ 0x014b, 0x0177,
+ 0x017a, 0x017c,
+ 0x0183, 0x0185,
+ 0x01a1, 0x01a5,
+ 0x01b4, 0x01b6,
+ 0x01cc, 0x01da,
+ 0x01df, 0x01ed,
+ 0x01f3, 0x01f5,
+ 0x01f9, 0x0231,
+ 0x0247, 0x024d,
+ 0x03d9, 0x03ed,
+ 0x0461, 0x0481,
+ 0x048b, 0x04bf,
+ 0x04c2, 0x04cc,
+ 0x04d1, 0x0513,
+ 0x1e01, 0x1e93,
+ 0x1ea1, 0x1ef9,
+ 0x2c68, 0x2c6c,
+ 0x2c81, 0x2ce1,
+};
+
+static Rune __islowers[] = {
+ 0x00aa,
+ 0x00b5,
+ 0x00ba,
+ 0x0188,
+ 0x0192,
+ 0x0195,
+ 0x019e,
+ 0x01a8,
+ 0x01ad,
+ 0x01b0,
+ 0x01c6,
+ 0x01c9,
+ 0x023c,
+ 0x0242,
+ 0x0390,
+ 0x03f5,
+ 0x03f8,
+ 0x1fbe,
+ 0x2071,
+ 0x207f,
+ 0x210a,
+ 0x2113,
+ 0x212f,
+ 0x2134,
+ 0x2139,
+ 0x214e,
+ 0x2184,
+ 0x2c61,
+ 0x2c74,
+ 0x1d4bb,
+ 0x1d7cb,
+};
+
+int
+islowerrune(Rune c)
+{
+ Rune *p;
+
+ p = rbsearch(c, __islowerr, nelem(__islowerr)/2, 2);
+ if(p && c >= p[0] && c <= p[1])
+ return 1;
+ p = rbsearch(c, __islowerp, nelem(__islowerp)/2, 2);
+ if(p && c >= p[0] && c <= p[1] && !((c - p[0]) & 1))
+ return 1;
+ p = rbsearch(c, __islowers, nelem(__islowers), 1);
+ if(p && c == p[0])
+ return 1;
+ return 0;
+}
+
+static Rune __istitler[] = {
+ 0x0041, 0x005a,
+ 0x00c0, 0x00d6,
+ 0x00d8, 0x00de,
+ 0x0178, 0x0179,
+ 0x0181, 0x0182,
+ 0x0186, 0x0187,
+ 0x0189, 0x018b,
+ 0x018e, 0x0191,
+ 0x0193, 0x0194,
+ 0x0196, 0x0198,
+ 0x019c, 0x019d,
+ 0x019f, 0x01a0,
+ 0x01a6, 0x01a7,
+ 0x01ae, 0x01af,
+ 0x01b1, 0x01b3,
+ 0x01b7, 0x01b8,
+ 0x01f6, 0x01f8,
+ 0x023a, 0x023b,
+ 0x023d, 0x023e,
+ 0x0243, 0x0246,
+ 0x0388, 0x038a,
+ 0x038e, 0x038f,
+ 0x0391, 0x03a1,
+ 0x03a3, 0x03ab,
+ 0x03f9, 0x03fa,
+ 0x03fd, 0x042f,
+ 0x04c0, 0x04c1,
+ 0x0531, 0x0556,
+ 0x10a0, 0x10c5,
+ 0x1f08, 0x1f0f,
+ 0x1f18, 0x1f1d,
+ 0x1f28, 0x1f2f,
+ 0x1f38, 0x1f3f,
+ 0x1f48, 0x1f4d,
+ 0x1f68, 0x1f6f,
+ 0x1f88, 0x1f8f,
+ 0x1f98, 0x1f9f,
+ 0x1fa8, 0x1faf,
+ 0x1fb8, 0x1fbc,
+ 0x1fc8, 0x1fcc,
+ 0x1fd8, 0x1fdb,
+ 0x1fe8, 0x1fec,
+ 0x1ff8, 0x1ffc,
+ 0x2160, 0x216f,
+ 0x24b6, 0x24cf,
+ 0x2c00, 0x2c2e,
+ 0x2c62, 0x2c64,
+ 0xff21, 0xff3a,
+ 0x10400, 0x10427,
+};
+
+static Rune __istitlep[] = {
+ 0x0100, 0x012e,
+ 0x0132, 0x0136,
+ 0x0139, 0x0147,
+ 0x014a, 0x0176,
+ 0x017b, 0x017d,
+ 0x01a2, 0x01a4,
+ 0x01cb, 0x01db,
+ 0x01de, 0x01ee,
+ 0x01f2, 0x01f4,
+ 0x01fa, 0x0232,
+ 0x0248, 0x024e,
+ 0x03d8, 0x03ee,
+ 0x0460, 0x0480,
+ 0x048a, 0x04be,
+ 0x04c3, 0x04cd,
+ 0x04d0, 0x0512,
+ 0x1e00, 0x1e94,
+ 0x1ea0, 0x1ef8,
+ 0x1f59, 0x1f5f,
+ 0x2c67, 0x2c6b,
+ 0x2c80, 0x2ce2,
+};
+
+static Rune __istitles[] = {
+ 0x0184,
+ 0x01a9,
+ 0x01ac,
+ 0x01b5,
+ 0x01bc,
+ 0x01c5,
+ 0x01c8,
+ 0x0241,
+ 0x0386,
+ 0x038c,
+ 0x03f7,
+ 0x2132,
+ 0x2183,
+ 0x2c60,
+ 0x2c75,
+};
+
+int
+istitlerune(Rune c)
+{
+ Rune *p;
+
+ p = rbsearch(c, __istitler, nelem(__istitler)/2, 2);
+ if(p && c >= p[0] && c <= p[1])
+ return 1;
+ p = rbsearch(c, __istitlep, nelem(__istitlep)/2, 2);
+ if(p && c >= p[0] && c <= p[1] && !((c - p[0]) & 1))
+ return 1;
+ p = rbsearch(c, __istitles, nelem(__istitles), 1);
+ if(p && c == p[0])
+ return 1;
+ return 0;
+}
+
+static Rune __toupperr[] = {
+ 0x0061, 0x007a, 1048544,
+ 0x00e0, 0x00f6, 1048544,
+ 0x00f8, 0x00fe, 1048544,
+ 0x0256, 0x0257, 1048371,
+ 0x028a, 0x028b, 1048359,
+ 0x037b, 0x037d, 1048706,
+ 0x03ad, 0x03af, 1048539,
+ 0x03b1, 0x03c1, 1048544,
+ 0x03c3, 0x03cb, 1048544,
+ 0x03cd, 0x03ce, 1048513,
+ 0x0430, 0x044f, 1048544,
+ 0x0450, 0x045f, 1048496,
+ 0x0561, 0x0586, 1048528,
+ 0x1f00, 0x1f07, 1048584,
+ 0x1f10, 0x1f15, 1048584,
+ 0x1f20, 0x1f27, 1048584,
+ 0x1f30, 0x1f37, 1048584,
+ 0x1f40, 0x1f45, 1048584,
+ 0x1f60, 0x1f67, 1048584,
+ 0x1f70, 0x1f71, 1048650,
+ 0x1f72, 0x1f75, 1048662,
+ 0x1f76, 0x1f77, 1048676,
+ 0x1f78, 0x1f79, 1048704,
+ 0x1f7a, 0x1f7b, 1048688,
+ 0x1f7c, 0x1f7d, 1048702,
+ 0x1f80, 0x1f87, 1048584,
+ 0x1f90, 0x1f97, 1048584,
+ 0x1fa0, 0x1fa7, 1048584,
+ 0x1fb0, 0x1fb1, 1048584,
+ 0x1fd0, 0x1fd1, 1048584,
+ 0x1fe0, 0x1fe1, 1048584,
+ 0x2170, 0x217f, 1048560,
+ 0x24d0, 0x24e9, 1048550,
+ 0x2c30, 0x2c5e, 1048528,
+ 0x2d00, 0x2d25, 1041312,
+ 0xff41, 0xff5a, 1048544,
+ 0x10428, 0x1044f, 1048536,
+};
+
+static Rune __toupperp[] = {
+ 0x0101, 0x012f, 1048575,
+ 0x0133, 0x0137, 1048575,
+ 0x013a, 0x0148, 1048575,
+ 0x014b, 0x0177, 1048575,
+ 0x017a, 0x017e, 1048575,
+ 0x0183, 0x0185, 1048575,
+ 0x01a1, 0x01a5, 1048575,
+ 0x01b4, 0x01b6, 1048575,
+ 0x01ce, 0x01dc, 1048575,
+ 0x01df, 0x01ef, 1048575,
+ 0x01f9, 0x021f, 1048575,
+ 0x0223, 0x0233, 1048575,
+ 0x0247, 0x024f, 1048575,
+ 0x03d9, 0x03ef, 1048575,
+ 0x0461, 0x0481, 1048575,
+ 0x048b, 0x04bf, 1048575,
+ 0x04c2, 0x04ce, 1048575,
+ 0x04d1, 0x0513, 1048575,
+ 0x1e01, 0x1e95, 1048575,
+ 0x1ea1, 0x1ef9, 1048575,
+ 0x1f51, 0x1f57, 1048584,
+ 0x2c68, 0x2c6c, 1048575,
+ 0x2c81, 0x2ce3, 1048575,
+};
+
+static Rune __touppers[] = {
+ 0x00b5, 1049319,
+ 0x00ff, 1048697,
+ 0x0131, 1048344,
+ 0x017f, 1048276,
+ 0x0180, 1048771,
+ 0x0188, 1048575,
+ 0x018c, 1048575,
+ 0x0192, 1048575,
+ 0x0195, 1048673,
+ 0x0199, 1048575,
+ 0x019a, 1048739,
+ 0x019e, 1048706,
+ 0x01a8, 1048575,
+ 0x01ad, 1048575,
+ 0x01b0, 1048575,
+ 0x01b9, 1048575,
+ 0x01bd, 1048575,
+ 0x01bf, 1048632,
+ 0x01c5, 1048575,
+ 0x01c6, 1048574,
+ 0x01c8, 1048575,
+ 0x01c9, 1048574,
+ 0x01cb, 1048575,
+ 0x01cc, 1048574,
+ 0x01dd, 1048497,
+ 0x01f2, 1048575,
+ 0x01f3, 1048574,
+ 0x01f5, 1048575,
+ 0x023c, 1048575,
+ 0x0242, 1048575,
+ 0x0253, 1048366,
+ 0x0254, 1048370,
+ 0x0259, 1048374,
+ 0x025b, 1048373,
+ 0x0260, 1048371,
+ 0x0263, 1048369,
+ 0x0268, 1048367,
+ 0x0269, 1048365,
+ 0x026b, 1059319,
+ 0x026f, 1048365,
+ 0x0272, 1048363,
+ 0x0275, 1048362,
+ 0x027d, 1059303,
+ 0x0280, 1048358,
+ 0x0283, 1048358,
+ 0x0288, 1048358,
+ 0x0289, 1048507,
+ 0x028c, 1048505,
+ 0x0292, 1048357,
+ 0x0345, 1048660,
+ 0x03ac, 1048538,
+ 0x03c2, 1048545,
+ 0x03cc, 1048512,
+ 0x03d0, 1048514,
+ 0x03d1, 1048519,
+ 0x03d5, 1048529,
+ 0x03d6, 1048522,
+ 0x03f0, 1048490,
+ 0x03f1, 1048496,
+ 0x03f2, 1048583,
+ 0x03f5, 1048480,
+ 0x03f8, 1048575,
+ 0x03fb, 1048575,
+ 0x04cf, 1048561,
+ 0x1d7d, 1052390,
+ 0x1e9b, 1048517,
+ 0x1fb3, 1048585,
+ 0x1fbe, 1041371,
+ 0x1fc3, 1048585,
+ 0x1fe5, 1048583,
+ 0x1ff3, 1048585,
+ 0x214e, 1048548,
+ 0x2184, 1048575,
+ 0x2c61, 1048575,
+ 0x2c65, 1037781,
+ 0x2c66, 1037784,
+ 0x2c76, 1048575,
+};
+
+Rune
+toupperrune(Rune c)
+{
+ Rune *p;
+
+ p = rbsearch(c, __toupperr, nelem(__toupperr)/3, 3);
+ if(p && c >= p[0] && c <= p[1])
+ return c + p[2] - 1048576;
+ p = rbsearch(c, __toupperp, nelem(__toupperp)/3, 3);
+ if(p && c >= p[0] && c <= p[1] && !((c - p[0]) & 1))
+ return c + p[2] - 1048576;
+ p = rbsearch(c, __touppers, nelem(__touppers)/2, 2);
+ if(p && c == p[0])
+ return c + p[1] - 1048576;
+ return c;
+}
+
+static Rune __tolowerr[] = {
+ 0x0041, 0x005a, 1048608,
+ 0x00c0, 0x00d6, 1048608,
+ 0x00d8, 0x00de, 1048608,
+ 0x0189, 0x018a, 1048781,
+ 0x01b1, 0x01b2, 1048793,
+ 0x0388, 0x038a, 1048613,
+ 0x038e, 0x038f, 1048639,
+ 0x0391, 0x03a1, 1048608,
+ 0x03a3, 0x03ab, 1048608,
+ 0x03fd, 0x03ff, 1048446,
+ 0x0400, 0x040f, 1048656,
+ 0x0410, 0x042f, 1048608,
+ 0x0531, 0x0556, 1048624,
+ 0x10a0, 0x10c5, 1055840,
+ 0x1f08, 0x1f0f, 1048568,
+ 0x1f18, 0x1f1d, 1048568,
+ 0x1f28, 0x1f2f, 1048568,
+ 0x1f38, 0x1f3f, 1048568,
+ 0x1f48, 0x1f4d, 1048568,
+ 0x1f68, 0x1f6f, 1048568,
+ 0x1f88, 0x1f8f, 1048568,
+ 0x1f98, 0x1f9f, 1048568,
+ 0x1fa8, 0x1faf, 1048568,
+ 0x1fb8, 0x1fb9, 1048568,
+ 0x1fba, 0x1fbb, 1048502,
+ 0x1fc8, 0x1fcb, 1048490,
+ 0x1fd8, 0x1fd9, 1048568,
+ 0x1fda, 0x1fdb, 1048476,
+ 0x1fe8, 0x1fe9, 1048568,
+ 0x1fea, 0x1feb, 1048464,
+ 0x1ff8, 0x1ff9, 1048448,
+ 0x1ffa, 0x1ffb, 1048450,
+ 0x2160, 0x216f, 1048592,
+ 0x24b6, 0x24cf, 1048602,
+ 0x2c00, 0x2c2e, 1048624,
+ 0xff21, 0xff3a, 1048608,
+ 0x10400, 0x10427, 1048616,
+};
+
+static Rune __tolowerp[] = {
+ 0x0100, 0x012e, 1048577,
+ 0x0132, 0x0136, 1048577,
+ 0x0139, 0x0147, 1048577,
+ 0x014a, 0x0176, 1048577,
+ 0x017b, 0x017d, 1048577,
+ 0x01a2, 0x01a4, 1048577,
+ 0x01b3, 0x01b5, 1048577,
+ 0x01cd, 0x01db, 1048577,
+ 0x01de, 0x01ee, 1048577,
+ 0x01f8, 0x021e, 1048577,
+ 0x0222, 0x0232, 1048577,
+ 0x0248, 0x024e, 1048577,
+ 0x03d8, 0x03ee, 1048577,
+ 0x0460, 0x0480, 1048577,
+ 0x048a, 0x04be, 1048577,
+ 0x04c3, 0x04cd, 1048577,
+ 0x04d0, 0x0512, 1048577,
+ 0x1e00, 0x1e94, 1048577,
+ 0x1ea0, 0x1ef8, 1048577,
+ 0x1f59, 0x1f5f, 1048568,
+ 0x2c67, 0x2c6b, 1048577,
+ 0x2c80, 0x2ce2, 1048577,
+};
+
+static Rune __tolowers[] = {
+ 0x0130, 1048377,
+ 0x0178, 1048455,
+ 0x0179, 1048577,
+ 0x0181, 1048786,
+ 0x0182, 1048577,
+ 0x0184, 1048577,
+ 0x0186, 1048782,
+ 0x0187, 1048577,
+ 0x018b, 1048577,
+ 0x018e, 1048655,
+ 0x018f, 1048778,
+ 0x0190, 1048779,
+ 0x0191, 1048577,
+ 0x0193, 1048781,
+ 0x0194, 1048783,
+ 0x0196, 1048787,
+ 0x0197, 1048785,
+ 0x0198, 1048577,
+ 0x019c, 1048787,
+ 0x019d, 1048789,
+ 0x019f, 1048790,
+ 0x01a0, 1048577,
+ 0x01a6, 1048794,
+ 0x01a7, 1048577,
+ 0x01a9, 1048794,
+ 0x01ac, 1048577,
+ 0x01ae, 1048794,
+ 0x01af, 1048577,
+ 0x01b7, 1048795,
+ 0x01b8, 1048577,
+ 0x01bc, 1048577,
+ 0x01c4, 1048578,
+ 0x01c5, 1048577,
+ 0x01c7, 1048578,
+ 0x01c8, 1048577,
+ 0x01ca, 1048578,
+ 0x01cb, 1048577,
+ 0x01f1, 1048578,
+ 0x01f2, 1048577,
+ 0x01f4, 1048577,
+ 0x01f6, 1048479,
+ 0x01f7, 1048520,
+ 0x0220, 1048446,
+ 0x023a, 1059371,
+ 0x023b, 1048577,
+ 0x023d, 1048413,
+ 0x023e, 1059368,
+ 0x0241, 1048577,
+ 0x0243, 1048381,
+ 0x0244, 1048645,
+ 0x0245, 1048647,
+ 0x0246, 1048577,
+ 0x0386, 1048614,
+ 0x038c, 1048640,
+ 0x03f4, 1048516,
+ 0x03f7, 1048577,
+ 0x03f9, 1048569,
+ 0x03fa, 1048577,
+ 0x04c0, 1048591,
+ 0x04c1, 1048577,
+ 0x1fbc, 1048567,
+ 0x1fcc, 1048567,
+ 0x1fec, 1048569,
+ 0x1ffc, 1048567,
+ 0x2126, 1041059,
+ 0x212a, 1040193,
+ 0x212b, 1040314,
+ 0x2132, 1048604,
+ 0x2183, 1048577,
+ 0x2c60, 1048577,
+ 0x2c62, 1037833,
+ 0x2c63, 1044762,
+ 0x2c64, 1037849,
+ 0x2c75, 1048577,
+};
+
+Rune
+tolowerrune(Rune c)
+{
+ Rune *p;
+
+ p = rbsearch(c, __tolowerr, nelem(__tolowerr)/3, 3);
+ if(p && c >= p[0] && c <= p[1])
+ return c + p[2] - 1048576;
+ p = rbsearch(c, __tolowerp, nelem(__tolowerp)/3, 3);
+ if(p && c >= p[0] && c <= p[1] && !((c - p[0]) & 1))
+ return c + p[2] - 1048576;
+ p = rbsearch(c, __tolowers, nelem(__tolowers)/2, 2);
+ if(p && c == p[0])
+ return c + p[1] - 1048576;
+ return c;
+}
+
+static Rune __totitler[] = {
+ 0x0061, 0x007a, 1048544,
+ 0x00e0, 0x00f6, 1048544,
+ 0x00f8, 0x00fe, 1048544,
+ 0x0256, 0x0257, 1048371,
+ 0x028a, 0x028b, 1048359,
+ 0x037b, 0x037d, 1048706,
+ 0x03ad, 0x03af, 1048539,
+ 0x03b1, 0x03c1, 1048544,
+ 0x03c3, 0x03cb, 1048544,
+ 0x03cd, 0x03ce, 1048513,
+ 0x0430, 0x044f, 1048544,
+ 0x0450, 0x045f, 1048496,
+ 0x0561, 0x0586, 1048528,
+ 0x1f00, 0x1f07, 1048584,
+ 0x1f10, 0x1f15, 1048584,
+ 0x1f20, 0x1f27, 1048584,
+ 0x1f30, 0x1f37, 1048584,
+ 0x1f40, 0x1f45, 1048584,
+ 0x1f60, 0x1f67, 1048584,
+ 0x1f70, 0x1f71, 1048650,
+ 0x1f72, 0x1f75, 1048662,
+ 0x1f76, 0x1f77, 1048676,
+ 0x1f78, 0x1f79, 1048704,
+ 0x1f7a, 0x1f7b, 1048688,
+ 0x1f7c, 0x1f7d, 1048702,
+ 0x1f80, 0x1f87, 1048584,
+ 0x1f90, 0x1f97, 1048584,
+ 0x1fa0, 0x1fa7, 1048584,
+ 0x1fb0, 0x1fb1, 1048584,
+ 0x1fd0, 0x1fd1, 1048584,
+ 0x1fe0, 0x1fe1, 1048584,
+ 0x2170, 0x217f, 1048560,
+ 0x24d0, 0x24e9, 1048550,
+ 0x2c30, 0x2c5e, 1048528,
+ 0x2d00, 0x2d25, 1041312,
+ 0xff41, 0xff5a, 1048544,
+ 0x10428, 0x1044f, 1048536,
+};
+
+static Rune __totitlep[] = {
+ 0x0101, 0x012f, 1048575,
+ 0x0133, 0x0137, 1048575,
+ 0x013a, 0x0148, 1048575,
+ 0x014b, 0x0177, 1048575,
+ 0x017a, 0x017e, 1048575,
+ 0x0183, 0x0185, 1048575,
+ 0x01a1, 0x01a5, 1048575,
+ 0x01b4, 0x01b6, 1048575,
+ 0x01cc, 0x01dc, 1048575,
+ 0x01df, 0x01ef, 1048575,
+ 0x01f3, 0x01f5, 1048575,
+ 0x01f9, 0x021f, 1048575,
+ 0x0223, 0x0233, 1048575,
+ 0x0247, 0x024f, 1048575,
+ 0x03d9, 0x03ef, 1048575,
+ 0x0461, 0x0481, 1048575,
+ 0x048b, 0x04bf, 1048575,
+ 0x04c2, 0x04ce, 1048575,
+ 0x04d1, 0x0513, 1048575,
+ 0x1e01, 0x1e95, 1048575,
+ 0x1ea1, 0x1ef9, 1048575,
+ 0x1f51, 0x1f57, 1048584,
+ 0x2c68, 0x2c6c, 1048575,
+ 0x2c81, 0x2ce3, 1048575,
+};
+
+static Rune __totitles[] = {
+ 0x00b5, 1049319,
+ 0x00ff, 1048697,
+ 0x0131, 1048344,
+ 0x017f, 1048276,
+ 0x0180, 1048771,
+ 0x0188, 1048575,
+ 0x018c, 1048575,
+ 0x0192, 1048575,
+ 0x0195, 1048673,
+ 0x0199, 1048575,
+ 0x019a, 1048739,
+ 0x019e, 1048706,
+ 0x01a8, 1048575,
+ 0x01ad, 1048575,
+ 0x01b0, 1048575,
+ 0x01b9, 1048575,
+ 0x01bd, 1048575,
+ 0x01bf, 1048632,
+ 0x01c4, 1048577,
+ 0x01c6, 1048575,
+ 0x01c7, 1048577,
+ 0x01c9, 1048575,
+ 0x01ca, 1048577,
+ 0x01dd, 1048497,
+ 0x01f1, 1048577,
+ 0x023c, 1048575,
+ 0x0242, 1048575,
+ 0x0253, 1048366,
+ 0x0254, 1048370,
+ 0x0259, 1048374,
+ 0x025b, 1048373,
+ 0x0260, 1048371,
+ 0x0263, 1048369,
+ 0x0268, 1048367,
+ 0x0269, 1048365,
+ 0x026b, 1059319,
+ 0x026f, 1048365,
+ 0x0272, 1048363,
+ 0x0275, 1048362,
+ 0x027d, 1059303,
+ 0x0280, 1048358,
+ 0x0283, 1048358,
+ 0x0288, 1048358,
+ 0x0289, 1048507,
+ 0x028c, 1048505,
+ 0x0292, 1048357,
+ 0x0345, 1048660,
+ 0x03ac, 1048538,
+ 0x03c2, 1048545,
+ 0x03cc, 1048512,
+ 0x03d0, 1048514,
+ 0x03d1, 1048519,
+ 0x03d5, 1048529,
+ 0x03d6, 1048522,
+ 0x03f0, 1048490,
+ 0x03f1, 1048496,
+ 0x03f2, 1048583,
+ 0x03f5, 1048480,
+ 0x03f8, 1048575,
+ 0x03fb, 1048575,
+ 0x04cf, 1048561,
+ 0x1d7d, 1052390,
+ 0x1e9b, 1048517,
+ 0x1fb3, 1048585,
+ 0x1fbe, 1041371,
+ 0x1fc3, 1048585,
+ 0x1fe5, 1048583,
+ 0x1ff3, 1048585,
+ 0x214e, 1048548,
+ 0x2184, 1048575,
+ 0x2c61, 1048575,
+ 0x2c65, 1037781,
+ 0x2c66, 1037784,
+ 0x2c76, 1048575,
+};
+
+Rune
+totitlerune(Rune c)
+{
+ Rune *p;
+
+ p = rbsearch(c, __totitler, nelem(__totitler)/3, 3);
+ if(p && c >= p[0] && c <= p[1])
+ return c + p[2] - 1048576;
+ p = rbsearch(c, __totitlep, nelem(__totitlep)/3, 3);
+ if(p && c >= p[0] && c <= p[1] && !((c - p[0]) & 1))
+ return c + p[2] - 1048576;
+ p = rbsearch(c, __totitles, nelem(__totitles)/2, 2);
+ if(p && c == p[0])
+ return c + p[1] - 1048576;
+ return c;
+}
+
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/utf.h b/trunk/src/third_party/css_parser/src/third_party/utf/utf.h
new file mode 100644
index 0000000..02ba472
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/utf.h
@@ -0,0 +1,233 @@
+#ifndef _UTFH_
+#define _UTFH_ 1
+
+#include <stdint.h>
+
+typedef signed int Rune; /* Code-point values in Unicode 4.0 are 21 bits wide.*/
+
+enum
+{
+ UTFmax = 4, /* maximum bytes per rune */
+ Runesync = 0x80, /* cannot represent part of a UTF sequence (<) */
+ Runeself = 0x80, /* rune and UTF sequences are the same (<) */
+ Runeerror = 0xFFFD, /* decoding error in UTF */
+ Runemax = 0x10FFFF, /* maximum rune value */
+};
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/*
+ * rune routines
+ */
+
+/*
+ * These routines were written by Rob Pike and Ken Thompson
+ * and first appeared in Plan 9.
+ * SEE ALSO
+ * utf (7)
+ * tcs (1)
+*/
+
+// runetochar copies (encodes) one rune, pointed to by r, to at most
+// UTFmax bytes starting at s and returns the number of bytes generated.
+
+int runetochar(char* s, const Rune* r);
+
+
+// chartorune copies (decodes) at most UTFmax bytes starting at s to
+// one rune, pointed to by r, and returns the number of bytes consumed.
+// If the input is not exactly in UTF format, chartorune will set *r
+// to Runeerror and return 1.
+//
+// Note: There is no special case for a "null-terminated" string. A
+// string whose first byte has the value 0 is the UTF8 encoding of the
+// Unicode value 0 (i.e., ASCII NULL). A byte value of 0 is illegal
+// anywhere else in a UTF sequence.
+
+int chartorune(Rune* r, const char* s);
+
+
+// charntorune is like chartorune, except that it will access at most
+// n bytes of s. If the UTF sequence is incomplete within n bytes,
+// charntorune will set *r to Runeerror and return 0. If it is complete
+// but not in UTF format, it will set *r to Runeerror and return 1.
+//
+// Added 2004-09-24 by Wei-Hwa Huang
+
+int charntorune(Rune* r, const char* s, int n);
+
+// isvalidcharntorune(str, n, r, consumed)
+// is a convenience function that calls "*consumed = charntorune(r, str, n)"
+// and returns an int (logically boolean) indicating whether the first
+// n bytes of str was a valid and complete UTF sequence.
+
+int isvalidcharntorune(const char* str, int n, Rune* r, int* consumed);
+
+// runelen returns the number of bytes required to convert r into UTF.
+
+int runelen(Rune r);
+
+
+// runenlen returns the number of bytes required to convert the n
+// runes pointed to by r into UTF.
+
+int runenlen(const Rune* r, int n);
+
+
+// fullrune returns 1 if the string s of length n is long enough to be
+// decoded by chartorune, and 0 otherwise. This does not guarantee
+// that the string contains a legal UTF encoding. This routine is used
+// by programs that obtain input one byte at a time and need to know
+// when a full rune has arrived.
+
+int fullrune(const char* s, int n);
+
+// The following routines are analogous to the corresponding string
+// routines with "utf" substituted for "str", and "rune" substituted
+// for "chr".
+
+// utflen returns the number of runes that are represented by the UTF
+// string s. (cf. strlen)
+
+int utflen(const char* s);
+
+
+// utfnlen returns the number of complete runes that are represented
+// by the first n bytes of the UTF string s. If the last few bytes of
+// the string contain an incompletely coded rune, utfnlen will not
+// count them; in this way, it differs from utflen, which includes
+// every byte of the string. (cf. strnlen)
+
+int utfnlen(const char* s, long n);
+
+
+// utfrune returns a pointer to the first occurrence of rune r in the
+// UTF string s, or 0 if r does not occur in the string. The NULL
+// byte terminating a string is considered to be part of the string s.
+// (cf. strchr)
+
+const char* utfrune(const char* s, Rune r);
+
+
+// utfrrune returns a pointer to the last occurrence of rune r in the
+// UTF string s, or 0 if r does not occur in the string. The NULL
+// byte terminating a string is considered to be part of the string s.
+// (cf. strrchr)
+
+const char* utfrrune(const char* s, Rune r);
+
+
+// utfutf returns a pointer to the first occurrence of the UTF string
+// s2 as a UTF substring of s1, or 0 if there is none. If s2 is the
+// null string, utfutf returns s1. (cf. strstr)
+
+const char* utfutf(const char* s1, const char* s2);
+
+
+// utfecpy copies UTF sequences until a null sequence has been copied,
+// but writes no sequences beyond es1. If any sequences are copied,
+// s1 is terminated by a null sequence, and a pointer to that sequence
+// is returned. Otherwise, the original s1 is returned. (cf. strecpy)
+
+char* utfecpy(char *s1, char *es1, const char *s2);
+
+
+
+// These functions are rune-string analogues of the corresponding
+// functions in strcat (3).
+//
+// These routines first appeared in Plan 9.
+// SEE ALSO
+// memmove (3)
+// rune (3)
+// strcat (2)
+//
+// BUGS: The outcome of overlapping moves varies among implementations.
+
+Rune* runestrcat(Rune* s1, const Rune* s2);
+Rune* runestrncat(Rune* s1, const Rune* s2, long n);
+
+const Rune* runestrchr(const Rune* s, Rune c);
+
+int runestrcmp(const Rune* s1, const Rune* s2);
+int runestrncmp(const Rune* s1, const Rune* s2, long n);
+
+Rune* runestrcpy(Rune* s1, const Rune* s2);
+Rune* runestrncpy(Rune* s1, const Rune* s2, long n);
+Rune* runestrecpy(Rune* s1, Rune* es1, const Rune* s2);
+
+Rune* runestrdup(const Rune* s);
+
+const Rune* runestrrchr(const Rune* s, Rune c);
+long runestrlen(const Rune* s);
+const Rune* runestrstr(const Rune* s1, const Rune* s2);
+
+
+
+// The following routines test types and modify cases for Unicode
+// characters. Unicode defines some characters as letters and
+// specifies three cases: upper, lower, and title. Mappings among the
+// cases are also defined, although they are not exhaustive: some
+// upper case letters have no lower case mapping, and so on. Unicode
+// also defines several character properties, a subset of which are
+// checked by these routines. These routines are based on Unicode
+// version 3.0.0.
+//
+// NOTE: The routines are implemented in C, so the boolean functions
+// (e.g., isupperrune) return 0 for false and 1 for true.
+//
+//
+// toupperrune, tolowerrune, and totitlerune are the Unicode case
+// mappings. These routines return the character unchanged if it has
+// no defined mapping.
+
+Rune toupperrune(Rune r);
+Rune tolowerrune(Rune r);
+Rune totitlerune(Rune r);
+
+
+// isupperrune tests for upper case characters, including Unicode
+// upper case letters and targets of the toupper mapping. islowerrune
+// and istitlerune are defined analogously.
+
+int isupperrune(Rune r);
+int islowerrune(Rune r);
+int istitlerune(Rune r);
+
+
+// isalpharune tests for Unicode letters; this includes ideographs in
+// addition to alphabetic characters.
+
+int isalpharune(Rune r);
+
+
+// isdigitrune tests for digits. Non-digit numbers, such as Roman
+// numerals, are not included.
+
+int isdigitrune(Rune r);
+
+
+// isideographicrune tests for ideographic characters and numbers, as
+// defined by the Unicode standard.
+
+int isideographicrune(Rune r);
+
+
+// isspacerune tests for whitespace characters, including "C" locale
+// whitespace, Unicode defined whitespace, and the "zero-width
+// non-break space" character.
+
+int isspacerune(Rune r);
+
+
+// (The comments in this file were copied from the manpage files rune.3,
+// isalpharune.3, and runestrcat.3. Some formatting changes were also made
+// to conform to Google style. /JRM 11/11/05)
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/utfdef.h b/trunk/src/third_party/css_parser/src/third_party/utf/utfdef.h
new file mode 100644
index 0000000..4b58ae8
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/utfdef.h
@@ -0,0 +1,14 @@
+#define uchar _utfuchar
+#define ushort _utfushort
+#define uint _utfuint
+#define ulong _utfulong
+#define vlong _utfvlong
+#define uvlong _utfuvlong
+
+typedef unsigned char uchar;
+typedef unsigned short ushort;
+typedef unsigned int uint;
+typedef unsigned long ulong;
+
+#define nelem(x) (sizeof(x)/sizeof((x)[0]))
+#define nil ((void*)0)
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/utfecpy.c b/trunk/src/third_party/css_parser/src/third_party/utf/utfecpy.c
new file mode 100644
index 0000000..5cf19af
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/utfecpy.c
@@ -0,0 +1,36 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+char*
+utfecpy(char *to, char *e, const char *from)
+{
+ char *end;
+
+ if(to >= e)
+ return to;
+ end = (char*)memccpy(to, from, '\0', e - to);
+ if(end == nil){
+ end = e-1;
+ while(end>to && (*--end&0xC0)==0x80)
+ ;
+ *end = '\0';
+ }else{
+ end--;
+ }
+ return end;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/utflen.c b/trunk/src/third_party/css_parser/src/third_party/utf/utflen.c
new file mode 100644
index 0000000..5a23e0a
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/utflen.c
@@ -0,0 +1,38 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+int
+utflen(const char *s)
+{
+ int c;
+ long n;
+ Rune rune;
+
+ n = 0;
+ for(;;) {
+ c = *(uchar*)s;
+ if(c < Runeself) {
+ if(c == 0)
+ return n;
+ s++;
+ } else
+ s += chartorune(&rune, s);
+ n++;
+ }
+ return 0;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/utfnlen.c b/trunk/src/third_party/css_parser/src/third_party/utf/utfnlen.c
new file mode 100644
index 0000000..1f191fd
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/utfnlen.c
@@ -0,0 +1,41 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+int
+utfnlen(const char *s, long m)
+{
+ int c;
+ long n;
+ Rune rune;
+ const char *es;
+
+ es = s + m;
+ for(n = 0; s < es; n++) {
+ c = *(uchar*)s;
+ if(c < Runeself){
+ if(c == '\0')
+ break;
+ s++;
+ continue;
+ }
+ if(!fullrune(s, es-s))
+ break;
+ s += chartorune(&rune, s);
+ }
+ return n;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/utfrrune.c b/trunk/src/third_party/css_parser/src/third_party/utf/utfrrune.c
new file mode 100644
index 0000000..bd3351e
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/utfrrune.c
@@ -0,0 +1,47 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+const
+char*
+utfrrune(const char *s, Rune c)
+{
+ long c1;
+ Rune r;
+ const char *s1;
+
+ if(c < Runesync) /* not part of utf sequence */
+ return strrchr(s, c);
+
+ s1 = 0;
+ for(;;) {
+ c1 = *(uchar*)s;
+ if(c1 < Runeself) { /* one byte rune */
+ if(c1 == 0)
+ return s1;
+ if(c1 == c)
+ s1 = s;
+ s++;
+ continue;
+ }
+ c1 = chartorune(&r, s);
+ if(r == c)
+ s1 = s;
+ s += c1;
+ }
+ return 0;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/utfrune.c b/trunk/src/third_party/css_parser/src/third_party/utf/utfrune.c
new file mode 100644
index 0000000..2d8fa18
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/utfrune.c
@@ -0,0 +1,46 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+const
+char*
+utfrune(const char *s, Rune c)
+{
+ long c1;
+ Rune r;
+ int n;
+
+ if(c < Runesync) /* not part of utf sequence */
+ return strchr(s, c);
+
+ for(;;) {
+ c1 = *(uchar*)s;
+ if(c1 < Runeself) { /* one byte rune */
+ if(c1 == 0)
+ return 0;
+ if(c1 == c)
+ return s;
+ s++;
+ continue;
+ }
+ n = chartorune(&r, s);
+ if(r == c)
+ return s;
+ s += n;
+ }
+ return 0;
+}
diff --git a/trunk/src/third_party/css_parser/src/third_party/utf/utfutf.c b/trunk/src/third_party/css_parser/src/third_party/utf/utfutf.c
new file mode 100644
index 0000000..df242d1
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/third_party/utf/utfutf.c
@@ -0,0 +1,42 @@
+/*
+ * The authors of this software are Rob Pike and Ken Thompson.
+ * Copyright (c) 2002 by Lucent Technologies.
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose without fee is hereby granted, provided that this entire notice
+ * is included in all copies of any software which is or includes a copy
+ * or modification of this software and in all copies of the supporting
+ * documentation for such software.
+ * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
+ * WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
+ * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
+ * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+ */
+#include <stdarg.h>
+#include <string.h>
+#include "third_party/utf/utf.h"
+#include "third_party/utf/utfdef.h"
+
+
+/*
+ * Return pointer to first occurrence of s2 in s1,
+ * 0 if none
+ */
+const
+char*
+utfutf(const char *s1, const char *s2)
+{
+ const char *p;
+ long f, n1, n2;
+ Rune r;
+
+ n1 = chartorune(&r, s2);
+ f = r;
+ if(f <= Runesync) /* represents self */
+ return strstr(s1, s2);
+
+ n2 = strlen(s2);
+ for(p=s1; (p=utfrune(p, f)) != 0; p+=n1)
+ if(strncmp(p, s2, n2) == 0)
+ return p;
+ return 0;
+}
diff --git a/trunk/src/third_party/css_parser/src/util/gtl/dense_hash_map.h b/trunk/src/third_party/css_parser/src/util/gtl/dense_hash_map.h
new file mode 100644
index 0000000..7de44ea
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/util/gtl/dense_hash_map.h
@@ -0,0 +1,24 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef UTIL_GTL_DENSE_HASH_MAP_H_
+#define UTIL_GTL_DENSE_HASH_MAP_H_
+
+//#include "third_party/google-sparsehash/src/google/dense_hash_map"
+#include <google/dense_hash_map>
+
+using google::dense_hash_map;
+
+#endif // UTIL_GTL_DENSE_HASH_MAP_H_
diff --git a/trunk/src/third_party/css_parser/src/util/gtl/map-util.h b/trunk/src/third_party/css_parser/src/util/gtl/map-util.h
new file mode 100644
index 0000000..6bc8344
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/util/gtl/map-util.h
@@ -0,0 +1,22 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef UTIL_GTL_MAP_UTIL_H_
+#define UTIL_GTL_MAP_UTIL_H_
+
+//#include "third_party/chromium/src/base/stl_util-inl.h"
+#include "base/stl_util-inl.h"
+
+#endif // UTIL_GTL_MAP_UTIL_H_
diff --git a/trunk/src/third_party/css_parser/src/util/gtl/singleton.h b/trunk/src/third_party/css_parser/src/util/gtl/singleton.h
new file mode 100644
index 0000000..ff03672
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/util/gtl/singleton.h
@@ -0,0 +1,22 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef UTIL_GTL_SINGLETON_H_
+#define UTIL_GTL_SINGLETON_H_
+
+//#include "third_party/chromium/src/base/singleton.h"
+#include "base/singleton.h"
+
+#endif // UTIL_GTL_SINGLETON_H_
diff --git a/trunk/src/third_party/css_parser/src/util/gtl/stl_util-inl.h b/trunk/src/third_party/css_parser/src/util/gtl/stl_util-inl.h
new file mode 100644
index 0000000..d3f1326
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/util/gtl/stl_util-inl.h
@@ -0,0 +1,22 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef UTIL_GTL_STL_UTIL_INL_H_
+#define UTIL_GTL_STL_UTIL_INL_H_
+
+//#include "third_party/chromium/src/base/stl_util-inl.h"
+#include "base/stl_util-inl.h"
+
+#endif // UTIL_GTL_STL_UTIL_INL_H_
diff --git a/trunk/src/third_party/css_parser/src/util/hash/hash.h b/trunk/src/third_party/css_parser/src/util/hash/hash.h
new file mode 100644
index 0000000..d09072c
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/util/hash/hash.h
@@ -0,0 +1,15 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
diff --git a/trunk/src/third_party/css_parser/src/util/utf8/internal/unicodetext.cc b/trunk/src/third_party/css_parser/src/util/utf8/internal/unicodetext.cc
new file mode 100644
index 0000000..c1a885b
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/util/utf8/internal/unicodetext.cc
@@ -0,0 +1,487 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2006 Google Inc. All Rights Reserved.
+// Author: jrm@google.com (Jim Meehan)
+
+
+#include "util/utf8/public/unicodetext.h"
+
+#include "base/logging.h"
+#include "strings/stringpiece.h"
+#include "strings/stringprintf.h"
+#include "third_party/utf/utf.h"
+#include "util/utf8/public/unilib.h"
+
+static int CodepointDistance(const char* start, const char* end) {
+ int n = 0;
+ // Increment n on every non-trail-byte.
+ for (const char* p = start; p < end; ++p) {
+ n += (*reinterpret_cast<const signed char*>(p) >= -0x40);
+ }
+ return n;
+}
+
+static int CodepointCount(const char* utf8, int len) {
+ return CodepointDistance(utf8, utf8 + len);
+}
+
+UnicodeText::const_iterator::difference_type
+distance(const UnicodeText::const_iterator& first,
+ const UnicodeText::const_iterator& last) {
+ return CodepointDistance(first.it_, last.it_);
+}
+
+// ---------- Utility ----------
+
+static int ConvertToInterchangeValid(char* start, int len) {
+ // This routine is called only when we've discovered that a UTF-8 buffer
+ // that was passed to CopyUTF8, TakeOwnershipOfUTF8, or PointToUTF8
+ // was not interchange valid. This indicates a bug in the caller, and
+ // a LOG(WARNING) is done in that case.
+ // This is similar to CoerceToInterchangeValid, but it replaces each
+ // structurally valid byte with a space, and each non-interchange
+ // character with a space, even when that character requires more
+ // than one byte in UTF8. E.g., "\xEF\xB7\x90" (U+FDD0) is
+ // structurally valid UTF8, but U+FDD0 is not an interchange-valid
+ // code point. The result should contain one space, not three.
+ //
+ // Since the conversion never needs to write more data than it
+ // reads, it is safe to change the buffer in place. It returns the
+ // number of bytes written.
+ char* const in = start;
+ char* out = start;
+ char* const end = start + len;
+ while (start < end) {
+ int good = UniLib::SpanInterchangeValid(start, end - start);
+ if (good > 0) {
+ if (out != start) {
+ memmove(out, start, good);
+ }
+ out += good;
+ start += good;
+ if (start == end) {
+ break;
+ }
+ }
+ // Is the current string invalid UTF8 or just non-interchange UTF8?
+ char32 rune;
+ int n;
+ if (isvalidcharntorune(start, end - start, &rune, &n)) {
+ // structurally valid UTF8, but not interchange valid
+ start += n; // Skip over the whole character.
+ } else { // bad UTF8
+ start += 1; // Skip over just one byte
+ }
+ *out++ = ' ';
+ }
+ return out - in;
+}
+
+
+// *************** Data representation **********
+
+// Note: the copy constructor is undefined.
+
+// After reserve(), resize(), or clear(), we're an owner, not an alias.
+
+void UnicodeText::Repr::reserve(int new_capacity) {
+ // If there's already enough capacity, and we're an owner, do nothing.
+ if (capacity_ >= new_capacity && ours_) return;
+
+ // Otherwise, allocate a new buffer.
+ capacity_ = max(new_capacity, (3 * capacity_) / 2 + 20);
+ char* new_data = new char[capacity_];
+
+ // If there is an old buffer, copy it into the new buffer.
+ if (data_) {
+ memcpy(new_data, data_, size_);
+ if (ours_) delete[] data_; // If we owned the old buffer, free it.
+ }
+ data_ = new_data;
+ ours_ = true; // We own the new buffer.
+ // size_ is unchanged.
+}
+
+void UnicodeText::Repr::resize(int new_size) {
+ if (new_size == 0) {
+ clear();
+ } else {
+ if (!ours_ || new_size > capacity_) reserve(new_size);
+ // Clear the memory in the expanded part.
+ if (size_ < new_size) memset(data_ + size_, 0, new_size - size_);
+ size_ = new_size;
+ ours_ = true;
+ }
+}
+
+// This implementation of clear() deallocates the buffer if we're an owner.
+// That's not strictly necessary; we could just set size_ to 0.
+void UnicodeText::Repr::clear() {
+ if (ours_) delete[] data_;
+ data_ = NULL;
+ size_ = capacity_ = 0;
+ ours_ = true;
+}
+
+void UnicodeText::Repr::Copy(const char* data, int size) {
+ resize(size);
+ memcpy(data_, data, size);
+}
+
+void UnicodeText::Repr::TakeOwnershipOf(char* data, int size, int capacity) {
+ if (data == data_) return; // We already own this memory. (Weird case.)
+ if (ours_ && data_) delete[] data_; // If we owned the old buffer, free it.
+ data_ = data;
+ size_ = size;
+ capacity_ = capacity;
+ ours_ = true;
+}
+
+void UnicodeText::Repr::PointTo(const char* data, int size) {
+ if (ours_ && data_) delete[] data_; // If we owned the old buffer, free it.
+ data_ = const_cast<char*>(data);
+ size_ = size;
+ capacity_ = size;
+ ours_ = false;
+}
+
+void UnicodeText::Repr::append(const char* bytes, int byte_length) {
+ reserve(size_ + byte_length);
+ memcpy(data_ + size_, bytes, byte_length);
+ size_ += byte_length;
+}
+
+string UnicodeText::Repr::DebugString() const {
+ return StringPrintf("{Repr %p data=%p size=%d capacity=%d %s}",
+ this,
+ data_, size_, capacity_,
+ ours_ ? "Owned" : "Alias");
+}
+
+
+
+// *************** UnicodeText ******************
+
+// ----- Constructors -----
+
+// Default constructor
+UnicodeText::UnicodeText() {
+}
+
+// Copy constructor
+UnicodeText::UnicodeText(const UnicodeText& src) {
+ Copy(src);
+}
+
+// Substring constructor
+UnicodeText::UnicodeText(const UnicodeText::const_iterator& first,
+ const UnicodeText::const_iterator& last) {
+ CHECK(first <= last) << " Incompatible iterators";
+ repr_.append(first.it_, last.it_ - first.it_);
+}
+
+string UnicodeText::UTF8Substring(const const_iterator& first,
+ const const_iterator& last) {
+ CHECK(first <= last) << " Incompatible iterators";
+ return string(first.it_, last.it_ - first.it_);
+}
+
+
+// ----- Copy -----
+
+UnicodeText& UnicodeText::operator=(const UnicodeText& src) {
+ if (this != &src) {
+ Copy(src);
+ }
+ return *this;
+}
+
+UnicodeText& UnicodeText::Copy(const UnicodeText& src) {
+ repr_.Copy(src.repr_.data_, src.repr_.size_);
+ return *this;
+}
+
+UnicodeText& UnicodeText::CopyUTF8(const char* buffer, int byte_length) {
+ repr_.Copy(buffer, byte_length);
+ if (!UniLib:: IsInterchangeValid(buffer, byte_length)) {
+ LOG(WARNING) << "UTF-8 buffer is not interchange-valid.";
+ repr_.size_ = ConvertToInterchangeValid(repr_.data_, byte_length);
+ }
+ return *this;
+}
+
+UnicodeText& UnicodeText::UnsafeCopyUTF8(const char* buffer,
+ int byte_length) {
+ repr_.Copy(buffer, byte_length);
+ return *this;
+}
+
+// ----- TakeOwnershipOf -----
+
+UnicodeText& UnicodeText::TakeOwnershipOfUTF8(char* buffer,
+ int byte_length,
+ int byte_capacity) {
+ repr_.TakeOwnershipOf(buffer, byte_length, byte_capacity);
+ if (!UniLib:: IsInterchangeValid(buffer, byte_length)) {
+ LOG(WARNING) << "UTF-8 buffer is not interchange-valid.";
+ repr_.size_ = ConvertToInterchangeValid(repr_.data_, byte_length);
+ }
+ return *this;
+}
+
+UnicodeText& UnicodeText::UnsafeTakeOwnershipOfUTF8(char* buffer,
+ int byte_length,
+ int byte_capacity) {
+ repr_.TakeOwnershipOf(buffer, byte_length, byte_capacity);
+ return *this;
+}
+
+// ----- PointTo -----
+
+UnicodeText& UnicodeText::PointToUTF8(const char* buffer, int byte_length) {
+ if (UniLib:: IsInterchangeValid(buffer, byte_length)) {
+ repr_.PointTo(buffer, byte_length);
+ } else {
+ LOG(WARNING) << "UTF-8 buffer is not interchange-valid.";
+ repr_.Copy(buffer, byte_length);
+ repr_.size_ = ConvertToInterchangeValid(repr_.data_, byte_length);
+ }
+ return *this;
+}
+
+UnicodeText& UnicodeText::UnsafePointToUTF8(const char* buffer,
+ int byte_length) {
+ repr_.PointTo(buffer, byte_length);
+ return *this;
+}
+
+UnicodeText& UnicodeText::PointTo(const UnicodeText& src) {
+ repr_.PointTo(src.repr_.data_, src.repr_.size_);
+ return *this;
+}
+
+UnicodeText& UnicodeText::PointTo(const const_iterator &first,
+ const const_iterator &last) {
+ CHECK(first <= last) << " Incompatible iterators";
+ repr_.PointTo(first.utf8_data(), last.utf8_data() - first.utf8_data());
+ return *this;
+}
+
+// ----- Append -----
+
+UnicodeText& UnicodeText::append(const UnicodeText& u) {
+ repr_.append(u.repr_.data_, u.repr_.size_);
+ return *this;
+}
+
+UnicodeText& UnicodeText::append(const const_iterator& first,
+ const const_iterator& last) {
+ CHECK(first <= last) << " Incompatible iterators";
+ repr_.append(first.it_, last.it_ - first.it_);
+ return *this;
+}
+
+UnicodeText& UnicodeText::UnsafeAppendUTF8(const char* utf8, int len) {
+ repr_.append(utf8, len);
+ return *this;
+}
+
+// ----- substring searching -----
+
+UnicodeText::const_iterator UnicodeText::find(const UnicodeText& look,
+ const_iterator start_pos) const {
+ CHECK_GE(start_pos.utf8_data(), utf8_data());
+ CHECK_LE(start_pos.utf8_data(), utf8_data() + utf8_length());
+ return UnsafeFind(look, start_pos);
+}
+
+UnicodeText::const_iterator UnicodeText::find(const UnicodeText& look) const {
+ return UnsafeFind(look, begin());
+}
+
+UnicodeText::const_iterator UnicodeText::UnsafeFind(
+ const UnicodeText& look, const_iterator start_pos) const {
+ // Due to the magic of the UTF8 encoding, searching for a sequence of
+ // letters is equivalent to substring search.
+ StringPiece searching(utf8_data(), utf8_length());
+ StringPiece look_piece(look.utf8_data(), look.utf8_length());
+ StringPiece::size_type found =
+ searching.find(look_piece, start_pos.utf8_data() - utf8_data());
+ if (found == StringPiece::npos) return end();
+ return const_iterator(utf8_data() + found);
+}
+
+bool UnicodeText::HasReplacementChar() const {
+ // Equivalent to:
+ // UnicodeText replacement_char;
+ // replacement_char.push_back(0xFFFD);
+ // return find(replacement_char) != end();
+ StringPiece searching(utf8_data(), utf8_length());
+ StringPiece looking_for("\xEF\xBF\xBD", 3);
+ return searching.find(looking_for) != StringPiece::npos;
+}
+
+// ----- other methods -----
+
+// Clear operator
+void UnicodeText::clear() {
+ repr_.clear();
+}
+
+// Destructor
+UnicodeText::~UnicodeText() {}
+
+
+void UnicodeText::push_back(char32 c) {
+ if (UniLib::IsValidCodepoint(c)) {
+ char buf[UTFmax];
+ int len = runetochar(buf, &c);
+ if (UniLib::IsInterchangeValid(buf, len)) {
+ repr_.append(buf, len);
+ } else {
+ LOG(WARNING) << "Unicode value 0x" << hex << c
+ << " is not valid for interchange";
+ repr_.append(" ", 1);
+ }
+ } else {
+ LOG(WARNING) << "Illegal Unicode value: 0x" << hex << c;
+ repr_.append(" ", 1);
+ }
+}
+
+int UnicodeText::size() const {
+ return CodepointCount(repr_.data_, repr_.size_);
+}
+
+bool operator==(const UnicodeText& lhs, const UnicodeText& rhs) {
+ if (&lhs == &rhs) return true;
+ if (lhs.repr_.size_ != rhs.repr_.size_) return false;
+ return memcmp(lhs.repr_.data_, rhs.repr_.data_, lhs.repr_.size_) == 0;
+}
+
+string UnicodeText::DebugString() const {
+ return StringPrintf("{UnicodeText %p chars=%d repr=%s}",
+ this,
+ size(),
+ repr_.DebugString().c_str());
+}
+
+
+// ******************* UnicodeText::const_iterator *********************
+
+// The implementation of const_iterator would be nicer if it
+// inherited from boost::iterator_facade
+// (http://boost.org/libs/iterator/doc/iterator_facade.html).
+
+UnicodeText::const_iterator::const_iterator() : it_(0) {}
+
+UnicodeText::const_iterator::const_iterator(const const_iterator& other)
+ : it_(other.it_) {
+}
+
+UnicodeText::const_iterator&
+UnicodeText::const_iterator::operator=(const const_iterator& other) {
+ if (&other != this)
+ it_ = other.it_;
+ return *this;
+}
+
+UnicodeText::const_iterator UnicodeText::begin() const {
+ return const_iterator(repr_.data_);
+}
+
+UnicodeText::const_iterator UnicodeText::end() const {
+ return const_iterator(repr_.data_ + repr_.size_);
+}
+
+bool operator<(const UnicodeText::const_iterator& lhs,
+ const UnicodeText::const_iterator& rhs) {
+ return lhs.it_ < rhs.it_;
+}
+
+char32 UnicodeText::const_iterator::operator*() const {
+ // (We could call chartorune here, but that does some
+ // error-checking, and we're guaranteed that our data is valid
+ // UTF-8. Also, we expect this routine to be called very often. So
+ // for speed, we do the calculation ourselves.)
+
+ // Convert from UTF-8
+ int byte1 = it_[0];
+ if (byte1 < 0x80)
+ return byte1;
+
+ int byte2 = it_[1];
+ if (byte1 < 0xE0)
+ return ((byte1 & 0x1F) << 6)
+ | (byte2 & 0x3F);
+
+ int byte3 = it_[2];
+ if (byte1 < 0xF0)
+ return ((byte1 & 0x0F) << 12)
+ | ((byte2 & 0x3F) << 6)
+ | (byte3 & 0x3F);
+
+ int byte4 = it_[3];
+ return ((byte1 & 0x07) << 18)
+ | ((byte2 & 0x3F) << 12)
+ | ((byte3 & 0x3F) << 6)
+ | (byte4 & 0x3F);
+}
+
+UnicodeText::const_iterator& UnicodeText::const_iterator::operator++() {
+ it_ += UniLib::OneCharLen(it_);
+ return *this;
+}
+
+UnicodeText::const_iterator& UnicodeText::const_iterator::operator--() {
+ while (UniLib::IsTrailByte(*--it_));
+ return *this;
+}
+
+int UnicodeText::const_iterator::get_utf8(char* utf8_output) const {
+ utf8_output[0] = it_[0]; if (it_[0] < 0x80) return 1;
+ utf8_output[1] = it_[1]; if (it_[0] < 0xE0) return 2;
+ utf8_output[2] = it_[2]; if (it_[0] < 0xF0) return 3;
+ utf8_output[3] = it_[3];
+ return 4;
+}
+
+
+UnicodeText::const_iterator UnicodeText::MakeIterator(const char* p) const {
+ CHECK(p != NULL);
+ const char* start = utf8_data();
+ int len = utf8_length();
+ const char* end = start + len;
+ CHECK(p >= start);
+ CHECK(p <= end);
+ CHECK(p == end || !UniLib::IsTrailByte(*p));
+ return const_iterator(p);
+}
+
+string UnicodeText::const_iterator::DebugString() const {
+ return StringPrintf("{iter %p}", it_);
+}
+
+
+// *************************** Utilities *************************
+
+string CodepointString(const UnicodeText& t) {
+ string s;
+ UnicodeText::const_iterator it = t.begin(), end = t.end();
+ while (it != end) StringAppendF(&s, "%X ", *it++);
+ return s;
+}
diff --git a/trunk/src/third_party/css_parser/src/util/utf8/internal/unilib.cc b/trunk/src/third_party/css_parser/src/util/utf8/internal/unilib.cc
new file mode 100644
index 0000000..5efa47b
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/util/utf8/internal/unilib.cc
@@ -0,0 +1,64 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "util/utf8/public/unilib.h"
+
+#include "base/basictypes.h"
+#include "third_party/utf/utf.h"
+
+namespace UniLib {
+
+namespace {
+
+// MOE: start_strip
+// MOE: end_strip
+// Codepoints not allowed for interchange are:
+// C0 (ASCII) controls: U+0000 to U+001F excluding Space (SP, U+0020),
+// Horizontal Tab (HT, U+0009), Line-Feed (LF, U+000A),
+// Form Feed (FF, U+000C) and Carriage-Return (CR, U+000D)
+// C1 controls: U+007F to U+009F
+// Surrogates: U+D800 to U+DFFF
+// Non-characters: U+FDD0 to U+FDEF and U+xxFFFE to U+xxFFFF for all xx
+inline bool IsInterchangeValidCodepoint(char32 c) {
+ return !((c >= 0x00 && c <= 0x08) || c == 0x0B || (c >= 0x0E && c <= 0x1F) ||
+ (c >= 0x7F && c <= 0x9F) ||
+ (c >= 0xD800 && c <= 0xDFFF) ||
+ (c >= 0xFDD0 && c <= 0xFDEF) || (c&0xFFFE) == 0xFFFE);
+}
+
+} // namespace
+
+int SpanInterchangeValid(const char* begin, int byte_length) {
+ char32 rune;
+ const char* p = begin;
+ const char* end = begin + byte_length;
+ while (p < end) {
+ int bytes_consumed = charntorune(&rune, p, end - p);
+ // We want to accept Runeerror == U+FFFD as a valid char, but it is used
+ // by chartorune to indicate error. Luckily, the real codepoint is size 3
+ // while errors return bytes_consumed == 1.
+ if ((rune == Runeerror && bytes_consumed == 1) ||
+ !IsInterchangeValidCodepoint(rune)) {
+ break; // Found
+ }
+ p += bytes_consumed;
+ }
+ return p - begin;
+}
+
+} // namespace UniLib
diff --git a/trunk/src/third_party/css_parser/src/util/utf8/public/config.h b/trunk/src/third_party/css_parser/src/util/utf8/public/config.h
new file mode 100644
index 0000000..9fb78b4
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/util/utf8/public/config.h
@@ -0,0 +1,24 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef UTIL_UTF8_PUBLIC_CONFIG_H_
+#define UTIL_UTF8_PUBLIC_CONFIG_H_
+
+using namespace std;
+
+#endif // UTIL_UTF8_PUBLIC_CONFIG_H_
diff --git a/trunk/src/third_party/css_parser/src/util/utf8/public/unicodetext.h b/trunk/src/third_party/css_parser/src/util/utf8/public/unicodetext.h
new file mode 100644
index 0000000..0741365
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/util/utf8/public/unicodetext.h
@@ -0,0 +1,462 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2006 Google Inc. All Rights Reserved.
+// Author: jrm@google.com (Jim Meehan)
+
+
+#ifndef UTIL_UTF8_UNICODETEXT_H__
+#define UTIL_UTF8_UNICODETEXT_H__
+
+#include <iterator>
+#include <string>
+#include <utility>
+#include "base/basictypes.h"
+#include "util/utf8/public/config.h"
+
+// ***************************** UnicodeText **************************
+//
+// A UnicodeText object is a container for a sequence of Unicode
+// codepoint values. It has default, copy, and assignment constructors.
+// Data can be appended to it from another UnicodeText, from
+// iterators, or from a single codepoint.
+//
+// The internal representation of the text is UTF-8. Since UTF-8 is a
+// variable-width format, UnicodeText does not provide random access
+// to the text, and changes to the text are permitted only at the end.
+//
+// The UnicodeText class defines a const_iterator. The dereferencing
+// operator (*) returns a codepoint (char32). The iterator is a
+// bidirectional, read-only iterator. It becomes invalid if the text
+// is changed.
+//
+// There are methods for appending and retrieving UTF-8 data directly.
+// The 'utf8_data' method returns a const char* that contains the
+// UTF-8-encoded version of the text; 'utf8_length' returns the number
+// of bytes in the UTF-8 data. An iterator's 'get' method stores up to
+// 4 bytes of UTF-8 data in a char array and returns the number of
+// bytes that it stored.
+//
+// Codepoints are integers in the range [0, 0xD7FF] or [0xE000,
+// 0x10FFFF], but UnicodeText has the additional restriction that it
+// can contain only those characters that are valid for interchange on
+// the Web. This excludes all of the control codes except for carriage
+// return, line feed, and horizontal tab. It also excludes
+// non-characters, but codepoints that are in the Private Use regions
+// are allowed, as are codepoints that are unassigned. (See the
+// Unicode reference for details.) The function UniLib::IsInterchangeValid
+// can be used as a test for this property.
+//
+// UnicodeTexts are safe. Every method that constructs or modifies a
+// UnicodeText tests for interchange-validity, and will substitute a
+// space for the invalid data. Such cases are reported via
+// LOG(WARNING).
+//
+// MEMORY MANAGEMENT: copy, take ownership, or point to
+//
+// A UnicodeText is either an "owner", meaning that it owns the memory
+// for the data buffer and will free it when the UnicodeText is
+// destroyed, or it is an "alias", meaning that it does not.
+//
+// There are three methods for storing UTF-8 data in a UnicodeText:
+//
+// CopyUTF8(buffer, len) copies buffer.
+//
+// TakeOwnershipOfUTF8(buffer, size, capacity) takes ownership of buffer.
+//
+// PointToUTF8(buffer, size) creates an alias pointing to buffer.
+//
+// All three methods perform a validity check on the buffer. There are
+// private, "unsafe" versions of these functions that bypass the
+// validity check. They are used internally and by friend-functions
+// that are handling UTF-8 data that has already been validated.
+//
+// The purpose of an alias is to avoid making an unnecessary copy of a
+// UTF-8 buffer while still providing access to the Unicode values
+// within that text through iterators or the fast scanners that are
+// based on UTF-8 state tables. The lifetime of an alias must not
+// exceed the lifetime of the buffer from which it was constructed.
+//
+// The semantics of an alias might be described as "copy on write or
+// repair." The source data is never modified. If push_back() or
+// append() is called on an alias, a copy of the data will be created,
+// and the UnicodeText will become an owner. If clear() is called on
+// an alias, it becomes an (empty) owner.
+//
+// The copy constructor and the assignment operator produce an owner.
+// That is, after direct initialization ("UnicodeText x(y);") or copy
+// initialization ("UnicodeText x = y;") x will be an owner, even if y
+// was an alias. The assignment operator ("x = y;") also produces an
+// owner unless x and y are the same object and y is an alias.
+//
+// Aliases should be used with care. If the source from which an alias
+// was created is freed, or if the contents are changed, while the
+// alias is still in use, fatal errors could result. But it can be
+// quite useful to have a UnicodeText "window" through which to see a
+// UTF-8 buffer without having to pay the price of making a copy.
+//
+// UTILITIES
+//
+// The interfaces in util/utf8/public/textutils.h provide higher-level
+// utilities for dealing with UnicodeTexts, including routines for
+// creating UnicodeTexts (both owners and aliases) from UTF-8 buffers or
+// strings, creating strings from UnicodeTexts, normalizing text for
+// efficient matching or display, and others.
+
+class UnicodeText {
+ public:
+ class const_iterator;
+
+ typedef char32 value_type;
+
+ // Constructors. These always produce owners.
+ UnicodeText(); // Create an empty text.
+ UnicodeText(const UnicodeText& src); // copy constructor
+ // Construct a substring (copies the data).
+ UnicodeText(const const_iterator& first, const const_iterator& last);
+
+ // Assignment operator. This copies the data and produces an owner
+ // unless this == &src, e.g., "x = x;", which is a no-op.
+ UnicodeText& operator=(const UnicodeText& src);
+
+ // x.Copy(y) copies the data from y into x.
+ UnicodeText& Copy(const UnicodeText& src);
+ inline UnicodeText& assign(const UnicodeText& src) { return Copy(src); }
+
+ // x.PointTo(y) changes x so that it points to y's data.
+ // It does not copy y or take ownership of y's data.
+ UnicodeText& PointTo(const UnicodeText& src);
+ UnicodeText& PointTo(const const_iterator& first,
+ const const_iterator& last);
+
+ ~UnicodeText();
+
+ void clear(); // Clear text.
+ bool empty() { return repr_.size_ == 0; } // Test if text is empty.
+
+ // Add a codepoint to the end of the text.
+ // If the codepoint is not interchange-valid, add a space instead
+ // and log a warning.
+ void push_back(char32 codepoint);
+
+ // Generic appending operation.
+ // iterator_traits<ForwardIterator>::value_type must be implicitly
+ // convertible to char32. Typical uses of this method might include:
+ // char32 chars[] = {0x1, 0x2, ...};
+ // vector<char32> more_chars = ...;
+ // utext.append(chars, chars+arraysize(chars));
+ // utext.append(more_chars.begin(), more_chars.end());
+ template<typename ForwardIterator>
+ UnicodeText& append(ForwardIterator first, const ForwardIterator last) {
+ while (first != last) { push_back(*first++); }
+ return *this;
+ }
+
+ // A specialization of the generic append() method.
+ UnicodeText& append(const const_iterator& first, const const_iterator& last);
+
+ // An optimization of append(source.begin(), source.end()).
+ UnicodeText& append(const UnicodeText& source);
+
+ int size() const; // the number of Unicode characters (codepoints)
+
+ friend bool operator==(const UnicodeText& lhs, const UnicodeText& rhs);
+ friend bool operator!=(const UnicodeText& lhs, const UnicodeText& rhs);
+
+ class const_iterator {
+ typedef const_iterator CI;
+ public:
+ typedef bidirectional_iterator_tag iterator_category;
+ typedef char32 value_type;
+ typedef ptrdiff_t difference_type;
+ typedef void pointer; // (Not needed.)
+ typedef const char32 reference; // (Needed for const_reverse_iterator)
+
+ // Iterators are default-constructible.
+ const_iterator();
+
+ // It's safe to make multiple passes over a UnicodeText.
+ const_iterator(const const_iterator& other);
+ const_iterator& operator=(const const_iterator& other);
+
+ char32 operator*() const; // Dereference
+
+ const_iterator& operator++(); // Advance (++iter)
+ const_iterator operator++(int) { // (iter++)
+ const_iterator result(*this);
+ ++*this;
+ return result;
+ }
+
+ const_iterator& operator--(); // Retreat (--iter)
+ const_iterator operator--(int) { // (iter--)
+ const_iterator result(*this);
+ --*this;
+ return result;
+ }
+
+ // We love relational operators.
+ friend bool operator==(const CI& lhs, const CI& rhs) {
+ return lhs.it_ == rhs.it_; }
+ friend bool operator!=(const CI& lhs, const CI& rhs) {
+ return !(lhs == rhs); }
+ friend bool operator<(const CI& lhs, const CI& rhs);
+ friend bool operator>(const CI& lhs, const CI& rhs) {
+ return rhs < lhs; }
+ friend bool operator<=(const CI& lhs, const CI& rhs) {
+ return !(rhs < lhs); }
+ friend bool operator>=(const CI& lhs, const CI& rhs) {
+ return !(lhs < rhs); }
+
+ friend difference_type distance(const CI& first, const CI& last);
+
+ // UTF-8-specific methods
+ // Store the UTF-8 encoding of the current codepoint into buf,
+ // which must be at least 4 bytes long. Return the number of
+ // bytes written.
+ int get_utf8(char* buf) const;
+ // Return the iterator's pointer into the UTF-8 data.
+ const char* utf8_data() const { return it_; }
+
+ string DebugString() const;
+
+ private:
+ friend class UnicodeText;
+ friend class UnicodeTextUtils;
+ friend class UTF8StateTableProperty;
+ explicit const_iterator(const char* it) : it_(it) {}
+
+ const char* it_;
+ };
+
+ const_iterator begin() const;
+ const_iterator end() const;
+
+ class const_reverse_iterator : public std::reverse_iterator<const_iterator> {
+ public:
+ const_reverse_iterator(const_iterator it) :
+ std::reverse_iterator<const_iterator>(it) {}
+ const char* utf8_data() const {
+ const_iterator tmp_it = base();
+ return (--tmp_it).utf8_data();
+ }
+ int get_utf8(char* buf) const {
+ const_iterator tmp_it = base();
+ return (--tmp_it).get_utf8(buf);
+ }
+ };
+ const_reverse_iterator rbegin() const {
+ return const_reverse_iterator(end());
+ }
+ const_reverse_iterator rend() const {
+ return const_reverse_iterator(begin());
+ }
+
+ // Substring searching. Returns the beginning of the first
+ // occurrence of "look", or end() if not found.
+ const_iterator find(const UnicodeText& look, const_iterator start_pos) const;
+ // Equivalent to find(look, begin())
+ const_iterator find(const UnicodeText& look) const;
+
+ // Returns whether this contains the character U+FFFD. This can
+ // occur, for example, if the input to Encodings::Decode() had byte
+ // sequences that were invalid in the source encoding.
+ bool HasReplacementChar() const;
+
+ // UTF-8-specific methods
+ //
+ // Return the data, length, and capacity of UTF-8-encoded version of
+ // the text. Length and capacity are measured in bytes.
+ const char* utf8_data() const { return repr_.data_; }
+ int utf8_length() const { return repr_.size_; }
+ int utf8_capacity() const { return repr_.capacity_; }
+
+ // Return the UTF-8 data as a string.
+ static string UTF8Substring(const const_iterator& first,
+ const const_iterator& last);
+
+ // There are three methods for initializing a UnicodeText from UTF-8
+ // data. They vary in details of memory management. In all cases,
+ // the data is tested for interchange-validity. If it is not
+ // interchange-valid, a LOG(WARNING) is issued, and each
+ // structurally invalid byte and each interchange-invalid codepoint
+ // is replaced with a space.
+
+ // x.CopyUTF8(buf, len) copies buf into x.
+ UnicodeText& CopyUTF8(const char* utf8_buffer, int byte_length);
+
+ // x.TakeOwnershipOfUTF8(buf, len, capacity). x takes ownership of
+ // buf. buf is not copied.
+ UnicodeText& TakeOwnershipOfUTF8(char* utf8_buffer,
+ int byte_length,
+ int byte_capacity);
+
+ // x.PointToUTF8(buf,len) changes x so that it points to buf
+ // ("becomes an alias"). It does not take ownership or copy buf.
+ // If the buffer is not valid, this has the same effect as
+ // CopyUTF8(utf8_buffer, byte_length).
+ UnicodeText& PointToUTF8(const char* utf8_buffer, int byte_length);
+
+ // Occasionally it is necessary to use functions that operate on the
+ // pointer returned by utf8_data(). MakeIterator(p) provides a way
+ // to get back to the UnicodeText level. It uses CHECK to ensure
+ // that p is a pointer within this object's UTF-8 data, and that it
+ // points to the beginning of a character.
+ const_iterator MakeIterator(const char* p) const;
+
+ string DebugString() const;
+
+ private:
+ friend class const_iterator;
+ friend class UnicodeTextUtils;
+
+ class Repr { // A byte-string.
+ public:
+ char* data_;
+ int size_;
+ int capacity_;
+ bool ours_; // Do we own data_?
+
+ Repr() : data_(NULL), size_(0), capacity_(0), ours_(true) {}
+ ~Repr() { if (ours_) delete[] data_; }
+
+ void clear();
+ void reserve(int capacity);
+ void resize(int size);
+
+ void append(const char* bytes, int byte_length);
+ void Copy(const char* data, int size);
+ void TakeOwnershipOf(char* data, int size, int capacity);
+ void PointTo(const char* data, int size);
+
+ string DebugString() const;
+
+ private:
+ Repr& operator=(const Repr&);
+ Repr(const Repr& other);
+ };
+
+ Repr repr_;
+
+ // UTF-8-specific private methods.
+ // These routines do not perform a validity check when compiled
+ // in opt mode.
+ // It is an error to call these methods with UTF-8 data that
+ // is not interchange-valid.
+ //
+ UnicodeText& UnsafeCopyUTF8(const char* utf8_buffer, int byte_length);
+ UnicodeText& UnsafeTakeOwnershipOfUTF8(
+ char* utf8_buffer, int byte_length, int byte_capacity);
+ UnicodeText& UnsafePointToUTF8(const char* utf8_buffer, int byte_length);
+ UnicodeText& UnsafeAppendUTF8(const char* utf8_buffer, int byte_length);
+ const_iterator UnsafeFind(const UnicodeText& look,
+ const_iterator start_pos) const;
+};
+
+bool operator==(const UnicodeText& lhs, const UnicodeText& rhs);
+
+inline bool operator!=(const UnicodeText& lhs, const UnicodeText& rhs) {
+ return !(lhs == rhs);
+}
+
+// UnicodeTextRange is a pair of iterators, useful for specifying text
+// segments. If the iterators are ==, the segment is empty.
+typedef pair<UnicodeText::const_iterator,
+ UnicodeText::const_iterator> UnicodeTextRange;
+
+inline bool UnicodeTextRangeIsEmpty(const UnicodeTextRange& r) {
+ return r.first == r.second;
+}
+
+
+// *************************** Utilities *************************
+
+// A factory function for creating a UnicodeText from a buffer of
+// UTF-8 data. The new UnicodeText takes ownership of the buffer. (It
+// is an "owner.")
+//
+// Each byte that is structurally invalid will be replaced with a
+// space. Each codepoint that is interchange-invalid will also be
+// replaced with a space, even if the codepoint was represented with a
+// multibyte sequence in the UTF-8 data.
+//
+inline UnicodeText MakeUnicodeTextAcceptingOwnership(
+ char* utf8_buffer, int byte_length, int byte_capacity) {
+ return UnicodeText().TakeOwnershipOfUTF8(
+ utf8_buffer, byte_length, byte_capacity);
+}
+
+// A factory function for creating a UnicodeText from a buffer of
+// UTF-8 data. The new UnicodeText does not take ownership of the
+// buffer. (It is an "alias.")
+//
+inline UnicodeText MakeUnicodeTextWithoutAcceptingOwnership(
+ const char* utf8_buffer, int byte_length) {
+ return UnicodeText().PointToUTF8(utf8_buffer, byte_length);
+}
+
+// Create a UnicodeText from a UTF-8 string or buffer.
+//
+// If do_copy is true, then a copy of the string is made. The copy is
+// owned by the resulting UnicodeText object and will be freed when
+// the object is destroyed. This UnicodeText object is referred to
+// as an "owner."
+//
+// If do_copy is false, then no copy is made. The resulting
+// UnicodeText object does NOT take ownership of the string; in this
+// case, the lifetime of the UnicodeText object must not exceed the
+// lifetime of the string. This Unicodetext object is referred to as
+// an "alias." This is the same as MakeUnicodeTextWithoutAcceptingOwnership.
+//
+// If the input string does not contain valid UTF-8, then a copy is
+// made (as if do_copy were true) and coerced to valid UTF-8 by
+// replacing each invalid byte with a space.
+//
+inline UnicodeText UTF8ToUnicodeText(const char* utf8_buf, int len,
+ bool do_copy) {
+ UnicodeText t;
+ if (do_copy) {
+ t.CopyUTF8(utf8_buf, len);
+ } else {
+ t.PointToUTF8(utf8_buf, len);
+ }
+ return t;
+}
+
+inline UnicodeText UTF8ToUnicodeText(const string& utf_string, bool do_copy) {
+ return UTF8ToUnicodeText(utf_string.data(), utf_string.size(), do_copy);
+}
+
+inline UnicodeText UTF8ToUnicodeText(const char* utf8_buf, int len) {
+ return UTF8ToUnicodeText(utf8_buf, len, true);
+}
+inline UnicodeText UTF8ToUnicodeText(const string& utf8_string) {
+ return UTF8ToUnicodeText(utf8_string, true);
+}
+
+// Return a string containing the UTF-8 encoded version of all the
+// Unicode characters in t.
+inline string UnicodeTextToUTF8(const UnicodeText& t) {
+ return string(t.utf8_data(), t.utf8_length());
+}
+
+
+// For debugging. Return a string of integers, written in uppercase
+// hex (%X), corresponding to the codepoints within the text. Each
+// integer is followed by a space. E.g., "61 62 6A 3005 ".
+string CodepointString(const UnicodeText& t);
+
+#endif // UTIL_UTF8_UNICODETEXT_H__
diff --git a/trunk/src/third_party/css_parser/src/util/utf8/public/unilib.h b/trunk/src/third_party/css_parser/src/util/utf8/public/unilib.h
new file mode 100644
index 0000000..4cfc787
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/util/utf8/public/unilib.h
@@ -0,0 +1,95 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Routines to do manipulation of Unicode characters or text
+//
+// The StructurallyValid routines accept buffers of arbitrary bytes.
+// For CoerceToStructurallyValid(), the input buffer and output buffers may
+// point to exactly the same memory.
+//
+// In all other cases, the UTF-8 string must be structurally valid and
+// have all codepoints in the range U+0000 to U+D7FF or U+E000 to U+10FFFF.
+// Debug builds take a fatal error for invalid UTF-8 input.
+// The input and output buffers may not overlap at all.
+//
+// The char32 routines are here only for convenience; they convert to UTF-8
+// internally and use the UTF-8 routines.
+
+#ifndef UTIL_UTF8_UNILIB_H__
+#define UTIL_UTF8_UNILIB_H__
+
+#include <string>
+#include "base/basictypes.h"
+
+namespace UniLib {
+
+// Returns true unless a surrogate code point
+inline bool IsValidCodepoint(char32 c) {
+ // In the range [0, 0xD800) or [0xE000, 0x10FFFF]
+ return (static_cast<uint32>(c) < 0xD800)
+ || (c >= 0xE000 && c <= 0x10FFFF);
+}
+
+// Table of UTF-8 character lengths, based on first byte
+static const unsigned char kUTF8LenTbl[256] = {
+ 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1,
+ 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1,
+ 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1,
+ 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1,
+
+ 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1,
+ 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1,
+ 2,2,2,2,2,2,2,2, 2,2,2,2,2,2,2,2, 2,2,2,2,2,2,2,2, 2,2,2,2,2,2,2,2,
+ 3,3,3,3,3,3,3,3, 3,3,3,3,3,3,3,3, 4,4,4,4,4,4,4,4, 4,4,4,4,4,4,4,4
+};
+
+// Return length of a single UTF-8 source character
+inline int OneCharLen(const char* src) {
+ return kUTF8LenTbl[*reinterpret_cast<const uint8*>(src)];
+}
+
+// Return length of a single UTF-8 source character
+inline int OneCharLen(const uint8* src) {
+ return kUTF8LenTbl[*src];
+}
+
+// Return true if this byte is a trailing UTF-8 byte (10xx xxxx)
+inline bool IsTrailByte(char x) {
+ // return (x & 0xC0) == 0x80;
+ // Since trail bytes are always in [0x80, 0xBF], we can optimize:
+ return static_cast<signed char>(x) < -0x40;
+}
+
+// Returns the length in bytes of the prefix of src that is all
+// interchange valid UTF-8
+int SpanInterchangeValid(const char* src, int byte_length);
+inline int SpanInterchangeValid(const std::string& src) {
+ return SpanInterchangeValid(src.data(), src.size());
+}
+
+// Returns true if the source is all interchange valid UTF-8
+// "Interchange valid" is a stronger than structurally valid --
+// no C0 or C1 control codes (other than CR LF HT FF) and no non-characters.
+inline bool IsInterchangeValid(const char* src, int byte_length) {
+ return (byte_length == SpanInterchangeValid(src, byte_length));
+}
+inline bool IsInterchangeValid(const std::string& src) {
+ return IsInterchangeValid(src.data(), src.size());
+}
+
+} // namespace UniLib
+
+#endif // UTIL_UTF8_PUBLIC_UNILIB_H_
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/gtest_main.cc b/trunk/src/third_party/css_parser/src/webutil/css/gtest_main.cc
new file mode 100644
index 0000000..1d97136
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/gtest_main.cc
@@ -0,0 +1,26 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+//
+// Build all tests with this main to run all tests.
+
+#include "gtest-1.5.0/install/include/gtest/gtest.h"
+
+int main(int argc, char **argv) {
+ ::testing::InitGoogleTest(&argc, argv);
+ return RUN_ALL_TESTS();
+}
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/identifier.cc b/trunk/src/third_party/css_parser/src/webutil/css/identifier.cc
new file mode 100644
index 0000000..eaad0e5
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/identifier.cc
@@ -0,0 +1,985 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/* C++ code produced by gperf version 3.0.3 */
+/* Computed positions: -k'1-3,10,$' */
+
+#if !((' ' == 32) && ('!' == 33) && ('"' == 34) && ('#' == 35) \
+ && ('%' == 37) && ('&' == 38) && ('\'' == 39) && ('(' == 40) \
+ && (')' == 41) && ('*' == 42) && ('+' == 43) && (',' == 44) \
+ && ('-' == 45) && ('.' == 46) && ('/' == 47) && ('0' == 48) \
+ && ('1' == 49) && ('2' == 50) && ('3' == 51) && ('4' == 52) \
+ && ('5' == 53) && ('6' == 54) && ('7' == 55) && ('8' == 56) \
+ && ('9' == 57) && (':' == 58) && (';' == 59) && ('<' == 60) \
+ && ('=' == 61) && ('>' == 62) && ('?' == 63) && ('A' == 65) \
+ && ('B' == 66) && ('C' == 67) && ('D' == 68) && ('E' == 69) \
+ && ('F' == 70) && ('G' == 71) && ('H' == 72) && ('I' == 73) \
+ && ('J' == 74) && ('K' == 75) && ('L' == 76) && ('M' == 77) \
+ && ('N' == 78) && ('O' == 79) && ('P' == 80) && ('Q' == 81) \
+ && ('R' == 82) && ('S' == 83) && ('T' == 84) && ('U' == 85) \
+ && ('V' == 86) && ('W' == 87) && ('X' == 88) && ('Y' == 89) \
+ && ('Z' == 90) && ('[' == 91) && ('\\' == 92) && (']' == 93) \
+ && ('^' == 94) && ('_' == 95) && ('a' == 97) && ('b' == 98) \
+ && ('c' == 99) && ('d' == 100) && ('e' == 101) && ('f' == 102) \
+ && ('g' == 103) && ('h' == 104) && ('i' == 105) && ('j' == 106) \
+ && ('k' == 107) && ('l' == 108) && ('m' == 109) && ('n' == 110) \
+ && ('o' == 111) && ('p' == 112) && ('q' == 113) && ('r' == 114) \
+ && ('s' == 115) && ('t' == 116) && ('u' == 117) && ('v' == 118) \
+ && ('w' == 119) && ('x' == 120) && ('y' == 121) && ('z' == 122) \
+ && ('{' == 123) && ('|' == 124) && ('}' == 125) && ('~' == 126))
+/* The character set is not based on ISO-646. */
+#error "gperf generated tables don't work with this execution character set. Please report a bug to <bug-gnu-gperf@gnu.org>."
+#endif
+
+#line 1 "webutil/css/identifier.gperf"
+
+#include "webutil/css/identifier.h"
+
+#include "base/googleinit.h"
+#include "base/logging.h"
+#include "webutil/css/string_util.h"
+
+namespace Css {
+#line 11 "webutil/css/identifier.gperf"
+struct idents {
+ const char *name;
+ Identifier::Ident id;
+};
+enum
+ {
+ TOTAL_KEYWORDS = 144,
+ MIN_WORD_LENGTH = 3,
+ MAX_WORD_LENGTH = 24,
+ MIN_HASH_VALUE = 5,
+ MAX_HASH_VALUE = 401
+ };
+
+/* maximum key range = 397, duplicates = 0 */
+
+#ifndef GPERF_DOWNCASE
+#define GPERF_DOWNCASE 1
+static unsigned char gperf_downcase[256] =
+ {
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
+ 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29,
+ 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,
+ 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59,
+ 60, 61, 62, 63, 64, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106,
+ 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121,
+ 122, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104,
+ 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119,
+ 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134,
+ 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149,
+ 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164,
+ 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179,
+ 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194,
+ 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209,
+ 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224,
+ 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239,
+ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254,
+ 255
+ };
+#endif
+
+#ifndef GPERF_CASE_STRNCMP
+#define GPERF_CASE_STRNCMP 1
+static int
+gperf_case_strncmp (register const char *s1, register const char *s2, register unsigned int n)
+{
+ for (; n > 0;)
+ {
+ unsigned char c1 = gperf_downcase[(unsigned char)*s1++];
+ unsigned char c2 = gperf_downcase[(unsigned char)*s2++];
+ if (c1 != 0 && c1 == c2)
+ {
+ n--;
+ continue;
+ }
+ return (int)c1 - (int)c2;
+ }
+ return 0;
+}
+#endif
+
+class IdentifierMapper
+{
+private:
+ static inline unsigned int hash (const char *str, unsigned int len);
+public:
+ static const struct idents *in_word_set (const char *str, unsigned int len);
+};
+
+inline unsigned int
+IdentifierMapper::hash (register const char *str, register unsigned int len)
+{
+ static const unsigned short asso_values[] =
+ {
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 60, 125, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 30, 0, 50, 5, 0,
+ 25, 30, 145, 65, 110, 50, 10, 140, 45, 75,
+ 15, 155, 20, 5, 0, 45, 110, 20, 75, 95,
+ 402, 402, 402, 402, 402, 402, 402, 30, 0, 50,
+ 5, 0, 25, 30, 145, 65, 110, 50, 10, 140,
+ 45, 75, 15, 155, 20, 5, 0, 45, 110, 20,
+ 75, 95, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402, 402, 402, 402,
+ 402, 402, 402, 402, 402, 402, 402
+ };
+ register int hval = len;
+
+ switch (hval)
+ {
+ default:
+ hval += asso_values[(unsigned char)str[9]];
+ /*FALLTHROUGH*/
+ case 9:
+ case 8:
+ case 7:
+ case 6:
+ case 5:
+ case 4:
+ case 3:
+ hval += asso_values[(unsigned char)str[2]];
+ /*FALLTHROUGH*/
+ case 2:
+ hval += asso_values[(unsigned char)str[1]+1];
+ /*FALLTHROUGH*/
+ case 1:
+ hval += asso_values[(unsigned char)str[0]];
+ break;
+ }
+ return hval + asso_values[(unsigned char)str[len - 1]];
+}
+
+static const struct idents wordlist[] =
+ {
+#line 78 "webutil/css/identifier.gperf"
+ {"table", Identifier::TABLE},
+#line 145 "webutil/css/identifier.gperf"
+ {"baseline", Identifier::BASELINE},
+#line 42 "webutil/css/identifier.gperf"
+ {"dashed", Identifier::DASHED},
+#line 151 "webutil/css/identifier.gperf"
+ {"pre", Identifier::PRE},
+#line 154 "webutil/css/identifier.gperf"
+ {"pre-line", Identifier::PRE_LINE},
+#line 83 "webutil/css/identifier.gperf"
+ {"table-row", Identifier::TABLE_ROW},
+#line 86 "webutil/css/identifier.gperf"
+ {"table-cell", Identifier::TABLE_CELL},
+#line 41 "webutil/css/identifier.gperf"
+ {"dotted", Identifier::DOTTED},
+#line 82 "webutil/css/identifier.gperf"
+ {"table-footer-group", Identifier::TABLE_FOOTER_GROUP},
+#line 106 "webutil/css/identifier.gperf"
+ {"bold", Identifier::BOLD},
+#line 98 "webutil/css/identifier.gperf"
+ {"large", Identifier::LARGE},
+#line 81 "webutil/css/identifier.gperf"
+ {"table-header-group", Identifier::TABLE_HEADER_GROUP},
+#line 43 "webutil/css/identifier.gperf"
+ {"solid", Identifier::SOLID},
+#line 153 "webutil/css/identifier.gperf"
+ {"pre-wrap", Identifier::PRE_WRAP},
+#line 34 "webutil/css/identifier.gperf"
+ {"scroll", Identifier::SCROLL},
+#line 31 "webutil/css/identifier.gperf"
+ {"top", Identifier::TOP},
+#line 107 "webutil/css/identifier.gperf"
+ {"bolder", Identifier::BOLDER},
+#line 40 "webutil/css/identifier.gperf"
+ {"separate", Identifier::SEPARATE},
+#line 142 "webutil/css/identifier.gperf"
+ {"lowercase", Identifier::LOWERCASE},
+#line 143 "webutil/css/identifier.gperf"
+ {"embed", Identifier::EMBED},
+#line 102 "webutil/css/identifier.gperf"
+ {"larger", Identifier::LARGER},
+#line 87 "webutil/css/identifier.gperf"
+ {"table-caption", Identifier::TABLE_CAPTION},
+#line 56 "webutil/css/identifier.gperf"
+ {"default", Identifier::DEFAULT},
+#line 133 "webutil/css/identifier.gperf"
+ {"relative", Identifier::RELATIVE},
+#line 28 "webutil/css/identifier.gperf"
+ {"left", Identifier::LEFT},
+#line 26 "webutil/css/identifier.gperf"
+ {"repeat", Identifier::REPEAT},
+#line 44 "webutil/css/identifier.gperf"
+ {"double", Identifier::DOUBLE},
+#line 140 "webutil/css/identifier.gperf"
+ {"capitalize", Identifier::CAPITALIZE},
+#line 119 "webutil/css/identifier.gperf"
+ {"square", Identifier::SQUARE},
+#line 84 "webutil/css/identifier.gperf"
+ {"table-column-group", Identifier::TABLE_COLUMN_GROUP},
+#line 90 "webutil/css/identifier.gperf"
+ {"serif", Identifier::SERIF},
+#line 27 "webutil/css/identifier.gperf"
+ {"collapse", Identifier::COLLAPSE},
+#line 72 "webutil/css/identifier.gperf"
+ {"rtl", Identifier::RTL,},
+#line 68 "webutil/css/identifier.gperf"
+ {"wait", Identifier::WAIT},
+#line 80 "webutil/css/identifier.gperf"
+ {"table-row-group", Identifier::TABLE_ROW_GROUP},
+#line 36 "webutil/css/identifier.gperf"
+ {"transparent", Identifier::TRANSPARENT},
+#line 134 "webutil/css/identifier.gperf"
+ {"absolute", Identifier::ABSOLUTE},
+#line 97 "webutil/css/identifier.gperf"
+ {"small", Identifier::SMALL},
+#line 20 "webutil/css/identifier.gperf"
+ {"normal", Identifier::NORMAL},
+#line 120 "webutil/css/identifier.gperf"
+ {"decimal", Identifier::DECIMAL},
+#line 71 "webutil/css/identifier.gperf"
+ {"ltr", Identifier::LTR,},
+#line 63 "webutil/css/identifier.gperf"
+ {"se-resize", Identifier::SE_RESIZE},
+#line 105 "webutil/css/identifier.gperf"
+ {"small-caps", Identifier::SMALL_CAPS},
+#line 152 "webutil/css/identifier.gperf"
+ {"nowrap", Identifier::NOWRAP},
+#line 85 "webutil/css/identifier.gperf"
+ {"table-column", Identifier::TABLE_COLUMN},
+#line 137 "webutil/css/identifier.gperf"
+ {"overline", Identifier::OVERLINE},
+#line 67 "webutil/css/identifier.gperf"
+ {"text", Identifier::TEXT},
+#line 124 "webutil/css/identifier.gperf"
+ {"lower-greek", Identifier::LOWER_GREEK},
+#line 101 "webutil/css/identifier.gperf"
+ {"smaller", Identifier::SMALLER},
+#line 70 "webutil/css/identifier.gperf"
+ {"progress", Identifier::PROGRESS},
+#line 18 "webutil/css/identifier.gperf"
+ {"none", Identifier::NONE},
+#line 91 "webutil/css/identifier.gperf"
+ {"sans-serif", Identifier::SANS_SERIF},
+#line 45 "webutil/css/identifier.gperf"
+ {"groove", Identifier::GROOVE},
+#line 109 "webutil/css/identifier.gperf"
+ {"caption", Identifier::CAPTION},
+#line 146 "webutil/css/identifier.gperf"
+ {"sub", Identifier::SUB},
+#line 57 "webutil/css/identifier.gperf"
+ {"pointer", Identifier::POINTER},
+#line 148 "webutil/css/identifier.gperf"
+ {"text-top", Identifier::TEXT_TOP},
+#line 39 "webutil/css/identifier.gperf"
+ {"no-repeat", Identifier::NO_REPEAT},
+#line 114 "webutil/css/identifier.gperf"
+ {"status-bar", Identifier::STATUS_BAR},
+#line 122 "webutil/css/identifier.gperf"
+ {"lower-roman", Identifier::LOWER_ROMAN},
+#line 136 "webutil/css/identifier.gperf"
+ {"underline", Identifier::UNDERLINE},
+#line 24 "webutil/css/identifier.gperf"
+ {"avoid", Identifier::AVOID},
+#line 132 "webutil/css/identifier.gperf"
+ {"static", Identifier::STATIC},
+#line 113 "webutil/css/identifier.gperf"
+ {"small-caption", Identifier::SMALL_CAPTION},
+#line 60 "webutil/css/identifier.gperf"
+ {"ne-resize", Identifier::NE_RESIZE},
+#line 46 "webutil/css/identifier.gperf"
+ {"ridge", Identifier::RIDGE},
+#line 104 "webutil/css/identifier.gperf"
+ {"oblique", Identifier::OBLIQUE},
+#line 37 "webutil/css/identifier.gperf"
+ {"repeat-x", Identifier::REPEAT_X},
+#line 29 "webutil/css/identifier.gperf"
+ {"center", Identifier::CENTER},
+#line 144 "webutil/css/identifier.gperf"
+ {"bidi-override", Identifier::BIDI_OVERRIDE},
+#line 64 "webutil/css/identifier.gperf"
+ {"sw-resize", Identifier::SW_RESIZE},
+#line 47 "webutil/css/identifier.gperf"
+ {"inset", Identifier::INSET},
+#line 115 "webutil/css/identifier.gperf"
+ {"inside", Identifier::INSIDE},
+#line 59 "webutil/css/identifier.gperf"
+ {"e-resize", Identifier::E_RESIZE},
+#line 147 "webutil/css/identifier.gperf"
+ {"super", Identifier::SUPER},
+#line 73 "webutil/css/identifier.gperf"
+ {"inline", Identifier::INLINE},
+#line 65 "webutil/css/identifier.gperf"
+ {"s-resize", Identifier::S_RESIZE},
+#line 55 "webutil/css/identifier.gperf"
+ {"crosshair", Identifier::CROSSHAIR},
+#line 32 "webutil/css/identifier.gperf"
+ {"bottom", Identifier::BOTTOM},
+#line 79 "webutil/css/identifier.gperf"
+ {"inline-table", Identifier::INLINE_TABLE},
+#line 38 "webutil/css/identifier.gperf"
+ {"repeat-y", Identifier::REPEAT_Y},
+#line 33 "webutil/css/identifier.gperf"
+ {"both", Identifier::BOTH},
+#line 30 "webutil/css/identifier.gperf"
+ {"right", Identifier::RIGHT},
+#line 125 "webutil/css/identifier.gperf"
+ {"lower-latin", Identifier::LOWER_LATIN,},
+#line 88 "webutil/css/identifier.gperf"
+ {"show", Identifier::SHOW},
+#line 93 "webutil/css/identifier.gperf"
+ {"fantasy", Identifier::FANTASY},
+#line 66 "webutil/css/identifier.gperf"
+ {"w-resize", Identifier::W_RESIZE},
+#line 117 "webutil/css/identifier.gperf"
+ {"disc", Identifier::DISC},
+#line 121 "webutil/css/identifier.gperf"
+ {"decimal-leading-zero", Identifier::DECIMAL_LEADING_ZERO},
+#line 108 "webutil/css/identifier.gperf"
+ {"lighter", Identifier::LIGHTER},
+#line 53 "webutil/css/identifier.gperf"
+ {"no-open-quote", Identifier::NO_OPEN_QUOTE},
+#line 49 "webutil/css/identifier.gperf"
+ {"thin", Identifier::THIN},
+#line 128 "webutil/css/identifier.gperf"
+ {"georgian", Identifier::GEORGIAN},
+#line 50 "webutil/css/identifier.gperf"
+ {"thick", Identifier::THICK},
+#line 118 "webutil/css/identifier.gperf"
+ {"circle", Identifier::CIRCLE},
+#line 92 "webutil/css/identifier.gperf"
+ {"cursive", Identifier::CURSIVE},
+#line 61 "webutil/css/identifier.gperf"
+ {"nw-resize", Identifier::NW_RESIZE},
+#line 48 "webutil/css/identifier.gperf"
+ {"outset", Identifier::OUTSET},
+#line 116 "webutil/css/identifier.gperf"
+ {"outside", Identifier::OUTSIDE},
+#line 110 "webutil/css/identifier.gperf"
+ {"icon", Identifier::ICON},
+#line 103 "webutil/css/identifier.gperf"
+ {"italic", Identifier::ITALIC},
+#line 62 "webutil/css/identifier.gperf"
+ {"n-resize", Identifier::N_RESIZE},
+#line 69 "webutil/css/identifier.gperf"
+ {"help", Identifier::HELP},
+#line 23 "webutil/css/identifier.gperf"
+ {"always", Identifier::ALWAYS},
+#line 94 "webutil/css/identifier.gperf"
+ {"monospace", Identifier::MONOSPACE},
+#line 99 "webutil/css/identifier.gperf"
+ {"x-large", Identifier::X_LARGE},
+#line 19 "webutil/css/identifier.gperf"
+ {"auto", Identifier::AUTO},
+#line 35 "webutil/css/identifier.gperf"
+ {"fixed", Identifier::FIXED},
+#line 96 "webutil/css/identifier.gperf"
+ {"x-small", Identifier::X_SMALL},
+#line 141 "webutil/css/identifier.gperf"
+ {"uppercase", Identifier::UPPERCASE},
+#line 76 "webutil/css/identifier.gperf"
+ {"run-in", Identifier::RUN_IN},
+#line 127 "webutil/css/identifier.gperf"
+ {"armenian", Identifier::ARMENIAN},
+#line 129 "webutil/css/identifier.gperf"
+ {"lower-alpha", Identifier::LOWER_ALPHA},
+#line 21 "webutil/css/identifier.gperf"
+ {"visible", Identifier::VISIBLE},
+#line 100 "webutil/css/identifier.gperf"
+ {"xx-large", Identifier::XX_LARGE},
+#line 51 "webutil/css/identifier.gperf"
+ {"open-quote", Identifier::OPEN_QUOTE},
+#line 95 "webutil/css/identifier.gperf"
+ {"xx-small", Identifier::XX_SMALL},
+#line 131 "webutil/css/identifier.gperf"
+ {"invert", Identifier::INVERT},
+#line 111 "webutil/css/identifier.gperf"
+ {"menu", Identifier::MENU},
+#line 139 "webutil/css/identifier.gperf"
+ {"blink", Identifier::BLINK},
+#line 149 "webutil/css/identifier.gperf"
+ {"middle", Identifier::MIDDLE},
+#line 89 "webutil/css/identifier.gperf"
+ {"hide", Identifier::HIDE},
+#line 58 "webutil/css/identifier.gperf"
+ {"move", Identifier::MOVE},
+#line 74 "webutil/css/identifier.gperf"
+ {"block", Identifier::BLOCK},
+#line 75 "webutil/css/identifier.gperf"
+ {"list-item", Identifier::LIST_ITEM},
+#line 52 "webutil/css/identifier.gperf"
+ {"close-quote", Identifier::CLOSE_QUOTE},
+#line 77 "webutil/css/identifier.gperf"
+ {"inline-block", Identifier::INLINE_BLOCK},
+#line 54 "webutil/css/identifier.gperf"
+ {"no-close-quote", Identifier::NO_CLOSE_QUOTE},
+#line 17 "webutil/css/identifier.gperf"
+ {"inherit", Identifier::INHERIT},
+#line 156 "webutil/css/identifier.gperf"
+ {"--goog-body-color--", Identifier::GOOG_BODY_COLOR},
+#line 123 "webutil/css/identifier.gperf"
+ {"upper-roman", Identifier::UPPER_ROMAN},
+#line 157 "webutil/css/identifier.gperf"
+ {"--goog-body-link-color--", Identifier::GOOG_BODY_LINK_COLOR},
+#line 22 "webutil/css/identifier.gperf"
+ {"hidden", Identifier::HIDDEN},
+#line 25 "webutil/css/identifier.gperf"
+ {"medium", Identifier::MEDIUM},
+#line 158 "webutil/css/identifier.gperf"
+ {"--goog-big--", Identifier::GOOG_BIG},
+#line 159 "webutil/css/identifier.gperf"
+ {"--goog-small--", Identifier::GOOG_SMALL},
+#line 150 "webutil/css/identifier.gperf"
+ {"text-bottom", Identifier::TEXT_BOTTOM},
+#line 135 "webutil/css/identifier.gperf"
+ {"justify", Identifier::JUSTIFY},
+#line 112 "webutil/css/identifier.gperf"
+ {"message-box", Identifier::MESSAGE_BOX},
+#line 126 "webutil/css/identifier.gperf"
+ {"upper-latin", Identifier::UPPER_LATIN},
+#line 16 "webutil/css/identifier.gperf"
+ {"--goog-unknown--", Identifier::GOOG_UNKNOWN},
+#line 155 "webutil/css/identifier.gperf"
+ {"--goog-initial--", Identifier::GOOG_INITIAL},
+#line 138 "webutil/css/identifier.gperf"
+ {"line-through", Identifier::LINE_THROUGH},
+#line 130 "webutil/css/identifier.gperf"
+ {"upper-alpha", Identifier::UPPER_ALPHA}
+ };
+
+const struct idents *
+IdentifierMapper::in_word_set (register const char *str, register unsigned int len)
+{
+ if (len <= MAX_WORD_LENGTH && len >= MIN_WORD_LENGTH)
+ {
+ register int key = hash (str, len);
+
+ if (key <= MAX_HASH_VALUE && key >= MIN_HASH_VALUE)
+ {
+ register const struct idents *resword;
+
+ switch (key - 5)
+ {
+ case 0:
+ resword = &wordlist[0];
+ goto compare;
+ case 8:
+ resword = &wordlist[1];
+ goto compare;
+ case 16:
+ resword = &wordlist[2];
+ goto compare;
+ case 18:
+ resword = &wordlist[3];
+ goto compare;
+ case 23:
+ resword = &wordlist[4];
+ goto compare;
+ case 24:
+ resword = &wordlist[5];
+ goto compare;
+ case 25:
+ resword = &wordlist[6];
+ goto compare;
+ case 26:
+ resword = &wordlist[7];
+ goto compare;
+ case 28:
+ resword = &wordlist[8];
+ goto compare;
+ case 29:
+ resword = &wordlist[9];
+ goto compare;
+ case 30:
+ resword = &wordlist[10];
+ goto compare;
+ case 33:
+ resword = &wordlist[11];
+ goto compare;
+ case 35:
+ resword = &wordlist[12];
+ goto compare;
+ case 38:
+ resword = &wordlist[13];
+ goto compare;
+ case 41:
+ resword = &wordlist[14];
+ goto compare;
+ case 43:
+ resword = &wordlist[15];
+ goto compare;
+ case 46:
+ resword = &wordlist[16];
+ goto compare;
+ case 48:
+ resword = &wordlist[17];
+ goto compare;
+ case 49:
+ resword = &wordlist[18];
+ goto compare;
+ case 50:
+ resword = &wordlist[19];
+ goto compare;
+ case 51:
+ resword = &wordlist[20];
+ goto compare;
+ case 53:
+ resword = &wordlist[21];
+ goto compare;
+ case 57:
+ resword = &wordlist[22];
+ goto compare;
+ case 58:
+ resword = &wordlist[23];
+ goto compare;
+ case 59:
+ resword = &wordlist[24];
+ goto compare;
+ case 61:
+ resword = &wordlist[25];
+ goto compare;
+ case 66:
+ resword = &wordlist[26];
+ goto compare;
+ case 70:
+ resword = &wordlist[27];
+ goto compare;
+ case 71:
+ resword = &wordlist[28];
+ goto compare;
+ case 73:
+ resword = &wordlist[29];
+ goto compare;
+ case 75:
+ resword = &wordlist[30];
+ goto compare;
+ case 78:
+ resword = &wordlist[31];
+ goto compare;
+ case 83:
+ resword = &wordlist[32];
+ goto compare;
+ case 84:
+ resword = &wordlist[33];
+ goto compare;
+ case 85:
+ resword = &wordlist[34];
+ goto compare;
+ case 86:
+ resword = &wordlist[35];
+ goto compare;
+ case 88:
+ resword = &wordlist[36];
+ goto compare;
+ case 90:
+ resword = &wordlist[37];
+ goto compare;
+ case 91:
+ resword = &wordlist[38];
+ goto compare;
+ case 92:
+ resword = &wordlist[39];
+ goto compare;
+ case 93:
+ resword = &wordlist[40];
+ goto compare;
+ case 94:
+ resword = &wordlist[41];
+ goto compare;
+ case 95:
+ resword = &wordlist[42];
+ goto compare;
+ case 96:
+ resword = &wordlist[43];
+ goto compare;
+ case 97:
+ resword = &wordlist[44];
+ goto compare;
+ case 98:
+ resword = &wordlist[45];
+ goto compare;
+ case 99:
+ resword = &wordlist[46];
+ goto compare;
+ case 101:
+ resword = &wordlist[47];
+ goto compare;
+ case 102:
+ resword = &wordlist[48];
+ goto compare;
+ case 103:
+ resword = &wordlist[49];
+ goto compare;
+ case 104:
+ resword = &wordlist[50];
+ goto compare;
+ case 105:
+ resword = &wordlist[51];
+ goto compare;
+ case 111:
+ resword = &wordlist[52];
+ goto compare;
+ case 112:
+ resword = &wordlist[53];
+ goto compare;
+ case 113:
+ resword = &wordlist[54];
+ goto compare;
+ case 117:
+ resword = &wordlist[55];
+ goto compare;
+ case 118:
+ resword = &wordlist[56];
+ goto compare;
+ case 124:
+ resword = &wordlist[57];
+ goto compare;
+ case 125:
+ resword = &wordlist[58];
+ goto compare;
+ case 126:
+ resword = &wordlist[59];
+ goto compare;
+ case 129:
+ resword = &wordlist[60];
+ goto compare;
+ case 130:
+ resword = &wordlist[61];
+ goto compare;
+ case 131:
+ resword = &wordlist[62];
+ goto compare;
+ case 133:
+ resword = &wordlist[63];
+ goto compare;
+ case 134:
+ resword = &wordlist[64];
+ goto compare;
+ case 135:
+ resword = &wordlist[65];
+ goto compare;
+ case 137:
+ resword = &wordlist[66];
+ goto compare;
+ case 138:
+ resword = &wordlist[67];
+ goto compare;
+ case 141:
+ resword = &wordlist[68];
+ goto compare;
+ case 143:
+ resword = &wordlist[69];
+ goto compare;
+ case 144:
+ resword = &wordlist[70];
+ goto compare;
+ case 145:
+ resword = &wordlist[71];
+ goto compare;
+ case 146:
+ resword = &wordlist[72];
+ goto compare;
+ case 148:
+ resword = &wordlist[73];
+ goto compare;
+ case 150:
+ resword = &wordlist[74];
+ goto compare;
+ case 151:
+ resword = &wordlist[75];
+ goto compare;
+ case 153:
+ resword = &wordlist[76];
+ goto compare;
+ case 154:
+ resword = &wordlist[77];
+ goto compare;
+ case 156:
+ resword = &wordlist[78];
+ goto compare;
+ case 157:
+ resword = &wordlist[79];
+ goto compare;
+ case 158:
+ resword = &wordlist[80];
+ goto compare;
+ case 159:
+ resword = &wordlist[81];
+ goto compare;
+ case 160:
+ resword = &wordlist[82];
+ goto compare;
+ case 161:
+ resword = &wordlist[83];
+ goto compare;
+ case 164:
+ resword = &wordlist[84];
+ goto compare;
+ case 167:
+ resword = &wordlist[85];
+ goto compare;
+ case 168:
+ resword = &wordlist[86];
+ goto compare;
+ case 169:
+ resword = &wordlist[87];
+ goto compare;
+ case 170:
+ resword = &wordlist[88];
+ goto compare;
+ case 172:
+ resword = &wordlist[89];
+ goto compare;
+ case 173:
+ resword = &wordlist[90];
+ goto compare;
+ case 174:
+ resword = &wordlist[91];
+ goto compare;
+ case 178:
+ resword = &wordlist[92];
+ goto compare;
+ case 180:
+ resword = &wordlist[93];
+ goto compare;
+ case 181:
+ resword = &wordlist[94];
+ goto compare;
+ case 182:
+ resword = &wordlist[95];
+ goto compare;
+ case 184:
+ resword = &wordlist[96];
+ goto compare;
+ case 186:
+ resword = &wordlist[97];
+ goto compare;
+ case 187:
+ resword = &wordlist[98];
+ goto compare;
+ case 189:
+ resword = &wordlist[99];
+ goto compare;
+ case 191:
+ resword = &wordlist[100];
+ goto compare;
+ case 193:
+ resword = &wordlist[101];
+ goto compare;
+ case 194:
+ resword = &wordlist[102];
+ goto compare;
+ case 196:
+ resword = &wordlist[103];
+ goto compare;
+ case 204:
+ resword = &wordlist[104];
+ goto compare;
+ case 212:
+ resword = &wordlist[105];
+ goto compare;
+ case 214:
+ resword = &wordlist[106];
+ goto compare;
+ case 215:
+ resword = &wordlist[107];
+ goto compare;
+ case 217:
+ resword = &wordlist[108];
+ goto compare;
+ case 219:
+ resword = &wordlist[109];
+ goto compare;
+ case 221:
+ resword = &wordlist[110];
+ goto compare;
+ case 223:
+ resword = &wordlist[111];
+ goto compare;
+ case 226:
+ resword = &wordlist[112];
+ goto compare;
+ case 227:
+ resword = &wordlist[113];
+ goto compare;
+ case 233:
+ resword = &wordlist[114];
+ goto compare;
+ case 235:
+ resword = &wordlist[115];
+ goto compare;
+ case 243:
+ resword = &wordlist[116];
+ goto compare;
+ case 251:
+ resword = &wordlist[117];
+ goto compare;
+ case 254:
+ resword = &wordlist[118];
+ goto compare;
+ case 255:
+ resword = &wordlist[119];
+ goto compare;
+ case 256:
+ resword = &wordlist[120];
+ goto compare;
+ case 259:
+ resword = &wordlist[121];
+ goto compare;
+ case 264:
+ resword = &wordlist[122];
+ goto compare;
+ case 265:
+ resword = &wordlist[123];
+ goto compare;
+ case 269:
+ resword = &wordlist[124];
+ goto compare;
+ case 271:
+ resword = &wordlist[125];
+ goto compare;
+ case 282:
+ resword = &wordlist[126];
+ goto compare;
+ case 284:
+ resword = &wordlist[127];
+ goto compare;
+ case 287:
+ resword = &wordlist[128];
+ goto compare;
+ case 294:
+ resword = &wordlist[129];
+ goto compare;
+ case 296:
+ resword = &wordlist[130];
+ goto compare;
+ case 299:
+ resword = &wordlist[131];
+ goto compare;
+ case 306:
+ resword = &wordlist[132];
+ goto compare;
+ case 311:
+ resword = &wordlist[133];
+ goto compare;
+ case 312:
+ resword = &wordlist[134];
+ goto compare;
+ case 314:
+ resword = &wordlist[135];
+ goto compare;
+ case 321:
+ resword = &wordlist[136];
+ goto compare;
+ case 322:
+ resword = &wordlist[137];
+ goto compare;
+ case 326:
+ resword = &wordlist[138];
+ goto compare;
+ case 331:
+ resword = &wordlist[139];
+ goto compare;
+ case 336:
+ resword = &wordlist[140];
+ goto compare;
+ case 351:
+ resword = &wordlist[141];
+ goto compare;
+ case 362:
+ resword = &wordlist[142];
+ goto compare;
+ case 396:
+ resword = &wordlist[143];
+ goto compare;
+ }
+ return 0;
+ compare:
+ {
+ register const char *s = resword->name;
+
+ if ((((unsigned char)*str ^ (unsigned char)*s) & ~32) == 0 && !gperf_case_strncmp (str, s, len) && s[len] == '\0')
+ return resword;
+ }
+ }
+ }
+ return 0;
+}
+#line 160 "webutil/css/identifier.gperf"
+
+
+//
+// Constructor.
+//
+
+Identifier::Identifier(const UnicodeText& s) : ident_(IdentFromText(s)) {
+ if (ident_ == OTHER)
+ other_ = LowercaseAscii(s);
+}
+
+//
+// Static methods mapping Ident's to strings
+//
+
+Identifier::Ident Identifier::IdentFromText(const UnicodeText& s) {
+ const idents* a = IdentifierMapper::in_word_set(s.utf8_data(),
+ s.utf8_length());
+ if (a)
+ return a->id;
+ else
+ return OTHER;
+}
+
+static struct {
+ const char* name;
+ int len;
+} gKnownIdentifiers[TOTAL_KEYWORDS];
+
+static void InitializeIdentifierNameLookupTable() {
+ for (int i = 0; i < TOTAL_KEYWORDS; ++i) {
+ gKnownIdentifiers[wordlist[i].id].name = wordlist[i].name;
+ gKnownIdentifiers[wordlist[i].id].len = strlen(wordlist[i].name);
+ }
+}
+
+UnicodeText Identifier::TextFromIdent(Ident p) {
+ if (p == OTHER) {
+ return UTF8ToUnicodeText("OTHER", 5, false);
+ } else {
+ DCHECK_LT(p, OTHER);
+ return UTF8ToUnicodeText(gKnownIdentifiers[p].name,
+ gKnownIdentifiers[p].len,
+ false);
+ }
+}
+
+} // namespace
+
+REGISTER_MODULE_INITIALIZER(identifier, {
+ Css::InitializeIdentifierNameLookupTable();
+});
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/identifier.gperf b/trunk/src/third_party/css_parser/src/webutil/css/identifier.gperf
new file mode 100644
index 0000000..2c11fb9
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/identifier.gperf
@@ -0,0 +1,227 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+%{
+#include "webutil/css/identifier.h"
+
+#include "base/googleinit.h"
+#include "base/logging.h"
+#include "webutil/css/string_util.h"
+
+namespace Css {
+%}
+// This is the set of identifiers.
+struct idents {
+ const char *name;
+ Identifier::Ident id;
+};
+%%
+--goog-unknown--, Identifier::GOOG_UNKNOWN
+inherit, Identifier::INHERIT
+none, Identifier::NONE
+auto, Identifier::AUTO
+normal, Identifier::NORMAL
+visible, Identifier::VISIBLE
+hidden, Identifier::HIDDEN
+always, Identifier::ALWAYS
+avoid, Identifier::AVOID
+medium, Identifier::MEDIUM
+repeat, Identifier::REPEAT
+collapse, Identifier::COLLAPSE
+left, Identifier::LEFT
+center, Identifier::CENTER
+right, Identifier::RIGHT
+top, Identifier::TOP
+bottom, Identifier::BOTTOM
+both, Identifier::BOTH
+scroll, Identifier::SCROLL
+fixed, Identifier::FIXED
+transparent, Identifier::TRANSPARENT
+repeat-x, Identifier::REPEAT_X
+repeat-y, Identifier::REPEAT_Y
+no-repeat, Identifier::NO_REPEAT
+separate, Identifier::SEPARATE
+dotted, Identifier::DOTTED
+dashed, Identifier::DASHED
+solid, Identifier::SOLID
+double, Identifier::DOUBLE
+groove, Identifier::GROOVE
+ridge, Identifier::RIDGE
+inset, Identifier::INSET
+outset, Identifier::OUTSET
+thin, Identifier::THIN
+thick, Identifier::THICK
+open-quote, Identifier::OPEN_QUOTE
+close-quote, Identifier::CLOSE_QUOTE
+no-open-quote, Identifier::NO_OPEN_QUOTE
+no-close-quote, Identifier::NO_CLOSE_QUOTE
+crosshair, Identifier::CROSSHAIR
+default, Identifier::DEFAULT
+pointer, Identifier::POINTER
+move, Identifier::MOVE
+e-resize, Identifier::E_RESIZE
+ne-resize, Identifier::NE_RESIZE
+nw-resize, Identifier::NW_RESIZE
+n-resize, Identifier::N_RESIZE
+se-resize, Identifier::SE_RESIZE
+sw-resize, Identifier::SW_RESIZE
+s-resize, Identifier::S_RESIZE
+w-resize, Identifier::W_RESIZE
+text, Identifier::TEXT
+wait, Identifier::WAIT
+help, Identifier::HELP
+progress, Identifier::PROGRESS
+ltr, Identifier::LTR,
+rtl, Identifier::RTL,
+inline, Identifier::INLINE
+block, Identifier::BLOCK
+list-item, Identifier::LIST_ITEM
+run-in, Identifier::RUN_IN
+inline-block, Identifier::INLINE_BLOCK
+table, Identifier::TABLE
+inline-table, Identifier::INLINE_TABLE
+table-row-group, Identifier::TABLE_ROW_GROUP
+table-header-group, Identifier::TABLE_HEADER_GROUP
+table-footer-group, Identifier::TABLE_FOOTER_GROUP
+table-row, Identifier::TABLE_ROW
+table-column-group, Identifier::TABLE_COLUMN_GROUP
+table-column, Identifier::TABLE_COLUMN
+table-cell, Identifier::TABLE_CELL
+table-caption, Identifier::TABLE_CAPTION
+show, Identifier::SHOW
+hide, Identifier::HIDE
+serif, Identifier::SERIF
+sans-serif, Identifier::SANS_SERIF
+cursive, Identifier::CURSIVE
+fantasy, Identifier::FANTASY
+monospace, Identifier::MONOSPACE
+xx-small, Identifier::XX_SMALL
+x-small, Identifier::X_SMALL
+small, Identifier::SMALL
+large, Identifier::LARGE
+x-large, Identifier::X_LARGE
+xx-large, Identifier::XX_LARGE
+smaller, Identifier::SMALLER
+larger, Identifier::LARGER
+italic, Identifier::ITALIC
+oblique, Identifier::OBLIQUE
+small-caps, Identifier::SMALL_CAPS
+bold, Identifier::BOLD
+bolder, Identifier::BOLDER
+lighter, Identifier::LIGHTER
+caption, Identifier::CAPTION
+icon, Identifier::ICON
+menu, Identifier::MENU
+message-box, Identifier::MESSAGE_BOX
+small-caption, Identifier::SMALL_CAPTION
+status-bar, Identifier::STATUS_BAR
+inside, Identifier::INSIDE
+outside, Identifier::OUTSIDE
+disc, Identifier::DISC
+circle, Identifier::CIRCLE
+square, Identifier::SQUARE
+decimal, Identifier::DECIMAL
+decimal-leading-zero, Identifier::DECIMAL_LEADING_ZERO
+lower-roman, Identifier::LOWER_ROMAN
+upper-roman, Identifier::UPPER_ROMAN
+lower-greek, Identifier::LOWER_GREEK
+lower-latin, Identifier::LOWER_LATIN,
+upper-latin, Identifier::UPPER_LATIN
+armenian, Identifier::ARMENIAN
+georgian, Identifier::GEORGIAN
+lower-alpha, Identifier::LOWER_ALPHA
+upper-alpha, Identifier::UPPER_ALPHA
+invert, Identifier::INVERT
+static, Identifier::STATIC
+relative, Identifier::RELATIVE
+absolute, Identifier::ABSOLUTE
+justify, Identifier::JUSTIFY
+underline, Identifier::UNDERLINE
+overline, Identifier::OVERLINE
+line-through, Identifier::LINE_THROUGH
+blink, Identifier::BLINK
+capitalize, Identifier::CAPITALIZE
+uppercase, Identifier::UPPERCASE
+lowercase, Identifier::LOWERCASE
+embed, Identifier::EMBED
+bidi-override, Identifier::BIDI_OVERRIDE
+baseline, Identifier::BASELINE
+sub, Identifier::SUB
+super, Identifier::SUPER
+text-top, Identifier::TEXT_TOP
+middle, Identifier::MIDDLE
+text-bottom, Identifier::TEXT_BOTTOM
+pre, Identifier::PRE
+nowrap, Identifier::NOWRAP
+pre-wrap, Identifier::PRE_WRAP
+pre-line, Identifier::PRE_LINE
+--goog-initial--, Identifier::GOOG_INITIAL
+--goog-body-color--, Identifier::GOOG_BODY_COLOR
+--goog-body-link-color--, Identifier::GOOG_BODY_LINK_COLOR
+--goog-big--, Identifier::GOOG_BIG
+--goog-small--, Identifier::GOOG_SMALL
+%%
+
+//
+// Constructor.
+//
+
+Identifier::Identifier(const UnicodeText& s) : ident_(IdentFromText(s)) {
+ if (ident_ == OTHER)
+ other_ = LowercaseAscii(s);
+}
+
+//
+// Static methods mapping Ident's to strings
+//
+
+Identifier::Ident Identifier::IdentFromText(const UnicodeText& s) {
+ const idents* a = IdentifierMapper::in_word_set(s.utf8_data(),
+ s.utf8_length());
+ if (a)
+ return a->id;
+ else
+ return OTHER;
+}
+
+static struct {
+ const char* name;
+ int len;
+} gKnownIdentifiers[TOTAL_KEYWORDS];
+
+static void InitializeIdentifierNameLookupTable() {
+ for (int i = 0; i < TOTAL_KEYWORDS; ++i) {
+ gKnownIdentifiers[wordlist[i].id].name = wordlist[i].name;
+ gKnownIdentifiers[wordlist[i].id].len = strlen(wordlist[i].name);
+ }
+}
+
+UnicodeText Identifier::TextFromIdent(Ident p) {
+ if (p == OTHER) {
+ return UTF8ToUnicodeText("OTHER", 5, false);
+ } else {
+ DCHECK_LT(p, OTHER);
+ return UTF8ToUnicodeText(gKnownIdentifiers[p].name,
+ gKnownIdentifiers[p].len,
+ false);
+ }
+}
+
+} // namespace
+
+REGISTER_MODULE_INITIALIZER(identifier, {
+ Css::InitializeIdentifierNameLookupTable();
+});
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/identifier.h b/trunk/src/third_party/css_parser/src/webutil/css/identifier.h
new file mode 100644
index 0000000..f9d8ac0
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/identifier.h
@@ -0,0 +1,151 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2007 Google Inc. All Rights Reserved.
+// Author: yian@google.com (Yi-an Huang)
+//
+// Identifier represents the value of a CSS identifier (e.g.,
+// normal, repeat, small, inherit). If we recognize the ident,
+// we store it as an enum. Otherwise, we store the text for the
+// string value of the identifier.
+//
+// This code is based on CSS 2.1.
+
+#ifndef WEBUTIL_CSS_IDENTIFIER_H__
+#define WEBUTIL_CSS_IDENTIFIER_H__
+
+#include "util/utf8/public/unicodetext.h"
+
+namespace Css {
+
+class Identifier {
+ public:
+ enum Ident {
+ // local add
+ // UNKNOWN identifiers. Reserved for internal use.
+ GOOG_UNKNOWN,
+ // common values
+ INHERIT, NONE, AUTO, NORMAL, VISIBLE, HIDDEN, ALWAYS, AVOID, MEDIUM,
+ REPEAT, COLLAPSE, LEFT, CENTER, RIGHT, TOP, BOTTOM, BOTH,
+ SCROLL, FIXED,
+ // background-color
+ TRANSPARENT,
+ // background-repeat
+ REPEAT_X, REPEAT_Y, NO_REPEAT,
+ // border-collapse
+ SEPARATE,
+ // border-style
+ DOTTED, DASHED, SOLID, DOUBLE, GROOVE, RIDGE, INSET, OUTSET,
+ // border-width
+ THIN, THICK,
+ // content
+ OPEN_QUOTE, CLOSE_QUOTE, NO_OPEN_QUOTE, NO_CLOSE_QUOTE,
+ // cursor
+ CROSSHAIR, DEFAULT, POINTER, MOVE, E_RESIZE, NE_RESIZE, NW_RESIZE,
+ N_RESIZE, SE_RESIZE, SW_RESIZE, S_RESIZE, W_RESIZE, TEXT, WAIT, HELP,
+ PROGRESS,
+ // direction,
+ LTR, RTL,
+ // display
+ INLINE, BLOCK, LIST_ITEM, RUN_IN, INLINE_BLOCK, TABLE, INLINE_TABLE,
+ TABLE_ROW_GROUP, TABLE_HEADER_GROUP, TABLE_FOOTER_GROUP, TABLE_ROW,
+ TABLE_COLUMN_GROUP, TABLE_COLUMN, TABLE_CELL, TABLE_CAPTION,
+ // empty-cells
+ SHOW, HIDE,
+ // font-family
+ SERIF, SANS_SERIF, CURSIVE, FANTASY, MONOSPACE,
+ // font-size
+ XX_SMALL, X_SMALL, SMALL, LARGE, X_LARGE, XX_LARGE, SMALLER, LARGER,
+ // font-style
+ ITALIC, OBLIQUE,
+ // font-variant
+ SMALL_CAPS,
+ // font-weight
+ BOLD, BOLDER, LIGHTER,
+ // font
+ CAPTION, ICON, MENU, MESSAGE_BOX, SMALL_CAPTION, STATUS_BAR,
+ // list-style-position
+ INSIDE, OUTSIDE,
+ // list-style-type
+ DISC, CIRCLE, SQUARE, DECIMAL, DECIMAL_LEADING_ZERO, LOWER_ROMAN,
+ UPPER_ROMAN, LOWER_GREEK, LOWER_LATIN, UPPER_LATIN, ARMENIAN, GEORGIAN,
+ LOWER_ALPHA, UPPER_ALPHA,
+ // outline-color
+ INVERT,
+ // position
+ STATIC, RELATIVE, ABSOLUTE,
+ // text-align
+ JUSTIFY,
+ // text-decoration
+ UNDERLINE, OVERLINE, LINE_THROUGH, BLINK,
+ // text-transform
+ CAPITALIZE, UPPERCASE, LOWERCASE,
+ // unicode-bidi
+ EMBED, BIDI_OVERRIDE,
+ // vertical-align
+ BASELINE, SUB, SUPER, TEXT_TOP, MIDDLE, TEXT_BOTTOM,
+ // white-space
+ PRE, NOWRAP, PRE_WRAP, PRE_LINE,
+ // google specific. Internal use only.
+ // For property with context-dependent initial values. such as border-color
+ // and text-align.
+ GOOG_INITIAL,
+ // color specified by <body text=color>
+ GOOG_BODY_COLOR,
+ // color specified by <body link=color>
+ GOOG_BODY_LINK_COLOR,
+ // identifier reserved for font-size in <big> and <small>. IE has special
+ // semantics for them.
+ GOOG_BIG, GOOG_SMALL,
+ OTHER
+ };
+
+ // Constructor.
+ Identifier() : ident_(GOOG_UNKNOWN) { }
+ explicit Identifier(const UnicodeText& s);
+ explicit Identifier(Ident ident) : ident_(ident) { }
+
+ // Accessors.
+ //
+ // ident() returns the ident enum -- OTHER if unrecognized.
+ Ident ident() const { return ident_; }
+
+ // ident_text() returns the identifier as a string.
+ UnicodeText ident_text() const {
+ if (ident_ == OTHER)
+ return other_;
+ else
+ return TextFromIdent(ident_);
+ }
+
+ // Static methods mapping between Ident and strings:
+ //
+ // Given the text of a CSS identifier, IdentFromText returns the
+ // corresponding enum. If no such identifier is found, IdentromText returns
+ // OTHER.
+ static Ident IdentFromText(const UnicodeText& s);
+ // Given a Ident, returns its string representation. If u is OTHER, we
+ // return "OTHER", but this may not be what you want.
+ static UnicodeText TextFromIdent(Ident p);
+
+ private:
+ Ident ident_;
+ UnicodeText other_; // valid if ident_ is OTHER.
+};
+
+} // namespace
+
+#endif // WEBUTIL_CSS_IDENTIFIER_H__
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/identifier_test.cc b/trunk/src/third_party/css_parser/src/webutil/css/identifier_test.cc
new file mode 100644
index 0000000..fd2da6a
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/identifier_test.cc
@@ -0,0 +1,66 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2007 Google Inc. All Rights Reserved.
+// Author: yian@google.com (Yi-an Huang)
+
+#include "webutil/css/identifier.h"
+
+#include <string>
+
+#include "testing/base/public/googletest.h"
+#include "testing/base/public/gunit.h"
+#include "webutil/css/string.h"
+
+namespace {
+
+class IdentifierTest : public testing::Test {
+};
+
+TEST_F(IdentifierTest, IdentFromText) {
+ UnicodeText s = UTF8ToUnicodeText(string("Inherit"), true);
+ EXPECT_EQ(Css::Identifier::INHERIT, Css::Identifier::IdentFromText(s));
+ EXPECT_EQ(Css::Identifier::INHERIT, Css::Identifier(s).ident());
+ EXPECT_EQ("inherit", UnicodeTextToUTF8(Css::Identifier(s).ident_text()));
+}
+
+TEST_F(IdentifierTest, TextFromIdent) {
+ EXPECT_EQ("inherit",
+ UnicodeTextToUTF8(Css::Identifier::TextFromIdent(
+ Css::Identifier::INHERIT)));
+ EXPECT_EQ("OTHER",
+ UnicodeTextToUTF8(Css::Identifier::TextFromIdent(
+ Css::Identifier::OTHER)));
+}
+
+TEST_F(IdentifierTest, UnicodeText) {
+ UnicodeText s = UTF8ToUnicodeText(string("宋体"), true);
+ EXPECT_EQ(Css::Identifier::OTHER, Css::Identifier::IdentFromText(s));
+ Css::Identifier id(s);
+ EXPECT_EQ(Css::Identifier::OTHER, id.ident());
+ EXPECT_EQ("宋体", UnicodeTextToUTF8(id.ident_text()));
+}
+
+TEST_F(IdentifierTest, Inverses) {
+ for (int i = 0; i < Css::Identifier::OTHER; ++i) {
+ UnicodeText s = Css::Identifier::TextFromIdent(
+ static_cast<Css::Identifier::Ident>(i));
+ EXPECT_EQ(static_cast<Css::Identifier::Ident>(i),
+ Css::Identifier::IdentFromText(s));
+ }
+}
+
+} // namespace
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/parse_arg.cc b/trunk/src/third_party/css_parser/src/webutil/css/parse_arg.cc
new file mode 100644
index 0000000..b1110e6
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/parse_arg.cc
@@ -0,0 +1,40 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include <stdio.h>
+#include <stdlib.h>
+
+#include <string>
+
+#include "webutil/css/parser.h"
+
+int main(int argc, char** argv) {
+ if (argc > 1) {
+ std::string text(argv[1]);
+
+ printf("Parsing:\n%s\n\n", text.c_str());
+ Css::Parser parser(text);
+ Css::Stylesheet* stylesheet = parser.ParseStylesheet();
+
+ printf("Stylesheet:\n%s\n", stylesheet->ToString().c_str());
+
+ delete stylesheet;
+ }
+
+ return EXIT_SUCCESS;
+}
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/parser.cc b/trunk/src/third_party/css_parser/src/webutil/css/parser.cc
new file mode 100644
index 0000000..d92b7a0
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/parser.cc
@@ -0,0 +1,1802 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2006 Google Inc. All Rights Reserved.
+// Author: dpeng@google.com (Daniel Peng)
+
+#include "webutil/css/parser.h"
+
+#include <ctype.h> // isascii
+
+#include <string>
+#include <vector>
+
+#include "base/logging.h"
+#include "base/scoped_ptr.h"
+#include "strings/memutil.h"
+#include "strings/strutil.h"
+#include "third_party/utf/utf.h"
+#include "util/gtl/stl_util-inl.h"
+#include "webutil/css/string.h"
+#include "webutil/css/string_util.h"
+#include "webutil/css/util.h"
+#include "webutil/css/value.h"
+#include "webutil/css/valuevalidator.h"
+
+
+namespace Css {
+
+const uint64 Parser::kNoError;
+const uint64 Parser::kUtf8Error;
+const uint64 Parser::kDeclarationError;
+const uint64 Parser::kSelectorError;
+const uint64 Parser::kFunctionError;
+const uint64 Parser::kMediaError;
+
+
+// Using isascii with signed chars is unfortunately undefined.
+static inline bool IsAscii(char c) {
+ return isascii(static_cast<unsigned char>(c));
+}
+
+class Tracer { // in opt mode, do nothing.
+ public:
+ Tracer(const char* name, const char** in) { }
+};
+
+
+// ****************
+// constructors
+// ****************
+
+Parser::Parser(const char* utf8text, const char* textend)
+ : in_(utf8text),
+ end_(textend),
+ quirks_mode_(true),
+ errors_seen_mask_(kNoError) {
+}
+
+Parser::Parser(const char* utf8text)
+ : in_(utf8text),
+ end_(utf8text + strlen(utf8text)),
+ quirks_mode_(true),
+ errors_seen_mask_(kNoError) {
+}
+
+Parser::Parser(StringPiece s)
+ : in_(s.begin()),
+ end_(s.end()),
+ quirks_mode_(true),
+ errors_seen_mask_(kNoError) {
+}
+
+// ****************
+// Helper functions
+// ****************
+
+// is c a space? Only the characters "space" (Unicode code 32), "tab"
+// (9), "line feed" (10), "carriage return" (13), and "form feed" (12)
+// can occur in whitespace. Other space-like characters, such as
+// "em-space" (8195) and "ideographic space" (12288), are never part
+// of whitespace.
+// http://www.w3.org/TR/REC-CSS2/syndata.html#whitespace
+static bool IsSpace(char c) {
+ switch (c) {
+ case ' ': case '\t': case '\r': case '\n': case '\f':
+ return true;
+ default:
+ return false;
+ }
+}
+
+// If the character c is a hex digit, DeHex returns the number it
+// represents ('0' => 0, 'A' => 10, 'F' => 15). Otherwise, DeHex
+// returns -1.
+static int DeHex(char c) {
+ if (c >= '0' && c <= '9') {
+ return c - '0';
+ } else if (c >= 'A' && c <= 'F') {
+ return (c - 'A') + 10;
+ } else if (c >= 'a' && c <= 'f') {
+ return (c - 'a') + 10;
+ } else {
+ return -1;
+ }
+}
+
+// ****************
+// Recursive-descent functions.
+//
+// The best documentation for these is in cssparser.h.
+//
+// ****************
+
+// consume whitespace and comments.
+void Parser::SkipSpace() {
+ Tracer trace(__func__, &in_);
+ while (in_ < end_) {
+ if (IsSpace(*in_))
+ in_++;
+ else if (in_ + 1 < end_ && in_[0] == '/' && in_[1] == '*')
+ SkipComment();
+ else
+ return;
+ }
+}
+
+// consume comment /* aoeuaoe */
+void Parser::SkipComment() {
+ DCHECK(in_ + 2 <= end_ && in_[0] == '/' && in_[1] == '*');
+ in_ += 2; // skip the /*
+ while (in_ + 1 < end_) {
+ if (in_[0] == '*' && in_[1] == '/') {
+ in_ += 2;
+ return;
+ } else {
+ in_++;
+ }
+ }
+ in_ = end_;
+}
+
+// skips until delim is seen or end-of-stream. returns if delim is actually
+// seen.
+bool Parser::SkipPastDelimiter(char delim) {
+ SkipSpace();
+ while (in_ < end_ && *in_ != delim) {
+ ++in_;
+ SkipSpace();
+ }
+
+ if (Done()) return false;
+ ++in_;
+ return true;
+}
+
+// returns true if there might be a token to read
+bool Parser::SkipToNextToken() {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ while (in_ < end_) {
+ switch (*in_) {
+ case '{':
+ ParseBlock(); // ignore
+ break;
+ case '@':
+ in_++;
+ ParseIdent(); // ignore
+ break;
+ case ';': case '}':
+ case '!':
+ return false;
+ default:
+ return true;
+ }
+ SkipSpace();
+ }
+ return false;
+}
+
+// In CSS2, identifiers (including element names, classes, and IDs in
+// selectors) can contain only the characters [A-Za-z0-9] and ISO
+// 10646 characters 161 and higher, plus the hyphen (-); they cannot
+// start with a hyphen or a digit. They can also contain escaped
+// characters and any ISO 10646 character as a numeric code (see next
+// item). For instance, the identifier "B&W?" may be written as
+// "B\&W\?" or "B\26 W\3F".
+//
+// We're a little more forgiving than the standard and permit hyphens
+// and digits to start identifiers.
+//
+// FIXME(yian): actually, IE is more forgiving than Firefox in using a class
+// selector starting with digits.
+//
+// http://www.w3.org/TR/REC-CSS2/syndata.html#value-def-identifier
+static bool StartsIdent(char c) {
+ return ((c >= 'A' && c <= 'Z')
+ || (c >= 'a' && c <= 'z')
+ || (c >= '0' && c <= '9')
+ || c == '-' || c == '_'
+ || !IsAscii(c));
+}
+
+UnicodeText Parser::ParseIdent() {
+ Tracer trace(__func__, &in_);
+ UnicodeText s;
+ while (in_ < end_) {
+ if ((*in_ >= 'A' && *in_ <= 'Z')
+ || (*in_ >= 'a' && *in_ <= 'z')
+ || (*in_ >= '0' && *in_ <= '9')
+ || *in_ == '-' || *in_ == '_') {
+ s.push_back(*in_);
+ in_++;
+ } else if (!IsAscii(*in_)) {
+ Rune rune;
+ int len = charntorune(&rune, in_, end_-in_);
+ if (len && rune != Runeerror) {
+ if (rune >= 161) {
+ s.push_back(rune);
+ in_ += len;
+ } else { // characters 128-160 can't be in identifiers.
+ return s;
+ }
+ } else { // Encoding error. Be a little forgiving.
+ errors_seen_mask_ |= kUtf8Error;
+ in_++;
+ }
+ } else if (*in_ == '\\') {
+ s.push_back(ParseEscape());
+ } else {
+ return s;
+ }
+ }
+ return s;
+}
+
+// Returns the codepoint for the current escape.
+// \abcdef => codepoint 0xabcdef. also consumes whitespace afterwards.
+// \(UTF8-encoded unicode character) => codepoint for that character
+char32 Parser::ParseEscape() {
+ SkipSpace();
+ DCHECK_LT(in_, end_);
+ DCHECK_EQ(*in_, '\\');
+ in_++;
+ if (Done()) return static_cast<char32>('\\');
+
+ int dehexed = DeHex(*in_);
+ if (dehexed == -1) {
+ Rune rune;
+ int len = charntorune(&rune, in_, end_-in_);
+ if (len && rune != Runeerror) {
+ in_ += len;
+ } else {
+ errors_seen_mask_ |= kUtf8Error;
+ in_++;
+ }
+ return rune;
+ } else {
+ char32 codepoint = 0;
+ for (int count = 0; count < 6 && in_ < end_; count++) {
+ dehexed = DeHex(*in_);
+ if (dehexed == -1)
+ break;
+ in_++;
+ codepoint = codepoint << 4 | dehexed;
+ }
+ if (end_ - in_ >= 2 && memcmp(in_, "\r\n", 2) == 0)
+ in_ += 2;
+ else if (IsSpace(*in_))
+ in_++;
+ return codepoint;
+ }
+}
+
+// Starts at delim.
+template<char delim>
+UnicodeText Parser::ParseString() {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ DCHECK_LT(in_, end_);
+ DCHECK_EQ(*in_, delim);
+ in_++;
+ if (Done()) return UnicodeText();
+
+ UnicodeText s;
+ while (in_ < end_) {
+ switch (*in_) {
+ case delim:
+ in_++;
+ return s;
+ case '\n':
+ return s;
+ case '\\':
+ if (in_ + 1 < end_ && in_[1] == '\n') {
+ in_ += 2;
+ } else {
+ s.push_back(ParseEscape());
+ }
+ break;
+ default:
+ if (!IsAscii(*in_)) {
+ Rune rune;
+ int len = charntorune(&rune, in_, end_-in_);
+ if (len && rune != Runeerror) {
+ s.push_back(rune);
+ in_ += len;
+ } else {
+ errors_seen_mask_ |= kUtf8Error;
+ in_++;
+ }
+ } else {
+ s.push_back(*in_);
+ in_++;
+ }
+ break;
+ }
+ }
+ return s;
+}
+
+// parse ident or 'string'
+UnicodeText Parser::ParseStringOrIdent() {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return UnicodeText();
+ DCHECK_LT(in_, end_);
+
+ if (*in_ == '\'') {
+ return ParseString<'\''>();
+ } else if (*in_ == '"') {
+ return ParseString<'"'>();
+ } else {
+ return ParseIdent();
+ }
+}
+
+// Parse a CSS number, including unit or percent sign.
+Value* Parser::ParseNumber() {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return NULL;
+ DCHECK_LT(in_, end_);
+
+ const char* begin = in_;
+ if (in_ < end_ && (*in_ == '-' || *in_ == '+')) // sign
+ in_++;
+ while (in_ < end_ && isdigit(*in_)) {
+ in_++;
+ }
+ if (*in_ == '.') {
+ in_++;
+
+ while (in_ < end_ && isdigit(*in_)) {
+ in_++;
+ }
+ }
+ double num = 0;
+ if (in_ == begin || !ParseDouble(begin, in_ - begin, &num)) {
+ return NULL;
+ }
+ if (*in_ == '%') {
+ in_++;
+ return new Value(num, Value::PERCENT);
+ } else if (StartsIdent(*in_)) {
+ return new Value(num, ParseIdent());
+ } else {
+ return new Value(num, Value::NO_UNIT);
+ }
+}
+
+HtmlColor Parser::ParseColor() {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return HtmlColor("", 0);
+ DCHECK_LT(in_, end_);
+
+ unsigned char hexdigits[6] = {0};
+ int dehexed;
+ int i = 0;
+
+ const char* oldin = in_;
+
+ // To further mess things up, IE also accepts string values happily.
+ if (*in_ == '"' || *in_ == '\'') {
+ in_++;
+ if (Done()) return HtmlColor("", 0);
+ }
+
+ bool rgb_valid = quirks_mode_ || *in_ == '#';
+
+ if (*in_ == '#') in_++;
+
+ while (in_ < end_ && i < 6 && (dehexed = DeHex(*in_)) != -1) {
+ hexdigits[i] = static_cast<unsigned char>(dehexed);
+ i++;
+ in_++;
+ }
+
+ // close strings. Assume a named color if there are trailing characters
+ if (*oldin == '"' || *oldin == '\'') {
+ if (Done() || *in_ != *oldin) // no need to touch in_, will redo anyway.
+ i = 0;
+ else
+ in_++;
+ }
+
+ // Normally, ParseXXX() routines stop wherever it cannot be consumed and
+ // doesn't check whether the next character is valid. which should be caught
+ // by the next ParseXXX() routine. But ParseColor may be called to test
+ // whether a numerical value can be used as color, and fail over to a normal
+ // ParseAny(). We need to do an immediate check here to guarantine a valid
+ // non-color number (such as 100%) will not be accepted as a color.
+ //
+ // We also do not want rrggbb (without #) to be accepted in non-quirks mode,
+ // but HtmlColor will happily accept it anyway. Do a sanity check here.
+ if (i == 3 || i == 6) {
+ if (!rgb_valid ||
+ (!Done() && (*in_ == '%' || StartsIdent(*in_))))
+ return HtmlColor("", 0);
+ }
+
+ if (i == 3) {
+ return HtmlColor(hexdigits[0] | hexdigits[0] << 4,
+ hexdigits[1] | hexdigits[1] << 4,
+ hexdigits[2] | hexdigits[2] << 4);
+ } else if (i == 6) {
+ return HtmlColor(hexdigits[1] | hexdigits[0] << 4,
+ hexdigits[3] | hexdigits[2] << 4,
+ hexdigits[5] | hexdigits[4] << 4);
+ } else {
+ in_ = oldin;
+
+ // A named color must not begin with #, but we need to parse it anyway and
+ // report failure later.
+ bool name_valid = true;
+ if (*in_ == '#') {
+ in_++;
+ name_valid = false;
+ }
+
+ string ident = UnicodeTextToUTF8(ParseStringOrIdent());
+ HtmlColor val("", 0);
+ if (name_valid) {
+ val.SetValueFromName(ident.c_str());
+ if (!val.IsDefined())
+ Util::GetSystemColor(ident, &val);
+ }
+ return val;
+ }
+}
+
+// Returns the 0-255 RGB value corresponding to Value v. Only
+// unusual thing is percentages are interpreted as percentages of
+// 255.0.
+unsigned char Parser::ValueToRGB(Value* v) {
+ int toret = 0;
+ if (v == NULL) {
+ toret = 0;
+ } else if (v->GetLexicalUnitType() == Value::NUMBER) {
+ if (v->GetDimension() == Value::PERCENT) {
+ toret = static_cast<int>(v->GetFloatValue()/100.0 * 255.0);
+ } else {
+ toret = v->GetIntegerValue();
+ }
+ } else {
+ toret = 0;
+ }
+
+ // RGB values outside the device gamut should be clipped according to spec.
+ if (toret > 255)
+ toret = 255;
+ if (toret < 0)
+ toret = 0;
+ return static_cast<unsigned char>(toret);
+}
+
+// parse RGB color 25, 32, 12 or 25%, 1%, 7%.
+// stops without consuming final right-paren
+Value* Parser::ParseRgbColor() {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return NULL;
+ DCHECK_LT(in_, end_);
+
+ unsigned char rgb[3];
+
+ for (int i = 0; i < 3; i++) {
+ scoped_ptr<Value> val(ParseNumber());
+ if (!val.get() || val->GetLexicalUnitType() != Value::NUMBER ||
+ (val->GetDimension() != Value::PERCENT &&
+ val->GetDimension() != Value::NO_UNIT))
+ break;
+ rgb[i] = ValueToRGB(val.get());
+ SkipSpace();
+ // Make sure the correct syntax is followed.
+ if (Done() || (*in_ != ',' && *in_ != ')') || (*in_ == ')' && i != 2))
+ break;
+
+ if (*in_ == ')')
+ return new Value(HtmlColor(rgb[0], rgb[1], rgb[2]));
+ in_++; // ','
+ }
+
+ return NULL;
+}
+
+// parse url yellow.png or 'yellow.png'
+// (doesn't consume subsequent right-paren).
+Value* Parser::ParseUrl() {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return NULL;
+ DCHECK_LT(in_, end_);
+
+ UnicodeText s;
+ if (*in_ == '\'') {
+ s = ParseString<'\''>();
+ } else if (*in_ == '"') {
+ s = ParseString<'"'>();
+ } else {
+ while (in_ < end_) {
+ if (IsSpace(*in_) || *in_ == ')') {
+ break;
+ } else if (*in_ == '\\') {
+ s.push_back(ParseEscape());
+ } else if (!IsAscii(*in_)) {
+ Rune rune;
+ int len = charntorune(&rune, in_, end_-in_);
+ if (len && rune != Runeerror) {
+ s.push_back(rune);
+ in_ += len;
+ } else {
+ errors_seen_mask_ |= kUtf8Error;
+ in_++;
+ }
+ } else {
+ s.push_back(*in_);
+ in_++;
+ }
+ }
+ }
+ SkipSpace();
+ if (!Done() && *in_ == ')')
+ return new Value(Value::URI, s);
+
+ return NULL;
+}
+
+// parse rect(top, right, bottom, left) without consuming final right-paren.
+// Spaces are allowed as delimiters here for historical reasons.
+Value* Parser::ParseRect() {
+ scoped_ptr<Values> params(new Values);
+
+ SkipSpace();
+ if (Done()) return NULL;
+ DCHECK_LT(in_, end_);
+
+ // Never parse after the final right-paren!
+ if (*in_ == ')') return NULL;
+
+ for (int i = 0; i < 4; i++) {
+ scoped_ptr<Value> val(ParseAny());
+ if (!val.get())
+ break;
+ params->push_back(val.release());
+ SkipSpace();
+ // Make sure the correct syntax is followed.
+ if (Done() || (*in_ == ')' && i != 3))
+ break;
+
+ if (*in_ == ')')
+ return new Value(Value::RECT, params.release());
+ else if (*in_ == ',')
+ in_++;
+ }
+
+ return NULL;
+}
+
+Value* Parser::ParseAnyExpectingColor() {
+ Tracer trace(__func__, &in_);
+ Value* toret = NULL;
+
+ SkipSpace();
+ if (Done()) return NULL;
+ DCHECK_LT(in_, end_);
+
+ const char* oldin = in_;
+ HtmlColor c = ParseColor();
+ if (c.IsDefined()) {
+ toret = new Value(c);
+ } else {
+ in_ = oldin; // no valid color. rollback.
+ toret = ParseAny();
+ }
+ return toret;
+}
+
+// Parses a CSS value. Could be just about anything.
+Value* Parser::ParseAny() {
+ Tracer trace(__func__, &in_);
+ Value* toret = NULL;
+
+ SkipSpace();
+ if (Done()) return NULL;
+ DCHECK_LT(in_, end_);
+
+ const char* oldin = in_;
+ switch (*in_) {
+ case '0':
+ case '1':
+ case '2':
+ case '3':
+ case '4':
+ case '5':
+ case '6':
+ case '7':
+ case '8':
+ case '9':
+ case '.':
+ toret = ParseNumber();
+ break;
+ case '(': case '[': {
+ char delim = *in_ == '(' ? ')' : ']';
+ SkipPastDelimiter(delim);
+ toret = NULL; // we don't understand this construct.
+ break;
+ }
+ case '"':
+ toret = new Value(Value::STRING, ParseString<'"'>());
+ break;
+ case '\'':
+ toret = new Value(Value::STRING, ParseString<'\''>());
+ break;
+ case '#': {
+ HtmlColor color = ParseColor();
+ if (color.IsDefined())
+ toret = new Value(color);
+ else
+ toret = NULL;
+ break;
+ }
+ case '+':
+ toret = ParseNumber();
+ break;
+ case '-':
+ // ambiguity between a negative number and an identifier starting with -.
+ if (in_ < end_ - 1 &&
+ ((*(in_ + 1) >= '0' && *(in_ + 1) <= '9') || *(in_ + 1) == '.')) {
+ toret = ParseNumber();
+ break;
+ }
+ // fail through
+ default: {
+ UnicodeText id = ParseIdent();
+ if (id.empty()) {
+ toret = NULL;
+ } else if (*in_ == '(') {
+ in_++;
+ if (id.utf8_length() == 3
+ && memcasecmp("url", id.utf8_data(), 3) == 0) {
+ toret = ParseUrl();
+ } else if (id.utf8_length() == 7
+ && memcasecmp("counter", id.utf8_data(), 7) == 0) {
+ // TODO(yian): parse COUNTER parameters
+ toret = new Value(Value::COUNTER, new Values());
+ } else if (id.utf8_length() == 8
+ && memcasecmp("counters", id.utf8_data(), 8) == 0) {
+ // TODO(yian): parse COUNTERS parameters
+ toret = new Value(Value::COUNTER, new Values());
+ } else if (id.utf8_length() == 3
+ && memcasecmp("rgb", id.utf8_data(), 3) == 0) {
+ toret = ParseRgbColor();
+ } else if (id.utf8_length() == 4
+ && memcasecmp("rect", id.utf8_data(), 4) == 0) {
+ toret = ParseRect();
+ } else {
+ errors_seen_mask_ |= kFunctionError;
+ // TODO(yian): parse FUNCTION parameters
+ toret = new Value(id, new Values());
+ }
+ SkipPastDelimiter(')');
+ } else {
+ toret = new Value(Identifier(id));
+ }
+ break;
+ }
+ }
+ // Deadlock prevention: always make progress even if nothing can be parsed.
+ if (toret == NULL && in_ == oldin) ++in_;
+ return toret;
+}
+
+static bool IsPropExpectingColor(Property::Prop prop) {
+ switch (prop) {
+ case Property::BORDER_COLOR:
+ case Property::BORDER_TOP_COLOR:
+ case Property::BORDER_RIGHT_COLOR:
+ case Property::BORDER_BOTTOM_COLOR:
+ case Property::BORDER_LEFT_COLOR:
+ case Property::BORDER:
+ case Property::BORDER_TOP:
+ case Property::BORDER_RIGHT:
+ case Property::BORDER_BOTTOM:
+ case Property::BORDER_LEFT:
+ case Property::BACKGROUND_COLOR:
+ case Property::BACKGROUND:
+ case Property::COLOR:
+ case Property::OUTLINE_COLOR:
+ case Property::OUTLINE:
+ return true;
+ default:
+ return false;
+ }
+}
+
+// Parse values like "12pt Arial"
+// If you make any change to this function, please also update
+// ParseBackground, ParseFont and ParseFontFamily accordingly.
+Values* Parser::ParseValues(Property::Prop prop) {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return new Values();
+ DCHECK_LT(in_, end_);
+
+ // If expecting_color is true, color values are expected.
+ bool expecting_color = IsPropExpectingColor(prop);
+
+ scoped_ptr<Values> values(new Values);
+ while (SkipToNextToken()) {
+ scoped_ptr<Value> v(expecting_color ?
+ ParseAnyExpectingColor() :
+ ParseAny());
+ if (v.get() && ValueValidator::Get()->IsValidValue(prop, *v, quirks_mode_))
+ values->push_back(v.release());
+ else
+ return NULL;
+ }
+ return values.release();
+}
+
+// Parse background. It is a shortcut property for individual background
+// properties.
+//
+// The output is a tuple in the following order:
+// "background-color background-image background-repeat background-attachment
+// background-position-x background-position-y"
+// or NULL if invalid
+//
+// The x-y position parsing is somewhat complicated. The following spec is from
+// CSS 2.1.
+// http://www.w3.org/TR/CSS21/colors.html#propdef-background-position
+//
+// "If a background image has been specified, this property specifies its
+// initial position. If only one value is specified, the second value is
+// assumed to be 'center'. If at least one value is not a keyword, then the
+// first value represents the horizontal position and the second represents the
+// vertical position. Negative <percentage> and <length> values are allowed.
+// <percentage> ...
+// <length> ...
+// top ...
+// right ...
+// bottom ...
+// left ...
+// center ..."
+//
+// In addition, we have some IE specific behavior:
+// 1) you can specifiy more than two values, but once both x and y have
+// specified values, further values will be discarded.
+// 2) if y is not specified and x has seen two or more values, the last value
+// counts. The same for y.
+// 3) [length, left/right] is valid and the length becomes a value for y.
+// [top/bottom, length] is also valid and the length becomes a value for x.
+// If you make any change to this function, please also update ParseValues,
+// ParseFont and ParseFontFamily if applicable.
+bool Parser::ExpandBackground(const Declaration& original_declaration,
+ Declarations* new_declarations) {
+ const Values* vals = original_declaration.values();
+ bool important = original_declaration.IsImportant();
+ DCHECK(vals != NULL);
+
+ Value background_color(Identifier::TRANSPARENT);
+ Value background_image(Identifier::NONE);
+ Value background_repeat(Identifier::REPEAT);
+ Value background_attachment(Identifier::SCROLL);
+ scoped_ptr<Value> background_position_x;
+ scoped_ptr<Value> background_position_y;
+
+ bool is_first = true;
+
+ // The following flag is used to implement IE quirks #3. When the first
+ // positional value is a length or CENTER, it is stored in
+ // background-position-x, but the value may actually be used as
+ // background-position-y if a keyword LEFT or RIGHT appears later.
+ bool first_is_ambiguous = false; // Value::NUMBER or Identifier::CENTER
+
+ for (Values::const_iterator iter = vals->begin(); iter != vals->end();
+ ++iter) {
+ const Value* val = *iter;
+
+ // Firefox allows only one value to be set per property, IE need not.
+ switch (val->GetLexicalUnitType()) {
+ case Value::COLOR:
+ // background_color, etc. take ownership of val. We will clear vals
+ // at the end to make sure we don't have double ownership.
+ background_color = *val;
+ break;
+ case Value::URI:
+ background_image = *val;
+ break;
+ case Value::NUMBER:
+ if (!background_position_x.get()) {
+ background_position_x.reset(new Value(*val));
+ first_is_ambiguous = true;
+ } else if (!background_position_y.get()) {
+ background_position_y.reset(new Value(*val));
+ }
+ break;
+ case Value::IDENT:
+ switch (val->GetIdentifier().ident()) {
+ case Identifier::CENTER:
+ if (!background_position_x.get()) {
+ background_position_x.reset(new Value(*val));
+ first_is_ambiguous = true;
+ } else if (!background_position_y.get()) {
+ background_position_y.reset(new Value(*val));
+ }
+ break;
+ case Identifier::LEFT:
+ case Identifier::RIGHT:
+ // This is IE-specific behavior.
+ if (!background_position_x.get() || !background_position_y.get()) {
+ if (background_position_x.get() && first_is_ambiguous)
+ background_position_y.reset(background_position_x.release());
+ background_position_x.reset(new Value(*val));
+ first_is_ambiguous = false;
+ }
+ break;
+ case Identifier::TOP:
+ case Identifier::BOTTOM:
+ if (!background_position_x.get() || !background_position_y.get())
+ background_position_y.reset(new Value(*val));
+ break;
+ case Identifier::REPEAT:
+ case Identifier::REPEAT_X:
+ case Identifier::REPEAT_Y:
+ case Identifier::NO_REPEAT:
+ background_repeat = *val;
+ break;
+ case Identifier::SCROLL:
+ case Identifier::FIXED:
+ background_attachment = *val;
+ break;
+ case Identifier::TRANSPARENT:
+ background_color = *val;
+ break;
+ case Identifier::NONE:
+ background_image = *val;
+ break;
+ case Identifier::INHERIT:
+ // Inherit must be the one and only value.
+ if (!(iter == vals->begin() && vals->size() == 1))
+ return false;
+ // We copy the inherit value into each background_* value.
+ background_color = *val;
+ background_image = *val;
+ background_repeat = *val;
+ background_attachment = *val;
+ background_position_x.reset(new Value(*val));
+ background_position_y.reset(new Value(*val));
+ break;
+ default:
+ return false;
+ }
+ break;
+ default:
+ return false;
+ }
+ is_first = false;
+ }
+ if (is_first) return false;
+
+ new_declarations->push_back(new Declaration(Property::BACKGROUND_COLOR,
+ background_color,
+ important));
+ new_declarations->push_back(new Declaration(Property::BACKGROUND_IMAGE,
+ background_image,
+ important));
+ new_declarations->push_back(new Declaration(Property::BACKGROUND_REPEAT,
+ background_repeat,
+ important));
+ new_declarations->push_back(new Declaration(Property::BACKGROUND_ATTACHMENT,
+ background_attachment,
+ important));
+
+ // Fix up x and y position.
+ if (!background_position_x.get() && !background_position_y.get()) {
+ background_position_x.reset(new Value(0, Value::PERCENT));
+ background_position_y.reset(new Value(0, Value::PERCENT));
+ } else if (!background_position_x.get()) {
+ background_position_x.reset(new Value(50, Value::PERCENT));
+ } else if (!background_position_y.get()) {
+ background_position_y.reset(new Value(50, Value::PERCENT));
+ }
+ new_declarations->push_back(new Declaration(Property::BACKGROUND_POSITION_X,
+ *background_position_x,
+ important));
+ new_declarations->push_back(new Declaration(Property::BACKGROUND_POSITION_Y,
+ *background_position_y,
+ important));
+
+ return true;
+}
+
+// Parses font-family. It is special in that it uses commas as delimiters. It
+// also concatenates adjacent idents into one name. Strings can be also used
+// and they are separate from others even without commas.
+// E.g, Courier New, Sans -> "Courier New", "Sans"
+// Arial "MS Times" monospace -> "Arial", "MS Times", "monospace".
+// If you make any change to this function, please also update ParseValues,
+// ParseBackground and ParseFont if applicable.
+bool Parser::ParseFontFamily(Values* values) {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return true;
+ DCHECK_LT(in_, end_);
+
+ UnicodeText family;
+ while (SkipToNextToken()) {
+ if (*in_ == ',') {
+ if (!family.empty()) {
+ values->push_back(new Value(Identifier(family)));
+ family.clear();
+ }
+ in_++;
+ } else {
+ scoped_ptr<Value> v(ParseAny());
+ if (!v.get()) return false;
+ switch (v->GetLexicalUnitType()) {
+ case Value::STRING:
+ if (!family.empty()) {
+ values->push_back(new Value(Identifier(family)));
+ family.clear();
+ }
+ values->push_back(v.release());
+ break;
+ case Value::IDENT:
+ if (!family.empty())
+ family.push_back(static_cast<char32>(' '));
+ family.append(v->GetIdentifierText());
+ break;
+ default:
+ return false;
+ }
+ }
+ }
+ if (!family.empty())
+ values->push_back(new Value(Identifier(family)));
+ return true;
+}
+
+// Parse font. It is special in that it uses a special format (see spec):
+// [ [ <'font-style'> || <'font-variant'> || <'font-weight'> ]?
+// <'font-size'> [ / <'line-height'> ]? <'font-family'> ]
+// | caption | icon | menu | message-box | small-caption | status-bar | inherit
+//
+// The output is a tuple in the following order:
+// "font-style font-variant font-weight font-size line-height font-family*"
+// or NULL if invalid
+// IE pecularity: font-family is optional (hence the *).
+// If you make any change to this function, please also update ParseValues,
+// ParseBackground and ParseFontFamily if applicable.
+Values* Parser::ParseFont() {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return NULL;
+ DCHECK_LT(in_, end_);
+
+ scoped_ptr<Values> values(new Values);
+
+ if (!SkipToNextToken())
+ return NULL;
+
+ scoped_ptr<Value> v(ParseAny());
+ if (!v.get()) return NULL;
+
+ // For special one-valued font: notations, just return with that one value.
+ // Note: these can be expanded by ExpandShorthandProperties
+ if (v->GetLexicalUnitType() == Value::IDENT) {
+ switch (v->GetIdentifier().ident()) {
+ case Identifier::CAPTION:
+ case Identifier::ICON:
+ case Identifier::MENU:
+ case Identifier::MESSAGE_BOX:
+ case Identifier::SMALL_CAPTION:
+ case Identifier::STATUS_BAR:
+ case Identifier::INHERIT:
+ // These special identifiers must be the only one in a declaration.
+ // Fail if there are others.
+ // TODO(sligocki): We should probably raise an error here.
+ if (SkipToNextToken()) return NULL;
+ // If everything is good, push these out.
+ values->push_back(v.release());
+ return values.release();
+ default:
+ break;
+ }
+ }
+
+ scoped_ptr<Value> font_style(new Value(Identifier::NORMAL));
+ scoped_ptr<Value> font_variant(new Value(Identifier::NORMAL));
+ scoped_ptr<Value> font_weight(new Value(Identifier::NORMAL));
+ scoped_ptr<Value> font_size(new Value(Identifier::MEDIUM));
+ scoped_ptr<Value> line_height(new Value(Identifier::NORMAL));
+ scoped_ptr<Value> font_family;
+
+ // parse style, variant and weight
+ while (true) {
+ // Firefox allows only one value to be set per property, IE need not.
+ if (v->GetLexicalUnitType() == Value::IDENT) {
+ switch (v->GetIdentifier().ident()) {
+ case Identifier::NORMAL:
+ // no-op
+ break;
+ case Identifier::ITALIC:
+ case Identifier::OBLIQUE:
+ font_style.reset(v.release());
+ break;
+ case Identifier::SMALL_CAPS:
+ font_variant.reset(v.release());
+ break;
+ case Identifier::BOLD:
+ case Identifier::BOLDER:
+ case Identifier::LIGHTER:
+ font_weight.reset(v.release());
+ break;
+ default:
+ goto check_fontsize;
+ }
+ } else if (v->GetLexicalUnitType() == Value::NUMBER &&
+ v->GetDimension() == Value::NO_UNIT) {
+ switch (v->GetIntegerValue()) {
+ // Different browsers handle this quite differently. But there
+ // is at least a test that is consistent between IE and
+ // firefox: try <span style="font:120 serif"> and <span
+ // style="font:100 serif">, the first one treats 120 as
+ // font-size, and the second does not.
+ case 100: case 200: case 300: case 400:
+ case 500: case 600: case 700: case 800:
+ case 900:
+ font_weight.reset(v.release());
+ break;
+ default:
+ goto check_fontsize;
+ }
+ } else {
+ goto check_fontsize;
+ }
+ if (!SkipToNextToken())
+ return NULL;
+ v.reset(ParseAny());
+ if (!v.get()) return NULL;
+ }
+
+ check_fontsize:
+ // parse font-size
+ switch (v->GetLexicalUnitType()) {
+ case Value::IDENT:
+ switch (v->GetIdentifier().ident()) {
+ case Identifier::XX_SMALL:
+ case Identifier::X_SMALL:
+ case Identifier::SMALL:
+ case Identifier::MEDIUM:
+ case Identifier::LARGE:
+ case Identifier::X_LARGE:
+ case Identifier::XX_LARGE:
+ case Identifier::LARGER:
+ case Identifier::SMALLER:
+ font_size.reset(v.release());
+ break;
+ default:
+ return NULL;
+ }
+ break;
+ case Value::NUMBER:
+ font_size.reset(v.release());
+ break;
+ default:
+ return NULL;
+ }
+
+ // parse line-height if '/' is seen, or use the default line-height
+ if (SkipToNextToken() && *in_ == '/') {
+ in_++;
+ if (!SkipToNextToken()) return NULL;
+ v.reset(ParseAny());
+ if (!v.get()) return NULL;
+
+ switch (v->GetLexicalUnitType()) {
+ case Value::IDENT:
+ if (v->GetIdentifier().ident() == Identifier::NORMAL)
+ break;
+ else
+ return NULL;
+ case Value::NUMBER:
+ line_height.reset(v.release());
+ break;
+ default:
+ return NULL;
+ }
+ }
+
+ values->push_back(font_style.release());
+ values->push_back(font_variant.release());
+ values->push_back(font_weight.release());
+ values->push_back(font_size.release());
+ values->push_back(line_height.release());
+
+ if (!ParseFontFamily(values.get())) // empty is okay.
+ return NULL;
+ return values.release();
+}
+
+static void ExpandShorthandProperties(Declarations* declarations,
+ const Declaration& declaration) {
+ Property prop = declaration.property();
+ const Values* vals = declaration.values();
+ bool important = declaration.IsImportant();
+
+ // Buffer to build up values used instead of vals above.
+ scoped_ptr<Values> edit_vals;
+ switch (prop.prop()) {
+ case Property::FONT: {
+ // Expand the value vector for special font: values.
+ if (vals->size() == 1) {
+ const Value* val = vals->at(0);
+ switch (val->GetIdentifier().ident()) {
+ case Identifier::CAPTION:
+ case Identifier::ICON:
+ case Identifier::MENU:
+ case Identifier::MESSAGE_BOX:
+ case Identifier::SMALL_CAPTION:
+ case Identifier::STATUS_BAR:
+ edit_vals.reset(new Values());
+ // Reasonable defaults to use for special font: declarations.
+ edit_vals->push_back(new Value(Identifier::NORMAL)); // font-style
+ edit_vals->push_back(new Value(Identifier::NORMAL)); // font-variant
+ edit_vals->push_back(new Value(Identifier::NORMAL)); // font-weight
+ // In this case, the actual font size will depend on browser,
+ // this is a common value found in IE and Firefox:
+ edit_vals->push_back(new Value(32.0/3, Value::PX)); // font-size
+ edit_vals->push_back(new Value(Identifier::NORMAL)); // line-height
+ // We store the special font type as font-family:
+ edit_vals->push_back(new Value(*val)); // font-family
+ vals = edit_vals.get(); // Move pointer to new, built-up values.
+ break;
+ case Identifier::INHERIT:
+ edit_vals.reset(new Values());
+ // font: inherit means all properties inherit.
+ edit_vals->push_back(new Value(*val)); // font-style
+ edit_vals->push_back(new Value(*val)); // font-variant
+ edit_vals->push_back(new Value(*val)); // font-weight
+ edit_vals->push_back(new Value(*val)); // font-size
+ edit_vals->push_back(new Value(*val)); // line-height
+ edit_vals->push_back(new Value(*val)); // font-family
+ vals = edit_vals.get(); // Move pointer to new, built-up values.
+ break;
+ default:
+ break;
+ }
+ }
+ // Only expand valid font: declarations (ones created by ParseFont, which
+ // requires at least 5 values in a specific order).
+ if (vals->size() < 5) {
+ LOG(ERROR) << "font: values are not in the correct format.\n" << vals;
+ break;
+ }
+ declarations->push_back(
+ new Declaration(Property::FONT_STYLE, *vals->get(0), important));
+ declarations->push_back(
+ new Declaration(Property::FONT_VARIANT, *vals->get(1), important));
+ declarations->push_back(
+ new Declaration(Property::FONT_WEIGHT, *vals->get(2), important));
+ declarations->push_back(
+ new Declaration(Property::FONT_SIZE, *vals->get(3), important));
+ declarations->push_back(
+ new Declaration(Property::LINE_HEIGHT, *vals->get(4), important));
+ if (vals->size() > 5) {
+ Values* family_vals = new Values;
+ for (int i = 5, n = vals->size(); i < n; ++i)
+ family_vals->push_back(new Value(*vals->get(i)));
+ declarations->push_back(
+ new Declaration(Property::FONT_FAMILY, family_vals, important));
+ }
+ }
+ break;
+ default:
+ // TODO(yian): other shorthand properties:
+ // background-position
+ // border-color border-style border-width
+ // border-top border-right border-bottom border-left
+ // border
+ // margin padding
+ // outline
+ break;
+ }
+}
+
+// Parse declarations like "background: white; color: #333; line-height: 1.3;"
+Declarations* Parser::ParseRawDeclarations() {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return new Declarations();
+ DCHECK_LT(in_, end_);
+
+ Declarations* declarations = new Declarations();
+ while (in_ < end_) {
+ bool ignore_this_decl = false;
+ switch (*in_) {
+ case ';':
+ in_++;
+ break;
+ case '}':
+ return declarations;
+ default: {
+ UnicodeText id = ParseIdent();
+ if (id.empty()) {
+ ignore_this_decl = true;
+ break;
+ }
+ Property prop(id);
+ SkipSpace();
+ if (Done() || *in_ != ':') {
+ ignore_this_decl = true;
+ break;
+ }
+ in_++;
+
+ Values* vals;
+ switch (prop.prop()) {
+ // TODO(sligocki): stop special-casing.
+ case Property::FONT:
+ vals = ParseFont();
+ break;
+ case Property::FONT_FAMILY:
+ vals = new Values();
+ if (!ParseFontFamily(vals) || vals->empty()) {
+ delete vals;
+ vals = NULL;
+ }
+ break;
+ default:
+ vals = ParseValues(prop.prop());
+ break;
+ }
+
+ if (vals == NULL) {
+ ignore_this_decl = true;
+ break;
+ }
+
+ bool important = false;
+ if (in_ < end_ && *in_ == '!') {
+ in_++;
+ SkipSpace();
+ UnicodeText ident = ParseIdent();
+ if (ident.utf8_length() == 9 &&
+ !memcasecmp(ident.utf8_data(), "important", 9))
+ important = true;
+ }
+ declarations->push_back(new Declaration(prop, vals, important));
+ }
+ }
+ SkipSpace();
+ if (ignore_this_decl) { // on bad syntax, we skip till the next declaration
+ errors_seen_mask_ |= kDeclarationError;
+ while (in_ < end_ && *in_ != ';' && *in_ != '}') {
+ // IE (and IE only) ignores {} blocks in quirks mode.
+ if (*in_ == '{' && !quirks_mode_) {
+ ParseBlock(); // ignore
+ } else {
+ in_++;
+ SkipSpace();
+ }
+ }
+ }
+ }
+ return declarations;
+}
+
+Declarations* Parser::ExpandDeclarations(Declarations* orig_declarations) {
+ scoped_ptr<Declarations> new_declarations(new Declarations);
+ for (int j = 0; j < orig_declarations->size(); ++j) {
+ // new_declarations takes ownership of declaration.
+ Declaration* declaration = orig_declarations->at(j);
+ orig_declarations->at(j) = NULL;
+ // TODO(yian): We currently store both expanded properties and the original
+ // property because only limited expansion is supported. In future, we
+ // should discard the original property after expansion.
+ new_declarations->push_back(declaration);
+ ExpandShorthandProperties(new_declarations.get(), *declaration);
+ // TODO(sligocki): Get ExpandBackground back into ExpandShorthandProperties.
+ switch (declaration->property().prop()) {
+ case Css::Property::BACKGROUND: {
+ ExpandBackground(*declaration, new_declarations.get());
+ break;
+ }
+ default:
+ break;
+ }
+ }
+ return new_declarations.release();
+}
+
+Declarations* Parser::ParseDeclarations() {
+ scoped_ptr<Declarations> orig_declarations(ParseRawDeclarations());
+ return ExpandDeclarations(orig_declarations.get());
+}
+
+// Starts from [ and parses to the closing ]
+// in [ foo ~= bar ].
+// Whitespace is not skipped at beginning or the end.
+SimpleSelector* Parser::ParseAttributeSelector() {
+ Tracer trace(__func__, &in_);
+
+ DCHECK_LT(in_, end_);
+ DCHECK_EQ('[', *in_);
+ in_++;
+ SkipSpace();
+
+ UnicodeText attr = ParseIdent();
+ SkipSpace();
+ scoped_ptr<SimpleSelector> newcond;
+ if (!attr.empty() && in_ < end_) {
+ char oper = *in_;
+ switch (*in_) {
+ case '~':
+ case '|':
+ case '^':
+ case '$':
+ case '*':
+ in_++;
+ if (Done() || *in_ != '=')
+ break;
+ // fall through
+ case '=': {
+ in_++;
+ UnicodeText value = ParseStringOrIdent();
+ if (!value.empty())
+ newcond.reset(SimpleSelector::NewBinaryAttribute(
+ SimpleSelector::AttributeTypeFromOperator(oper),
+ attr,
+ value));
+ break;
+ }
+ default:
+ newcond.reset(SimpleSelector::NewExistAttribute(attr));
+ break;
+ }
+ }
+ if (SkipPastDelimiter(']'))
+ return newcond.release();
+ else
+ return NULL;
+}
+
+SimpleSelector* Parser::ParseSimpleSelector() {
+ Tracer trace(__func__, &in_);
+
+ if (Done()) return NULL;
+ DCHECK_LT(in_, end_);
+
+ switch (*in_) {
+ case '#': {
+ in_++;
+ UnicodeText id = ParseIdent();
+ if (!id.empty())
+ return SimpleSelector::NewId(id);
+ break;
+ }
+ case '.': {
+ in_++;
+ UnicodeText classname = ParseIdent();
+ if (!classname.empty())
+ return SimpleSelector::NewClass(classname);
+ break;
+ }
+ case ':': {
+ in_++;
+ UnicodeText pseudoclass = ParseIdent();
+ // FIXME(yian): skip constructs "(en)" in lang(en) for now.
+ if (in_ < end_ && *in_ == '(') {
+ in_++;
+ SkipSpace();
+ ParseIdent();
+ if (!SkipPastDelimiter(')'))
+ break;
+ }
+ if (!pseudoclass.empty())
+ return SimpleSelector::NewPseudoclass(pseudoclass);
+ break;
+ }
+ case '[': {
+ SimpleSelector* newcond = ParseAttributeSelector();
+ if (newcond)
+ return newcond;
+ break;
+ }
+ case '*':
+ in_++;
+ return SimpleSelector::NewUniversal();
+ break;
+ default: {
+ UnicodeText ident = ParseIdent();
+ if (!ident.empty())
+ return SimpleSelector::NewElementType(ident);
+ break;
+ }
+ }
+ // We did not parse anything or we parsed something incorrectly.
+ return NULL;
+}
+
+bool Parser::AtValidSimpleSelectorsTerminator() const {
+ if (Done()) return true;
+ switch (*in_) {
+ case ' ': case '\t': case '\r': case '\n': case '\f':
+ case ',': case '{': case '>': case '+':
+ return true;
+ case '/':
+ if (in_ + 1 < end_ && *(in_ + 1) == '*')
+ return true;
+ break;
+ }
+ return false;
+}
+
+SimpleSelectors* Parser::ParseSimpleSelectors(bool expecting_combinator) {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return NULL;
+ DCHECK_LT(in_, end_);
+
+ SimpleSelectors::Combinator combinator;
+ if (!expecting_combinator)
+ combinator = SimpleSelectors::NONE;
+ else
+ switch (*in_) {
+ case '>':
+ in_++;
+ combinator = SimpleSelectors::CHILD;
+ break;
+ case '+':
+ in_++;
+ combinator = SimpleSelectors::SIBLING;
+ break;
+ default:
+ combinator = SimpleSelectors::DESCENDANT;
+ break;
+ }
+
+ scoped_ptr<SimpleSelectors> selectors(
+ new SimpleSelectors(combinator));
+
+ SkipSpace();
+ if (Done()) return NULL;
+
+ const char* oldin = in_;
+ while (SimpleSelector* simpleselector = ParseSimpleSelector()) {
+ selectors->push_back(simpleselector);
+ oldin = in_;
+ }
+
+ if (selectors->size() > 0 && // at least one simple selector stored
+ in_ == oldin && // the last NULL does not make progress
+ AtValidSimpleSelectorsTerminator()) // stop at a valid terminator
+ return selectors.release();
+
+ return NULL;
+}
+
+Selectors* Parser::ParseSelectors() {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return NULL;
+ DCHECK_LT(in_, end_);
+
+ // Remember whether anything goes wrong, but continue parsing until the
+ // declaration starts or the position comes to the end. Then discard the
+ // selectors.
+ bool success = true;
+
+ scoped_ptr<Selectors> selectors(new Selectors());
+ Selector* selector = new Selector();
+ selectors->push_back(selector);
+
+ // The first simple selector sequence in a chain of simple selector
+ // sequences does not have a combinator. ParseSimpleSelectors needs
+ // to know this, so we set this to false here and after ',', and
+ // true after we see a simple selector sequence.
+ bool expecting_combinator = false;
+ while (in_ < end_ && *in_ != '{') {
+ switch (*in_) {
+ case ',':
+ if (selector->size() == 0) {
+ success = false;
+ } else {
+ selector = new Selector();
+ selectors->push_back(selector);
+ }
+ in_++;
+ expecting_combinator = false;
+ break;
+ default: {
+ const char* oldin = in_;
+ SimpleSelectors* simple_selectors
+ = ParseSimpleSelectors(expecting_combinator);
+ if (!simple_selectors) {
+ success = false;
+ if (in_ == oldin)
+ in_++;
+ } else {
+ selector->push_back(simple_selectors);
+ }
+
+ expecting_combinator = true;
+ break;
+ }
+ }
+ SkipSpace();
+ }
+
+ if (selector->size() == 0)
+ success = false;
+
+ if (success)
+ return selectors.release();
+ else
+ return NULL;
+}
+
+Ruleset* Parser::ParseRuleset() {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return NULL;
+ DCHECK_LT(in_, end_);
+
+ // Remember whether anything goes wrong, but continue parsing until the
+ // closing }. Then discard the whole ruleset if necessary. This allows the
+ // parser to make progress anyway.
+ bool success = true;
+
+ scoped_ptr<Ruleset> ruleset(new Ruleset());
+ scoped_ptr<Selectors> selectors(ParseSelectors());
+
+ if (Done())
+ return NULL;
+
+ if (selectors.get() == NULL) {
+ // http://www.w3.org/TR/CSS21/syndata.html#rule-sets
+ // When a user agent can't parse the selector (i.e., it is not
+ // valid CSS 2.1), it must ignore the declaration block as
+ // well.
+ success = false;
+ errors_seen_mask_ |= kSelectorError;
+ } else {
+ ruleset->set_selectors(selectors.release());
+ }
+
+ in_++; // '{'
+ ruleset->set_declarations(ParseRawDeclarations());
+ SkipPastDelimiter('}');
+
+ if (success)
+ return ruleset.release();
+ else
+ return NULL;
+}
+
+void Parser::ParseMediumList(std::vector<UnicodeText>* media) {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return;
+ DCHECK_LT(in_, end_);
+
+ while (in_ < end_) {
+ switch (*in_) {
+ case ';':
+ case '{':
+ return;
+ case ',':
+ in_++;
+ break;
+ default:
+ scoped_ptr<Value> v(ParseAny());
+ if (v.get() && v->GetLexicalUnitType() == Value::IDENT)
+ media->push_back(v->GetIdentifierText());
+ else
+ errors_seen_mask_ |= kMediaError;
+ break;
+ }
+ SkipSpace();
+ }
+}
+
+// Start after @import is parsed.
+Import* Parser::ParseImport() {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return NULL;
+ DCHECK_LT(in_, end_);
+
+ scoped_ptr<Value> v(ParseAny());
+ if (!v.get() || (v->GetLexicalUnitType() != Value::STRING &&
+ v->GetLexicalUnitType() != Value::URI))
+ return NULL;
+
+ Import* import = new Import();
+ import->link = v->GetStringValue();
+
+ ParseMediumList(&import->media);
+ if (in_ < end_ && *in_ == ';') in_++;
+ return import;
+}
+
+void Parser::ParseAtrule(Stylesheet* stylesheet) {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ DCHECK_LT(in_, end_);
+ DCHECK_EQ('@', *in_);
+ in_++;
+
+ UnicodeText ident = ParseIdent();
+
+ // @import string|uri medium-list ? ;
+ if (ident.utf8_length() == 6 &&
+ memcasecmp(ident.utf8_data(), "import", 6) == 0) {
+ scoped_ptr<Import> import(ParseImport());
+ if (import.get() && stylesheet)
+ stylesheet->mutable_imports().push_back(import.release());
+
+ // @charset string ;
+ } else if (ident.utf8_length() == 7 &&
+ memcasecmp(ident.utf8_data(), "charset", 7) == 0) {
+ SkipPastDelimiter(';');
+
+ // @media medium-list { ruleset-list }
+ } else if (ident.utf8_length() == 5 &&
+ memcasecmp(ident.utf8_data(), "media", 5) == 0) {
+ std::vector<UnicodeText> media;
+ ParseMediumList(&media);
+ if (Done() || *in_ != '{')
+ return;
+ in_++;
+ SkipSpace();
+ while (in_ < end_ && *in_ != '}') {
+ const char* oldin = in_;
+ scoped_ptr<Ruleset> ruleset(ParseRuleset());
+ if (!ruleset.get() && in_ == oldin)
+ in_++;
+ if (ruleset.get()) {
+ ruleset->set_media(media);
+ stylesheet->mutable_rulesets().push_back(ruleset.release());
+ }
+ SkipSpace();
+ }
+ if (in_ < end_) in_++;
+
+ // @page pseudo_page? { declaration-list }
+ } else if (ident.utf8_length() == 4 &&
+ memcasecmp(ident.utf8_data(), "page", 4) == 0) {
+ delete ParseRuleset();
+ }
+}
+
+// TODO(dpeng): What exactly does this code do?
+void Parser::ParseBlock() {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ DCHECK_LT(in_, end_);
+ DCHECK_EQ('{', *in_);
+ int depth = 0;
+ while (in_ < end_) {
+ switch (*in_) {
+ case '{':
+ in_++;
+ depth++;
+ break;
+ case '@':
+ in_++;
+ ParseIdent();
+ break;
+ case ';':
+ in_++;
+ break;
+ case '}':
+ in_++;
+ depth--;
+ if (depth == 0)
+ return;
+ break;
+ default:
+ scoped_ptr<Value> v(ParseAny());
+ break;
+ }
+ SkipSpace();
+ }
+}
+
+Stylesheet* Parser::ParseRawStylesheet() {
+ Tracer trace(__func__, &in_);
+
+ SkipSpace();
+ if (Done()) return new Stylesheet();
+ DCHECK_LT(in_, end_);
+
+ Stylesheet* stylesheet = new Stylesheet();
+ while (in_ < end_) {
+ switch (*in_) {
+ case '<':
+ in_++;
+ if (end_ - in_ >= 3 && memcmp(in_, "!--", 3) == 0) {
+ in_ += 3;
+ }
+ break;
+ case '-':
+ in_++;
+ if (end_ - in_ >= 2 && memcmp(in_, "->", 2) == 0) {
+ in_ += 2;
+ }
+ break;
+ case '@':
+ ParseAtrule(stylesheet);
+ break;
+ default: {
+ const char* oldin = in_;
+ scoped_ptr<Ruleset> ruleset(ParseRuleset());
+ if (!ruleset.get() && oldin == in_)
+ in_++;
+ if (ruleset.get())
+ stylesheet->mutable_rulesets().push_back(ruleset.release());
+ break;
+ }
+ }
+ SkipSpace();
+ }
+ return stylesheet;
+}
+
+Stylesheet* Parser::ParseStylesheet() {
+ Tracer trace(__func__, &in_);
+
+ Stylesheet* stylesheet = ParseRawStylesheet();
+
+ Rulesets& rulesets = stylesheet->mutable_rulesets();
+ for (int i = 0; i < rulesets.size(); ++i) {
+ Declarations& orig_declarations = rulesets[i]->mutable_declarations();
+ rulesets[i]->set_declarations(ExpandDeclarations(&orig_declarations));
+ }
+
+ return stylesheet;
+}
+
+//
+// Some destructors that need STLDeleteElements() from stl_util-inl.h
+//
+
+Declarations::~Declarations() { STLDeleteElements(this); }
+Rulesets::~Rulesets() { STLDeleteElements(this); }
+Imports::~Imports() { STLDeleteElements(this); }
+
+} // namespace
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/parser.h b/trunk/src/third_party/css_parser/src/webutil/css/parser.h
new file mode 100644
index 0000000..e421dd7
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/parser.h
@@ -0,0 +1,642 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2006 Google Inc. All Rights Reserved.
+// Author: dpeng@google.com (Daniel Peng)
+
+#ifndef WEBUTIL_CSS_PARSER_H__
+#define WEBUTIL_CSS_PARSER_H__
+
+#include <string>
+#include <vector>
+
+#include "base/scoped_ptr.h"
+#include "strings/stringpiece.h"
+#include "testing/production_stub/public/gunit_prod.h"
+#include "util/utf8/public/unicodetext.h"
+#include "webutil/css/property.h" // while these CSS includes can be
+#include "webutil/css/selector.h" // forward-declared, who is really
+#include "webutil/css/string.h"
+#include "webutil/css/value.h" // going to want parser.h but not values.h?
+#include "webutil/html/htmlcolor.h"
+
+namespace Css {
+
+// These are defined below Parser.
+struct Import;
+class Declaration;
+class Declarations;
+class Stylesheet;
+class Ruleset;
+
+// Recursive descent parser for CSS.
+// Based on: http://www.w3.org/TR/CSS21/syndata.html
+// http://www.w3.org/TR/CSS21/grammar.html
+//
+// Say you want to parse a fragment of CSS. Then construct a new
+// Parser object (this is very lightweight) and pass in the fragment to parse.
+// Then, call the top-level ParseXXX() function for what you want to parse.
+// This parses the fragment and returns a pointer to the abstract syntax tree.
+// You own this pointer and must delete it when you're done.
+//
+// The data structures comprising the abstract syntax tree are described in
+// cssvalue.h, cssparser-inl.h, csscondition.h, and cssproperty.h.
+//
+// Essentially, each stylesheet is a collection of rulesets.
+
+// Each ruleset has some selectors to describe what HTML elements it
+// applies to and some declarations to describe how the HTML elements
+// should be styled.
+// The ruleset may apply to multiple comma-separated selectors, which
+// means they apply to an element matching any of the selectors.
+// Ex: h1, h2 > p, h3 { color: red; }
+//
+// Each selector consists of a chain of simple selectors, separated by
+// combinators.
+// Ex: h2 > p selects a P element that is a child of an H2 element.
+//
+// Each simple selector may have conditions which impose further
+// restrictions, such as [foo], #id, .class, or :hover. We represent
+// these as a list, which are semantically AND-ed together.
+//
+// Each declaration consists of a property and a list of values.
+//
+// Say, for example, you wish to parse a CSS declaration:
+// Parser a("background: white; color: #333; line-height: 1.3;
+// text-align: justify; font-family: \"Gill Sans MT\",
+// \"Gill Sans\", GillSans, Arial, Helvetica, sans-serif");
+// scoped_ptr<Declarations> t(a.ParseDeclarations());
+// See the 'declarations' unit test case for more details.
+//
+// We've made most of the recursive-descent ParseXXX functions private
+// to shrink the size of the public interface. We expose
+// ParseStylesheet() and ParseDeclarations() because those are the
+// top-level functions necessary to parse stylesheets in HTML
+// documents. And ParseSelectors is exposed to parse selectors. If it's
+// useful to expose more of the functions, please just send a CL for approval,
+// so we know what people depend on.
+//
+// The CSS parser runs in either quirks mode (the default) and standard
+// compliant mode. The latter is stricter in many aspects. Currently, it
+// affects color parsing (see below for details). Please refer to:
+// http://developer.mozilla.org/en/docs/Mozilla_Quirks_Mode_Behavior
+// for the difference in Mozilla browsers.
+class Parser {
+ public:
+ Parser(const char* utf8text, const char* textend);
+ explicit Parser(const char* utf8text);
+ explicit Parser(StringPiece s);
+
+ // ParseRawSytlesheet and ParseStylesheet consume the entire document and
+ // return a Stylesheet* containing all the imports and rulesets that it
+ // found. You must delete the return pointer.
+
+ // ParseRawStylesheet simply parses the document into an abstract syntax tree.
+ Stylesheet* ParseRawStylesheet();
+ // ParseStylesheet also runs a second pass to convert shorthand
+ // declarations such as background, font and font-family into sets of
+ // declarations that they represent.
+ Stylesheet* ParseStylesheet();
+
+ // ParserRawDeclarations and ParseDeclarations parse declarations like
+ // "background: white; color: #333; line-height: 1.3;", consuming until
+ // (but not including) the closing '}' or EOF. You must delete the return
+ // pointer.
+
+ // ParseRawDeclarations simply parses the declarations into an AST.
+ Declarations* ParseRawDeclarations();
+
+ // ParseDeclarations also runs a second pass to convert *some* syntactic
+ // sugar declarations such as background, font and font-family.
+ // Currently, both the expanded properties (such as background-color) and the
+ // original property (background) are stored because the impl. is incomplete.
+ // For details, see parser.cc.
+ Declarations* ParseDeclarations();
+
+ // Expand the values of shorthand declarations. Currently expands background
+ // and font. Clears (but does not delete) input orig_declarartions in the
+ // process. orig_declarations should be a std::vector of NULLs on exit.
+ Declarations* ExpandDeclarations(Declarations* orig_declarations);
+
+ // Starting at the first simple selector or whitespace, ParseSelectors parses
+ // a sequence of selectors. Return NULL if the parsing fails. The parser would
+ // consume anything up to the declaration starting '{' or the end of document.
+ Selectors* ParseSelectors();
+
+ // current position in the parse.
+ const char* getpos() const { return in_; }
+
+ // Done with the parse?
+ bool Done() const { return in_ == end_; }
+
+ // Whether quirks mode (the default) is used in parsing. Standard compliant
+ // (non-quirks) mode is stricter in color parsing, where a form of "rrgbbb"
+ // without a leading # is not allowed.
+ bool quirks_mode() const { return quirks_mode_; }
+ void set_quirks_mode(bool quirks_mode) { quirks_mode_ = quirks_mode; }
+
+ // This is a bitmask of errors seen during the parse. This is decidedly
+ // incomplete --- there are definitely many errors that are not reported here.
+ static const uint64 kNoError = 0;
+ static const uint64 kUtf8Error = 1ULL << 0;
+ static const uint64 kDeclarationError = 1ULL << 1;
+ static const uint64 kSelectorError = 1ULL << 2;
+ static const uint64 kFunctionError = 1ULL << 3;
+ static const uint64 kMediaError = 1ULL << 4;
+ uint64 errors_seen_mask() const { return errors_seen_mask_; }
+
+ friend class ParserTest; // we need to unit test private Parse functions.
+
+ private:
+ //
+ // Syntactic methods
+ //
+
+ // SkipSpace() skips whitespace ([ \t\r\n\f]) and comments
+ // (/* .... */) until we reach a non-whitespace, non-comment
+ // character, or the end of the document.
+ void SkipSpace();
+
+ // Starting at /*, SkipComment() skips past the matching */ or to
+ // the end of the document.
+ void SkipComment();
+
+ // Skips following characters until delimiter delim or end is seen, delim is
+ // consumed if found. Be smart enough not to stop at character delim in
+ // comments. Returns whether delim is actually seen.
+ bool SkipPastDelimiter(char delim);
+
+ // Skips whitespace, comments, blocks ({..}), and @tokens, and returns true
+ // unless we are at the end of the document or the next character is a token
+ // ending delimiter ([;}!]).
+ bool SkipToNextToken();
+
+ //
+ // Parse functions.
+ //
+ // When the comment reads 'starting at foo', it's a dchecked runtime
+ // error to call the function if the input does not start with
+ // 'foo'.
+ //
+ // If a ParseXXX method returns a pointer, you own it and must
+ // delete it.
+
+ //
+ // 'leaves' of the parse tree: strings, urls, identifiers, numbers,
+ // etc
+ //
+
+ // ParseIdent() consumes the identifier and returns its unescaped
+ // representation. If we are at the end of the document, or if no
+ // identifier is found, ParseIdent() returns the empty string.
+ //
+ // In CSS2, identifiers (including element names, classes, and IDs in
+ // selectors) can contain only the characters [A-Za-z0-9] and ISO
+ // 10646 characters 161 and higher, plus the hyphen (-); they cannot
+ // start with a hyphen or a digit. They can also contain escaped
+ // characters and any ISO 10646 character as a numeric code (see next
+ // item). For instance, the identifier "B&W?" may be written as
+ // "B\&W\?" or "B\26 W\3F".
+ // http://www.w3.org/TR/REC-CSS2/syndata.html#value-def-identifier
+ //
+ // We're a little more forgiving than the standard and permit hyphens
+ // and digits to start identifiers.
+ // This method does not skip spaces like most other methods do, because it
+ // may be used to identify things like "import" in "@import", which is
+ // different from "@ import".
+ UnicodeText ParseIdent(); // parse an identifier like justify
+
+ // Starting at \, parse the escape and return the corresponding
+ // unicode codepoint. If the \ is the last character in the
+ // document, we return '\'; there is no other malformed input. This
+ // implements the second and third types of character escapes at
+ // http://www.w3.org/TR/REC-CSS2/syndata.html#escaped-characters
+ //
+ // 2) It cancels the meaning of special CSS characters. Any
+ // character (except a hexadecimal digit) can be escaped with a
+ // backslash to remove its special meaning. For example,
+ // ParseEscape() returns 0x6240 for \所 and 71 for \G (but \C is a
+ // hex escape, treated below:)
+ //
+ // 3) Backslash escapes allow authors to refer to characters
+ // they can't easily put in a document. In this case, the backslash
+ // is followed by at most six hexadecimal digits (0..9A..Fa..f), which
+ // stand for the ISO 10646 ([ISO10646]) character with that
+ // number. If a digit or letter follows the hexadecimal number, the
+ // end of the number needs to be made clear. There are two ways to
+ // do that:
+ // 1. with a space (or other whitespace character): "\26 B" ("&B")
+ // 2. by providing exactly 6 hexadecimal digits: "\000026B" ("&B")
+ //
+ // So, if the escape sequence is a hex escape and the character following
+ // the last hex digit is a space, then ParseEscape() consumes it.
+ char32 ParseEscape(); // return the codepoint for the current escape \12a76f
+
+ // Starting at delim, ParseString<char delim>() consumes the string,
+ // including the matching end-delim, and returns its unescaped
+ // representation, without the delimiters. If we fail to find the
+ // matching delimiter, we consume the rest of the document and
+ // return it.
+ //
+ // Strings can either be written with double quotes or with single
+ // quotes. Double quotes cannot occur inside double quotes, unless
+ // escaped (as '\"' or as '\22'). Analogously for single quotes
+ // ("\'" or "\27"). A string cannot directly contain a newline,
+ // unless hex-escaped as "\A".
+ //
+ // It is possible to break strings over several lines, for aesthetic
+ // or other reasons, but in such a case the newline itself has to be
+ // escaped with a backslash (\). For instance, the following two
+ // selectors are exactly the same:
+ // http://www.w3.org/TR/REC-CSS2/syndata.html#strings
+ template<char delim> UnicodeText ParseString();
+
+ // If the current character is a string-delimiter (' or "),
+ // ParseStringOrIdent() parses a string and returns the contents.
+ // Otherwise, it tries to parse an identifier. We must not be at
+ // the end of the document.
+ UnicodeText ParseStringOrIdent();
+
+ // ParseNumber parses a number and an optional unit, consuming to
+ // the end of the number or unit and returning a Value*.
+ // Real numbers and integers are specified in decimal notation
+ // only. An <integer> consists of one or more digits "0" to "9". A
+ // <number> can either be an <integer>, or it can be zero or more
+ // digits followed by a dot (.) followed by one or more digits. Both
+ // integers and real numbers may be preceded by a "-" or "+" to
+ // indicate the sign.
+ //
+ // If no number is found, ParseNumber returns NULL.
+ Value* ParseNumber();
+
+ // ParseColor parses several different representations of colors:
+ // 1) rgb
+ // 2) #rgb
+ // 3) rrggbb
+ // 4) #rrggbb
+ // 5) The 16 HTML4 color names (aqua, black, blue,
+ // fuchsia, gray, green, lime, maroon, navy, olive, purple, red,
+ // silver, teal, white, and yellow), with or without quotes (' or ").
+ // It's designed to handle all the ill-formed CSS color values out there.
+ // It consumes the color if it finds a valid color. Otherwise, it returns
+ // an undefined HtmlColor (HtmlColor::IsDefined()) and does not consume
+ // anything.
+ //
+ // However, if quirks_mode_ is false (standard compliant mode), forms 1 and 3
+ // (without #) would not be accepted.
+ HtmlColor ParseColor(); // parse a hex or named
+ // color like #fff, #bcdefa
+ // or black
+
+ //
+ // FUNCTION-like objects: rgb(), url(), rect()
+ //
+
+ // Converts a Value number or percentage to an RGB value.
+ static unsigned char ValueToRGB(Value* v);
+
+ // ParseRgbColor parsers the part between the parentheses of rgb( )
+ // according to http://www.w3.org/TR/REC-CSS2/syndata.html#color-units .
+ //
+ // The format of an RGB value in the functional notation is 'rgb('
+ // followed by a comma-separated list of three numerical values
+ // (either three integer values or three percentage values)
+ // followed by ')'. The integer value 255 corresponds to 100%, and
+ // to F or FF in the hexadecimal notation: rgb(255,255,255) =
+ // rgb(100%,100%,100%) = #FFF. Whitespace characters are allowed
+ // around the numerical values.
+ //
+ // Starting just past 'rgb(', ParseColor() consumes up to (but not
+ // including) the closing ) and returns the color it finds.
+ // Returns NULL if mal-formed.
+ Value* ParseRgbColor(); // parse an rgbcolor like 125, 25, 12
+ // or 12%, 57%, 89%
+
+ // ParseUrl parses the part between the parentheses of url( )
+ // according to http://www.w3.org/TR/REC-CSS2/syndata.html#uri .
+ //
+ // The format of a URI value is 'url(' followed by optional
+ // whitespace followed by an optional single quote (') or double
+ // quote (") character followed by the URI itself, followed by an
+ // optional single quote (') or double quote (") character followed
+ // by optional whitespace followed by ')'. The two quote characters
+ // must be the same.
+ //
+ // Starting just past 'url(', ParseUrl() consumes the url as well as
+ // the optional whitespace. If the url is well-formed, the next
+ // character must be ')'.
+ // Returns NULL for mal-formed URLs.
+ Value* ParseUrl(); // parse a url like yellow.png or 'blah.png'
+
+ // Parses between the parentheses of rect(top, right, bottom, left).
+ //
+ // The contents should be a comma or space separated list of four numerical
+ // values or the keyword "auto". Note that spaces are allowed to separate
+ // values for historical reasons.
+ //
+ // Returns NULL if the contents is not well-formed.
+ Value* ParseRect(); // parse rect(top, right, bottom, left)
+
+ //
+ // Value and Values
+ //
+
+ // Parses a value which is expected to be color values. It can be
+ // different from ParseAny, for example, for black or ccddff, both
+ // are translated into color values here but are returned as idents
+ // in the latter case. We call this instead of ParseAny() after
+ // color, background-color, and background properties to accomodate bad CSS.
+ // If no value is found, ParseAnyExpectingColor returns NULL.
+ Value* ParseAnyExpectingColor();
+
+ // ParseAny() parses a css value and consumes it. It does not skip
+ // leading or trailing whitespace.
+ // If no value is found, ParseAny returns NULL and make sure at least one
+ // character is consumed (to make progress).
+ Value* ParseAny(); // parse a value, which is pretty much anything.
+
+ // Parse a list of values for the given property.
+ // We parse until we see a !, ;, or } delimiter. However, if there are any
+ // malformed values, stop parsing and return NULL immediately.
+ // For special shortcut properties, use the following specialized methods
+ // instead.
+ Values* ParseValues(Property::Prop prop);
+
+ // Expand a background property into all the sub-properties (background-color,
+ // background-image, etc.). Return false on malformed original_declaration.
+ static bool ExpandBackground(const Declaration& original_declaration,
+ Declarations* new_declarations);
+
+ // Parses FONT. Returnss NULL if malformed. Otherwise, the output is a tuple
+ // in the following order
+ // "font-style font-variant font-weight font-size line-height font-family+"
+ Values* ParseFont();
+
+ // Parses FONT-FAMILY and the tailing part in FONT and appends the results in
+ // values. Returns false if there are any malformed values.
+ // This interface is different from the others because it is also used by
+ // ParseFont(), where family names are appended to other CSS values.
+ bool ParseFontFamily(Values* values);
+
+ //
+ // Selectors and Rulesets
+ //
+
+ // ParseAttributeSelector() starts at [ and parses an attribute
+ // selector like [ foo ~= bar], consuming the final ]. Returns NULL
+ // on error but still consumes to the matching ].
+ // This method does not skip spaces like most other methods do.
+ // Whitespace is syntactically significant here, because a sequence of simple
+ // selectors contains no whitespace. 'div[align=center]' is a sequence of
+ // simple selectors, but 'div [align=center]' is a syntax error (though we
+ // will parse it as a selector, i.e., two simple selector sequences separated
+ // by a whitespace combinator).
+ SimpleSelector* ParseAttributeSelector();
+
+ // ParseSimpleSelector() parses one simple sector. Starts from
+ // anything and returns NULL if no simple selector found or parse error.
+ // This method does not skip spaces like most other methods do.
+ // See comment above.
+ SimpleSelector* ParseSimpleSelector();
+
+ // Checks if the parser stops at a character (or characters) that will
+ // legally terminate a SimpleSelectors. The checked characters are not eaten.
+ // Valid terminators are whitespaces, comments, combinators ('>', '+'), ','
+ // and '{'. A stop at the end is also considered valid.
+ bool AtValidSimpleSelectorsTerminator() const;
+
+ // Starting at whitespace, a combinator, or the first simple
+ // selector, ParseSimpleSelectors parses a sequence of simple
+ // selectors, i.e., a chain of simple selectors that are not
+ // separated by a combinator. The chain itself may be preceeded by
+ // a combinator, in which case you should pass true for
+ // expecting_combinator, and we will parse the combinator.
+ // Typically, when you're parsing a selector (i.e., a chain of
+ // sequences of simple selectors separated by combinators), you pass
+ // false on the first simple selector and true on the subsequent
+ // ones.
+ SimpleSelectors* ParseSimpleSelectors(bool expecting_combinator);
+
+ // ParseRuleset() starts from the first character of the first
+ // selector (note: it does not skip whitespace) and consumes the
+ // ruleset, including the closing '}'. Return NULL if the parsing fails.
+ // However, the parser would consume anything up to the closing '}', if any,
+ // even if it fails somehow in the middle, per CSS spec.
+ Ruleset* ParseRuleset();
+
+ //
+ // Miscellaneous
+ //
+
+ // Starting at whitespace or the first medium, ParseMediumList
+ // parses a medium list separated by commas and whitespace, stopping
+ // at (without consuming) ; or {. It adds each medium to the back of media.
+ void ParseMediumList(std::vector<UnicodeText>* media);
+
+ // ParseImport starts just after @import and consumes the import
+ // declaration, including the closing ;. It returns a Import*
+ // containing the imported name and the media.
+ Import* ParseImport();
+
+ // Starting at @, ParseAtrule parses @import, @charset, @medium, and
+ // @page declarations and adds the information to the stylesheet.
+ // Consumes the @-rule, including the closing ';' or '}'. Does not
+ // consume trailing whitespace.
+ void ParseAtrule(Stylesheet* stylesheet); // parse @ rules.
+
+ // Starting at '{', ParseBlock consumes to the matching '}', respecting
+ // nested blocks. We discard the result.
+ void ParseBlock();
+
+ const char *in_; // The current point in the parse.
+ const char *end_; // The end of the document to parse.
+
+ bool quirks_mode_; // Whether we are in quirks mode.
+ uint64 errors_seen_mask_;
+
+ FRIEND_TEST(ParserTest, color);
+ FRIEND_TEST(ParserTest, url);
+ FRIEND_TEST(ParserTest, rect);
+ FRIEND_TEST(ParserTest, background);
+ FRIEND_TEST(ParserTest, font_family);
+ friend void ParseFontFamily(Parser* parser);
+ FRIEND_TEST(ParserTest, ParseBlock);
+ FRIEND_TEST(ParserTest, font);
+ FRIEND_TEST(ParserTest, values);
+ FRIEND_TEST(ParserTest, declarations);
+ FRIEND_TEST(ParserTest, universalselector);
+ FRIEND_TEST(ParserTest, universalselectorcondition);
+ FRIEND_TEST(ParserTest, comment_breaking_descendant_combinator);
+ FRIEND_TEST(ParserTest, comment_breaking_child_combinator);
+ FRIEND_TEST(ParserTest, simple_selectors);
+ FRIEND_TEST(ParserTest, bad_simple_selectors);
+ FRIEND_TEST(ParserTest, rulesets);
+ FRIEND_TEST(ParserTest, ruleset_starts_with_combinator);
+ FRIEND_TEST(ParserTest, atrules);
+ FRIEND_TEST(ParserTest, percentage_colors);
+ DISALLOW_COPY_AND_ASSIGN(Parser);
+};
+
+// Definitions of various data structures returned by the parser.
+// More in selector.h and value.h.
+
+// A single declaration such as font: 12pt Arial.
+// A declaration consists of a property name (Property) and a list
+// of values (Values*).
+// It could also be important (font: 12pt Arial !important).
+class Declaration {
+ public:
+ // constructor. We take ownership of v.
+ Declaration(Property p, Values* v, bool important) :
+ property_(p), values_(v), important_(important) { }
+ // constructor with a single Value. We make a copy of the value.
+ Declaration(Property p, const Value& v, bool important)
+ : property_(p), values_(new Values), important_(important) {
+ values_->push_back(new Value(v));
+ }
+
+ // accessors
+ Property property() const { return property_; }
+ const Values* values() const { return values_.get(); }
+ bool IsImportant() const { return important_; }
+
+ // convenience accessors
+ Property::Prop prop() const { return property_.prop(); }
+ string prop_text() const { return property_.prop_text(); }
+
+ Values* mutable_values() { return values_.get(); }
+
+ void set_property(Property property) { property_ = property; }
+ // Takes ownership of values.
+ void set_values(Values* values) { values_.reset(values); }
+ void set_important(bool important) { important_ = important; }
+
+ string ToString() const;
+ private:
+ Property property_;
+ scoped_ptr<Values> values_;
+ bool important_; // Whether !important is declared on this declaration.
+
+ DISALLOW_COPY_AND_ASSIGN(Declaration);
+};
+
+// Declarations is a vector of Declaration*, which we own and
+// will delete upon destruction. If you remove elements from
+// Declarations, you are responsible for deleting them.
+// Also, be careful --- there's no virtual destructor, so this must be
+// deleted as a Declarations.
+class Declarations : public std::vector<Declaration*> {
+ public:
+ Declarations() : std::vector<Declaration*>() { }
+ ~Declarations();
+
+ // We provide syntactic sugar for accessing elements.
+ // declarations->get(i) looks better than (*declarations)[i])
+ const Declaration* get(int i) const { return (*this)[i]; }
+
+ string ToString() const;
+ private:
+ DISALLOW_COPY_AND_ASSIGN(Declarations);
+};
+
+// A ruleset consists of a list of selectors followed by a declaration block.
+// It can also optionally include a list of medium description.
+class Ruleset {
+ public:
+ Ruleset() : selectors_(new Selectors), declarations_(new Declarations) { }
+ // Takes ownership of selectors and declarations.
+ Ruleset(Selectors* selectors, const std::vector<UnicodeText>& media,
+ Declarations* declarations)
+ : selectors_(selectors), media_(media), declarations_(declarations) { }
+ ~Ruleset() { }
+
+ const Selectors& selectors() const { return *selectors_; }
+ const Selector& selector(int i) const { return *selectors_->at(i); }
+ const std::vector<UnicodeText>& media() const { return media_; }
+ const UnicodeText& medium(int i) const { return media_.at(i); }
+ const Declarations& declarations() const { return *declarations_; }
+ const Declaration& declaration(int i) const { return *declarations_->at(i); }
+
+ Selectors& mutable_selectors() { return *selectors_; }
+ Declarations& mutable_declarations() { return *declarations_; }
+
+ // set_media copies input media.
+ void set_media(const std::vector<UnicodeText>& media) {
+ media_.assign(media.begin(), media.end());
+ }
+ // set_selectors and _declarations take ownership of parameters.
+ void set_selectors(Selectors* selectors) { selectors_.reset(selectors); }
+ void set_declarations(Declarations* decls) { declarations_.reset(decls); }
+
+ string ToString() const;
+ private:
+ scoped_ptr<Selectors> selectors_;
+ std::vector<UnicodeText> media_;
+ scoped_ptr<Declarations> declarations_;
+
+ DISALLOW_COPY_AND_ASSIGN(Ruleset);
+};
+
+class Rulesets : public std::vector<Css::Ruleset*> {
+ public:
+ Rulesets() : std::vector<Css::Ruleset*>() { }
+ ~Rulesets();
+};
+
+struct Import {
+ std::vector<UnicodeText> media;
+ UnicodeText link;
+
+ string ToString() const;
+};
+
+class Imports : public std::vector<Css::Import*> {
+ public:
+ Imports() : std::vector<Css::Import*>() { }
+ ~Imports();
+};
+
+// A stylesheet consists of a list of import information and a list of
+// rulesets.
+class Stylesheet {
+ public:
+ Stylesheet() : type_(AUTHOR) {}
+
+ // USER is currently unused.
+ enum StylesheetType { AUTHOR, USER, SYSTEM };
+ StylesheetType type() const { return type_; }
+ const Imports& imports() const { return imports_; }
+ const Rulesets& rulesets() const { return rulesets_; }
+
+ const Import& import(int i) const { return *imports_[i]; }
+ const Ruleset& ruleset(int i) const { return *rulesets_[i]; }
+
+ void set_type(StylesheetType type) { type_ = type; }
+ Imports& mutable_imports() { return imports_; }
+ Rulesets& mutable_rulesets() { return rulesets_; }
+
+ string ToString() const;
+ private:
+ StylesheetType type_;
+ Imports imports_;
+ Rulesets rulesets_;
+
+ DISALLOW_COPY_AND_ASSIGN(Stylesheet);
+};
+
+} // namespace
+
+#endif // WEBUTIL_CSS_PARSER_H__
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/parser_unittest.cc b/trunk/src/third_party/css_parser/src/webutil/css/parser_unittest.cc
new file mode 100644
index 0000000..db1626d
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/parser_unittest.cc
@@ -0,0 +1,1455 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2006 Google Inc. All Rights Reserved.
+// Author: dpeng@google.com (Daniel Peng)
+
+#include "webutil/css/parser.h"
+
+#include "base/callback.h"
+#include "base/logging.h"
+#include "base/macros.h"
+#include "base/scoped_ptr.h"
+#include "testing/base/public/googletest.h"
+#include "testing/base/public/gunit.h"
+
+
+namespace Css {
+
+class ParserTest : public testing::Test {
+ protected:
+ // The various Test* functions below check that parselen characters
+ // are parsed. Pass -1 to indicate that the entire string should be
+ // parsed.
+
+ // Checks that unescaping s returns value.
+ void TestUnescape(const char* s, int parselen, char32 value) {
+ SCOPED_TRACE(s);
+ Parser a(s);
+ if (parselen == -1) parselen = strlen(s);
+ EXPECT_EQ(value, a.ParseEscape());
+ EXPECT_EQ(parselen, a.getpos() - s);
+ }
+
+ // Checks that ParseIdent(s) returns utf8golden.
+ void TestIdent(const char* s, int parselen, string utf8golden) {
+ SCOPED_TRACE(s);
+ Parser a(s);
+ if (parselen == -1) parselen = strlen(s);
+ EXPECT_EQ(utf8golden, UnicodeTextToUTF8(a.ParseIdent()));
+ EXPECT_EQ(parselen, a.getpos() - s);
+ }
+
+ // Checks that ParseString<">(s) returns utf8golden.
+ void TestDstring(const char* s, int parselen, string utf8golden) {
+ SCOPED_TRACE(s);
+ Parser a(s);
+ if (parselen == -1) parselen = strlen(s);
+ EXPECT_EQ(utf8golden, UnicodeTextToUTF8(a.ParseString<'"'>()));
+ EXPECT_EQ(parselen, a.getpos() - s);
+ }
+
+ // Checks that ParseString<'>(s) returns utf8golden.
+ void TestSstring(const char* s, int parselen, string utf8golden) {
+ SCOPED_TRACE(s);
+ Parser a(s);
+ if (parselen == -1) parselen = strlen(s);
+ EXPECT_EQ(utf8golden, UnicodeTextToUTF8(a.ParseString<'\''>()));
+ EXPECT_EQ(parselen, a.getpos() - s);
+ }
+
+ // Checks that ParseAny(s) returns goldennum with goldenunit unit.
+ void TestAnyNum(const char* s, int parselen, float goldennum,
+ Value::Unit goldenunit) {
+ SCOPED_TRACE(s);
+ Parser a(s);
+ if (parselen == -1) parselen = strlen(s);
+ scoped_ptr<Value> t(a.ParseAny());
+ EXPECT_EQ(t->GetLexicalUnitType(), Value::NUMBER);
+ EXPECT_EQ(t->GetDimension(), goldenunit);
+ EXPECT_FLOAT_EQ(t->GetFloatValue(), goldennum);
+ EXPECT_EQ(parselen, a.getpos() - s);
+ }
+
+ // Checks that ParseAny(s) returns goldennum with OTHER unit (with
+ // unit text goldenunit).
+ void TestAnyNumOtherUnit(const char* s, int parselen, float goldennum,
+ string goldenunit) {
+ SCOPED_TRACE(s);
+ Parser a(s);
+ if (parselen == -1) parselen = strlen(s);
+ scoped_ptr<Value> t(a.ParseAny());
+ EXPECT_EQ(t->GetLexicalUnitType(), Value::NUMBER);
+ EXPECT_EQ(t->GetDimension(), Value::OTHER);
+ EXPECT_EQ(t->GetDimensionUnitText(), goldenunit);
+ EXPECT_EQ(parselen, a.getpos() - s);
+ }
+
+ // Checks that ParseAny(s) returns a string-type value with type goldenty
+ // and value utf8golden.
+ void TestAnyString(const char* s, int parselen, Value::ValueType goldenty,
+ string utf8golden) {
+ SCOPED_TRACE(s);
+ Parser a(s);
+ if (parselen == -1) parselen = strlen(s);
+ scoped_ptr<Value> t(a.ParseAny());
+ EXPECT_EQ(goldenty, t->GetLexicalUnitType());
+ EXPECT_EQ(utf8golden, UnicodeTextToUTF8(t->GetStringValue()));
+ EXPECT_EQ(parselen, a.getpos() - s);
+ }
+
+ // Checks that ParseAny(s) returns a ident value with identifier goldenty
+ // and value utf8golden.
+ void TestAnyIdent(const char* s, int parselen, Identifier::Ident goldenty) {
+ SCOPED_TRACE(s);
+ Parser a(s);
+ if (parselen == -1) parselen = strlen(s);
+ scoped_ptr<Value> t(a.ParseAny());
+ EXPECT_EQ(Value::IDENT, t->GetLexicalUnitType());
+ EXPECT_EQ(goldenty, t->GetIdentifier().ident());
+ EXPECT_EQ(parselen, a.getpos() - s);
+ }
+
+ // Checks that ParseAny(s) returns OTHER identifier (with text goldenident).
+ void TestAnyOtherIdent(const char* s, int parselen,
+ const string& goldenident) {
+ SCOPED_TRACE(s);
+ Parser a(s);
+ if (parselen == -1) parselen = strlen(s);
+ scoped_ptr<Value> t(a.ParseAny());
+ EXPECT_EQ(Value::IDENT, t->GetLexicalUnitType());
+ EXPECT_EQ(Identifier::OTHER, t->GetIdentifier().ident());
+ EXPECT_EQ(goldenident, UnicodeTextToUTF8(t->GetIdentifierText()));
+ EXPECT_EQ(parselen, a.getpos() - s);
+ }
+
+ Declarations* ParseAndExpandBackground(const string& str,
+ bool quirks_mode=true) {
+ scoped_ptr<Parser> p(new Parser(str));
+ p->set_quirks_mode(quirks_mode);
+ scoped_ptr<Values> vals(p->ParseValues(Property::BACKGROUND));
+ if (vals.get() == NULL || vals->size() == 0) {
+ return NULL;
+ }
+ Declaration background(Property::BACKGROUND, vals.release(), false);
+ scoped_ptr<Declarations> decls(new Declarations);
+ Parser::ExpandBackground(background, decls.get());
+ if (decls->size() == 0) {
+ return NULL;
+ }
+ return decls.release();
+ }
+
+ void TestBackgroundPosition(const string& str, const string& x,
+ const string& y) {
+ scoped_ptr<Declarations> decls(ParseAndExpandBackground(str));
+ // Find and check position x and y values.
+ bool found_x = false;
+ bool found_y = false;
+ for (Declarations::const_iterator iter = decls->begin();
+ iter != decls->end(); ++iter) {
+ Declaration* decl = *iter;
+ switch (decl->prop()) {
+ case Property::BACKGROUND_POSITION_X:
+ EXPECT_EQ(x, decl->values()->get(0)->ToString());
+ found_x = true;
+ break;
+ case Property::BACKGROUND_POSITION_Y:
+ EXPECT_EQ(y, decl->values()->get(0)->ToString());
+ found_y = true;
+ break;
+ default:
+ break;
+ }
+ }
+ EXPECT_TRUE(found_x);
+ EXPECT_TRUE(found_y);
+ }
+};
+
+
+
+TEST_F(ParserTest, unescape) {
+ TestUnescape("\\abcdef aabc", 8, 0xABCDEF);
+ TestUnescape("\\A", 2, 0xA);
+ TestUnescape("\\A0b5C\r\n", 8, 0xa0b5C);
+ TestUnescape("\\AB ", 4, 0xAB);
+}
+
+TEST_F(ParserTest, ident) {
+ // We're a little more forgiving than the standard:
+ //
+ // In CSS 2.1, identifiers (including element names, classes, and
+ // IDs in selectors) can contain only the characters [A-Za-z0-9]
+ // and ISO 10646 characters U+00A1 and higher, plus the hyphen (-)
+ // and the underscore (_); they cannot start with a digit, or a
+ // hyphen followed by a digit. Only properties, values, units,
+ // pseudo-classes, pseudo-elements, and at-rules may start with a
+ // hyphen (-); other identifiers (e.g. element names, classes, or
+ // IDs) may not. Identifiers can also contain escaped characters
+ // and any ISO 10646 character as a numeric code (see next
+ // item). For instance, the identifier "B&W?" may be written as
+ // "B\&W\?" or "B\26 W\3F".
+ TestIdent("abcd rexo\n", 4, "abcd");
+ TestIdent("台灣華語", 12, "台灣華語");
+ TestIdent("\\41\\42 \\43 \\44", 14, "ABCD");
+ TestIdent("\\41\\42 \\43 \\44g'r,'rcg.,',", 15, "ABCDg");
+ TestIdent("\\41\\42 \\43 \\44\r\ng'r,'rcg.,',", 17, "ABCDg");
+ TestIdent("-blah-_67", 9, "-blah-_67");
+ TestIdent("\\!\\&\\^\\*\\\\e", 11, "!&^*\\e");
+}
+
+TEST_F(ParserTest, string) {
+ TestSstring("'ab\\'aoe\"\\'eo灣'灣", 17, "ab'aoe\"'eo灣");
+ TestDstring("\"ab'aoe\\\"'eo灣\"灣", 16, "ab'aoe\"'eo灣");
+ TestSstring("'ab\naoeu", 3, "ab");
+ TestDstring("\"ab\naoeu", 3, "ab");
+ TestDstring("\"ab\\\naoeu\"", 10, "abaoeu");
+}
+
+TEST_F(ParserTest, anynum) {
+ TestAnyNum("3.1415 4aone", 6, 3.1415, Value::NO_UNIT);
+ TestAnyNum(".1415 4aone", 5, .1415, Value::NO_UNIT);
+ TestAnyNum("5 4aone", 1, 5, Value::NO_UNIT);
+
+ TestAnyNum("3.1415pt 4aone", 8, 3.1415, Value::PT);
+ TestAnyNum(".1415pc 4aone", 7, .1415, Value::PC);
+ TestAnyNum("5s 4aone", 2, 5, Value::S);
+
+ TestAnyNumOtherUnit("5sacks 4aone", 6, 5, "sacks");
+ TestAnyNumOtherUnit("5灣 4aone", 4, 5, "灣");
+}
+
+TEST_F(ParserTest, anystring) {
+ TestAnyIdent("none b c d e", 4, Identifier::NONE);
+ TestAnyIdent("none; b c d e", 4, Identifier::NONE);
+ TestAnyIdent("none ; b c d e", 4, Identifier::NONE);
+ TestAnyOtherIdent("a b c d e", 1, "a");
+ TestAnyOtherIdent("a; b c d e", 1, "a");
+ TestAnyOtherIdent("a ; b c d e", 1, "a");
+ TestAnyString("'ab\\'aoe\"\\'eo灣'灣 ; b c d e", 17,
+ Value::STRING, "ab'aoe\"'eo灣");
+}
+
+TEST_F(ParserTest, color) {
+ // allowed in quirks mode
+ scoped_ptr<Parser> a(new Parser("abCdEF brc.,aoek"));
+ EXPECT_EQ(a->ParseColor().ToString(), "#abcdef");
+
+ // not allowed in stanard compliant mode.
+ a.reset(new Parser("abCdEF brc.,aoek"));
+ a->set_quirks_mode(false);
+ EXPECT_FALSE(a->ParseColor().IsDefined());
+
+ // this is allowed
+ a.reset(new Parser("#abCdEF brc.,aoek"));
+ a->set_quirks_mode(false);
+ EXPECT_EQ(a->ParseColor().ToString(), "#abcdef");
+
+ a.reset(new Parser("abC btneo"));
+ EXPECT_EQ(a->ParseColor().ToString(), "#aabbcc");
+
+ // no longer allowed
+ a.reset(new Parser("#white something"));
+ EXPECT_FALSE(a->ParseColor().IsDefined());
+
+ a.reset(new Parser("#white something"));
+ a->set_quirks_mode(false);
+ EXPECT_FALSE(a->ParseColor().IsDefined());
+
+ // this is allowed
+ a.reset(new Parser("white something"));
+ EXPECT_EQ(a->ParseColor().ToString(), "#ffffff");
+
+ a.reset(new Parser("white something"));
+ a->set_quirks_mode(false);
+ EXPECT_EQ(a->ParseColor().ToString(), "#ffffff");
+
+ // system color
+ a.reset(new Parser("buttonface something"));
+ EXPECT_EQ(a->ParseColor().ToString(), "#ece9d8");
+
+ // string patterns
+
+ a.reset(new Parser("'abCdEF' brc.,aoek"));
+ EXPECT_EQ(a->ParseColor().ToString(), "#abcdef");
+
+ a.reset(new Parser("'abCdEF' brc.,aoek"));
+ a->set_quirks_mode(false);
+ EXPECT_FALSE(a->ParseColor().IsDefined());
+
+ // this is not allowed since color values must end on string boundary
+ a.reset(new Parser("'#abCdEF brc'.,aoek"));
+ a->set_quirks_mode(false);
+ EXPECT_FALSE(a->ParseColor().IsDefined());
+
+ a.reset(new Parser("\"abC\" btneo"));
+ EXPECT_EQ(a->ParseColor().ToString(), "#aabbcc");
+
+ // no longer allowed
+ a.reset(new Parser("'#white' something"));
+ EXPECT_FALSE(a->ParseColor().IsDefined());
+
+ a.reset(new Parser("'#white' something"));
+ a->set_quirks_mode(false);
+ EXPECT_FALSE(a->ParseColor().IsDefined());
+
+ // this is allowed
+ a.reset(new Parser("'white' something"));
+ EXPECT_EQ(a->ParseColor().ToString(), "#ffffff");
+
+ a.reset(new Parser("'white' something"));
+ a->set_quirks_mode(false);
+ EXPECT_EQ(a->ParseColor().ToString(), "#ffffff");
+
+ // no longer allowed
+ a.reset(new Parser("100%"));
+ EXPECT_FALSE(a->ParseColor().IsDefined());
+
+ // no longer allowed
+ a.reset(new Parser("100px"));
+ EXPECT_FALSE(a->ParseColor().IsDefined());
+
+ // this is allowed
+ a.reset(new Parser("100"));
+ EXPECT_EQ(a->ParseColor().ToString(), "#110000");
+
+ // should be parsed as a number
+ a.reset(new Parser("100px"));
+ scoped_ptr<Value> t(a->ParseAnyExpectingColor());
+ EXPECT_EQ(Value::NUMBER, t->GetLexicalUnitType());
+ EXPECT_EQ("100px", t->ToString());
+
+ a.reset(new Parser("rgb(12,25,30)"));
+ t.reset(a->ParseAny());
+ EXPECT_EQ(t->GetColorValue().ToString(), "#0c191e");
+
+ a.reset(new Parser("rgb( 12% , 25%, 30%)"));
+ t.reset(a->ParseAny());
+ EXPECT_EQ(t->GetColorValue().ToString(), "#1e3f4c");
+
+ a.reset(new Parser("rgb( 12% , 25% 30%)"));
+ t.reset(a->ParseAny());
+ EXPECT_FALSE(t.get());
+}
+
+TEST_F(ParserTest, url) {
+ scoped_ptr<Parser> a(new Parser("url(blah)"));
+ scoped_ptr<Value> t(a->ParseAny());
+
+ EXPECT_EQ(Value::URI, t->GetLexicalUnitType());
+ EXPECT_EQ("blah", UnicodeTextToUTF8(t->GetStringValue()));
+
+ a.reset(new Parser("url( blah )"));
+ t.reset(a->ParseAny());
+
+ EXPECT_EQ(Value::URI, t->GetLexicalUnitType());
+ EXPECT_EQ("blah", UnicodeTextToUTF8(t->GetStringValue()));
+
+ a.reset(new Parser("url( blah extra)"));
+ t.reset(a->ParseAny());
+
+ EXPECT_EQ(static_cast<Value *>(NULL), t.get());
+}
+
+TEST_F(ParserTest, rect) {
+ // rect can be either comma or space delimited
+ scoped_ptr<Parser> a(new Parser("rect( 12, 10,auto 200px)"));
+ scoped_ptr<Value> t(a->ParseAny());
+
+ EXPECT_EQ(Value::RECT, t->GetLexicalUnitType());
+ ASSERT_EQ(4, t->GetParameters()->size());
+ EXPECT_EQ(Value::NUMBER,
+ t->GetParameters()->get(0)->GetLexicalUnitType());
+ EXPECT_EQ(12, t->GetParameters()->get(0)->GetIntegerValue());
+ EXPECT_EQ(Value::IDENT,
+ t->GetParameters()->get(2)->GetLexicalUnitType());
+ EXPECT_EQ(Identifier::AUTO,
+ t->GetParameters()->get(2)->GetIdentifier().ident());
+
+ a.reset(new Parser("rect(auto)"));
+ t.reset(a->ParseAny());
+
+ EXPECT_EQ(static_cast<Value *>(NULL), t.get());
+
+ a.reset(new Parser("rect()"));
+ t.reset(a->ParseAny());
+
+ EXPECT_EQ(static_cast<Value *>(NULL), t.get());
+
+ a.reset(new Parser("rect(13 10 auto 4)"));
+ t.reset(a->ParseAny());
+
+ EXPECT_EQ(13, t->GetParameters()->get(0)->GetIntegerValue());
+
+ a.reset(new Parser("rect(14,10,1,2)"));
+ t.reset(a->ParseAny());
+
+ EXPECT_EQ(14, t->GetParameters()->get(0)->GetIntegerValue());
+
+ a.reset(new Parser("rect(15 10 1)"));
+ t.reset(a->ParseAny());
+
+ EXPECT_EQ(static_cast<Value *>(NULL), t.get());
+
+ a.reset(new Parser("rect(16 10 1 2 3)"));
+ t.reset(a->ParseAny());
+
+ EXPECT_EQ(static_cast<Value *>(NULL), t.get());
+}
+
+TEST_F(ParserTest, background) {
+ scoped_ptr<Declarations> decls(ParseAndExpandBackground("#333"));
+
+ EXPECT_EQ(6, decls->size());
+
+ decls.reset(ParseAndExpandBackground("fff"));
+ EXPECT_TRUE(decls.get());
+ // Not valid for quirks_mode=false
+ EXPECT_FALSE(ParseAndExpandBackground("fff", false));
+
+ decls.reset(ParseAndExpandBackground("fff000"));
+ EXPECT_TRUE(decls.get());
+ // Not valid for quirks_mode=false
+ EXPECT_FALSE(ParseAndExpandBackground("fff000", false));
+
+ // This should now be parsed as background position instead of color.
+ decls.reset(ParseAndExpandBackground("100%"));
+ ASSERT_TRUE(decls.get());
+ ASSERT_EQ(6, decls->size());
+ EXPECT_EQ(Property::BACKGROUND_COLOR, decls->get(0)->prop());
+ EXPECT_EQ(Identifier::TRANSPARENT,
+ decls->get(0)->values()->get(0)->GetIdentifier().ident());
+ EXPECT_EQ(Property::BACKGROUND_POSITION_X, decls->get(4)->prop());
+ EXPECT_EQ("100%", decls->get(4)->values()->get(0)->ToString());
+
+ EXPECT_FALSE(ParseAndExpandBackground(""));
+ EXPECT_FALSE(ParseAndExpandBackground(";"));
+ EXPECT_FALSE(ParseAndExpandBackground("\"string\""));
+ EXPECT_FALSE(ParseAndExpandBackground("normal"));
+
+ decls.reset(ParseAndExpandBackground("inherit"));
+ ASSERT_EQ(6, decls->size());
+ for (int i = 0; i < 6; ++i) {
+ EXPECT_EQ(Identifier::INHERIT,
+ decls->get(i)->values()->get(0)->GetIdentifier().ident());
+ }
+
+ EXPECT_FALSE(ParseAndExpandBackground("inherit none"));
+ EXPECT_FALSE(ParseAndExpandBackground("none inherit"));
+
+ decls.reset(ParseAndExpandBackground("none"));
+ EXPECT_EQ(Identifier::TRANSPARENT,
+ decls->get(0)->values()->get(0)->GetIdentifier().ident());
+ EXPECT_EQ(Identifier::NONE,
+ decls->get(1)->values()->get(0)->GetIdentifier().ident());
+ EXPECT_EQ(Identifier::REPEAT,
+ decls->get(2)->values()->get(0)->GetIdentifier().ident());
+ EXPECT_EQ(Identifier::SCROLL,
+ decls->get(3)->values()->get(0)->GetIdentifier().ident());
+
+ decls.reset(ParseAndExpandBackground("fixed"));
+ EXPECT_EQ(Identifier::FIXED,
+ decls->get(3)->values()->get(0)->GetIdentifier().ident());
+
+ decls.reset(ParseAndExpandBackground("transparent"));
+ EXPECT_EQ(Identifier::TRANSPARENT,
+ decls->get(0)->values()->get(0)->GetIdentifier().ident());
+
+ // IE specific. Firefox should bail out.
+ decls.reset(ParseAndExpandBackground("none url(abc)"));
+ EXPECT_EQ(Value::URI, decls->get(1)->values()->get(0)->GetLexicalUnitType());
+
+ decls.reset(ParseAndExpandBackground("none red fixed"));
+ EXPECT_EQ("#ff0000",
+ decls->get(0)->values()->get(0)->GetColorValue().ToString());
+ EXPECT_EQ(Identifier::NONE,
+ decls->get(1)->values()->get(0)->GetIdentifier().ident());
+ EXPECT_EQ(Identifier::FIXED,
+ decls->get(3)->values()->get(0)->GetIdentifier().ident());
+
+ // The rest are position tests
+ TestBackgroundPosition("none", "0%", "0%");
+ TestBackgroundPosition("10", "10", "50%");
+ TestBackgroundPosition("10 20%", "10", "20%");
+ TestBackgroundPosition("10 100%", "10", "100%");
+ TestBackgroundPosition("top left", "left", "top");
+ TestBackgroundPosition("left top", "left", "top");
+ TestBackgroundPosition("bottom", "50%", "bottom");
+ TestBackgroundPosition("bottom center", "center", "bottom");
+ TestBackgroundPosition("center bottom", "center", "bottom");
+ TestBackgroundPosition("left", "left", "50%");
+ TestBackgroundPosition("left center", "left", "center");
+ TestBackgroundPosition("center left", "left", "center");
+ TestBackgroundPosition("center", "center", "50%");
+ TestBackgroundPosition("center center", "center", "center");
+ TestBackgroundPosition("center 30%", "center", "30%");
+ TestBackgroundPosition("30% center", "30%", "center");
+ TestBackgroundPosition("30% bottom", "30%", "bottom");
+ TestBackgroundPosition("left 30%", "left", "30%");
+ TestBackgroundPosition("30% left", "left", "30%");
+ // IE specific
+ TestBackgroundPosition("30% 20% 50%", "30%", "20%");
+ TestBackgroundPosition("bottom center right", "center", "bottom");
+ TestBackgroundPosition("bottom right top", "right", "bottom");
+ TestBackgroundPosition("bottom top right", "right", "top");
+ TestBackgroundPosition("top right left", "right", "top");
+ TestBackgroundPosition("right left top", "left", "top");
+}
+
+TEST_F(ParserTest, font_family) {
+ scoped_ptr<Parser> a(new Parser(
+ " Arial font 'Sans' system, menu new "));
+ scoped_ptr<Values> t(new Values);
+
+ EXPECT_TRUE(a->ParseFontFamily(t.get()));
+ ASSERT_EQ(4, t->size());
+ EXPECT_EQ(Value::IDENT, t->get(0)->GetLexicalUnitType());
+ EXPECT_EQ("arial font", UnicodeTextToUTF8(t->get(0)->GetIdentifierText()));
+ EXPECT_EQ(Value::STRING, t->get(1)->GetLexicalUnitType());
+ EXPECT_EQ("system", UnicodeTextToUTF8(t->get(2)->GetIdentifierText()));
+ EXPECT_EQ("menu new", UnicodeTextToUTF8(t->get(3)->GetIdentifierText()));
+
+ a.reset(new Parser("Verdana 3"));
+ t.reset(new Values);
+ EXPECT_FALSE(a->ParseFontFamily(t.get()));
+
+ a.reset(new Parser("Verdana :"));
+ t.reset(new Values);
+ EXPECT_FALSE(a->ParseFontFamily(t.get()));
+
+ a.reset(new Parser("Verdana ;"));
+ t.reset(new Values);
+ EXPECT_TRUE(a->ParseFontFamily(t.get()));
+ ASSERT_EQ(1, t->size());
+ EXPECT_EQ(Value::IDENT, t->get(0)->GetLexicalUnitType());
+ EXPECT_EQ("verdana", UnicodeTextToUTF8(t->get(0)->GetIdentifierText()));
+}
+
+TEST_F(ParserTest, font) {
+ scoped_ptr<Parser> a(new Parser("font: caption"));
+ scoped_ptr<Declarations> declarations(a->ParseDeclarations());
+ const char expected_caption_expansion[] =
+ "font: caption; "
+ "font-style: normal; "
+ "font-variant: normal; "
+ "font-weight: normal; "
+ "font-size: 10.6667px; "
+ "line-height: normal; "
+ "font-family: caption";
+ EXPECT_EQ(expected_caption_expansion, declarations->ToString());
+
+ a.reset(new Parser("font: inherit"));
+ declarations.reset(a->ParseDeclarations());
+ const char expected_inherit_expansion[] =
+ "font: inherit; "
+ "font-style: inherit; "
+ "font-variant: inherit; "
+ "font-weight: inherit; "
+ "font-size: inherit; "
+ "line-height: inherit; "
+ "font-family: inherit";
+ EXPECT_EQ(expected_inherit_expansion, declarations->ToString());
+
+ a.reset(new Parser("normal 10px /120% Arial 'Sans'"));
+ scoped_ptr<Values> t(a->ParseFont());
+ ASSERT_EQ(7, t->size());
+ EXPECT_FLOAT_EQ(10, t->get(3)->GetFloatValue());
+ EXPECT_EQ(Value::PERCENT, t->get(4)->GetDimension());
+
+ a.reset(new Parser("italic 10px Arial, Sans"));
+ t.reset(a->ParseFont());
+ ASSERT_EQ(7, t->size());
+ EXPECT_FLOAT_EQ(10, t->get(3)->GetFloatValue());
+ EXPECT_EQ(Identifier::NORMAL, t->get(4)->GetIdentifier().ident());
+
+ a.reset(new Parser("SMALL-caps normal x-large Arial"));
+ t.reset(a->ParseFont());
+ ASSERT_EQ(6, t->size());
+ EXPECT_EQ(Identifier::NORMAL, t->get(0)->GetIdentifier().ident());
+ EXPECT_EQ(Identifier::SMALL_CAPS, t->get(1)->GetIdentifier().ident());
+ EXPECT_EQ(Identifier::X_LARGE, t->get(3)->GetIdentifier().ident());
+ EXPECT_EQ(Identifier::NORMAL, t->get(4)->GetIdentifier().ident());
+
+ a.reset(new Parser("bolder 100 120 Arial"));
+ t.reset(a->ParseFont());
+ ASSERT_EQ(6, t->size());
+ EXPECT_EQ(100, t->get(2)->GetIntegerValue());
+ EXPECT_EQ(120, t->get(3)->GetIntegerValue());
+ EXPECT_EQ(Identifier::NORMAL, t->get(4)->GetIdentifier().ident());
+
+ a.reset(new Parser("10px normal"));
+ t.reset(a->ParseFont());
+ ASSERT_EQ(6, t->size());
+ EXPECT_EQ(10, t->get(3)->GetIntegerValue());
+ EXPECT_EQ(Identifier::NORMAL, t->get(5)->GetIdentifier().ident());
+
+ a.reset(new Parser("normal 10px "));
+ t.reset(a->ParseFont());
+ EXPECT_EQ(5, t->size()) << "missing font-family should be allowed";
+
+ a.reset(new Parser("10px/12pt "));
+ t.reset(a->ParseFont());
+ EXPECT_EQ(5, t->size()) << "missing font-family should be allowed";
+
+ a.reset(new Parser("menu 10px"));
+ t.reset(a->ParseFont());
+ EXPECT_EQ(static_cast<Values *>(NULL), t.get())
+ << "system font with extra value";
+
+ a.reset(new Parser("Arial, menu "));
+ t.reset(a->ParseFont());
+ EXPECT_EQ(static_cast<Values *>(NULL), t.get()) << "missing font-size";
+
+ a.reset(new Parser("transparent 10px "));
+ t.reset(a->ParseFont());
+ EXPECT_EQ(static_cast<Values *>(NULL), t.get()) << "unknown property";
+
+ a.reset(new Parser("normal / 10px Arial"));
+ t.reset(a->ParseFont());
+ EXPECT_EQ(static_cast<Values *>(NULL), t.get())
+ << "line-height without font-size";
+
+ a.reset(new Parser("normal 10px/ Arial"));
+ t.reset(a->ParseFont());
+ EXPECT_EQ(static_cast<Values *>(NULL), t.get())
+ << "slash without line-height";
+
+ a.reset(new Parser("normal 10px Arial #333"));
+ t.reset(a->ParseFont());
+ EXPECT_EQ(static_cast<Values *>(NULL), t.get()) << "invalid type";
+}
+
+TEST_F(ParserTest, values) {
+ scoped_ptr<Parser> a(new Parser(
+ "rgb(12,25,30) url(blah) url('blah.png') 12% !important 'arial'"));
+ scoped_ptr<Values> t(a->ParseValues(Property::OTHER));
+
+ ASSERT_EQ(4, t->size());
+ EXPECT_EQ(Value::COLOR, t->get(0)->GetLexicalUnitType());
+ EXPECT_EQ(Value::URI, t->get(1)->GetLexicalUnitType());
+ EXPECT_EQ(Value::URI, t->get(2)->GetLexicalUnitType());
+ EXPECT_EQ(Value::NUMBER, t->get(3)->GetLexicalUnitType());
+ EXPECT_EQ(Value::PERCENT, t->get(3)->GetDimension());
+
+ a.reset(new Parser("rgb( 12, 25,30) @ignored url( blah )"
+ " rect(12 10 auto 200px)"
+ " { should be {nested }discarded } ident;"));
+ t.reset(a->ParseValues(Property::OTHER));
+
+ ASSERT_EQ(4, t->size());
+ EXPECT_EQ(Value::COLOR, t->get(0)->GetLexicalUnitType());
+ EXPECT_EQ(Value::URI, t->get(1)->GetLexicalUnitType());
+ EXPECT_EQ(Value::RECT, t->get(2)->GetLexicalUnitType());
+ EXPECT_EQ(Value::IDENT, t->get(3)->GetLexicalUnitType());
+ EXPECT_EQ("ident", UnicodeTextToUTF8(t->get(3)->GetIdentifierText()));
+
+ // test value copy constructor.
+ scoped_ptr<Value> val(new Value(*(t->get(2))));
+ EXPECT_EQ(Value::RECT, val->GetLexicalUnitType());
+ ASSERT_EQ(4, val->GetParameters()->size());
+ EXPECT_EQ(Value::NUMBER,
+ val->GetParameters()->get(0)->GetLexicalUnitType());
+ EXPECT_EQ(12, val->GetParameters()->get(0)->GetIntegerValue());
+ EXPECT_EQ(Value::IDENT,
+ val->GetParameters()->get(2)->GetLexicalUnitType());
+ EXPECT_EQ("auto", UnicodeTextToUTF8(val->GetParameters()->get(2)
+ ->GetIdentifierText()));
+}
+
+void ParseFontFamily(Parser* parser) {
+ Values values;
+ parser->ParseFontFamily(&values);
+}
+
+TEST_F(ParserTest, ParseBlock) {
+ static const char* truetestcases[] = {
+ "{{{{}}}} serif",
+ "{ { { { } } } } serif", // whitespace
+ "{@ident1{{ @ident {}}}} serif", // @-idents
+ "{{}} @ident {{}} @ident serif", // multiple blocks
+ "{{ident{{}ident2}}} serif", // idents
+ };
+ for (int i = 0; i < arraysize(truetestcases); ++i) {
+ SCOPED_TRACE(truetestcases[i]);
+ Values values;
+ EXPECT_TRUE(Parser(truetestcases[i]).ParseFontFamily(&values));
+ ASSERT_EQ(1, values.size());
+ EXPECT_EQ(Value::IDENT, values.get(0)->GetLexicalUnitType());
+ EXPECT_EQ("serif", UnicodeTextToUTF8(values.get(0)->GetIdentifierText()));
+ }
+
+ static const char* falsetestcases[] = {
+ "{{{{}}} serif", // too many opens
+ "{{{{}}}}} serif", // too many closes
+ "{{{{}}}}}", // no tokenxs
+ };
+ for (int i = 0; i < arraysize(falsetestcases); ++i) {
+ SCOPED_TRACE(falsetestcases[i]);
+ Values values;
+ Parser(falsetestcases[i]).ParseFontFamily(&values);
+ EXPECT_EQ(0, values.size());
+ }
+
+}
+
+
+TEST_F(ParserTest, declarations) {
+ scoped_ptr<Parser> a(new Parser(
+ "color: #333; line-height: 1.3;"
+ "text-align: justify; font-family: \"Gill Sans MT\","
+ "\"Gill Sans\", GillSans, Arial, Helvetica, sans-serif"));
+ scoped_ptr<Declarations> t(a->ParseDeclarations());
+
+ // Declarations is a vector of Declaration, and we go through them:
+ ASSERT_EQ(4, t->size());
+ EXPECT_EQ(Property::COLOR, t->get(0)->prop());
+ EXPECT_EQ(Property::LINE_HEIGHT, t->get(1)->prop());
+ EXPECT_EQ(Property::TEXT_ALIGN, t->get(2)->prop());
+ EXPECT_EQ(Property::FONT_FAMILY, t->get(3)->prop());
+
+ ASSERT_EQ(1, t->get(0)->values()->size());
+ EXPECT_EQ(Value::COLOR, t->get(0)->values()->get(0)->GetLexicalUnitType());
+ EXPECT_EQ("#333333", t->get(0)->values()->get(0)->GetColorValue().ToString());
+
+ ASSERT_EQ(6, (*t->get(3)->values()).size());
+ EXPECT_EQ(Value::STRING,
+ t->get(3)->values()->get(0)->GetLexicalUnitType());
+ EXPECT_EQ("Gill Sans MT",
+ UnicodeTextToUTF8(t->get(3)->values()->get(0)->GetStringValue()));
+
+ a.reset(new Parser(
+ "background-color: 333; color: \"abcdef\";"
+ "background-color: #red; color: \"white\";"
+ "background-color: rgb(255, 10%, 10)"));
+ t.reset(a->ParseDeclarations());
+
+ ASSERT_EQ(4, t->size()) << "#red is not valid";
+ EXPECT_EQ(Property::BACKGROUND_COLOR, t->get(0)->prop());
+ EXPECT_EQ(Property::COLOR, t->get(1)->prop());
+ EXPECT_EQ(Property::COLOR, t->get(2)->prop());
+ EXPECT_EQ(Property::BACKGROUND_COLOR, t->get(3)->prop());
+ ASSERT_EQ(1, t->get(0)->values()->size());
+ EXPECT_EQ(Value::COLOR, t->get(0)->values()->get(0)->GetLexicalUnitType());
+ EXPECT_EQ("#333333", t->get(0)->values()->get(0)->GetColorValue().ToString());
+ ASSERT_EQ(1, t->get(1)->values()->size());
+ EXPECT_EQ(Value::COLOR, t->get(1)->values()->get(0)->GetLexicalUnitType());
+ EXPECT_EQ("#abcdef", t->get(1)->values()->get(0)->GetColorValue().ToString());
+ ASSERT_EQ(1, t->get(2)->values()->size());
+ EXPECT_EQ(Value::COLOR, t->get(2)->values()->get(0)->GetLexicalUnitType());
+ EXPECT_EQ("#ffffff", t->get(2)->values()->get(0)->GetColorValue().ToString());
+ ASSERT_EQ(1, t->get(3)->values()->size());
+ EXPECT_EQ(Value::COLOR, t->get(3)->values()->get(0)->GetLexicalUnitType());
+ EXPECT_EQ("#ff190a", t->get(3)->values()->get(0)->GetColorValue().ToString());
+
+ // expand background
+ a.reset(new Parser("background: #333 fixed no-repeat; "));
+ t.reset(a->ParseDeclarations());
+ ASSERT_EQ(7, t->size());
+ EXPECT_EQ(Property::BACKGROUND, t->get(0)->prop());
+ EXPECT_EQ(3, t->get(0)->values()->size());
+ EXPECT_EQ(Property::BACKGROUND_COLOR, t->get(1)->prop());
+ EXPECT_EQ(1, t->get(1)->values()->size());
+ EXPECT_EQ(Property::BACKGROUND_IMAGE, t->get(2)->prop());
+ EXPECT_EQ(1, t->get(2)->values()->size());
+ EXPECT_EQ(Property::BACKGROUND_REPEAT, t->get(3)->prop());
+ EXPECT_EQ(1, t->get(3)->values()->size());
+ EXPECT_EQ(Property::BACKGROUND_ATTACHMENT, t->get(4)->prop());
+ EXPECT_EQ(1, t->get(4)->values()->size());
+ EXPECT_EQ(Property::BACKGROUND_POSITION_X, t->get(5)->prop());
+ EXPECT_EQ(1, t->get(5)->values()->size());
+ EXPECT_EQ(Property::BACKGROUND_POSITION_Y, t->get(6)->prop());
+ EXPECT_EQ(1, t->get(6)->values()->size());
+
+ // expand font
+ a.reset(new Parser("font: small-caps 24px Arial 'Sans', monospace; "));
+ t.reset(a->ParseDeclarations());
+ ASSERT_EQ(7, t->size());
+ EXPECT_EQ(Property::FONT, t->get(0)->prop());
+ EXPECT_EQ(8, t->get(0)->values()->size());
+ EXPECT_EQ(Property::FONT_STYLE, t->get(1)->prop());
+ EXPECT_EQ(Property::FONT_VARIANT, t->get(2)->prop());
+ EXPECT_EQ(Property::FONT_WEIGHT, t->get(3)->prop());
+ EXPECT_EQ(Property::FONT_SIZE, t->get(4)->prop());
+ EXPECT_EQ(Property::LINE_HEIGHT, t->get(5)->prop());
+ EXPECT_EQ(Property::FONT_FAMILY, t->get(6)->prop());
+ ASSERT_EQ(3, t->get(6)->values()->size());
+ EXPECT_EQ("monospace", UnicodeTextToUTF8(t->get(6)->values()->get(2)->
+ GetIdentifierText()));
+
+ a.reset(new Parser(
+ "{font-size: #333; color:red"));
+ t.reset(a->ParseDeclarations());
+ ASSERT_EQ(1, t->size());
+ EXPECT_EQ(Property::COLOR, t->get(0)->prop());
+
+ a.reset(new Parser(
+ "{font-size: #333; color:red"));
+ a->set_quirks_mode(false);
+ t.reset(a->ParseDeclarations());
+ EXPECT_EQ(0, t->size());
+
+ a.reset(new Parser(
+ "font-size {background: #333; color:red"));
+ t.reset(a->ParseDeclarations());
+ ASSERT_EQ(1, t->size());
+ EXPECT_EQ(Property::COLOR, t->get(0)->prop());
+
+ a.reset(new Parser(
+ "font-size {background: #333; color:red"));
+ a->set_quirks_mode(false);
+ t.reset(a->ParseDeclarations());
+ EXPECT_EQ(0, t->size());
+
+ a.reset(new Parser(
+ "font-size }background: #333; color:red"));
+ t.reset(a->ParseDeclarations());
+ EXPECT_EQ(0, t->size());
+
+ a.reset(new Parser(
+ "font-size }background: #333; color:red"));
+ a->set_quirks_mode(false);
+ t.reset(a->ParseDeclarations());
+ EXPECT_EQ(0, t->size());
+
+ a.reset(new Parser(
+ "top:1px; {font-size: #333; color:red}"));
+ t.reset(a->ParseDeclarations());
+ ASSERT_EQ(2, t->size());
+ EXPECT_EQ(Property::TOP, t->get(0)->prop());
+ EXPECT_EQ(Property::COLOR, t->get(1)->prop());
+
+ a.reset(new Parser(
+ "top:1px; {font-size: #333; color:red}"));
+ a->set_quirks_mode(false);
+ t.reset(a->ParseDeclarations());
+ ASSERT_EQ(1, t->size());
+ EXPECT_EQ(Property::TOP, t->get(0)->prop());
+}
+
+TEST_F(ParserTest, illegal_constructs) {
+ scoped_ptr<Parser> a(new Parser("width: {$width}"));
+ scoped_ptr<Declarations> t(a->ParseDeclarations());
+
+ ASSERT_EQ(1, t->size());
+ EXPECT_EQ(Property::WIDTH, t->get(0)->prop());
+ EXPECT_EQ(0, t->get(0)->values()->size());
+
+ a.reset(new Parser("font-family: \"Gill Sans MT;"));
+ t.reset(a->ParseDeclarations());
+
+ ASSERT_EQ(1, t->size());
+ EXPECT_EQ(Property::FONT_FAMILY, t->get(0)->prop());
+ ASSERT_EQ(1, t->get(0)->values()->size());
+ EXPECT_EQ(Value::STRING, t->get(0)->values()->get(0)->GetLexicalUnitType());
+ EXPECT_EQ("Gill Sans MT;",
+ UnicodeTextToUTF8(t->get(0)->values()->get(0)->GetStringValue()));
+
+ a.reset(new Parser("font-family: 'Gill Sans MT"));
+ t.reset(a->ParseDeclarations());
+
+ ASSERT_EQ(1, t->size());
+ EXPECT_EQ(Property::FONT_FAMILY, t->get(0)->prop());
+ ASSERT_EQ(1, t->get(0)->values()->size());
+ EXPECT_EQ(Value::STRING, t->get(0)->values()->get(0)->GetLexicalUnitType());
+ EXPECT_EQ("Gill Sans MT",
+ UnicodeTextToUTF8(t->get(0)->values()->get(0)->GetStringValue()));
+}
+
+TEST_F(ParserTest, value_validation) {
+ scoped_ptr<Parser> a(new Parser("width: {$width}"));
+ scoped_ptr<Declarations> t(a->ParseDeclarations());
+
+ // Let's take border-color as an example. It only accepts color and the
+ // transparent keyword in particular (and inherit is a common one).
+ a.reset(new Parser(
+ "border-color: \"string\"; "
+ "border-color: url(\"abc\"); "
+ "border-color: 12345; "
+ "border-color: none; "
+ "border-color: inherited; "
+ "border-color: red; "
+ "border-color: #123456; "
+ "border-color: transparent; "
+ "border-color: inherit; "
+ "border-color: unknown; "
+ ));
+ t.reset(a->ParseDeclarations());
+
+ ASSERT_EQ(4, t->size());
+ EXPECT_EQ(Value::COLOR, t->get(0)->values()->get(0)->GetLexicalUnitType());
+ EXPECT_EQ(Value::COLOR, t->get(1)->values()->get(0)->GetLexicalUnitType());
+ EXPECT_EQ(Value::IDENT, t->get(2)->values()->get(0)->GetLexicalUnitType());
+ EXPECT_EQ(Identifier::TRANSPARENT,
+ t->get(2)->values()->get(0)->GetIdentifier().ident());
+ EXPECT_EQ(Value::IDENT, t->get(3)->values()->get(0)->GetLexicalUnitType());
+ EXPECT_EQ(Identifier::INHERIT,
+ t->get(3)->values()->get(0)->GetIdentifier().ident());
+}
+
+TEST_F(ParserTest, universalselector) {
+ Parser p("*");
+ scoped_ptr<SimpleSelectors> t(p.ParseSimpleSelectors(false));
+
+ EXPECT_EQ(SimpleSelectors::NONE, t->combinator());
+ ASSERT_EQ(1, t->size());
+ EXPECT_EQ(SimpleSelector::UNIVERSAL, t->get(0)->type());
+}
+
+TEST_F(ParserTest, universalselectorcondition) {
+ scoped_ptr<Parser> a(new Parser(" *[foo=bar]"));
+ scoped_ptr<SimpleSelectors> t(a->ParseSimpleSelectors(true));
+
+ EXPECT_EQ(SimpleSelectors::DESCENDANT, t->combinator());
+ ASSERT_EQ(2, t->size());
+ EXPECT_EQ(SimpleSelector::UNIVERSAL, t->get(0)->type());
+ EXPECT_EQ(SimpleSelector::EXACT_ATTRIBUTE, t->get(1)->type());
+
+ a.reset(new Parser(" *[foo="));
+ t.reset(a->ParseSimpleSelectors(true));
+
+ // This is not a valid selector.
+ EXPECT_FALSE(t.get());
+}
+
+TEST_F(ParserTest, comment_breaking_descendant_combinator) {
+ Parser p(" a b/*foo*/c /*foo*/d/*foo*/ e { }");
+ scoped_ptr<Ruleset> t(p.ParseRuleset());
+
+ ASSERT_EQ(1, t->selectors().size());
+ const Selector* s = t->selectors().get(0);
+ ASSERT_EQ(5, s->size());
+ EXPECT_EQ(SimpleSelectors::NONE, s->get(0)->combinator());
+ EXPECT_EQ(SimpleSelectors::DESCENDANT, s->get(1)->combinator());
+ EXPECT_EQ(SimpleSelectors::DESCENDANT, s->get(2)->combinator());
+ EXPECT_EQ(SimpleSelectors::DESCENDANT, s->get(3)->combinator());
+ EXPECT_EQ(SimpleSelectors::DESCENDANT, s->get(4)->combinator());
+}
+
+TEST_F(ParserTest, comment_breaking_child_combinator) {
+ Parser p(" a >b/*f>oo*/>c /*fo>o*/>d/*f>oo*/ > e>f { }");
+ scoped_ptr<Ruleset> t(p.ParseRuleset());
+
+ ASSERT_EQ(1, t->selectors().size());
+ const Selector* s = t->selectors().get(0);
+ ASSERT_EQ(6, s->size());
+ EXPECT_EQ(SimpleSelectors::NONE, s->get(0)->combinator());
+ EXPECT_EQ(SimpleSelectors::CHILD, s->get(1)->combinator());
+ EXPECT_EQ(SimpleSelectors::CHILD, s->get(2)->combinator());
+ EXPECT_EQ(SimpleSelectors::CHILD, s->get(3)->combinator());
+ EXPECT_EQ(SimpleSelectors::CHILD, s->get(4)->combinator());
+ EXPECT_EQ(SimpleSelectors::CHILD, s->get(5)->combinator());
+}
+
+TEST_F(ParserTest, ruleset_starts_with_combinator) {
+ Parser p(" >a { }");
+ scoped_ptr<Ruleset> t(p.ParseRuleset());
+
+ // This is not a valid selector.
+ EXPECT_FALSE(t.get());
+}
+
+
+TEST_F(ParserTest, simple_selectors) {
+ // First, a basic case
+ scoped_ptr<Parser> a(new Parser("*[lang|=fr]"));
+ scoped_ptr<SimpleSelectors> t(a->ParseSimpleSelectors(false));
+
+ EXPECT_EQ(SimpleSelectors::NONE, t->combinator());
+
+ {
+ const SimpleSelector* c = t->get(0);
+ ASSERT_TRUE(c != NULL);
+ ASSERT_EQ(SimpleSelector::UNIVERSAL, c->type());
+ }
+ {
+ const SimpleSelector* c = t->get(1);
+ ASSERT_TRUE(c != NULL);
+ ASSERT_EQ(SimpleSelector::BEGIN_HYPHEN_ATTRIBUTE, c->type());
+ EXPECT_EQ("lang", UnicodeTextToUTF8(c->attribute()));
+ EXPECT_EQ("fr", UnicodeTextToUTF8(c->value()));
+ }
+
+ // Now, a very complex one.
+ a.reset(new Parser(
+ "> P:first_child:hover[class~='hidden'][width]#content"
+ "[id*=logo][id^=logo][id$=\"logo\"]"
+ "[lang=en].anotherclass.moreclass #next"));
+ t.reset(a->ParseSimpleSelectors(true));
+
+ EXPECT_EQ(SimpleSelectors::CHILD, t->combinator());
+
+ // We're going to go through the conditions in reverse order, for kicks.
+ SimpleSelectors::const_reverse_iterator it = t->rbegin();
+
+ // .moreclass
+ {
+ SimpleSelector* c = *it++;
+ ASSERT_EQ(SimpleSelector::CLASS, c->type());
+ EXPECT_EQ("moreclass", UnicodeTextToUTF8(c->value()));
+ }
+
+ // .anotherclass
+ {
+ SimpleSelector* c = *it++;
+ ASSERT_EQ(SimpleSelector::CLASS, c->type());
+ EXPECT_EQ("anotherclass", UnicodeTextToUTF8(c->value()));
+ }
+
+ // EXACT_ATTRIBUTE [lang=en]
+ {
+ SimpleSelector* c = *it++;
+ ASSERT_EQ(SimpleSelector::EXACT_ATTRIBUTE, c->type());
+ EXPECT_EQ("lang", UnicodeTextToUTF8(c->attribute()));
+ EXPECT_EQ("en", UnicodeTextToUTF8(c->value()));
+ }
+
+ // END_WITH_ATTRIBUTE [id$="logo"]
+ {
+ SimpleSelector* c = *it++;
+ ASSERT_EQ(SimpleSelector::END_WITH_ATTRIBUTE, c->type());
+ EXPECT_EQ("id", UnicodeTextToUTF8(c->attribute()));
+ EXPECT_EQ("logo", UnicodeTextToUTF8(c->value()));
+ }
+
+ // BEGIN_WITH_ATTRIBUTE [id^=logo]
+ {
+ SimpleSelector* c = *it++;
+ ASSERT_EQ(SimpleSelector::BEGIN_WITH_ATTRIBUTE, c->type());
+ EXPECT_EQ("id", UnicodeTextToUTF8(c->attribute()));
+ EXPECT_EQ("logo", UnicodeTextToUTF8(c->value()));
+ }
+
+ // SUBSTRING_ATTRIBUTE [id*=logo]
+ {
+ SimpleSelector* c = *it++;
+ ASSERT_EQ(SimpleSelector::SUBSTRING_ATTRIBUTE, c->type());
+ EXPECT_EQ("id", UnicodeTextToUTF8(c->attribute()));
+ EXPECT_EQ("logo", UnicodeTextToUTF8(c->value()));
+ }
+
+ // ID #content
+ {
+ SimpleSelector* c = *it++;
+ ASSERT_EQ(SimpleSelector::ID, c->type());
+ EXPECT_EQ("content", UnicodeTextToUTF8(c->value()));
+ }
+
+ // EXIST_ATTRIBUTE [width]
+ {
+ SimpleSelector* c = *it++;
+ ASSERT_EQ(SimpleSelector::EXIST_ATTRIBUTE, c->type());
+ EXPECT_EQ("width", UnicodeTextToUTF8(c->attribute()));
+ }
+
+ // ONE_OF_ATTRIBUTE [class~=hidden]
+ {
+ SimpleSelector* c = *it++;
+ ASSERT_EQ(SimpleSelector::ONE_OF_ATTRIBUTE, c->type());
+ EXPECT_EQ("class", UnicodeTextToUTF8(c->attribute()));
+ EXPECT_EQ("hidden", UnicodeTextToUTF8(c->value()));
+ }
+
+ // PSEUDOCLASS :hover
+ {
+ SimpleSelector* c = *it++;
+ ASSERT_EQ(SimpleSelector::PSEUDOCLASS, c->type());
+ EXPECT_EQ("hover", UnicodeTextToUTF8(c->pseudoclass()));
+ }
+
+ // PSEUDOCLASS :first_child
+ {
+ SimpleSelector* c = *it++;
+ ASSERT_EQ(SimpleSelector::PSEUDOCLASS, c->type());
+ EXPECT_EQ("first_child", UnicodeTextToUTF8(c->pseudoclass()));
+ }
+
+ // P
+ {
+ SimpleSelector* c = *it++;
+ ASSERT_EQ(SimpleSelector::ELEMENT_TYPE, c->type());
+ EXPECT_EQ("P", UnicodeTextToUTF8(c->element_text()));
+ EXPECT_EQ(kHtmlTagP, c->element_type());
+ }
+
+
+ ASSERT_TRUE(it == t->rend());
+}
+
+TEST_F(ParserTest, bad_simple_selectors) {
+ scoped_ptr<Parser> a;
+ scoped_ptr<SimpleSelectors> t;
+
+ // valid or not?
+ a.reset(new Parser(""));
+ t.reset(a->ParseSimpleSelectors(false));
+ EXPECT_FALSE(t.get());
+
+ a.reset(new Parser("{}"));
+ t.reset(a->ParseSimpleSelectors(false));
+ EXPECT_FALSE(t.get());
+
+ a.reset(new Parser("#"));
+ t.reset(a->ParseSimpleSelectors(false));
+ EXPECT_FALSE(t.get());
+
+ a.reset(new Parser("# {"));
+ t.reset(a->ParseSimpleSelectors(false));
+ EXPECT_FALSE(t.get());
+
+ a.reset(new Parser("#{"));
+ t.reset(a->ParseSimpleSelectors(false));
+ EXPECT_FALSE(t.get());
+
+ a.reset(new Parser("##"));
+ t.reset(a->ParseSimpleSelectors(false));
+ EXPECT_FALSE(t.get());
+
+ a.reset(new Parser("*[class="));
+ t.reset(a->ParseSimpleSelectors(false));
+ EXPECT_FALSE(t.get());
+
+ a.reset(new Parser("*[class=hidden];"));
+ t.reset(a->ParseSimpleSelectors(false));
+ EXPECT_FALSE(t.get());
+
+ a.reset(new Parser("*[class=hidden].;"));
+ t.reset(a->ParseSimpleSelectors(false));
+ EXPECT_FALSE(t.get());
+
+ a.reset(new Parser("#a {"));
+ t.reset(a->ParseSimpleSelectors(false));
+ EXPECT_TRUE(t.get());
+}
+
+TEST_F(ParserTest, selectors) {
+ scoped_ptr<Parser> a(new Parser("h1 p #id {"));
+ scoped_ptr<Selectors> t(a->ParseSelectors());
+ EXPECT_TRUE(t.get());
+ ASSERT_EQ(1, t->size());
+ EXPECT_EQ(3, (*t)[0]->size());
+ EXPECT_EQ('{', *a->getpos());
+
+ a.reset(new Parser(" h1 p #id , div.p > h2 > div.t #id"));
+ t.reset(a->ParseSelectors());
+ EXPECT_TRUE(t.get());
+ ASSERT_EQ(2, t->size());
+ EXPECT_EQ(3, (*t)[0]->size());
+ EXPECT_EQ(4, (*t)[1]->size());
+ EXPECT_TRUE(a->Done());
+
+ a.reset(new Parser("/*c*/h1 p #id/*c*/,/*c*/div.p > h2 > div.t #id/*c*/"));
+ t.reset(a->ParseSelectors());
+ EXPECT_TRUE(t.get());
+ ASSERT_EQ(2, t->size());
+ EXPECT_EQ(3, (*t)[0]->size());
+ EXPECT_EQ(4, (*t)[1]->size());
+ EXPECT_TRUE(a->Done());
+
+ a.reset(new Parser("{}"));
+ t.reset(a->ParseSelectors());
+ EXPECT_FALSE(t.get());
+ EXPECT_EQ('{', *a->getpos());
+
+ a.reset(new Parser(""));
+ t.reset(a->ParseSelectors());
+ EXPECT_FALSE(t.get());
+ EXPECT_TRUE(a->Done());
+
+ a.reset(new Parser(" ,h1 p #id {"));
+ t.reset(a->ParseSelectors());
+ EXPECT_FALSE(t.get());
+ EXPECT_EQ('{', *a->getpos());
+
+ a.reset(new Parser(" , {"));
+ t.reset(a->ParseSelectors());
+ EXPECT_FALSE(t.get());
+ EXPECT_EQ('{', *a->getpos());
+
+ a.reset(new Parser("h1 p #id, {"));
+ t.reset(a->ParseSelectors());
+ EXPECT_FALSE(t.get());
+ EXPECT_EQ('{', *a->getpos());
+
+ a.reset(new Parser("h1 p #id, {"));
+ t.reset(a->ParseSelectors());
+ EXPECT_FALSE(t.get());
+ EXPECT_EQ('{', *a->getpos());
+
+ a.reset(new Parser("h1 p #id;"));
+ t.reset(a->ParseSelectors());
+ EXPECT_FALSE(t.get());
+ EXPECT_TRUE(a->Done());
+
+ a.reset(new Parser(" h1 p[class=/*{*/ #id , h2 #id"));
+ t.reset(a->ParseSelectors());
+ EXPECT_FALSE(t.get());
+ EXPECT_TRUE(a->Done());
+
+ a.reset(new Parser(" h1 #id. , h2 #id"));
+ t.reset(a->ParseSelectors());
+ EXPECT_FALSE(t.get());
+ EXPECT_TRUE(a->Done());
+}
+
+TEST_F(ParserTest, rulesets) {
+ scoped_ptr<Parser> a(new Parser("h1 p #id ;"));
+ scoped_ptr<Ruleset> t(a->ParseRuleset());
+
+ EXPECT_EQ(static_cast<Ruleset *>(NULL), t.get());
+
+ a.reset(new Parser(", { }"));
+ t.reset(a->ParseRuleset());
+
+ EXPECT_FALSE(t.get());
+
+ a.reset(new Parser(", h1 p #id, { };"));
+ t.reset(a->ParseRuleset());
+
+ EXPECT_FALSE(t.get());
+
+ a.reset(new Parser(
+ "h1 p + #id { font-size: 7px; width:10pt !important;}"));
+ t.reset(a->ParseRuleset());
+
+ ASSERT_EQ(1, (*t).selectors().size());
+ ASSERT_EQ(3, (*t).selector(0).size());
+ EXPECT_EQ(SimpleSelectors::SIBLING,
+ (*(*t).selectors()[0])[2]->combinator());
+ ASSERT_EQ(2, (*t).declarations().size());
+ EXPECT_EQ(false, (*t).declarations()[0]->IsImportant());
+ EXPECT_EQ(Property::WIDTH, (*t).declarations()[1]->prop());
+ EXPECT_EQ(true, (*t).declarations()[1]->IsImportant());
+
+ a.reset(new Parser("h1 p + #id , h1:first_child { font-size: 10px; }"));
+ t.reset(a->ParseRuleset());
+
+ ASSERT_EQ(2, (*t).selectors().size());
+ ASSERT_EQ(3, (*t).selector(0).size());
+ ASSERT_EQ(1, (*t).selector(1).size());
+ EXPECT_EQ(SimpleSelectors::SIBLING,
+ (*(*t).selectors()[0])[2]->combinator());
+ ASSERT_EQ(1, (*t).declarations().size());
+ EXPECT_EQ(false, (*t).declarations()[0]->IsImportant());
+}
+
+TEST_F(ParserTest, atrules) {
+ scoped_ptr<Parser> a(new Parser(
+ "@IMPORT url(assets/style.css) screen,printer"));
+ scoped_ptr<Stylesheet> t(new Stylesheet());
+ a->ParseAtrule(t.get());
+
+ ASSERT_EQ(1, (*t).imports().size());
+ EXPECT_EQ("assets/style.css", UnicodeTextToUTF8((*t).import(0).link));
+ EXPECT_EQ(true, a->Done());
+
+ a.reset(new Parser("@charset \"ISO-8859-1\" ;"));
+ t.reset(new Stylesheet());
+ a->ParseAtrule(t.get());
+
+ EXPECT_EQ(true, a->Done());
+
+ a.reset(new Parser(
+ "@media print,screen {\n\tbody { font-size: 10pt }\n}"));
+ t.reset(new Stylesheet());
+ a->ParseAtrule(t.get());
+
+ ASSERT_EQ(1, (*t).rulesets().size());
+ ASSERT_EQ(1, (*t).ruleset(0).selectors().size());
+ ASSERT_EQ(2, (*t).ruleset(0).media().size());
+ EXPECT_EQ("print", UnicodeTextToUTF8((*t).ruleset(0).medium(0)));
+ EXPECT_EQ("screen", UnicodeTextToUTF8((*t).ruleset(0).medium(1)));
+ ASSERT_EQ(1, (*(*t).ruleset(0).selectors()[0]).size());
+ EXPECT_EQ(kHtmlTagBody,
+ (*t).ruleset(0).selector(0)[0]->get(0)->element_type());
+ EXPECT_EQ(Property::FONT_SIZE,
+ (*t).ruleset(0).declarations()[0]->prop());
+ EXPECT_EQ(true, a->Done());
+
+ a.reset(new Parser(
+ "@page :left { margin-left: 4cm; margin-right: 3cm; }"));
+ t.reset(new Stylesheet());
+ a->ParseAtrule(t.get());
+
+ EXPECT_EQ(0, (*t).rulesets().size());
+ EXPECT_EQ(true, a->Done());
+
+ // Make sure media strings can be shared between multiple rulesets.
+ a.reset(new Parser(
+ "@media print { a { color: red; } p { color: blue; } }"));
+ t.reset(new Stylesheet());
+ a->ParseAtrule(t.get());
+
+ ASSERT_EQ(2, t->rulesets().size());
+ EXPECT_EQ("print", UnicodeTextToUTF8(t->ruleset(0).medium(0)));
+ EXPECT_EQ("print", UnicodeTextToUTF8(t->ruleset(1).medium(0)));
+ t->ToString(); // Make sure it can be written as a string.
+}
+
+TEST_F(ParserTest, stylesheets) {
+ scoped_ptr<Parser> a(new Parser("\n"
+ "\t@import \"mystyle.css\" all; "
+ "@import url(\"mystyle.css\" );\n"
+ "\tBODY {\n"
+ "color:black !important; \n"
+ "background: white !important; }\n"
+ "* {\n"
+ "\tcolor: inherit !important;\n"
+ "background: transparent;\n"
+ "}\n"
+ "\n"
+ "<!-- html comments * { font-size: 1 } -->\n"
+ "H1 + *[REL-up] {}"));
+
+ scoped_ptr<Stylesheet> t(a->ParseStylesheet());
+ EXPECT_EQ(Parser::kNoError, a->errors_seen_mask());
+ ASSERT_EQ(2, t->imports().size());
+ EXPECT_EQ("mystyle.css", UnicodeTextToUTF8(t->import(0).link));
+ ASSERT_EQ(1, t->import(0).media.size());
+ EXPECT_EQ("all", UnicodeTextToUTF8(t->import(0).media[0]));
+ EXPECT_EQ("mystyle.css", UnicodeTextToUTF8(t->import(1).link));
+ // html-style comment should NOT work
+ EXPECT_EQ(4, t->rulesets().size());
+ EXPECT_TRUE(a->Done());
+}
+
+TEST_F(ParserTest, ParseRawStylesheetDoesNotExpand) {
+ {
+ Parser p("a { background: none; }");
+ scoped_ptr<Stylesheet> stylesheet(p.ParseRawStylesheet());
+ ASSERT_EQ(1, stylesheet->rulesets().size());
+ ASSERT_EQ(1, stylesheet->ruleset(0).declarations().size());
+ ASSERT_EQ(1, stylesheet->ruleset(0).declaration(0).values()->size());
+ EXPECT_TRUE(p.Done());
+ }
+ {
+ Parser p("a { font: 12px verdana; }");
+ scoped_ptr<Stylesheet> stylesheet(p.ParseRawStylesheet());
+ ASSERT_EQ(1, stylesheet->rulesets().size());
+ ASSERT_EQ(1, stylesheet->ruleset(0).declarations().size());
+ const Values& values = *stylesheet->ruleset(0).declaration(0).values();
+ ASSERT_EQ(6, values.size());
+ // ParseRaw will expand the values out to:
+ // font: normal normal normal 12px/normal verdana
+ // But it will not expand out the six other declarations.
+ // TODO(sligocki): There has got to be a nicer way to test this.
+ EXPECT_EQ(Identifier::NORMAL, values[0]->GetIdentifier().ident());
+ EXPECT_EQ(Identifier::NORMAL, values[1]->GetIdentifier().ident());
+ EXPECT_EQ(Identifier::NORMAL, values[2]->GetIdentifier().ident());
+ EXPECT_EQ(12.0, values[3]->GetFloatValue());
+ EXPECT_EQ(Value::PX, values[3]->GetDimension());
+ EXPECT_EQ(Identifier::NORMAL, values[4]->GetIdentifier().ident());
+ EXPECT_EQ("verdana", UnicodeTextToUTF8(values[5]->GetIdentifierText()));
+ EXPECT_TRUE(p.Done());
+ }
+}
+
+TEST_F(ParserTest, ParseStylesheetDoesExpand) {
+ {
+ Parser p("a { background: none; }");
+ scoped_ptr<Stylesheet> stylesheet(p.ParseStylesheet());
+ ASSERT_EQ(1, stylesheet->rulesets().size());
+ const Declarations& declarations = stylesheet->ruleset(0).declarations();
+ ASSERT_EQ(7, declarations.size());
+ EXPECT_EQ(Property::BACKGROUND, declarations[0]->prop());
+ EXPECT_EQ(Property::BACKGROUND_COLOR, declarations[1]->prop());
+ EXPECT_EQ(Property::BACKGROUND_IMAGE, declarations[2]->prop());
+ EXPECT_EQ(Property::BACKGROUND_REPEAT, declarations[3]->prop());
+ EXPECT_EQ(Property::BACKGROUND_ATTACHMENT, declarations[4]->prop());
+ EXPECT_EQ(Property::BACKGROUND_POSITION_X, declarations[5]->prop());
+ EXPECT_EQ(Property::BACKGROUND_POSITION_Y, declarations[6]->prop());
+ EXPECT_TRUE(p.Done());
+ }
+ {
+ Parser p("a { font: 12px verdana; }");
+ scoped_ptr<Stylesheet> stylesheet(p.ParseStylesheet());
+ ASSERT_EQ(1, stylesheet->rulesets().size());
+ const Declarations& declarations = stylesheet->ruleset(0).declarations();
+ ASSERT_EQ(7, declarations.size());
+ EXPECT_EQ(Property::FONT, declarations[0]->prop());
+ EXPECT_EQ(6, declarations[0]->values()->size());
+ EXPECT_EQ(Property::FONT_STYLE, declarations[1]->prop());
+ EXPECT_EQ(Property::FONT_VARIANT, declarations[2]->prop());
+ EXPECT_EQ(Property::FONT_WEIGHT, declarations[3]->prop());
+ EXPECT_EQ(Property::FONT_SIZE, declarations[4]->prop());
+ EXPECT_EQ(Property::LINE_HEIGHT, declarations[5]->prop());
+ EXPECT_EQ(Property::FONT_FAMILY, declarations[6]->prop());
+ }
+}
+
+TEST_F(ParserTest, percentage_colors) {
+ Value hundred(100.0, Value::PERCENT);
+ EXPECT_EQ(255, Parser::ValueToRGB(&hundred));
+ Value zero(0.0, Value::PERCENT);
+ EXPECT_EQ(0, Parser::ValueToRGB(&zero));
+}
+
+TEST_F(ParserTest, value_equality) {
+ Value hundred(100.0, Value::PERCENT);
+ Value hundred2(100.0, Value::PERCENT);
+ Value zero(0.0, Value::PERCENT);
+ Identifier auto_ident(Identifier::AUTO);
+ Value ident(auto_ident);
+ EXPECT_TRUE(hundred.Equals(hundred2));
+ EXPECT_FALSE(hundred.Equals(zero));
+ EXPECT_FALSE(hundred.Equals(ident));
+}
+
+TEST_F(ParserTest, Utf8Error) {
+ Parser p("font-family: \"\xCB\xCE\xCC\xE5\"");
+ scoped_ptr<Declarations> declarations(p.ParseDeclarations());
+ EXPECT_EQ(1, declarations->size());
+ EXPECT_EQ(Parser::kUtf8Error, p.errors_seen_mask());
+}
+
+TEST_F(ParserTest, DeclarationError) {
+ Parser p("font-family ; ");
+ scoped_ptr<Declarations> declarations(p.ParseDeclarations());
+ EXPECT_EQ(0, declarations->size());
+ EXPECT_EQ(Parser::kDeclarationError, p.errors_seen_mask());
+}
+
+TEST_F(ParserTest, SelectorError) {
+ Parser p(".bold: { font-weight: bold }");
+ scoped_ptr<Stylesheet> stylesheet(p.ParseStylesheet());
+ EXPECT_EQ(0, stylesheet->rulesets().size());
+ EXPECT_EQ(Parser::kSelectorError, p.errors_seen_mask());
+}
+
+TEST_F(ParserTest, FunctionError) {
+ Parser p("box-shadow: -1px -2px 2px rgba(0, 0, 0, .15)");
+ scoped_ptr<Declarations> declarations(p.ParseDeclarations());
+ EXPECT_EQ(1, declarations->size());
+ EXPECT_EQ(Parser::kFunctionError, p.errors_seen_mask());
+}
+
+TEST_F(ParserTest, MediaError) {
+ Parser p("@media screen and (max-width: 290px) {}");
+ scoped_ptr<Stylesheet> stylesheet(p.ParseStylesheet());
+ EXPECT_EQ(0, stylesheet->rulesets().size());
+ EXPECT_EQ(Parser::kMediaError, p.errors_seen_mask());
+}
+
+TEST_F(ParserTest, AcceptCorrectValues) {
+ // http://code.google.com/p/modpagespeed/issues/detail?id=128
+ Parser p("list-style-type: none");
+ scoped_ptr<Declarations> declarations(p.ParseDeclarations());
+ EXPECT_EQ(1, declarations->size());
+ EXPECT_EQ(Parser::kNoError, p.errors_seen_mask());
+ EXPECT_EQ("list-style-type: none", declarations->ToString());
+}
+
+} // namespace
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/property.cc b/trunk/src/third_party/css_parser/src/webutil/css/property.cc
new file mode 100644
index 0000000..341590d
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/property.cc
@@ -0,0 +1,1150 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/* C++ code produced by gperf version 3.0.3 */
+/* Computed positions: -k'1-2,13,$' */
+
+#if !((' ' == 32) && ('!' == 33) && ('"' == 34) && ('#' == 35) \
+ && ('%' == 37) && ('&' == 38) && ('\'' == 39) && ('(' == 40) \
+ && (')' == 41) && ('*' == 42) && ('+' == 43) && (',' == 44) \
+ && ('-' == 45) && ('.' == 46) && ('/' == 47) && ('0' == 48) \
+ && ('1' == 49) && ('2' == 50) && ('3' == 51) && ('4' == 52) \
+ && ('5' == 53) && ('6' == 54) && ('7' == 55) && ('8' == 56) \
+ && ('9' == 57) && (':' == 58) && (';' == 59) && ('<' == 60) \
+ && ('=' == 61) && ('>' == 62) && ('?' == 63) && ('A' == 65) \
+ && ('B' == 66) && ('C' == 67) && ('D' == 68) && ('E' == 69) \
+ && ('F' == 70) && ('G' == 71) && ('H' == 72) && ('I' == 73) \
+ && ('J' == 74) && ('K' == 75) && ('L' == 76) && ('M' == 77) \
+ && ('N' == 78) && ('O' == 79) && ('P' == 80) && ('Q' == 81) \
+ && ('R' == 82) && ('S' == 83) && ('T' == 84) && ('U' == 85) \
+ && ('V' == 86) && ('W' == 87) && ('X' == 88) && ('Y' == 89) \
+ && ('Z' == 90) && ('[' == 91) && ('\\' == 92) && (']' == 93) \
+ && ('^' == 94) && ('_' == 95) && ('a' == 97) && ('b' == 98) \
+ && ('c' == 99) && ('d' == 100) && ('e' == 101) && ('f' == 102) \
+ && ('g' == 103) && ('h' == 104) && ('i' == 105) && ('j' == 106) \
+ && ('k' == 107) && ('l' == 108) && ('m' == 109) && ('n' == 110) \
+ && ('o' == 111) && ('p' == 112) && ('q' == 113) && ('r' == 114) \
+ && ('s' == 115) && ('t' == 116) && ('u' == 117) && ('v' == 118) \
+ && ('w' == 119) && ('x' == 120) && ('y' == 121) && ('z' == 122) \
+ && ('{' == 123) && ('|' == 124) && ('}' == 125) && ('~' == 126))
+/* The character set is not based on ISO-646. */
+#error "gperf generated tables don't work with this execution character set. Please report a bug to <bug-gnu-gperf@gnu.org>."
+#endif
+
+#line 1 "webutil/css/property.gperf"
+
+#include "webutil/css/property.h"
+
+#include "base/googleinit.h"
+#include "base/logging.h"
+#include "webutil/css/string_util.h"
+
+namespace Css {
+#line 11 "webutil/css/property.gperf"
+struct props {
+ const char *name;
+ Property::Prop id;
+};
+enum
+ {
+ TOTAL_KEYWORDS = 178,
+ MIN_WORD_LENGTH = 3,
+ MAX_WORD_LENGTH = 43,
+ MIN_HASH_VALUE = 17,
+ MAX_HASH_VALUE = 563
+ };
+
+/* maximum key range = 547, duplicates = 0 */
+
+#ifndef GPERF_DOWNCASE
+#define GPERF_DOWNCASE 1
+static unsigned char gperf_downcase[256] =
+ {
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
+ 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29,
+ 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,
+ 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59,
+ 60, 61, 62, 63, 64, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106,
+ 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121,
+ 122, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104,
+ 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119,
+ 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134,
+ 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149,
+ 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164,
+ 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179,
+ 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194,
+ 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209,
+ 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224,
+ 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239,
+ 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254,
+ 255
+ };
+#endif
+
+#ifndef GPERF_CASE_STRNCMP
+#define GPERF_CASE_STRNCMP 1
+static int
+gperf_case_strncmp (register const char *s1, register const char *s2, register unsigned int n)
+{
+ for (; n > 0;)
+ {
+ unsigned char c1 = gperf_downcase[(unsigned char)*s1++];
+ unsigned char c2 = gperf_downcase[(unsigned char)*s2++];
+ if (c1 != 0 && c1 == c2)
+ {
+ n--;
+ continue;
+ }
+ return (int)c1 - (int)c2;
+ }
+ return 0;
+}
+#endif
+
+class PropertyMapper
+{
+private:
+ static inline unsigned int hash (const char *str, unsigned int len);
+public:
+ static const struct props *in_word_set (const char *str, unsigned int len);
+};
+
+inline unsigned int
+PropertyMapper::hash (register const char *str, register unsigned int len)
+{
+ static const unsigned short asso_values[] =
+ {
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 0, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 50, 25, 130, 140, 0,
+ 144, 60, 155, 150, 564, 0, 130, 70, 105, 10,
+ 35, 5, 95, 185, 15, 80, 20, 5, 10, 210,
+ 0, 564, 564, 564, 564, 564, 564, 50, 25, 130,
+ 140, 0, 144, 60, 155, 150, 564, 0, 130, 70,
+ 105, 10, 35, 5, 95, 185, 15, 80, 20, 5,
+ 10, 210, 0, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564, 564, 564, 564, 564,
+ 564, 564, 564, 564, 564, 564
+ };
+ register int hval = len;
+
+ switch (hval)
+ {
+ default:
+ hval += asso_values[(unsigned char)str[12]];
+ /*FALLTHROUGH*/
+ case 12:
+ case 11:
+ case 10:
+ case 9:
+ case 8:
+ case 7:
+ case 6:
+ case 5:
+ case 4:
+ case 3:
+ case 2:
+ hval += asso_values[(unsigned char)str[1]];
+ /*FALLTHROUGH*/
+ case 1:
+ hval += asso_values[(unsigned char)str[0]];
+ break;
+ }
+ return hval + asso_values[(unsigned char)str[len - 1]];
+}
+
+static const struct props wordlist[] =
+ {
+#line 170 "webutil/css/property.gperf"
+ {"z-index", Property::Z_INDEX},
+#line 109 "webutil/css/property.gperf"
+ {"-webkit-nbsp-mode", Property::_WEBKIT_NBSP_MODE},
+#line 93 "webutil/css/property.gperf"
+ {"-webkit-line-break", Property::_WEBKIT_LINE_BREAK},
+#line 30 "webutil/css/property.gperf"
+ {"-webkit-border-image", Property::_WEBKIT_BORDER_IMAGE},
+#line 142 "webutil/css/property.gperf"
+ {"text-overline", Property::TEXT_OVERLINE},
+#line 148 "webutil/css/property.gperf"
+ {"text-shadow", Property::TEXT_SHADOW},
+#line 144 "webutil/css/property.gperf"
+ {"text-overline-mode", Property::TEXT_OVERLINE_MODE},
+#line 145 "webutil/css/property.gperf"
+ {"text-overline-style", Property::TEXT_OVERLINE_STYLE},
+#line 141 "webutil/css/property.gperf"
+ {"text-overflow", Property::TEXT_OVERFLOW},
+#line 162 "webutil/css/property.gperf"
+ {"-webkit-user-select", Property::_WEBKIT_USER_SELECT},
+#line 135 "webutil/css/property.gperf"
+ {"text-indent", Property::TEXT_INDENT},
+#line 116 "webutil/css/property.gperf"
+ {"overflow", Property::OVERFLOW},
+#line 156 "webutil/css/property.gperf"
+ {"-webkit-text-size-adjust", Property::_WEBKIT_TEXT_SIZE_ADJUST},
+#line 174 "webutil/css/property.gperf"
+ {"border-style", Property::BORDER_STYLE},
+#line 58 "webutil/css/property.gperf"
+ {"-webkit-box-orient", Property::_WEBKIT_BOX_ORIENT},
+#line 117 "webutil/css/property.gperf"
+ {"overflow-x", Property::OVERFLOW_X},
+#line 44 "webutil/css/property.gperf"
+ {"border-right-style", Property::BORDER_RIGHT_STYLE},
+#line 192 "webutil/css/property.gperf"
+ {"-webkit-text-decorations-in-effect", Property::_WEBKIT_TEXT_DECORATIONS_IN_EFFECT},
+#line 59 "webutil/css/property.gperf"
+ {"-webkit-box-pack", Property::_WEBKIT_BOX_PACK},
+#line 84 "webutil/css/property.gperf"
+ {"-webkit-line-clamp", Property::_WEBKIT_LINE_CLAMP},
+#line 168 "webutil/css/property.gperf"
+ {"word-wrap", Property::WORD_WRAP},
+#line 178 "webutil/css/property.gperf"
+ {"border-left", Property::BORDER_LEFT},
+#line 176 "webutil/css/property.gperf"
+ {"border-right", Property::BORDER_RIGHT},
+#line 158 "webutil/css/property.gperf"
+ {"top", Property::TOP},
+#line 43 "webutil/css/property.gperf"
+ {"border-top-style", Property::BORDER_TOP_STYLE},
+#line 16 "webutil/css/property.gperf"
+ {"-webkit-appearance", Property::_WEBKIT_APPEARANCE},
+#line 57 "webutil/css/property.gperf"
+ {"-webkit-box-ordinal-group", Property::_WEBKIT_BOX_ORDINAL_GROUP},
+#line 75 "webutil/css/property.gperf"
+ {"-webkit-font-size-delta", Property::_WEBKIT_FONT_SIZE_DELTA},
+#line 175 "webutil/css/property.gperf"
+ {"border-top", Property::BORDER_TOP},
+#line 160 "webutil/css/property.gperf"
+ {"-webkit-user-drag", Property::_WEBKIT_USER_DRAG},
+#line 29 "webutil/css/property.gperf"
+ {"border-collapse", Property::BORDER_COLLAPSE},
+#line 169 "webutil/css/property.gperf"
+ {"word-spacing", Property::WORD_SPACING},
+#line 27 "webutil/css/property.gperf"
+ {"-webkit-background-size", Property::_WEBKIT_BACKGROUND_SIZE},
+#line 124 "webutil/css/property.gperf"
+ {"page", Property::PAGE},
+#line 132 "webutil/css/property.gperf"
+ {"table-layout", Property::TABLE_LAYOUT},
+#line 20 "webutil/css/property.gperf"
+ {"-webkit-background-composite", Property::_WEBKIT_BACKGROUND_COMPOSITE},
+#line 193 "webutil/css/property.gperf"
+ {"-webkit-rtl-ordering", Property::_WEBKIT_RTL_ORDERING},
+#line 33 "webutil/css/property.gperf"
+ {"-webkit-border-vertical-spacing", Property::_WEBKIT_BORDER_VERTICAL_SPACING},
+#line 183 "webutil/css/property.gperf"
+ {"outline", Property::OUTLINE},
+#line 32 "webutil/css/property.gperf"
+ {"-webkit-border-horizontal-spacing", Property::_WEBKIT_BORDER_HORIZONTAL_SPACING},
+#line 98 "webutil/css/property.gperf"
+ {"-webkit-marquee", Property::_WEBKIT_MARQUEE},
+#line 155 "webutil/css/property.gperf"
+ {"resize", Property::RESIZE},
+#line 126 "webutil/css/property.gperf"
+ {"page-break-before", Property::PAGE_BREAK_BEFORE},
+#line 114 "webutil/css/property.gperf"
+ {"outline-style", Property::OUTLINE_STYLE},
+#line 60 "webutil/css/property.gperf"
+ {"box-sizing", Property::BOX_SIZING},
+#line 103 "webutil/css/property.gperf"
+ {"-webkit-marquee-style", Property::_WEBKIT_MARQUEE_STYLE},
+#line 26 "webutil/css/property.gperf"
+ {"background-repeat", Property::BACKGROUND_REPEAT},
+#line 51 "webutil/css/property.gperf"
+ {"bottom", Property::BOTTOM},
+#line 122 "webutil/css/property.gperf"
+ {"padding-left", Property::PADDING_LEFT},
+#line 24 "webutil/css/property.gperf"
+ {"background-position-x", Property::BACKGROUND_POSITION_X},
+#line 113 "webutil/css/property.gperf"
+ {"outline-offset", Property::OUTLINE_OFFSET},
+#line 18 "webutil/css/property.gperf"
+ {"-webkit-background-clip", Property::_WEBKIT_BACKGROUND_CLIP},
+#line 45 "webutil/css/property.gperf"
+ {"border-bottom-style", Property::BORDER_BOTTOM_STYLE},
+#line 100 "webutil/css/property.gperf"
+ {"-webkit-marquee-increment", Property::_WEBKIT_MARQUEE_INCREMENT},
+#line 17 "webutil/css/property.gperf"
+ {"background-attachment", Property::BACKGROUND_ATTACHMENT},
+#line 120 "webutil/css/property.gperf"
+ {"padding-right", Property::PADDING_RIGHT},
+#line 143 "webutil/css/property.gperf"
+ {"text-overline-color", Property::TEXT_OVERLINE_COLOR},
+#line 133 "webutil/css/property.gperf"
+ {"text-align", Property::TEXT_ALIGN},
+#line 119 "webutil/css/property.gperf"
+ {"padding-top", Property::PADDING_TOP},
+#line 138 "webutil/css/property.gperf"
+ {"text-line-through-mode", Property::TEXT_LINE_THROUGH_MODE},
+#line 139 "webutil/css/property.gperf"
+ {"text-line-through-style", Property::TEXT_LINE_THROUGH_STYLE},
+#line 150 "webutil/css/property.gperf"
+ {"text-underline", Property::TEXT_UNDERLINE},
+#line 172 "webutil/css/property.gperf"
+ {"border", Property::BORDER},
+#line 152 "webutil/css/property.gperf"
+ {"text-underline-mode", Property::TEXT_UNDERLINE_MODE},
+#line 153 "webutil/css/property.gperf"
+ {"text-underline-style", Property::TEXT_UNDERLINE_STYLE},
+#line 173 "webutil/css/property.gperf"
+ {"border-color", Property::BORDER_COLOR},
+#line 105 "webutil/css/property.gperf"
+ {"max-height", Property::MAX_HEIGHT},
+#line 92 "webutil/css/property.gperf"
+ {"margin-left", Property::MARGIN_LEFT},
+#line 90 "webutil/css/property.gperf"
+ {"margin-right", Property::MARGIN_RIGHT},
+#line 40 "webutil/css/property.gperf"
+ {"border-right-color", Property::BORDER_RIGHT_COLOR},
+#line 82 "webutil/css/property.gperf"
+ {"left", Property::LEFT},
+#line 184 "webutil/css/property.gperf"
+ {"padding", Property::PADDING},
+#line 39 "webutil/css/property.gperf"
+ {"border-top-color", Property::BORDER_TOP_COLOR},
+#line 128 "webutil/css/property.gperf"
+ {"position", Property::POSITION},
+#line 157 "webutil/css/property.gperf"
+ {"-webkit-dashboard-region", Property::_WEBKIT_DASHBOARD_REGION},
+#line 21 "webutil/css/property.gperf"
+ {"background-image", Property::BACKGROUND_IMAGE},
+#line 65 "webutil/css/property.gperf"
+ {"content", Property::CONTENT},
+#line 74 "webutil/css/property.gperf"
+ {"font-size", Property::FONT_SIZE},
+#line 77 "webutil/css/property.gperf"
+ {"font-style", Property::FONT_STYLE},
+#line 89 "webutil/css/property.gperf"
+ {"margin-top", Property::MARGIN_TOP},
+#line 81 "webutil/css/property.gperf"
+ {"-webkit-highlight", Property::_WEBKIT_HIGHLIGHT},
+#line 165 "webutil/css/property.gperf"
+ {"white-space", Property::WHITE_SPACE},
+#line 66 "webutil/css/property.gperf"
+ {"counter-increment", Property::COUNTER_INCREMENT},
+#line 180 "webutil/css/property.gperf"
+ {"font", Property::FONT},
+#line 54 "webutil/css/property.gperf"
+ {"-webkit-box-flex", Property::_WEBKIT_BOX_FLEX},
+#line 80 "webutil/css/property.gperf"
+ {"height", Property::HEIGHT},
+#line 52 "webutil/css/property.gperf"
+ {"-webkit-box-align", Property::_WEBKIT_BOX_ALIGN},
+#line 94 "webutil/css/property.gperf"
+ {"-webkit-margin-collapse", Property::_WEBKIT_MARGIN_COLLAPSE},
+#line 121 "webutil/css/property.gperf"
+ {"padding-bottom", Property::PADDING_BOTTOM},
+#line 79 "webutil/css/property.gperf"
+ {"font-weight", Property::FONT_WEIGHT},
+#line 78 "webutil/css/property.gperf"
+ {"font-variant", Property::FONT_VARIANT},
+#line 95 "webutil/css/property.gperf"
+ {"-webkit-margin-top-collapse", Property::_WEBKIT_MARGIN_TOP_COLLAPSE},
+#line 67 "webutil/css/property.gperf"
+ {"counter-reset", Property::COUNTER_RESET},
+#line 96 "webutil/css/property.gperf"
+ {"-webkit-margin-bottom-collapse", Property::_WEBKIT_MARGIN_BOTTOM_COLLAPSE},
+#line 177 "webutil/css/property.gperf"
+ {"border-bottom", Property::BORDER_BOTTOM},
+#line 146 "webutil/css/property.gperf"
+ {"text-overline-width", Property::TEXT_OVERLINE_WIDTH},
+#line 97 "webutil/css/property.gperf"
+ {"-webkit-margin-start", Property::_WEBKIT_MARGIN_START},
+#line 123 "webutil/css/property.gperf"
+ {"-webkit-padding-start", Property::_WEBKIT_PADDING_START},
+#line 61 "webutil/css/property.gperf"
+ {"caption-side", Property::CAPTION_SIDE},
+#line 149 "webutil/css/property.gperf"
+ {"text-transform", Property::TEXT_TRANSFORM},
+#line 22 "webutil/css/property.gperf"
+ {"-webkit-background-origin", Property::_WEBKIT_BACKGROUND_ORIGIN},
+#line 19 "webutil/css/property.gperf"
+ {"background-color", Property::BACKGROUND_COLOR},
+#line 163 "webutil/css/property.gperf"
+ {"vertical-align", Property::VERTICAL_ALIGN},
+#line 179 "webutil/css/property.gperf"
+ {"border-width", Property::BORDER_WIDTH},
+#line 55 "webutil/css/property.gperf"
+ {"-webkit-box-flex-group", Property::_WEBKIT_BOX_FLEX_GROUP},
+#line 127 "webutil/css/property.gperf"
+ {"page-break-inside", Property::PAGE_BREAK_INSIDE},
+#line 48 "webutil/css/property.gperf"
+ {"border-right-width", Property::BORDER_RIGHT_WIDTH},
+#line 23 "webutil/css/property.gperf"
+ {"background-position", Property::BACKGROUND_POSITION},
+#line 34 "webutil/css/property.gperf"
+ {"-webkit-border-radius", Property::_WEBKIT_BORDER_RADIUS},
+#line 50 "webutil/css/property.gperf"
+ {"border-left-width", Property::BORDER_LEFT_WIDTH},
+#line 31 "webutil/css/property.gperf"
+ {"border-spacing", Property::BORDER_SPACING},
+#line 99 "webutil/css/property.gperf"
+ {"-webkit-marquee-direction", Property::_WEBKIT_MARQUEE_DIRECTION},
+#line 101 "webutil/css/property.gperf"
+ {"-webkit-marquee-repetition", Property::_WEBKIT_MARQUEE_REPETITION},
+#line 41 "webutil/css/property.gperf"
+ {"border-bottom-color", Property::BORDER_BOTTOM_COLOR},
+#line 35 "webutil/css/property.gperf"
+ {"-webkit-border-top-left-radius", Property::_WEBKIT_BORDER_TOP_LEFT_RADIUS},
+#line 36 "webutil/css/property.gperf"
+ {"-webkit-border-top-right-radius", Property::_WEBKIT_BORDER_TOP_RIGHT_RADIUS},
+#line 37 "webutil/css/property.gperf"
+ {"-webkit-border-bottom-left-radius", Property::_WEBKIT_BORDER_BOTTOM_LEFT_RADIUS},
+#line 38 "webutil/css/property.gperf"
+ {"-webkit-border-bottom-right-radius", Property::_WEBKIT_BORDER_BOTTOM_RIGHT_RADIUS},
+#line 171 "webutil/css/property.gperf"
+ {"background", Property::BACKGROUND},
+#line 137 "webutil/css/property.gperf"
+ {"text-line-through-color", Property::TEXT_LINE_THROUGH_COLOR},
+#line 28 "webutil/css/property.gperf"
+ {"-webkit-binding", Property::_WEBKIT_BINDING},
+#line 182 "webutil/css/property.gperf"
+ {"margin", Property::MARGIN},
+#line 161 "webutil/css/property.gperf"
+ {"-webkit-user-modify", Property::_WEBKIT_USER_MODIFY},
+#line 151 "webutil/css/property.gperf"
+ {"text-underline-color", Property::TEXT_UNDERLINE_COLOR},
+#line 147 "webutil/css/property.gperf"
+ {"-webkit-text-security", Property::_WEBKIT_TEXT_SECURITY},
+#line 46 "webutil/css/property.gperf"
+ {"border-left-style", Property::BORDER_LEFT_STYLE},
+#line 64 "webutil/css/property.gperf"
+ {"color", Property::COLOR},
+#line 107 "webutil/css/property.gperf"
+ {"min-height", Property::MIN_HEIGHT},
+#line 102 "webutil/css/property.gperf"
+ {"-webkit-marquee-speed", Property::_WEBKIT_MARQUEE_SPEED},
+#line 118 "webutil/css/property.gperf"
+ {"overflow-y", Property::OVERFLOW_Y},
+#line 110 "webutil/css/property.gperf"
+ {"opacity", Property::OPACITY},
+#line 130 "webutil/css/property.gperf"
+ {"right", Property::RIGHT},
+#line 71 "webutil/css/property.gperf"
+ {"empty-cells", Property::EMPTY_CELLS},
+#line 53 "webutil/css/property.gperf"
+ {"-webkit-box-direction", Property::_WEBKIT_BOX_DIRECTION},
+#line 91 "webutil/css/property.gperf"
+ {"margin-bottom", Property::MARGIN_BOTTOM},
+#line 129 "webutil/css/property.gperf"
+ {"quotes", Property::QUOTES},
+#line 42 "webutil/css/property.gperf"
+ {"border-left-color", Property::BORDER_LEFT_COLOR},
+#line 49 "webutil/css/property.gperf"
+ {"border-bottom-width", Property::BORDER_BOTTOM_WIDTH},
+#line 136 "webutil/css/property.gperf"
+ {"text-line-through", Property::TEXT_LINE_THROUGH},
+#line 106 "webutil/css/property.gperf"
+ {"max-width", Property::MAX_WIDTH},
+#line 134 "webutil/css/property.gperf"
+ {"text-decoration", Property::TEXT_DECORATION},
+#line 140 "webutil/css/property.gperf"
+ {"text-line-through-width", Property::TEXT_LINE_THROUGH_WIDTH},
+#line 181 "webutil/css/property.gperf"
+ {"list-style", Property::LIST_STYLE},
+#line 112 "webutil/css/property.gperf"
+ {"outline-color", Property::OUTLINE_COLOR},
+#line 72 "webutil/css/property.gperf"
+ {"float", Property::FLOAT},
+#line 154 "webutil/css/property.gperf"
+ {"text-underline-width", Property::TEXT_UNDERLINE_WIDTH},
+#line 111 "webutil/css/property.gperf"
+ {"orphans", Property::ORPHANS},
+#line 104 "webutil/css/property.gperf"
+ {"-webkit-match-nearest-mail-blockquote-color", Property::_WEBKIT_MATCH_NEAREST_MAIL_BLOCKQUOTE_COLOR},
+#line 63 "webutil/css/property.gperf"
+ {"clip", Property::CLIP},
+#line 85 "webutil/css/property.gperf"
+ {"line-height", Property::LINE_HEIGHT},
+#line 83 "webutil/css/property.gperf"
+ {"letter-spacing", Property::LETTER_SPACING},
+#line 68 "webutil/css/property.gperf"
+ {"cursor", Property::CURSOR},
+#line 167 "webutil/css/property.gperf"
+ {"width", Property::WIDTH},
+#line 25 "webutil/css/property.gperf"
+ {"background-position-y", Property::BACKGROUND_POSITION_Y},
+#line 76 "webutil/css/property.gperf"
+ {"font-stretch", Property::FONT_STRETCH},
+#line 56 "webutil/css/property.gperf"
+ {"-webkit-box-lines", Property::_WEBKIT_BOX_LINES},
+#line 131 "webutil/css/property.gperf"
+ {"size", Property::SIZE},
+#line 125 "webutil/css/property.gperf"
+ {"page-break-after", Property::PAGE_BREAK_AFTER},
+#line 166 "webutil/css/property.gperf"
+ {"widows", Property::WIDOWS},
+#line 159 "webutil/css/property.gperf"
+ {"unicode-bidi", Property::UNICODE_BIDI},
+#line 47 "webutil/css/property.gperf"
+ {"border-top-width", Property::BORDER_TOP_WIDTH},
+#line 62 "webutil/css/property.gperf"
+ {"clear", Property::CLEAR},
+#line 86 "webutil/css/property.gperf"
+ {"list-style-image", Property::LIST_STYLE_IMAGE},
+#line 73 "webutil/css/property.gperf"
+ {"font-family", Property::FONT_FAMILY},
+#line 108 "webutil/css/property.gperf"
+ {"min-width", Property::MIN_WIDTH},
+#line 164 "webutil/css/property.gperf"
+ {"visibility", Property::VISIBILITY},
+#line 69 "webutil/css/property.gperf"
+ {"direction", Property::DIRECTION},
+#line 115 "webutil/css/property.gperf"
+ {"outline-width", Property::OUTLINE_WIDTH},
+#line 87 "webutil/css/property.gperf"
+ {"list-style-position", Property::LIST_STYLE_POSITION},
+#line 190 "webutil/css/property.gperf"
+ {"scrollbar-track-color", Property::SCROLLBAR_TRACK_COLOR},
+#line 186 "webutil/css/property.gperf"
+ {"scrollbar-shadow-color", Property::SCROLLBAR_SHADOW_COLOR},
+#line 187 "webutil/css/property.gperf"
+ {"scrollbar-highlight-color", Property::SCROLLBAR_HIGHLIGHT_COLOR},
+#line 88 "webutil/css/property.gperf"
+ {"list-style-type", Property::LIST_STYLE_TYPE},
+#line 70 "webutil/css/property.gperf"
+ {"display", Property::DISPLAY},
+#line 191 "webutil/css/property.gperf"
+ {"scrollbar-arrow-color", Property::SCROLLBAR_ARROW_COLOR},
+#line 189 "webutil/css/property.gperf"
+ {"scrollbar-darkshadow-color", Property::SCROLLBAR_DARKSHADOW_COLOR},
+#line 185 "webutil/css/property.gperf"
+ {"scrollbar-face-color", Property::SCROLLBAR_FACE_COLOR},
+#line 188 "webutil/css/property.gperf"
+ {"scrollbar-3dlight-color", Property::SCROLLBAR_3DLIGHT_COLOR}
+ };
+
+const struct props *
+PropertyMapper::in_word_set (register const char *str, register unsigned int len)
+{
+ if (len <= MAX_WORD_LENGTH && len >= MIN_WORD_LENGTH)
+ {
+ register int key = hash (str, len);
+
+ if (key <= MAX_HASH_VALUE && key >= MIN_HASH_VALUE)
+ {
+ register const struct props *resword;
+
+ switch (key - 17)
+ {
+ case 0:
+ resword = &wordlist[0];
+ goto compare;
+ case 5:
+ resword = &wordlist[1];
+ goto compare;
+ case 6:
+ resword = &wordlist[2];
+ goto compare;
+ case 8:
+ resword = &wordlist[3];
+ goto compare;
+ case 11:
+ resword = &wordlist[4];
+ goto compare;
+ case 14:
+ resword = &wordlist[5];
+ goto compare;
+ case 16:
+ resword = &wordlist[6];
+ goto compare;
+ case 17:
+ resword = &wordlist[7];
+ goto compare;
+ case 21:
+ resword = &wordlist[8];
+ goto compare;
+ case 22:
+ resword = &wordlist[9];
+ goto compare;
+ case 24:
+ resword = &wordlist[10];
+ goto compare;
+ case 26:
+ resword = &wordlist[11];
+ goto compare;
+ case 27:
+ resword = &wordlist[12];
+ goto compare;
+ case 30:
+ resword = &wordlist[13];
+ goto compare;
+ case 31:
+ resword = &wordlist[14];
+ goto compare;
+ case 33:
+ resword = &wordlist[15];
+ goto compare;
+ case 36:
+ resword = &wordlist[16];
+ goto compare;
+ case 37:
+ resword = &wordlist[17];
+ goto compare;
+ case 39:
+ resword = &wordlist[18];
+ goto compare;
+ case 41:
+ resword = &wordlist[19];
+ goto compare;
+ case 42:
+ resword = &wordlist[20];
+ goto compare;
+ case 44:
+ resword = &wordlist[21];
+ goto compare;
+ case 45:
+ resword = &wordlist[22];
+ goto compare;
+ case 46:
+ resword = &wordlist[23];
+ goto compare;
+ case 49:
+ resword = &wordlist[24];
+ goto compare;
+ case 56:
+ resword = &wordlist[25];
+ goto compare;
+ case 58:
+ resword = &wordlist[26];
+ goto compare;
+ case 61:
+ resword = &wordlist[27];
+ goto compare;
+ case 63:
+ resword = &wordlist[28];
+ goto compare;
+ case 65:
+ resword = &wordlist[29];
+ goto compare;
+ case 68:
+ resword = &wordlist[30];
+ goto compare;
+ case 70:
+ resword = &wordlist[31];
+ goto compare;
+ case 71:
+ resword = &wordlist[32];
+ goto compare;
+ case 72:
+ resword = &wordlist[33];
+ goto compare;
+ case 75:
+ resword = &wordlist[34];
+ goto compare;
+ case 76:
+ resword = &wordlist[35];
+ goto compare;
+ case 78:
+ resword = &wordlist[36];
+ goto compare;
+ case 79:
+ resword = &wordlist[37];
+ goto compare;
+ case 80:
+ resword = &wordlist[38];
+ goto compare;
+ case 81:
+ resword = &wordlist[39];
+ goto compare;
+ case 83:
+ resword = &wordlist[40];
+ goto compare;
+ case 84:
+ resword = &wordlist[41];
+ goto compare;
+ case 85:
+ resword = &wordlist[42];
+ goto compare;
+ case 86:
+ resword = &wordlist[43];
+ goto compare;
+ case 88:
+ resword = &wordlist[44];
+ goto compare;
+ case 89:
+ resword = &wordlist[45];
+ goto compare;
+ case 90:
+ resword = &wordlist[46];
+ goto compare;
+ case 94:
+ resword = &wordlist[47];
+ goto compare;
+ case 95:
+ resword = &wordlist[48];
+ goto compare;
+ case 99:
+ resword = &wordlist[49];
+ goto compare;
+ case 102:
+ resword = &wordlist[50];
+ goto compare;
+ case 106:
+ resword = &wordlist[51];
+ goto compare;
+ case 107:
+ resword = &wordlist[52];
+ goto compare;
+ case 108:
+ resword = &wordlist[53];
+ goto compare;
+ case 109:
+ resword = &wordlist[54];
+ goto compare;
+ case 111:
+ resword = &wordlist[55];
+ goto compare;
+ case 112:
+ resword = &wordlist[56];
+ goto compare;
+ case 113:
+ resword = &wordlist[57];
+ goto compare;
+ case 114:
+ resword = &wordlist[58];
+ goto compare;
+ case 115:
+ resword = &wordlist[59];
+ goto compare;
+ case 116:
+ resword = &wordlist[60];
+ goto compare;
+ case 117:
+ resword = &wordlist[61];
+ goto compare;
+ case 119:
+ resword = &wordlist[62];
+ goto compare;
+ case 122:
+ resword = &wordlist[63];
+ goto compare;
+ case 123:
+ resword = &wordlist[64];
+ goto compare;
+ case 125:
+ resword = &wordlist[65];
+ goto compare;
+ case 128:
+ resword = &wordlist[66];
+ goto compare;
+ case 129:
+ resword = &wordlist[67];
+ goto compare;
+ case 130:
+ resword = &wordlist[68];
+ goto compare;
+ case 131:
+ resword = &wordlist[69];
+ goto compare;
+ case 132:
+ resword = &wordlist[70];
+ goto compare;
+ case 135:
+ resword = &wordlist[71];
+ goto compare;
+ case 139:
+ resword = &wordlist[72];
+ goto compare;
+ case 141:
+ resword = &wordlist[73];
+ goto compare;
+ case 142:
+ resword = &wordlist[74];
+ goto compare;
+ case 144:
+ resword = &wordlist[75];
+ goto compare;
+ case 145:
+ resword = &wordlist[76];
+ goto compare;
+ case 146:
+ resword = &wordlist[77];
+ goto compare;
+ case 147:
+ resword = &wordlist[78];
+ goto compare;
+ case 148:
+ resword = &wordlist[79];
+ goto compare;
+ case 150:
+ resword = &wordlist[80];
+ goto compare;
+ case 154:
+ resword = &wordlist[81];
+ goto compare;
+ case 155:
+ resword = &wordlist[82];
+ goto compare;
+ case 156:
+ resword = &wordlist[83];
+ goto compare;
+ case 158:
+ resword = &wordlist[84];
+ goto compare;
+ case 159:
+ resword = &wordlist[85];
+ goto compare;
+ case 160:
+ resword = &wordlist[86];
+ goto compare;
+ case 161:
+ resword = &wordlist[87];
+ goto compare;
+ case 162:
+ resword = &wordlist[88];
+ goto compare;
+ case 163:
+ resword = &wordlist[89];
+ goto compare;
+ case 164:
+ resword = &wordlist[90];
+ goto compare;
+ case 165:
+ resword = &wordlist[91];
+ goto compare;
+ case 166:
+ resword = &wordlist[92];
+ goto compare;
+ case 168:
+ resword = &wordlist[93];
+ goto compare;
+ case 171:
+ resword = &wordlist[94];
+ goto compare;
+ case 172:
+ resword = &wordlist[95];
+ goto compare;
+ case 173:
+ resword = &wordlist[96];
+ goto compare;
+ case 174:
+ resword = &wordlist[97];
+ goto compare;
+ case 175:
+ resword = &wordlist[98];
+ goto compare;
+ case 177:
+ resword = &wordlist[99];
+ goto compare;
+ case 178:
+ resword = &wordlist[100];
+ goto compare;
+ case 179:
+ resword = &wordlist[101];
+ goto compare;
+ case 182:
+ resword = &wordlist[102];
+ goto compare;
+ case 185:
+ resword = &wordlist[103];
+ goto compare;
+ case 189:
+ resword = &wordlist[104];
+ goto compare;
+ case 190:
+ resword = &wordlist[105];
+ goto compare;
+ case 191:
+ resword = &wordlist[106];
+ goto compare;
+ case 192:
+ resword = &wordlist[107];
+ goto compare;
+ case 194:
+ resword = &wordlist[108];
+ goto compare;
+ case 195:
+ resword = &wordlist[109];
+ goto compare;
+ case 197:
+ resword = &wordlist[110];
+ goto compare;
+ case 198:
+ resword = &wordlist[111];
+ goto compare;
+ case 199:
+ resword = &wordlist[112];
+ goto compare;
+ case 202:
+ resword = &wordlist[113];
+ goto compare;
+ case 203:
+ resword = &wordlist[114];
+ goto compare;
+ case 204:
+ resword = &wordlist[115];
+ goto compare;
+ case 206:
+ resword = &wordlist[116];
+ goto compare;
+ case 207:
+ resword = &wordlist[117];
+ goto compare;
+ case 208:
+ resword = &wordlist[118];
+ goto compare;
+ case 211:
+ resword = &wordlist[119];
+ goto compare;
+ case 213:
+ resword = &wordlist[120];
+ goto compare;
+ case 214:
+ resword = &wordlist[121];
+ goto compare;
+ case 217:
+ resword = &wordlist[122];
+ goto compare;
+ case 218:
+ resword = &wordlist[123];
+ goto compare;
+ case 219:
+ resword = &wordlist[124];
+ goto compare;
+ case 220:
+ resword = &wordlist[125];
+ goto compare;
+ case 223:
+ resword = &wordlist[126];
+ goto compare;
+ case 228:
+ resword = &wordlist[127];
+ goto compare;
+ case 229:
+ resword = &wordlist[128];
+ goto compare;
+ case 233:
+ resword = &wordlist[129];
+ goto compare;
+ case 245:
+ resword = &wordlist[130];
+ goto compare;
+ case 248:
+ resword = &wordlist[131];
+ goto compare;
+ case 249:
+ resword = &wordlist[132];
+ goto compare;
+ case 254:
+ resword = &wordlist[133];
+ goto compare;
+ case 256:
+ resword = &wordlist[134];
+ goto compare;
+ case 259:
+ resword = &wordlist[135];
+ goto compare;
+ case 260:
+ resword = &wordlist[136];
+ goto compare;
+ case 262:
+ resword = &wordlist[137];
+ goto compare;
+ case 265:
+ resword = &wordlist[138];
+ goto compare;
+ case 267:
+ resword = &wordlist[139];
+ goto compare;
+ case 268:
+ resword = &wordlist[140];
+ goto compare;
+ case 271:
+ resword = &wordlist[141];
+ goto compare;
+ case 273:
+ resword = &wordlist[142];
+ goto compare;
+ case 276:
+ resword = &wordlist[143];
+ goto compare;
+ case 277:
+ resword = &wordlist[144];
+ goto compare;
+ case 278:
+ resword = &wordlist[145];
+ goto compare;
+ case 280:
+ resword = &wordlist[146];
+ goto compare;
+ case 281:
+ resword = &wordlist[147];
+ goto compare;
+ case 282:
+ resword = &wordlist[148];
+ goto compare;
+ case 289:
+ resword = &wordlist[149];
+ goto compare;
+ case 292:
+ resword = &wordlist[150];
+ goto compare;
+ case 294:
+ resword = &wordlist[151];
+ goto compare;
+ case 298:
+ resword = &wordlist[152];
+ goto compare;
+ case 299:
+ resword = &wordlist[153];
+ goto compare;
+ case 304:
+ resword = &wordlist[154];
+ goto compare;
+ case 320:
+ resword = &wordlist[155];
+ goto compare;
+ case 322:
+ resword = &wordlist[156];
+ goto compare;
+ case 323:
+ resword = &wordlist[157];
+ goto compare;
+ case 329:
+ resword = &wordlist[158];
+ goto compare;
+ case 330:
+ resword = &wordlist[159];
+ goto compare;
+ case 339:
+ resword = &wordlist[160];
+ goto compare;
+ case 343:
+ resword = &wordlist[161];
+ goto compare;
+ case 349:
+ resword = &wordlist[162];
+ goto compare;
+ case 358:
+ resword = &wordlist[163];
+ goto compare;
+ case 367:
+ resword = &wordlist[164];
+ goto compare;
+ case 373:
+ resword = &wordlist[165];
+ goto compare;
+ case 387:
+ resword = &wordlist[166];
+ goto compare;
+ case 396:
+ resword = &wordlist[167];
+ goto compare;
+ case 397:
+ resword = &wordlist[168];
+ goto compare;
+ case 464:
+ resword = &wordlist[169];
+ goto compare;
+ case 465:
+ resword = &wordlist[170];
+ goto compare;
+ case 478:
+ resword = &wordlist[171];
+ goto compare;
+ case 488:
+ resword = &wordlist[172];
+ goto compare;
+ case 490:
+ resword = &wordlist[173];
+ goto compare;
+ case 509:
+ resword = &wordlist[174];
+ goto compare;
+ case 514:
+ resword = &wordlist[175];
+ goto compare;
+ case 543:
+ resword = &wordlist[176];
+ goto compare;
+ case 546:
+ resword = &wordlist[177];
+ goto compare;
+ }
+ return 0;
+ compare:
+ {
+ register const char *s = resword->name;
+
+ if ((((unsigned char)*str ^ (unsigned char)*s) & ~32) == 0 && !gperf_case_strncmp (str, s, len) && s[len] == '\0')
+ return resword;
+ }
+ }
+ }
+ return 0;
+}
+#line 194 "webutil/css/property.gperf"
+
+
+//
+// Constructor.
+//
+
+Property::Property(UnicodeText s)
+ : prop_(PropFromText(s.utf8_data(), s.utf8_length()))
+{
+ if (prop_ == OTHER)
+ other_ = LowercaseAscii(s);
+}
+
+//
+// Static methods mapping Prop's to strings
+//
+
+Property::Prop Property::PropFromText(const char* str, int len) {
+ const props* a = PropertyMapper::in_word_set(str, len);
+ if (a)
+ return a->id;
+ else
+ return OTHER;
+}
+
+static const char* name_lookup[TOTAL_KEYWORDS];
+
+static void InitializeNameLookupTable() {
+ for (int i = 0; i < TOTAL_KEYWORDS; ++i)
+ name_lookup[wordlist[i].id] = wordlist[i].name;
+}
+
+const char* Property::TextFromProp(Prop p) {
+ if (p == OTHER) {
+ return "OTHER";
+ } else {
+ DCHECK_LT(p, OTHER);
+ return name_lookup[p];
+ }
+}
+
+} // namespace
+
+REGISTER_MODULE_INITIALIZER(property, {
+ Css::InitializeNameLookupTable();
+});
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/property.gperf b/trunk/src/third_party/css_parser/src/webutil/css/property.gperf
new file mode 100644
index 0000000..5311ae8
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/property.gperf
@@ -0,0 +1,255 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+%{
+#include "webutil/css/property.h"
+
+#include "base/googleinit.h"
+#include "base/logging.h"
+#include "webutil/css/string_util.h"
+
+namespace Css {
+%}
+// This is the set of properties supported by WebKit.
+struct props {
+ const char *name;
+ Property::Prop id;
+};
+%%
+-webkit-appearance, Property::_WEBKIT_APPEARANCE
+background-attachment, Property::BACKGROUND_ATTACHMENT
+-webkit-background-clip, Property::_WEBKIT_BACKGROUND_CLIP
+background-color, Property::BACKGROUND_COLOR
+-webkit-background-composite, Property::_WEBKIT_BACKGROUND_COMPOSITE
+background-image, Property::BACKGROUND_IMAGE
+-webkit-background-origin, Property::_WEBKIT_BACKGROUND_ORIGIN
+background-position, Property::BACKGROUND_POSITION
+background-position-x, Property::BACKGROUND_POSITION_X
+background-position-y, Property::BACKGROUND_POSITION_Y
+background-repeat, Property::BACKGROUND_REPEAT
+-webkit-background-size, Property::_WEBKIT_BACKGROUND_SIZE
+-webkit-binding, Property::_WEBKIT_BINDING
+border-collapse, Property::BORDER_COLLAPSE
+-webkit-border-image, Property::_WEBKIT_BORDER_IMAGE
+border-spacing, Property::BORDER_SPACING
+-webkit-border-horizontal-spacing, Property::_WEBKIT_BORDER_HORIZONTAL_SPACING
+-webkit-border-vertical-spacing, Property::_WEBKIT_BORDER_VERTICAL_SPACING
+-webkit-border-radius, Property::_WEBKIT_BORDER_RADIUS
+-webkit-border-top-left-radius, Property::_WEBKIT_BORDER_TOP_LEFT_RADIUS
+-webkit-border-top-right-radius, Property::_WEBKIT_BORDER_TOP_RIGHT_RADIUS
+-webkit-border-bottom-left-radius, Property::_WEBKIT_BORDER_BOTTOM_LEFT_RADIUS
+-webkit-border-bottom-right-radius, Property::_WEBKIT_BORDER_BOTTOM_RIGHT_RADIUS
+border-top-color, Property::BORDER_TOP_COLOR
+border-right-color, Property::BORDER_RIGHT_COLOR
+border-bottom-color, Property::BORDER_BOTTOM_COLOR
+border-left-color, Property::BORDER_LEFT_COLOR
+border-top-style, Property::BORDER_TOP_STYLE
+border-right-style, Property::BORDER_RIGHT_STYLE
+border-bottom-style, Property::BORDER_BOTTOM_STYLE
+border-left-style, Property::BORDER_LEFT_STYLE
+border-top-width, Property::BORDER_TOP_WIDTH
+border-right-width, Property::BORDER_RIGHT_WIDTH
+border-bottom-width, Property::BORDER_BOTTOM_WIDTH
+border-left-width, Property::BORDER_LEFT_WIDTH
+bottom, Property::BOTTOM
+-webkit-box-align, Property::_WEBKIT_BOX_ALIGN
+-webkit-box-direction, Property::_WEBKIT_BOX_DIRECTION
+-webkit-box-flex, Property::_WEBKIT_BOX_FLEX
+-webkit-box-flex-group, Property::_WEBKIT_BOX_FLEX_GROUP
+-webkit-box-lines, Property::_WEBKIT_BOX_LINES
+-webkit-box-ordinal-group, Property::_WEBKIT_BOX_ORDINAL_GROUP
+-webkit-box-orient, Property::_WEBKIT_BOX_ORIENT
+-webkit-box-pack, Property::_WEBKIT_BOX_PACK
+box-sizing, Property::BOX_SIZING
+caption-side, Property::CAPTION_SIDE
+clear, Property::CLEAR
+clip, Property::CLIP
+color, Property::COLOR
+content, Property::CONTENT
+counter-increment, Property::COUNTER_INCREMENT
+counter-reset, Property::COUNTER_RESET
+cursor, Property::CURSOR
+direction, Property::DIRECTION
+display, Property::DISPLAY
+empty-cells, Property::EMPTY_CELLS
+float, Property::FLOAT
+font-family, Property::FONT_FAMILY
+font-size, Property::FONT_SIZE
+-webkit-font-size-delta, Property::_WEBKIT_FONT_SIZE_DELTA
+font-stretch, Property::FONT_STRETCH
+font-style, Property::FONT_STYLE
+font-variant, Property::FONT_VARIANT
+font-weight, Property::FONT_WEIGHT
+height, Property::HEIGHT
+-webkit-highlight, Property::_WEBKIT_HIGHLIGHT
+left, Property::LEFT
+letter-spacing, Property::LETTER_SPACING
+-webkit-line-clamp, Property::_WEBKIT_LINE_CLAMP
+line-height, Property::LINE_HEIGHT
+list-style-image, Property::LIST_STYLE_IMAGE
+list-style-position, Property::LIST_STYLE_POSITION
+list-style-type, Property::LIST_STYLE_TYPE
+margin-top, Property::MARGIN_TOP
+margin-right, Property::MARGIN_RIGHT
+margin-bottom, Property::MARGIN_BOTTOM
+margin-left, Property::MARGIN_LEFT
+-webkit-line-break, Property::_WEBKIT_LINE_BREAK
+-webkit-margin-collapse, Property::_WEBKIT_MARGIN_COLLAPSE
+-webkit-margin-top-collapse, Property::_WEBKIT_MARGIN_TOP_COLLAPSE
+-webkit-margin-bottom-collapse, Property::_WEBKIT_MARGIN_BOTTOM_COLLAPSE
+-webkit-margin-start, Property::_WEBKIT_MARGIN_START
+-webkit-marquee, Property::_WEBKIT_MARQUEE
+-webkit-marquee-direction, Property::_WEBKIT_MARQUEE_DIRECTION
+-webkit-marquee-increment, Property::_WEBKIT_MARQUEE_INCREMENT
+-webkit-marquee-repetition, Property::_WEBKIT_MARQUEE_REPETITION
+-webkit-marquee-speed, Property::_WEBKIT_MARQUEE_SPEED
+-webkit-marquee-style, Property::_WEBKIT_MARQUEE_STYLE
+-webkit-match-nearest-mail-blockquote-color, Property::_WEBKIT_MATCH_NEAREST_MAIL_BLOCKQUOTE_COLOR
+max-height, Property::MAX_HEIGHT
+max-width, Property::MAX_WIDTH
+min-height, Property::MIN_HEIGHT
+min-width, Property::MIN_WIDTH
+-webkit-nbsp-mode, Property::_WEBKIT_NBSP_MODE
+opacity, Property::OPACITY
+orphans, Property::ORPHANS
+outline-color, Property::OUTLINE_COLOR
+outline-offset, Property::OUTLINE_OFFSET
+outline-style, Property::OUTLINE_STYLE
+outline-width, Property::OUTLINE_WIDTH
+overflow, Property::OVERFLOW
+overflow-x, Property::OVERFLOW_X
+overflow-y, Property::OVERFLOW_Y
+padding-top, Property::PADDING_TOP
+padding-right, Property::PADDING_RIGHT
+padding-bottom, Property::PADDING_BOTTOM
+padding-left, Property::PADDING_LEFT
+-webkit-padding-start, Property::_WEBKIT_PADDING_START
+page, Property::PAGE
+page-break-after, Property::PAGE_BREAK_AFTER
+page-break-before, Property::PAGE_BREAK_BEFORE
+page-break-inside, Property::PAGE_BREAK_INSIDE
+position, Property::POSITION
+quotes, Property::QUOTES
+right, Property::RIGHT
+size, Property::SIZE
+table-layout, Property::TABLE_LAYOUT
+text-align, Property::TEXT_ALIGN
+text-decoration, Property::TEXT_DECORATION
+text-indent, Property::TEXT_INDENT
+text-line-through, Property::TEXT_LINE_THROUGH
+text-line-through-color, Property::TEXT_LINE_THROUGH_COLOR
+text-line-through-mode, Property::TEXT_LINE_THROUGH_MODE
+text-line-through-style, Property::TEXT_LINE_THROUGH_STYLE
+text-line-through-width, Property::TEXT_LINE_THROUGH_WIDTH
+text-overflow, Property::TEXT_OVERFLOW
+text-overline, Property::TEXT_OVERLINE
+text-overline-color, Property::TEXT_OVERLINE_COLOR
+text-overline-mode, Property::TEXT_OVERLINE_MODE
+text-overline-style, Property::TEXT_OVERLINE_STYLE
+text-overline-width, Property::TEXT_OVERLINE_WIDTH
+-webkit-text-security, Property::_WEBKIT_TEXT_SECURITY
+text-shadow, Property::TEXT_SHADOW
+text-transform, Property::TEXT_TRANSFORM
+text-underline, Property::TEXT_UNDERLINE
+text-underline-color, Property::TEXT_UNDERLINE_COLOR
+text-underline-mode, Property::TEXT_UNDERLINE_MODE
+text-underline-style, Property::TEXT_UNDERLINE_STYLE
+text-underline-width, Property::TEXT_UNDERLINE_WIDTH
+resize, Property::RESIZE
+-webkit-text-size-adjust, Property::_WEBKIT_TEXT_SIZE_ADJUST
+-webkit-dashboard-region, Property::_WEBKIT_DASHBOARD_REGION
+top, Property::TOP
+unicode-bidi, Property::UNICODE_BIDI
+-webkit-user-drag, Property::_WEBKIT_USER_DRAG
+-webkit-user-modify, Property::_WEBKIT_USER_MODIFY
+-webkit-user-select, Property::_WEBKIT_USER_SELECT
+vertical-align, Property::VERTICAL_ALIGN
+visibility, Property::VISIBILITY
+white-space, Property::WHITE_SPACE
+widows, Property::WIDOWS
+width, Property::WIDTH
+word-wrap, Property::WORD_WRAP
+word-spacing, Property::WORD_SPACING
+z-index, Property::Z_INDEX
+background, Property::BACKGROUND
+border, Property::BORDER
+border-color, Property::BORDER_COLOR
+border-style, Property::BORDER_STYLE
+border-top, Property::BORDER_TOP
+border-right, Property::BORDER_RIGHT
+border-bottom, Property::BORDER_BOTTOM
+border-left, Property::BORDER_LEFT
+border-width, Property::BORDER_WIDTH
+font, Property::FONT
+list-style, Property::LIST_STYLE
+margin, Property::MARGIN
+outline, Property::OUTLINE
+padding, Property::PADDING
+scrollbar-face-color, Property::SCROLLBAR_FACE_COLOR
+scrollbar-shadow-color, Property::SCROLLBAR_SHADOW_COLOR
+scrollbar-highlight-color, Property::SCROLLBAR_HIGHLIGHT_COLOR
+scrollbar-3dlight-color, Property::SCROLLBAR_3DLIGHT_COLOR
+scrollbar-darkshadow-color, Property::SCROLLBAR_DARKSHADOW_COLOR
+scrollbar-track-color, Property::SCROLLBAR_TRACK_COLOR
+scrollbar-arrow-color, Property::SCROLLBAR_ARROW_COLOR
+-webkit-text-decorations-in-effect, Property::_WEBKIT_TEXT_DECORATIONS_IN_EFFECT
+-webkit-rtl-ordering, Property::_WEBKIT_RTL_ORDERING
+%%
+
+//
+// Constructor.
+//
+
+Property::Property(UnicodeText s)
+ : prop_(PropFromText(s.utf8_data(), s.utf8_length()))
+{
+ if (prop_ == OTHER)
+ other_ = LowercaseAscii(s);
+}
+
+//
+// Static methods mapping Prop's to strings
+//
+
+Property::Prop Property::PropFromText(const char* str, int len) {
+ const props* a = PropertyMapper::in_word_set(str, len);
+ if (a)
+ return a->id;
+ else
+ return OTHER;
+}
+
+static const char* name_lookup[TOTAL_KEYWORDS];
+
+static void InitializeNameLookupTable() {
+ for (int i = 0; i < TOTAL_KEYWORDS; ++i)
+ name_lookup[wordlist[i].id] = wordlist[i].name;
+}
+
+const char* Property::TextFromProp(Prop p) {
+ if (p == OTHER) {
+ return "OTHER";
+ } else {
+ DCHECK_LT(p, OTHER);
+ return name_lookup[p];
+ }
+}
+
+} // namespace
+
+REGISTER_MODULE_INITIALIZER(property, {
+ Css::InitializeNameLookupTable();
+});
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/property.h b/trunk/src/third_party/css_parser/src/webutil/css/property.h
new file mode 100644
index 0000000..43af536
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/property.h
@@ -0,0 +1,140 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2006 Google Inc. All Rights Reserved.
+// Author: dpeng@google.com (Daniel Peng)
+//
+// Property represents the name of a CSS property (e.g.,
+// background, font-face, font-size). If we recognize the property,
+// we store it as an enum. Otherwise, we store the text for the name
+// of the property.
+
+
+#ifndef WEBUTIL_CSS_PROPERTY_H__
+#define WEBUTIL_CSS_PROPERTY_H__
+
+#include <cmath>
+#include <string>
+
+#include "util/utf8/public/unicodetext.h"
+#include "webutil/css/string.h"
+
+// resolving conflict on macro OVERFLOW defined in math.h.
+#ifdef OVERFLOW
+const int OVERFLOW_temporary = OVERFLOW;
+#undef OVERFLOW
+const int OVERFLOW = OVERFLOW_temporary;
+#endif
+
+namespace Css {
+
+class Property {
+ public:
+ enum Prop {
+ _WEBKIT_APPEARANCE, BACKGROUND_ATTACHMENT,
+ _WEBKIT_BACKGROUND_CLIP, BACKGROUND_COLOR,
+ _WEBKIT_BACKGROUND_COMPOSITE, BACKGROUND_IMAGE,
+ _WEBKIT_BACKGROUND_ORIGIN, BACKGROUND_POSITION, BACKGROUND_POSITION_X,
+ BACKGROUND_POSITION_Y, BACKGROUND_REPEAT, _WEBKIT_BACKGROUND_SIZE,
+ _WEBKIT_BINDING, BORDER_COLLAPSE, _WEBKIT_BORDER_IMAGE,
+ BORDER_SPACING, _WEBKIT_BORDER_HORIZONTAL_SPACING,
+ _WEBKIT_BORDER_VERTICAL_SPACING, _WEBKIT_BORDER_RADIUS,
+ _WEBKIT_BORDER_TOP_LEFT_RADIUS, _WEBKIT_BORDER_TOP_RIGHT_RADIUS,
+ _WEBKIT_BORDER_BOTTOM_LEFT_RADIUS, _WEBKIT_BORDER_BOTTOM_RIGHT_RADIUS,
+ BORDER_TOP_COLOR, BORDER_RIGHT_COLOR, BORDER_BOTTOM_COLOR,
+ BORDER_LEFT_COLOR, BORDER_TOP_STYLE, BORDER_RIGHT_STYLE,
+ BORDER_BOTTOM_STYLE, BORDER_LEFT_STYLE, BORDER_TOP_WIDTH,
+ BORDER_RIGHT_WIDTH, BORDER_BOTTOM_WIDTH, BORDER_LEFT_WIDTH, BOTTOM,
+ _WEBKIT_BOX_ALIGN, _WEBKIT_BOX_DIRECTION, _WEBKIT_BOX_FLEX,
+ _WEBKIT_BOX_FLEX_GROUP, _WEBKIT_BOX_LINES, _WEBKIT_BOX_ORDINAL_GROUP,
+ _WEBKIT_BOX_ORIENT, _WEBKIT_BOX_PACK, BOX_SIZING, CAPTION_SIDE, CLEAR,
+ CLIP, COLOR, CONTENT, COUNTER_INCREMENT, COUNTER_RESET, CURSOR,
+ DIRECTION, DISPLAY, EMPTY_CELLS, FLOAT, FONT_FAMILY, FONT_SIZE,
+ _WEBKIT_FONT_SIZE_DELTA, FONT_STRETCH, FONT_STYLE, FONT_VARIANT,
+ FONT_WEIGHT, HEIGHT, _WEBKIT_HIGHLIGHT, LEFT, LETTER_SPACING,
+ _WEBKIT_LINE_CLAMP, LINE_HEIGHT, LIST_STYLE_IMAGE,
+ LIST_STYLE_POSITION, LIST_STYLE_TYPE, MARGIN_TOP, MARGIN_RIGHT,
+ MARGIN_BOTTOM, MARGIN_LEFT, _WEBKIT_LINE_BREAK,
+ _WEBKIT_MARGIN_COLLAPSE, _WEBKIT_MARGIN_TOP_COLLAPSE,
+ _WEBKIT_MARGIN_BOTTOM_COLLAPSE, _WEBKIT_MARGIN_START, _WEBKIT_MARQUEE,
+ _WEBKIT_MARQUEE_DIRECTION, _WEBKIT_MARQUEE_INCREMENT,
+ _WEBKIT_MARQUEE_REPETITION, _WEBKIT_MARQUEE_SPEED,
+ _WEBKIT_MARQUEE_STYLE, _WEBKIT_MATCH_NEAREST_MAIL_BLOCKQUOTE_COLOR,
+ MAX_HEIGHT, MAX_WIDTH, MIN_HEIGHT, MIN_WIDTH, _WEBKIT_NBSP_MODE,
+ OPACITY, ORPHANS, OUTLINE_COLOR, OUTLINE_OFFSET, OUTLINE_STYLE,
+ OUTLINE_WIDTH, OVERFLOW, OVERFLOW_X, OVERFLOW_Y, PADDING_TOP,
+ PADDING_RIGHT, PADDING_BOTTOM, PADDING_LEFT, _WEBKIT_PADDING_START,
+ PAGE, PAGE_BREAK_AFTER, PAGE_BREAK_BEFORE, PAGE_BREAK_INSIDE,
+ POSITION, QUOTES, RIGHT, SIZE, TABLE_LAYOUT, TEXT_ALIGN,
+ TEXT_DECORATION, TEXT_INDENT, TEXT_LINE_THROUGH,
+ TEXT_LINE_THROUGH_COLOR, TEXT_LINE_THROUGH_MODE,
+ TEXT_LINE_THROUGH_STYLE, TEXT_LINE_THROUGH_WIDTH, TEXT_OVERFLOW,
+ TEXT_OVERLINE, TEXT_OVERLINE_COLOR, TEXT_OVERLINE_MODE,
+ TEXT_OVERLINE_STYLE, TEXT_OVERLINE_WIDTH, _WEBKIT_TEXT_SECURITY,
+ TEXT_SHADOW, TEXT_TRANSFORM, TEXT_UNDERLINE, TEXT_UNDERLINE_COLOR,
+ TEXT_UNDERLINE_MODE, TEXT_UNDERLINE_STYLE, TEXT_UNDERLINE_WIDTH,
+ RESIZE, _WEBKIT_TEXT_SIZE_ADJUST, _WEBKIT_DASHBOARD_REGION, TOP,
+ UNICODE_BIDI, _WEBKIT_USER_DRAG, _WEBKIT_USER_MODIFY,
+ _WEBKIT_USER_SELECT, VERTICAL_ALIGN, VISIBILITY, WHITE_SPACE, WIDOWS,
+ WIDTH, WORD_WRAP, WORD_SPACING, Z_INDEX, BACKGROUND, BORDER,
+ BORDER_COLOR, BORDER_STYLE, BORDER_TOP, BORDER_RIGHT, BORDER_BOTTOM,
+ BORDER_LEFT, BORDER_WIDTH, FONT, LIST_STYLE, MARGIN, OUTLINE, PADDING,
+ SCROLLBAR_FACE_COLOR, SCROLLBAR_SHADOW_COLOR,
+ SCROLLBAR_HIGHLIGHT_COLOR, SCROLLBAR_3DLIGHT_COLOR,
+ SCROLLBAR_DARKSHADOW_COLOR, SCROLLBAR_TRACK_COLOR,
+ SCROLLBAR_ARROW_COLOR, _WEBKIT_TEXT_DECORATIONS_IN_EFFECT,
+ _WEBKIT_RTL_ORDERING, OTHER
+ };
+
+ // Constructor.
+ explicit Property(UnicodeText s);
+ Property(Prop prop) : prop_(prop) { }
+
+ // Accessors.
+ //
+ // prop() returns the property enum -- OTHER if unrecognized.
+ Prop prop() const { return prop_; }
+
+ // prop_text() returns the property as a string.
+ string prop_text() const {
+ if (prop_ == OTHER)
+ return string(other_.utf8_data(), other_.utf8_length());
+ else
+ return TextFromProp(prop_);
+ }
+
+ // Static methods mapping between Prop and strings:
+ //
+ // Given the text of a CSS property, PropFromText returns the
+ // corresponding enum. If no such property is found, UnitFromText
+ // returns OTHER. Since all CSS properties are ASCII, we are happy
+ // with ASCII, UTF8, Latin-1, etc.
+ static Prop PropFromText(const char* s, int len);
+ // Given a Prop, returns its string representation. If u is
+ // NO_UNIT, returns "". If u is OTHER, we return "OTHER", but this
+ // may not be what you want.
+ static const char* TextFromProp(Prop p);
+
+ private:
+ Prop prop_;
+ UnicodeText other_; // valid if prop_ is OTHER.
+
+ Property(); // sorry, not default-constructible.
+};
+
+} // namespace
+
+#endif // WEBUTIL_CSS_PROPERTY_H__
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/property_test.cc b/trunk/src/third_party/css_parser/src/webutil/css/property_test.cc
new file mode 100644
index 0000000..82b5287
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/property_test.cc
@@ -0,0 +1,49 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2007 Google Inc. All Rights Reserved.
+// Author: dpeng@google.com (Daniel Peng)
+
+#include "webutil/css/property.h"
+
+#include "testing/base/public/googletest.h"
+#include "testing/base/public/gunit.h"
+
+namespace {
+
+class PropertyTest : public testing::Test {
+};
+
+TEST_F(PropertyTest, PropFromText) {
+ string s("border-width");
+ EXPECT_EQ(Css::Property::BORDER_WIDTH,
+ Css::Property::PropFromText(s.c_str(), s.length()));
+}
+
+TEST_F(PropertyTest, TextFromProp) {
+ EXPECT_STREQ("border-width",
+ Css::Property::TextFromProp(Css::Property::BORDER_WIDTH));
+}
+
+TEST_F(PropertyTest, Inverses) {
+ for(int i = 0; i < Css::Property::OTHER; ++i) {
+ string s(Css::Property::TextFromProp(static_cast<Css::Property::Prop>(i)));
+ EXPECT_EQ(static_cast<Css::Property::Prop>(i),
+ Css::Property::PropFromText(s.c_str(), s.length()));
+ }
+}
+
+} // namespace
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/selector.cc b/trunk/src/third_party/css_parser/src/webutil/css/selector.cc
new file mode 100644
index 0000000..3428207
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/selector.cc
@@ -0,0 +1,92 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2006 Google Inc. All Rights Reserved.
+// Author: dpeng@google.com (Daniel Peng)
+
+#include "util/gtl/stl_util-inl.h"
+
+#include "webutil/css/selector.h"
+
+namespace Css {
+
+//
+// static
+//
+
+const HtmlTagIndex SimpleSelector::tagindex_;
+
+//
+// SimpleSelector factory methods
+//
+
+SimpleSelector* SimpleSelector::NewElementType(const UnicodeText& name) {
+ HtmlTagEnum tag = static_cast<HtmlTagEnum>(
+ tagindex_.FindHtmlTag(name.utf8_data(), name.utf8_length()));
+ return new SimpleSelector(tag, name);
+}
+
+SimpleSelector* SimpleSelector::NewUniversal() {
+ return new SimpleSelector(SimpleSelector::UNIVERSAL,
+ UnicodeText(), UnicodeText());
+}
+
+SimpleSelector* SimpleSelector::NewExistAttribute(
+ const UnicodeText& attribute) {
+ return new SimpleSelector(SimpleSelector::EXIST_ATTRIBUTE,
+ attribute, UnicodeText());
+}
+
+SimpleSelector* SimpleSelector::NewBinaryAttribute(
+ Type type, const UnicodeText& attribute, const UnicodeText& value) {
+ return new SimpleSelector(type, attribute, value);
+}
+
+static const char kClassText[] = "class";
+SimpleSelector* SimpleSelector::NewClass(const UnicodeText& classname) {
+ static const UnicodeText kClass =
+ UTF8ToUnicodeText(kClassText, strlen(kClassText));
+ return new SimpleSelector(SimpleSelector::CLASS,
+ kClass, classname);
+}
+
+static const char kIdText[] = "id";
+SimpleSelector* SimpleSelector::NewId(const UnicodeText& id) {
+ static const UnicodeText kId = UTF8ToUnicodeText(kIdText, strlen(kIdText));
+ return new SimpleSelector(SimpleSelector::ID,
+ kId, id);
+}
+
+SimpleSelector* SimpleSelector::NewPseudoclass(
+ const UnicodeText& pseudoclass) {
+ return new SimpleSelector(SimpleSelector::PSEUDOCLASS,
+ UnicodeText(), pseudoclass);
+}
+
+SimpleSelector* SimpleSelector::NewLang(const UnicodeText& lang) {
+ return new SimpleSelector(SimpleSelector::LANG,
+ UnicodeText(), lang);
+}
+
+//
+// Some destructors that need STLDeleteElements() from stl_util-inl.h
+//
+
+SimpleSelectors::~SimpleSelectors() { STLDeleteElements(this); }
+Selector::~Selector() { STLDeleteElements(this); }
+Selectors::~Selectors() { STLDeleteElements(this); }
+
+} // namespace
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/selector.h b/trunk/src/third_party/css_parser/src/webutil/css/selector.h
new file mode 100644
index 0000000..6d405b3
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/selector.h
@@ -0,0 +1,305 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2006 Google Inc. All Rights Reserved.
+// Author: dpeng@google.com (Daniel Peng)
+
+#ifndef WEBUTIL_CSS_SELECTOR_H__
+#define WEBUTIL_CSS_SELECTOR_H__
+
+#include <string>
+#include <vector>
+
+#include "base/logging.h"
+#include "base/scoped_ptr.h"
+#include "util/utf8/public/unicodetext.h"
+#include "webutil/css/string.h"
+#include "webutil/html/htmltagenum.h"
+#include "webutil/html/htmltagindex.h"
+
+namespace Css {
+
+// -------------
+// Overview:
+//
+// We adopt CSS3's naming conventions:
+// http://www.w3.org/TR/css3-selectors/#selector-syntax
+//
+// A selector is a chain of one or more sequences of simple selectors
+// separated by combinators. We represent this in the Selector
+// class.
+// Ex: div[align=center] > h1
+//
+// A sequence of simple selectors is a chain of simple selectors that
+// are not separated by a combinator. It always begins with a type
+// selector or a universal selector. No other type selector or
+// universal selector is allowed in the sequence. We represent this
+// in the SimpleSelectors class as just a list of simple selectors;
+// the simple selectors are logically AND-ed together.
+// Ex: div[align=center] is a sequence of two simple selectors, 'div'
+// and '[align=center]'
+//
+// A simple selector is either a type selector, universal selector,
+// attribute selector, class selector, ID selector, content selector,
+// or pseudo-class. One pseudo-element may be appended to the last
+// sequence of simple selectors. We represent this in the
+// SimpleSelector class.
+// Ex: [align=center]
+//
+// ------------
+
+// ------------
+// SimpleSelector:
+//
+// A SimpleSelector represents a simple selector. We currently
+// don't distinguish pseudo-elements (:first-line) from pseudo-classes
+// (:hover).
+//
+// For example, 'div' is a simple selector of type TYPE_SELECTOR,
+// matching HTML div tags. '[align=center]' is simple selector
+// of type EXACT_ATTRIBUTE, matching tags with attribute 'align' equal
+// to 'center'. If you put them together as div[align=center], you
+// get a chain of simple selectors matching, e.g., <div align=center
+// foo=bar>.
+//
+// There are several different types of simple selectors, so you can
+// think of a SimpleSelector as a tagged union of various values.
+// The tag is set by the factory method and accessed with type(). The
+// values are also set by the factory and accessed with the various
+// accessors. Each accessor is valid with certain types.
+// ------------
+class SimpleSelector {
+ public:
+ enum Type {
+ // An element type selector matches the HTML element type (e.g., h1, h2, h3)
+ ELEMENT_TYPE,
+
+ // This type of simple selector matches anything ('*').
+ UNIVERSAL,
+
+ // These types check HTML element attributes in various ways.
+ EXIST_ATTRIBUTE, // [attr]: element sets the "attr" attribute
+ EXACT_ATTRIBUTE, // [attr=val]: element's "attr" attribute value is "val".
+ ONE_OF_ATTRIBUTE, // [attr~=val]: element's "attr" attribute value is a
+ // space-separated list of words, one of which
+ // is exactly "val".
+ BEGIN_HYPHEN_ATTRIBUTE, // [attr|=val] element's "attr" attribute value
+ // is a hypen-separated list that begins
+ // with "val-".
+
+ // New in CSS3, but well supported.
+ BEGIN_WITH_ATTRIBUTE, // [attr^=val]: attribute value starts with "val".
+ END_WITH_ATTRIBUTE, // [attr$=val]: attribute value ends with "val".
+ SUBSTRING_ATTRIBUTE, // [attr*=val]: attribute value contains "val".
+
+ // CLASS and ID are equivalent to ONE_OF_ATTRIBUTE and
+ // EXACT_ATTRIBUTE, but CSS provides special syntax for them, and
+ // they're very common. GetLocalName() returns "class" and "id"
+ // for these:
+ CLASS, // .class: element's class attribute is "val"
+ ID, // #id: the element's id attribute is "id"
+
+ // Miscellaneous conditions:
+ PSEUDOCLASS, // a:hover matches <a href=blah> when mouse is hovering
+ LANG, // :lang(en) matches if the element is in English.
+
+ // We don't implement these (yet).
+ // AND, OR, NOT, ONLY_CHILD, ONLY_TYPE, CONTENT, POSITIONAL
+ // ROOT, TEXT, PSEUDOELEMENT, PROCESSING_INSTRUCTION,
+ // NEGATIVE, COMMENT, CDATA_SECTION,
+ };
+
+ // Factory methods to generate SimpleSelectors of various types.
+ static SimpleSelector* NewElementType(const UnicodeText& name);
+ static SimpleSelector* NewUniversal();
+ static SimpleSelector* NewExistAttribute(const UnicodeText& attribute);
+ // *_ATTRIBUTE.
+ static SimpleSelector* NewBinaryAttribute(Type type,
+ const UnicodeText& attribute,
+ const UnicodeText& value);
+ static SimpleSelector* NewClass(const UnicodeText& classname);
+ static SimpleSelector* NewId(const UnicodeText& id);
+ static SimpleSelector* NewPseudoclass(const UnicodeText& pseudoclass);
+ static SimpleSelector* NewLang(const UnicodeText& lang);
+
+ // oper is '=' for EXACT_ATTRIBUTE, or the first character of the attribute
+ // selector operator, i.e. '~', '|', etc.
+ static Type AttributeTypeFromOperator(char oper) {
+ switch (oper) {
+ case '=':
+ return EXACT_ATTRIBUTE;
+ case '~':
+ return ONE_OF_ATTRIBUTE;
+ case '|':
+ return BEGIN_HYPHEN_ATTRIBUTE;
+ case '^':
+ return BEGIN_WITH_ATTRIBUTE;
+ case '$':
+ return END_WITH_ATTRIBUTE;
+ case '*':
+ return SUBSTRING_ATTRIBUTE;
+ default:
+ LOG(FATAL) << "Invalid attribute operator " << oper;
+ }
+ }
+
+ // Accessors.
+ //
+ Type type() const { return type_; } // The type of selector
+
+ // ELEMENT_TYPE accessor
+ // element_type() returns kHtmlTagUnknown if we don't recognize the tag.
+ HtmlTagEnum element_type() const { return element_type_; }
+ // element_text() returns the element text. We preserve the original case.
+ const UnicodeText& element_text() const { return element_text_; }
+
+ // *_ATTRIBUTE, CLASS, ID accessors
+ const UnicodeText& namespace_uri(); // Not implemented.
+ const UnicodeText& attribute() const {
+ DCHECK(IsAttributeCondition());
+ return attribute_;
+ }
+ const UnicodeText& value() const {
+ DCHECK(IsAttributeCondition());
+ return value_;
+ }
+ // IsAttributeCondition indicates whether this is an attribute
+ // selector, with valid attribute() and value() fields.
+ bool IsAttributeCondition() const {
+ return (EXIST_ATTRIBUTE == type_ || EXACT_ATTRIBUTE == type_
+ || ONE_OF_ATTRIBUTE == type_ || BEGIN_HYPHEN_ATTRIBUTE == type_
+ || BEGIN_WITH_ATTRIBUTE == type_ || END_WITH_ATTRIBUTE == type_
+ || SUBSTRING_ATTRIBUTE == type_ || CLASS == type_ || ID == type_);
+ }
+
+ // PSEUDOCLASS accessor:
+ const UnicodeText& pseudoclass() const {
+ DCHECK_EQ(PSEUDOCLASS, type_);
+ return value_;
+ }
+
+ // lang accessor
+ const UnicodeText& lang() const {
+ DCHECK_EQ(LANG, type_);
+ return value_;
+ }
+
+ string ToString() const;
+ private:
+ Type type_;
+
+ HtmlTagEnum element_type_; // ELEMENT_TYPE
+ UnicodeText element_text_; // ELEMENT_TYPE
+ static const HtmlTagIndex tagindex_; // Look up HTML tags. Thread-safe.
+
+ UnicodeText attribute_; // Attribute name, valid for *_ATTRIBUTE, CLASS, ID
+ UnicodeText value_; // Valid for *_ATTRIBUTE, CLASS, ID, PSEUDOCLASS, LANG
+
+ // Private constructors, for use by factory methods
+ SimpleSelector(Type type,
+ const UnicodeText& attribute, const UnicodeText& value)
+ : type_(type), attribute_(attribute), value_(value) { }
+ SimpleSelector(HtmlTagEnum element_type, const UnicodeText& element_text)
+ : type_(ELEMENT_TYPE),
+ element_type_(element_type), element_text_(element_text) { }
+
+ DISALLOW_IMPLICIT_CONSTRUCTORS(SimpleSelector);
+};
+
+// ------------
+// SimpleSelectors:
+//
+// SimpleSelectors is a vector of SimpleSelector*, which we own and
+// will delete upon destruction. If you remove elements from
+// SimpleSelectors, you are responsible for deleting them.
+//
+// Semantically, SimpleSelectors is the AND of each of its constituent
+// simple selectors. Although SAC permits other logical connectives like OR
+// and NOT, it appears that CSS3 will not, so we favor the simplicity
+// of this list representation over the complexity of a full logical tree
+// (and its pie-in-the-sky possibilities). CSS3's :not pseudoclass
+// takes an entire simple selector as negation, not just a condition.
+//
+// SimpleSelectors also has the combinator() field, which describes
+// how it relates to the previous SimpleSelectors in the
+// Selector chain. For example, consider E > F + G. E's
+// combinator() is NONE, F's combinator is CHILD, and G's combinator
+// is SIBLING.
+// ------------
+class SimpleSelectors : public std::vector<SimpleSelector*> {
+ public:
+ enum Combinator {
+ NONE, // first one in the chain
+ DESCENDANT, // this one is a descendant of the previous one
+ CHILD, // this one is a child (direct descendant) of the previous one
+ SIBLING // this one is an adjacent sibling of the previous
+ // one, non-element nodes (such as text nodes and
+ // comments) excluded.
+ };
+
+ SimpleSelectors(Combinator c)
+ : std::vector<SimpleSelector*>(), combinator_(c) { }
+ ~SimpleSelectors();
+
+ Combinator combinator() const { return combinator_; }
+ const SimpleSelector* get(int i) const { return (*this)[i]; } // sugar.
+
+ string ToString() const;
+ private:
+ const Combinator combinator_;
+ DISALLOW_COPY_AND_ASSIGN(SimpleSelectors);
+};
+
+// ------------
+// Selector:
+//
+// A selector is a chain of sequences of simple selectors separated by
+// combinators. Each SimpleSelectors stores the combinator between
+// it and the previous one in the chain.
+// ------------
+class Selector: public std::vector<SimpleSelectors*> {
+ public:
+ Selector() { }
+ ~Selector();
+ // We provide syntactic sugar for accessing elements.
+ // conditions->get(i) looks better than (*conditions)[i])
+ const SimpleSelectors* get(int i) const { return (*this)[i]; }
+
+ string ToString() const;
+ private:
+ DISALLOW_COPY_AND_ASSIGN(Selector);
+};
+
+// ------------
+// Selectors:
+//
+// When several selectors share the same declarations, they may be
+// grouped into a comma-separated list:
+// ------------
+class Selectors: public std::vector<Selector*> {
+ public:
+ Selectors() { }
+ ~Selectors();
+ const Selector* get(int i) const { return (*this)[i]; }
+
+ string ToString() const;
+ private:
+ DISALLOW_COPY_AND_ASSIGN(Selectors);
+};
+
+} // namespace
+
+#endif // WEBUTIL_CSS_SELECTOR_H__
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/string.h b/trunk/src/third_party/css_parser/src/webutil/css/string.h
new file mode 100644
index 0000000..60f1bd8
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/string.h
@@ -0,0 +1,23 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#ifndef WEBUTIL_CSS_STRING_H_
+#define WEBUTIL_CSS_STRING_H_
+
+#include "string_using.h"
+
+#endif // WEBUTIL_CSS_STRING_H_
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/string_util.cc b/trunk/src/third_party/css_parser/src/webutil/css/string_util.cc
new file mode 100644
index 0000000..e427c32
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/string_util.cc
@@ -0,0 +1,75 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2010 Google Inc. All Rights Reserved.
+// Author: sligocki@google.com (Shawn Ligocki)
+
+#include "webutil/css/string_util.h"
+
+#include <cerrno>
+#include <cstdlib> // strtod
+#include <cstring> // memcpy
+
+#include "strings/ascii_ctype.h" // ascii_tolower
+#include "util/utf8/public/unicodetext.h"
+
+namespace Css {
+
+// Addapted from RE2::Arg::parse_double() from the Regular Expressions package
+// RE2 (http://code.google.com/p/re2/).
+bool ParseDouble(const char* str, int len, double* dest) {
+ static const int kMaxLength = 200;
+ if (dest == NULL || len == 0 || len >= kMaxLength) {
+ return false;
+ }
+ char buf[kMaxLength];
+ memcpy(buf, str, len);
+ buf[len] = '\0';
+
+ char* end;
+ errno = 0;
+ *dest = strtod(buf, &end);
+ if (errno != 0 || end != buf + len) {
+ return false;
+ }
+ return true;
+}
+
+namespace {
+
+inline bool IsAscii(char32 c) {
+ return c < 0x80 && c >= 0;
+}
+
+} // namespace
+
+UnicodeText LowercaseAscii(const UnicodeText& in_text) {
+ UnicodeText out_text;
+ // TODO(sligocki): out_text.reserve(in_text.utf8_length())
+ for (UnicodeText::const_iterator iter = in_text.begin();
+ iter < in_text.end(); ++iter) {
+ char32 c = *iter;
+ if (IsAscii(c)) {
+ out_text.push_back(ascii_tolower(c));
+ } else {
+ out_text.push_back(c);
+ }
+ }
+ return out_text;
+}
+
+
+} // namespace Css
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/string_util.h b/trunk/src/third_party/css_parser/src/webutil/css/string_util.h
new file mode 100644
index 0000000..6208426
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/string_util.h
@@ -0,0 +1,38 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2010 Google Inc. All Rights Reserved.
+// Author: sligocki@google.com (Shawn Ligocki)
+//
+// Useful string utils.
+
+#ifndef WEBUTIL_CSS_STRING_UTIL_H_
+#define WEBUTIL_CSS_STRING_UTIL_H_
+
+class UnicodeText;
+
+namespace Css {
+
+// Convert a given block of chars to a double.
+bool ParseDouble(const char* str, int len, double* dest);
+
+// Lowercase all ASCII chars in the UnicodeText in_text.
+// Leaves non-ASCII chars alone.
+UnicodeText LowercaseAscii(const UnicodeText& in_text);
+
+} // namespace Css
+
+#endif // WEBUTIL_CSS_STRING_UTIL_H_
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/tostring.cc b/trunk/src/third_party/css_parser/src/webutil/css/tostring.cc
new file mode 100644
index 0000000..c7b9c2d
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/tostring.cc
@@ -0,0 +1,273 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2007 Google Inc. All Rights Reserved.
+// Author: yian@google.com (Yi-An Huang)
+
+#include <string>
+#include <vector>
+
+#include "base/stringprintf.h"
+#include "strings/join.h"
+#include "strings/strutil.h"
+#include "webutil/css/parser.h"
+#include "webutil/css/string.h"
+
+namespace Css {
+
+// Escape [(), \t\r\n\\'"]
+// Based on CEscape/CEscapeString from strings/strutil.cc.
+static string CSSEscapeString(const StringPiece& src) {
+ const int dest_length = src.size() * 2 + 1; // Maximum possible expansion
+ scoped_array<char> dest(new char[dest_length]);
+
+ const char* src_end = src.data() + src.size();
+ int used = 0;
+
+ for (const char* p = src.data(); p < src_end; p++) {
+ switch (*p) {
+ case '\n': dest[used++] = '\\'; dest[used++] = 'n'; break;
+ case '\r': dest[used++] = '\\'; dest[used++] = 'r'; break;
+ case '\t': dest[used++] = '\\'; dest[used++] = 't'; break;
+ case '\"': case '\'': case '\\': case ',': case '(': case ')':
+ dest[used++] = '\\';
+ dest[used++] = *p;
+ break;
+ default: dest[used++] = *p; break;
+ }
+ }
+
+ return string(dest.get(), used);
+}
+
+static string CSSEscapeString(const UnicodeText& src) {
+ return CSSEscapeString(StringPiece(src.utf8_data(), src.utf8_length()));
+}
+
+template <typename Container>
+static string JoinElementStrings(const Container& c, const char* delim) {
+ std::vector<string> vals;
+ vals.reserve(c.size());
+ for (typename Container::const_iterator it = c.begin(); it != c.end(); ++it)
+ vals.push_back((*it)->ToString());
+ string result;
+ JoinStrings(vals, delim, &result);
+ return result;
+}
+
+static string JoinMediaStrings(const std::vector<UnicodeText>& media,
+ const char* delim) {
+ std::vector<string> vals;
+ vals.reserve(media.size());
+ for (std::vector<UnicodeText>::const_iterator
+ it = media.begin(); it != media.end(); ++it)
+ vals.push_back(CSSEscapeString(*it));
+ string result;
+ JoinStrings(vals, delim, &result);
+ return result;
+}
+
+static string StylesheetTypeString(Stylesheet::StylesheetType type) {
+ switch (type) {
+ CONSIDER_IN_CLASS(Stylesheet, AUTHOR);
+ CONSIDER_IN_CLASS(Stylesheet, USER);
+ CONSIDER_IN_CLASS(Stylesheet, SYSTEM);
+ default:
+ LOG(FATAL) << "Invalid type";
+ }
+}
+
+string Value::ToString() const {
+ switch (GetLexicalUnitType()) {
+ case NUMBER:
+ return StringPrintf("%g%s",
+ GetFloatValue(),
+ GetDimensionUnitText().c_str());
+ case URI:
+ return StringPrintf("url(%s)",
+ CSSEscapeString(GetStringValue()).c_str());
+ case COUNTER:
+ return StringPrintf("counter(%s)",
+ GetParameters()->ToString().c_str());
+ case FUNCTION:
+ return StringPrintf("%s(%s)",
+ CSSEscapeString(GetFunctionName()).c_str(),
+ GetParameters()->ToString().c_str());
+ case RECT:
+ return StringPrintf("rect(%s)",
+ GetParameters()->ToString().c_str());
+ case COLOR:
+ if (GetColorValue().IsDefined())
+ return GetColorValue().ToString();
+ else
+ return "bad";
+ case STRING:
+ return StringPrintf("\"%s\"",
+ CSSEscapeString(GetStringValue()).c_str());
+ case IDENT:
+ return CSSEscapeString(GetIdentifierText()).c_str();
+ case UNKNOWN:
+ return "UNKNOWN";
+ case DEFAULT:
+ return "";
+ default:
+ LOG(FATAL) << "Invalid type";
+ }
+}
+
+string Values::ToString() const {
+ return JoinElementStrings(*this, " ");
+}
+
+string SimpleSelector::ToString() const {
+ switch (type()) {
+ case ELEMENT_TYPE:
+ return UnicodeTextToUTF8(element_text());
+ case UNIVERSAL:
+ return "*";
+ case EXIST_ATTRIBUTE:
+ return StringPrintf("[%s]",
+ CSSEscapeString(attribute()).c_str());
+ case EXACT_ATTRIBUTE:
+ return StringPrintf("[%s=%s]",
+ CSSEscapeString(attribute()).c_str(),
+ CSSEscapeString(value()).c_str());
+ case ONE_OF_ATTRIBUTE:
+ return StringPrintf("[%s~=%s]",
+ CSSEscapeString(attribute()).c_str(),
+ CSSEscapeString(value()).c_str());
+ case BEGIN_HYPHEN_ATTRIBUTE:
+ return StringPrintf("[%s|=%s]",
+ CSSEscapeString(attribute()).c_str(),
+ CSSEscapeString(value()).c_str());
+ case SUBSTRING_ATTRIBUTE:
+ return StringPrintf("[%s*=%s]",
+ CSSEscapeString(attribute()).c_str(),
+ CSSEscapeString(value()).c_str());
+ case BEGIN_WITH_ATTRIBUTE:
+ return StringPrintf("[%s^=%s]",
+ CSSEscapeString(attribute()).c_str(),
+ CSSEscapeString(value()).c_str());
+ case END_WITH_ATTRIBUTE:
+ return StringPrintf("[%s$=%s]",
+ CSSEscapeString(attribute()).c_str(),
+ CSSEscapeString(value()).c_str());
+ case CLASS:
+ return StringPrintf(".%s",
+ CSSEscapeString(value()).c_str());
+ case ID:
+ return StringPrintf("#%s",
+ CSSEscapeString(value()).c_str());
+ case PSEUDOCLASS:
+ return StringPrintf(":%s",
+ CSSEscapeString(pseudoclass()).c_str());
+ case LANG:
+ return StringPrintf(":lang(%s)",
+ CSSEscapeString(lang()).c_str());
+ default:
+ LOG(FATAL) << "Invalid type";
+ }
+}
+
+string SimpleSelectors::ToString() const {
+ string prefix;
+ switch (combinator()) {
+ case CHILD:
+ prefix = "> ";
+ break;
+ case SIBLING:
+ prefix = "+ ";
+ break;
+ default:
+ break;
+ }
+ return prefix + JoinElementStrings(*this, "");
+}
+
+string Selector::ToString() const {
+ return JoinElementStrings(*this, " ");
+}
+
+string Selectors::ToString() const {
+ return JoinElementStrings(*this, ", ");
+}
+
+string Declaration::ToString() const {
+ string result = prop_text() + ": ";
+ switch (prop()) {
+ case Property::FONT_FAMILY:
+ result += JoinElementStrings(*values(), ",");
+ break;
+ case Property::FONT:
+ if (values()->size() == 1) {
+ // Special one-value font: notations like "font: menu".
+ result += values()->ToString();
+ } else if (values()->size() < 5) {
+ result += "bad";
+ } else {
+ string tmp;
+ tmp = values()->get(0)->ToString();
+ if (tmp != "normal") result += tmp + " ";
+ tmp = values()->get(1)->ToString();
+ if (tmp != "normal") result += tmp + " ";
+ tmp = values()->get(2)->ToString();
+ if (tmp != "normal") result += tmp + " ";
+ result += values()->get(3)->ToString();
+ tmp = values()->get(4)->ToString();
+ if (tmp != "normal") result += "/" + tmp;
+ for (int i = 5, n = values()->size(); i < n; ++i)
+ result += (i == 5 ? " " : ",") + values()->get(i)->ToString();
+ }
+ break;
+ default:
+ result += values()->ToString();
+ break;
+ }
+ if (IsImportant())
+ result += " !important";
+ return result;
+}
+
+string Declarations::ToString() const {
+ return JoinElementStrings(*this, "; ");
+}
+
+string Ruleset::ToString() const {
+ string result;
+ if (!media().empty())
+ result += StringPrintf("@media %s { ",
+ JoinMediaStrings(media(), ",").c_str());
+ result += selectors().ToString() + " {" + declarations().ToString() + "}";
+ if (!media().empty())
+ result += " }";
+ return result;
+}
+
+string Import::ToString() const {
+ return StringPrintf("@import url(\"%s\") %s;",
+ CSSEscapeString(link).c_str(),
+ JoinMediaStrings(media, ",").c_str());
+}
+
+string Stylesheet::ToString() const {
+ string result;
+ result += "/* " + StylesheetTypeString(type()) + " */\n";
+ result += JoinElementStrings(imports(), "\n") + "\n";
+ result += JoinElementStrings(rulesets(), "\n") + "\n";
+ return result;
+}
+
+} // namespace
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/tostring_test.cc b/trunk/src/third_party/css_parser/src/webutil/css/tostring_test.cc
new file mode 100644
index 0000000..bb8ae18
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/tostring_test.cc
@@ -0,0 +1,95 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2007 Google Inc. All Rights Reserved.
+// Author: yian@google.com (Yi-An Huang)
+
+#include <string>
+
+#include "base/scoped_ptr.h"
+#include "base/stringprintf.h"
+#include "testing/base/public/googletest.h"
+#include "testing/base/public/gunit.h"
+#include "webutil/css/parser.h"
+#include "webutil/css/string.h"
+
+namespace {
+
+#define TESTDECLARATIONS(css) TestDeclarations(css, __LINE__)
+#define TESTSTYLESHEET(css) TestStylesheet(css, __LINE__)
+
+class ToStringTest : public testing::Test {
+ protected:
+ // Checks if ParseDeclarations()->ToString() returns identity.
+ void TestDeclarations(const string& css, int line) {
+ SCOPED_TRACE(StringPrintf("at line %d", line));
+ Css::Parser parser(css);
+ scoped_ptr<Css::Declarations> decls(parser.ParseDeclarations());
+ EXPECT_EQ(css, decls->ToString());
+ }
+
+ // Checks if ParseStylesheet()->ToString() returns identity.
+ void TestStylesheet(const string& css, int line) {
+ SCOPED_TRACE(StringPrintf("at line %d", line));
+ Css::Parser parser(css);
+ scoped_ptr<Css::Stylesheet> stylesheet(parser.ParseStylesheet());
+ EXPECT_EQ(css, stylesheet->ToString());
+ }
+};
+
+TEST_F(ToStringTest, declarations) {
+ TESTDECLARATIONS("left: inherit; "
+ "color: #abcdef; "
+ "content: \"text\"; "
+ "top: 1; right: 2px !important; "
+ "background-image: url(link\\(a\\,b\\,\\\"c\\\"\\).html)");
+ TESTDECLARATIONS("content: counter(); clip: rect(auto 1px 2em auto)");
+ TESTDECLARATIONS("font-family: arial,serif,\"Courier New\"");
+
+ // FONT is special
+ Css::Parser parser("font: 3px/1.1 Arial");
+ scoped_ptr<Css::Declarations> decls(parser.ParseDeclarations());
+ EXPECT_EQ("font: 3px/1.1 arial; "
+ "font-style: normal; font-variant: normal; font-weight: normal; "
+ "font-size: 3px; line-height: 1.1; font-family: arial",
+ decls->ToString());
+}
+
+TEST_F(ToStringTest, selectors) {
+ TESTSTYLESHEET("/* AUTHOR */\n\n"
+ "a, *, b#id, c.class, :hover:focus {top: 1}\n");
+ TESTSTYLESHEET("/* AUTHOR */\n\n"
+ "table[width], [disable=no], [x~=y], [lang|=fr] {top: 1}\n");
+ TESTSTYLESHEET("/* AUTHOR */\n\n"
+ "a > b, a + b, a b + c > d > e f {top: 1}\n");
+}
+
+TEST_F(ToStringTest, misc) {
+ TESTSTYLESHEET("/* AUTHOR */\n"
+ "@import url(\"a.html\") ;\n"
+ "@import url(\"b.html\") print;\n"
+ "@media print,screen { a {top: 1} }\n"
+ "b {color: #ff0000}\n");
+
+ Css::Parser parser("a {top: 1}");
+ scoped_ptr<Css::Stylesheet> stylesheet(parser.ParseStylesheet());
+ stylesheet->set_type(Css::Stylesheet::SYSTEM);
+ EXPECT_EQ("/* SYSTEM */\n\n"
+ "a {top: 1}\n",
+ stylesheet->ToString());
+}
+
+} // namespace
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/util.cc b/trunk/src/third_party/css_parser/src/webutil/css/util.cc
new file mode 100644
index 0000000..b8c7575
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/util.cc
@@ -0,0 +1,309 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2007 Google Inc. All Rights Reserved.
+// Author: dpeng@google.com (Daniel Peng)
+
+#include "webutil/css/util.h"
+
+#include "base/commandlineflags.h"
+#include "strings/memutil.h"
+#include "strings/stringpiece_utils.h"
+#include "strings/strutil.h"
+#include "util/utf8/public/unicodetext.h"
+#include "webutil/css/parser.h"
+
+DEFINE_double(font_size_adjustment,
+ 0.58,
+ "Ratio of x-height to font-size in CSS terms");
+
+namespace Css {
+namespace Util {
+
+HtmlColor GetCssColor(const Css::Value* val,
+ const HtmlColor& def,
+ COLOR_ATTR* attr) {
+ if (val) {
+ switch (val->GetLexicalUnitType()) {
+ case Value::COLOR:
+ if (val->GetColorValue().IsDefined()) {
+ if (attr) *attr = ORIGINAL;
+ return val->GetColorValue();
+ }
+ break;
+ case Value::UNKNOWN:
+ if (attr) *attr = UNKNOWN;
+ return def;
+ case Value::IDENT:
+ switch (val->GetIdentifier().ident()) {
+ case Identifier::INHERIT:
+ if (attr) *attr = INHERIT;
+ return def;
+ case Identifier::TRANSPARENT:
+ if (attr) *attr = TRANSPARENT;
+ return def;
+ default:
+ break;
+ }
+ break;
+ default:
+ break;
+ }
+ }
+ if (attr) *attr = TRANSPARENT;
+ return def;
+}
+
+bool GetCssLength(const Css::Value* val, double parent_size,
+ double font_size, double unit,
+ bool can_negative, bool can_unitless,
+ double* size) {
+ if (val == NULL || val->GetLexicalUnitType() != Css::Value::NUMBER ||
+ (!can_negative && val->GetFloatValue() < 0))
+ return false;
+
+ switch (val->GetDimension()) {
+ case Css::Value::PERCENT:
+ if (parent_size != -1) {
+ *size = val->GetFloatValue() * parent_size / 100.0;
+ return true;
+ }
+ break;
+ case Css::Value::PX:
+ *size = val->GetFloatValue();
+ return true;
+ case Css::Value::EM:
+ *size = val->GetFloatValue() * font_size;
+ return true;
+ case Css::Value::EX:
+ *size = val->GetFloatValue() * font_size * FLAGS_font_size_adjustment;
+ return true;
+ case Css::Value::MM:
+ *size = val->GetFloatValue() / 0.265;
+ return true;
+ case Css::Value::CM:
+ *size = val->GetFloatValue() / 0.265 * 10;
+ return true;
+ case Css::Value::IN:
+ *size = val->GetFloatValue() * 96;
+ return true;
+ case Css::Value::PT:
+ *size = val->GetFloatValue() * 4 / 3;
+ return true;
+ case Css::Value::PC:
+ *size = val->GetFloatValue() * 16;
+ return true;
+ case Css::Value::NO_UNIT:
+ // 0 without unit is always allowed
+ if (can_unitless || val->GetFloatValue() == 0.0) {
+ *size = val->GetFloatValue() * unit;
+ return true;
+ }
+ break;
+ default:
+ break;
+ }
+ return false;
+}
+
+// Css system colors.
+// The structure is nearly literally copied from webutil/html/htmlcolor.cc.
+
+typedef struct RgbValue {
+ unsigned char r_;
+ unsigned char g_;
+ unsigned char b_;
+} RgbValue;
+
+// Color table for system colors.
+// This is only a rough estimation based on a typical setup.
+// when making change to known_system_color_values, please
+// also change the GetKnownSystemColorValue function because
+// entire table is hardcoded into the function for efficiency
+static const RgbValue known_system_color_values[] = {
+/* 0 activeborder */{212, 208, 200},
+/* 1 activecaption */{ 0, 84, 227},
+/* 2 appworkspace */{128, 128, 128},
+/* 3 background */{ 0, 78, 152},
+/* 4 buttonface */{236, 233, 216},
+/* 5 buttonhighlight */{255, 255, 255},
+/* 6 buttonshadow */{172, 168, 153},
+/* 7 buttontext */{ 0, 0, 0},
+/* 8 captiontext */{255, 255, 255},
+/* 9 graytext */{172, 168, 153},
+/*10 highlight */{ 49, 106, 197},
+/*11 highlighttext */{255, 255, 255},
+/*12 inactiveborder */{212, 208, 200},
+/*13 inactivecaption */{122, 150, 223},
+/*14 inactivecaptiontext */{216, 228, 248},
+/*15 infobackground */{255, 255, 225},
+/*16 infotext */{ 0, 0, 0},
+/*17 menu */{255, 255, 255},
+/*18 menutext */{ 0, 0, 0},
+/*19 scrollbar */{212, 208, 200},
+/*20 threeddarkshadow */{113, 111, 100},
+/*21 threedface */{236, 233, 216},
+/*22 threedhighlight */{255, 255, 255},
+/*23 threedlightshadow */{241, 239, 226},
+/*24 threedshadow */{172, 168, 153},
+/*25 window */{255, 255, 255},
+/*26 windowframe */{ 0, 0, 0},
+/*27 windowtext */{ 0, 0, 0},
+};
+
+const RgbValue* GetKnownSystemColorValue(const char *colorstr) {
+ switch (tolower(colorstr[0])) {
+ case 'a':
+ switch (tolower(colorstr[1])) {
+ case 'c':
+ if (!strcasecmp("activeborder", colorstr)) {
+ return &known_system_color_values[0];
+ } else if (!strcasecmp("activecaption", colorstr)) {
+ return &known_system_color_values[1];
+ }
+ return NULL;
+ case 'p':
+ if (!strcasecmp("appworkspace", colorstr)) {
+ return &known_system_color_values[2];
+ }
+ return NULL;
+ }
+ return NULL;
+ case 'b':
+ switch (tolower(colorstr[1])) {
+ case 'a':
+ if (!strcasecmp("background", colorstr)) {
+ return &known_system_color_values[3];
+ }
+ return NULL;
+ case 'u':
+ if (!strcasecmp("buttonface", colorstr)) {
+ return &known_system_color_values[4];
+ } else if (!strcasecmp("buttonhighlight", colorstr)) {
+ return &known_system_color_values[5];
+ } else if (!strcasecmp("buttonshadow", colorstr)) {
+ return &known_system_color_values[6];
+ } else if (!strcasecmp("buttontext", colorstr)) {
+ return &known_system_color_values[7];
+ }
+ return NULL;
+ }
+ return NULL;
+ case 'c':
+ if (!strcasecmp("captiontext", colorstr)) {
+ return &known_system_color_values[8];
+ }
+ return NULL;
+ case 'g':
+ if (!strcasecmp("graytext", colorstr)) {
+ return &known_system_color_values[9];
+ }
+ return NULL;
+ case 'h':
+ if (!strcasecmp("highlight", colorstr)) {
+ return &known_system_color_values[10];
+ } else if (!strcasecmp("highlighttext", colorstr)) {
+ return &known_system_color_values[11];
+ }
+ return NULL;
+ case 'i':
+ if (!strcasecmp("inactiveborder", colorstr)) {
+ return &known_system_color_values[12];
+ } else if (!strcasecmp("inactivecaption", colorstr)) {
+ return &known_system_color_values[13];
+ } else if (!strcasecmp("inactivecaptiontext", colorstr)) {
+ return &known_system_color_values[14];
+ } else if (!strcasecmp("infobackground", colorstr)) {
+ return &known_system_color_values[15];
+ } else if (!strcasecmp("infotext", colorstr)) {
+ return &known_system_color_values[16];
+ }
+ return NULL;
+ case 'm':
+ if (!strcasecmp("menu", colorstr)) {
+ return &known_system_color_values[17];
+ } else if (!strcasecmp("menutext", colorstr)) {
+ return &known_system_color_values[18];
+ }
+ return NULL;
+ case 's':
+ if (!strcasecmp("scrollbar", colorstr)) {
+ return &known_system_color_values[19];
+ }
+ return NULL;
+ case 't':
+ if (!strcasecmp("threeddarkshadow", colorstr)) {
+ return &known_system_color_values[20];
+ } else if (!strcasecmp("threedface", colorstr)) {
+ return &known_system_color_values[21];
+ } else if (!strcasecmp("threedhighlight", colorstr)) {
+ return &known_system_color_values[22];
+ } else if (!strcasecmp("threedlightshadow", colorstr)) {
+ return &known_system_color_values[23];
+ } else if (!strcasecmp("threedshadow", colorstr)) {
+ return &known_system_color_values[24];
+ }
+ return NULL;
+ case 'w':
+ if (!strcasecmp("window", colorstr)) {
+ return &known_system_color_values[25];
+ } else if (!strcasecmp("windowframe", colorstr)) {
+ return &known_system_color_values[26];
+ } else if (!strcasecmp("windowtext", colorstr)) {
+ return &known_system_color_values[27];
+ }
+ return NULL;
+ }
+ return NULL;
+}
+
+bool GetSystemColor(const string& colorstr, HtmlColor* color) {
+ const RgbValue *p_rgb = GetKnownSystemColorValue(colorstr.c_str());
+ if (p_rgb) {
+ color->SetValueFromRGB(p_rgb->r_, p_rgb->g_, p_rgb->b_);
+ return true;
+ } else {
+ return false;
+ }
+}
+
+bool MediaAppliesToScreen(const StringPiece& media) {
+ std::vector<StringPiece> values;
+ StringPieceUtils::Split(media, ",", &values);
+ for (std::vector<StringPiece>::iterator iter = values.begin();
+ iter < values.end(); ++iter) {
+ StringPieceUtils::RemoveWhitespaceContext(&(*iter));
+ if (memcaseis(iter->data(), iter->size(), "all") ||
+ memcaseis(iter->data(), iter->size(), "screen"))
+ return true;
+ }
+ return false;
+}
+
+bool MediaAppliesToScreen(const std::vector<UnicodeText>& media) {
+ if (media.size() == 0) return true;
+
+ for (std::vector<UnicodeText>::const_iterator iter = media.begin();
+ iter < media.end(); ++iter) {
+ if (memcaseis(iter->utf8_data(), iter->utf8_length(), "all") ||
+ memcaseis(iter->utf8_data(), iter->utf8_length(), "screen"))
+ return true;
+ }
+ return false;
+}
+
+} // namespace Util
+} // namespace Css
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/util.h b/trunk/src/third_party/css_parser/src/webutil/css/util.h
new file mode 100644
index 0000000..cbd0f58
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/util.h
@@ -0,0 +1,82 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2007 Google Inc. All Rights Reserved.
+// Author: yian@google.com (Yian Huang)
+// Author: dpeng@google.com (Daniel Peng)
+
+#ifndef WEBUTIL_CSS_UTIL_H__
+#define WEBUTIL_CSS_UTIL_H__
+
+#include <string>
+#include <vector>
+
+#include "strings/stringpiece.h"
+#include "webutil/css/string.h"
+
+class HtmlColor;
+class UnicodeText;
+
+namespace Css {
+
+class Value;
+
+namespace Util {
+
+enum COLOR_ATTR {ORIGINAL, TRANSPARENT, UNKNOWN, INHERIT};
+
+// Parses CSS color value (may be a string, identifier or a color) into
+// HtmlColor. returns def if the color is invalid. set attr to one of the
+// following attributes: ORIGINAL if the color is valid, INHERIT if it is the
+// keyword inherit, UNKNOWN if it is the keywoard unknown, TRANSPARENT
+// if it is invalid otherwise.
+HtmlColor GetCssColor(const Css::Value* val, const HtmlColor& def,
+ COLOR_ATTR* attr);
+
+// Converts length or percentage string to absolute px units. Refer to
+// parent_size when seeing 10% (invalid if parent_size is -1). Refer to
+// font_size when seeing 1.2EM or 1.2EX. Invalid if val is NULL or it is not
+// a number. It can also be invalid if can_negative is set and the value is
+// negative, can_unitless works similarly. Returns if parsing succeeds, and
+// if so, size stores the result.
+bool GetCssLength(const Css::Value* val, double parent_size,
+ double font_size, double unit, bool can_negative,
+ bool can_unitless, double* size);
+
+// Updates color with system color specified in colorstr. The change is only
+// done only when the conversion succeeds, indicated by the return value.
+// For a list of system colors, please see
+// http://www.w3.org/TR/CSS21/ui.html#system-colors
+// Actual system colors depend on OS's graphic environment. For the purpose
+// of hidden text detection, we assume a typical setting based on Windows XP
+// default theme.
+bool GetSystemColor(const string& colorstr, HtmlColor* color);
+
+// Whether a media string (comma separated list of media) is compatible with
+// screen-oriented applications. It is valid if no media is specified or some
+// medium has the name "screen" or "all".
+bool MediaAppliesToScreen(const StringPiece& media);
+
+// Whether a media list is compatible with screen-oriented applications. It
+// is valid if no media is specified or some medium has the name "screen" or
+// "all".
+bool MediaAppliesToScreen(const std::vector<UnicodeText>& media);
+
+
+} // namespace Util
+} // namespace Css
+
+#endif // WEBUTIL_CSS_UTIL_H__
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/util_test.cc b/trunk/src/third_party/css_parser/src/webutil/css/util_test.cc
new file mode 100644
index 0000000..d6740aa
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/util_test.cc
@@ -0,0 +1,78 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2007 Google Inc. All Rights Reserved.
+// Author: yian@google.com (Yi-An Huang)
+
+#include "webutil/css/util.h"
+
+#include <string>
+
+#include "base/logging.h"
+#include "base/scoped_ptr.h"
+#include "testing/base/public/googletest.h"
+#include "testing/base/public/gunit.h"
+#include "webutil/css/string.h"
+#include "webutil/html/htmlcolor.h"
+
+namespace {
+
+class CsssystemcolorTest : public testing::Test {
+ protected:
+ virtual void SetUp() {
+ color_.reset(new HtmlColor(0, 0, 0));
+ }
+
+ virtual void TearDown() {
+ }
+
+ void TestColor(const char* name, const char* mapped_to) {
+ CHECK(Css::Util::GetSystemColor(name, color_.get()));
+ // TODO(sligocki): Chromium CHECK_STREQ appears to be buggy. Fixit.
+ //CHECK_STREQ(color_->ToString().c_str(), mapped_to);
+ CHECK_EQ(color_->ToString(), string(mapped_to));
+ }
+
+ void TestInvalidColor(const char* name) {
+ CHECK(!Css::Util::GetSystemColor(name, color_.get()));
+ }
+
+ private:
+ scoped_ptr<HtmlColor> color_;
+};
+
+TEST_F(CsssystemcolorTest, common_colors) {
+ // test some "intuitive" colors
+ TestColor("menu", "#ffffff"); // menu background is white
+ TestColor("menutext", "#000000"); // menu foreground is black
+ TestColor("buttonface", "#ece9d8"); // button background is grey
+ TestColor("captiontext", "#ffffff"); // caption text is white
+ TestColor("windowtext", "#000000"); // window text is black
+}
+
+TEST_F(CsssystemcolorTest, case_sensitiveness) {
+ // colors are case insensitive
+ TestColor("activeBorder", "#d4d0c8");
+ TestColor("BACKGROUND", "#004e98");
+}
+
+TEST_F(CsssystemcolorTest, invalid_colors) {
+ TestInvalidColor("menux");
+ TestInvalidColor("text");
+ TestInvalidColor("win");
+}
+
+} // namespace
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/value.cc b/trunk/src/third_party/css_parser/src/webutil/css/value.cc
new file mode 100644
index 0000000..e6e675f
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/value.cc
@@ -0,0 +1,323 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2006 Google Inc. All Rights Reserved.
+// Author: dpeng@google.com (Daniel Peng)
+
+#include "util/gtl/stl_util-inl.h"
+
+#include "webutil/css/value.h"
+
+#include "base/logging.h"
+#include "strings/memutil.h"
+#include "webutil/html/htmlcolor.h"
+
+namespace Css {
+
+//
+// Static constants.
+//
+
+const char* const Value::kDimensionUnitText[] = {
+ "em", "ex", "px", "cm", "mm", "in", "pt", "pc",
+ "deg", "rad", "grad", "ms", "s", "hz", "khz", "%", "OTHER", "" };
+
+//
+// Constructors.
+//
+
+Value::Value(ValueType ty)
+ : type_(ty),
+ color_(0, 0, 0) {
+ DCHECK(ty == DEFAULT || ty == UNKNOWN);
+}
+
+Value::Value(float num, const UnicodeText& unit)
+ : type_(NUMBER),
+ num_(num),
+ color_(0, 0, 0) {
+ unit_ = UnitFromText(unit.utf8_data(), unit.utf8_length());
+ if (unit_ == OTHER)
+ str_ = unit;
+}
+
+Value::Value(float num, Unit unit)
+ : type_(NUMBER),
+ num_(num),
+ unit_(unit),
+ color_(0, 0, 0) {
+ DCHECK_NE(unit, OTHER);
+}
+
+Value::Value(ValueType ty, const UnicodeText& str)
+ : type_(ty),
+ str_(str),
+ color_(0, 0, 0) {
+ DCHECK(ty == STRING || ty == URI);
+}
+
+Value::Value(const Identifier& identifier)
+ : type_(IDENT),
+ identifier_(identifier),
+ color_(0, 0, 0) {
+}
+
+Value::Value(const Identifier::Ident ident)
+ : type_(IDENT),
+ identifier_(Identifier(ident)),
+ color_(0, 0, 0) {
+}
+
+Value::Value(ValueType ty, Values* params)
+ : type_(ty),
+ params_(params),
+ color_(0, 0, 0) {
+ DCHECK(params != NULL);
+ DCHECK(ty == RECT || ty == COUNTER);
+}
+
+Value::Value(const UnicodeText& func, Values* params)
+ : type_(FUNCTION),
+ str_(func),
+ params_(params),
+ color_(0, 0, 0) {
+ DCHECK(params != NULL);
+}
+
+Value::Value(HtmlColor c)
+ : type_(COLOR),
+ color_(c) {
+}
+
+Value::Value(const Value& other)
+ : type_(other.type_),
+ num_(other.num_),
+ unit_(other.unit_),
+ identifier_(other.identifier_),
+ str_(other.str_),
+ color_(other.color_) {
+ if (other.params_.get() != NULL) {
+ Values* vals = new Values;
+ vals->reserve(other.params_->size());
+ for (Values::const_iterator iter = other.params_->begin();
+ iter < other.params_->end(); ++iter)
+ vals->push_back(new Value(**iter));
+ params_.reset(vals);
+ }
+}
+
+Value& Value::operator=(const Value& other) {
+ if (this == &other) return *this;
+ type_ = other.type_;
+ num_ = other.num_;
+ unit_ = other.unit_;
+ identifier_ = other.identifier_;
+ str_ = other.str_;
+ color_ = other.color_;
+ if (other.params_.get() != NULL) {
+ Values* vals = new Values;
+ vals->reserve(other.params_->size());
+ for (Values::const_iterator iter = other.params_->begin();
+ iter < other.params_->end(); ++iter)
+ vals->push_back(new Value(**iter));
+ params_.reset(vals);
+ } else {
+ params_.reset();
+ }
+ return *this;
+}
+
+bool Value::Equals(const Value& other) const {
+ if (type_ != other.type_) return false;
+ switch (type_) {
+ case DEFAULT:
+ case UNKNOWN:
+ return true;
+ case NUMBER:
+ return unit_ == other.unit_ && num_ == other.num_;
+ case URI:
+ case STRING:
+ return str_ == other.str_;
+ case IDENT:
+ if (identifier_.ident() != other.identifier_.ident())
+ return false;
+ if (identifier_.ident() == Identifier::OTHER)
+ return identifier_.ident_text() == other.identifier_.ident_text();
+ return true;
+ case COLOR:
+ if (color_.IsDefined() != other.color_.IsDefined())
+ return false;
+ if (color_.IsDefined())
+ return color_.rgb() == other.color_.rgb();
+ return true;
+ case FUNCTION:
+ if (str_ != other.str_)
+ return false;
+ // pass through
+ case COUNTER:
+ case RECT:
+ if (params_.get() == NULL)
+ return other.params_.get() == NULL;
+ if (params_->size() != other.params_->size())
+ return false;
+ for (Values::const_iterator iter = params_->begin(),
+ other_iter = other.params_->begin();
+ iter < params_->end(); ++iter, ++other_iter)
+ if (!(**iter).Equals(**other_iter))
+ return false;
+ return true;
+ default:
+ LOG(FATAL) << "Unknown type:" << type_;
+ }
+}
+
+//
+// Static functions mapping between units and strings
+//
+
+Value::Unit Value::UnitFromText(const char* str, int len) {
+ switch (len) {
+ case 0:
+ return NO_UNIT;
+ case 1:
+ switch (str[0]) {
+ case '%': return PERCENT;
+ case 's': case 'S': return S;
+ default: return OTHER;
+ }
+ case 2:
+ switch (str[0]) {
+ case 'e': case 'E':
+ switch (str[1]) {
+ case 'm': case 'M': return EM;
+ case 'x': case 'X': return EX;
+ default: return OTHER;
+ }
+ case 'p': case 'P':
+ switch (str[1]) {
+ case 'x': case 'X': return PX;
+ case 't': case 'T': return PT;
+ case 'c': case 'C': return PC;
+ default: return OTHER;
+ }
+ case 'c': case 'C':
+ switch (str[1]) {
+ case 'm': case 'M': return CM;
+ default: return OTHER;
+ }
+ case 'm': case 'M':
+ switch (str[1]) {
+ case 'm': case 'M': return MM;
+ case 's': case 'S': return MS;
+ default: return OTHER;
+ }
+ case 'i': case 'I':
+ switch (str[1]) {
+ case 'n': case 'N': return IN;
+ default: return OTHER;
+ }
+ case 'h': case 'H':
+ switch (str[1]) {
+ case 'z': case 'Z': return HZ;
+ default: return OTHER;
+ }
+ default:
+ return OTHER;
+ }
+ case 3:
+ if (memcasecmp(str, "deg", 3) == 0) return DEG;
+ if (memcasecmp(str, "rad", 3) == 0) return RAD;
+ if (memcasecmp(str, "khz", 3) == 0) return KHZ;
+ return OTHER;
+ case 4:
+ if (memcasecmp(str, "grad", 3) == 0) return GRAD;
+ return OTHER;
+ default:
+ return OTHER;
+ }
+}
+
+const char* Value::TextFromUnit(Unit u) {
+ DCHECK_LT(u, NUM_UNITS);
+ COMPILE_ASSERT(arraysize(kDimensionUnitText) == NUM_UNITS, dimensionunitsize);
+
+ return kDimensionUnitText[u];
+}
+
+//
+// Accessors.
+//
+
+Value::ValueType Value::GetLexicalUnitType() const {
+ return type_;
+}
+
+string Value::GetDimensionUnitText() const {
+ DCHECK_EQ(type_, NUMBER);
+ if (unit_ == OTHER)
+ return string(str_.utf8_data(), str_.utf8_length());
+ else
+ return TextFromUnit(unit_);
+}
+
+Value::Unit Value::GetDimension() const {
+ DCHECK_EQ(type_, NUMBER);
+ return unit_;
+}
+
+int Value::GetIntegerValue() const {
+ DCHECK_EQ(type_, NUMBER);
+ return static_cast<int>(num_);
+}
+
+float Value::GetFloatValue() const {
+ DCHECK_EQ(type_, NUMBER);
+ return num_;
+}
+
+const Values* Value::GetParameters() const {
+ DCHECK(type_ == FUNCTION || type_ == RECT || type_ == COUNTER);
+ return params_.get();
+}
+
+const UnicodeText& Value::GetFunctionName() const {
+ DCHECK_EQ(type_, FUNCTION);
+ return str_;
+}
+
+const UnicodeText& Value::GetStringValue() const {
+ DCHECK(type_ == URI || type_ == STRING);
+ return str_;
+}
+
+UnicodeText Value::GetIdentifierText() const {
+ DCHECK_EQ(type_, IDENT);
+ return identifier_.ident_text();
+}
+
+const Identifier& Value::GetIdentifier() const {
+ DCHECK_EQ(type_, IDENT);
+ return identifier_;
+}
+
+const HtmlColor& Value::GetColorValue() const {
+ DCHECK_EQ(type_, COLOR);
+ return color_;
+}
+
+Values::~Values() { STLDeleteElements(this); }
+
+} // namespace
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/value.h b/trunk/src/third_party/css_parser/src/webutil/css/value.h
new file mode 100644
index 0000000..c3402c7
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/value.h
@@ -0,0 +1,166 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2006 Google Inc. All Rights Reserved.
+// Author: dpeng@google.com (Daniel Peng)
+
+#ifndef WEBUTIL_CSS_VALUE_H__
+#define WEBUTIL_CSS_VALUE_H__
+
+#include <string>
+#include <vector>
+
+#include "base/macros.h"
+#include "base/scoped_ptr.h"
+#include "util/utf8/public/unicodetext.h"
+#include "webutil/css/identifier.h"
+#include "webutil/css/string.h"
+#include "webutil/html/htmlcolor.h"
+
+// resolving conflict on macro HZ defined elsewhere.
+#ifdef HZ
+const int HZ_temporary = HZ;
+#undef HZ
+const int HZ = HZ_temporary;
+#endif
+
+namespace Css {
+
+class Values;
+
+// A Value represents a value in CSS (maybe more generally, a
+// lexical unit). There are many different types of these, so you can
+// think of a Value as a tagged union of various values. The tag
+// is set by the constructor and accessed with GetLexicalUnitType().
+// The values are also set by the constructor and accessed with the
+// various accessors.
+class Value {
+ public:
+ enum ValueType { NUMBER, URI, COUNTER, FUNCTION, RECT,
+ COLOR, STRING, IDENT, UNKNOWN, DEFAULT };
+ enum Unit { EM, EX, PX, CM, MM, IN, PT, PC,
+ DEG, RAD, GRAD, MS, S, HZ, KHZ, PERCENT, OTHER, NO_UNIT,
+ NUM_UNITS };
+
+ // These constructors generate Values of various types.
+
+ Value() : type_(DEFAULT), color_(0, 0, 0) { }
+
+ // UNKNOWN or DEFAULT
+ Value(ValueType ty); // NOLINT
+
+ // NUMBER with unit. OTHER is not a valid unit here. Use the next form:
+ Value(float num, Unit unit);
+
+ // NUMBER with unit; we convert unit to an enum for you. If it's
+ // not a known unit, we use the OTHER enum and save the text.
+ Value(float num, const UnicodeText& unit);
+
+ // Any of the string types (URI, STRING). For IDENT, use the next
+ // constructor instead.
+ Value(ValueType ty, const UnicodeText& str);
+
+ // IDENT from an identifier.
+ explicit Value(const Identifier& identifier);
+ explicit Value(const Identifier::Ident ident);
+
+ // Any of the special function types (COUNTER, RECT)
+ // NOTE: The ownership of params will be taken.
+ // params cannot be NULL, if no parameters are needed, pass an empty Values.
+ explicit Value(ValueType ty, Values* params);
+
+ // FUNCTION with name func
+ // NOTE: The ownership of params will be taken.
+ // params cannot be NULL, if no parameters are needed, pass an empty Values.
+ explicit Value(const UnicodeText& func, Values* params);
+
+ // COLOR.
+ explicit Value(HtmlColor color);
+
+ // copy constructor and assignment operator
+ Value(const Value& other);
+ Value& operator=(const Value& other);
+
+ // equality.
+ bool Equals(const Value& other) const;
+
+ // Given the text of a CSS unit, UnitFromText returns the
+ // corresponding enum. If no such unit is found, UnitFromText
+ // returns OTHER. Since all CSS units are ASCII, we are happy with
+ // ASCII, UTF8, Latin-1, etc.
+ static Unit UnitFromText(const char* s, int len);
+ // Given a unit, returns its string representation. If u is
+ // NO_UNIT, returns "". If u is OTHER, we return "OTHER", but this
+ // may not be what you want.
+ static const char* TextFromUnit(Unit u);
+
+ // Returns a string representation of the value.
+ string ToString() const;
+
+ // Accessors. Modeled after
+ // http://www.w3.org/Style/CSS/SAC/doc/org/w3c/css/sac/LexicalUnit.html
+
+ ValueType GetLexicalUnitType() const; // The type of value
+
+ // Each of these accessors is only valid for certain types. The
+ // comment indicates for which types they are valid; we DCHECK this
+ // precondition.
+ string GetDimensionUnitText() const; // NUMBER: the unit as a string.
+ Unit GetDimension() const; // NUMBER: the unit.
+ int GetIntegerValue() const; // NUMBER: the integer value.
+ float GetFloatValue() const; // NUMBER: the float value.
+ const Values* GetParameters() const; // FUNCTION: the function params
+ const UnicodeText& GetFunctionName() const; // FUNCTION: the function name.
+ const UnicodeText& GetStringValue() const; // URI, STRING: the string value
+ UnicodeText GetIdentifierText() const; // IDENT: the ident as a string.
+ const Identifier& GetIdentifier() const; // IDENT: identifier.
+ const HtmlColor& GetColorValue() const; // COLOR: the color value
+
+ private:
+ ValueType type_; // indicates the type of value. Always valid.
+ float num_; // for NUMBER (integer values are stored as floats)
+ Unit unit_; // for NUMBER
+ Identifier identifier_; // for IDENT
+ UnicodeText str_; // for NUMBER (OTHER unit_), URI, STRING, FUNCTION
+ scoped_ptr<Values> params_; // FUNCTION, COUNTER and RECT params
+ HtmlColor color_; // COLOR
+
+ // kDimensionUnitText stores the name of each unit (see TextFromUnit)
+ static const char* const kDimensionUnitText[];
+};
+
+// Values is a vector of Value*, which we own and will delete
+// upon destruction. If you remove elements from Values, you are
+// responsible for deleting them.
+// Also, be careful --- there's no virtual destructor, so this must be
+// deleted as a Values.
+class Values : public std::vector<Value*> {
+ public:
+ Values() : std::vector<Value*>() { }
+ ~Values();
+
+ // We provide syntactic sugar for accessing elements.
+ // values->get(i) looks better than (*values)[i])
+ const Value* get(int i) const { return (*this)[i]; }
+
+ string ToString() const;
+ private:
+ DISALLOW_COPY_AND_ASSIGN(Values);
+};
+
+} // namespace
+
+#endif // WEBUTIL_CSS_VALUE_H__
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/valuevalidator.cc b/trunk/src/third_party/css_parser/src/webutil/css/valuevalidator.cc
new file mode 100644
index 0000000..0af595c
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/valuevalidator.cc
@@ -0,0 +1,601 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2007 Google Inc. All Rights Reserved.
+// Author: yian@google.com (Yi-An Huang)
+
+#include "base/logging.h"
+#include "base/macros.h"
+#include "util/gtl/map-util.h"
+#include "util/gtl/stl_util-inl.h"
+#include "webutil/css/valuevalidator.h"
+
+namespace Css {
+
+const Property::Prop kEndProp = static_cast<Property::Prop>(-1);
+const Value::ValueType kEndType = static_cast<Value::ValueType>(-1);
+const Identifier::Ident kEndIdent = static_cast<Identifier::Ident>(-1);
+
+// For each CSS property (or a group of properties), the valid value types and
+// valid IDENT values. We do not list the IDENT, DEFAULT and UNKOWN types
+// because they are always valid, we also do not list INHERIT values explicitly
+// for the same reason. Arrays are terminated with -1.
+static struct valid_prop_info_t {
+ Property::Prop props[10];
+ Value::ValueType types[10];
+ Identifier::Ident idents[30];
+ // the following properties make sense for numbers.
+ bool accept_percent;
+ bool accept_no_unit;
+ bool accept_length;
+ bool accept_negative;
+} kValidPropInfo[] = {
+ // Chapter 8: Box model
+
+ { { Property::BORDER_COLOR, Property::BORDER_TOP_COLOR,
+ Property::BORDER_RIGHT_COLOR, Property::BORDER_BOTTOM_COLOR,
+ Property::BORDER_LEFT_COLOR, kEndProp },
+ { Value::COLOR, kEndType },
+ { Identifier::TRANSPARENT, Identifier::GOOG_INITIAL, kEndIdent },
+ },
+ { { Property::BORDER_STYLE, Property::BORDER_TOP_STYLE,
+ Property::BORDER_RIGHT_STYLE, Property::BORDER_BOTTOM_STYLE,
+ Property::BORDER_LEFT_STYLE, kEndProp },
+ { kEndType },
+ { Identifier::NONE, Identifier::HIDDEN, Identifier::DOTTED,
+ Identifier::DASHED, Identifier::SOLID, Identifier::DOUBLE,
+ Identifier::GROOVE, Identifier::RIDGE, Identifier::INSET,
+ Identifier::OUTSET, kEndIdent },
+ },
+ { { Property::BORDER_WIDTH, Property::BORDER_TOP_WIDTH,
+ Property::BORDER_RIGHT_WIDTH, Property::BORDER_BOTTOM_WIDTH,
+ Property::BORDER_LEFT_WIDTH, kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::THIN, Identifier::MEDIUM, Identifier::THICK, kEndIdent },
+ false, false, true, false,
+ },
+ { { Property::BORDER, Property::BORDER_TOP, Property::BORDER_RIGHT,
+ Property::BORDER_BOTTOM, Property::BORDER_LEFT, kEndProp },
+ { Value::COLOR, Value::NUMBER, kEndType },
+ { Identifier::TRANSPARENT, Identifier::GOOG_INITIAL,
+ Identifier::NONE, Identifier::HIDDEN, Identifier::DOTTED,
+ Identifier::DASHED, Identifier::SOLID, Identifier::DOUBLE,
+ Identifier::GROOVE, Identifier::RIDGE, Identifier::INSET,
+ Identifier::OUTSET,
+ Identifier::THIN, Identifier::MEDIUM, Identifier::THICK, kEndIdent },
+ false, false, true, false,
+ },
+
+ { { Property::MARGIN, Property::MARGIN_RIGHT, Property::MARGIN_LEFT,
+ Property::MARGIN_TOP, Property::MARGIN_BOTTOM, kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::AUTO, kEndIdent },
+ true, false, true, true,
+ },
+
+ { { Property::PADDING, Property::PADDING_RIGHT, Property::PADDING_LEFT,
+ Property::PADDING_TOP, Property::PADDING_BOTTOM, kEndProp },
+ { Value::NUMBER, kEndType },
+ { kEndIdent },
+ true, false, true, false,
+ },
+
+ // Chapter 9: Visual formatting model
+
+ { { Property::BOTTOM, Property::LEFT, Property::RIGHT, Property::TOP,
+ kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::AUTO, kEndIdent },
+ true, false, true, true,
+ },
+
+ { { Property::CLEAR, kEndProp },
+ { kEndType },
+ { Identifier::NONE, Identifier::LEFT, Identifier::RIGHT, Identifier::BOTH,
+ kEndIdent },
+ },
+
+ { { Property::DIRECTION, kEndProp },
+ { kEndType },
+ { Identifier::LTR, Identifier::RTL, kEndIdent },
+ },
+
+ { { Property::DISPLAY, kEndProp },
+ { kEndType },
+ { Identifier::INLINE, Identifier::BLOCK, Identifier::LIST_ITEM,
+ Identifier::RUN_IN, Identifier::INLINE_BLOCK, Identifier::TABLE,
+ Identifier::INLINE_TABLE, Identifier::TABLE_ROW_GROUP,
+ Identifier::TABLE_HEADER_GROUP, Identifier::TABLE_FOOTER_GROUP,
+ Identifier::TABLE_ROW, Identifier::TABLE_COLUMN_GROUP,
+ Identifier::TABLE_COLUMN, Identifier::TABLE_CELL,
+ Identifier::TABLE_CAPTION, Identifier::NONE, kEndIdent },
+ },
+
+ { { Property::FLOAT, kEndProp },
+ { kEndType },
+ { Identifier::LEFT, Identifier::RIGHT, Identifier::NONE, kEndIdent },
+ },
+
+ { { Property::POSITION, kEndProp },
+ { kEndType },
+ { Identifier::STATIC, Identifier::RELATIVE, Identifier::ABSOLUTE,
+ Identifier::FIXED, kEndIdent },
+ },
+
+ { { Property::UNICODE_BIDI, kEndProp },
+ { kEndType },
+ { Identifier::NORMAL, Identifier::EMBED, Identifier::BIDI_OVERRIDE,
+ kEndIdent },
+ },
+
+ { { Property::Z_INDEX, kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::AUTO, kEndIdent },
+ false, true, false, true,
+ },
+
+ // Chapter 10: Visual formatting model details
+
+ { { Property::HEIGHT, Property::WIDTH, kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::AUTO, kEndIdent },
+ true, false, true, false,
+ },
+
+ { { Property::LINE_HEIGHT, kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::NORMAL, kEndIdent },
+ true, true, true, false,
+ },
+
+ { { Property::MAX_HEIGHT, Property::MAX_WIDTH, kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::NONE, kEndIdent },
+ true, false, true, false,
+ },
+
+ { { Property::MIN_HEIGHT, Property::MIN_WIDTH, kEndProp },
+ { Value::NUMBER, kEndType },
+ { kEndIdent },
+ true, false, true, false,
+ },
+
+ { { Property::VERTICAL_ALIGN, kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::BASELINE, Identifier::SUB, Identifier::SUPER,
+ Identifier::TOP, Identifier::TEXT_TOP, Identifier::MIDDLE,
+ Identifier::BOTTOM, Identifier::TEXT_BOTTOM, kEndIdent },
+ true, false, true, true,
+ },
+
+ // Chapter 11: Visual effects
+
+ { { Property::CLIP, kEndProp },
+ { Value::RECT, kEndType },
+ { Identifier::AUTO, kEndIdent },
+ false, false, true, true,
+ },
+
+ { { Property::OVERFLOW, kEndProp },
+ { kEndType },
+ { Identifier::VISIBLE, Identifier::HIDDEN, Identifier::SCROLL,
+ Identifier::AUTO, kEndIdent },
+ },
+
+ { { Property::VISIBILITY, kEndProp },
+ { kEndType },
+ { Identifier::VISIBLE, Identifier::HIDDEN, Identifier::COLLAPSE, kEndIdent
+ },
+ },
+
+ // Chapter 12: Generated content, automatic numbering, and lists
+
+ { { Property::CONTENT, kEndProp },
+ { Value::STRING, Value::URI, Value::COUNTER, Value::FUNCTION, kEndType },
+ { Identifier::NORMAL, Identifier::NONE,
+ Identifier::OPEN_QUOTE, Identifier::CLOSE_QUOTE,
+ Identifier::NO_OPEN_QUOTE, Identifier::NO_CLOSE_QUOTE, kEndIdent },
+ },
+
+ { { Property::COUNTER_INCREMENT, Property::COUNTER_RESET, kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::NONE, Identifier::OTHER, kEndIdent },
+ false, true, false, true,
+ },
+
+ { { Property::LIST_STYLE_IMAGE, kEndProp },
+ { Value::URI, kEndType },
+ { Identifier::NONE, kEndIdent },
+ },
+ { { Property::LIST_STYLE_POSITION, kEndProp },
+ { kEndType },
+ { Identifier::INSIDE, Identifier::OUTSIDE, kEndIdent },
+ },
+ { { Property::LIST_STYLE_TYPE, kEndProp },
+ { kEndType },
+ { Identifier::DISC, Identifier::CIRCLE, Identifier::SQUARE,
+ Identifier::DECIMAL, Identifier::DECIMAL_LEADING_ZERO,
+ Identifier::LOWER_ROMAN, Identifier::UPPER_ROMAN,
+ Identifier::LOWER_GREEK, Identifier::LOWER_LATIN,
+ Identifier::UPPER_LATIN, Identifier::ARMENIAN, Identifier::GEORGIAN,
+ Identifier::LOWER_ALPHA, Identifier::UPPER_ALPHA,
+ Identifier::NONE, kEndIdent },
+ },
+ { { Property::LIST_STYLE, kEndProp },
+ { Value::URI, kEndType },
+ { Identifier::NONE, Identifier::INSIDE, Identifier::OUTSIDE,
+ Identifier::DISC, Identifier::CIRCLE, Identifier::SQUARE,
+ Identifier::DECIMAL, Identifier::DECIMAL_LEADING_ZERO,
+ Identifier::LOWER_ROMAN, Identifier::UPPER_ROMAN,
+ Identifier::LOWER_GREEK, Identifier::LOWER_LATIN,
+ Identifier::UPPER_LATIN, Identifier::ARMENIAN, Identifier::GEORGIAN,
+ Identifier::LOWER_ALPHA, Identifier::UPPER_ALPHA, kEndIdent },
+ },
+
+ { { Property::QUOTES, kEndProp },
+ { Value::STRING, kEndType },
+ { Identifier::NONE, kEndIdent },
+ },
+
+ // Chapter 13: Paged media
+
+ { { Property::ORPHANS, kEndProp },
+ { Value::NUMBER, kEndType },
+ { kEndIdent },
+ false, true, false, false,
+ },
+
+ { { Property::PAGE_BREAK_AFTER, Property::PAGE_BREAK_BEFORE, kEndProp },
+ { kEndType },
+ { Identifier::AUTO, Identifier::ALWAYS, Identifier::AVOID,
+ Identifier::LEFT, Identifier::RIGHT, kEndIdent },
+ },
+ { { Property::PAGE_BREAK_INSIDE, kEndProp },
+ { kEndType },
+ { Identifier::AVOID, Identifier::AUTO, kEndIdent },
+ },
+
+ { { Property::WIDOWS, kEndProp },
+ { Value::NUMBER, kEndType },
+ { kEndIdent },
+ false, true, false, false,
+ },
+
+ // Chapter 14: Colors and Backgrounds
+
+ { { Property::BACKGROUND_ATTACHMENT, kEndProp },
+ { kEndType },
+ { Identifier::SCROLL, Identifier::FIXED, kEndIdent },
+ },
+ { { Property::BACKGROUND_COLOR, kEndProp },
+ { Value::COLOR, kEndType },
+ { Identifier::TRANSPARENT, kEndIdent },
+ },
+ { { Property::BACKGROUND_IMAGE, kEndProp },
+ { Value::URI, kEndType },
+ { Identifier::NONE, kEndIdent },
+ },
+ { { Property::BACKGROUND_POSITION, kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::LEFT, Identifier::CENTER, Identifier::RIGHT, Identifier::TOP,
+ Identifier::BOTTOM, kEndIdent },
+ true, false, true, true,
+ },
+ { { Property::BACKGROUND_REPEAT, kEndProp },
+ { kEndType },
+ { Identifier::REPEAT, Identifier::REPEAT_X, Identifier::REPEAT_Y,
+ Identifier::NO_REPEAT, kEndIdent },
+ },
+ { { Property::BACKGROUND, kEndProp },
+ { Value::COLOR, Value::URI, Value::NUMBER, kEndType },
+ { Identifier::SCROLL, Identifier::FIXED, Identifier::TRANSPARENT,
+ Identifier::NONE, Identifier::LEFT, Identifier::CENTER,
+ Identifier::RIGHT, Identifier::TOP, Identifier::BOTTOM,
+ Identifier::REPEAT, Identifier::REPEAT_X, Identifier::REPEAT_Y,
+ Identifier::NO_REPEAT, kEndIdent },
+ true, false, true, true,
+ },
+
+ { { Property::COLOR, kEndProp },
+ { Value::COLOR, kEndType },
+ { Identifier::GOOG_BODY_COLOR, Identifier::GOOG_BODY_LINK_COLOR,
+ kEndIdent },
+ },
+
+ // Chapter 15: Fonts
+
+ { { Property::FONT_FAMILY, kEndProp },
+ { Value::STRING, kEndType },
+ { Identifier::SERIF, Identifier::SANS_SERIF, Identifier::CURSIVE,
+ Identifier::FANTASY, Identifier::MONOSPACE, Identifier::OTHER, kEndIdent
+ },
+ },
+ { { Property::FONT_SIZE, kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::XX_SMALL, Identifier::X_SMALL, Identifier::SMALL,
+ Identifier::MEDIUM, Identifier::LARGE, Identifier::X_LARGE,
+ Identifier::XX_LARGE, Identifier::LARGER, Identifier::SMALLER,
+ Identifier::GOOG_BIG, Identifier::GOOG_SMALL, kEndIdent
+ },
+ true, false, true, false,
+ },
+ { { Property::FONT_STYLE, kEndProp },
+ { kEndType },
+ { Identifier::NORMAL, Identifier::ITALIC, Identifier::OBLIQUE, kEndIdent },
+ },
+ { { Property::FONT_VARIANT, kEndProp },
+ { kEndType },
+ { Identifier::NORMAL, Identifier::SMALL_CAPS, kEndIdent },
+ },
+ { { Property::FONT_WEIGHT, kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::NORMAL, Identifier::BOLD, Identifier::BOLDER,
+ Identifier::LIGHTER, kEndIdent },
+ false, true, true, false,
+ },
+ { { Property::FONT, kEndProp },
+ { Value::STRING, Value::NUMBER, kEndType },
+ { Identifier::SERIF, Identifier::SANS_SERIF, Identifier::CURSIVE,
+ Identifier::FANTASY, Identifier::MONOSPACE, Identifier::OTHER,
+ Identifier::XX_SMALL, Identifier::X_SMALL, Identifier::SMALL,
+ Identifier::MEDIUM, Identifier::LARGE, Identifier::X_LARGE,
+ Identifier::XX_LARGE, Identifier::LARGER, Identifier::SMALLER,
+ Identifier::NORMAL, Identifier::ITALIC, Identifier::OBLIQUE,
+ Identifier::SMALL_CAPS, Identifier::BOLD, Identifier::BOLDER,
+ Identifier::LIGHTER,
+ Identifier::CAPTION, Identifier::ICON, Identifier::MENU,
+ Identifier::MESSAGE_BOX, Identifier::MESSAGE_BOX,
+ Identifier::SMALL_CAPTION, Identifier::STATUS_BAR, kEndIdent },
+ true, true, true, false,
+ },
+
+ // Chapter 16: Text
+
+ { { Property::LETTER_SPACING, kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::NORMAL, kEndIdent },
+ false, false, true, true,
+ },
+
+ { { Property::TEXT_ALIGN, kEndProp },
+ { kEndType },
+ { Identifier::LEFT, Identifier::RIGHT, Identifier::CENTER,
+ Identifier::JUSTIFY, Identifier::GOOG_INITIAL, kEndIdent },
+ },
+ { { Property::TEXT_DECORATION, kEndProp },
+ { kEndType },
+ { Identifier::NONE, Identifier::UNDERLINE, Identifier::OVERLINE,
+ Identifier::LINE_THROUGH, Identifier::BLINK, kEndIdent },
+ },
+ { { Property::TEXT_INDENT, kEndProp },
+ { Value::NUMBER, kEndType },
+ { kEndIdent },
+ true, false, true, true,
+ },
+ { { Property::TEXT_TRANSFORM, kEndProp },
+ { kEndType },
+ { Identifier::CAPITALIZE, Identifier::UPPERCASE, Identifier::LOWERCASE,
+ Identifier::NONE, kEndIdent },
+ },
+
+ { { Property::WHITE_SPACE, kEndProp },
+ { kEndType },
+ { Identifier::NORMAL, Identifier::PRE, Identifier::NOWRAP,
+ Identifier::PRE_WRAP, Identifier::PRE_LINE, kEndIdent },
+ },
+
+ { { Property::WORD_SPACING, kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::NORMAL, kEndIdent },
+ false, false, true, true,
+ },
+
+ // Chapter 17: Tables
+
+ { { Property::BORDER_COLLAPSE, kEndProp },
+ { kEndType },
+ { Identifier::COLLAPSE, Identifier::SEPARATE, kEndIdent },
+ },
+ { { Property::BORDER_SPACING, kEndProp },
+ { Value::NUMBER, kEndType },
+ { kEndIdent },
+ false, false, true, false,
+ },
+
+ { { Property::CAPTION_SIDE, kEndProp },
+ { kEndType },
+ { Identifier::TOP, Identifier::BOTTOM, kEndIdent },
+ },
+
+ { { Property::EMPTY_CELLS, kEndProp },
+ { kEndType },
+ { Identifier::SHOW, Identifier::HIDE, kEndIdent },
+ },
+
+ { { Property::TABLE_LAYOUT, kEndProp },
+ { kEndType },
+ { Identifier::AUTO, Identifier::FIXED, kEndIdent },
+ },
+
+ // Chapter 18: User interface
+
+ { { Property::CURSOR, kEndProp },
+ { Value::URI, kEndType },
+ { Identifier::AUTO, Identifier::CROSSHAIR, Identifier::DEFAULT,
+ Identifier::POINTER, Identifier::MOVE, Identifier::E_RESIZE,
+ Identifier::NE_RESIZE, Identifier::NW_RESIZE, Identifier::N_RESIZE,
+ Identifier::SE_RESIZE, Identifier::SW_RESIZE, Identifier::S_RESIZE,
+ Identifier::W_RESIZE, Identifier::TEXT, Identifier::WAIT,
+ Identifier::HELP, Identifier::PROGRESS, kEndIdent },
+ },
+
+ { { Property::OUTLINE_COLOR, kEndProp },
+ { Value::COLOR, kEndType },
+ { Identifier::INVERT, kEndIdent },
+ },
+ { { Property::OUTLINE_STYLE, kEndProp },
+ { kEndType },
+ { Identifier::NONE, Identifier::DOTTED, Identifier::DASHED,
+ Identifier::SOLID, Identifier::DOUBLE, Identifier::GROOVE,
+ Identifier::RIDGE, Identifier::INSET, Identifier::OUTSET, kEndIdent },
+ },
+ { { Property::OUTLINE_WIDTH, kEndProp },
+ { Value::NUMBER, kEndType },
+ { Identifier::THIN, Identifier::MEDIUM, Identifier::THICK, kEndIdent },
+ false, false, true, false,
+ },
+ { { Property::OUTLINE, kEndProp },
+ { Value::COLOR, Value::NUMBER, kEndType },
+ { Identifier::INVERT, Identifier::NONE, Identifier::DOTTED,
+ Identifier::DASHED, Identifier::SOLID, Identifier::DOUBLE,
+ Identifier::GROOVE, Identifier::RIDGE, Identifier::INSET,
+ Identifier::OUTSET, Identifier::THIN, Identifier::MEDIUM,
+ Identifier::THICK, kEndIdent },
+ false, false, true, false,
+ }
+};
+
+struct PropertyValidationInfo {
+ std::set<Value::ValueType> valid_types;
+ std::set<Identifier::Ident> valid_idents;
+ bool accept_percent;
+ bool accept_no_unit;
+ bool accept_length;
+ bool accept_negative;
+};
+
+ValueValidator::ValueValidator() {
+ validation_info_.resize(Property::OTHER + 1);
+ for (int i = 0, n = arraysize(kValidPropInfo); i < n; ++i) {
+ std::set<Value::ValueType> t;
+ t.insert(Value::IDENT);
+ t.insert(Value::UNKNOWN);
+ t.insert(Value::DEFAULT);
+ for (int j = 0; kValidPropInfo[i].types[j] != -1; ++j) {
+ CHECK_LT(j, arraysize(kValidPropInfo[i].types));
+ t.insert(kValidPropInfo[i].types[j]);
+ }
+
+ std::set<Identifier::Ident> s;
+ s.insert(Identifier::INHERIT);
+ for (int j = 0; kValidPropInfo[i].idents[j] != -1; ++j) {
+ CHECK_LT(j, arraysize(kValidPropInfo[i].idents));
+ s.insert(kValidPropInfo[i].idents[j]);
+ }
+
+ for (int j = 0; kValidPropInfo[i].props[j] != -1; ++j) {
+ CHECK_LT(j, arraysize(kValidPropInfo[i].props));
+ PropertyValidationInfo* info = new PropertyValidationInfo;
+ validation_info_[kValidPropInfo[i].props[j]] = info;
+ info->valid_types = t;
+ info->valid_idents = s;
+ info->accept_percent = kValidPropInfo[i].accept_percent;
+ info->accept_no_unit = kValidPropInfo[i].accept_no_unit;
+ info->accept_length = kValidPropInfo[i].accept_length;
+ info->accept_negative = kValidPropInfo[i].accept_negative;
+ }
+ }
+}
+
+ValueValidator::~ValueValidator() {
+ STLDeleteElements(&validation_info_);
+}
+
+bool ValueValidator::IsValidValue(Property::Prop prop,
+ const Value& value,
+ bool quirks_mode) const {
+ if (!IsValidType(prop, value.GetLexicalUnitType()))
+ return false;
+ if (value.GetLexicalUnitType() == Value::IDENT &&
+ !IsValidIdentifier(prop, value.GetIdentifier().ident()))
+ return false;
+ if (value.GetLexicalUnitType() == Value::NUMBER &&
+ !IsValidNumber(prop, value, quirks_mode))
+ return false;
+ if (value.GetLexicalUnitType() == Value::RECT) {
+ const Values* params = value.GetParameters();
+ CHECK(params != NULL && params->size() == 4);
+ for (Values::const_iterator iter = params->begin();
+ iter < params->end(); ++iter) {
+ const Value* param = *iter;
+ if (param->GetLexicalUnitType() == Value::IDENT) {
+ if (!IsValidIdentifier(prop, param->GetIdentifier().ident()))
+ return false;
+ } else if (param->GetLexicalUnitType() == Value::NUMBER) {
+ if (!IsValidNumber(prop, *param, quirks_mode))
+ return false;
+ } else {
+ return false;
+ }
+ }
+ }
+ return true;
+}
+
+bool ValueValidator::IsValidType(Property::Prop prop,
+ Value::ValueType type) const {
+ const PropertyValidationInfo* info = validation_info_[prop];
+ // If we don't have information about prop, then all values are accepted.
+ return info == NULL || ContainsKey(info->valid_types, type);
+}
+
+bool ValueValidator::IsValidIdentifier(Property::Prop prop,
+ Identifier::Ident ident) const {
+ const PropertyValidationInfo* info = validation_info_[prop];
+ // If we don't have information about prop, then all values are accepted.
+ return info == NULL || ContainsKey(info->valid_idents, ident);
+}
+
+bool ValueValidator::IsValidNumber(Property::Prop prop, const Value& value,
+ bool quirks_mode) const {
+ const PropertyValidationInfo* info = validation_info_[prop];
+ // If we don't have information about prop, then all values are accepted.
+ if (info == NULL) return true;
+
+ switch (value.GetDimension()) {
+ case Value::OTHER:
+ return false;
+ case Value::DEG:
+ case Value::RAD:
+ case Value::GRAD:
+ case Value::HZ:
+ case Value::KHZ:
+ case Value::MS:
+ case Value::S:
+ return false; // unless we handle aural properties
+ case Value::PERCENT:
+ if (!info->accept_percent)
+ return false;
+ break;
+ case Value::NO_UNIT:
+ // We accept no-unit numbers if any of
+ // 1) accepted by the property,
+ // 2) 0 is always accepted,
+ // 3) in quirks mode.
+ if (!info->accept_no_unit && !quirks_mode &&
+ value.GetFloatValue() != 0.0)
+ return false;
+ break;
+ default:
+ if (!info->accept_length)
+ return false;
+ break;
+ }
+ if (value.GetFloatValue() < 0 && !info->accept_negative)
+ return false;
+ return true;
+}
+} // namespace
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/valuevalidator.h b/trunk/src/third_party/css_parser/src/webutil/css/valuevalidator.h
new file mode 100644
index 0000000..93bd228
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/valuevalidator.h
@@ -0,0 +1,71 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2007 Google Inc. All Rights Reserved.
+// Author: yian@google.com (Yi-An Huang)
+
+#ifndef WEBUTIL_CSS_VALUEVALIDATOR_H__
+#define WEBUTIL_CSS_VALUEVALIDATOR_H__
+
+#include <vector>
+
+#include "testing/production_stub/public/gunit_prod.h"
+#include "util/gtl/singleton.h"
+#include "webutil/css/property.h"
+#include "webutil/css/value.h"
+#include "webutil/css/identifier.h"
+
+namespace Css {
+
+struct PropertyValidationInfo;
+
+class ValueValidator {
+ public:
+ static ValueValidator* Get() { return Singleton<ValueValidator>::get(); }
+
+ // Is value is a valid value for property prop?
+ bool IsValidValue(Property::Prop prop, const Value& value,
+ bool quirks_mode) const;
+
+ // TODO(sligocki): Chromium's Singleton<> is not playing well with this class
+ // and so we've had to make the constructor/destructor public. Look into this.
+ // Default Constructor; only used for the Singleton, default instance
+ ValueValidator();
+ ~ValueValidator();
+
+private:
+ // Is type is a valid type for property prop?
+ bool IsValidType(Property::Prop prop, Value::ValueType type) const;
+
+ // Is ident is a valid identifier for property prop?
+ bool IsValidIdentifier(Property::Prop prop, Identifier::Ident ident) const;
+
+ // Is number valid for property prop?
+ bool IsValidNumber(Property::Prop prop, const Value& value,
+ bool quirks_mode) const;
+
+ std::vector<PropertyValidationInfo*> validation_info_;
+ friend class Singleton<ValueValidator>;
+ FRIEND_TEST(ValueValidatorTest, types);
+ FRIEND_TEST(ValueValidatorTest, identifiers);
+ FRIEND_TEST(ValueValidatorTest, numbers);
+
+ DISALLOW_COPY_AND_ASSIGN(ValueValidator);
+};
+
+} // namespace
+
+#endif // WEBUTIL_CSS_VALUEVALIDATOR_H__
diff --git a/trunk/src/third_party/css_parser/src/webutil/css/valuevalidator_test.cc b/trunk/src/third_party/css_parser/src/webutil/css/valuevalidator_test.cc
new file mode 100644
index 0000000..595f0c8
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/css/valuevalidator_test.cc
@@ -0,0 +1,105 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2008 Google Inc. All Rights Reserved.
+// Author: yian@google.com (Yi-An Huang)
+
+#include "webutil/css/valuevalidator.h"
+
+#include <string>
+
+#include "testing/base/public/googletest.h"
+#include "testing/base/public/gunit.h"
+#include "webutil/css/string.h"
+
+namespace Css {
+
+class ValueValidatorTest : public testing::Test {
+ protected:
+ const ValueValidator& validator() {
+ return *ValueValidator::Get();
+ }
+};
+
+TEST_F(ValueValidatorTest, types) {
+ EXPECT_TRUE(validator().IsValidType(Property::COLOR, Value::IDENT));
+ EXPECT_TRUE(validator().IsValidType(Property::COLOR, Value::DEFAULT));
+ EXPECT_TRUE(validator().IsValidType(Property::COLOR, Value::UNKNOWN));
+ EXPECT_TRUE(validator().IsValidType(Property::COLOR, Value::COLOR));
+ EXPECT_FALSE(validator().IsValidType(Property::COLOR, Value::STRING));
+ EXPECT_FALSE(validator().IsValidType(Property::COLOR, Value::URI));
+ EXPECT_FALSE(validator().IsValidType(Property::COLOR, Value::FUNCTION));
+}
+
+TEST_F(ValueValidatorTest, identifiers) {
+ EXPECT_TRUE(validator().IsValidIdentifier(Property::COLOR,
+ Identifier::INHERIT));
+ EXPECT_FALSE(validator().IsValidIdentifier(Property::COLOR,
+ Identifier::OTHER));
+ // font-family take all identifiers.
+ EXPECT_TRUE(validator().IsValidIdentifier(Property::FONT_FAMILY,
+ Identifier::SERIF));
+ EXPECT_TRUE(validator().IsValidIdentifier(Property::FONT_FAMILY,
+ Identifier::OTHER));
+ // FIXME(yian): Is this right?
+ EXPECT_FALSE(validator().IsValidIdentifier(Property::FONT_FAMILY,
+ Identifier::NORMAL));
+}
+
+TEST_F(ValueValidatorTest, numbers) {
+ // misc units
+ EXPECT_FALSE(validator().IsValidNumber(
+ Property::HEIGHT, Value(0, UTF8ToUnicodeText(string("unit"), false)),
+ false));
+ EXPECT_FALSE(validator().IsValidNumber(
+ Property::HEIGHT, Value(0, Value::RAD), false));
+ // percent
+ EXPECT_TRUE(validator().IsValidNumber(
+ Property::HEIGHT, Value(0, Value::PERCENT), false));
+ EXPECT_FALSE(validator().IsValidNumber(
+ Property::Z_INDEX, Value(0, Value::PERCENT), false));
+ // no-unit
+ EXPECT_FALSE(validator().IsValidNumber(
+ Property::HEIGHT, Value(1, Value::NO_UNIT), false));
+ EXPECT_TRUE(validator().IsValidNumber(
+ Property::HEIGHT, Value(0, Value::NO_UNIT), false));
+ EXPECT_TRUE(validator().IsValidNumber(
+ Property::HEIGHT, Value(1, Value::NO_UNIT), true));
+ EXPECT_TRUE(validator().IsValidNumber(
+ Property::Z_INDEX, Value(1, Value::NO_UNIT), false));
+ // lengths
+ EXPECT_TRUE(validator().IsValidNumber(
+ Property::HEIGHT, Value(1, Value::PX), false));
+ EXPECT_FALSE(validator().IsValidNumber(
+ Property::Z_INDEX, Value(1, Value::PX), false));
+ // negative
+ EXPECT_TRUE(validator().IsValidNumber(
+ Property::BOTTOM, Value(-1, Value::PX), false));
+ EXPECT_FALSE(validator().IsValidNumber(
+ Property::HEIGHT, Value(-1, Value::PX), false));
+}
+
+TEST_F(ValueValidatorTest, combined) {
+ Identifier transparent_identifier(Identifier::TRANSPARENT);
+ Value transparent(transparent_identifier);
+ Value some_string(Value::STRING, UTF8ToUnicodeText(string("string"), false));
+ EXPECT_TRUE(validator().IsValidValue(Property::BACKGROUND_COLOR,
+ transparent, false));
+ EXPECT_FALSE(validator().IsValidValue(Property::COLOR, transparent, false));
+ EXPECT_FALSE(validator().IsValidValue(Property::COLOR, some_string, false));
+}
+
+} // namespace
diff --git a/trunk/src/third_party/css_parser/src/webutil/html/htmlcolor.cc b/trunk/src/third_party/css_parser/src/webutil/html/htmlcolor.cc
new file mode 100644
index 0000000..1d623ea
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/html/htmlcolor.cc
@@ -0,0 +1,1001 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright (C) 2000 and onwards Google, Inc.
+//
+//
+// .cc for the HtmlColor class
+
+#include "webutil/html/htmlcolor.h"
+
+#include <cmath>
+
+#include "base/stringprintf.h"
+#include "strings/ascii_ctype.h"
+#include "strings/escaping.h"
+#include "strings/strutil.h"
+
+typedef struct RgbValue {
+ unsigned char r_;
+ unsigned char g_;
+ unsigned char b_;
+} RgbValue;
+
+// color table
+// when making change to known_color_values, please
+// also change the GetKnownColorValue function because
+// entire table is hardcoded into the function for efficiency
+static const RgbValue known_color_values[] = {
+/* 49 aliceblue */{240, 248, 255},
+/* 50 antiquewhite */{250, 235, 215},
+/* 51 aqua */{ 0, 255, 255},
+/* 52 aquamarine */{127, 255, 212},
+/* 53 azure */{240, 255, 255},
+/* 54 beige */{245, 245, 220},
+/* 55 bisque */{255, 228, 196},
+/* 56 black */{ 0, 0, 0},
+/* 57 blanchedalmond */{255, 235, 205},
+/* 58 blue */{ 0, 0, 255},
+/* 59 blueviolet */{138, 43, 226},
+/* 60 brown */{165, 42, 42},
+/* 61 burlywood */{222, 184, 135},
+/* 62 cadetblue */{ 95, 158, 160},
+/* 63 chartreuse */{127, 255, 0},
+/* 64 chocolate */{210, 105, 30},
+/* 65 coral */{255, 127, 80},
+/* 66 cornflowerblue */{100, 149, 237},
+/* 67 cornsilk */{255, 248, 220},
+/* 68 crimson */{220, 20, 60},
+/* 69 cyan */{ 0, 255, 255},
+/* 70 darkblue */{ 0, 0, 139},
+/* 71 darkcyan */{ 0, 139, 139},
+/* 72 darkgoldenrod */{184, 134, 11},
+/* 73 darkgray */{169, 169, 169},
+/* 74 darkgreen */{ 0, 100, 0},
+/* 75 darkgrey */{169, 169, 169},
+/* 76 darkkhaki */{189, 183, 107},
+/* 77 darkmagenta */{139, 0, 139},
+/* 78 darkolivegreen */{ 85, 107, 47},
+/* 79 darkorange */{255, 140, 0},
+/* 80 darkorchid */{153, 50, 204},
+/* 81 darkred */{139, 0, 0},
+/* 82 darksalmon */{233, 150, 122},
+/* 83 darkseagreen */{143, 188, 143},
+/* 84 darkslateblue */{ 72, 61, 139},
+/* 85 darkslategray */{ 47, 79, 79},
+/* 86 darkslategrey */{ 47, 79, 79},
+/* 87 darkturquoise */{ 0, 206, 209},
+/* 88 darkviolet */{148, 0, 211},
+/* 89 deeppink */{255, 20, 147},
+/* 90 deepskyblue */{ 0, 191, 255},
+/* 91 dimgray */{105, 105, 105},
+/* 92 dimgrey */{105, 105, 105},
+/* 93 dodgerblue */{ 30, 144, 255},
+/* 94 firebrick */{178, 34, 34},
+/* 95 floralwhite */{255, 250, 240},
+/* 96 forestgreen */{ 34, 139, 34},
+/* 97 fuchsia */{255, 0, 255},
+/* 98 gainsboro */{220, 220, 220},
+/* 99 ghostwhite */{248, 248, 255},
+/*100 gold */{255, 215, 0},
+/*101 goldenrod */{218, 165, 32},
+/*102 gray */{128, 128, 128},
+/*103 green */{ 0, 128, 0},
+/*104 grey */{128, 128, 128},
+/*105 greenyellow */{173, 255, 47},
+/*106 honeydew */{240, 255, 240},
+/*107 hotpink */{255, 105, 180},
+/*108 indianred */{205, 92, 92},
+/*109 indigo */{ 75, 0, 130},
+/*110 ivory */{255, 255, 240},
+/*111 khaki */{240, 230, 140},
+/*112 lavender */{230, 230, 250},
+/*113 lavenderblush */{255, 240, 245},
+/*114 lawngreen */{124, 252, 0},
+/*115 lemonchiffon */{255, 250, 205},
+/*116 lightblue */{173, 216, 230},
+/*117 lightcoral */{240, 128, 128},
+/*118 lightcyan */{224, 255, 255},
+/*119 lightgoldenrodyellow */{250, 250, 210},
+/*120 lightgray */{211, 211, 211},
+/*121 lightgreen */{144, 238, 144},
+/*122 lightgrey */{211, 211, 211},
+/*123 lightpink */{255, 182, 193},
+/*124 lightsalmon */{255, 160, 122},
+/*125 lightseagreen */{ 32, 178, 170},
+/*126 lightskyblue */{135, 206, 250},
+/*127 lightslategray */{119, 136, 153},
+/*128 lightslategrey */{119, 136, 153},
+/*129 lightsteelblue */{176, 196, 222},
+/*130 lightyellow */{255, 255, 224},
+/*131 lime */{ 0, 255, 0},
+/*132 limegreen */{ 50, 205, 50},
+/*133 linen */{250, 240, 230},
+/*134 magenta */{255, 0, 255},
+/*135 maroon */{128, 0, 0},
+/*136 mediumaquamarine */{102, 205, 170},
+/*137 mediumblue */{ 0, 0, 205},
+/*138 mediumorchid */{186, 85, 211},
+/*139 mediumpurple */{147, 112, 219},
+/*140 mediumseagreen */{ 60, 179, 113},
+/*141 mediumslateblue */{123, 104, 238},
+/*142 mediumspringgreen */{ 0, 250, 154},
+/*143 mediumturquoise */{ 72, 209, 204},
+/*144 mediumvioletred */{199, 21, 133},
+/*145 midnightblue */{ 25, 25, 112},
+/*146 mintcream */{245, 255, 250},
+/*147 mistyrose */{255, 228, 225},
+/*148 moccasin */{255, 228, 181},
+/*149 navajowhite */{255, 222, 173},
+/*150 navy */{ 0, 0, 128},
+/*151 oldlace */{253, 245, 230},
+/*152 olive */{128, 128, 0},
+/*153 olivedrab */{107, 142, 35},
+/*154 orange */{255, 165, 0},
+/*155 orangered */{255, 69, 0},
+/*156 orchid */{218, 112, 214},
+/*157 palegoldenrod */{238, 232, 170},
+/*158 palegreen */{152, 251, 152},
+/*159 paleturquoise */{175, 238, 238},
+/*160 palevioletred */{219, 112, 147},
+/*161 papayawhip */{255, 239, 213},
+/*162 peachpuff */{255, 218, 185},
+/*163 peru */{205, 133, 63},
+/*164 pink */{255, 192, 203},
+/*165 plum */{221, 160, 221},
+/*166 powderblue */{176, 224, 230},
+/*167 purple */{128, 0, 128},
+/*168 red */{255, 0, 0},
+/*169 rosybrown */{188, 143, 143},
+/*170 royalblue */{ 65, 105, 225},
+/*171 saddlebrown */{139, 69, 19},
+/*172 salmon */{250, 128, 114},
+/*173 sandybrown */{244, 164, 96},
+/*174 seagreen */{ 46, 139, 87},
+/*175 seashell */{255, 245, 238},
+/*176 sienna */{160, 82, 45},
+/*177 silver */{192, 192, 192},
+/*178 skyblue */{135, 206, 235},
+/*179 slateblue */{106, 90, 205},
+/*180 slategray */{112, 128, 144},
+/*181 slategrey */{112, 128, 144},
+/*182 snow */{255, 250, 250},
+/*183 springgreen */{ 0, 255, 127},
+/*184 steelblue */{ 70, 130, 180},
+/*185 tan */{210, 180, 140},
+/*186 teal */{ 0, 128, 128},
+/*187 thistle */{216, 191, 216},
+/*188 tomato */{255, 99, 71},
+/*189 turquoise */{ 64, 224, 208},
+/*190 violet */{238, 130, 238},
+/*191 wheat */{245, 222, 179},
+/*192 white */{255, 255, 255},
+/*193 whitesmoke */{245, 245, 245},
+/*194 yellow */{255, 255, 0},
+/*195 yellowgreen */{154, 205, 50},
+};
+
+// here the entiry table is hardcoded into the function
+// mainly because of consideration of efficiency
+static const RgbValue * GetKnownColorValue(const char *colorstr) {
+ switch(ascii_tolower(colorstr[0])) {
+ case 'a':
+ switch(ascii_tolower(colorstr[1])) {
+ case 'l':
+ if (!strcasecmp("aliceblue", colorstr)) {
+ return &known_color_values[0];
+ }
+ return NULL;
+ case 'n':
+ if (!strcasecmp("antiquewhite", colorstr)) {
+ return &known_color_values[1];
+ }
+ return NULL;
+ case 'q':
+ if (!strcasecmp("aqua", colorstr)) {
+ return &known_color_values[2];
+ } else if (!strcasecmp("aquamarine", colorstr)) {
+ return &known_color_values[3];
+ }
+ return NULL;
+ case 'z':
+ if (!strcasecmp("azure", colorstr)) {
+ return &known_color_values[4];
+ }
+ return NULL;
+ }
+ return NULL;
+ case 'b':
+ switch(ascii_tolower(colorstr[1])) {
+ case 'e':
+ if (!strcasecmp("beige", colorstr)) {
+ return &known_color_values[5];
+ }
+ return NULL;
+ case 'i':
+ if (!strcasecmp("bisque", colorstr)) {
+ return &known_color_values[6];
+ }
+ return NULL;
+ case 'l':
+ if (!strcasecmp("black", colorstr)) {
+ return &known_color_values[7];
+ } else if (!strcasecmp("blanchedalmond", colorstr)) {
+ return &known_color_values[8];
+ } else if (!strcasecmp("blue", colorstr)) {
+ return &known_color_values[9];
+ } else if (!strcasecmp("blueviolet", colorstr)) {
+ return &known_color_values[10];
+ }
+ return NULL;
+ case 'r':
+ if (!strcasecmp("brown", colorstr)) {
+ return &known_color_values[11];
+ }
+ return NULL;
+ case 'u':
+ if (!strcasecmp("burlywood", colorstr)) {
+ return &known_color_values[12];
+ }
+ return NULL;
+ }
+ return NULL;
+ case 'c':
+ switch(ascii_tolower(colorstr[1])) {
+ case 'a':
+ if (!strcasecmp("cadetblue", colorstr)) {
+ return &known_color_values[13];
+ }
+ return NULL;
+ case 'h':
+ if (!strcasecmp("chartreuse", colorstr)) {
+ return &known_color_values[14];
+ } else if (!strcasecmp("chocolate", colorstr)) {
+ return &known_color_values[15];
+ }
+ return NULL;
+ case 'o':
+ if (!strcasecmp("coral", colorstr)) {
+ return &known_color_values[16];
+ } else if (!strcasecmp("cornflowerblue", colorstr)) {
+ return &known_color_values[17];
+ } else if (!strcasecmp("cornsilk", colorstr)) {
+ return &known_color_values[18];
+ }
+ return NULL;
+ case 'r':
+ if (!strcasecmp("crimson", colorstr)) {
+ return &known_color_values[19];
+ }
+ return NULL;
+ case 'y':
+ if (!strcasecmp("cyan", colorstr)) {
+ return &known_color_values[20];
+ }
+ return NULL;
+ }
+
+ return NULL;
+ case 'd':
+ switch(ascii_tolower(colorstr[1])) {
+ case 'a':
+ if (!strcasecmp("darkblue", colorstr)) {
+ return &known_color_values[21];
+ } else if (!strcasecmp("darkcyan", colorstr)) {
+ return &known_color_values[22];
+ } else if (!strcasecmp("darkgoldenrod", colorstr)) {
+ return &known_color_values[23];
+ } else if (!strcasecmp("darkgray", colorstr)) {
+ return &known_color_values[24];
+ } else if (!strcasecmp("darkgreen", colorstr)) {
+ return &known_color_values[25];
+ } else if (!strcasecmp("darkgrey", colorstr)) {
+ return &known_color_values[26];
+ } else if (!strcasecmp("darkkhaki", colorstr)) {
+ return &known_color_values[27];
+ } else if (!strcasecmp("darkmagenta", colorstr)) {
+ return &known_color_values[28];
+ } else if (!strcasecmp("darkolivegreen", colorstr)) {
+ return &known_color_values[29];
+ } else if (!strcasecmp("darkorange", colorstr)) {
+ return &known_color_values[30];
+ } else if (!strcasecmp("darkorchid", colorstr)) {
+ return &known_color_values[31];
+ } else if (!strcasecmp("darkred", colorstr)) {
+ return &known_color_values[32];
+ } else if (!strcasecmp("darksalmon", colorstr)) {
+ return &known_color_values[33];
+ } else if (!strcasecmp("darkseagreen", colorstr)) {
+ return &known_color_values[34];
+ } else if (!strcasecmp("darkslateblue", colorstr)) {
+ return &known_color_values[35];
+ } else if (!strcasecmp("darkslategray", colorstr)) {
+ return &known_color_values[36];
+ } else if (!strcasecmp("darkslategrey", colorstr)) {
+ return &known_color_values[37];
+ } else if (!strcasecmp("darkturquoise", colorstr)) {
+ return &known_color_values[38];
+ } else if (!strcasecmp("darkviolet", colorstr)) {
+ return &known_color_values[39];
+ }
+ return NULL;
+ case 'e':
+ if (!strcasecmp("deeppink", colorstr)) {
+ return &known_color_values[40];
+ } else if (!strcasecmp("deepskyblue", colorstr)) {
+ return &known_color_values[41];
+ }
+ return NULL;
+ case 'i':
+ if (!strcasecmp("dimgray", colorstr)) {
+ return &known_color_values[42];
+ } else if (!strcasecmp("dimgrey", colorstr)) {
+ return &known_color_values[43];
+ }
+ return NULL;
+ case 'o':
+ if (!strcasecmp("dodgerblue", colorstr)) {
+ return &known_color_values[44];
+ }
+ return NULL;
+ }
+ return NULL;
+ case 'f':
+ switch(ascii_tolower(colorstr[1])) {
+ case 'i':
+ if (!strcasecmp("firebrick", colorstr)) {
+ return &known_color_values[45];
+ }
+ return NULL;
+ case 'l':
+ if (!strcasecmp("floralwhite", colorstr)) {
+ return &known_color_values[46];
+ }
+ return NULL;
+ case 'o':
+ if (!strcasecmp("forestgreen", colorstr)) {
+ return &known_color_values[47];
+ }
+ return NULL;
+ case 'u':
+ if (!strcasecmp("fuchsia", colorstr)) {
+ return &known_color_values[48];
+ }
+ return NULL;
+ }
+ return NULL;
+ case 'g':
+ switch(ascii_tolower(colorstr[1])) {
+ case 'a':
+ if (!strcasecmp("gainsboro", colorstr)) {
+ return &known_color_values[49];
+ }
+ return NULL;
+ case 'h':
+ if (!strcasecmp("ghostwhite", colorstr)) {
+ return &known_color_values[50];
+ }
+ return NULL;
+ case 'o':
+ if (!strcasecmp("gold", colorstr)) {
+ return &known_color_values[51];
+ } else if (!strcasecmp("goldenrod", colorstr)) {
+ return &known_color_values[52];
+ }
+ return NULL;
+ case 'r':
+ if (!strcasecmp("gray", colorstr)) {
+ return &known_color_values[53];
+ } else if (!strcasecmp("green", colorstr)) {
+ return &known_color_values[54];
+ } else if (!strcasecmp("grey", colorstr)) {
+ return &known_color_values[55];
+ } else if (!strcasecmp("greenyellow", colorstr)) {
+ return &known_color_values[56];
+ }
+ return NULL;
+ }
+ return NULL;
+ case 'h':
+ if (!strcasecmp("honeydew", colorstr)) {
+ return &known_color_values[57];
+ } else if (!strcasecmp("hotpink", colorstr)) {
+ return &known_color_values[58];
+ }
+ return NULL;
+ case 'i':
+ if (!strcasecmp("indianred", colorstr)) {
+ return &known_color_values[59];
+ } else if (!strcasecmp("indigo", colorstr)) {
+ return &known_color_values[60];
+ } else if (!strcasecmp("ivory", colorstr)) {
+ return &known_color_values[61];
+ }
+ return NULL;
+ case 'k':
+ if (!strcasecmp("khaki", colorstr)) {
+ return &known_color_values[62];
+ }
+ return NULL;
+ case 'l':
+ switch(ascii_tolower(colorstr[1])) {
+ case 'a':
+ if (!strcasecmp("lavender", colorstr)) {
+ return &known_color_values[63];
+ } else if (!strcasecmp("lavenderblush", colorstr)) {
+ return &known_color_values[64];
+ } else if (!strcasecmp("lawngreen", colorstr)) {
+ return &known_color_values[65];
+ }
+ return NULL;
+ case 'e':
+ if (!strcasecmp("lemonchiffon", colorstr)) {
+ return &known_color_values[66];
+ }
+ return NULL;
+ case 'i':
+ if (!strcasecmp("lightblue", colorstr)) {
+ return &known_color_values[67];
+ } else if (!strcasecmp("lightcoral", colorstr)) {
+ return &known_color_values[68];
+ } else if (!strcasecmp("lightcyan", colorstr)) {
+ return &known_color_values[69];
+ } else if (!strcasecmp("lightgoldenrodyellow", colorstr)) {
+ return &known_color_values[70];
+ } else if (!strcasecmp("lightgray", colorstr)) {
+ return &known_color_values[71];
+ } else if (!strcasecmp("lightgreen", colorstr)) {
+ return &known_color_values[72];
+ } else if (!strcasecmp("lightgrey", colorstr)) {
+ return &known_color_values[73];
+ } else if (!strcasecmp("lightpink", colorstr)) {
+ return &known_color_values[74];
+ } else if (!strcasecmp("lightsalmon", colorstr)) {
+ return &known_color_values[75];
+ } else if (!strcasecmp("lightseagreen", colorstr)) {
+ return &known_color_values[76];
+ } else if (!strcasecmp("lightskyblue", colorstr)) {
+ return &known_color_values[77];
+ } else if (!strcasecmp("lightslategray", colorstr)) {
+ return &known_color_values[78];
+ } else if (!strcasecmp("lightslategrey", colorstr)) {
+ return &known_color_values[79];
+ } else if (!strcasecmp("lightsteelblue", colorstr)) {
+ return &known_color_values[80];
+ } else if (!strcasecmp("lightyellow", colorstr)) {
+ return &known_color_values[81];
+ } else if (!strcasecmp("lime", colorstr)) {
+ return &known_color_values[82];
+ } else if (!strcasecmp("limegreen", colorstr)) {
+ return &known_color_values[83];
+ } else if (!strcasecmp("linen", colorstr)) {
+ return &known_color_values[84];
+ }
+ return NULL;
+ }
+ return NULL;
+ case 'm':
+ switch(ascii_tolower(colorstr[1])) {
+ case 'a':
+ if (!strcasecmp("magenta", colorstr)) {
+ return &known_color_values[85];
+ } else if (!strcasecmp("maroon", colorstr)) {
+ return &known_color_values[86];
+ }
+ return NULL;
+ case 'e':
+ if (!strcasecmp("mediumaquamarine", colorstr)) {
+ return &known_color_values[87];
+ } else if (!strcasecmp("mediumblue", colorstr)) {
+ return &known_color_values[88];
+ } else if (!strcasecmp("mediumorchid", colorstr)) {
+ return &known_color_values[89];
+ } else if (!strcasecmp("mediumpurple", colorstr)) {
+ return &known_color_values[90];
+ } else if (!strcasecmp("mediumseagreen", colorstr)) {
+ return &known_color_values[91];
+ } else if (!strcasecmp("mediumslateblue", colorstr)) {
+ return &known_color_values[92];
+ } else if (!strcasecmp("mediumspringgreen", colorstr)) {
+ return &known_color_values[93];
+ } else if (!strcasecmp("mediumturquoise", colorstr)) {
+ return &known_color_values[94];
+ } else if (!strcasecmp("mediumvioletred", colorstr)) {
+ return &known_color_values[95];
+ }
+ return NULL;
+ case 'i':
+ if (!strcasecmp("midnightblue", colorstr)) {
+ return &known_color_values[96];
+ } else if (!strcasecmp("mintcream", colorstr)) {
+ return &known_color_values[97];
+ } else if (!strcasecmp("mistyrose", colorstr)) {
+ return &known_color_values[98];
+ }
+ return NULL;
+ case 'o':
+ if (!strcasecmp("moccasin", colorstr)) {
+ return &known_color_values[99];
+ }
+ return NULL;
+ }
+ return NULL;
+ case 'n':
+ if (!strcasecmp("navajowhite", colorstr)) {
+ return &known_color_values[100];
+ } else if (!strcasecmp("navy", colorstr)) {
+ return &known_color_values[101];
+ }
+ return NULL;
+ case 'o':
+ switch(ascii_tolower(colorstr[1])) {
+ case 'l':
+ if (!strcasecmp("oldlace", colorstr)) {
+ return &known_color_values[102];
+ } else if (!strcasecmp("olive", colorstr)) {
+ return &known_color_values[103];
+ } else if (!strcasecmp("olivedrab", colorstr)) {
+ return &known_color_values[104];
+ }
+ return NULL;
+ case 'r':
+ if (!strcasecmp("orange", colorstr)) {
+ return &known_color_values[105];
+ } else if (!strcasecmp("orangered", colorstr)) {
+ return &known_color_values[106];
+ } else if (!strcasecmp("orchid", colorstr)) {
+ return &known_color_values[107];
+ }
+ return NULL;
+ }
+ return NULL;
+ case 'p':
+ switch(ascii_tolower(colorstr[1])) {
+ case 'a':
+ if (!strcasecmp("palegoldenrod", colorstr)) {
+ return &known_color_values[108];
+ } else if (!strcasecmp("palegreen", colorstr)) {
+ return &known_color_values[109];
+ } else if (!strcasecmp("paleturquoise", colorstr)) {
+ return &known_color_values[110];
+ } else if (!strcasecmp("palevioletred", colorstr)) {
+ return &known_color_values[111];
+ } else if (!strcasecmp("papayawhip", colorstr)) {
+ return &known_color_values[112];
+ }
+ return NULL;
+ case 'e':
+ if (!strcasecmp("peachpuff", colorstr)) {
+ return &known_color_values[113];
+ } else if (!strcasecmp("peru", colorstr)) {
+ return &known_color_values[114];
+ }
+ return NULL;
+ case 'i':
+ if (!strcasecmp("pink", colorstr)) {
+ return &known_color_values[115];
+ }
+ return NULL;
+ case 'l':
+ if (!strcasecmp("plum", colorstr)) {
+ return &known_color_values[116];
+ }
+ return NULL;
+ case 'o':
+ if (!strcasecmp("powderblue", colorstr)) {
+ return &known_color_values[117];
+ }
+ return NULL;
+ case 'u':
+ if (!strcasecmp("purple", colorstr)) {
+ return &known_color_values[118];
+ }
+ return NULL;
+ }
+ return NULL;
+ case 'r':
+ if (!strcasecmp("red", colorstr)) {
+ return &known_color_values[119];
+ } else if (!strcasecmp("rosybrown", colorstr)) {
+ return &known_color_values[120];
+ } else if (!strcasecmp("royalblue", colorstr)) {
+ return &known_color_values[121];
+ }
+ return NULL;
+ case 's':
+ switch(ascii_tolower(colorstr[1])) {
+ case 'a':
+ if (!strcasecmp("saddlebrown", colorstr)) {
+ return &known_color_values[122];
+ } else if (!strcasecmp("salmon", colorstr)) {
+ return &known_color_values[123];
+ } else if (!strcasecmp("sandybrown", colorstr)) {
+ return &known_color_values[124];
+ }
+ return NULL;
+ case 'e':
+ if (!strcasecmp("seagreen", colorstr)) {
+ return &known_color_values[125];
+ } else if (!strcasecmp("seashell", colorstr)) {
+ return &known_color_values[126];
+ }
+ return NULL;
+ case 'i':
+ if (!strcasecmp("sienna", colorstr)) {
+ return &known_color_values[127];
+ } else if (!strcasecmp("silver", colorstr)) {
+ return &known_color_values[128];
+ }
+ return NULL;
+ case 'k':
+ if (!strcasecmp("skyblue", colorstr)) {
+ return &known_color_values[129];
+ }
+ return NULL;
+ case 'l':
+ if (!strcasecmp("slateblue", colorstr)) {
+ return &known_color_values[130];
+ } else if (!strcasecmp("slategray", colorstr)) {
+ return &known_color_values[131];
+ } else if (!strcasecmp("slategrey", colorstr)) {
+ return &known_color_values[132];
+ }
+ return NULL;
+ case 'n':
+ if (!strcasecmp("snow", colorstr)) {
+ return &known_color_values[133];
+ }
+ return NULL;
+ case 'p':
+ if (!strcasecmp("springgreen", colorstr)) {
+ return &known_color_values[134];
+ }
+ return NULL;
+ case 't':
+ if (!strcasecmp("steelblue", colorstr)) {
+ return &known_color_values[135];
+ }
+ return NULL;
+ }
+ return NULL;
+ case 't':
+ switch(ascii_tolower(colorstr[1])) {
+ case 'a':
+ if (!strcasecmp("tan", colorstr)) {
+ return &known_color_values[136];
+ }
+ return NULL;
+ case 'e':
+ if (!strcasecmp("teal", colorstr)) {
+ return &known_color_values[137];
+ }
+ return NULL;
+ case 'h':
+ if (!strcasecmp("thistle", colorstr)) {
+ return &known_color_values[138];
+ }
+ return NULL;
+ case 'o':
+ if (!strcasecmp("tomato", colorstr)) {
+ return &known_color_values[139];
+ }
+ return NULL;
+ case 'u':
+ if (!strcasecmp("turquoise", colorstr)) {
+ return &known_color_values[140];
+ }
+ return NULL;
+ }
+ return NULL;
+ case 'v':
+ if (!strcasecmp("violet", colorstr)) {
+ return &known_color_values[141];
+ }
+ return NULL;
+ case 'w':
+ if (!strcasecmp("wheat", colorstr)) {
+ return &known_color_values[142];
+ } else if (!strcasecmp("white", colorstr)) {
+ return &known_color_values[143];
+ } else if (!strcasecmp("whitesmoke", colorstr)) {
+ return &known_color_values[144];
+ }
+ return NULL;
+ case 'y':
+ if (!strcasecmp("yellow", colorstr)) {
+ return &known_color_values[145];
+ } else if (!strcasecmp("yellowgreen", colorstr)) {
+ return &known_color_values[146];
+ }
+ return NULL;
+ };
+
+ return NULL;
+}
+
+const unsigned char HtmlColor::kGoodColorValue;
+const unsigned char HtmlColor::kBadColorName;
+const unsigned char HtmlColor::kBadColorHex;
+
+static inline int TwoXDigitsToNum(const char *xstr) {
+ return (hex_digit_to_int(xstr[0])*16 + hex_digit_to_int(xstr[1]));
+}
+
+void HtmlColor::SetValueFromHexStr(const char *hexstr) {
+ int hexstr_len = strlen(hexstr);
+ const char* finalstr = hexstr;
+ char hexbuf[7];
+
+ if (hexstr_len == 3) {
+ for (int i = 0; i < 3; i++) {
+ if (!ascii_isxdigit(hexstr[i])) {
+ SetBadHexValue();
+ return;
+ }
+ hexbuf[2 * i] = hexstr[i];
+ hexbuf[2 * i + 1] = hexstr[i];
+ }
+ hexbuf[6] = '\0';
+ finalstr = hexbuf;
+ } else if (hexstr_len == 6) {
+ for (int i = 0; i < 6; i++) {
+ if (!ascii_isxdigit(hexstr[i])) {
+ SetBadHexValue();
+ return;
+ }
+ }
+ } else {
+ SetBadHexValue();
+ return;
+ }
+
+ r_ = static_cast<unsigned char>(TwoXDigitsToNum(finalstr));
+ g_ = static_cast<unsigned char>(TwoXDigitsToNum(finalstr + 2));
+ b_ = static_cast<unsigned char>(TwoXDigitsToNum(finalstr + 4));
+ is_bad_value_ = kGoodColorValue;
+}
+
+HtmlColor::HtmlColor(const string& str) {
+ SetValueFromStr(str.c_str());
+}
+
+HtmlColor::HtmlColor(const char *str, int colorstrlen) {
+ string tmp(str,colorstrlen);
+ SetValueFromStr(tmp.c_str());
+}
+
+void HtmlColor::SetValueFromStr(const char* colorstr) {
+ if(colorstr[0]=='#') { // rgb value
+ SetValueFromHexStr(colorstr + 1);
+ } else {
+ SetValueFromName(colorstr);
+ if (!IsDefined() && strlen(colorstr) == 6) {
+ SetValueFromHexStr(colorstr);
+ if (!IsDefined())
+ SetBadNameValue();
+ }
+ }
+}
+
+void HtmlColor::SetValueFromName(const char* str) {
+ const RgbValue *p_rgb = GetKnownColorValue(str);
+ if (p_rgb) {
+ r_ = p_rgb->r_;
+ g_ = p_rgb->g_;
+ b_ = p_rgb->b_;
+ is_bad_value_ = kGoodColorValue;
+ } else {
+ SetBadNameValue();
+ }
+}
+
+void HtmlColor::SetValueFromRGB(unsigned char r, unsigned char g,
+ unsigned char b) {
+ r_ = r;
+ g_ = g;
+ b_ = b;
+ is_bad_value_ = kGoodColorValue;
+}
+
+HtmlColor::HtmlColor(unsigned char r, unsigned char g, unsigned char b) {
+ SetValueFromRGB(r, g, b);
+}
+
+static const float kLumR = 0.30;
+static const float kLumG = 0.59;
+static const float kLumB = 0.11;
+
+// Converts from RGB to HSL. This is an algorithm derived from Fundamentals of
+// Interactive Computer Graphics by Foley and van Dam (1982). A (slightly)
+// modified formula can be found at
+// http://en.wikipedia.org/wiki/HSL_color_space
+// We adopt the notation that HSL values are expressed in the range of 0 to 1.
+// Specifically, H is in [0, 1), while S and L are in [0, 1].
+// (Another popular scheme normalizes HSL to [0, 360), [0, 100], and [0, 100],
+// respectively).
+static void RGBtoHSL(const HtmlColor& rgb, double* h, double* s, double* l) {
+ int r = rgb.r();
+ int g = rgb.g();
+ int b = rgb.b();
+ int max_v = (r < g) ? ((g < b) ? b : g) : ((b < r) ? r : b);
+ int min_v = (r < g) ? ((r < b) ? r : b) : ((b < g) ? b : g);
+ int sum = max_v + min_v;
+ double delta = static_cast<double>(max_v - min_v);
+
+ double dR = (max_v - r) / delta;
+ double dG = (max_v - g) / delta;
+ double dB = (max_v - b) / delta;
+
+ if (min_v == max_v)
+ *h = 0.0;
+ else if (r == max_v)
+ *h = (dB - dG) / 6.0;
+ else if (g == max_v)
+ *h = (2.0 + dR - dB) / 6.0;
+ else
+ *h = (4.0 + dG - dR) / 6.0;
+
+ if (*h < 0.0)
+ *h += 1.0;
+ if (*h >= 1.0)
+ *h -= 1.0;
+
+ *l = 0.5 * sum / 255.0;
+
+ if (max_v == 0 || min_v == 255)
+ *s = 0.0;
+ else if (sum <= 255)
+ *s = delta / sum;
+ else
+ *s = delta / (2 * 255 - sum);
+}
+
+// Calculates the Euclidean distance between two color vectors on a HSL sphere.
+// A demo of the sphere can also be found at:
+// http://en.wikipedia.org/wiki/HSL_color_space
+// In short, a vector for color (H, S, L) in this system can be expressed as
+// (S*L'*cos(2*PI*H), S*L'*sin(2*PI*H), L), where L' = abs(L - 0.5).
+// And we simply calculate the l-2 distance using these coordinates.
+static double HSLDistance(double h1, double s1, double l1, double h2,
+ double s2, double l2) {
+ double sl1, sl2;
+ if (l1 <= 0.5)
+ sl1 = s1 * l1;
+ else
+ sl1 = s1 * (1.0 - l1);
+
+ if (l2 <= 0.5)
+ sl2 = s2 * l2;
+ else
+ sl2 = s2 * (1.0 - l2);
+
+ double dH = (h1 - h2) * 2.0 * M_PI;
+ return (l1 - l2) * (l1 - l2) +
+ sl1 * sl1 + sl2 * sl2 - 2 * sl1 * sl2 * cos(dH);
+}
+
+bool HtmlColor::IsSimilarInHSL(const HtmlColor &color, double level) const {
+ double h1, s1, l1, h2, s2, l2;
+ RGBtoHSL(*this, &h1, &s1, &l1);
+ RGBtoHSL(color, &h2, &s2, &l2);
+ return HSLDistance(h1, s1, l1, h2, s2, l2) <= level;
+}
+
+// Calculate the luminance of the color
+// Luminance is an integer value from 0 - 255, which
+// represents the gray level that most closely corresponds
+// to the perceived brightness of the color, based on
+// the way the human eye sees color. The weights in
+// the function below are pretty standard. See
+// http://www.google.com/search?q=rgb+luminance+formula
+int HtmlColor::Luminance() const {
+ if (!IsDefined())
+ return 0;
+
+ float luminance = (kLumR * static_cast<float>(r_)) +
+ (kLumG * static_cast<float>(g_)) +
+ (kLumB * static_cast<float>(b_));
+ return static_cast<int>(luminance);
+}
+
+void HtmlColor::Lighten(float factor) {
+ HtmlColor white("ffffff", 6);
+ BlendWithColor(1.0-factor, white);
+}
+
+void HtmlColor::Darken(float factor) {
+ HtmlColor black("000000", 6);
+ BlendWithColor(1.0-factor, black);
+}
+
+void HtmlColor::Desaturate(float factor) {
+ if (!IsDefined() || factor < 0 || factor > 1)
+ return;
+
+ unsigned char lum = static_cast<unsigned char>(Luminance());
+ HtmlColor gray(lum, lum, lum);
+ BlendWithColor(1.0-factor, gray);
+}
+
+void HtmlColor::BlendWithColor(float factor, const HtmlColor& c) {
+ if (!IsDefined() || factor < 0 || factor > 1)
+ return;
+
+ r_ = static_cast<unsigned char>(factor * r_ + (1-factor) * c.r_);
+ g_ = static_cast<unsigned char>(factor * g_ + (1-factor) * c.g_);
+ b_ = static_cast<unsigned char>(factor * b_ + (1-factor) * c.b_);
+}
+
+// Return the color as a string for use in HTML
+// This isn't the most efficient function. If you're using it a lot,
+// you might want to rewrite it.
+string HtmlColor::ToString() const {
+ return StringPrintf("#%02x%02x%02x", r_, g_, b_);
+}
+
+
+//
+// == HtmlColorUtils ==
+//
+string HtmlColorUtils::MaybeConvertToCssShorthand(const char* orig_color) {
+ HtmlColor color(orig_color);
+ if ( !color.IsDefined() )
+ return orig_color;
+
+ string shorthand = MaybeConvertToCssShorthand(color);
+ if (shorthand.size() < strlen(orig_color)) {
+ return shorthand;
+ } else {
+ return orig_color;
+ }
+}
+
+string HtmlColorUtils::MaybeConvertToCssShorthand(const HtmlColor& color) {
+ // There are 16 color names which are supported by all known CSS compliant
+ // browsers. Of these 16, 9 are shorter than their hex equivalents. For
+ // this reason, we prefer to use the shorter color names in order to save
+ // bytes.
+ switch (color.rgb()) {
+ case 0x000080:
+ return "navy";
+ case 0x008000:
+ return "green";
+ case 0x008080:
+ return "teal";
+ case 0x800000:
+ return "maroon";
+ case 0x800080:
+ return "purple";
+ case 0x808000:
+ return "olive";
+ case 0x808080:
+ return "gray";
+ case 0xC0C0C0:
+ return "silver";
+ case 0xFF0000:
+ return "red";
+ }
+
+ if ( (color.r() >> 4) != (color.r() & 0xF) ||
+ (color.g() >> 4) != (color.g() & 0xF) ||
+ (color.b() >> 4) != (color.b() & 0xF) )
+ return color.ToString();
+
+ return StringPrintf("#%01x%01x%01x",
+ color.r() & 0xF,
+ color.g() & 0xF,
+ color.b() & 0xF);
+}
diff --git a/trunk/src/third_party/css_parser/src/webutil/html/htmlcolor.h b/trunk/src/third_party/css_parser/src/webutil/html/htmlcolor.h
new file mode 100644
index 0000000..46c0885
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/html/htmlcolor.h
@@ -0,0 +1,155 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright (C) 2000 and onwards Google, Inc.
+//
+//
+// .h for the HtmlColor class
+// HtmlColor provides 'IsSimilar' for comparing HTML colors
+// of different representations ( "#xxxxxx" or color names such as "white").
+// Before doing comparison, check 'IsDefined' first, it's because
+// that not all given HTML color strings are valid.
+
+#ifndef _HTML_COLOR_H_
+#define _HTML_COLOR_H_
+
+#include <string>
+#include "string_using.h"
+#include <stdlib.h>
+
+class HtmlColor {
+ private:
+ unsigned char r_;
+ unsigned char g_;
+ unsigned char b_;
+
+ // a colorstr is well defined if it is "#xxxxxx" ('x' is a hexdigit)
+ // or it's a known color name such as "black".
+ // 0: the RGB value is good!
+ // 1: bad (name) value, caused by bad color name
+ // 2: bad (hex) value, caused by bad hex string value
+ // -- with the browser (Netscape Communicator 4.75, linux-2.2.14,
+ // it shows that the color displayed is sometimes 'black' under case '2'.
+ unsigned char is_bad_value_;
+ static const unsigned char kGoodColorValue = 0x00;
+ static const unsigned char kBadColorName = 0x01;
+ static const unsigned char kBadColorHex = 0x02;
+
+ void SetBadNameValue() {
+ r_ = g_ = b_ = 0x00;
+ is_bad_value_ = kBadColorName;
+ }
+ void SetBadHexValue() {
+ r_ = g_ = b_ = 0x00;
+ is_bad_value_ = kBadColorHex;
+ }
+ void SetDefaultValue() {
+ r_ = g_ = b_ = 0x00;
+ is_bad_value_ = kGoodColorValue;
+ }
+
+ public:
+ enum TolerateLevel { EXACTLY_SAME=0,
+ HIGHLY_SIMILAR=5,
+ SIMILAR=10 };
+
+ // These methods also accept a CSS shorthand string "#xyz" for convenience.
+ // "#xyz" is expanded to "#xxyyzz" before processing.
+ explicit HtmlColor(const string& colorstr);
+ explicit HtmlColor(const char *colorstr, int colorstrlen);
+ explicit HtmlColor(unsigned char r, unsigned char g, unsigned char b);
+
+ bool IsDefined() const { return (is_bad_value_==0); }
+
+ bool IsSimilar(const HtmlColor &color, int level) const {
+ if (!IsDefined() || !color.IsDefined())
+ return false;
+
+ if ( (abs(static_cast<int>(r_) - static_cast<int>(color.r_)) <= level) &&
+ (abs(static_cast<int>(g_) - static_cast<int>(color.g_)) <= level) &&
+ (abs(static_cast<int>(b_) - static_cast<int>(color.b_)) <= level)
+ )
+ return true;
+ return false;
+ }
+
+ // Compares color similarity in HSL (Hue, Saturation, Lightness) space.
+ // This is assumed to be more accurate based on human perception.
+ // Note the difference level is a float number and it may vary from 0.0 to
+ // 1.0, inclusive. A suggested value for level is 0.02.
+ // WARNING: this is more expensive than IsSimilar() as it involves float
+ // arithmetic and cosine operations.
+ // TODO(yian): may need to disintegrate it into a separate HSL class.
+ bool IsSimilarInHSL(const HtmlColor &color, double level) const;
+
+ // return the luminance (0-255) of the color.
+ // this corresponds to a human's perception of the color's brightness
+ int Luminance() const;
+
+ // Lighten or darken the color by a given factor (between 0 and 1)
+ // Lightening with factor 1.0 => white
+ // Darkening with factor 1.0 => black
+ void Lighten(float factor);
+ void Darken(float factor);
+
+ // Desaturate the color (0.0 = no change, 1.0 = equivalent shade of gray)
+ void Desaturate(float factor);
+
+ // Blend the color with a second color by a certain factor between 0 and 1
+ // 1.0 => original color
+ // 0.0 => other color
+ void BlendWithColor(float factor, const HtmlColor& c);
+
+ string ToString() const;
+
+ // hexstr is in form of "xxxxxx"
+ void SetValueFromHexStr(const char *hexstr);
+
+ // either a color name or a hex string "#xxxxxx"
+ // This method also accepts a CSS shorthand string "#xyz" for convenience.
+ // "#xyz" is expanded to "#xxyyzz" before processing.
+ void SetValueFromStr(const char* str);
+
+ // Set the html color object from rgb values
+ void SetValueFromRGB(unsigned char r, unsigned char g,
+ unsigned char b);
+
+ // must be a color name. It can be one of 147 colors defined in CSS3 color
+ // module or SVG 1.0, which is supported by all major browsers. A reference
+ // can be found at:
+ // http://www.w3.org/TR/css3-color/#svg-color
+ void SetValueFromName(const char* str);
+
+ int r() const { return static_cast<int>(r_); }
+ int g() const { return static_cast<int>(g_); }
+ int b() const { return static_cast<int>(b_); }
+ int rgb() const { return b() + (g() << 8) + (r() << 16); }
+};
+
+class HtmlColorUtils {
+ public:
+
+ // Converts a color into its shortest possible CSS representation.
+ // For 9 colors, that is their color name. Example: "#008000" returns "green".
+ // For colors in the form #rrggbb, where r=r, g=g, and b=b, that is #rgb.
+ // Example: "#aabbcc" returns "#abc".
+ // For all other colors, the six hex-digit representation is shortest.
+ // Example: "lightgoldenrodyellow" returns "#FAFAD2"
+ static string MaybeConvertToCssShorthand(const HtmlColor& color);
+ static string MaybeConvertToCssShorthand(const char* orig);
+};
+
+#endif // _HTML_COLOR_H_
diff --git a/trunk/src/third_party/css_parser/src/webutil/html/htmltagenum.cc b/trunk/src/third_party/css_parser/src/webutil/html/htmltagenum.cc
new file mode 100644
index 0000000..0348545
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/html/htmltagenum.cc
@@ -0,0 +1,82 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2006 Google Inc. All Rights Reserved.
+// Author: dpeng@google.com (Daniel Peng)
+
+#include "base/basictypes.h"
+#include "webutil/html/htmltagenum.h"
+#include "strings/stringprintf.h"
+
+const char* HtmlTagEnumNames[] =
+{ "Unknown",
+ "A", "Abbr", "Acronym", "Address", "Applet",
+ "Area", "B", "Base", "Basefont", "Bdo",
+ "Big", "Blockquote", "Body", "Br", "Button",
+ "Caption", "Center", "Cite", "Code", "Col",
+ "Colgroup", "Dd", "Del", "Dfn", "Dir",
+ "Div", "Dl", "Dt", "Em", "Fieldset",
+ "Font", "Form", "Frame", "Frameset", "H1",
+ "H2", "H3", "H4", "H5", "H6", "Head",
+ "Hr", "Html", "I", "Iframe", "Img",
+ "Input", "Ins", "Isindex", "Kbd", "Label",
+ "Legend", "Li", "Link", "Map", "Menu",
+ "Meta", "Noframes", "Noscript", "Object",
+ "Ol", "Optgroup", "Option", "P", "Param",
+ "Pre", "Q", "S", "Samp", "Script",
+ "Select", "Small", "Span", "Strike",
+ "Strong", "Style", "Sub", "Sup", "Table",
+ "Tbody", "Td", "Textarea", "Tfoot", "Th",
+ "Thead", "Title", "Tr", "Tt",
+ "U", "Ul", "Var",
+ // Empty tag
+ "ZeroLength",
+ // Used in repository/lexer/html_lexer.cc
+ "!--", "Blink",
+ // Used in repository/parsers/base/handler-parser.cc
+ "Embed", "Marquee",
+ // Legacy backwards-compatible tags mentioned in HTML5.
+ "Nobr", "Wbr", "Bgsound", "Image",
+ "Listing", "Noembed", "Plaintext", "Spacer",
+ "Xmp",
+ // From Netscape Navigator 4.0
+ "Ilayer", "Keygen", "Layer", "Multicol", "Nolayer", "Server",
+ // !doctype
+ "!Doctype",
+ // Legacy tag used mostly by Russian sites.
+ "Noindex",
+ // Old style comments,
+ "!Comment",
+};
+
+COMPILE_ASSERT(arraysize(HtmlTagEnumNames) == kHtmlTagBuiltinMax,
+ ForgotToAddTagToHtmlTagEnumNames);
+
+const char* HtmlTagName(HtmlTagEnum tag) {
+ if (tag < kHtmlTagBuiltinMax) {
+ return HtmlTagEnumNames[tag];
+ } else {
+ return NULL;
+ }
+}
+
+string HtmlTagNameOrUnknown(int i) {
+ if (i < kHtmlTagBuiltinMax) {
+ return HtmlTagEnumNames[i];
+ } else {
+ return StringPrintf("UNKNOWN%d", i);
+ }
+}
diff --git a/trunk/src/third_party/css_parser/src/webutil/html/htmltagenum.h b/trunk/src/third_party/css_parser/src/webutil/html/htmltagenum.h
new file mode 100644
index 0000000..09c5762
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/html/htmltagenum.h
@@ -0,0 +1,194 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2006 Google Inc. All Rights Reserved.
+// Author: mec@google.com (Michael Chastain)
+// Author: dpeng@google.com (Daniel Peng)
+
+#ifndef WEBUTIL_HTML_HTMLTAGENUM_H__
+#define WEBUTIL_HTML_HTMLTAGENUM_H__
+
+#include <string>
+#include "string_using.h"
+
+// This is public at the top level because I think a lot of people
+// will want to use it.
+//
+// NOTE: These values may be stored in proto buffers. Do not change or remove
+// any existing values. If you want to add support for a new tag, use the
+// next available value (as specified by kHtmlTagBuiltinMax), and increment
+// kHtmlTagBuiltinMax. Also make sure to add the new tag to HtmlTagEnumNames
+// in htmltagenum.cc.
+//
+// This tag list came from:
+// http://www.w3.org/TR/REC-html40/index/elements.html
+// With additional tags used in:
+// repository/lexer/html_lexer.cc
+// repository/parsers/base/handler-parser.cc
+// And some additional legacy backwards-compatible tags from HTML5:
+// http://whatwg.org/specs/web-apps/current-work/#stack
+// [accessed 2006-10-10]
+// Plus some Netscape Navigator 4.0 tags from:
+// http://devedge-temp.mozilla.org/library/manuals/1998/htmlguide/
+// [accessed 2006-11-21]
+// The !doctype tag defines a reference to DTD (doctype type definition).
+// It is a SGML tag and is somehow mentioned in the HTML 4.01 spec.
+// http://www.w3.org/TR/html401/intro/sgmltut.html
+// The noindex tag is a non-standard tag used mostly by Russian sites:
+// http://ru.wikipedia.org/wiki/Noindex
+// http://translate.google.com/translate?u=http://ru.wikipedia.org/wiki/Noindex&sl=ru&tl=en
+
+enum HtmlTagEnum {
+ // Unknown tag: must be 0
+ kHtmlTagUnknown = 0,
+ // From html 4.01 spec
+ kHtmlTagA = 1,
+ kHtmlTagAbbr = 2,
+ kHtmlTagAcronym = 3,
+ kHtmlTagAddress = 4,
+ kHtmlTagApplet = 5,
+ kHtmlTagArea = 6,
+ kHtmlTagB = 7,
+ kHtmlTagBase = 8,
+ kHtmlTagBasefont = 9,
+ kHtmlTagBdo = 10,
+ kHtmlTagBig = 11,
+ kHtmlTagBlockquote = 12,
+ kHtmlTagBody = 13,
+ kHtmlTagBr = 14,
+ kHtmlTagButton = 15,
+ kHtmlTagCaption = 16,
+ kHtmlTagCenter = 17,
+ kHtmlTagCite = 18,
+ kHtmlTagCode = 19,
+ kHtmlTagCol = 20,
+ kHtmlTagColgroup = 21,
+ kHtmlTagDd = 22,
+ kHtmlTagDel = 23,
+ kHtmlTagDfn = 24,
+ kHtmlTagDir = 25,
+ kHtmlTagDiv = 26,
+ kHtmlTagDl = 27,
+ kHtmlTagDt = 28,
+ kHtmlTagEm = 29,
+ kHtmlTagFieldset = 30,
+ kHtmlTagFont = 31,
+ kHtmlTagForm = 32,
+ kHtmlTagFrame = 33,
+ kHtmlTagFrameset = 34,
+ kHtmlTagH1 = 35,
+ kHtmlTagH2 = 36,
+ kHtmlTagH3 = 37,
+ kHtmlTagH4 = 38,
+ kHtmlTagH5 = 39,
+ kHtmlTagH6 = 40,
+ kHtmlTagHead = 41,
+ kHtmlTagHr = 42,
+ kHtmlTagHtml = 43,
+ kHtmlTagI = 44,
+ kHtmlTagIframe = 45,
+ kHtmlTagImg = 46,
+ kHtmlTagInput = 47,
+ kHtmlTagIns = 48,
+ kHtmlTagIsindex = 49,
+ kHtmlTagKbd = 50,
+ kHtmlTagLabel = 51,
+ kHtmlTagLegend = 52,
+ kHtmlTagLi = 53,
+ kHtmlTagLink = 54,
+ kHtmlTagMap = 55,
+ kHtmlTagMenu = 56,
+ kHtmlTagMeta = 57,
+ kHtmlTagNoframes = 58,
+ kHtmlTagNoscript = 59,
+ kHtmlTagObject = 60,
+ kHtmlTagOl = 61,
+ kHtmlTagOptgroup = 62,
+ kHtmlTagOption = 63,
+ kHtmlTagP = 64,
+ kHtmlTagParam = 65,
+ kHtmlTagPre = 66,
+ kHtmlTagQ = 67,
+ kHtmlTagS = 68,
+ kHtmlTagSamp = 69,
+ kHtmlTagScript = 70,
+ kHtmlTagSelect = 71,
+ kHtmlTagSmall = 72,
+ kHtmlTagSpan = 73,
+ kHtmlTagStrike = 74,
+ kHtmlTagStrong = 75,
+ kHtmlTagStyle = 76,
+ kHtmlTagSub = 77,
+ kHtmlTagSup = 78,
+ kHtmlTagTable = 79,
+ kHtmlTagTbody = 80,
+ kHtmlTagTd = 81,
+ kHtmlTagTextarea = 82,
+ kHtmlTagTfoot = 83,
+ kHtmlTagTh = 84,
+ kHtmlTagThead = 85,
+ kHtmlTagTitle = 86,
+ kHtmlTagTr = 87,
+ kHtmlTagTt = 88,
+ kHtmlTagU = 89,
+ kHtmlTagUl = 90,
+ kHtmlTagVar = 91,
+ // Empty tag
+ kHtmlTagZeroLength = 92,
+ // Used in repository/lexer/html_lexer.cc
+ kHtmlTagBangDashDash = 93,
+ kHtmlTagBlink = 94,
+ // Used in repository/parsers/base/handler-parser.cc
+ kHtmlTagEmbed = 95,
+ kHtmlTagMarquee = 96,
+ // Legacy backwards-compatible tags mentioned in HTML5.
+ kHtmlTagNobr = 97,
+ kHtmlTagWbr = 98,
+ kHtmlTagBgsound = 99,
+ kHtmlTagImage = 100,
+ kHtmlTagListing = 101,
+ kHtmlTagNoembed = 102,
+ kHtmlTagPlaintext = 103,
+ kHtmlTagSpacer = 104,
+ kHtmlTagXmp = 105,
+ // From Netscape Navigator 4.0
+ kHtmlTagIlayer = 106,
+ kHtmlTagKeygen = 107,
+ kHtmlTagLayer = 108,
+ kHtmlTagMulticol = 109,
+ kHtmlTagNolayer = 110,
+ kHtmlTagServer = 111,
+ // !doctype from SGML and also from HTML 4.01 spec.
+ kHtmlTagBangDoctype = 112,
+ // Legacy tag used mostly by Russian sites.
+ kHtmlTagNoindex = 113,
+ // Anything starts with ! (except those marked above) or ?
+ kHtmlTagBogusComment = 114,
+
+ // Add new tag values here. Make sure you also add new tags to
+ // HtmlTagEnumNames in htmltagenum.cc and update kHtmlTagBuiltinMax.
+
+ // Sentinel.
+ kHtmlTagBuiltinMax = 115
+};
+
+// NULL if tag >= kHtmlTagBuiltinMax.
+extern const char* HtmlTagName(HtmlTagEnum tag);
+
+// StringPrintf("UNKNOWN%d", tag) if tag >= kHtmlTag
+extern string HtmlTagNameOrUnknown(int i);
+
+#endif // WEBUTIL_HTML_HTMLTAGENUM_H__
diff --git a/trunk/src/third_party/css_parser/src/webutil/html/htmltagindex.cc b/trunk/src/third_party/css_parser/src/webutil/html/htmltagindex.cc
new file mode 100644
index 0000000..f6327bf
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/html/htmltagindex.cc
@@ -0,0 +1,425 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2006, Google Inc. All rights reserved.
+// Author: mec@google.com (Michael Chastain)
+
+#include <ctype.h>
+#include <string.h>
+#include "webutil/html/htmltagindex.h"
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "base/paranoid.h"
+#include "strings/ascii_ctype.h"
+#include "util/gtl/dense_hash_map.h"
+#include "util/hash/hash.h"
+
+// Assert character properties that the fast path depends on.
+// (Uppercase letter | 0x20) == lowercase letter
+// All digits have 0x20 bit set
+// Several special chars have 0x20 bit set
+COMPILE_ASSERT(('A'|0x20)=='a', A_a);
+COMPILE_ASSERT(('B'|0x20)=='b', B_b);
+COMPILE_ASSERT(('C'|0x20)=='c', C_c);
+COMPILE_ASSERT(('D'|0x20)=='d', D_d);
+COMPILE_ASSERT(('E'|0x20)=='e', E_e);
+COMPILE_ASSERT(('F'|0x20)=='f', F_f);
+COMPILE_ASSERT(('G'|0x20)=='g', G_g);
+COMPILE_ASSERT(('H'|0x20)=='h', H_h);
+COMPILE_ASSERT(('I'|0x20)=='i', I_i);
+COMPILE_ASSERT(('J'|0x20)=='j', J_j);
+COMPILE_ASSERT(('K'|0x20)=='k', K_k);
+COMPILE_ASSERT(('L'|0x20)=='l', L_l);
+COMPILE_ASSERT(('M'|0x20)=='m', M_m);
+COMPILE_ASSERT(('N'|0x20)=='n', N_n);
+COMPILE_ASSERT(('O'|0x20)=='o', O_o);
+COMPILE_ASSERT(('P'|0x20)=='p', P_p);
+COMPILE_ASSERT(('Q'|0x20)=='q', Q_q);
+COMPILE_ASSERT(('R'|0x20)=='r', R_r);
+COMPILE_ASSERT(('S'|0x20)=='s', S_s);
+COMPILE_ASSERT(('T'|0x20)=='t', T_t);
+COMPILE_ASSERT(('U'|0x20)=='u', U_u);
+COMPILE_ASSERT(('V'|0x20)=='v', V_v);
+COMPILE_ASSERT(('W'|0x20)=='w', W_w);
+COMPILE_ASSERT(('X'|0x20)=='x', X_x);
+COMPILE_ASSERT(('Y'|0x20)=='y', Y_y);
+COMPILE_ASSERT(('Z'|0x20)=='z', Z_z);
+COMPILE_ASSERT('0'&0x20, d0);
+COMPILE_ASSERT('1'&0x20, d1);
+COMPILE_ASSERT('2'&0x20, d2);
+COMPILE_ASSERT('3'&0x20, d3);
+COMPILE_ASSERT('4'&0x20, d4);
+COMPILE_ASSERT('5'&0x20, d5);
+COMPILE_ASSERT('6'&0x20, d6);
+COMPILE_ASSERT('7'&0x20, d7);
+COMPILE_ASSERT('8'&0x20, d8);
+COMPILE_ASSERT('9'&0x20, d9);
+COMPILE_ASSERT('!'&0x20, cBang);
+COMPILE_ASSERT('-'&0x20, cDash);
+COMPILE_ASSERT('?'&0x20, cQuestion);
+
+// Ctor.
+
+HtmlTagIndex::HtmlTagIndex()
+ : case_sensitive_fixed_(false),
+ index_max_(kHtmlTagBuiltinMax),
+ custom_tag_map_(NULL) {
+ this->SetCaseSensitive(false);
+}
+
+// Dtor.
+
+HtmlTagIndex::~HtmlTagIndex() {
+}
+
+// Case sensitivity.
+
+void HtmlTagIndex::SetCaseSensitive(bool case_sensitive) {
+ CHECK(!case_sensitive_fixed_);
+ case_sensitive_ = SanitizeBool(case_sensitive);
+ if (case_sensitive_) {
+ case_mask_1_ = case_mask_2_ = case_mask_3_ = case_mask_4_ = 0;
+ case_mask_5_ = case_mask_6_ = case_mask_7_ = case_mask_8_ = 0;
+ } else {
+ case_mask_1_ = 0x20;
+ case_mask_2_ = 0x2020;
+ case_mask_3_ = 0x202020;
+ case_mask_4_ = 0x20202020;
+ case_mask_5_ = 0x2020202020ull;
+ case_mask_6_ = 0x202020202020ull;
+ case_mask_7_ = 0x20202020202020ull;
+ case_mask_8_ = 0x2020202020202020ull;
+ }
+}
+
+// Return a case-aware string. If case-sensitive, this is the same string.
+// If case-insensitive, this is the lower-case version.
+
+static string CaseAwareString(bool case_sensitive,
+ const char* tag,
+ int length) {
+ CHECK_GE(length, 0);
+ string str;
+ if (case_sensitive) {
+ str.assign(tag, length);
+ } else {
+ for (int i = 0; i < length; ++i) {
+ str += ascii_tolower(tag[i]);
+ }
+ }
+ return str;
+}
+
+// Add a tag.
+
+int HtmlTagIndex::AddHtmlTag(const char* tag, int length) {
+ // No more changing the case sensitivity.
+ case_sensitive_fixed_ = true;
+
+ // Look for existing tag.
+ const int tag_id = FindHtmlTag(tag, length);
+ if (tag_id != kHtmlTagUnknown)
+ return tag_id;
+
+ // Add to the custom table.
+ if (custom_tag_map_.get() == NULL) {
+ custom_tag_map_.reset(new CustomTagMap);
+ custom_tag_map_->set_empty_key(string(""));
+ }
+ string tag_copy(CaseAwareString(case_sensitive_, tag, length));
+ (*custom_tag_map_)[tag_copy] = index_max_;
+ return index_max_++;
+}
+
+// Find a tag.
+// This is hard-wired to go fast.
+
+int HtmlTagIndex::FindHtmlTag(const char* tag, int length) const {
+// These macros convert a string to a uint32 or uint64
+#define S1(s) ((s)[0])
+#define S2(s) (S1(s) | ((s)[1] << 8))
+#define S3(s) (S2(s) | ((s)[2] << 16))
+#define S4(s) (S3(s) | ((s)[3] << 24))
+#define S5(s) (S4(s) | (static_cast<uint64>((s)[4]) << 32))
+#define S6(s) (S5(s) | (static_cast<uint64>((s)[5]) << 40))
+#define S7(s) (S6(s) | (static_cast<uint64>((s)[6]) << 48))
+#define S8(s) (S7(s) | (static_cast<uint64>((s)[7]) << 56))
+
+// These macros convert a sequence of characters to a uint32 or uint64
+#define C1(c0) (c0)
+#define C2(c0, c1) (C1(c0) | ((c1) << 8))
+#define C3(c0, c1, c2) (C2(c0, c1) | ((c2) << 16))
+#define C4(c0, c1, c2, c3) (C3(c0, c1, c2) | ((c3) << 24))
+#define C5(c0, c1, c2, c3, c4) \
+ (C4(c0, c1, c2, c3) | (static_cast<uint64>(c4) << 32))
+#define C6(c0, c1, c2, c3, c4, c5) \
+ (C5(c0, c1, c2, c3, c4) | (static_cast<uint64>(c5) << 40))
+#define C7(c0, c1, c2, c3, c4, c5, c6) \
+ (C6(c0, c1, c2, c3, c4, c5) | (static_cast<uint64>(c6) << 48))
+#define C8(c0, c1, c2, c3, c4, c5, c6, c7) \
+ (C7(c0, c1, c2, c3, c4, c5, c6) | (static_cast<uint64>(c7) << 56))
+
+ switch (length) {
+ case 0:
+ // Empty tag
+ return kHtmlTagZeroLength;
+ case 1:
+ {
+ switch (S1(tag)|case_mask_1_) {
+ default: break;
+ // From html 4.01 spec
+ case C1('a'): return kHtmlTagA;
+ case C1('b'): return kHtmlTagB;
+ case C1('i'): return kHtmlTagI;
+ case C1('p'): return kHtmlTagP;
+ case C1('q'): return kHtmlTagQ;
+ case C1('s'): return kHtmlTagS;
+ case C1('u'): return kHtmlTagU;
+ }
+ }
+ break;
+ case 2:
+ {
+ switch (S2(tag)|case_mask_2_) {
+ default: break;
+ // From html 4.01 spec
+ case C2('b','r'): return kHtmlTagBr;
+ case C2('d','d'): return kHtmlTagDd;
+ case C2('d','l'): return kHtmlTagDl;
+ case C2('d','t'): return kHtmlTagDt;
+ case C2('e','m'): return kHtmlTagEm;
+ // Beware of matching "h\x11" or "H\x11" in case-insensitive mode.
+ case C2('h','1'): if (tag[1] == '1') return kHtmlTagH1; else break;
+ case C2('h','2'): if (tag[1] == '2') return kHtmlTagH2; else break;
+ case C2('h','3'): if (tag[1] == '3') return kHtmlTagH3; else break;
+ case C2('h','4'): if (tag[1] == '4') return kHtmlTagH4; else break;
+ case C2('h','5'): if (tag[1] == '5') return kHtmlTagH5; else break;
+ case C2('h','6'): if (tag[1] == '6') return kHtmlTagH6; else break;
+ case C2('h','r'): return kHtmlTagHr;
+ case C2('l','i'): return kHtmlTagLi;
+ case C2('o','l'): return kHtmlTagOl;
+ case C2('t','d'): return kHtmlTagTd;
+ case C2('t','h'): return kHtmlTagTh;
+ case C2('t','r'): return kHtmlTagTr;
+ case C2('t','t'): return kHtmlTagTt;
+ case C2('u','l'): return kHtmlTagUl;
+ }
+ }
+ break;
+ case 3:
+ {
+ switch (S3(tag)|case_mask_3_) {
+ default: break;
+ // From html 4.01 spec
+ case C3('b','d','o'): return kHtmlTagBdo;
+ case C3('b','i','g'): return kHtmlTagBig;
+ case C3('c','o','l'): return kHtmlTagCol;
+ case C3('d','e','l'): return kHtmlTagDel;
+ case C3('d','i','r'): return kHtmlTagDir;
+ case C3('d','i','v'): return kHtmlTagDiv;
+ case C3('d','f','n'): return kHtmlTagDfn;
+ case C3('i','m','g'): return kHtmlTagImg;
+ case C3('i','n','s'): return kHtmlTagIns;
+ case C3('k','b','d'): return kHtmlTagKbd;
+ case C3('m','a','p'): return kHtmlTagMap;
+ case C3('p','r','e'): return kHtmlTagPre;
+ case C3('s','u','b'): return kHtmlTagSub;
+ case C3('s','u','p'): return kHtmlTagSup;
+ case C3('v','a','r'): return kHtmlTagVar;
+ case C3('w','b','r'): return kHtmlTagWbr;
+ case C3('x','m','p'): return kHtmlTagXmp;
+ // Used in repository/lexer/html_lexer.cc
+ case C3('!','-','-'):
+ // These chars have no upper-lower case form so I haev to rematch.
+ if (S3(tag) == C3('!','-','-'))
+ return kHtmlTagBangDashDash;
+ break;
+ }
+ }
+ break;
+ case 4:
+ {
+ switch (S4(tag)|case_mask_4_) {
+ default: break;
+ // From html 4.01 spec
+ case C4('a','b','b','r'): return kHtmlTagAbbr;
+ case C4('a','r','e','a'): return kHtmlTagArea;
+ case C4('b','a','s','e'): return kHtmlTagBase;
+ case C4('b','o','d','y'): return kHtmlTagBody;
+ case C4('c','i','t','e'): return kHtmlTagCite;
+ case C4('c','o','d','e'): return kHtmlTagCode;
+ case C4('f','o','n','t'): return kHtmlTagFont;
+ case C4('f','o','r','m'): return kHtmlTagForm;
+ case C4('h','e','a','d'): return kHtmlTagHead;
+ case C4('h','t','m','l'): return kHtmlTagHtml;
+ case C4('l','i','n','k'): return kHtmlTagLink;
+ case C4('m','e','n','u'): return kHtmlTagMenu;
+ case C4('m','e','t','a'): return kHtmlTagMeta;
+ case C4('s','a','m','p'): return kHtmlTagSamp;
+ case C4('s','p','a','n'): return kHtmlTagSpan;
+ case C4('n','o','b','r'): return kHtmlTagNobr;
+ }
+ }
+ break;
+ case 5:
+ {
+ switch (S5(tag)|case_mask_5_) {
+ default: break;
+ // From html 4.01 spec
+ case C5('f','r','a','m','e'): return kHtmlTagFrame;
+ case C5('i','n','p','u','t'): return kHtmlTagInput;
+ case C5('l','a','b','e','l'): return kHtmlTagLabel;
+ case C5('p','a','r','a','m'): return kHtmlTagParam;
+ case C5('s','m','a','l','l'): return kHtmlTagSmall;
+ case C5('s','t','y','l','e'): return kHtmlTagStyle;
+ case C5('t','a','b','l','e'): return kHtmlTagTable;
+ case C5('t','b','o','d','y'): return kHtmlTagTbody;
+ case C5('t','f','o','o','t'): return kHtmlTagTfoot;
+ case C5('t','h','e','a','d'): return kHtmlTagThead;
+ case C5('t','i','t','l','e'): return kHtmlTagTitle;
+ // Used in repository/lexer/html_lexer.cc
+ case C5('b','l','i','n','k'): return kHtmlTagBlink;
+ // Used in repository/parsers/base/handler-parser.cc
+ case C5('e','m','b','e','d'): return kHtmlTagEmbed;
+ case C5('i','m','a','g','e'): return kHtmlTagImage;
+ // From Netscape Navigator 4.0
+ case C5('l','a','y','e','r'): return kHtmlTagLayer;
+ }
+ }
+ break;
+ case 6:
+ {
+ switch (S6(tag)|case_mask_6_) {
+ default: break;
+ // From html 4.01 spec
+ case C6('a','p','p','l','e','t'): return kHtmlTagApplet;
+ case C6('b','u','t','t','o','n'): return kHtmlTagButton;
+ case C6('c','e','n','t','e','r'): return kHtmlTagCenter;
+ case C6('i','f','r','a','m','e'): return kHtmlTagIframe;
+ case C6('l','e','g','e','n','d'): return kHtmlTagLegend;
+ case C6('o','b','j','e','c','t'): return kHtmlTagObject;
+ case C6('o','p','t','i','o','n'): return kHtmlTagOption;
+ case C6('s','c','r','i','p','t'): return kHtmlTagScript;
+ case C6('s','e','l','e','c','t'): return kHtmlTagSelect;
+ case C6('s','t','r','i','k','e'): return kHtmlTagStrike;
+ case C6('s','t','r','o','n','g'): return kHtmlTagStrong;
+ case C6('s','p','a','c','e','r'): return kHtmlTagSpacer;
+ // From Netscape Navigator 4.0
+ case C6('i','l','a','y','e','r'): return kHtmlTagIlayer;
+ case C6('k','e','y','g','e','n'): return kHtmlTagKeygen;
+ case C6('s','e','r','v','e','r'): return kHtmlTagServer;
+ }
+ }
+ break;
+ case 7:
+ {
+ switch (S7(tag)|case_mask_7_) {
+ default: break;
+ // From html 4.01 spec
+ case C7('a','c','r','o','n','y','m'): return kHtmlTagAcronym;
+ case C7('a','d','d','r','e','s','s'): return kHtmlTagAddress;
+ case C7('c','a','p','t','i','o','n'): return kHtmlTagCaption;
+ case C7('i','s','i','n','d','e','x'): return kHtmlTagIsindex;
+ // Used in repository/parsers/base/handler-parser.cc
+ case C7('m','a','r','q','u','e','e'): return kHtmlTagMarquee;
+ case C7('b','g','s','o','u','n','d'): return kHtmlTagBgsound;
+ case C7('l','i','s','t','i','n','g'): return kHtmlTagListing;
+ case C7('n','o','e','m','b','e','d'): return kHtmlTagNoembed;
+ // From Netscape Navigator 4.0
+ case C7('n','o','l','a','y','e','r'): return kHtmlTagNolayer;
+ // Legacy tag used mostly by Russian sites
+ case C7('n','o','i','n','d','e','x'): return kHtmlTagNoindex;
+ }
+ }
+ break;
+ case 8:
+ {
+ switch (S8(tag)|case_mask_8_) {
+ default: break;
+ // From html 4.01 spec
+ case C8('b','a','s','e','f','o','n','t'): return kHtmlTagBasefont;
+ case C8('c','o','l','g','r','o','u','p'): return kHtmlTagColgroup;
+ case C8('f','i','e','l','d','s','e','t'): return kHtmlTagFieldset;
+ case C8('f','r','a','m','e','s','e','t'): return kHtmlTagFrameset;
+ case C8('n','o','f','r','a','m','e','s'): return kHtmlTagNoframes;
+ case C8('n','o','s','c','r','i','p','t'): return kHtmlTagNoscript;
+ case C8('o','p','t','g','r','o','u','p'): return kHtmlTagOptgroup;
+ case C8('t','e','x','t','a','r','e','a'): return kHtmlTagTextarea;
+ // From Netscape Navigator 4.0
+ case C8('m','u','l','t','i','c','o','l'): return kHtmlTagMulticol;
+ }
+ }
+ break;
+ case 9:
+ {
+ if ((S4(tag+0)|case_mask_4_) == C4('p','l','a','i') &&
+ (S4(tag+4)|case_mask_4_) == C4('n','t','e','x') &&
+ (S1(tag+8)|case_mask_1_) == C1('t'))
+ return kHtmlTagPlaintext;
+ }
+ break;
+ case 10:
+ {
+ // From html 4.01 spec
+ if ((S4(tag+0)|case_mask_4_) == C4('b','l','o','c') &&
+ (S4(tag+4)|case_mask_4_) == C4('k','q','u','o') &&
+ (S2(tag+8)|case_mask_2_) == C2('t','e'))
+ return kHtmlTagBlockquote;
+ }
+ break;
+ }
+
+ // !doctype is special. Any tag name starting with !doctype (no need for
+ // exact match) is considered !doctype. Tested on IE 7.0 and Firefox 2.0.
+ // You can also refer to (ongoing work):
+ // http://whatwg.org/specs/web-apps/current-work/#markup
+ if (length >= 8 && S1(tag) == C1('!') &&
+ (S8(tag)|case_mask_8_) == C8('!','d','o','c','t','y','p','e'))
+ return kHtmlTagBangDoctype;
+
+ // Otherwise, !blahblah and ?blahblah are comments.
+ if (S1(tag) == C1('!') || S1(tag) == C1('?'))
+ return kHtmlTagBogusComment;
+
+#undef C8
+#undef C7
+#undef C6
+#undef C5
+#undef C4
+#undef C3
+#undef C2
+#undef C1
+#undef S8
+#undef S7
+#undef S6
+#undef S5
+#undef S4
+#undef S3
+#undef S2
+#undef S1
+
+ // Look in the custom table.
+ if (custom_tag_map_.get() != NULL) {
+ string tag_copy(CaseAwareString(case_sensitive_, tag, length));
+ CustomTagMap::const_iterator it = custom_tag_map_->find(tag_copy);
+ if (it != custom_tag_map_->end()) {
+ return it->second;
+ }
+ }
+
+ // Unknown tag.
+ return kHtmlTagUnknown;
+}
diff --git a/trunk/src/third_party/css_parser/src/webutil/html/htmltagindex.h b/trunk/src/third_party/css_parser/src/webutil/html/htmltagindex.h
new file mode 100644
index 0000000..827ff10
--- /dev/null
+++ b/trunk/src/third_party/css_parser/src/webutil/html/htmltagindex.h
@@ -0,0 +1,89 @@
+/**
+ * Copyright 2010 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Copyright 2006, Google Inc. All rights reserved.
+// Author: mec@google.com (Michael Chastain)
+//
+// Map an html tag to a dense index number.
+// Hardwired for speed on builtin tags.
+// Caller can add tags on top of the builtins.
+// Caller can choose case-sensitive or case-insensitive.
+//
+// TODO(mec): merge this with webutil/html/htmltag
+
+#ifndef WEBUTIL_HTML_HTML_TAG_INDEX_H__
+#define WEBUTIL_HTML_HTML_TAG_INDEX_H__
+
+#include <string>
+#include "string_using.h"
+#include "base/basictypes.h"
+#include "base/scoped_ptr.h"
+#include "util/gtl/dense_hash_map.h"
+#include "webutil/html/htmltagenum.h"
+
+class HtmlTagIndex {
+ public:
+ HtmlTagIndex();
+ ~HtmlTagIndex();
+
+ // Add a tag and return its index. It is okay to add a builtin
+ // tag or to add the same tag more than once.
+ int AddHtmlTag(const char* tag, int length);
+ int AddHtmlTag(const char* tag) {
+ return AddHtmlTag(tag, strlen(tag));
+ }
+
+ // Find returns a value in the half-open range [0..GetIndexMax()).
+ // 0 == unknown tag.
+ COMPILE_ASSERT(kHtmlTagUnknown == 0, unknown_tag_equals_zero);
+ int FindHtmlTag(const char* tag, int length) const;
+ int FindHtmlTag(const char* tag) const {
+ return FindHtmlTag(tag, strlen(tag));
+ }
+
+ // Return the half-open upper bound on lookup return value.
+ // If GetIndexMax returns 10, then find will return [0..9).
+ int GetIndexMax() const {
+ return index_max_;
+ };
+
+ // Set case sensitivity. This cannot be done after any calls to AddHtmlTag.
+ void SetCaseSensitive(bool case_sensitive);
+ bool IsCaseSensitive() const {
+ return case_sensitive_;
+ }
+
+ private:
+ DISALLOW_EVIL_CONSTRUCTORS(HtmlTagIndex);
+
+ // Case sensitive stuff.
+ bool case_sensitive_fixed_;
+ bool case_sensitive_;
+ uint32 case_mask_1_;
+ uint32 case_mask_2_;
+ uint32 case_mask_3_;
+ uint32 case_mask_4_;
+ uint64 case_mask_5_;
+ uint64 case_mask_6_;
+ uint64 case_mask_7_;
+ uint64 case_mask_8_;
+
+ int index_max_;
+ typedef dense_hash_map<string, int> CustomTagMap;
+ scoped_ptr<CustomTagMap> custom_tag_map_;
+};
+
+#endif // WEBUTIL_HTML_HTML_TAG_INDEX_H__
diff --git a/trunk/src/third_party/gflags/gen/arch/linux/ia32/include/gflags/gflags.h b/trunk/src/third_party/gflags/gen/arch/linux/ia32/include/gflags/gflags.h
new file mode 100644
index 0000000..821d194
--- /dev/null
+++ b/trunk/src/third_party/gflags/gen/arch/linux/ia32/include/gflags/gflags.h
@@ -0,0 +1,538 @@
+// Copyright (c) 2006, Google Inc.
+// All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions are
+// met:
+//
+// * Redistributions of source code must retain the above copyright
+// notice, this list of conditions and the following disclaimer.
+// * Redistributions in binary form must reproduce the above
+// copyright notice, this list of conditions and the following disclaimer
+// in the documentation and/or other materials provided with the
+// distribution.
+// * Neither the name of Google Inc. nor the names of its
+// contributors may be used to endorse or promote products derived from
+// this software without specific prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+// ---
+// Author: Ray Sidney
+// Revamped and reorganized by Craig Silverstein
+//
+// This is the file that should be included by any file which declares
+// or defines a command line flag or wants to parse command line flags
+// or print a program usage message (which will include information about
+// flags). Executive summary, in the form of an example foo.cc file:
+//
+// #include "foo.h" // foo.h has a line "DECLARE_int32(start);"
+//
+// DEFINE_int32(end, 1000, "The last record to read");
+// DECLARE_bool(verbose); // some other file has a DEFINE_bool(verbose, ...)
+//
+// void MyFunc() {
+// if (FLAGS_verbose) printf("Records %d-%d\n", FLAGS_start, FLAGS_end);
+// }
+//
+// Then, at the command-line:
+// ./foo --noverbose --start=5 --end=100
+//
+// For more details, see
+// doc/gflags.html
+//
+// --- A note about thread-safety:
+//
+// We describe many functions in this routine as being thread-hostile,
+// thread-compatible, or thread-safe. Here are the meanings we use:
+//
+// thread-safe: it is safe for multiple threads to call this routine
+// (or, when referring to a class, methods of this class)
+// concurrently.
+// thread-hostile: it is not safe for multiple threads to call this
+// routine (or methods of this class) concurrently. In gflags,
+// most thread-hostile routines are intended to be called early in,
+// or even before, main() -- that is, before threads are spawned.
+// thread-compatible: it is safe for multiple threads to read from
+// this variable (when applied to variables), or to call const
+// methods of this class (when applied to classes), as long as no
+// other thread is writing to the variable or calling non-const
+// methods of this class.
+
+#ifndef GOOGLE_GFLAGS_H_
+#define GOOGLE_GFLAGS_H_
+
+#include <string>
+#include <vector>
+
+// We care a lot about number of bits things take up. Unfortunately,
+// systems define their bit-specific ints in a lot of different ways.
+// We use our own way, and have a typedef to get there.
+// Note: these commands below may look like "#if 1" or "#if 0", but
+// that's because they were constructed that way at ./configure time.
+// Look at gflags.h.in to see how they're calculated (based on your config).
+#if 1
+#include <stdint.h> // the normal place uint16_t is defined
+#endif
+#if 1
+#include <sys/types.h> // the normal place u_int16_t is defined
+#endif
+#if 1
+#include <inttypes.h> // a third place for uint16_t or u_int16_t
+#endif
+
+namespace google {
+
+#if 1 // the C99 format
+typedef int32_t int32;
+typedef uint32_t uint32;
+typedef int64_t int64;
+typedef uint64_t uint64;
+#elif 1 // the BSD format
+typedef int32_t int32;
+typedef u_int32_t uint32;
+typedef int64_t int64;
+typedef u_int64_t uint64;
+#elif 0 // the windows (vc7) format
+typedef __int32 int32;
+typedef unsigned __int32 uint32;
+typedef __int64 int64;
+typedef unsigned __int64 uint64;
+#else
+#error Do not know how to define a 32-bit integer quantity on your system
+#endif
+
+// --------------------------------------------------------------------
+// To actually define a flag in a file, use DEFINE_bool,
+// DEFINE_string, etc. at the bottom of this file. You may also find
+// it useful to register a validator with the flag. This ensures that
+// when the flag is parsed from the commandline, or is later set via
+// SetCommandLineOption, we call the validation function.
+//
+// The validation function should return true if the flag value is valid, and
+// false otherwise. If the function returns false for the new setting of the
+// flag, the flag will retain its current value. If it returns false for the
+// default value, InitGoogle will die.
+//
+// This function is safe to call at global construct time (as in the
+// example below).
+//
+// Example use:
+// static bool ValidatePort(const char* flagname, int32 value) {
+// if (value > 0 && value < 32768) // value is ok
+// return true;
+// printf("Invalid value for --%s: %d\n", flagname, (int)value);
+// return false;
+// }
+// DEFINE_int32(port, 0, "What port to listen on");
+// static bool dummy = RegisterFlagValidator(&FLAGS_port, &ValidatePort);
+
+// Returns true if successfully registered, false if not (because the
+// first argument doesn't point to a command-line flag, or because a
+// validator is already registered for this flag).
+bool RegisterFlagValidator(const bool* flag,
+ bool (*validate_fn)(const char*, bool));
+bool RegisterFlagValidator(const int32* flag,
+ bool (*validate_fn)(const char*, int32));
+bool RegisterFlagValidator(const int64* flag,
+ bool (*validate_fn)(const char*, int64));
+bool RegisterFlagValidator(const uint64* flag,
+ bool (*validate_fn)(const char*, uint64));
+bool RegisterFlagValidator(const double* flag,
+ bool (*validate_fn)(const char*, double));
+bool RegisterFlagValidator(const std::string* flag,
+ bool (*validate_fn)(const char*, const std::string&));
+
+
+// --------------------------------------------------------------------
+// These methods are the best way to get access to info about the
+// list of commandline flags. Note that these routines are pretty slow.
+// GetAllFlags: mostly-complete info about the list, sorted by file.
+// ShowUsageWithFlags: pretty-prints the list to stdout (what --help does)
+// ShowUsageWithFlagsRestrict: limit to filenames with restrict as a substr
+//
+// In addition to accessing flags, you can also access argv[0] (the program
+// name) and argv (the entire commandline), which we sock away a copy of.
+// These variables are static, so you should only set them once.
+
+struct CommandLineFlagInfo {
+ std::string name; // the name of the flag
+ std::string type; // the type of the flag: int32, etc
+ std::string description; // the "help text" associated with the flag
+ std::string current_value; // the current value, as a string
+ std::string default_value; // the default value, as a string
+ std::string filename; // 'cleaned' version of filename holding the flag
+ bool has_validator_fn; // true if RegisterFlagValidator called on flag
+ bool is_default; // true if the flag has default value
+};
+
+// Using this inside of a validator is a recipe for a deadlock.
+// TODO(wojtekm) Fix locking when validators are running, to make it safe to
+// call validators during ParseAllFlags.
+// Also make sure then to uncomment the corresponding unit test in
+// commandlineflags_unittest.sh
+extern void GetAllFlags(std::vector<CommandLineFlagInfo>* OUTPUT);
+// These two are actually defined in commandlineflags_reporting.cc.
+extern void ShowUsageWithFlags(const char *argv0); // what --help does
+extern void ShowUsageWithFlagsRestrict(const char *argv0, const char *restrict);
+
+// Create a descriptive string for a flag.
+// Goes to some trouble to make pretty line breaks.
+extern std::string DescribeOneFlag(const CommandLineFlagInfo& flag);
+
+// Thread-hostile; meant to be called before any threads are spawned.
+extern void SetArgv(int argc, const char** argv);
+// The following functions are thread-safe as long as SetArgv() is
+// only called before any threads start.
+extern const std::vector<std::string>& GetArgvs(); // all of argv as a vector
+extern const char* GetArgv(); // all of argv as a string
+extern const char* GetArgv0(); // only argv0
+extern uint32 GetArgvSum(); // simple checksum of argv
+extern const char* ProgramInvocationName(); // argv0, or "UNKNOWN" if not set
+extern const char* ProgramInvocationShortName(); // basename(argv0)
+// ProgramUsage() is thread-safe as long as SetUsageMessage() is only
+// called before any threads start.
+extern const char* ProgramUsage(); // string set by SetUsageMessage()
+
+
+// --------------------------------------------------------------------
+// Normally you access commandline flags by just saying "if (FLAGS_foo)"
+// or whatever, and set them by calling "FLAGS_foo = bar" (or, more
+// commonly, via the DEFINE_foo macro). But if you need a bit more
+// control, we have programmatic ways to get/set the flags as well.
+// These programmatic ways to access flags are thread-safe, but direct
+// access is only thread-compatible.
+
+// Return true iff the flagname was found.
+// OUTPUT is set to the flag's value, or unchanged if we return false.
+extern bool GetCommandLineOption(const char* name, std::string* OUTPUT);
+
+// Return true iff the flagname was found. OUTPUT is set to the flag's
+// CommandLineFlagInfo or unchanged if we return false.
+extern bool GetCommandLineFlagInfo(const char* name,
+ CommandLineFlagInfo* OUTPUT);
+
+// Return the CommandLineFlagInfo of the flagname. exit() if name not found.
+// Example usage, to check if a flag's value is currently the default value:
+// if (GetCommandLineFlagInfoOrDie("foo").is_default) ...
+extern CommandLineFlagInfo GetCommandLineFlagInfoOrDie(const char* name);
+
+enum FlagSettingMode {
+ // update the flag's value (can call this multiple times).
+ SET_FLAGS_VALUE,
+ // update the flag's value, but *only if* it has not yet been updated
+ // with SET_FLAGS_VALUE, SET_FLAG_IF_DEFAULT, or "FLAGS_xxx = nondef".
+ SET_FLAG_IF_DEFAULT,
+ // set the flag's default value to this. If the flag has not yet updated
+ // yet (via SET_FLAGS_VALUE, SET_FLAG_IF_DEFAULT, or "FLAGS_xxx = nondef")
+ // change the flag's current value to the new default value as well.
+ SET_FLAGS_DEFAULT
+};
+
+// Set a particular flag ("command line option"). Returns a string
+// describing the new value that the option has been set to. The
+// return value API is not well-specified, so basically just depend on
+// it to be empty if the setting failed for some reason -- the name is
+// not a valid flag name, or the value is not a valid value -- and
+// non-empty else.
+
+// SetCommandLineOption uses set_mode == SET_FLAGS_VALUE (the common case)
+extern std::string SetCommandLineOption(const char* name, const char* value);
+extern std::string SetCommandLineOptionWithMode(const char* name, const char* value,
+ FlagSettingMode set_mode);
+
+
+// --------------------------------------------------------------------
+// Saves the states (value, default value, whether the user has set
+// the flag, registered validators, etc) of all flags, and restores
+// them when the FlagSaver is destroyed. This is very useful in
+// tests, say, when you want to let your tests change the flags, but
+// make sure that they get reverted to the original states when your
+// test is complete.
+//
+// Example usage:
+// void TestFoo() {
+// FlagSaver s1;
+// FLAG_foo = false;
+// FLAG_bar = "some value";
+//
+// // test happens here. You can return at any time
+// // without worrying about restoring the FLAG values.
+// }
+//
+// Note: This class is marked with __attribute__((unused)) because all the
+// work is done in the constructor and destructor, so in the standard
+// usage example above, the compiler would complain that it's an
+// unused variable.
+//
+// This class is thread-safe.
+
+class FlagSaver {
+ public:
+ FlagSaver();
+ ~FlagSaver();
+
+ private:
+ class FlagSaverImpl* impl_; // we use pimpl here to keep API steady
+
+ FlagSaver(const FlagSaver&); // no copying!
+ void operator=(const FlagSaver&);
+} __attribute__ ((unused));
+
+// --------------------------------------------------------------------
+// Some deprecated or hopefully-soon-to-be-deprecated functions.
+
+// This is often used for logging. TODO(csilvers): figure out a better way
+extern std::string CommandlineFlagsIntoString();
+// Usually where this is used, a FlagSaver should be used instead.
+extern bool ReadFlagsFromString(const std::string& flagfilecontents,
+ const char* prog_name,
+ bool errors_are_fatal); // uses SET_FLAGS_VALUE
+
+// These let you manually implement --flagfile functionality.
+// DEPRECATED.
+extern bool AppendFlagsIntoFile(const std::string& filename, const char* prog_name);
+extern bool SaveCommandFlags(); // actually defined in google.cc !
+extern bool ReadFromFlagsFile(const std::string& filename, const char* prog_name,
+ bool errors_are_fatal); // uses SET_FLAGS_VALUE
+
+
+// --------------------------------------------------------------------
+// Useful routines for initializing flags from the environment.
+// In each case, if 'varname' does not exist in the environment
+// return defval. If 'varname' does exist but is not valid
+// (e.g., not a number for an int32 flag), abort with an error.
+// Otherwise, return the value. NOTE: for booleans, for true use
+// 't' or 'T' or 'true' or '1', for false 'f' or 'F' or 'false' or '0'.
+
+extern bool BoolFromEnv(const char *varname, bool defval);
+extern int32 Int32FromEnv(const char *varname, int32 defval);
+extern int64 Int64FromEnv(const char *varname, int64 defval);
+extern uint64 Uint64FromEnv(const char *varname, uint64 defval);
+extern double DoubleFromEnv(const char *varname, double defval);
+extern const char *StringFromEnv(const char *varname, const char *defval);
+
+
+// --------------------------------------------------------------------
+// The next two functions parse commandlineflags from main():
+
+// Set the "usage" message for this program. For example:
+// string usage("This program does nothing. Sample usage:\n");
+// usage += argv[0] + " <uselessarg1> <uselessarg2>";
+// SetUsageMessage(usage);
+// Do not include commandline flags in the usage: we do that for you!
+// Thread-hostile; meant to be called before any threads are spawned.
+extern void SetUsageMessage(const std::string& usage);
+
+// Looks for flags in argv and parses them. Rearranges argv to put
+// flags first, or removes them entirely if remove_flags is true.
+// If a flag is defined more than once in the command line or flag
+// file, the last definition is used.
+// See top-of-file for more details on this function.
+#ifndef SWIG // In swig, use ParseCommandLineFlagsScript() instead.
+extern uint32 ParseCommandLineFlags(int *argc, char*** argv,
+ bool remove_flags);
+#endif
+
+
+// Calls to ParseCommandLineNonHelpFlags and then to
+// HandleCommandLineHelpFlags can be used instead of a call to
+// ParseCommandLineFlags during initialization, in order to allow for
+// changing default values for some FLAGS (via
+// e.g. SetCommandLineOptionWithMode calls) between the time of
+// command line parsing and the time of dumping help information for
+// the flags as a result of command line parsing.
+// If a flag is defined more than once in the command line or flag
+// file, the last definition is used.
+extern uint32 ParseCommandLineNonHelpFlags(int *argc, char*** argv,
+ bool remove_flags);
+// This is actually defined in commandlineflags_reporting.cc.
+// This function is misnamed (it also handles --version, etc.), but
+// it's too late to change that now. :-(
+extern void HandleCommandLineHelpFlags(); // in commandlineflags_reporting.cc
+
+// Allow command line reparsing. Disables the error normally
+// generated when an unknown flag is found, since it may be found in a
+// later parse. Thread-hostile; meant to be called before any threads
+// are spawned.
+extern void AllowCommandLineReparsing();
+
+// Reparse the flags that have not yet been recognized.
+// Only flags registered since the last parse will be recognized.
+// Any flag value must be provided as part of the argument using "=",
+// not as a separate command line argument that follows the flag argument.
+// Intended for handling flags from dynamically loaded libraries,
+// since their flags are not registered until they are loaded.
+extern uint32 ReparseCommandLineNonHelpFlags();
+
+
+// --------------------------------------------------------------------
+// Now come the command line flag declaration/definition macros that
+// will actually be used. They're kind of hairy. A major reason
+// for this is initialization: we want people to be able to access
+// variables in global constructors and have that not crash, even if
+// their global constructor runs before the global constructor here.
+// (Obviously, we can't guarantee the flags will have the correct
+// default value in that case, but at least accessing them is safe.)
+// The only way to do that is have flags point to a static buffer.
+// So we make one, using a union to ensure proper alignment, and
+// then use placement-new to actually set up the flag with the
+// correct default value. In the same vein, we have to worry about
+// flag access in global destructors, so FlagRegisterer has to be
+// careful never to destroy the flag-values it constructs.
+//
+// Note that when we define a flag variable FLAGS_<name>, we also
+// preemptively define a junk variable, FLAGS_no<name>. This is to
+// cause a link-time error if someone tries to define 2 flags with
+// names like "logging" and "nologging". We do this because a bool
+// flag FLAG can be set from the command line to true with a "-FLAG"
+// argument, and to false with a "-noFLAG" argument, and so this can
+// potentially avert confusion.
+//
+// We also put flags into their own namespace. It is purposefully
+// named in an opaque way that people should have trouble typing
+// directly. The idea is that DEFINE puts the flag in the weird
+// namespace, and DECLARE imports the flag from there into the current
+// namespace. The net result is to force people to use DECLARE to get
+// access to a flag, rather than saying "extern bool FLAGS_whatever;"
+// or some such instead. We want this so we can put extra
+// functionality (like sanity-checking) in DECLARE if we want, and
+// make sure it is picked up everywhere.
+//
+// We also put the type of the variable in the namespace, so that
+// people can't DECLARE_int32 something that they DEFINE_bool'd
+// elsewhere.
+
+class FlagRegisterer {
+ public:
+ FlagRegisterer(const char* name, const char* type,
+ const char* help, const char* filename,
+ void* current_storage, void* defvalue_storage);
+};
+
+extern bool FlagsTypeWarn(const char *name);
+
+// If your application #defines STRIP_FLAG_HELP to a non-zero value
+// before #including this file, we remove the help message from the
+// binary file. This can reduce the size of the resulting binary
+// somewhat, and may also be useful for security reasons.
+
+extern const char kStrippedFlagHelp[];
+
+}
+
+#ifndef SWIG // In swig, ignore the main flag declarations
+
+#if defined(STRIP_FLAG_HELP) && STRIP_FLAG_HELP > 0
+// Need this construct to avoid the 'defined but not used' warning.
+#define MAYBE_STRIPPED_HELP(txt) (false ? (txt) : kStrippedFlagHelp)
+#else
+#define MAYBE_STRIPPED_HELP(txt) txt
+#endif
+
+// Each command-line flag has two variables associated with it: one
+// with the current value, and one with the default value. However,
+// we have a third variable, which is where value is assigned; it's a
+// constant. This guarantees that FLAG_##value is initialized at
+// static initialization time (e.g. before program-start) rather than
+// than global construction time (which is after program-start but
+// before main), at least when 'value' is a compile-time constant. We
+// use a small trick for the "default value" variable, and call it
+// FLAGS_no<name>. This serves the second purpose of assuring a
+// compile error if someone tries to define a flag named no<name>
+// which is illegal (--foo and --nofoo both affect the "foo" flag).
+#define DEFINE_VARIABLE(type, shorttype, name, value, help) \
+ namespace fL##shorttype { \
+ static const type FLAGS_nono##name = value; \
+ type FLAGS_##name = FLAGS_nono##name; \
+ type FLAGS_no##name = FLAGS_nono##name; \
+ static ::google::FlagRegisterer o_##name( \
+ #name, #type, MAYBE_STRIPPED_HELP(help), __FILE__, \
+ &FLAGS_##name, &FLAGS_no##name); \
+ } \
+ using fL##shorttype::FLAGS_##name
+
+#define DECLARE_VARIABLE(type, shorttype, name) \
+ namespace fL##shorttype { \
+ extern type FLAGS_##name; \
+ } \
+ using fL##shorttype::FLAGS_##name
+
+// For DEFINE_bool, we want to do the extra check that the passed-in
+// value is actually a bool, and not a string or something that can be
+// coerced to a bool. These declarations (no definition needed!) will
+// help us do that, and never evaluate From, which is important.
+// We'll use 'sizeof(IsBool(val))' to distinguish. This code requires
+// that the compiler have different sizes for bool & double. Since
+// this is not guaranteed by the standard, we check it with a
+// compile-time assert (msg[-1] will give a compile-time error).
+namespace fLB {
+struct CompileAssert {};
+typedef CompileAssert expected_sizeof_double_neq_sizeof_bool[
+ (sizeof(double) != sizeof(bool)) ? 1 : -1];
+template<typename From> double IsBoolFlag(const From& from);
+bool IsBoolFlag(bool from);
+} // namespace fLB
+
+#define DECLARE_bool(name) DECLARE_VARIABLE(bool, B, name)
+#define DEFINE_bool(name, val, txt) \
+ namespace fLB { \
+ typedef CompileAssert FLAG_##name##_value_is_not_a_bool[ \
+ (sizeof(::fLB::IsBoolFlag(val)) != sizeof(double)) ? 1 : -1]; \
+ } \
+ DEFINE_VARIABLE(bool, B, name, val, txt)
+
+#define DECLARE_int32(name) DECLARE_VARIABLE(::google::int32, I, name)
+#define DEFINE_int32(name,val,txt) DEFINE_VARIABLE(::google::int32, I, name, val, txt)
+
+#define DECLARE_int64(name) DECLARE_VARIABLE(::google::int64, I64, name)
+#define DEFINE_int64(name,val,txt) DEFINE_VARIABLE(::google::int64, I64, name, val, txt)
+
+#define DECLARE_uint64(name) DECLARE_VARIABLE(::google::uint64, U64, name)
+#define DEFINE_uint64(name,val,txt) DEFINE_VARIABLE(::google::uint64, U64, name, val, txt)
+
+#define DECLARE_double(name) DECLARE_VARIABLE(double, D, name)
+#define DEFINE_double(name, val, txt) DEFINE_VARIABLE(double, D, name, val, txt)
+
+// Strings are trickier, because they're not a POD, so we can't
+// construct them at static-initialization time (instead they get
+// constructed at global-constructor time, which is much later). To
+// try to avoid crashes in that case, we use a char buffer to store
+// the string, which we can static-initialize, and then placement-new
+// into it later. It's not perfect, but the best we can do.
+#define DECLARE_string(name) namespace fLS { extern std::string& FLAGS_##name; } \
+ using fLS::FLAGS_##name
+
+// We need to define a var named FLAGS_no##name so people don't define
+// --string and --nostring. And we need a temporary place to put val
+// so we don't have to evaluate it twice. Two great needs that go
+// great together!
+// The weird 'using' + 'extern' inside the fLS namespace is to work around
+// an unknown compiler bug/issue with the gcc 4.2.1 on SUSE 10. See
+// http://code.google.com/p/google-gflags/issues/detail?id=20
+#define DEFINE_string(name, val, txt) \
+ namespace fLS { \
+ static union { void* align; char s[sizeof(std::string)]; } s_##name[2]; \
+ const std::string* const FLAGS_no##name = new (s_##name[0].s) std::string(val); \
+ static ::google::FlagRegisterer o_##name( \
+ #name, "string", MAYBE_STRIPPED_HELP(txt), __FILE__, \
+ s_##name[0].s, new (s_##name[1].s) std::string(*FLAGS_no##name)); \
+ extern std::string& FLAGS_##name; \
+ using fLS::FLAGS_##name; \
+ std::string& FLAGS_##name = *(reinterpret_cast<std::string*>(s_##name[0].s)); \
+ } \
+ using fLS::FLAGS_##name
+
+#endif // SWIG
+
+#endif // GOOGLE_GFLAGS_H_
diff --git a/trunk/src/third_party/gflags/gen/arch/linux/ia32/include/gflags/gflags_completions.h b/trunk/src/third_party/gflags/gen/arch/linux/ia32/include/gflags/gflags_completions.h
new file mode 100644
index 0000000..9d9ce7a
--- /dev/null
+++ b/trunk/src/third_party/gflags/gen/arch/linux/ia32/include/gflags/gflags_completions.h
@@ -0,0 +1,121 @@
+// Copyright (c) 2008, Google Inc.
+// All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions are
+// met:
+//
+// * Redistributions of source code must retain the above copyright
+// notice, this list of conditions and the following disclaimer.
+// * Redistributions in binary form must reproduce the above
+// copyright notice, this list of conditions and the following disclaimer
+// in the documentation and/or other materials provided with the
+// distribution.
+// * Neither the name of Google Inc. nor the names of its
+// contributors may be used to endorse or promote products derived from
+// this software without specific prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+//
+// ---
+// Author: Dave Nicponski
+//
+// Implement helpful bash-style command line flag completions
+//
+// ** Functional API:
+// HandleCommandLineCompletions() should be called early during
+// program startup, but after command line flag code has been
+// initialized, such as the beginning of HandleCommandLineHelpFlags().
+// It checks the value of the flag --tab_completion_word. If this
+// flag is empty, nothing happens here. If it contains a string,
+// however, then HandleCommandLineCompletions() will hijack the
+// process, attempting to identify the intention behind this
+// completion. Regardless of the outcome of this deduction, the
+// process will be terminated, similar to --helpshort flag
+// handling.
+//
+// ** Overview of Bash completions:
+// Bash can be told to programatically determine completions for the
+// current 'cursor word'. It does this by (in this case) invoking a
+// command with some additional arguments identifying the command
+// being executed, the word being completed, and the previous word
+// (if any). Bash then expects a sequence of output lines to be
+// printed to stdout. If these lines all contain a common prefix
+// longer than the cursor word, bash will replace the cursor word
+// with that common prefix, and display nothing. If there isn't such
+// a common prefix, bash will display the lines in pages using 'more'.
+//
+// ** Strategy taken for command line completions:
+// If we can deduce either the exact flag intended, or a common flag
+// prefix, we'll output exactly that. Otherwise, if information
+// must be displayed to the user, we'll take the opportunity to add
+// some helpful information beyond just the flag name (specifically,
+// we'll include the default flag value and as much of the flag's
+// description as can fit on a single terminal line width, as specified
+// by the flag --tab_completion_columns). Furthermore, we'll try to
+// make bash order the output such that the most useful or relevent
+// flags are the most likely to be shown at the top.
+//
+// ** Additional features:
+// To assist in finding that one really useful flag, substring matching
+// was implemented. Before pressing a <TAB> to get completion for the
+// current word, you can append one or more '?' to the flag to do
+// substring matching. Here's the semantics:
+// --foo<TAB> Show me all flags with names prefixed by 'foo'
+// --foo?<TAB> Show me all flags with 'foo' somewhere in the name
+// --foo??<TAB> Same as prior case, but also search in module
+// definition path for 'foo'
+// --foo???<TAB> Same as prior case, but also search in flag
+// descriptions for 'foo'
+// Finally, we'll trim the output to a relatively small number of
+// flags to keep bash quiet about the verbosity of output. If one
+// really wanted to see all possible matches, appending a '+' to the
+// search word will force the exhaustive list of matches to be printed.
+//
+// ** How to have bash accept completions from a binary:
+// Bash requires that it be informed about each command that programmatic
+// completion should be enabled for. Example addition to a .bashrc
+// file would be (your path to gflags_completions.sh file may differ):
+
+/*
+$ complete -o bashdefault -o default -o nospace -C \
+ '/usr/local/bin/gflags_completions.sh --tab_completion_columns $COLUMNS' \
+ time env binary_name another_binary [...]
+*/
+
+// This would allow the following to work:
+// $ /path/to/binary_name --vmodule<TAB>
+// Or:
+// $ ./bin/path/another_binary --gfs_u<TAB>
+// (etc)
+//
+// Sadly, it appears that bash gives no easy way to force this behavior for
+// all commands. That's where the "time" in the above example comes in.
+// If you haven't specifically added a command to the list of completion
+// supported commands, you can still get completions by prefixing the
+// entire command with "env".
+// $ env /some/brand/new/binary --vmod<TAB>
+// Assuming that "binary" is a newly compiled binary, this should still
+// produce the expected completion output.
+
+
+#ifndef GOOGLE_GFLAGS_COMPLETIONS_H_
+#define GOOGLE_GFLAGS_COMPLETIONS_H_
+
+namespace google {
+
+void HandleCommandLineCompletions(void);
+
+}
+
+#endif // GOOGLE_GFLAGS_COMPLETIONS_H_
diff --git a/trunk/src/third_party/gflags/gen/arch/linux/ia32/include/private/config.h b/trunk/src/third_party/gflags/gen/arch/linux/ia32/include/private/config.h
new file mode 100644
index 0000000..b46ed79
--- /dev/null
+++ b/trunk/src/third_party/gflags/gen/arch/linux/ia32/include/private/config.h
@@ -0,0 +1,106 @@
+/* src/config.h. Generated from config.h.in by configure. */
+/* src/config.h.in. Generated from configure.ac by autoheader. */
+
+/* Always the empty-string on non-windows systems. On windows, should be
+ "__declspec(dllexport)". This way, when we compile the dll, we export our
+ functions/classes. It's safe to define this here because config.h is only
+ used internally, to compile the DLL, and every DLL source file #includes
+ "config.h" before anything else. */
+#define GFLAGS_DLL_DECL /**/
+
+/* Namespace for Google classes */
+#define GOOGLE_NAMESPACE ::google
+
+/* Define to 1 if you have the <dlfcn.h> header file. */
+#define HAVE_DLFCN_H 1
+
+/* Define to 1 if you have the <fnmatch.h> header file. */
+#define HAVE_FNMATCH_H 1
+
+/* Define to 1 if you have the <inttypes.h> header file. */
+#define HAVE_INTTYPES_H 1
+
+/* Define to 1 if you have the <memory.h> header file. */
+#define HAVE_MEMORY_H 1
+
+/* define if the compiler implements namespaces */
+#define HAVE_NAMESPACES 1
+
+/* Define if you have POSIX threads libraries and header files. */
+#define HAVE_PTHREAD 1
+
+/* Define to 1 if you have the `putenv' function. */
+#define HAVE_PUTENV 1
+
+/* Define to 1 if you have the `setenv' function. */
+#define HAVE_SETENV 1
+
+/* Define to 1 if you have the <stdint.h> header file. */
+#define HAVE_STDINT_H 1
+
+/* Define to 1 if you have the <stdlib.h> header file. */
+#define HAVE_STDLIB_H 1
+
+/* Define to 1 if you have the <strings.h> header file. */
+#define HAVE_STRINGS_H 1
+
+/* Define to 1 if you have the <string.h> header file. */
+#define HAVE_STRING_H 1
+
+/* Define to 1 if you have the `strtoll' function. */
+#define HAVE_STRTOLL 1
+
+/* Define to 1 if you have the `strtoq' function. */
+#define HAVE_STRTOQ 1
+
+/* Define to 1 if you have the <sys/stat.h> header file. */
+#define HAVE_SYS_STAT_H 1
+
+/* Define to 1 if you have the <sys/types.h> header file. */
+#define HAVE_SYS_TYPES_H 1
+
+/* Define to 1 if you have the <unistd.h> header file. */
+#define HAVE_UNISTD_H 1
+
+/* define if your compiler has __attribute__ */
+#define HAVE___ATTRIBUTE__ 1
+
+/* Name of package */
+#define PACKAGE "gflags"
+
+/* Define to the address where bug reports for this package should be sent. */
+#define PACKAGE_BUGREPORT "opensource@google.com"
+
+/* Define to the full name of this package. */
+#define PACKAGE_NAME "gflags"
+
+/* Define to the full name and version of this package. */
+#define PACKAGE_STRING "gflags 1.3"
+
+/* Define to the one symbol short name of this package. */
+#define PACKAGE_TARNAME "gflags"
+
+/* Define to the home page for this package. */
+#define PACKAGE_URL ""
+
+/* Define to the version of this package. */
+#define PACKAGE_VERSION "1.3"
+
+/* Define to necessary symbol if this constant uses a non-standard name on
+ your system. */
+/* #undef PTHREAD_CREATE_JOINABLE */
+
+/* Define to 1 if you have the ANSI C header files. */
+#define STDC_HEADERS 1
+
+/* the namespace where STL code like vector<> is defined */
+#define STL_NAMESPACE std
+
+/* Version number of package */
+#define VERSION "1.3"
+
+/* Stops putting the code inside the Google namespace */
+#define _END_GOOGLE_NAMESPACE_ }
+
+/* Puts following code inside the Google namespace */
+#define _START_GOOGLE_NAMESPACE_ namespace google {
diff --git a/trunk/src/third_party/gflags/gen/arch/linux/x64/include/gflags/gflags.h b/trunk/src/third_party/gflags/gen/arch/linux/x64/include/gflags/gflags.h
new file mode 100644
index 0000000..821d194
--- /dev/null
+++ b/trunk/src/third_party/gflags/gen/arch/linux/x64/include/gflags/gflags.h
@@ -0,0 +1,538 @@
+// Copyright (c) 2006, Google Inc.
+// All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions are
+// met:
+//
+// * Redistributions of source code must retain the above copyright
+// notice, this list of conditions and the following disclaimer.
+// * Redistributions in binary form must reproduce the above
+// copyright notice, this list of conditions and the following disclaimer
+// in the documentation and/or other materials provided with the
+// distribution.
+// * Neither the name of Google Inc. nor the names of its
+// contributors may be used to endorse or promote products derived from
+// this software without specific prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+// ---
+// Author: Ray Sidney
+// Revamped and reorganized by Craig Silverstein
+//
+// This is the file that should be included by any file which declares
+// or defines a command line flag or wants to parse command line flags
+// or print a program usage message (which will include information about
+// flags). Executive summary, in the form of an example foo.cc file:
+//
+// #include "foo.h" // foo.h has a line "DECLARE_int32(start);"
+//
+// DEFINE_int32(end, 1000, "The last record to read");
+// DECLARE_bool(verbose); // some other file has a DEFINE_bool(verbose, ...)
+//
+// void MyFunc() {
+// if (FLAGS_verbose) printf("Records %d-%d\n", FLAGS_start, FLAGS_end);
+// }
+//
+// Then, at the command-line:
+// ./foo --noverbose --start=5 --end=100
+//
+// For more details, see
+// doc/gflags.html
+//
+// --- A note about thread-safety:
+//
+// We describe many functions in this routine as being thread-hostile,
+// thread-compatible, or thread-safe. Here are the meanings we use:
+//
+// thread-safe: it is safe for multiple threads to call this routine
+// (or, when referring to a class, methods of this class)
+// concurrently.
+// thread-hostile: it is not safe for multiple threads to call this
+// routine (or methods of this class) concurrently. In gflags,
+// most thread-hostile routines are intended to be called early in,
+// or even before, main() -- that is, before threads are spawned.
+// thread-compatible: it is safe for multiple threads to read from
+// this variable (when applied to variables), or to call const
+// methods of this class (when applied to classes), as long as no
+// other thread is writing to the variable or calling non-const
+// methods of this class.
+
+#ifndef GOOGLE_GFLAGS_H_
+#define GOOGLE_GFLAGS_H_
+
+#include <string>
+#include <vector>
+
+// We care a lot about number of bits things take up. Unfortunately,
+// systems define their bit-specific ints in a lot of different ways.
+// We use our own way, and have a typedef to get there.
+// Note: these commands below may look like "#if 1" or "#if 0", but
+// that's because they were constructed that way at ./configure time.
+// Look at gflags.h.in to see how they're calculated (based on your config).
+#if 1
+#include <stdint.h> // the normal place uint16_t is defined
+#endif
+#if 1
+#include <sys/types.h> // the normal place u_int16_t is defined
+#endif
+#if 1
+#include <inttypes.h> // a third place for uint16_t or u_int16_t
+#endif
+
+namespace google {
+
+#if 1 // the C99 format
+typedef int32_t int32;
+typedef uint32_t uint32;
+typedef int64_t int64;
+typedef uint64_t uint64;
+#elif 1 // the BSD format
+typedef int32_t int32;
+typedef u_int32_t uint32;
+typedef int64_t int64;
+typedef u_int64_t uint64;
+#elif 0 // the windows (vc7) format
+typedef __int32 int32;
+typedef unsigned __int32 uint32;
+typedef __int64 int64;
+typedef unsigned __int64 uint64;
+#else
+#error Do not know how to define a 32-bit integer quantity on your system
+#endif
+
+// --------------------------------------------------------------------
+// To actually define a flag in a file, use DEFINE_bool,
+// DEFINE_string, etc. at the bottom of this file. You may also find
+// it useful to register a validator with the flag. This ensures that
+// when the flag is parsed from the commandline, or is later set via
+// SetCommandLineOption, we call the validation function.
+//
+// The validation function should return true if the flag value is valid, and
+// false otherwise. If the function returns false for the new setting of the
+// flag, the flag will retain its current value. If it returns false for the
+// default value, InitGoogle will die.
+//
+// This function is safe to call at global construct time (as in the
+// example below).
+//
+// Example use:
+// static bool ValidatePort(const char* flagname, int32 value) {
+// if (value > 0 && value < 32768) // value is ok
+// return true;
+// printf("Invalid value for --%s: %d\n", flagname, (int)value);
+// return false;
+// }
+// DEFINE_int32(port, 0, "What port to listen on");
+// static bool dummy = RegisterFlagValidator(&FLAGS_port, &ValidatePort);
+
+// Returns true if successfully registered, false if not (because the
+// first argument doesn't point to a command-line flag, or because a
+// validator is already registered for this flag).
+bool RegisterFlagValidator(const bool* flag,
+ bool (*validate_fn)(const char*, bool));
+bool RegisterFlagValidator(const int32* flag,
+ bool (*validate_fn)(const char*, int32));
+bool RegisterFlagValidator(const int64* flag,
+ bool (*validate_fn)(const char*, int64));
+bool RegisterFlagValidator(const uint64* flag,
+ bool (*validate_fn)(const char*, uint64));
+bool RegisterFlagValidator(const double* flag,
+ bool (*validate_fn)(const char*, double));
+bool RegisterFlagValidator(const std::string* flag,
+ bool (*validate_fn)(const char*, const std::string&));
+
+
+// --------------------------------------------------------------------
+// These methods are the best way to get access to info about the
+// list of commandline flags. Note that these routines are pretty slow.
+// GetAllFlags: mostly-complete info about the list, sorted by file.
+// ShowUsageWithFlags: pretty-prints the list to stdout (what --help does)
+// ShowUsageWithFlagsRestrict: limit to filenames with restrict as a substr
+//
+// In addition to accessing flags, you can also access argv[0] (the program
+// name) and argv (the entire commandline), which we sock away a copy of.
+// These variables are static, so you should only set them once.
+
+struct CommandLineFlagInfo {
+ std::string name; // the name of the flag
+ std::string type; // the type of the flag: int32, etc
+ std::string description; // the "help text" associated with the flag
+ std::string current_value; // the current value, as a string
+ std::string default_value; // the default value, as a string
+ std::string filename; // 'cleaned' version of filename holding the flag
+ bool has_validator_fn; // true if RegisterFlagValidator called on flag
+ bool is_default; // true if the flag has default value
+};
+
+// Using this inside of a validator is a recipe for a deadlock.
+// TODO(wojtekm) Fix locking when validators are running, to make it safe to
+// call validators during ParseAllFlags.
+// Also make sure then to uncomment the corresponding unit test in
+// commandlineflags_unittest.sh
+extern void GetAllFlags(std::vector<CommandLineFlagInfo>* OUTPUT);
+// These two are actually defined in commandlineflags_reporting.cc.
+extern void ShowUsageWithFlags(const char *argv0); // what --help does
+extern void ShowUsageWithFlagsRestrict(const char *argv0, const char *restrict);
+
+// Create a descriptive string for a flag.
+// Goes to some trouble to make pretty line breaks.
+extern std::string DescribeOneFlag(const CommandLineFlagInfo& flag);
+
+// Thread-hostile; meant to be called before any threads are spawned.
+extern void SetArgv(int argc, const char** argv);
+// The following functions are thread-safe as long as SetArgv() is
+// only called before any threads start.
+extern const std::vector<std::string>& GetArgvs(); // all of argv as a vector
+extern const char* GetArgv(); // all of argv as a string
+extern const char* GetArgv0(); // only argv0
+extern uint32 GetArgvSum(); // simple checksum of argv
+extern const char* ProgramInvocationName(); // argv0, or "UNKNOWN" if not set
+extern const char* ProgramInvocationShortName(); // basename(argv0)
+// ProgramUsage() is thread-safe as long as SetUsageMessage() is only
+// called before any threads start.
+extern const char* ProgramUsage(); // string set by SetUsageMessage()
+
+
+// --------------------------------------------------------------------
+// Normally you access commandline flags by just saying "if (FLAGS_foo)"
+// or whatever, and set them by calling "FLAGS_foo = bar" (or, more
+// commonly, via the DEFINE_foo macro). But if you need a bit more
+// control, we have programmatic ways to get/set the flags as well.
+// These programmatic ways to access flags are thread-safe, but direct
+// access is only thread-compatible.
+
+// Return true iff the flagname was found.
+// OUTPUT is set to the flag's value, or unchanged if we return false.
+extern bool GetCommandLineOption(const char* name, std::string* OUTPUT);
+
+// Return true iff the flagname was found. OUTPUT is set to the flag's
+// CommandLineFlagInfo or unchanged if we return false.
+extern bool GetCommandLineFlagInfo(const char* name,
+ CommandLineFlagInfo* OUTPUT);
+
+// Return the CommandLineFlagInfo of the flagname. exit() if name not found.
+// Example usage, to check if a flag's value is currently the default value:
+// if (GetCommandLineFlagInfoOrDie("foo").is_default) ...
+extern CommandLineFlagInfo GetCommandLineFlagInfoOrDie(const char* name);
+
+enum FlagSettingMode {
+ // update the flag's value (can call this multiple times).
+ SET_FLAGS_VALUE,
+ // update the flag's value, but *only if* it has not yet been updated
+ // with SET_FLAGS_VALUE, SET_FLAG_IF_DEFAULT, or "FLAGS_xxx = nondef".
+ SET_FLAG_IF_DEFAULT,
+ // set the flag's default value to this. If the flag has not yet updated
+ // yet (via SET_FLAGS_VALUE, SET_FLAG_IF_DEFAULT, or "FLAGS_xxx = nondef")
+ // change the flag's current value to the new default value as well.
+ SET_FLAGS_DEFAULT
+};
+
+// Set a particular flag ("command line option"). Returns a string
+// describing the new value that the option has been set to. The
+// return value API is not well-specified, so basically just depend on
+// it to be empty if the setting failed for some reason -- the name is
+// not a valid flag name, or the value is not a valid value -- and
+// non-empty else.
+
+// SetCommandLineOption uses set_mode == SET_FLAGS_VALUE (the common case)
+extern std::string SetCommandLineOption(const char* name, const char* value);
+extern std::string SetCommandLineOptionWithMode(const char* name, const char* value,
+ FlagSettingMode set_mode);
+
+
+// --------------------------------------------------------------------
+// Saves the states (value, default value, whether the user has set
+// the flag, registered validators, etc) of all flags, and restores
+// them when the FlagSaver is destroyed. This is very useful in
+// tests, say, when you want to let your tests change the flags, but
+// make sure that they get reverted to the original states when your
+// test is complete.
+//
+// Example usage:
+// void TestFoo() {
+// FlagSaver s1;
+// FLAG_foo = false;
+// FLAG_bar = "some value";
+//
+// // test happens here. You can return at any time
+// // without worrying about restoring the FLAG values.
+// }
+//
+// Note: This class is marked with __attribute__((unused)) because all the
+// work is done in the constructor and destructor, so in the standard
+// usage example above, the compiler would complain that it's an
+// unused variable.
+//
+// This class is thread-safe.
+
+class FlagSaver {
+ public:
+ FlagSaver();
+ ~FlagSaver();
+
+ private:
+ class FlagSaverImpl* impl_; // we use pimpl here to keep API steady
+
+ FlagSaver(const FlagSaver&); // no copying!
+ void operator=(const FlagSaver&);
+} __attribute__ ((unused));
+
+// --------------------------------------------------------------------
+// Some deprecated or hopefully-soon-to-be-deprecated functions.
+
+// This is often used for logging. TODO(csilvers): figure out a better way
+extern std::string CommandlineFlagsIntoString();
+// Usually where this is used, a FlagSaver should be used instead.
+extern bool ReadFlagsFromString(const std::string& flagfilecontents,
+ const char* prog_name,
+ bool errors_are_fatal); // uses SET_FLAGS_VALUE
+
+// These let you manually implement --flagfile functionality.
+// DEPRECATED.
+extern bool AppendFlagsIntoFile(const std::string& filename, const char* prog_name);
+extern bool SaveCommandFlags(); // actually defined in google.cc !
+extern bool ReadFromFlagsFile(const std::string& filename, const char* prog_name,
+ bool errors_are_fatal); // uses SET_FLAGS_VALUE
+
+
+// --------------------------------------------------------------------
+// Useful routines for initializing flags from the environment.
+// In each case, if 'varname' does not exist in the environment
+// return defval. If 'varname' does exist but is not valid
+// (e.g., not a number for an int32 flag), abort with an error.
+// Otherwise, return the value. NOTE: for booleans, for true use
+// 't' or 'T' or 'true' or '1', for false 'f' or 'F' or 'false' or '0'.
+
+extern bool BoolFromEnv(const char *varname, bool defval);
+extern int32 Int32FromEnv(const char *varname, int32 defval);
+extern int64 Int64FromEnv(const char *varname, int64 defval);
+extern uint64 Uint64FromEnv(const char *varname, uint64 defval);
+extern double DoubleFromEnv(const char *varname, double defval);
+extern const char *StringFromEnv(const char *varname, const char *defval);
+
+
+// --------------------------------------------------------------------
+// The next two functions parse commandlineflags from main():
+
+// Set the "usage" message for this program. For example:
+// string usage("This program does nothing. Sample usage:\n");
+// usage += argv[0] + " <uselessarg1> <uselessarg2>";
+// SetUsageMessage(usage);
+// Do not include commandline flags in the usage: we do that for you!
+// Thread-hostile; meant to be called before any threads are spawned.
+extern void SetUsageMessage(const std::string& usage);
+
+// Looks for flags in argv and parses them. Rearranges argv to put
+// flags first, or removes them entirely if remove_flags is true.
+// If a flag is defined more than once in the command line or flag
+// file, the last definition is used.
+// See top-of-file for more details on this function.
+#ifndef SWIG // In swig, use ParseCommandLineFlagsScript() instead.
+extern uint32 ParseCommandLineFlags(int *argc, char*** argv,
+ bool remove_flags);
+#endif
+
+
+// Calls to ParseCommandLineNonHelpFlags and then to
+// HandleCommandLineHelpFlags can be used instead of a call to
+// ParseCommandLineFlags during initialization, in order to allow for
+// changing default values for some FLAGS (via
+// e.g. SetCommandLineOptionWithMode calls) between the time of
+// command line parsing and the time of dumping help information for
+// the flags as a result of command line parsing.
+// If a flag is defined more than once in the command line or flag
+// file, the last definition is used.
+extern uint32 ParseCommandLineNonHelpFlags(int *argc, char*** argv,
+ bool remove_flags);
+// This is actually defined in commandlineflags_reporting.cc.
+// This function is misnamed (it also handles --version, etc.), but
+// it's too late to change that now. :-(
+extern void HandleCommandLineHelpFlags(); // in commandlineflags_reporting.cc
+
+// Allow command line reparsing. Disables the error normally
+// generated when an unknown flag is found, since it may be found in a
+// later parse. Thread-hostile; meant to be called before any threads
+// are spawned.
+extern void AllowCommandLineReparsing();
+
+// Reparse the flags that have not yet been recognized.
+// Only flags registered since the last parse will be recognized.
+// Any flag value must be provided as part of the argument using "=",
+// not as a separate command line argument that follows the flag argument.
+// Intended for handling flags from dynamically loaded libraries,
+// since their flags are not registered until they are loaded.
+extern uint32 ReparseCommandLineNonHelpFlags();
+
+
+// --------------------------------------------------------------------
+// Now come the command line flag declaration/definition macros that
+// will actually be used. They're kind of hairy. A major reason
+// for this is initialization: we want people to be able to access
+// variables in global constructors and have that not crash, even if
+// their global constructor runs before the global constructor here.
+// (Obviously, we can't guarantee the flags will have the correct
+// default value in that case, but at least accessing them is safe.)
+// The only way to do that is have flags point to a static buffer.
+// So we make one, using a union to ensure proper alignment, and
+// then use placement-new to actually set up the flag with the
+// correct default value. In the same vein, we have to worry about
+// flag access in global destructors, so FlagRegisterer has to be
+// careful never to destroy the flag-values it constructs.
+//
+// Note that when we define a flag variable FLAGS_<name>, we also
+// preemptively define a junk variable, FLAGS_no<name>. This is to
+// cause a link-time error if someone tries to define 2 flags with
+// names like "logging" and "nologging". We do this because a bool
+// flag FLAG can be set from the command line to true with a "-FLAG"
+// argument, and to false with a "-noFLAG" argument, and so this can
+// potentially avert confusion.
+//
+// We also put flags into their own namespace. It is purposefully
+// named in an opaque way that people should have trouble typing
+// directly. The idea is that DEFINE puts the flag in the weird
+// namespace, and DECLARE imports the flag from there into the current
+// namespace. The net result is to force people to use DECLARE to get
+// access to a flag, rather than saying "extern bool FLAGS_whatever;"
+// or some such instead. We want this so we can put extra
+// functionality (like sanity-checking) in DECLARE if we want, and
+// make sure it is picked up everywhere.
+//
+// We also put the type of the variable in the namespace, so that
+// people can't DECLARE_int32 something that they DEFINE_bool'd
+// elsewhere.
+
+class FlagRegisterer {
+ public:
+ FlagRegisterer(const char* name, const char* type,
+ const char* help, const char* filename,
+ void* current_storage, void* defvalue_storage);
+};
+
+extern bool FlagsTypeWarn(const char *name);
+
+// If your application #defines STRIP_FLAG_HELP to a non-zero value
+// before #including this file, we remove the help message from the
+// binary file. This can reduce the size of the resulting binary
+// somewhat, and may also be useful for security reasons.
+
+extern const char kStrippedFlagHelp[];
+
+}
+
+#ifndef SWIG // In swig, ignore the main flag declarations
+
+#if defined(STRIP_FLAG_HELP) && STRIP_FLAG_HELP > 0
+// Need this construct to avoid the 'defined but not used' warning.
+#define MAYBE_STRIPPED_HELP(txt) (false ? (txt) : kStrippedFlagHelp)
+#else
+#define MAYBE_STRIPPED_HELP(txt) txt
+#endif
+
+// Each command-line flag has two variables associated with it: one
+// with the current value, and one with the default value. However,
+// we have a third variable, which is where value is assigned; it's a
+// constant. This guarantees that FLAG_##value is initialized at
+// static initialization time (e.g. before program-start) rather than
+// than global construction time (which is after program-start but
+// before main), at least when 'value' is a compile-time constant. We
+// use a small trick for the "default value" variable, and call it
+// FLAGS_no<name>. This serves the second purpose of assuring a
+// compile error if someone tries to define a flag named no<name>
+// which is illegal (--foo and --nofoo both affect the "foo" flag).
+#define DEFINE_VARIABLE(type, shorttype, name, value, help) \
+ namespace fL##shorttype { \
+ static const type FLAGS_nono##name = value; \
+ type FLAGS_##name = FLAGS_nono##name; \
+ type FLAGS_no##name = FLAGS_nono##name; \
+ static ::google::FlagRegisterer o_##name( \
+ #name, #type, MAYBE_STRIPPED_HELP(help), __FILE__, \
+ &FLAGS_##name, &FLAGS_no##name); \
+ } \
+ using fL##shorttype::FLAGS_##name
+
+#define DECLARE_VARIABLE(type, shorttype, name) \
+ namespace fL##shorttype { \
+ extern type FLAGS_##name; \
+ } \
+ using fL##shorttype::FLAGS_##name
+
+// For DEFINE_bool, we want to do the extra check that the passed-in
+// value is actually a bool, and not a string or something that can be
+// coerced to a bool. These declarations (no definition needed!) will
+// help us do that, and never evaluate From, which is important.
+// We'll use 'sizeof(IsBool(val))' to distinguish. This code requires
+// that the compiler have different sizes for bool & double. Since
+// this is not guaranteed by the standard, we check it with a
+// compile-time assert (msg[-1] will give a compile-time error).
+namespace fLB {
+struct CompileAssert {};
+typedef CompileAssert expected_sizeof_double_neq_sizeof_bool[
+ (sizeof(double) != sizeof(bool)) ? 1 : -1];
+template<typename From> double IsBoolFlag(const From& from);
+bool IsBoolFlag(bool from);
+} // namespace fLB
+
+#define DECLARE_bool(name) DECLARE_VARIABLE(bool, B, name)
+#define DEFINE_bool(name, val, txt) \
+ namespace fLB { \
+ typedef CompileAssert FLAG_##name##_value_is_not_a_bool[ \
+ (sizeof(::fLB::IsBoolFlag(val)) != sizeof(double)) ? 1 : -1]; \
+ } \
+ DEFINE_VARIABLE(bool, B, name, val, txt)
+
+#define DECLARE_int32(name) DECLARE_VARIABLE(::google::int32, I, name)
+#define DEFINE_int32(name,val,txt) DEFINE_VARIABLE(::google::int32, I, name, val, txt)
+
+#define DECLARE_int64(name) DECLARE_VARIABLE(::google::int64, I64, name)
+#define DEFINE_int64(name,val,txt) DEFINE_VARIABLE(::google::int64, I64, name, val, txt)
+
+#define DECLARE_uint64(name) DECLARE_VARIABLE(::google::uint64, U64, name)
+#define DEFINE_uint64(name,val,txt) DEFINE_VARIABLE(::google::uint64, U64, name, val, txt)
+
+#define DECLARE_double(name) DECLARE_VARIABLE(double, D, name)
+#define DEFINE_double(name, val, txt) DEFINE_VARIABLE(double, D, name, val, txt)
+
+// Strings are trickier, because they're not a POD, so we can't
+// construct them at static-initialization time (instead they get
+// constructed at global-constructor time, which is much later). To
+// try to avoid crashes in that case, we use a char buffer to store
+// the string, which we can static-initialize, and then placement-new
+// into it later. It's not perfect, but the best we can do.
+#define DECLARE_string(name) namespace fLS { extern std::string& FLAGS_##name; } \
+ using fLS::FLAGS_##name
+
+// We need to define a var named FLAGS_no##name so people don't define
+// --string and --nostring. And we need a temporary place to put val
+// so we don't have to evaluate it twice. Two great needs that go
+// great together!
+// The weird 'using' + 'extern' inside the fLS namespace is to work around
+// an unknown compiler bug/issue with the gcc 4.2.1 on SUSE 10. See
+// http://code.google.com/p/google-gflags/issues/detail?id=20
+#define DEFINE_string(name, val, txt) \
+ namespace fLS { \
+ static union { void* align; char s[sizeof(std::string)]; } s_##name[2]; \
+ const std::string* const FLAGS_no##name = new (s_##name[0].s) std::string(val); \
+ static ::google::FlagRegisterer o_##name( \
+ #name, "string", MAYBE_STRIPPED_HELP(txt), __FILE__, \
+ s_##name[0].s, new (s_##name[1].s) std::string(*FLAGS_no##name)); \
+ extern std::string& FLAGS_##name; \
+ using fLS::FLAGS_##name; \
+ std::string& FLAGS_##name = *(reinterpret_cast<std::string*>(s_##name[0].s)); \
+ } \
+ using fLS::FLAGS_##name
+
+#endif // SWIG
+
+#endif // GOOGLE_GFLAGS_H_
diff --git a/trunk/src/third_party/gflags/gen/arch/linux/x64/include/gflags/gflags_completions.h b/trunk/src/third_party/gflags/gen/arch/linux/x64/include/gflags/gflags_completions.h
new file mode 100644
index 0000000..9d9ce7a
--- /dev/null
+++ b/trunk/src/third_party/gflags/gen/arch/linux/x64/include/gflags/gflags_completions.h
@@ -0,0 +1,121 @@
+// Copyright (c) 2008, Google Inc.
+// All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions are
+// met:
+//
+// * Redistributions of source code must retain the above copyright
+// notice, this list of conditions and the following disclaimer.
+// * Redistributions in binary form must reproduce the above
+// copyright notice, this list of conditions and the following disclaimer
+// in the documentation and/or other materials provided with the
+// distribution.
+// * Neither the name of Google Inc. nor the names of its
+// contributors may be used to endorse or promote products derived from
+// this software without specific prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+//
+// ---
+// Author: Dave Nicponski
+//
+// Implement helpful bash-style command line flag completions
+//
+// ** Functional API:
+// HandleCommandLineCompletions() should be called early during
+// program startup, but after command line flag code has been
+// initialized, such as the beginning of HandleCommandLineHelpFlags().
+// It checks the value of the flag --tab_completion_word. If this
+// flag is empty, nothing happens here. If it contains a string,
+// however, then HandleCommandLineCompletions() will hijack the
+// process, attempting to identify the intention behind this
+// completion. Regardless of the outcome of this deduction, the
+// process will be terminated, similar to --helpshort flag
+// handling.
+//
+// ** Overview of Bash completions:
+// Bash can be told to programatically determine completions for the
+// current 'cursor word'. It does this by (in this case) invoking a
+// command with some additional arguments identifying the command
+// being executed, the word being completed, and the previous word
+// (if any). Bash then expects a sequence of output lines to be
+// printed to stdout. If these lines all contain a common prefix
+// longer than the cursor word, bash will replace the cursor word
+// with that common prefix, and display nothing. If there isn't such
+// a common prefix, bash will display the lines in pages using 'more'.
+//
+// ** Strategy taken for command line completions:
+// If we can deduce either the exact flag intended, or a common flag
+// prefix, we'll output exactly that. Otherwise, if information
+// must be displayed to the user, we'll take the opportunity to add
+// some helpful information beyond just the flag name (specifically,
+// we'll include the default flag value and as much of the flag's
+// description as can fit on a single terminal line width, as specified
+// by the flag --tab_completion_columns). Furthermore, we'll try to
+// make bash order the output such that the most useful or relevent
+// flags are the most likely to be shown at the top.
+//
+// ** Additional features:
+// To assist in finding that one really useful flag, substring matching
+// was implemented. Before pressing a <TAB> to get completion for the
+// current word, you can append one or more '?' to the flag to do
+// substring matching. Here's the semantics:
+// --foo<TAB> Show me all flags with names prefixed by 'foo'
+// --foo?<TAB> Show me all flags with 'foo' somewhere in the name
+// --foo??<TAB> Same as prior case, but also search in module
+// definition path for 'foo'
+// --foo???<TAB> Same as prior case, but also search in flag
+// descriptions for 'foo'
+// Finally, we'll trim the output to a relatively small number of
+// flags to keep bash quiet about the verbosity of output. If one
+// really wanted to see all possible matches, appending a '+' to the
+// search word will force the exhaustive list of matches to be printed.
+//
+// ** How to have bash accept completions from a binary:
+// Bash requires that it be informed about each command that programmatic
+// completion should be enabled for. Example addition to a .bashrc
+// file would be (your path to gflags_completions.sh file may differ):
+
+/*
+$ complete -o bashdefault -o default -o nospace -C \
+ '/usr/local/bin/gflags_completions.sh --tab_completion_columns $COLUMNS' \
+ time env binary_name another_binary [...]
+*/
+
+// This would allow the following to work:
+// $ /path/to/binary_name --vmodule<TAB>
+// Or:
+// $ ./bin/path/another_binary --gfs_u<TAB>
+// (etc)
+//
+// Sadly, it appears that bash gives no easy way to force this behavior for
+// all commands. That's where the "time" in the above example comes in.
+// If you haven't specifically added a command to the list of completion
+// supported commands, you can still get completions by prefixing the
+// entire command with "env".
+// $ env /some/brand/new/binary --vmod<TAB>
+// Assuming that "binary" is a newly compiled binary, this should still
+// produce the expected completion output.
+
+
+#ifndef GOOGLE_GFLAGS_COMPLETIONS_H_
+#define GOOGLE_GFLAGS_COMPLETIONS_H_
+
+namespace google {
+
+void HandleCommandLineCompletions(void);
+
+}
+
+#endif // GOOGLE_GFLAGS_COMPLETIONS_H_
diff --git a/trunk/src/third_party/gflags/gen/arch/linux/x64/include/private/config.h b/trunk/src/third_party/gflags/gen/arch/linux/x64/include/private/config.h
new file mode 100644
index 0000000..b46ed79
--- /dev/null
+++ b/trunk/src/third_party/gflags/gen/arch/linux/x64/include/private/config.h
@@ -0,0 +1,106 @@
+/* src/config.h. Generated from config.h.in by configure. */
+/* src/config.h.in. Generated from configure.ac by autoheader. */
+
+/* Always the empty-string on non-windows systems. On windows, should be
+ "__declspec(dllexport)". This way, when we compile the dll, we export our
+ functions/classes. It's safe to define this here because config.h is only
+ used internally, to compile the DLL, and every DLL source file #includes
+ "config.h" before anything else. */
+#define GFLAGS_DLL_DECL /**/
+
+/* Namespace for Google classes */
+#define GOOGLE_NAMESPACE ::google
+
+/* Define to 1 if you have the <dlfcn.h> header file. */
+#define HAVE_DLFCN_H 1
+
+/* Define to 1 if you have the <fnmatch.h> header file. */
+#define HAVE_FNMATCH_H 1
+
+/* Define to 1 if you have the <inttypes.h> header file. */
+#define HAVE_INTTYPES_H 1
+
+/* Define to 1 if you have the <memory.h> header file. */
+#define HAVE_MEMORY_H 1
+
+/* define if the compiler implements namespaces */
+#define HAVE_NAMESPACES 1
+
+/* Define if you have POSIX threads libraries and header files. */
+#define HAVE_PTHREAD 1
+
+/* Define to 1 if you have the `putenv' function. */
+#define HAVE_PUTENV 1
+
+/* Define to 1 if you have the `setenv' function. */
+#define HAVE_SETENV 1
+
+/* Define to 1 if you have the <stdint.h> header file. */
+#define HAVE_STDINT_H 1
+
+/* Define to 1 if you have the <stdlib.h> header file. */
+#define HAVE_STDLIB_H 1
+
+/* Define to 1 if you have the <strings.h> header file. */
+#define HAVE_STRINGS_H 1
+
+/* Define to 1 if you have the <string.h> header file. */
+#define HAVE_STRING_H 1
+
+/* Define to 1 if you have the `strtoll' function. */
+#define HAVE_STRTOLL 1
+
+/* Define to 1 if you have the `strtoq' function. */
+#define HAVE_STRTOQ 1
+
+/* Define to 1 if you have the <sys/stat.h> header file. */
+#define HAVE_SYS_STAT_H 1
+
+/* Define to 1 if you have the <sys/types.h> header file. */
+#define HAVE_SYS_TYPES_H 1
+
+/* Define to 1 if you have the <unistd.h> header file. */
+#define HAVE_UNISTD_H 1
+
+/* define if your compiler has __attribute__ */
+#define HAVE___ATTRIBUTE__ 1
+
+/* Name of package */
+#define PACKAGE "gflags"
+
+/* Define to the address where bug reports for this package should be sent. */
+#define PACKAGE_BUGREPORT "opensource@google.com"
+
+/* Define to the full name of this package. */
+#define PACKAGE_NAME "gflags"
+
+/* Define to the full name and version of this package. */
+#define PACKAGE_STRING "gflags 1.3"
+
+/* Define to the one symbol short name of this package. */
+#define PACKAGE_TARNAME "gflags"
+
+/* Define to the home page for this package. */
+#define PACKAGE_URL ""
+
+/* Define to the version of this package. */
+#define PACKAGE_VERSION "1.3"
+
+/* Define to necessary symbol if this constant uses a non-standard name on
+ your system. */
+/* #undef PTHREAD_CREATE_JOINABLE */
+
+/* Define to 1 if you have the ANSI C header files. */
+#define STDC_HEADERS 1
+
+/* the namespace where STL code like vector<> is defined */
+#define STL_NAMESPACE std
+
+/* Version number of package */
+#define VERSION "1.3"
+
+/* Stops putting the code inside the Google namespace */
+#define _END_GOOGLE_NAMESPACE_ }
+
+/* Puts following code inside the Google namespace */
+#define _START_GOOGLE_NAMESPACE_ namespace google {
diff --git a/trunk/src/third_party/gflags/gen/arch/linux/x64/src/config.h b/trunk/src/third_party/gflags/gen/arch/linux/x64/src/config.h
new file mode 100644
index 0000000..b46ed79
--- /dev/null
+++ b/trunk/src/third_party/gflags/gen/arch/linux/x64/src/config.h
@@ -0,0 +1,106 @@
+/* src/config.h. Generated from config.h.in by configure. */
+/* src/config.h.in. Generated from configure.ac by autoheader. */
+
+/* Always the empty-string on non-windows systems. On windows, should be
+ "__declspec(dllexport)". This way, when we compile the dll, we export our
+ functions/classes. It's safe to define this here because config.h is only
+ used internally, to compile the DLL, and every DLL source file #includes
+ "config.h" before anything else. */
+#define GFLAGS_DLL_DECL /**/
+
+/* Namespace for Google classes */
+#define GOOGLE_NAMESPACE ::google
+
+/* Define to 1 if you have the <dlfcn.h> header file. */
+#define HAVE_DLFCN_H 1
+
+/* Define to 1 if you have the <fnmatch.h> header file. */
+#define HAVE_FNMATCH_H 1
+
+/* Define to 1 if you have the <inttypes.h> header file. */
+#define HAVE_INTTYPES_H 1
+
+/* Define to 1 if you have the <memory.h> header file. */
+#define HAVE_MEMORY_H 1
+
+/* define if the compiler implements namespaces */
+#define HAVE_NAMESPACES 1
+
+/* Define if you have POSIX threads libraries and header files. */
+#define HAVE_PTHREAD 1
+
+/* Define to 1 if you have the `putenv' function. */
+#define HAVE_PUTENV 1
+
+/* Define to 1 if you have the `setenv' function. */
+#define HAVE_SETENV 1
+
+/* Define to 1 if you have the <stdint.h> header file. */
+#define HAVE_STDINT_H 1
+
+/* Define to 1 if you have the <stdlib.h> header file. */
+#define HAVE_STDLIB_H 1
+
+/* Define to 1 if you have the <strings.h> header file. */
+#define HAVE_STRINGS_H 1
+
+/* Define to 1 if you have the <string.h> header file. */
+#define HAVE_STRING_H 1
+
+/* Define to 1 if you have the `strtoll' function. */
+#define HAVE_STRTOLL 1
+
+/* Define to 1 if you have the `strtoq' function. */
+#define HAVE_STRTOQ 1
+
+/* Define to 1 if you have the <sys/stat.h> header file. */
+#define HAVE_SYS_STAT_H 1
+
+/* Define to 1 if you have the <sys/types.h> header file. */
+#define HAVE_SYS_TYPES_H 1
+
+/* Define to 1 if you have the <unistd.h> header file. */
+#define HAVE_UNISTD_H 1
+
+/* define if your compiler has __attribute__ */
+#define HAVE___ATTRIBUTE__ 1
+
+/* Name of package */
+#define PACKAGE "gflags"
+
+/* Define to the address where bug reports for this package should be sent. */
+#define PACKAGE_BUGREPORT "opensource@google.com"
+
+/* Define to the full name of this package. */
+#define PACKAGE_NAME "gflags"
+
+/* Define to the full name and version of this package. */
+#define PACKAGE_STRING "gflags 1.3"
+
+/* Define to the one symbol short name of this package. */
+#define PACKAGE_TARNAME "gflags"
+
+/* Define to the home page for this package. */
+#define PACKAGE_URL ""
+
+/* Define to the version of this package. */
+#define PACKAGE_VERSION "1.3"
+
+/* Define to necessary symbol if this constant uses a non-standard name on
+ your system. */
+/* #undef PTHREAD_CREATE_JOINABLE */
+
+/* Define to 1 if you have the ANSI C header files. */
+#define STDC_HEADERS 1
+
+/* the namespace where STL code like vector<> is defined */
+#define STL_NAMESPACE std
+
+/* Version number of package */
+#define VERSION "1.3"
+
+/* Stops putting the code inside the Google namespace */
+#define _END_GOOGLE_NAMESPACE_ }
+
+/* Puts following code inside the Google namespace */
+#define _START_GOOGLE_NAMESPACE_ namespace google {
diff --git a/trunk/src/third_party/gflags/gflags.diff b/trunk/src/third_party/gflags/gflags.diff
new file mode 100644
index 0000000..aeab940
--- /dev/null
+++ b/trunk/src/third_party/gflags/gflags.diff
@@ -0,0 +1,64 @@
+Index: gflags.cc
+===================================================================
+--- gflags.cc (revision 42)
++++ gflags.cc (working copy)
+@@ -275,6 +275,10 @@
+ }
+
+ FlagValue::~FlagValue() {
++ /*
++ This class doesn't really own its value_buffer. I suspect that
++ this destructor is never called, otherwise it would surely crash.
++
+ switch (type_) {
+ case FV_BOOL: delete reinterpret_cast<bool*>(value_buffer_); break;
+ case FV_INT32: delete reinterpret_cast<int32*>(value_buffer_); break;
+@@ -283,6 +287,7 @@
+ case FV_DOUBLE: delete reinterpret_cast<double*>(value_buffer_); break;
+ case FV_STRING: delete reinterpret_cast<string*>(value_buffer_); break;
+ }
++ */
+ }
+
+ bool FlagValue::ParseFrom(const char* value) {
+@@ -611,6 +616,12 @@
+ class FlagRegistry {
+ public:
+ FlagRegistry() { }
++ ~FlagRegistry() {
++ for (FlagMap::iterator p = flags_.begin(), e = flags_.end(); p != e; ++p) {
++ CommandLineFlag* flag = p->second;
++ delete flag;
++ }
++ }
+
+ void Lock() { lock_.Lock(); }
+ void Unlock() { lock_.Unlock(); }
+@@ -642,6 +653,12 @@
+ FlagSettingMode set_mode, string* msg);
+
+ static FlagRegistry* GlobalRegistry(); // returns a singleton registry
++ static void Cleanup() {
++ if (global_registry_ != NULL) {
++ delete global_registry_;
++ global_registry_ = NULL;
++ }
++ }
+
+ private:
+ friend class GOOGLE_NAMESPACE::FlagSaverImpl; // reads all the flags in order to copy them
+@@ -679,6 +696,14 @@
+ return global_registry_;
+ }
+
++// It would be preferable to provide a cleanup method, rather than using
++// a static destructor, but this is easier to patch in.
++class CleanupGlobalRegistry {
++ public:
++ ~CleanupGlobalRegistry() { FlagRegistry::Cleanup(); }
++};
++CleanupGlobalRegistry cleanup_global_registry;
++
+ void FlagRegistry::RegisterFlag(CommandLineFlag* flag) {
+ Lock();
+ pair<FlagIterator, bool> ins =
diff --git a/trunk/src/third_party/gflags/gflags.gyp b/trunk/src/third_party/gflags/gflags.gyp
new file mode 100644
index 0000000..9c6dff6
--- /dev/null
+++ b/trunk/src/third_party/gflags/gflags.gyp
@@ -0,0 +1,42 @@
+# Copyright 2010 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'variables': {
+ 'gflags_root': '<(DEPTH)/third_party/gflags',
+ 'gflags_gen_arch_root': '<(gflags_root)/gen/arch/<(OS)/<(target_arch)',
+ },
+ 'targets': [
+ {
+ 'target_name': 'gflags',
+ 'type': '<(library)',
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '<(gflags_gen_arch_root)/include', # For configured files.
+ '<(gflags_root)/src', # For everything else.
+ ],
+ },
+ 'include_dirs': [
+ '<(gflags_gen_arch_root)/include/private', # For config.h
+ '<(gflags_gen_arch_root)/include', # For configured files.
+ '<(gflags_root)/src', # For everything else.
+ ],
+ 'sources': [
+ 'src/gflags.cc',
+ 'src/gflags_completions.cc',
+ 'src/gflags_reporting.cc',
+ ],
+ },
+ ],
+}
diff --git a/trunk/src/third_party/google-sparsehash/gen/arch/linux/ia32/include/google/sparsehash/sparseconfig.h b/trunk/src/third_party/google-sparsehash/gen/arch/linux/ia32/include/google/sparsehash/sparseconfig.h
new file mode 100644
index 0000000..f397d3b
--- /dev/null
+++ b/trunk/src/third_party/google-sparsehash/gen/arch/linux/ia32/include/google/sparsehash/sparseconfig.h
@@ -0,0 +1,49 @@
+/*
+ * NOTE: This file is for internal use only.
+ * Do not use these #defines in your own program!
+ */
+
+/* Namespace for Google classes */
+#define GOOGLE_NAMESPACE ::google
+
+/* the location of the header defining hash functions */
+#define HASH_FUN_H <tr1/functional>
+
+/* the namespace of the hash<> function */
+#define HASH_NAMESPACE std::tr1
+
+/* Define to 1 if you have the <inttypes.h> header file. */
+#define HAVE_INTTYPES_H 1
+
+/* Define to 1 if the system has the type `long long'. */
+#define HAVE_LONG_LONG 1
+
+/* Define to 1 if you have the `memcpy' function. */
+#define HAVE_MEMCPY 1
+
+/* Define to 1 if you have the <stdint.h> header file. */
+#define HAVE_STDINT_H 1
+
+/* Define to 1 if you have the <sys/types.h> header file. */
+#define HAVE_SYS_TYPES_H 1
+
+/* Define to 1 if the system has the type `uint16_t'. */
+#define HAVE_UINT16_T 1
+
+/* Define to 1 if the system has the type `u_int16_t'. */
+#define HAVE_U_INT16_T 1
+
+/* Define to 1 if the system has the type `__uint16'. */
+/* #undef HAVE___UINT16 */
+
+/* The system-provided hash function including the namespace. */
+#define SPARSEHASH_HASH HASH_NAMESPACE::hash
+
+/* the namespace where STL code like vector<> is defined */
+#define STL_NAMESPACE std
+
+/* Stops putting the code inside the Google namespace */
+#define _END_GOOGLE_NAMESPACE_ }
+
+/* Puts following code inside the Google namespace */
+#define _START_GOOGLE_NAMESPACE_ namespace google {
diff --git a/trunk/src/third_party/google-sparsehash/gen/arch/linux/x64/include/google/sparsehash/sparseconfig.h b/trunk/src/third_party/google-sparsehash/gen/arch/linux/x64/include/google/sparsehash/sparseconfig.h
new file mode 100644
index 0000000..f397d3b
--- /dev/null
+++ b/trunk/src/third_party/google-sparsehash/gen/arch/linux/x64/include/google/sparsehash/sparseconfig.h
@@ -0,0 +1,49 @@
+/*
+ * NOTE: This file is for internal use only.
+ * Do not use these #defines in your own program!
+ */
+
+/* Namespace for Google classes */
+#define GOOGLE_NAMESPACE ::google
+
+/* the location of the header defining hash functions */
+#define HASH_FUN_H <tr1/functional>
+
+/* the namespace of the hash<> function */
+#define HASH_NAMESPACE std::tr1
+
+/* Define to 1 if you have the <inttypes.h> header file. */
+#define HAVE_INTTYPES_H 1
+
+/* Define to 1 if the system has the type `long long'. */
+#define HAVE_LONG_LONG 1
+
+/* Define to 1 if you have the `memcpy' function. */
+#define HAVE_MEMCPY 1
+
+/* Define to 1 if you have the <stdint.h> header file. */
+#define HAVE_STDINT_H 1
+
+/* Define to 1 if you have the <sys/types.h> header file. */
+#define HAVE_SYS_TYPES_H 1
+
+/* Define to 1 if the system has the type `uint16_t'. */
+#define HAVE_UINT16_T 1
+
+/* Define to 1 if the system has the type `u_int16_t'. */
+#define HAVE_U_INT16_T 1
+
+/* Define to 1 if the system has the type `__uint16'. */
+/* #undef HAVE___UINT16 */
+
+/* The system-provided hash function including the namespace. */
+#define SPARSEHASH_HASH HASH_NAMESPACE::hash
+
+/* the namespace where STL code like vector<> is defined */
+#define STL_NAMESPACE std
+
+/* Stops putting the code inside the Google namespace */
+#define _END_GOOGLE_NAMESPACE_ }
+
+/* Puts following code inside the Google namespace */
+#define _START_GOOGLE_NAMESPACE_ namespace google {
diff --git a/trunk/src/third_party/google-sparsehash/google-sparsehash.gyp b/trunk/src/third_party/google-sparsehash/google-sparsehash.gyp
new file mode 100644
index 0000000..4df3352
--- /dev/null
+++ b/trunk/src/third_party/google-sparsehash/google-sparsehash.gyp
@@ -0,0 +1,32 @@
+# Copyright 2010 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'variables': {
+ 'sparsehash_root': '<(DEPTH)/third_party/google-sparsehash',
+ 'sparsehash_gen_arch_root': '<(sparsehash_root)/gen/arch/<(OS)/<(target_arch)',
+ },
+ 'targets': [
+ {
+ 'target_name': 'include',
+ 'type': 'none',
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '<(sparsehash_gen_arch_root)/include', # For sparseconfig.h
+ '<(sparsehash_root)/src', # For everything else.
+ ],
+ },
+ },
+ ],
+}
diff --git a/trunk/src/third_party/libpng/LICENSE b/trunk/src/third_party/libpng/LICENSE
new file mode 100644
index 0000000..e5561c2
--- /dev/null
+++ b/trunk/src/third_party/libpng/LICENSE
@@ -0,0 +1,111 @@
+
+This copy of the libpng notices is provided for your convenience. In case of
+any discrepancy between this copy and the notices in the file png.h that is
+included in the libpng distribution, the latter shall prevail.
+
+COPYRIGHT NOTICE, DISCLAIMER, and LICENSE:
+
+If you modify libpng you may insert additional notices immediately following
+this sentence.
+
+This code is released under the libpng license.
+
+libpng versions 1.2.6, August 15, 2004, through 1.2.44, June 26, 2010, are
+Copyright (c) 2004, 2006-2009 Glenn Randers-Pehrson, and are
+distributed according to the same disclaimer and license as libpng-1.2.5
+with the following individual added to the list of Contributing Authors
+
+ Cosmin Truta
+
+libpng versions 1.0.7, July 1, 2000, through 1.2.5 - October 3, 2002, are
+Copyright (c) 2000-2002 Glenn Randers-Pehrson, and are
+distributed according to the same disclaimer and license as libpng-1.0.6
+with the following individuals added to the list of Contributing Authors
+
+ Simon-Pierre Cadieux
+ Eric S. Raymond
+ Gilles Vollant
+
+and with the following additions to the disclaimer:
+
+ There is no warranty against interference with your enjoyment of the
+ library or against infringement. There is no warranty that our
+ efforts or the library will fulfill any of your particular purposes
+ or needs. This library is provided with all faults, and the entire
+ risk of satisfactory quality, performance, accuracy, and effort is with
+ the user.
+
+libpng versions 0.97, January 1998, through 1.0.6, March 20, 2000, are
+Copyright (c) 1998, 1999 Glenn Randers-Pehrson, and are
+distributed according to the same disclaimer and license as libpng-0.96,
+with the following individuals added to the list of Contributing Authors:
+
+ Tom Lane
+ Glenn Randers-Pehrson
+ Willem van Schaik
+
+libpng versions 0.89, June 1996, through 0.96, May 1997, are
+Copyright (c) 1996, 1997 Andreas Dilger
+Distributed according to the same disclaimer and license as libpng-0.88,
+with the following individuals added to the list of Contributing Authors:
+
+ John Bowler
+ Kevin Bracey
+ Sam Bushell
+ Magnus Holmgren
+ Greg Roelofs
+ Tom Tanner
+
+libpng versions 0.5, May 1995, through 0.88, January 1996, are
+Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.
+
+For the purposes of this copyright and license, "Contributing Authors"
+is defined as the following set of individuals:
+
+ Andreas Dilger
+ Dave Martindale
+ Guy Eric Schalnat
+ Paul Schmidt
+ Tim Wegner
+
+The PNG Reference Library is supplied "AS IS". The Contributing Authors
+and Group 42, Inc. disclaim all warranties, expressed or implied,
+including, without limitation, the warranties of merchantability and of
+fitness for any purpose. The Contributing Authors and Group 42, Inc.
+assume no liability for direct, indirect, incidental, special, exemplary,
+or consequential damages, which may result from the use of the PNG
+Reference Library, even if advised of the possibility of such damage.
+
+Permission is hereby granted to use, copy, modify, and distribute this
+source code, or portions hereof, for any purpose, without fee, subject
+to the following restrictions:
+
+1. The origin of this source code must not be misrepresented.
+
+2. Altered versions must be plainly marked as such and must not
+ be misrepresented as being the original source.
+
+3. This Copyright notice may not be removed or altered from any
+ source or altered source distribution.
+
+The Contributing Authors and Group 42, Inc. specifically permit, without
+fee, and encourage the use of this source code as a component to
+supporting the PNG file format in commercial products. If you use this
+source code in a product, acknowledgment is not required but would be
+appreciated.
+
+
+A "png_get_copyright" function is available, for convenient use in "about"
+boxes and the like:
+
+ printf("%s",png_get_copyright(NULL));
+
+Also, the PNG logo (in PNG format, of course) is supplied in the
+files "pngbar.png" and "pngbar.jpg (88x31) and "pngnow.png" (98x31).
+
+Libpng is OSI Certified Open Source Software. OSI Certified Open Source is a
+certification mark of the Open Source Initiative.
+
+Glenn Randers-Pehrson
+glennrp at users.sourceforge.net
+June 26, 2010
diff --git a/trunk/src/third_party/libpng/README b/trunk/src/third_party/libpng/README
new file mode 100644
index 0000000..ff7ac1f
--- /dev/null
+++ b/trunk/src/third_party/libpng/README
@@ -0,0 +1,275 @@
+README for libpng version 1.2.44 - June 26, 2010 (shared library 12.0)
+See the note about version numbers near the top of png.h
+
+See INSTALL for instructions on how to install libpng.
+
+Libpng comes in several distribution formats. Get libpng-*.tar.gz,
+libpng-*.tar.xz, or libpng-*.tar.bz2 if you want UNIX-style line
+endings in the text files, or lpng*.7z or lpng*.zip if you want DOS-style
+line endings. You can get UNIX-style line endings from the *.zip file
+by using "unzip -a" but there seems to be no simple way to recover
+UNIX-style line endings from the *.7z file. The *.tar.xz file is
+recommended for *NIX users instead.
+
+Version 0.89 was the first official release of libpng. Don't let the
+fact that it's the first release fool you. The libpng library has been in
+extensive use and testing since mid-1995. By late 1997 it had
+finally gotten to the stage where there hadn't been significant
+changes to the API in some time, and people have a bad feeling about
+libraries with versions < 1.0. Version 1.0.0 was released in
+March 1998.
+
+****
+Note that some of the changes to the png_info structure render this
+version of the library binary incompatible with libpng-0.89 or
+earlier versions if you are using a shared library. The type of the
+"filler" parameter for png_set_filler() has changed from png_byte to
+png_uint_32, which will affect shared-library applications that use
+this function.
+
+To avoid problems with changes to the internals of png_info_struct,
+new APIs have been made available in 0.95 to avoid direct application
+access to info_ptr. These functions are the png_set_<chunk> and
+png_get_<chunk> functions. These functions should be used when
+accessing/storing the info_struct data, rather than manipulating it
+directly, to avoid such problems in the future.
+
+It is important to note that the APIs do not make current programs
+that access the info struct directly incompatible with the new
+library. However, it is strongly suggested that new programs use
+the new APIs (as shown in example.c and pngtest.c), and older programs
+be converted to the new format, to facilitate upgrades in the future.
+****
+
+Additions since 0.90 include the ability to compile libpng as a
+Windows DLL, and new APIs for accessing data in the info struct.
+Experimental functions include the ability to set weighting and cost
+factors for row filter selection, direct reads of integers from buffers
+on big-endian processors that support misaligned data access, faster
+methods of doing alpha composition, and more accurate 16->8 bit color
+conversion.
+
+The additions since 0.89 include the ability to read from a PNG stream
+which has had some (or all) of the signature bytes read by the calling
+application. This also allows the reading of embedded PNG streams that
+do not have the PNG file signature. As well, it is now possible to set
+the library action on the detection of chunk CRC errors. It is possible
+to set different actions based on whether the CRC error occurred in a
+critical or an ancillary chunk.
+
+The changes made to the library, and bugs fixed are based on discussions
+on the png-mng-implement mailing list and not on material submitted
+privately to Guy, Andreas, or Glenn. They will forward any good
+suggestions to the list.
+
+For a detailed description on using libpng, read libpng.txt. For
+examples of libpng in a program, see example.c and pngtest.c. For usage
+information and restrictions (what little they are) on libpng, see
+png.h. For a description on using zlib (the compression library used by
+libpng) and zlib's restrictions, see zlib.h
+
+I have included a general makefile, as well as several machine and
+compiler specific ones, but you may have to modify one for your own needs.
+
+You should use zlib 1.0.4 or later to run this, but it MAY work with
+versions as old as zlib 0.95. Even so, there are bugs in older zlib
+versions which can cause the output of invalid compression streams for
+some images. You will definitely need zlib 1.0.4 or later if you are
+taking advantage of the MS-DOS "far" structure allocation for the small
+and medium memory models. You should also note that zlib is a
+compression library that is useful for more things than just PNG files.
+You can use zlib as a drop-in replacement for fread() and fwrite() if
+you are so inclined.
+
+zlib should be available at the same place that libpng is, or at
+ftp://ftp.simplesystems.org/pub/png/src/
+
+You may also want a copy of the PNG specification. It is available
+as an RFC, a W3C Recommendation, and an ISO/IEC Standard. You can find
+these at http://www.libpng.org/pub/png/pngdocs.html
+
+This code is currently being archived at libpng.sf.net in the
+[DOWNLOAD] area, and on CompuServe, Lib 20 (PNG SUPPORT)
+at GO GRAPHSUP. If you can't find it in any of those places,
+e-mail me, and I'll help you find it.
+
+If you have any code changes, requests, problems, etc., please e-mail
+them to me. Also, I'd appreciate any make files or project files,
+and any modifications you needed to make to get libpng to compile,
+along with a #define variable to tell what compiler/system you are on.
+If you needed to add transformations to libpng, or wish libpng would
+provide the image in a different way, drop me a note (and code, if
+possible), so I can consider supporting the transformation.
+Finally, if you get any warning messages when compiling libpng
+(note: not zlib), and they are easy to fix, I'd appreciate the
+fix. Please mention "libpng" somewhere in the subject line. Thanks.
+
+This release was created and will be supported by myself (of course
+based in a large way on Guy's and Andreas' earlier work), and the PNG
+development group.
+
+Send comments/corrections/commendations to png-mng-implement at lists.sf.net
+(subscription required; visit
+https://lists.sourceforge.net/lists/listinfo/png-mng-implement
+to subscribe) or to glennrp at users.sourceforge.net
+
+You can't reach Guy, the original libpng author, at the addresses
+given in previous versions of this document. He and Andreas will
+read mail addressed to the png-mng-implement list, however.
+
+Please do not send general questions about PNG. Send them to
+the (png-mng-misc at lists.sourceforge.net, subscription required, visit
+https://lists.sourceforge.net/lists/listinfo/png-mng-misc to
+subscribe). On the other hand, please do not send libpng questions to
+that address, send them to me or to the png-mng-implement list. I'll
+get them in the end anyway. If you have a question about something
+in the PNG specification that is related to using libpng, send it
+to me. Send me any questions that start with "I was using libpng,
+and ...". If in doubt, send questions to me. I'll bounce them
+to others, if necessary.
+
+Please do not send suggestions on how to change PNG. We have
+been discussing PNG for twelve years now, and it is official and
+finished. If you have suggestions for libpng, however, I'll
+gladly listen. Even if your suggestion is not used immediately,
+it may be used later.
+
+Files in this distribution:
+
+ ANNOUNCE => Announcement of this version, with recent changes
+ CHANGES => Description of changes between libpng versions
+ KNOWNBUG => List of known bugs and deficiencies
+ LICENSE => License to use and redistribute libpng
+ README => This file
+ TODO => Things not implemented in the current library
+ Y2KINFO => Statement of Y2K compliance
+ example.c => Example code for using libpng functions
+ libpng-*-*-diff.txt => Diff from previous release
+ libpng.3 => manual page for libpng (includes libpng.txt)
+ libpng.txt => Description of libpng and its functions
+ libpngpf.3 => manual page for libpng's private functions
+ png.5 => manual page for the PNG format
+ png.c => Basic interface functions common to library
+ png.h => Library function and interface declarations
+ pngconf.h => System specific library configuration
+ pngerror.c => Error/warning message I/O functions
+ pngget.c => Functions for retrieving info from struct
+ pngmem.c => Memory handling functions
+ pngbar.png => PNG logo, 88x31
+ pngnow.png => PNG logo, 98x31
+ pngpread.c => Progressive reading functions
+ pngread.c => Read data/helper high-level functions
+ pngrio.c => Lowest-level data read I/O functions
+ pngrtran.c => Read data transformation functions
+ pngrutil.c => Read data utility functions
+ pngset.c => Functions for storing data into the info_struct
+ pngtest.c => Library test program
+ pngtest.png => Library test sample image
+ pngtrans.c => Common data transformation functions
+ pngwio.c => Lowest-level write I/O functions
+ pngwrite.c => High-level write functions
+ pngwtran.c => Write data transformations
+ pngwutil.c => Write utility functions
+ contrib => Contributions
+ gregbook => source code for PNG reading and writing, from
+ Greg Roelofs' "PNG: The Definitive Guide",
+ O'Reilly, 1999
+ msvctest => Builds and runs pngtest using a MSVC workspace
+ pngminim => Simple pnm2pngm and png2pnmm programs
+ pngminus => Simple pnm2png and png2pnm programs
+ pngsuite => Test images
+ visupng => Contains a MSVC workspace for VisualPng
+ projects => Contains project files and workspaces for
+ building a DLL
+ beos => Contains a Beos workspace for building libpng
+ c5builder => Contains a Borland workspace for building
+ libpng and zlib
+ netware.txt => Contains instructions for downloading a set
+ of project files for building libpng and
+ zlib on Netware.
+ visualc6 => Contains a Microsoft Visual C++ (MSVC)
+ workspace for building libpng and zlib
+ wince.txt => Contains instructions for downloading a
+ Microsoft Visual C++ (Windows CD Toolkit)
+ workspace for building libpng and zlib on
+ WindowsCE
+ xcode => Contains xcode project files
+ scripts => Directory containing scripts for building libpng:
+ descrip.mms => VMS makefile for MMS or MMK
+ makefile.std => Generic UNIX makefile (cc, creates static
+ libpng.a)
+ makefile.elf => Linux/ELF gcc makefile symbol versioning,
+ creates libpng12.so.0.1.2.44)
+ makefile.linux => Linux/ELF makefile (gcc, creates
+ libpng12.so.0.1.2.44)
+ makefile.gcmmx => Linux/ELF makefile (gcc, creates
+ libpng12.so.0.1.2.44, previously
+ used assembler code tuned for Intel MMX
+ platform)
+ makefile.gcc => Generic makefile (gcc, creates static
+ libpng.a)
+ makefile.knr => Archaic UNIX Makefile that converts files
+ with ansi2knr (Requires ansi2knr.c from
+ ftp://ftp.cs.wisc.edu/ghost)
+ makefile.aix => AIX makefile
+ makefile.cygwin => Cygwin/gcc makefile
+ makefile.darwin => Darwin makefile
+ makefile.dec => DEC Alpha UNIX makefile
+ makefile.freebsd => FreeBSD makefile
+ makefile.hpgcc => HPUX makefile using gcc
+ makefile.hpux => HPUX (10.20 and 11.00) makefile
+ makefile.hp64 => HPUX (10.20 and 11.00) makefile, 64 bit
+ makefile.ibmc => IBM C/C++ version 3.x for Win32 and OS/2
+ (static)
+ makefile.intel => Intel C/C++ version 4.0 and later
+ libpng.icc => Project file, IBM VisualAge/C++ 4.0 or later
+ makefile.netbsd => NetBSD/cc makefile, makes libpng.so.
+ makefile.ne12bsd => NetBSD/cc makefile, makes libpng12.so
+ makefile.openbsd => OpenBSD makefile
+ makefile.sgi => Silicon Graphics IRIX (cc, creates static lib)
+ makefile.sggcc => Silicon Graphics
+ (gcc, creates libpng12.so.0.1.2.44)
+ makefile.sunos => Sun makefile
+ makefile.solaris => Solaris 2.X makefile
+ (gcc, creates libpng12.so.0.1.2.44)
+ makefile.so9 => Solaris 9 makefile
+ (gcc, creates libpng12.so.0.1.2.44)
+ makefile.32sunu => Sun Ultra 32-bit makefile
+ makefile.64sunu => Sun Ultra 64-bit makefile
+ makefile.sco => For SCO OSr5 ELF and Unixware 7 with Native cc
+ makefile.mips => MIPS makefile
+ makefile.acorn => Acorn makefile
+ makefile.amiga => Amiga makefile
+ smakefile.ppc => AMIGA smakefile for SAS C V6.58/7.00 PPC
+ compiler (Requires SCOPTIONS, copied from
+ scripts/SCOPTIONS.ppc)
+ makefile.atari => Atari makefile
+ makefile.beos => BEOS makefile for X86
+ makefile.bor => Borland makefile (uses bcc)
+ makefile.bc32 => 32-bit Borland C++ (all modules compiled in C mode)
+ makefile.tc3 => Turbo C 3.0 makefile
+ makefile.dj2 => DJGPP 2 makefile
+ makefile.msc => Microsoft C makefile
+ makefile.vcawin32=> makefile for Microsoft Visual C++ 5.0 and
+ later (previously used assembler code tuned
+ for Intel MMX platform)
+ makefile.vcwin32 => makefile for Microsoft Visual C++ 4.0 and
+ later (does not use assembler code)
+ makefile.os2 => OS/2 Makefile (gcc and emx, requires pngos2.def)
+ pngos2.def => OS/2 module definition file used by makefile.os2
+ makefile.watcom => Watcom 10a+ Makefile, 32-bit flat memory model
+ makevms.com => VMS build script
+ SCOPTIONS.ppc => Used with smakefile.ppc
+
+Good luck, and happy coding.
+
+-Glenn Randers-Pehrson (current maintainer, since 1998)
+ Internet: glennrp at users.sourceforge.net
+
+-Andreas Eric Dilger (former maintainer, 1996-1997)
+ Internet: adilger at enel.ucalgary.ca
+ Web: http://members.shaw.ca/adilger/
+
+-Guy Eric Schalnat (original author and former maintainer, 1995-1996)
+ (formerly of Group 42, Inc)
+ Internet: gschal at infinet.com
diff --git a/trunk/src/third_party/libpng/README.modpagespeed b/trunk/src/third_party/libpng/README.modpagespeed
new file mode 100644
index 0000000..f80cfc6
--- /dev/null
+++ b/trunk/src/third_party/libpng/README.modpagespeed
@@ -0,0 +1,5 @@
+Name: libpng
+URL: http://libpng.org/
+
+No changes to the source files.
+
diff --git a/trunk/src/third_party/libpng/libpng.gyp b/trunk/src/third_party/libpng/libpng.gyp
new file mode 100644
index 0000000..d46080f
--- /dev/null
+++ b/trunk/src/third_party/libpng/libpng.gyp
@@ -0,0 +1,117 @@
+# Copyright (c) 2009 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+{
+ 'variables': {
+ 'conditions': [
+ [ 'OS=="linux" or OS=="freebsd" or OS=="openbsd"', {
+ # Link to system .so since we already use it due to GTK.
+ 'use_system_libpng%': 1,
+ }, { # OS!="linux" and OS!="freebsd" and OS!="openbsd"
+ 'use_system_libpng%': 0,
+ }],
+ ],
+ },
+ 'conditions': [
+ ['use_system_libpng==0', {
+ 'targets': [
+ {
+ 'target_name': 'libpng',
+ 'type': '<(component)',
+ 'dependencies': [
+ '../zlib/zlib.gyp:zlib',
+ ],
+ 'msvs_guid': 'C564F145-9172-42C3-BFCB-6014CA97DBCD',
+ 'sources': [
+ 'png.c',
+ 'png.h',
+ 'pngconf.h',
+ 'pngerror.c',
+ 'pnggccrd.c',
+ 'pngget.c',
+ 'pngmem.c',
+ 'pngpread.c',
+ 'pngread.c',
+ 'pngrio.c',
+ 'pngrtran.c',
+ 'pngrutil.c',
+ 'pngset.c',
+ 'pngtrans.c',
+ 'pngusr.h',
+ 'pngvcrd.c',
+ 'pngwio.c',
+ 'pngwrite.c',
+ 'pngwtran.c',
+ 'pngwutil.c',
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '.',
+ ],
+ },
+ 'export_dependent_settings': [
+ '../zlib/zlib.gyp:zlib',
+ ],
+ 'conditions': [
+ ['OS!="win"', {'product_name': 'png'}],
+ ['OS=="win" and component=="shared_library"', {
+ 'defines': [
+ 'PNG_BUILD_DLL',
+ 'PNG_NO_MODULEDEF',
+ ],
+ 'direct_dependent_settings': {
+ 'defines': [
+ 'PNG_USE_DLL',
+ ],
+ },
+ }],
+ ],
+ },
+ ]
+ }, {
+ 'conditions': [
+ ['sysroot!=""', {
+ 'variables': {
+ 'pkg-config': '../../build/linux/pkg-config-wrapper "<(sysroot)"',
+ },
+ }, {
+ 'variables': {
+ 'pkg-config': 'pkg-config'
+ },
+ }],
+ ],
+ 'targets': [
+ {
+ 'target_name': 'libpng',
+ 'type': 'settings',
+ 'dependencies': [
+ '../zlib/zlib.gyp:zlib',
+ ],
+ 'direct_dependent_settings': {
+ 'cflags': [
+ '<!@(<(pkg-config) --cflags libpng)',
+ ],
+ 'defines': [
+ 'USE_SYSTEM_LIBPNG',
+ ],
+ },
+ 'link_settings': {
+ 'ldflags': [
+ '<!@(<(pkg-config) --libs-only-L --libs-only-other libpng)',
+ ],
+ 'libraries': [
+ '<!@(<(pkg-config) --libs-only-l libpng)',
+ ],
+ },
+ },
+ ],
+ }],
+ ],
+}
+
+# Local Variables:
+# tab-width:2
+# indent-tabs-mode:nil
+# End:
+# vim: set expandtab tabstop=2 shiftwidth=2:
diff --git a/trunk/src/third_party/libpng/png.c b/trunk/src/third_party/libpng/png.c
new file mode 100644
index 0000000..7ad9538
--- /dev/null
+++ b/trunk/src/third_party/libpng/png.c
@@ -0,0 +1,1100 @@
+
+/* png.c - location for general purpose libpng functions
+ *
+ * Last changed in libpng 1.2.43 [February 25, 2010]
+ * Copyright (c) 1998-2010 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ */
+
+#define PNG_INTERNAL
+#define PNG_NO_EXTERN
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+
+/* Generate a compiler error if there is an old png.h in the search path. */
+typedef version_1_2_44 Your_png_h_is_not_version_1_2_44;
+
+/* Version information for C files. This had better match the version
+ * string defined in png.h.
+ */
+
+#ifdef PNG_USE_GLOBAL_ARRAYS
+/* png_libpng_ver was changed to a function in version 1.0.5c */
+PNG_CONST char png_libpng_ver[18] = PNG_LIBPNG_VER_STRING;
+
+#ifdef PNG_READ_SUPPORTED
+
+/* png_sig was changed to a function in version 1.0.5c */
+/* Place to hold the signature string for a PNG file. */
+PNG_CONST png_byte FARDATA png_sig[8] = {137, 80, 78, 71, 13, 10, 26, 10};
+#endif /* PNG_READ_SUPPORTED */
+
+/* Invoke global declarations for constant strings for known chunk types */
+PNG_IHDR;
+PNG_IDAT;
+PNG_IEND;
+PNG_PLTE;
+PNG_bKGD;
+PNG_cHRM;
+PNG_gAMA;
+PNG_hIST;
+PNG_iCCP;
+PNG_iTXt;
+PNG_oFFs;
+PNG_pCAL;
+PNG_sCAL;
+PNG_pHYs;
+PNG_sBIT;
+PNG_sPLT;
+PNG_sRGB;
+PNG_tEXt;
+PNG_tIME;
+PNG_tRNS;
+PNG_zTXt;
+
+#ifdef PNG_READ_SUPPORTED
+/* Arrays to facilitate easy interlacing - use pass (0 - 6) as index */
+
+/* Start of interlace block */
+PNG_CONST int FARDATA png_pass_start[] = {0, 4, 0, 2, 0, 1, 0};
+
+/* Offset to next interlace block */
+PNG_CONST int FARDATA png_pass_inc[] = {8, 8, 4, 4, 2, 2, 1};
+
+/* Start of interlace block in the y direction */
+PNG_CONST int FARDATA png_pass_ystart[] = {0, 0, 4, 0, 2, 0, 1};
+
+/* Offset to next interlace block in the y direction */
+PNG_CONST int FARDATA png_pass_yinc[] = {8, 8, 8, 4, 4, 2, 2};
+
+/* Height of interlace block. This is not currently used - if you need
+ * it, uncomment it here and in png.h
+PNG_CONST int FARDATA png_pass_height[] = {8, 8, 4, 4, 2, 2, 1};
+*/
+
+/* Mask to determine which pixels are valid in a pass */
+PNG_CONST int FARDATA png_pass_mask[] =
+ {0x80, 0x08, 0x88, 0x22, 0xaa, 0x55, 0xff};
+
+/* Mask to determine which pixels to overwrite while displaying */
+PNG_CONST int FARDATA png_pass_dsp_mask[]
+ = {0xff, 0x0f, 0xff, 0x33, 0xff, 0x55, 0xff};
+
+#endif /* PNG_READ_SUPPORTED */
+#endif /* PNG_USE_GLOBAL_ARRAYS */
+
+/* Tells libpng that we have already handled the first "num_bytes" bytes
+ * of the PNG file signature. If the PNG data is embedded into another
+ * stream we can set num_bytes = 8 so that libpng will not attempt to read
+ * or write any of the magic bytes before it starts on the IHDR.
+ */
+
+#ifdef PNG_READ_SUPPORTED
+void PNGAPI
+png_set_sig_bytes(png_structp png_ptr, int num_bytes)
+{
+ png_debug(1, "in png_set_sig_bytes");
+
+ if (png_ptr == NULL)
+ return;
+
+ if (num_bytes > 8)
+ png_error(png_ptr, "Too many bytes for PNG signature.");
+
+ png_ptr->sig_bytes = (png_byte)(num_bytes < 0 ? 0 : num_bytes);
+}
+
+/* Checks whether the supplied bytes match the PNG signature. We allow
+ * checking less than the full 8-byte signature so that those apps that
+ * already read the first few bytes of a file to determine the file type
+ * can simply check the remaining bytes for extra assurance. Returns
+ * an integer less than, equal to, or greater than zero if sig is found,
+ * respectively, to be less than, to match, or be greater than the correct
+ * PNG signature (this is the same behaviour as strcmp, memcmp, etc).
+ */
+int PNGAPI
+png_sig_cmp(png_bytep sig, png_size_t start, png_size_t num_to_check)
+{
+ png_byte png_signature[8] = {137, 80, 78, 71, 13, 10, 26, 10};
+ if (num_to_check > 8)
+ num_to_check = 8;
+ else if (num_to_check < 1)
+ return (-1);
+
+ if (start > 7)
+ return (-1);
+
+ if (start + num_to_check > 8)
+ num_to_check = 8 - start;
+
+ return ((int)(png_memcmp(&sig[start], &png_signature[start], num_to_check)));
+}
+
+#if defined(PNG_1_0_X) || defined(PNG_1_2_X)
+/* (Obsolete) function to check signature bytes. It does not allow one
+ * to check a partial signature. This function might be removed in the
+ * future - use png_sig_cmp(). Returns true (nonzero) if the file is PNG.
+ */
+int PNGAPI
+png_check_sig(png_bytep sig, int num)
+{
+ return ((int)!png_sig_cmp(sig, (png_size_t)0, (png_size_t)num));
+}
+#endif
+#endif /* PNG_READ_SUPPORTED */
+
+#if defined(PNG_READ_SUPPORTED) || defined(PNG_WRITE_SUPPORTED)
+/* Function to allocate memory for zlib and clear it to 0. */
+#ifdef PNG_1_0_X
+voidpf PNGAPI
+#else
+voidpf /* PRIVATE */
+#endif
+png_zalloc(voidpf png_ptr, uInt items, uInt size)
+{
+ png_voidp ptr;
+ png_structp p=(png_structp)png_ptr;
+ png_uint_32 save_flags=p->flags;
+ png_uint_32 num_bytes;
+
+ if (png_ptr == NULL)
+ return (NULL);
+ if (items > PNG_UINT_32_MAX/size)
+ {
+ png_warning (p, "Potential overflow in png_zalloc()");
+ return (NULL);
+ }
+ num_bytes = (png_uint_32)items * size;
+
+ p->flags|=PNG_FLAG_MALLOC_NULL_MEM_OK;
+ ptr = (png_voidp)png_malloc((png_structp)png_ptr, num_bytes);
+ p->flags=save_flags;
+
+#if defined(PNG_1_0_X) && !defined(PNG_NO_ZALLOC_ZERO)
+ if (ptr == NULL)
+ return ((voidpf)ptr);
+
+ if (num_bytes > (png_uint_32)0x8000L)
+ {
+ png_memset(ptr, 0, (png_size_t)0x8000L);
+ png_memset((png_bytep)ptr + (png_size_t)0x8000L, 0,
+ (png_size_t)(num_bytes - (png_uint_32)0x8000L));
+ }
+ else
+ {
+ png_memset(ptr, 0, (png_size_t)num_bytes);
+ }
+#endif
+ return ((voidpf)ptr);
+}
+
+/* Function to free memory for zlib */
+#ifdef PNG_1_0_X
+void PNGAPI
+#else
+void /* PRIVATE */
+#endif
+png_zfree(voidpf png_ptr, voidpf ptr)
+{
+ png_free((png_structp)png_ptr, (png_voidp)ptr);
+}
+
+/* Reset the CRC variable to 32 bits of 1's. Care must be taken
+ * in case CRC is > 32 bits to leave the top bits 0.
+ */
+void /* PRIVATE */
+png_reset_crc(png_structp png_ptr)
+{
+ png_ptr->crc = crc32(0, Z_NULL, 0);
+}
+
+/* Calculate the CRC over a section of data. We can only pass as
+ * much data to this routine as the largest single buffer size. We
+ * also check that this data will actually be used before going to the
+ * trouble of calculating it.
+ */
+void /* PRIVATE */
+png_calculate_crc(png_structp png_ptr, png_bytep ptr, png_size_t length)
+{
+ int need_crc = 1;
+
+ if (png_ptr->chunk_name[0] & 0x20) /* ancillary */
+ {
+ if ((png_ptr->flags & PNG_FLAG_CRC_ANCILLARY_MASK) ==
+ (PNG_FLAG_CRC_ANCILLARY_USE | PNG_FLAG_CRC_ANCILLARY_NOWARN))
+ need_crc = 0;
+ }
+ else /* critical */
+ {
+ if (png_ptr->flags & PNG_FLAG_CRC_CRITICAL_IGNORE)
+ need_crc = 0;
+ }
+
+ if (need_crc)
+ png_ptr->crc = crc32(png_ptr->crc, ptr, (uInt)length);
+}
+
+/* Allocate the memory for an info_struct for the application. We don't
+ * really need the png_ptr, but it could potentially be useful in the
+ * future. This should be used in favour of malloc(png_sizeof(png_info))
+ * and png_info_init() so that applications that want to use a shared
+ * libpng don't have to be recompiled if png_info changes size.
+ */
+png_infop PNGAPI
+png_create_info_struct(png_structp png_ptr)
+{
+ png_infop info_ptr;
+
+ png_debug(1, "in png_create_info_struct");
+
+ if (png_ptr == NULL)
+ return (NULL);
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ info_ptr = (png_infop)png_create_struct_2(PNG_STRUCT_INFO,
+ png_ptr->malloc_fn, png_ptr->mem_ptr);
+#else
+ info_ptr = (png_infop)png_create_struct(PNG_STRUCT_INFO);
+#endif
+ if (info_ptr != NULL)
+ png_info_init_3(&info_ptr, png_sizeof(png_info));
+
+ return (info_ptr);
+}
+
+/* This function frees the memory associated with a single info struct.
+ * Normally, one would use either png_destroy_read_struct() or
+ * png_destroy_write_struct() to free an info struct, but this may be
+ * useful for some applications.
+ */
+void PNGAPI
+png_destroy_info_struct(png_structp png_ptr, png_infopp info_ptr_ptr)
+{
+ png_infop info_ptr = NULL;
+
+ png_debug(1, "in png_destroy_info_struct");
+
+ if (png_ptr == NULL)
+ return;
+
+ if (info_ptr_ptr != NULL)
+ info_ptr = *info_ptr_ptr;
+
+ if (info_ptr != NULL)
+ {
+ png_info_destroy(png_ptr, info_ptr);
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_destroy_struct_2((png_voidp)info_ptr, png_ptr->free_fn,
+ png_ptr->mem_ptr);
+#else
+ png_destroy_struct((png_voidp)info_ptr);
+#endif
+ *info_ptr_ptr = NULL;
+ }
+}
+
+/* Initialize the info structure. This is now an internal function (0.89)
+ * and applications using it are urged to use png_create_info_struct()
+ * instead.
+ */
+#if defined(PNG_1_0_X) || defined(PNG_1_2_X)
+#undef png_info_init
+void PNGAPI
+png_info_init(png_infop info_ptr)
+{
+ /* We only come here via pre-1.0.12-compiled applications */
+ png_info_init_3(&info_ptr, 0);
+}
+#endif
+
+void PNGAPI
+png_info_init_3(png_infopp ptr_ptr, png_size_t png_info_struct_size)
+{
+ png_infop info_ptr = *ptr_ptr;
+
+ png_debug(1, "in png_info_init_3");
+
+ if (info_ptr == NULL)
+ return;
+
+ if (png_sizeof(png_info) > png_info_struct_size)
+ {
+ png_destroy_struct(info_ptr);
+ info_ptr = (png_infop)png_create_struct(PNG_STRUCT_INFO);
+ *ptr_ptr = info_ptr;
+ }
+
+ /* Set everything to 0 */
+ png_memset(info_ptr, 0, png_sizeof(png_info));
+}
+
+#ifdef PNG_FREE_ME_SUPPORTED
+void PNGAPI
+png_data_freer(png_structp png_ptr, png_infop info_ptr,
+ int freer, png_uint_32 mask)
+{
+ png_debug(1, "in png_data_freer");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ if (freer == PNG_DESTROY_WILL_FREE_DATA)
+ info_ptr->free_me |= mask;
+ else if (freer == PNG_USER_WILL_FREE_DATA)
+ info_ptr->free_me &= ~mask;
+ else
+ png_warning(png_ptr,
+ "Unknown freer parameter in png_data_freer.");
+}
+#endif
+
+void PNGAPI
+png_free_data(png_structp png_ptr, png_infop info_ptr, png_uint_32 mask,
+ int num)
+{
+ png_debug(1, "in png_free_data");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+#ifdef PNG_TEXT_SUPPORTED
+ /* Free text item num or (if num == -1) all text items */
+#ifdef PNG_FREE_ME_SUPPORTED
+ if ((mask & PNG_FREE_TEXT) & info_ptr->free_me)
+#else
+ if (mask & PNG_FREE_TEXT)
+#endif
+ {
+ if (num != -1)
+ {
+ if (info_ptr->text && info_ptr->text[num].key)
+ {
+ png_free(png_ptr, info_ptr->text[num].key);
+ info_ptr->text[num].key = NULL;
+ }
+ }
+ else
+ {
+ int i;
+ for (i = 0; i < info_ptr->num_text; i++)
+ png_free_data(png_ptr, info_ptr, PNG_FREE_TEXT, i);
+ png_free(png_ptr, info_ptr->text);
+ info_ptr->text = NULL;
+ info_ptr->num_text=0;
+ }
+ }
+#endif
+
+#ifdef PNG_tRNS_SUPPORTED
+ /* Free any tRNS entry */
+#ifdef PNG_FREE_ME_SUPPORTED
+ if ((mask & PNG_FREE_TRNS) & info_ptr->free_me)
+#else
+ if ((mask & PNG_FREE_TRNS) && (png_ptr->flags & PNG_FLAG_FREE_TRNS))
+#endif
+ {
+ png_free(png_ptr, info_ptr->trans);
+ info_ptr->trans = NULL;
+ info_ptr->valid &= ~PNG_INFO_tRNS;
+#ifndef PNG_FREE_ME_SUPPORTED
+ png_ptr->flags &= ~PNG_FLAG_FREE_TRNS;
+#endif
+ }
+#endif
+
+#ifdef PNG_sCAL_SUPPORTED
+ /* Free any sCAL entry */
+#ifdef PNG_FREE_ME_SUPPORTED
+ if ((mask & PNG_FREE_SCAL) & info_ptr->free_me)
+#else
+ if (mask & PNG_FREE_SCAL)
+#endif
+ {
+#if defined(PNG_FIXED_POINT_SUPPORTED) && !defined(PNG_FLOATING_POINT_SUPPORTED)
+ png_free(png_ptr, info_ptr->scal_s_width);
+ png_free(png_ptr, info_ptr->scal_s_height);
+ info_ptr->scal_s_width = NULL;
+ info_ptr->scal_s_height = NULL;
+#endif
+ info_ptr->valid &= ~PNG_INFO_sCAL;
+ }
+#endif
+
+#ifdef PNG_pCAL_SUPPORTED
+ /* Free any pCAL entry */
+#ifdef PNG_FREE_ME_SUPPORTED
+ if ((mask & PNG_FREE_PCAL) & info_ptr->free_me)
+#else
+ if (mask & PNG_FREE_PCAL)
+#endif
+ {
+ png_free(png_ptr, info_ptr->pcal_purpose);
+ png_free(png_ptr, info_ptr->pcal_units);
+ info_ptr->pcal_purpose = NULL;
+ info_ptr->pcal_units = NULL;
+ if (info_ptr->pcal_params != NULL)
+ {
+ int i;
+ for (i = 0; i < (int)info_ptr->pcal_nparams; i++)
+ {
+ png_free(png_ptr, info_ptr->pcal_params[i]);
+ info_ptr->pcal_params[i] = NULL;
+ }
+ png_free(png_ptr, info_ptr->pcal_params);
+ info_ptr->pcal_params = NULL;
+ }
+ info_ptr->valid &= ~PNG_INFO_pCAL;
+ }
+#endif
+
+#ifdef PNG_iCCP_SUPPORTED
+ /* Free any iCCP entry */
+#ifdef PNG_FREE_ME_SUPPORTED
+ if ((mask & PNG_FREE_ICCP) & info_ptr->free_me)
+#else
+ if (mask & PNG_FREE_ICCP)
+#endif
+ {
+ png_free(png_ptr, info_ptr->iccp_name);
+ png_free(png_ptr, info_ptr->iccp_profile);
+ info_ptr->iccp_name = NULL;
+ info_ptr->iccp_profile = NULL;
+ info_ptr->valid &= ~PNG_INFO_iCCP;
+ }
+#endif
+
+#ifdef PNG_sPLT_SUPPORTED
+ /* Free a given sPLT entry, or (if num == -1) all sPLT entries */
+#ifdef PNG_FREE_ME_SUPPORTED
+ if ((mask & PNG_FREE_SPLT) & info_ptr->free_me)
+#else
+ if (mask & PNG_FREE_SPLT)
+#endif
+ {
+ if (num != -1)
+ {
+ if (info_ptr->splt_palettes)
+ {
+ png_free(png_ptr, info_ptr->splt_palettes[num].name);
+ png_free(png_ptr, info_ptr->splt_palettes[num].entries);
+ info_ptr->splt_palettes[num].name = NULL;
+ info_ptr->splt_palettes[num].entries = NULL;
+ }
+ }
+ else
+ {
+ if (info_ptr->splt_palettes_num)
+ {
+ int i;
+ for (i = 0; i < (int)info_ptr->splt_palettes_num; i++)
+ png_free_data(png_ptr, info_ptr, PNG_FREE_SPLT, i);
+
+ png_free(png_ptr, info_ptr->splt_palettes);
+ info_ptr->splt_palettes = NULL;
+ info_ptr->splt_palettes_num = 0;
+ }
+ info_ptr->valid &= ~PNG_INFO_sPLT;
+ }
+ }
+#endif
+
+#ifdef PNG_UNKNOWN_CHUNKS_SUPPORTED
+ if (png_ptr->unknown_chunk.data)
+ {
+ png_free(png_ptr, png_ptr->unknown_chunk.data);
+ png_ptr->unknown_chunk.data = NULL;
+ }
+
+#ifdef PNG_FREE_ME_SUPPORTED
+ if ((mask & PNG_FREE_UNKN) & info_ptr->free_me)
+#else
+ if (mask & PNG_FREE_UNKN)
+#endif
+ {
+ if (num != -1)
+ {
+ if (info_ptr->unknown_chunks)
+ {
+ png_free(png_ptr, info_ptr->unknown_chunks[num].data);
+ info_ptr->unknown_chunks[num].data = NULL;
+ }
+ }
+ else
+ {
+ int i;
+
+ if (info_ptr->unknown_chunks_num)
+ {
+ for (i = 0; i < (int)info_ptr->unknown_chunks_num; i++)
+ png_free_data(png_ptr, info_ptr, PNG_FREE_UNKN, i);
+
+ png_free(png_ptr, info_ptr->unknown_chunks);
+ info_ptr->unknown_chunks = NULL;
+ info_ptr->unknown_chunks_num = 0;
+ }
+ }
+ }
+#endif
+
+#ifdef PNG_hIST_SUPPORTED
+ /* Free any hIST entry */
+#ifdef PNG_FREE_ME_SUPPORTED
+ if ((mask & PNG_FREE_HIST) & info_ptr->free_me)
+#else
+ if ((mask & PNG_FREE_HIST) && (png_ptr->flags & PNG_FLAG_FREE_HIST))
+#endif
+ {
+ png_free(png_ptr, info_ptr->hist);
+ info_ptr->hist = NULL;
+ info_ptr->valid &= ~PNG_INFO_hIST;
+#ifndef PNG_FREE_ME_SUPPORTED
+ png_ptr->flags &= ~PNG_FLAG_FREE_HIST;
+#endif
+ }
+#endif
+
+ /* Free any PLTE entry that was internally allocated */
+#ifdef PNG_FREE_ME_SUPPORTED
+ if ((mask & PNG_FREE_PLTE) & info_ptr->free_me)
+#else
+ if ((mask & PNG_FREE_PLTE) && (png_ptr->flags & PNG_FLAG_FREE_PLTE))
+#endif
+ {
+ png_zfree(png_ptr, info_ptr->palette);
+ info_ptr->palette = NULL;
+ info_ptr->valid &= ~PNG_INFO_PLTE;
+#ifndef PNG_FREE_ME_SUPPORTED
+ png_ptr->flags &= ~PNG_FLAG_FREE_PLTE;
+#endif
+ info_ptr->num_palette = 0;
+ }
+
+#ifdef PNG_INFO_IMAGE_SUPPORTED
+ /* Free any image bits attached to the info structure */
+#ifdef PNG_FREE_ME_SUPPORTED
+ if ((mask & PNG_FREE_ROWS) & info_ptr->free_me)
+#else
+ if (mask & PNG_FREE_ROWS)
+#endif
+ {
+ if (info_ptr->row_pointers)
+ {
+ int row;
+ for (row = 0; row < (int)info_ptr->height; row++)
+ {
+ png_free(png_ptr, info_ptr->row_pointers[row]);
+ info_ptr->row_pointers[row] = NULL;
+ }
+ png_free(png_ptr, info_ptr->row_pointers);
+ info_ptr->row_pointers = NULL;
+ }
+ info_ptr->valid &= ~PNG_INFO_IDAT;
+ }
+#endif
+
+#ifdef PNG_FREE_ME_SUPPORTED
+ if (num == -1)
+ info_ptr->free_me &= ~mask;
+ else
+ info_ptr->free_me &= ~(mask & ~PNG_FREE_MUL);
+#endif
+}
+
+/* This is an internal routine to free any memory that the info struct is
+ * pointing to before re-using it or freeing the struct itself. Recall
+ * that png_free() checks for NULL pointers for us.
+ */
+void /* PRIVATE */
+png_info_destroy(png_structp png_ptr, png_infop info_ptr)
+{
+ png_debug(1, "in png_info_destroy");
+
+ png_free_data(png_ptr, info_ptr, PNG_FREE_ALL, -1);
+
+#ifdef PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+ if (png_ptr->num_chunk_list)
+ {
+ png_free(png_ptr, png_ptr->chunk_list);
+ png_ptr->chunk_list = NULL;
+ png_ptr->num_chunk_list = 0;
+ }
+#endif
+
+ png_info_init_3(&info_ptr, png_sizeof(png_info));
+}
+#endif /* defined(PNG_READ_SUPPORTED) || defined(PNG_WRITE_SUPPORTED) */
+
+/* This function returns a pointer to the io_ptr associated with the user
+ * functions. The application should free any memory associated with this
+ * pointer before png_write_destroy() or png_read_destroy() are called.
+ */
+png_voidp PNGAPI
+png_get_io_ptr(png_structp png_ptr)
+{
+ if (png_ptr == NULL)
+ return (NULL);
+ return (png_ptr->io_ptr);
+}
+
+#if defined(PNG_READ_SUPPORTED) || defined(PNG_WRITE_SUPPORTED)
+#ifdef PNG_STDIO_SUPPORTED
+/* Initialize the default input/output functions for the PNG file. If you
+ * use your own read or write routines, you can call either png_set_read_fn()
+ * or png_set_write_fn() instead of png_init_io(). If you have defined
+ * PNG_NO_STDIO, you must use a function of your own because "FILE *" isn't
+ * necessarily available.
+ */
+void PNGAPI
+png_init_io(png_structp png_ptr, png_FILE_p fp)
+{
+ png_debug(1, "in png_init_io");
+
+ if (png_ptr == NULL)
+ return;
+
+ png_ptr->io_ptr = (png_voidp)fp;
+}
+#endif
+
+#ifdef PNG_TIME_RFC1123_SUPPORTED
+/* Convert the supplied time into an RFC 1123 string suitable for use in
+ * a "Creation Time" or other text-based time string.
+ */
+png_charp PNGAPI
+png_convert_to_rfc1123(png_structp png_ptr, png_timep ptime)
+{
+ static PNG_CONST char short_months[12][4] =
+ {"Jan", "Feb", "Mar", "Apr", "May", "Jun",
+ "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"};
+
+ if (png_ptr == NULL)
+ return (NULL);
+ if (png_ptr->time_buffer == NULL)
+ {
+ png_ptr->time_buffer = (png_charp)png_malloc(png_ptr, (png_uint_32)(29*
+ png_sizeof(char)));
+ }
+
+#ifdef _WIN32_WCE
+ {
+ wchar_t time_buf[29];
+ wsprintf(time_buf, TEXT("%d %S %d %02d:%02d:%02d +0000"),
+ ptime->day % 32, short_months[(ptime->month - 1) % 12],
+ ptime->year, ptime->hour % 24, ptime->minute % 60,
+ ptime->second % 61);
+ WideCharToMultiByte(CP_ACP, 0, time_buf, -1, png_ptr->time_buffer,
+ 29, NULL, NULL);
+ }
+#else
+#ifdef USE_FAR_KEYWORD
+ {
+ char near_time_buf[29];
+ png_snprintf6(near_time_buf, 29, "%d %s %d %02d:%02d:%02d +0000",
+ ptime->day % 32, short_months[(ptime->month - 1) % 12],
+ ptime->year, ptime->hour % 24, ptime->minute % 60,
+ ptime->second % 61);
+ png_memcpy(png_ptr->time_buffer, near_time_buf,
+ 29*png_sizeof(char));
+ }
+#else
+ png_snprintf6(png_ptr->time_buffer, 29, "%d %s %d %02d:%02d:%02d +0000",
+ ptime->day % 32, short_months[(ptime->month - 1) % 12],
+ ptime->year, ptime->hour % 24, ptime->minute % 60,
+ ptime->second % 61);
+#endif
+#endif /* _WIN32_WCE */
+ return ((png_charp)png_ptr->time_buffer);
+}
+#endif /* PNG_TIME_RFC1123_SUPPORTED */
+
+#endif /* defined(PNG_READ_SUPPORTED) || defined(PNG_WRITE_SUPPORTED) */
+
+png_charp PNGAPI
+png_get_copyright(png_structp png_ptr)
+{
+ png_ptr = png_ptr; /* Silence compiler warning about unused png_ptr */
+#ifdef PNG_STRING_COPYRIGHT
+ return PNG_STRING_COPYRIGHT
+#else
+#ifdef __STDC__
+ return ((png_charp) PNG_STRING_NEWLINE \
+ "libpng version 1.2.44 - June 26, 2010" PNG_STRING_NEWLINE \
+ "Copyright (c) 1998-2010 Glenn Randers-Pehrson" PNG_STRING_NEWLINE \
+ "Copyright (c) 1996-1997 Andreas Dilger" PNG_STRING_NEWLINE \
+ "Copyright (c) 1995-1996 Guy Eric Schalnat, Group 42, Inc." \
+ PNG_STRING_NEWLINE);
+#else
+ return ((png_charp) "libpng version 1.2.44 - June 26, 2010\
+ Copyright (c) 1998-2010 Glenn Randers-Pehrson\
+ Copyright (c) 1996-1997 Andreas Dilger\
+ Copyright (c) 1995-1996 Guy Eric Schalnat, Group 42, Inc.");
+#endif
+#endif
+}
+
+/* The following return the library version as a short string in the
+ * format 1.0.0 through 99.99.99zz. To get the version of *.h files
+ * used with your application, print out PNG_LIBPNG_VER_STRING, which
+ * is defined in png.h.
+ * Note: now there is no difference between png_get_libpng_ver() and
+ * png_get_header_ver(). Due to the version_nn_nn_nn typedef guard,
+ * it is guaranteed that png.c uses the correct version of png.h.
+ */
+png_charp PNGAPI
+png_get_libpng_ver(png_structp png_ptr)
+{
+ /* Version of *.c files used when building libpng */
+ png_ptr = png_ptr; /* Silence compiler warning about unused png_ptr */
+ return ((png_charp) PNG_LIBPNG_VER_STRING);
+}
+
+png_charp PNGAPI
+png_get_header_ver(png_structp png_ptr)
+{
+ /* Version of *.h files used when building libpng */
+ png_ptr = png_ptr; /* Silence compiler warning about unused png_ptr */
+ return ((png_charp) PNG_LIBPNG_VER_STRING);
+}
+
+png_charp PNGAPI
+png_get_header_version(png_structp png_ptr)
+{
+ /* Returns longer string containing both version and date */
+ png_ptr = png_ptr; /* Silence compiler warning about unused png_ptr */
+#ifdef __STDC__
+ return ((png_charp) PNG_HEADER_VERSION_STRING
+#ifndef PNG_READ_SUPPORTED
+ " (NO READ SUPPORT)"
+#endif
+ PNG_STRING_NEWLINE);
+#else
+ return ((png_charp) PNG_HEADER_VERSION_STRING);
+#endif
+}
+
+#if defined(PNG_READ_SUPPORTED) || defined(PNG_WRITE_SUPPORTED)
+#ifdef PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+int PNGAPI
+png_handle_as_unknown(png_structp png_ptr, png_bytep chunk_name)
+{
+ /* Check chunk_name and return "keep" value if it's on the list, else 0 */
+ int i;
+ png_bytep p;
+ if (png_ptr == NULL || chunk_name == NULL || png_ptr->num_chunk_list<=0)
+ return 0;
+ p = png_ptr->chunk_list + png_ptr->num_chunk_list*5 - 5;
+ for (i = png_ptr->num_chunk_list; i; i--, p -= 5)
+ if (!png_memcmp(chunk_name, p, 4))
+ return ((int)*(p + 4));
+ return 0;
+}
+#endif
+
+/* This function, added to libpng-1.0.6g, is untested. */
+int PNGAPI
+png_reset_zstream(png_structp png_ptr)
+{
+ if (png_ptr == NULL)
+ return Z_STREAM_ERROR;
+ return (inflateReset(&png_ptr->zstream));
+}
+#endif /* defined(PNG_READ_SUPPORTED) || defined(PNG_WRITE_SUPPORTED) */
+
+/* This function was added to libpng-1.0.7 */
+png_uint_32 PNGAPI
+png_access_version_number(void)
+{
+ /* Version of *.c files used when building libpng */
+ return((png_uint_32) PNG_LIBPNG_VER);
+}
+
+
+#if defined(PNG_READ_SUPPORTED) && defined(PNG_ASSEMBLER_CODE_SUPPORTED)
+#ifndef PNG_1_0_X
+/* This function was added to libpng 1.2.0 */
+int PNGAPI
+png_mmx_support(void)
+{
+ /* Obsolete, to be removed from libpng-1.4.0 */
+ return -1;
+}
+#endif /* PNG_1_0_X */
+#endif /* PNG_READ_SUPPORTED && PNG_ASSEMBLER_CODE_SUPPORTED */
+
+#if defined(PNG_READ_SUPPORTED) || defined(PNG_WRITE_SUPPORTED)
+#ifdef PNG_SIZE_T
+/* Added at libpng version 1.2.6 */
+ PNG_EXTERN png_size_t PNGAPI png_convert_size PNGARG((size_t size));
+png_size_t PNGAPI
+png_convert_size(size_t size)
+{
+ if (size > (png_size_t)-1)
+ PNG_ABORT(); /* We haven't got access to png_ptr, so no png_error() */
+ return ((png_size_t)size);
+}
+#endif /* PNG_SIZE_T */
+
+/* Added at libpng version 1.2.34 and 1.4.0 (moved from pngset.c) */
+#ifdef PNG_cHRM_SUPPORTED
+#ifdef PNG_CHECK_cHRM_SUPPORTED
+
+/*
+ * Multiply two 32-bit numbers, V1 and V2, using 32-bit
+ * arithmetic, to produce a 64 bit result in the HI/LO words.
+ *
+ * A B
+ * x C D
+ * ------
+ * AD || BD
+ * AC || CB || 0
+ *
+ * where A and B are the high and low 16-bit words of V1,
+ * C and D are the 16-bit words of V2, AD is the product of
+ * A and D, and X || Y is (X << 16) + Y.
+*/
+
+void /* PRIVATE */
+png_64bit_product (long v1, long v2, unsigned long *hi_product,
+ unsigned long *lo_product)
+{
+ int a, b, c, d;
+ long lo, hi, x, y;
+
+ a = (v1 >> 16) & 0xffff;
+ b = v1 & 0xffff;
+ c = (v2 >> 16) & 0xffff;
+ d = v2 & 0xffff;
+
+ lo = b * d; /* BD */
+ x = a * d + c * b; /* AD + CB */
+ y = ((lo >> 16) & 0xffff) + x;
+
+ lo = (lo & 0xffff) | ((y & 0xffff) << 16);
+ hi = (y >> 16) & 0xffff;
+
+ hi += a * c; /* AC */
+
+ *hi_product = (unsigned long)hi;
+ *lo_product = (unsigned long)lo;
+}
+
+int /* PRIVATE */
+png_check_cHRM_fixed(png_structp png_ptr,
+ png_fixed_point white_x, png_fixed_point white_y, png_fixed_point red_x,
+ png_fixed_point red_y, png_fixed_point green_x, png_fixed_point green_y,
+ png_fixed_point blue_x, png_fixed_point blue_y)
+{
+ int ret = 1;
+ unsigned long xy_hi,xy_lo,yx_hi,yx_lo;
+
+ png_debug(1, "in function png_check_cHRM_fixed");
+
+ if (png_ptr == NULL)
+ return 0;
+
+ if (white_x < 0 || white_y <= 0 ||
+ red_x < 0 || red_y < 0 ||
+ green_x < 0 || green_y < 0 ||
+ blue_x < 0 || blue_y < 0)
+ {
+ png_warning(png_ptr,
+ "Ignoring attempt to set negative chromaticity value");
+ ret = 0;
+ }
+ if (white_x > (png_fixed_point) PNG_UINT_31_MAX ||
+ white_y > (png_fixed_point) PNG_UINT_31_MAX ||
+ red_x > (png_fixed_point) PNG_UINT_31_MAX ||
+ red_y > (png_fixed_point) PNG_UINT_31_MAX ||
+ green_x > (png_fixed_point) PNG_UINT_31_MAX ||
+ green_y > (png_fixed_point) PNG_UINT_31_MAX ||
+ blue_x > (png_fixed_point) PNG_UINT_31_MAX ||
+ blue_y > (png_fixed_point) PNG_UINT_31_MAX )
+ {
+ png_warning(png_ptr,
+ "Ignoring attempt to set chromaticity value exceeding 21474.83");
+ ret = 0;
+ }
+ if (white_x > 100000L - white_y)
+ {
+ png_warning(png_ptr, "Invalid cHRM white point");
+ ret = 0;
+ }
+ if (red_x > 100000L - red_y)
+ {
+ png_warning(png_ptr, "Invalid cHRM red point");
+ ret = 0;
+ }
+ if (green_x > 100000L - green_y)
+ {
+ png_warning(png_ptr, "Invalid cHRM green point");
+ ret = 0;
+ }
+ if (blue_x > 100000L - blue_y)
+ {
+ png_warning(png_ptr, "Invalid cHRM blue point");
+ ret = 0;
+ }
+
+ png_64bit_product(green_x - red_x, blue_y - red_y, &xy_hi, &xy_lo);
+ png_64bit_product(green_y - red_y, blue_x - red_x, &yx_hi, &yx_lo);
+
+ if (xy_hi == yx_hi && xy_lo == yx_lo)
+ {
+ png_warning(png_ptr,
+ "Ignoring attempt to set cHRM RGB triangle with zero area");
+ ret = 0;
+ }
+
+ return ret;
+}
+#endif /* PNG_CHECK_cHRM_SUPPORTED */
+#endif /* PNG_cHRM_SUPPORTED */
+
+void /* PRIVATE */
+png_check_IHDR(png_structp png_ptr,
+ png_uint_32 width, png_uint_32 height, int bit_depth,
+ int color_type, int interlace_type, int compression_type,
+ int filter_type)
+{
+ int error = 0;
+
+ /* Check for width and height valid values */
+ if (width == 0)
+ {
+ png_warning(png_ptr, "Image width is zero in IHDR");
+ error = 1;
+ }
+
+ if (height == 0)
+ {
+ png_warning(png_ptr, "Image height is zero in IHDR");
+ error = 1;
+ }
+
+#ifdef PNG_SET_USER_LIMITS_SUPPORTED
+ if (width > png_ptr->user_width_max || width > PNG_USER_WIDTH_MAX)
+#else
+ if (width > PNG_USER_WIDTH_MAX)
+#endif
+ {
+ png_warning(png_ptr, "Image width exceeds user limit in IHDR");
+ error = 1;
+ }
+
+#ifdef PNG_SET_USER_LIMITS_SUPPORTED
+ if (height > png_ptr->user_height_max || height > PNG_USER_HEIGHT_MAX)
+#else
+ if (height > PNG_USER_HEIGHT_MAX)
+#endif
+ {
+ png_warning(png_ptr, "Image height exceeds user limit in IHDR");
+ error = 1;
+ }
+
+ if (width > PNG_UINT_31_MAX)
+ {
+ png_warning(png_ptr, "Invalid image width in IHDR");
+ error = 1;
+ }
+
+ if ( height > PNG_UINT_31_MAX)
+ {
+ png_warning(png_ptr, "Invalid image height in IHDR");
+ error = 1;
+ }
+
+ if ( width > (PNG_UINT_32_MAX
+ >> 3) /* 8-byte RGBA pixels */
+ - 64 /* bigrowbuf hack */
+ - 1 /* filter byte */
+ - 7*8 /* rounding of width to multiple of 8 pixels */
+ - 8) /* extra max_pixel_depth pad */
+ png_warning(png_ptr, "Width is too large for libpng to process pixels");
+
+ /* Check other values */
+ if (bit_depth != 1 && bit_depth != 2 && bit_depth != 4 &&
+ bit_depth != 8 && bit_depth != 16)
+ {
+ png_warning(png_ptr, "Invalid bit depth in IHDR");
+ error = 1;
+ }
+
+ if (color_type < 0 || color_type == 1 ||
+ color_type == 5 || color_type > 6)
+ {
+ png_warning(png_ptr, "Invalid color type in IHDR");
+ error = 1;
+ }
+
+ if (((color_type == PNG_COLOR_TYPE_PALETTE) && bit_depth > 8) ||
+ ((color_type == PNG_COLOR_TYPE_RGB ||
+ color_type == PNG_COLOR_TYPE_GRAY_ALPHA ||
+ color_type == PNG_COLOR_TYPE_RGB_ALPHA) && bit_depth < 8))
+ {
+ png_warning(png_ptr, "Invalid color type/bit depth combination in IHDR");
+ error = 1;
+ }
+
+ if (interlace_type >= PNG_INTERLACE_LAST)
+ {
+ png_warning(png_ptr, "Unknown interlace method in IHDR");
+ error = 1;
+ }
+
+ if (compression_type != PNG_COMPRESSION_TYPE_BASE)
+ {
+ png_warning(png_ptr, "Unknown compression method in IHDR");
+ error = 1;
+ }
+
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+ /* Accept filter_method 64 (intrapixel differencing) only if
+ * 1. Libpng was compiled with PNG_MNG_FEATURES_SUPPORTED and
+ * 2. Libpng did not read a PNG signature (this filter_method is only
+ * used in PNG datastreams that are embedded in MNG datastreams) and
+ * 3. The application called png_permit_mng_features with a mask that
+ * included PNG_FLAG_MNG_FILTER_64 and
+ * 4. The filter_method is 64 and
+ * 5. The color_type is RGB or RGBA
+ */
+ if ((png_ptr->mode & PNG_HAVE_PNG_SIGNATURE) &&
+ png_ptr->mng_features_permitted)
+ png_warning(png_ptr, "MNG features are not allowed in a PNG datastream");
+
+ if (filter_type != PNG_FILTER_TYPE_BASE)
+ {
+ if (!((png_ptr->mng_features_permitted & PNG_FLAG_MNG_FILTER_64) &&
+ (filter_type == PNG_INTRAPIXEL_DIFFERENCING) &&
+ ((png_ptr->mode & PNG_HAVE_PNG_SIGNATURE) == 0) &&
+ (color_type == PNG_COLOR_TYPE_RGB ||
+ color_type == PNG_COLOR_TYPE_RGB_ALPHA)))
+ {
+ png_warning(png_ptr, "Unknown filter method in IHDR");
+ error = 1;
+ }
+
+ if (png_ptr->mode & PNG_HAVE_PNG_SIGNATURE)
+ {
+ png_warning(png_ptr, "Invalid filter method in IHDR");
+ error = 1;
+ }
+ }
+
+#else
+ if (filter_type != PNG_FILTER_TYPE_BASE)
+ {
+ png_warning(png_ptr, "Unknown filter method in IHDR");
+ error = 1;
+ }
+#endif
+
+ if (error == 1)
+ png_error(png_ptr, "Invalid IHDR data");
+}
+#endif /* defined(PNG_READ_SUPPORTED) || defined(PNG_WRITE_SUPPORTED) */
diff --git a/trunk/src/third_party/libpng/png.h b/trunk/src/third_party/libpng/png.h
new file mode 100644
index 0000000..cc1915d
--- /dev/null
+++ b/trunk/src/third_party/libpng/png.h
@@ -0,0 +1,3788 @@
+/* png.h - header file for PNG reference library
+ *
+ * libpng version 1.2.44 - June 26, 2010
+ * Copyright (c) 1998-2010 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license (See LICENSE, below)
+ *
+ * Authors and maintainers:
+ * libpng versions 0.71, May 1995, through 0.88, January 1996: Guy Schalnat
+ * libpng versions 0.89c, June 1996, through 0.96, May 1997: Andreas Dilger
+ * libpng versions 0.97, January 1998, through 1.2.44 - June 26, 2010: Glenn
+ * See also "Contributing Authors", below.
+ *
+ * Note about libpng version numbers:
+ *
+ * Due to various miscommunications, unforeseen code incompatibilities
+ * and occasional factors outside the authors' control, version numbering
+ * on the library has not always been consistent and straightforward.
+ * The following table summarizes matters since version 0.89c, which was
+ * the first widely used release:
+ *
+ * source png.h png.h shared-lib
+ * version string int version
+ * ------- ------ ----- ----------
+ * 0.89c "1.0 beta 3" 0.89 89 1.0.89
+ * 0.90 "1.0 beta 4" 0.90 90 0.90 [should have been 2.0.90]
+ * 0.95 "1.0 beta 5" 0.95 95 0.95 [should have been 2.0.95]
+ * 0.96 "1.0 beta 6" 0.96 96 0.96 [should have been 2.0.96]
+ * 0.97b "1.00.97 beta 7" 1.00.97 97 1.0.1 [should have been 2.0.97]
+ * 0.97c 0.97 97 2.0.97
+ * 0.98 0.98 98 2.0.98
+ * 0.99 0.99 98 2.0.99
+ * 0.99a-m 0.99 99 2.0.99
+ * 1.00 1.00 100 2.1.0 [100 should be 10000]
+ * 1.0.0 (from here on, the 100 2.1.0 [100 should be 10000]
+ * 1.0.1 png.h string is 10001 2.1.0
+ * 1.0.1a-e identical to the 10002 from here on, the shared library
+ * 1.0.2 source version) 10002 is 2.V where V is the source code
+ * 1.0.2a-b 10003 version, except as noted.
+ * 1.0.3 10003
+ * 1.0.3a-d 10004
+ * 1.0.4 10004
+ * 1.0.4a-f 10005
+ * 1.0.5 (+ 2 patches) 10005
+ * 1.0.5a-d 10006
+ * 1.0.5e-r 10100 (not source compatible)
+ * 1.0.5s-v 10006 (not binary compatible)
+ * 1.0.6 (+ 3 patches) 10006 (still binary incompatible)
+ * 1.0.6d-f 10007 (still binary incompatible)
+ * 1.0.6g 10007
+ * 1.0.6h 10007 10.6h (testing xy.z so-numbering)
+ * 1.0.6i 10007 10.6i
+ * 1.0.6j 10007 2.1.0.6j (incompatible with 1.0.0)
+ * 1.0.7beta11-14 DLLNUM 10007 2.1.0.7beta11-14 (binary compatible)
+ * 1.0.7beta15-18 1 10007 2.1.0.7beta15-18 (binary compatible)
+ * 1.0.7rc1-2 1 10007 2.1.0.7rc1-2 (binary compatible)
+ * 1.0.7 1 10007 (still compatible)
+ * 1.0.8beta1-4 1 10008 2.1.0.8beta1-4
+ * 1.0.8rc1 1 10008 2.1.0.8rc1
+ * 1.0.8 1 10008 2.1.0.8
+ * 1.0.9beta1-6 1 10009 2.1.0.9beta1-6
+ * 1.0.9rc1 1 10009 2.1.0.9rc1
+ * 1.0.9beta7-10 1 10009 2.1.0.9beta7-10
+ * 1.0.9rc2 1 10009 2.1.0.9rc2
+ * 1.0.9 1 10009 2.1.0.9
+ * 1.0.10beta1 1 10010 2.1.0.10beta1
+ * 1.0.10rc1 1 10010 2.1.0.10rc1
+ * 1.0.10 1 10010 2.1.0.10
+ * 1.0.11beta1-3 1 10011 2.1.0.11beta1-3
+ * 1.0.11rc1 1 10011 2.1.0.11rc1
+ * 1.0.11 1 10011 2.1.0.11
+ * 1.0.12beta1-2 2 10012 2.1.0.12beta1-2
+ * 1.0.12rc1 2 10012 2.1.0.12rc1
+ * 1.0.12 2 10012 2.1.0.12
+ * 1.1.0a-f - 10100 2.1.1.0a-f (branch abandoned)
+ * 1.2.0beta1-2 2 10200 2.1.2.0beta1-2
+ * 1.2.0beta3-5 3 10200 3.1.2.0beta3-5
+ * 1.2.0rc1 3 10200 3.1.2.0rc1
+ * 1.2.0 3 10200 3.1.2.0
+ * 1.2.1beta1-4 3 10201 3.1.2.1beta1-4
+ * 1.2.1rc1-2 3 10201 3.1.2.1rc1-2
+ * 1.2.1 3 10201 3.1.2.1
+ * 1.2.2beta1-6 12 10202 12.so.0.1.2.2beta1-6
+ * 1.0.13beta1 10 10013 10.so.0.1.0.13beta1
+ * 1.0.13rc1 10 10013 10.so.0.1.0.13rc1
+ * 1.2.2rc1 12 10202 12.so.0.1.2.2rc1
+ * 1.0.13 10 10013 10.so.0.1.0.13
+ * 1.2.2 12 10202 12.so.0.1.2.2
+ * 1.2.3rc1-6 12 10203 12.so.0.1.2.3rc1-6
+ * 1.2.3 12 10203 12.so.0.1.2.3
+ * 1.2.4beta1-3 13 10204 12.so.0.1.2.4beta1-3
+ * 1.0.14rc1 13 10014 10.so.0.1.0.14rc1
+ * 1.2.4rc1 13 10204 12.so.0.1.2.4rc1
+ * 1.0.14 10 10014 10.so.0.1.0.14
+ * 1.2.4 13 10204 12.so.0.1.2.4
+ * 1.2.5beta1-2 13 10205 12.so.0.1.2.5beta1-2
+ * 1.0.15rc1-3 10 10015 10.so.0.1.0.15rc1-3
+ * 1.2.5rc1-3 13 10205 12.so.0.1.2.5rc1-3
+ * 1.0.15 10 10015 10.so.0.1.0.15
+ * 1.2.5 13 10205 12.so.0.1.2.5
+ * 1.2.6beta1-4 13 10206 12.so.0.1.2.6beta1-4
+ * 1.0.16 10 10016 10.so.0.1.0.16
+ * 1.2.6 13 10206 12.so.0.1.2.6
+ * 1.2.7beta1-2 13 10207 12.so.0.1.2.7beta1-2
+ * 1.0.17rc1 10 10017 10.so.0.1.0.17rc1
+ * 1.2.7rc1 13 10207 12.so.0.1.2.7rc1
+ * 1.0.17 10 10017 10.so.0.1.0.17
+ * 1.2.7 13 10207 12.so.0.1.2.7
+ * 1.2.8beta1-5 13 10208 12.so.0.1.2.8beta1-5
+ * 1.0.18rc1-5 10 10018 10.so.0.1.0.18rc1-5
+ * 1.2.8rc1-5 13 10208 12.so.0.1.2.8rc1-5
+ * 1.0.18 10 10018 10.so.0.1.0.18
+ * 1.2.8 13 10208 12.so.0.1.2.8
+ * 1.2.9beta1-3 13 10209 12.so.0.1.2.9beta1-3
+ * 1.2.9beta4-11 13 10209 12.so.0.9[.0]
+ * 1.2.9rc1 13 10209 12.so.0.9[.0]
+ * 1.2.9 13 10209 12.so.0.9[.0]
+ * 1.2.10beta1-8 13 10210 12.so.0.10[.0]
+ * 1.2.10rc1-3 13 10210 12.so.0.10[.0]
+ * 1.2.10 13 10210 12.so.0.10[.0]
+ * 1.2.11beta1-4 13 10211 12.so.0.11[.0]
+ * 1.0.19rc1-5 10 10019 10.so.0.19[.0]
+ * 1.2.11rc1-5 13 10211 12.so.0.11[.0]
+ * 1.0.19 10 10019 10.so.0.19[.0]
+ * 1.2.11 13 10211 12.so.0.11[.0]
+ * 1.0.20 10 10020 10.so.0.20[.0]
+ * 1.2.12 13 10212 12.so.0.12[.0]
+ * 1.2.13beta1 13 10213 12.so.0.13[.0]
+ * 1.0.21 10 10021 10.so.0.21[.0]
+ * 1.2.13 13 10213 12.so.0.13[.0]
+ * 1.2.14beta1-2 13 10214 12.so.0.14[.0]
+ * 1.0.22rc1 10 10022 10.so.0.22[.0]
+ * 1.2.14rc1 13 10214 12.so.0.14[.0]
+ * 1.0.22 10 10022 10.so.0.22[.0]
+ * 1.2.14 13 10214 12.so.0.14[.0]
+ * 1.2.15beta1-6 13 10215 12.so.0.15[.0]
+ * 1.0.23rc1-5 10 10023 10.so.0.23[.0]
+ * 1.2.15rc1-5 13 10215 12.so.0.15[.0]
+ * 1.0.23 10 10023 10.so.0.23[.0]
+ * 1.2.15 13 10215 12.so.0.15[.0]
+ * 1.2.16beta1-2 13 10216 12.so.0.16[.0]
+ * 1.2.16rc1 13 10216 12.so.0.16[.0]
+ * 1.0.24 10 10024 10.so.0.24[.0]
+ * 1.2.16 13 10216 12.so.0.16[.0]
+ * 1.2.17beta1-2 13 10217 12.so.0.17[.0]
+ * 1.0.25rc1 10 10025 10.so.0.25[.0]
+ * 1.2.17rc1-3 13 10217 12.so.0.17[.0]
+ * 1.0.25 10 10025 10.so.0.25[.0]
+ * 1.2.17 13 10217 12.so.0.17[.0]
+ * 1.0.26 10 10026 10.so.0.26[.0]
+ * 1.2.18 13 10218 12.so.0.18[.0]
+ * 1.2.19beta1-31 13 10219 12.so.0.19[.0]
+ * 1.0.27rc1-6 10 10027 10.so.0.27[.0]
+ * 1.2.19rc1-6 13 10219 12.so.0.19[.0]
+ * 1.0.27 10 10027 10.so.0.27[.0]
+ * 1.2.19 13 10219 12.so.0.19[.0]
+ * 1.2.20beta01-04 13 10220 12.so.0.20[.0]
+ * 1.0.28rc1-6 10 10028 10.so.0.28[.0]
+ * 1.2.20rc1-6 13 10220 12.so.0.20[.0]
+ * 1.0.28 10 10028 10.so.0.28[.0]
+ * 1.2.20 13 10220 12.so.0.20[.0]
+ * 1.2.21beta1-2 13 10221 12.so.0.21[.0]
+ * 1.2.21rc1-3 13 10221 12.so.0.21[.0]
+ * 1.0.29 10 10029 10.so.0.29[.0]
+ * 1.2.21 13 10221 12.so.0.21[.0]
+ * 1.2.22beta1-4 13 10222 12.so.0.22[.0]
+ * 1.0.30rc1 10 10030 10.so.0.30[.0]
+ * 1.2.22rc1 13 10222 12.so.0.22[.0]
+ * 1.0.30 10 10030 10.so.0.30[.0]
+ * 1.2.22 13 10222 12.so.0.22[.0]
+ * 1.2.23beta01-05 13 10223 12.so.0.23[.0]
+ * 1.2.23rc01 13 10223 12.so.0.23[.0]
+ * 1.2.23 13 10223 12.so.0.23[.0]
+ * 1.2.24beta01-02 13 10224 12.so.0.24[.0]
+ * 1.2.24rc01 13 10224 12.so.0.24[.0]
+ * 1.2.24 13 10224 12.so.0.24[.0]
+ * 1.2.25beta01-06 13 10225 12.so.0.25[.0]
+ * 1.2.25rc01-02 13 10225 12.so.0.25[.0]
+ * 1.0.31 10 10031 10.so.0.31[.0]
+ * 1.2.25 13 10225 12.so.0.25[.0]
+ * 1.2.26beta01-06 13 10226 12.so.0.26[.0]
+ * 1.2.26rc01 13 10226 12.so.0.26[.0]
+ * 1.2.26 13 10226 12.so.0.26[.0]
+ * 1.0.32 10 10032 10.so.0.32[.0]
+ * 1.2.27beta01-06 13 10227 12.so.0.27[.0]
+ * 1.2.27rc01 13 10227 12.so.0.27[.0]
+ * 1.0.33 10 10033 10.so.0.33[.0]
+ * 1.2.27 13 10227 12.so.0.27[.0]
+ * 1.0.34 10 10034 10.so.0.34[.0]
+ * 1.2.28 13 10228 12.so.0.28[.0]
+ * 1.2.29beta01-03 13 10229 12.so.0.29[.0]
+ * 1.2.29rc01 13 10229 12.so.0.29[.0]
+ * 1.0.35 10 10035 10.so.0.35[.0]
+ * 1.2.29 13 10229 12.so.0.29[.0]
+ * 1.0.37 10 10037 10.so.0.37[.0]
+ * 1.2.30beta01-04 13 10230 12.so.0.30[.0]
+ * 1.0.38rc01-08 10 10038 10.so.0.38[.0]
+ * 1.2.30rc01-08 13 10230 12.so.0.30[.0]
+ * 1.0.38 10 10038 10.so.0.38[.0]
+ * 1.2.30 13 10230 12.so.0.30[.0]
+ * 1.0.39rc01-03 10 10039 10.so.0.39[.0]
+ * 1.2.31rc01-03 13 10231 12.so.0.31[.0]
+ * 1.0.39 10 10039 10.so.0.39[.0]
+ * 1.2.31 13 10231 12.so.0.31[.0]
+ * 1.2.32beta01-02 13 10232 12.so.0.32[.0]
+ * 1.0.40rc01 10 10040 10.so.0.40[.0]
+ * 1.2.32rc01 13 10232 12.so.0.32[.0]
+ * 1.0.40 10 10040 10.so.0.40[.0]
+ * 1.2.32 13 10232 12.so.0.32[.0]
+ * 1.2.33beta01-02 13 10233 12.so.0.33[.0]
+ * 1.2.33rc01-02 13 10233 12.so.0.33[.0]
+ * 1.0.41rc01 10 10041 10.so.0.41[.0]
+ * 1.2.33 13 10233 12.so.0.33[.0]
+ * 1.0.41 10 10041 10.so.0.41[.0]
+ * 1.2.34beta01-07 13 10234 12.so.0.34[.0]
+ * 1.0.42rc01 10 10042 10.so.0.42[.0]
+ * 1.2.34rc01 13 10234 12.so.0.34[.0]
+ * 1.0.42 10 10042 10.so.0.42[.0]
+ * 1.2.34 13 10234 12.so.0.34[.0]
+ * 1.2.35beta01-03 13 10235 12.so.0.35[.0]
+ * 1.0.43rc01-02 10 10043 10.so.0.43[.0]
+ * 1.2.35rc01-02 13 10235 12.so.0.35[.0]
+ * 1.0.43 10 10043 10.so.0.43[.0]
+ * 1.2.35 13 10235 12.so.0.35[.0]
+ * 1.2.36beta01-05 13 10236 12.so.0.36[.0]
+ * 1.2.36rc01 13 10236 12.so.0.36[.0]
+ * 1.0.44 10 10044 10.so.0.44[.0]
+ * 1.2.36 13 10236 12.so.0.36[.0]
+ * 1.2.37beta01-03 13 10237 12.so.0.37[.0]
+ * 1.2.37rc01 13 10237 12.so.0.37[.0]
+ * 1.2.37 13 10237 12.so.0.37[.0]
+ * 1.2.45 10 10045 12.so.0.45[.0]
+ * 1.0.46 10 10046 10.so.0.46[.0]
+ * 1.2.38beta01 13 10238 12.so.0.38[.0]
+ * 1.2.38rc01-03 13 10238 12.so.0.38[.0]
+ * 1.0.47 10 10047 10.so.0.47[.0]
+ * 1.2.38 13 10238 12.so.0.38[.0]
+ * 1.2.39beta01-05 13 10239 12.so.0.39[.0]
+ * 1.2.39rc01 13 10239 12.so.0.39[.0]
+ * 1.0.48 10 10048 10.so.0.48[.0]
+ * 1.2.39 13 10239 12.so.0.39[.0]
+ * 1.2.40beta01 13 10240 12.so.0.40[.0]
+ * 1.2.40rc01 13 10240 12.so.0.40[.0]
+ * 1.0.49 10 10049 10.so.0.49[.0]
+ * 1.2.40 13 10240 12.so.0.40[.0]
+ * 1.2.41beta01-18 13 10241 12.so.0.41[.0]
+ * 1.0.51rc01 10 10051 10.so.0.51[.0]
+ * 1.2.41rc01-03 13 10241 12.so.0.41[.0]
+ * 1.0.51 10 10051 10.so.0.51[.0]
+ * 1.2.41 13 10241 12.so.0.41[.0]
+ * 1.2.42beta01-02 13 10242 12.so.0.42[.0]
+ * 1.2.42rc01-05 13 10242 12.so.0.42[.0]
+ * 1.0.52 10 10052 10.so.0.52[.0]
+ * 1.2.42 13 10242 12.so.0.42[.0]
+ * 1.2.43beta01-05 13 10243 12.so.0.43[.0]
+ * 1.0.53rc01-02 10 10053 10.so.0.53[.0]
+ * 1.2.43rc01-02 13 10243 12.so.0.43[.0]
+ * 1.0.53 10 10053 10.so.0.53[.0]
+ * 1.2.43 13 10243 12.so.0.43[.0]
+ * 1.2.44beta01-03 13 10244 12.so.0.44[.0]
+ * 1.2.44rc01-03 13 10244 12.so.0.44[.0]
+ * 1.2.44 13 10244 12.so.0.44[.0]
+ *
+ * Henceforth the source version will match the shared-library major
+ * and minor numbers; the shared-library major version number will be
+ * used for changes in backward compatibility, as it is intended. The
+ * PNG_LIBPNG_VER macro, which is not used within libpng but is available
+ * for applications, is an unsigned integer of the form xyyzz corresponding
+ * to the source version x.y.z (leading zeros in y and z). Beta versions
+ * were given the previous public release number plus a letter, until
+ * version 1.0.6j; from then on they were given the upcoming public
+ * release number plus "betaNN" or "rcNN".
+ *
+ * Binary incompatibility exists only when applications make direct access
+ * to the info_ptr or png_ptr members through png.h, and the compiled
+ * application is loaded with a different version of the library.
+ *
+ * DLLNUM will change each time there are forward or backward changes
+ * in binary compatibility (e.g., when a new feature is added).
+ *
+ * See libpng.txt or libpng.3 for more information. The PNG specification
+ * is available as a W3C Recommendation and as an ISO Specification,
+ * <http://www.w3.org/TR/2003/REC-PNG-20031110/
+ */
+
+/*
+ * COPYRIGHT NOTICE, DISCLAIMER, and LICENSE:
+ *
+ * If you modify libpng you may insert additional notices immediately following
+ * this sentence.
+ *
+ * This code is released under the libpng license.
+ *
+ * libpng versions 1.2.6, August 15, 2004, through 1.2.44, June 26, 2010, are
+ * Copyright (c) 2004, 2006-2010 Glenn Randers-Pehrson, and are
+ * distributed according to the same disclaimer and license as libpng-1.2.5
+ * with the following individual added to the list of Contributing Authors:
+ *
+ * Cosmin Truta
+ *
+ * libpng versions 1.0.7, July 1, 2000, through 1.2.5, October 3, 2002, are
+ * Copyright (c) 2000-2002 Glenn Randers-Pehrson, and are
+ * distributed according to the same disclaimer and license as libpng-1.0.6
+ * with the following individuals added to the list of Contributing Authors:
+ *
+ * Simon-Pierre Cadieux
+ * Eric S. Raymond
+ * Gilles Vollant
+ *
+ * and with the following additions to the disclaimer:
+ *
+ * There is no warranty against interference with your enjoyment of the
+ * library or against infringement. There is no warranty that our
+ * efforts or the library will fulfill any of your particular purposes
+ * or needs. This library is provided with all faults, and the entire
+ * risk of satisfactory quality, performance, accuracy, and effort is with
+ * the user.
+ *
+ * libpng versions 0.97, January 1998, through 1.0.6, March 20, 2000, are
+ * Copyright (c) 1998, 1999, 2000 Glenn Randers-Pehrson, and are
+ * distributed according to the same disclaimer and license as libpng-0.96,
+ * with the following individuals added to the list of Contributing Authors:
+ *
+ * Tom Lane
+ * Glenn Randers-Pehrson
+ * Willem van Schaik
+ *
+ * libpng versions 0.89, June 1996, through 0.96, May 1997, are
+ * Copyright (c) 1996, 1997 Andreas Dilger
+ * Distributed according to the same disclaimer and license as libpng-0.88,
+ * with the following individuals added to the list of Contributing Authors:
+ *
+ * John Bowler
+ * Kevin Bracey
+ * Sam Bushell
+ * Magnus Holmgren
+ * Greg Roelofs
+ * Tom Tanner
+ *
+ * libpng versions 0.5, May 1995, through 0.88, January 1996, are
+ * Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.
+ *
+ * For the purposes of this copyright and license, "Contributing Authors"
+ * is defined as the following set of individuals:
+ *
+ * Andreas Dilger
+ * Dave Martindale
+ * Guy Eric Schalnat
+ * Paul Schmidt
+ * Tim Wegner
+ *
+ * The PNG Reference Library is supplied "AS IS". The Contributing Authors
+ * and Group 42, Inc. disclaim all warranties, expressed or implied,
+ * including, without limitation, the warranties of merchantability and of
+ * fitness for any purpose. The Contributing Authors and Group 42, Inc.
+ * assume no liability for direct, indirect, incidental, special, exemplary,
+ * or consequential damages, which may result from the use of the PNG
+ * Reference Library, even if advised of the possibility of such damage.
+ *
+ * Permission is hereby granted to use, copy, modify, and distribute this
+ * source code, or portions hereof, for any purpose, without fee, subject
+ * to the following restrictions:
+ *
+ * 1. The origin of this source code must not be misrepresented.
+ *
+ * 2. Altered versions must be plainly marked as such and
+ * must not be misrepresented as being the original source.
+ *
+ * 3. This Copyright notice may not be removed or altered from
+ * any source or altered source distribution.
+ *
+ * The Contributing Authors and Group 42, Inc. specifically permit, without
+ * fee, and encourage the use of this source code as a component to
+ * supporting the PNG file format in commercial products. If you use this
+ * source code in a product, acknowledgment is not required but would be
+ * appreciated.
+ */
+
+/*
+ * A "png_get_copyright" function is available, for convenient use in "about"
+ * boxes and the like:
+ *
+ * printf("%s",png_get_copyright(NULL));
+ *
+ * Also, the PNG logo (in PNG format, of course) is supplied in the
+ * files "pngbar.png" and "pngbar.jpg (88x31) and "pngnow.png" (98x31).
+ */
+
+/*
+ * Libpng is OSI Certified Open Source Software. OSI Certified is a
+ * certification mark of the Open Source Initiative.
+ */
+
+/*
+ * The contributing authors would like to thank all those who helped
+ * with testing, bug fixes, and patience. This wouldn't have been
+ * possible without all of you.
+ *
+ * Thanks to Frank J. T. Wojcik for helping with the documentation.
+ */
+
+/*
+ * Y2K compliance in libpng:
+ * =========================
+ *
+ * June 26, 2010
+ *
+ * Since the PNG Development group is an ad-hoc body, we can't make
+ * an official declaration.
+ *
+ * This is your unofficial assurance that libpng from version 0.71 and
+ * upward through 1.2.44 are Y2K compliant. It is my belief that earlier
+ * versions were also Y2K compliant.
+ *
+ * Libpng only has three year fields. One is a 2-byte unsigned integer
+ * that will hold years up to 65535. The other two hold the date in text
+ * format, and will hold years up to 9999.
+ *
+ * The integer is
+ * "png_uint_16 year" in png_time_struct.
+ *
+ * The strings are
+ * "png_charp time_buffer" in png_struct and
+ * "near_time_buffer", which is a local character string in png.c.
+ *
+ * There are seven time-related functions:
+ * png.c: png_convert_to_rfc_1123() in png.c
+ * (formerly png_convert_to_rfc_1152() in error)
+ * png_convert_from_struct_tm() in pngwrite.c, called in pngwrite.c
+ * png_convert_from_time_t() in pngwrite.c
+ * png_get_tIME() in pngget.c
+ * png_handle_tIME() in pngrutil.c, called in pngread.c
+ * png_set_tIME() in pngset.c
+ * png_write_tIME() in pngwutil.c, called in pngwrite.c
+ *
+ * All handle dates properly in a Y2K environment. The
+ * png_convert_from_time_t() function calls gmtime() to convert from system
+ * clock time, which returns (year - 1900), which we properly convert to
+ * the full 4-digit year. There is a possibility that applications using
+ * libpng are not passing 4-digit years into the png_convert_to_rfc_1123()
+ * function, or that they are incorrectly passing only a 2-digit year
+ * instead of "year - 1900" into the png_convert_from_struct_tm() function,
+ * but this is not under our control. The libpng documentation has always
+ * stated that it works with 4-digit years, and the APIs have been
+ * documented as such.
+ *
+ * The tIME chunk itself is also Y2K compliant. It uses a 2-byte unsigned
+ * integer to hold the year, and can hold years as large as 65535.
+ *
+ * zlib, upon which libpng depends, is also Y2K compliant. It contains
+ * no date-related code.
+ *
+ * Glenn Randers-Pehrson
+ * libpng maintainer
+ * PNG Development Group
+ */
+
+#ifndef PNG_H
+#define PNG_H
+
+/* This is not the place to learn how to use libpng. The file libpng.txt
+ * describes how to use libpng, and the file example.c summarizes it
+ * with some code on which to build. This file is useful for looking
+ * at the actual function definitions and structure components.
+ */
+
+/* Version information for png.h - this should match the version in png.c */
+#define PNG_LIBPNG_VER_STRING "1.2.44"
+#define PNG_HEADER_VERSION_STRING \
+ " libpng version 1.2.44 - June 26, 2010\n"
+
+#define PNG_LIBPNG_VER_SONUM 0
+#define PNG_LIBPNG_VER_DLLNUM 13
+
+/* These should match the first 3 components of PNG_LIBPNG_VER_STRING: */
+#define PNG_LIBPNG_VER_MAJOR 1
+#define PNG_LIBPNG_VER_MINOR 2
+#define PNG_LIBPNG_VER_RELEASE 44
+/* This should match the numeric part of the final component of
+ * PNG_LIBPNG_VER_STRING, omitting any leading zero:
+ */
+
+#define PNG_LIBPNG_VER_BUILD 0
+
+/* Release Status */
+#define PNG_LIBPNG_BUILD_ALPHA 1
+#define PNG_LIBPNG_BUILD_BETA 2
+#define PNG_LIBPNG_BUILD_RC 3
+#define PNG_LIBPNG_BUILD_STABLE 4
+#define PNG_LIBPNG_BUILD_RELEASE_STATUS_MASK 7
+
+/* Release-Specific Flags */
+#define PNG_LIBPNG_BUILD_PATCH 8 /* Can be OR'ed with
+ PNG_LIBPNG_BUILD_STABLE only */
+#define PNG_LIBPNG_BUILD_PRIVATE 16 /* Cannot be OR'ed with
+ PNG_LIBPNG_BUILD_SPECIAL */
+#define PNG_LIBPNG_BUILD_SPECIAL 32 /* Cannot be OR'ed with
+ PNG_LIBPNG_BUILD_PRIVATE */
+
+#define PNG_LIBPNG_BUILD_BASE_TYPE PNG_LIBPNG_BUILD_STABLE
+
+/* Careful here. At one time, Guy wanted to use 082, but that would be octal.
+ * We must not include leading zeros.
+ * Versions 0.7 through 1.0.0 were in the range 0 to 100 here (only
+ * version 1.0.0 was mis-numbered 100 instead of 10000). From
+ * version 1.0.1 it's xxyyzz, where x=major, y=minor, z=release
+ */
+#define PNG_LIBPNG_VER 10244 /* 1.2.44 */
+
+#ifndef PNG_VERSION_INFO_ONLY
+/* Include the compression library's header */
+#include "zlib.h"
+#endif
+
+/* Include all user configurable info, including optional assembler routines */
+#include "pngconf.h"
+
+/*
+ * Added at libpng-1.2.8 */
+/* Ref MSDN: Private as priority over Special
+ * VS_FF_PRIVATEBUILD File *was not* built using standard release
+ * procedures. If this value is given, the StringFileInfo block must
+ * contain a PrivateBuild string.
+ *
+ * VS_FF_SPECIALBUILD File *was* built by the original company using
+ * standard release procedures but is a variation of the standard
+ * file of the same version number. If this value is given, the
+ * StringFileInfo block must contain a SpecialBuild string.
+ */
+
+#ifdef PNG_USER_PRIVATEBUILD
+# define PNG_LIBPNG_BUILD_TYPE \
+ (PNG_LIBPNG_BUILD_BASE_TYPE | PNG_LIBPNG_BUILD_PRIVATE)
+#else
+# ifdef PNG_LIBPNG_SPECIALBUILD
+# define PNG_LIBPNG_BUILD_TYPE \
+ (PNG_LIBPNG_BUILD_BASE_TYPE | PNG_LIBPNG_BUILD_SPECIAL)
+# else
+# define PNG_LIBPNG_BUILD_TYPE (PNG_LIBPNG_BUILD_BASE_TYPE)
+# endif
+#endif
+
+#ifndef PNG_VERSION_INFO_ONLY
+
+/* Inhibit C++ name-mangling for libpng functions but not for system calls. */
+#ifdef __cplusplus
+extern "C" {
+#endif /* __cplusplus */
+
+/* This file is arranged in several sections. The first section contains
+ * structure and type definitions. The second section contains the external
+ * library functions, while the third has the internal library functions,
+ * which applications aren't expected to use directly.
+ */
+
+#ifndef PNG_NO_TYPECAST_NULL
+#define int_p_NULL (int *)NULL
+#define png_bytep_NULL (png_bytep)NULL
+#define png_bytepp_NULL (png_bytepp)NULL
+#define png_doublep_NULL (png_doublep)NULL
+#define png_error_ptr_NULL (png_error_ptr)NULL
+#define png_flush_ptr_NULL (png_flush_ptr)NULL
+#define png_free_ptr_NULL (png_free_ptr)NULL
+#define png_infopp_NULL (png_infopp)NULL
+#define png_malloc_ptr_NULL (png_malloc_ptr)NULL
+#define png_read_status_ptr_NULL (png_read_status_ptr)NULL
+#define png_rw_ptr_NULL (png_rw_ptr)NULL
+#define png_structp_NULL (png_structp)NULL
+#define png_uint_16p_NULL (png_uint_16p)NULL
+#define png_voidp_NULL (png_voidp)NULL
+#define png_write_status_ptr_NULL (png_write_status_ptr)NULL
+#else
+#define int_p_NULL NULL
+#define png_bytep_NULL NULL
+#define png_bytepp_NULL NULL
+#define png_doublep_NULL NULL
+#define png_error_ptr_NULL NULL
+#define png_flush_ptr_NULL NULL
+#define png_free_ptr_NULL NULL
+#define png_infopp_NULL NULL
+#define png_malloc_ptr_NULL NULL
+#define png_read_status_ptr_NULL NULL
+#define png_rw_ptr_NULL NULL
+#define png_structp_NULL NULL
+#define png_uint_16p_NULL NULL
+#define png_voidp_NULL NULL
+#define png_write_status_ptr_NULL NULL
+#endif
+
+/* Variables declared in png.c - only it needs to define PNG_NO_EXTERN */
+#if !defined(PNG_NO_EXTERN) || defined(PNG_ALWAYS_EXTERN)
+/* Version information for C files, stored in png.c. This had better match
+ * the version above.
+ */
+#ifdef PNG_USE_GLOBAL_ARRAYS
+PNG_EXPORT_VAR (PNG_CONST char) png_libpng_ver[18];
+ /* Need room for 99.99.99beta99z */
+#else
+#define png_libpng_ver png_get_header_ver(NULL)
+#endif
+
+#ifdef PNG_USE_GLOBAL_ARRAYS
+/* This was removed in version 1.0.5c */
+/* Structures to facilitate easy interlacing. See png.c for more details */
+PNG_EXPORT_VAR (PNG_CONST int FARDATA) png_pass_start[7];
+PNG_EXPORT_VAR (PNG_CONST int FARDATA) png_pass_inc[7];
+PNG_EXPORT_VAR (PNG_CONST int FARDATA) png_pass_ystart[7];
+PNG_EXPORT_VAR (PNG_CONST int FARDATA) png_pass_yinc[7];
+PNG_EXPORT_VAR (PNG_CONST int FARDATA) png_pass_mask[7];
+PNG_EXPORT_VAR (PNG_CONST int FARDATA) png_pass_dsp_mask[7];
+/* This isn't currently used. If you need it, see png.c for more details.
+PNG_EXPORT_VAR (PNG_CONST int FARDATA) png_pass_height[7];
+*/
+#endif
+
+#endif /* PNG_NO_EXTERN */
+
+/* Three color definitions. The order of the red, green, and blue, (and the
+ * exact size) is not important, although the size of the fields need to
+ * be png_byte or png_uint_16 (as defined below).
+ */
+typedef struct png_color_struct
+{
+ png_byte red;
+ png_byte green;
+ png_byte blue;
+} png_color;
+typedef png_color FAR * png_colorp;
+typedef png_color FAR * FAR * png_colorpp;
+
+typedef struct png_color_16_struct
+{
+ png_byte index; /* used for palette files */
+ png_uint_16 red; /* for use in red green blue files */
+ png_uint_16 green;
+ png_uint_16 blue;
+ png_uint_16 gray; /* for use in grayscale files */
+} png_color_16;
+typedef png_color_16 FAR * png_color_16p;
+typedef png_color_16 FAR * FAR * png_color_16pp;
+
+typedef struct png_color_8_struct
+{
+ png_byte red; /* for use in red green blue files */
+ png_byte green;
+ png_byte blue;
+ png_byte gray; /* for use in grayscale files */
+ png_byte alpha; /* for alpha channel files */
+} png_color_8;
+typedef png_color_8 FAR * png_color_8p;
+typedef png_color_8 FAR * FAR * png_color_8pp;
+
+/*
+ * The following two structures are used for the in-core representation
+ * of sPLT chunks.
+ */
+typedef struct png_sPLT_entry_struct
+{
+ png_uint_16 red;
+ png_uint_16 green;
+ png_uint_16 blue;
+ png_uint_16 alpha;
+ png_uint_16 frequency;
+} png_sPLT_entry;
+typedef png_sPLT_entry FAR * png_sPLT_entryp;
+typedef png_sPLT_entry FAR * FAR * png_sPLT_entrypp;
+
+/* When the depth of the sPLT palette is 8 bits, the color and alpha samples
+ * occupy the LSB of their respective members, and the MSB of each member
+ * is zero-filled. The frequency member always occupies the full 16 bits.
+ */
+
+typedef struct png_sPLT_struct
+{
+ png_charp name; /* palette name */
+ png_byte depth; /* depth of palette samples */
+ png_sPLT_entryp entries; /* palette entries */
+ png_int_32 nentries; /* number of palette entries */
+} png_sPLT_t;
+typedef png_sPLT_t FAR * png_sPLT_tp;
+typedef png_sPLT_t FAR * FAR * png_sPLT_tpp;
+
+#ifdef PNG_TEXT_SUPPORTED
+/* png_text holds the contents of a text/ztxt/itxt chunk in a PNG file,
+ * and whether that contents is compressed or not. The "key" field
+ * points to a regular zero-terminated C string. The "text", "lang", and
+ * "lang_key" fields can be regular C strings, empty strings, or NULL pointers.
+ * However, the * structure returned by png_get_text() will always contain
+ * regular zero-terminated C strings (possibly empty), never NULL pointers,
+ * so they can be safely used in printf() and other string-handling functions.
+ */
+typedef struct png_text_struct
+{
+ int compression; /* compression value:
+ -1: tEXt, none
+ 0: zTXt, deflate
+ 1: iTXt, none
+ 2: iTXt, deflate */
+ png_charp key; /* keyword, 1-79 character description of "text" */
+ png_charp text; /* comment, may be an empty string (ie "")
+ or a NULL pointer */
+ png_size_t text_length; /* length of the text string */
+#ifdef PNG_iTXt_SUPPORTED
+ png_size_t itxt_length; /* length of the itxt string */
+ png_charp lang; /* language code, 0-79 characters
+ or a NULL pointer */
+ png_charp lang_key; /* keyword translated UTF-8 string, 0 or more
+ chars or a NULL pointer */
+#endif
+} png_text;
+typedef png_text FAR * png_textp;
+typedef png_text FAR * FAR * png_textpp;
+#endif
+
+/* Supported compression types for text in PNG files (tEXt, and zTXt).
+ * The values of the PNG_TEXT_COMPRESSION_ defines should NOT be changed.
+ */
+#define PNG_TEXT_COMPRESSION_NONE_WR -3
+#define PNG_TEXT_COMPRESSION_zTXt_WR -2
+#define PNG_TEXT_COMPRESSION_NONE -1
+#define PNG_TEXT_COMPRESSION_zTXt 0
+#define PNG_ITXT_COMPRESSION_NONE 1
+#define PNG_ITXT_COMPRESSION_zTXt 2
+#define PNG_TEXT_COMPRESSION_LAST 3 /* Not a valid value */
+
+/* png_time is a way to hold the time in an machine independent way.
+ * Two conversions are provided, both from time_t and struct tm. There
+ * is no portable way to convert to either of these structures, as far
+ * as I know. If you know of a portable way, send it to me. As a side
+ * note - PNG has always been Year 2000 compliant!
+ */
+typedef struct png_time_struct
+{
+ png_uint_16 year; /* full year, as in, 1995 */
+ png_byte month; /* month of year, 1 - 12 */
+ png_byte day; /* day of month, 1 - 31 */
+ png_byte hour; /* hour of day, 0 - 23 */
+ png_byte minute; /* minute of hour, 0 - 59 */
+ png_byte second; /* second of minute, 0 - 60 (for leap seconds) */
+} png_time;
+typedef png_time FAR * png_timep;
+typedef png_time FAR * FAR * png_timepp;
+
+#if defined(PNG_UNKNOWN_CHUNKS_SUPPORTED) || \
+ defined(PNG_HANDLE_AS_UNKNOWN_SUPPORTED)
+/* png_unknown_chunk is a structure to hold queued chunks for which there is
+ * no specific support. The idea is that we can use this to queue
+ * up private chunks for output even though the library doesn't actually
+ * know about their semantics.
+ */
+#define PNG_CHUNK_NAME_LENGTH 5
+typedef struct png_unknown_chunk_t
+{
+ png_byte name[PNG_CHUNK_NAME_LENGTH];
+ png_byte *data;
+ png_size_t size;
+
+ /* libpng-using applications should NOT directly modify this byte. */
+ png_byte location; /* mode of operation at read time */
+}
+png_unknown_chunk;
+typedef png_unknown_chunk FAR * png_unknown_chunkp;
+typedef png_unknown_chunk FAR * FAR * png_unknown_chunkpp;
+#endif
+
+/* png_info is a structure that holds the information in a PNG file so
+ * that the application can find out the characteristics of the image.
+ * If you are reading the file, this structure will tell you what is
+ * in the PNG file. If you are writing the file, fill in the information
+ * you want to put into the PNG file, then call png_write_info().
+ * The names chosen should be very close to the PNG specification, so
+ * consult that document for information about the meaning of each field.
+ *
+ * With libpng < 0.95, it was only possible to directly set and read the
+ * the values in the png_info_struct, which meant that the contents and
+ * order of the values had to remain fixed. With libpng 0.95 and later,
+ * however, there are now functions that abstract the contents of
+ * png_info_struct from the application, so this makes it easier to use
+ * libpng with dynamic libraries, and even makes it possible to use
+ * libraries that don't have all of the libpng ancillary chunk-handing
+ * functionality.
+ *
+ * In any case, the order of the parameters in png_info_struct should NOT
+ * be changed for as long as possible to keep compatibility with applications
+ * that use the old direct-access method with png_info_struct.
+ *
+ * The following members may have allocated storage attached that should be
+ * cleaned up before the structure is discarded: palette, trans, text,
+ * pcal_purpose, pcal_units, pcal_params, hist, iccp_name, iccp_profile,
+ * splt_palettes, scal_unit, row_pointers, and unknowns. By default, these
+ * are automatically freed when the info structure is deallocated, if they were
+ * allocated internally by libpng. This behavior can be changed by means
+ * of the png_data_freer() function.
+ *
+ * More allocation details: all the chunk-reading functions that
+ * change these members go through the corresponding png_set_*
+ * functions. A function to clear these members is available: see
+ * png_free_data(). The png_set_* functions do not depend on being
+ * able to point info structure members to any of the storage they are
+ * passed (they make their own copies), EXCEPT that the png_set_text
+ * functions use the same storage passed to them in the text_ptr or
+ * itxt_ptr structure argument, and the png_set_rows and png_set_unknowns
+ * functions do not make their own copies.
+ */
+typedef struct png_info_struct
+{
+ /* The following are necessary for every PNG file */
+ png_uint_32 width PNG_DEPSTRUCT; /* width of image in pixels (from IHDR) */
+ png_uint_32 height PNG_DEPSTRUCT; /* height of image in pixels (from IHDR) */
+ png_uint_32 valid PNG_DEPSTRUCT; /* valid chunk data (see PNG_INFO_ below) */
+ png_uint_32 rowbytes PNG_DEPSTRUCT; /* bytes needed to hold an untransformed row */
+ png_colorp palette PNG_DEPSTRUCT; /* array of color values (valid & PNG_INFO_PLTE) */
+ png_uint_16 num_palette PNG_DEPSTRUCT; /* number of color entries in "palette" (PLTE) */
+ png_uint_16 num_trans PNG_DEPSTRUCT; /* number of transparent palette color (tRNS) */
+ png_byte bit_depth PNG_DEPSTRUCT; /* 1, 2, 4, 8, or 16 bits/channel (from IHDR) */
+ png_byte color_type PNG_DEPSTRUCT; /* see PNG_COLOR_TYPE_ below (from IHDR) */
+ /* The following three should have been named *_method not *_type */
+ png_byte compression_type PNG_DEPSTRUCT; /* must be PNG_COMPRESSION_TYPE_BASE (IHDR) */
+ png_byte filter_type PNG_DEPSTRUCT; /* must be PNG_FILTER_TYPE_BASE (from IHDR) */
+ png_byte interlace_type PNG_DEPSTRUCT; /* One of PNG_INTERLACE_NONE, PNG_INTERLACE_ADAM7 */
+
+ /* The following is informational only on read, and not used on writes. */
+ png_byte channels PNG_DEPSTRUCT; /* number of data channels per pixel (1, 2, 3, 4) */
+ png_byte pixel_depth PNG_DEPSTRUCT; /* number of bits per pixel */
+ png_byte spare_byte PNG_DEPSTRUCT; /* to align the data, and for future use */
+ png_byte signature[8] PNG_DEPSTRUCT; /* magic bytes read by libpng from start of file */
+
+ /* The rest of the data is optional. If you are reading, check the
+ * valid field to see if the information in these are valid. If you
+ * are writing, set the valid field to those chunks you want written,
+ * and initialize the appropriate fields below.
+ */
+
+#if defined(PNG_gAMA_SUPPORTED) && defined(PNG_FLOATING_POINT_SUPPORTED)
+ /* The gAMA chunk describes the gamma characteristics of the system
+ * on which the image was created, normally in the range [1.0, 2.5].
+ * Data is valid if (valid & PNG_INFO_gAMA) is non-zero.
+ */
+ float gamma PNG_DEPSTRUCT; /* gamma value of image, if (valid & PNG_INFO_gAMA) */
+#endif
+
+#ifdef PNG_sRGB_SUPPORTED
+ /* GR-P, 0.96a */
+ /* Data valid if (valid & PNG_INFO_sRGB) non-zero. */
+ png_byte srgb_intent PNG_DEPSTRUCT; /* sRGB rendering intent [0, 1, 2, or 3] */
+#endif
+
+#ifdef PNG_TEXT_SUPPORTED
+ /* The tEXt, and zTXt chunks contain human-readable textual data in
+ * uncompressed, compressed, and optionally compressed forms, respectively.
+ * The data in "text" is an array of pointers to uncompressed,
+ * null-terminated C strings. Each chunk has a keyword that describes the
+ * textual data contained in that chunk. Keywords are not required to be
+ * unique, and the text string may be empty. Any number of text chunks may
+ * be in an image.
+ */
+ int num_text PNG_DEPSTRUCT; /* number of comments read/to write */
+ int max_text PNG_DEPSTRUCT; /* current size of text array */
+ png_textp text PNG_DEPSTRUCT; /* array of comments read/to write */
+#endif /* PNG_TEXT_SUPPORTED */
+
+#ifdef PNG_tIME_SUPPORTED
+ /* The tIME chunk holds the last time the displayed image data was
+ * modified. See the png_time struct for the contents of this struct.
+ */
+ png_time mod_time PNG_DEPSTRUCT;
+#endif
+
+#ifdef PNG_sBIT_SUPPORTED
+ /* The sBIT chunk specifies the number of significant high-order bits
+ * in the pixel data. Values are in the range [1, bit_depth], and are
+ * only specified for the channels in the pixel data. The contents of
+ * the low-order bits is not specified. Data is valid if
+ * (valid & PNG_INFO_sBIT) is non-zero.
+ */
+ png_color_8 sig_bit PNG_DEPSTRUCT; /* significant bits in color channels */
+#endif
+
+#if defined(PNG_tRNS_SUPPORTED) || defined(PNG_READ_EXPAND_SUPPORTED) || \
+defined(PNG_READ_BACKGROUND_SUPPORTED)
+ /* The tRNS chunk supplies transparency data for paletted images and
+ * other image types that don't need a full alpha channel. There are
+ * "num_trans" transparency values for a paletted image, stored in the
+ * same order as the palette colors, starting from index 0. Values
+ * for the data are in the range [0, 255], ranging from fully transparent
+ * to fully opaque, respectively. For non-paletted images, there is a
+ * single color specified that should be treated as fully transparent.
+ * Data is valid if (valid & PNG_INFO_tRNS) is non-zero.
+ */
+ png_bytep trans PNG_DEPSTRUCT; /* transparent values for paletted image */
+ png_color_16 trans_values PNG_DEPSTRUCT; /* transparent color for non-palette image */
+#endif
+
+#if defined(PNG_bKGD_SUPPORTED) || defined(PNG_READ_BACKGROUND_SUPPORTED)
+ /* The bKGD chunk gives the suggested image background color if the
+ * display program does not have its own background color and the image
+ * is needs to composited onto a background before display. The colors
+ * in "background" are normally in the same color space/depth as the
+ * pixel data. Data is valid if (valid & PNG_INFO_bKGD) is non-zero.
+ */
+ png_color_16 background PNG_DEPSTRUCT;
+#endif
+
+#ifdef PNG_oFFs_SUPPORTED
+ /* The oFFs chunk gives the offset in "offset_unit_type" units rightwards
+ * and downwards from the top-left corner of the display, page, or other
+ * application-specific co-ordinate space. See the PNG_OFFSET_ defines
+ * below for the unit types. Valid if (valid & PNG_INFO_oFFs) non-zero.
+ */
+ png_int_32 x_offset PNG_DEPSTRUCT; /* x offset on page */
+ png_int_32 y_offset PNG_DEPSTRUCT; /* y offset on page */
+ png_byte offset_unit_type PNG_DEPSTRUCT; /* offset units type */
+#endif
+
+#ifdef PNG_pHYs_SUPPORTED
+ /* The pHYs chunk gives the physical pixel density of the image for
+ * display or printing in "phys_unit_type" units (see PNG_RESOLUTION_
+ * defines below). Data is valid if (valid & PNG_INFO_pHYs) is non-zero.
+ */
+ png_uint_32 x_pixels_per_unit PNG_DEPSTRUCT; /* horizontal pixel density */
+ png_uint_32 y_pixels_per_unit PNG_DEPSTRUCT; /* vertical pixel density */
+ png_byte phys_unit_type PNG_DEPSTRUCT; /* resolution type (see PNG_RESOLUTION_ below) */
+#endif
+
+#ifdef PNG_hIST_SUPPORTED
+ /* The hIST chunk contains the relative frequency or importance of the
+ * various palette entries, so that a viewer can intelligently select a
+ * reduced-color palette, if required. Data is an array of "num_palette"
+ * values in the range [0,65535]. Data valid if (valid & PNG_INFO_hIST)
+ * is non-zero.
+ */
+ png_uint_16p hist PNG_DEPSTRUCT;
+#endif
+
+#ifdef PNG_cHRM_SUPPORTED
+ /* The cHRM chunk describes the CIE color characteristics of the monitor
+ * on which the PNG was created. This data allows the viewer to do gamut
+ * mapping of the input image to ensure that the viewer sees the same
+ * colors in the image as the creator. Values are in the range
+ * [0.0, 0.8]. Data valid if (valid & PNG_INFO_cHRM) non-zero.
+ */
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ float x_white PNG_DEPSTRUCT;
+ float y_white PNG_DEPSTRUCT;
+ float x_red PNG_DEPSTRUCT;
+ float y_red PNG_DEPSTRUCT;
+ float x_green PNG_DEPSTRUCT;
+ float y_green PNG_DEPSTRUCT;
+ float x_blue PNG_DEPSTRUCT;
+ float y_blue PNG_DEPSTRUCT;
+#endif
+#endif
+
+#ifdef PNG_pCAL_SUPPORTED
+ /* The pCAL chunk describes a transformation between the stored pixel
+ * values and original physical data values used to create the image.
+ * The integer range [0, 2^bit_depth - 1] maps to the floating-point
+ * range given by [pcal_X0, pcal_X1], and are further transformed by a
+ * (possibly non-linear) transformation function given by "pcal_type"
+ * and "pcal_params" into "pcal_units". Please see the PNG_EQUATION_
+ * defines below, and the PNG-Group's PNG extensions document for a
+ * complete description of the transformations and how they should be
+ * implemented, and for a description of the ASCII parameter strings.
+ * Data values are valid if (valid & PNG_INFO_pCAL) non-zero.
+ */
+ png_charp pcal_purpose PNG_DEPSTRUCT; /* pCAL chunk description string */
+ png_int_32 pcal_X0 PNG_DEPSTRUCT; /* minimum value */
+ png_int_32 pcal_X1 PNG_DEPSTRUCT; /* maximum value */
+ png_charp pcal_units PNG_DEPSTRUCT; /* Latin-1 string giving physical units */
+ png_charpp pcal_params PNG_DEPSTRUCT; /* ASCII strings containing parameter values */
+ png_byte pcal_type PNG_DEPSTRUCT; /* equation type (see PNG_EQUATION_ below) */
+ png_byte pcal_nparams PNG_DEPSTRUCT; /* number of parameters given in pcal_params */
+#endif
+
+/* New members added in libpng-1.0.6 */
+#ifdef PNG_FREE_ME_SUPPORTED
+ png_uint_32 free_me PNG_DEPSTRUCT; /* flags items libpng is responsible for freeing */
+#endif
+
+#if defined(PNG_UNKNOWN_CHUNKS_SUPPORTED) || \
+ defined(PNG_HANDLE_AS_UNKNOWN_SUPPORTED)
+ /* Storage for unknown chunks that the library doesn't recognize. */
+ png_unknown_chunkp unknown_chunks PNG_DEPSTRUCT;
+ png_size_t unknown_chunks_num PNG_DEPSTRUCT;
+#endif
+
+#ifdef PNG_iCCP_SUPPORTED
+ /* iCCP chunk data. */
+ png_charp iccp_name PNG_DEPSTRUCT; /* profile name */
+ png_charp iccp_profile PNG_DEPSTRUCT; /* International Color Consortium profile data */
+ /* Note to maintainer: should be png_bytep */
+ png_uint_32 iccp_proflen PNG_DEPSTRUCT; /* ICC profile data length */
+ png_byte iccp_compression PNG_DEPSTRUCT; /* Always zero */
+#endif
+
+#ifdef PNG_sPLT_SUPPORTED
+ /* Data on sPLT chunks (there may be more than one). */
+ png_sPLT_tp splt_palettes PNG_DEPSTRUCT;
+ png_uint_32 splt_palettes_num PNG_DEPSTRUCT;
+#endif
+
+#ifdef PNG_sCAL_SUPPORTED
+ /* The sCAL chunk describes the actual physical dimensions of the
+ * subject matter of the graphic. The chunk contains a unit specification
+ * a byte value, and two ASCII strings representing floating-point
+ * values. The values are width and height corresponsing to one pixel
+ * in the image. This external representation is converted to double
+ * here. Data values are valid if (valid & PNG_INFO_sCAL) is non-zero.
+ */
+ png_byte scal_unit PNG_DEPSTRUCT; /* unit of physical scale */
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ double scal_pixel_width PNG_DEPSTRUCT; /* width of one pixel */
+ double scal_pixel_height PNG_DEPSTRUCT; /* height of one pixel */
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ png_charp scal_s_width PNG_DEPSTRUCT; /* string containing height */
+ png_charp scal_s_height PNG_DEPSTRUCT; /* string containing width */
+#endif
+#endif
+
+#ifdef PNG_INFO_IMAGE_SUPPORTED
+ /* Memory has been allocated if (valid & PNG_ALLOCATED_INFO_ROWS) non-zero */
+ /* Data valid if (valid & PNG_INFO_IDAT) non-zero */
+ png_bytepp row_pointers PNG_DEPSTRUCT; /* the image bits */
+#endif
+
+#if defined(PNG_FIXED_POINT_SUPPORTED) && defined(PNG_gAMA_SUPPORTED)
+ png_fixed_point int_gamma PNG_DEPSTRUCT; /* gamma of image, if (valid & PNG_INFO_gAMA) */
+#endif
+
+#if defined(PNG_cHRM_SUPPORTED) && defined(PNG_FIXED_POINT_SUPPORTED)
+ png_fixed_point int_x_white PNG_DEPSTRUCT;
+ png_fixed_point int_y_white PNG_DEPSTRUCT;
+ png_fixed_point int_x_red PNG_DEPSTRUCT;
+ png_fixed_point int_y_red PNG_DEPSTRUCT;
+ png_fixed_point int_x_green PNG_DEPSTRUCT;
+ png_fixed_point int_y_green PNG_DEPSTRUCT;
+ png_fixed_point int_x_blue PNG_DEPSTRUCT;
+ png_fixed_point int_y_blue PNG_DEPSTRUCT;
+#endif
+
+} png_info;
+
+typedef png_info FAR * png_infop;
+typedef png_info FAR * FAR * png_infopp;
+
+/* Maximum positive integer used in PNG is (2^31)-1 */
+#define PNG_UINT_31_MAX ((png_uint_32)0x7fffffffL)
+#define PNG_UINT_32_MAX ((png_uint_32)(-1))
+#define PNG_SIZE_MAX ((png_size_t)(-1))
+#if defined(PNG_1_0_X) || defined (PNG_1_2_X)
+/* PNG_MAX_UINT is deprecated; use PNG_UINT_31_MAX instead. */
+#define PNG_MAX_UINT PNG_UINT_31_MAX
+#endif
+
+/* These describe the color_type field in png_info. */
+/* color type masks */
+#define PNG_COLOR_MASK_PALETTE 1
+#define PNG_COLOR_MASK_COLOR 2
+#define PNG_COLOR_MASK_ALPHA 4
+
+/* color types. Note that not all combinations are legal */
+#define PNG_COLOR_TYPE_GRAY 0
+#define PNG_COLOR_TYPE_PALETTE (PNG_COLOR_MASK_COLOR | PNG_COLOR_MASK_PALETTE)
+#define PNG_COLOR_TYPE_RGB (PNG_COLOR_MASK_COLOR)
+#define PNG_COLOR_TYPE_RGB_ALPHA (PNG_COLOR_MASK_COLOR | PNG_COLOR_MASK_ALPHA)
+#define PNG_COLOR_TYPE_GRAY_ALPHA (PNG_COLOR_MASK_ALPHA)
+/* aliases */
+#define PNG_COLOR_TYPE_RGBA PNG_COLOR_TYPE_RGB_ALPHA
+#define PNG_COLOR_TYPE_GA PNG_COLOR_TYPE_GRAY_ALPHA
+
+/* This is for compression type. PNG 1.0-1.2 only define the single type. */
+#define PNG_COMPRESSION_TYPE_BASE 0 /* Deflate method 8, 32K window */
+#define PNG_COMPRESSION_TYPE_DEFAULT PNG_COMPRESSION_TYPE_BASE
+
+/* This is for filter type. PNG 1.0-1.2 only define the single type. */
+#define PNG_FILTER_TYPE_BASE 0 /* Single row per-byte filtering */
+#define PNG_INTRAPIXEL_DIFFERENCING 64 /* Used only in MNG datastreams */
+#define PNG_FILTER_TYPE_DEFAULT PNG_FILTER_TYPE_BASE
+
+/* These are for the interlacing type. These values should NOT be changed. */
+#define PNG_INTERLACE_NONE 0 /* Non-interlaced image */
+#define PNG_INTERLACE_ADAM7 1 /* Adam7 interlacing */
+#define PNG_INTERLACE_LAST 2 /* Not a valid value */
+
+/* These are for the oFFs chunk. These values should NOT be changed. */
+#define PNG_OFFSET_PIXEL 0 /* Offset in pixels */
+#define PNG_OFFSET_MICROMETER 1 /* Offset in micrometers (1/10^6 meter) */
+#define PNG_OFFSET_LAST 2 /* Not a valid value */
+
+/* These are for the pCAL chunk. These values should NOT be changed. */
+#define PNG_EQUATION_LINEAR 0 /* Linear transformation */
+#define PNG_EQUATION_BASE_E 1 /* Exponential base e transform */
+#define PNG_EQUATION_ARBITRARY 2 /* Arbitrary base exponential transform */
+#define PNG_EQUATION_HYPERBOLIC 3 /* Hyperbolic sine transformation */
+#define PNG_EQUATION_LAST 4 /* Not a valid value */
+
+/* These are for the sCAL chunk. These values should NOT be changed. */
+#define PNG_SCALE_UNKNOWN 0 /* unknown unit (image scale) */
+#define PNG_SCALE_METER 1 /* meters per pixel */
+#define PNG_SCALE_RADIAN 2 /* radians per pixel */
+#define PNG_SCALE_LAST 3 /* Not a valid value */
+
+/* These are for the pHYs chunk. These values should NOT be changed. */
+#define PNG_RESOLUTION_UNKNOWN 0 /* pixels/unknown unit (aspect ratio) */
+#define PNG_RESOLUTION_METER 1 /* pixels/meter */
+#define PNG_RESOLUTION_LAST 2 /* Not a valid value */
+
+/* These are for the sRGB chunk. These values should NOT be changed. */
+#define PNG_sRGB_INTENT_PERCEPTUAL 0
+#define PNG_sRGB_INTENT_RELATIVE 1
+#define PNG_sRGB_INTENT_SATURATION 2
+#define PNG_sRGB_INTENT_ABSOLUTE 3
+#define PNG_sRGB_INTENT_LAST 4 /* Not a valid value */
+
+/* This is for text chunks */
+#define PNG_KEYWORD_MAX_LENGTH 79
+
+/* Maximum number of entries in PLTE/sPLT/tRNS arrays */
+#define PNG_MAX_PALETTE_LENGTH 256
+
+/* These determine if an ancillary chunk's data has been successfully read
+ * from the PNG header, or if the application has filled in the corresponding
+ * data in the info_struct to be written into the output file. The values
+ * of the PNG_INFO_<chunk> defines should NOT be changed.
+ */
+#define PNG_INFO_gAMA 0x0001
+#define PNG_INFO_sBIT 0x0002
+#define PNG_INFO_cHRM 0x0004
+#define PNG_INFO_PLTE 0x0008
+#define PNG_INFO_tRNS 0x0010
+#define PNG_INFO_bKGD 0x0020
+#define PNG_INFO_hIST 0x0040
+#define PNG_INFO_pHYs 0x0080
+#define PNG_INFO_oFFs 0x0100
+#define PNG_INFO_tIME 0x0200
+#define PNG_INFO_pCAL 0x0400
+#define PNG_INFO_sRGB 0x0800 /* GR-P, 0.96a */
+#define PNG_INFO_iCCP 0x1000 /* ESR, 1.0.6 */
+#define PNG_INFO_sPLT 0x2000 /* ESR, 1.0.6 */
+#define PNG_INFO_sCAL 0x4000 /* ESR, 1.0.6 */
+#define PNG_INFO_IDAT 0x8000L /* ESR, 1.0.6 */
+
+/* This is used for the transformation routines, as some of them
+ * change these values for the row. It also should enable using
+ * the routines for other purposes.
+ */
+typedef struct png_row_info_struct
+{
+ png_uint_32 width; /* width of row */
+ png_uint_32 rowbytes; /* number of bytes in row */
+ png_byte color_type; /* color type of row */
+ png_byte bit_depth; /* bit depth of row */
+ png_byte channels; /* number of channels (1, 2, 3, or 4) */
+ png_byte pixel_depth; /* bits per pixel (depth * channels) */
+} png_row_info;
+
+typedef png_row_info FAR * png_row_infop;
+typedef png_row_info FAR * FAR * png_row_infopp;
+
+/* These are the function types for the I/O functions and for the functions
+ * that allow the user to override the default I/O functions with his or her
+ * own. The png_error_ptr type should match that of user-supplied warning
+ * and error functions, while the png_rw_ptr type should match that of the
+ * user read/write data functions.
+ */
+typedef struct png_struct_def png_struct;
+typedef png_struct FAR * png_structp;
+
+typedef void (PNGAPI *png_error_ptr) PNGARG((png_structp, png_const_charp));
+typedef void (PNGAPI *png_rw_ptr) PNGARG((png_structp, png_bytep, png_size_t));
+typedef void (PNGAPI *png_flush_ptr) PNGARG((png_structp));
+typedef void (PNGAPI *png_read_status_ptr) PNGARG((png_structp, png_uint_32,
+ int));
+typedef void (PNGAPI *png_write_status_ptr) PNGARG((png_structp, png_uint_32,
+ int));
+
+#ifdef PNG_PROGRESSIVE_READ_SUPPORTED
+typedef void (PNGAPI *png_progressive_info_ptr) PNGARG((png_structp, png_infop));
+typedef void (PNGAPI *png_progressive_end_ptr) PNGARG((png_structp, png_infop));
+typedef void (PNGAPI *png_progressive_row_ptr) PNGARG((png_structp, png_bytep,
+ png_uint_32, int));
+#endif
+
+#if defined(PNG_READ_USER_TRANSFORM_SUPPORTED) || \
+ defined(PNG_WRITE_USER_TRANSFORM_SUPPORTED) || \
+ defined(PNG_LEGACY_SUPPORTED)
+typedef void (PNGAPI *png_user_transform_ptr) PNGARG((png_structp,
+ png_row_infop, png_bytep));
+#endif
+
+#ifdef PNG_USER_CHUNKS_SUPPORTED
+typedef int (PNGAPI *png_user_chunk_ptr) PNGARG((png_structp, png_unknown_chunkp));
+#endif
+#ifdef PNG_UNKNOWN_CHUNKS_SUPPORTED
+typedef void (PNGAPI *png_unknown_chunk_ptr) PNGARG((png_structp));
+#endif
+
+/* Transform masks for the high-level interface */
+#define PNG_TRANSFORM_IDENTITY 0x0000 /* read and write */
+#define PNG_TRANSFORM_STRIP_16 0x0001 /* read only */
+#define PNG_TRANSFORM_STRIP_ALPHA 0x0002 /* read only */
+#define PNG_TRANSFORM_PACKING 0x0004 /* read and write */
+#define PNG_TRANSFORM_PACKSWAP 0x0008 /* read and write */
+#define PNG_TRANSFORM_EXPAND 0x0010 /* read only */
+#define PNG_TRANSFORM_INVERT_MONO 0x0020 /* read and write */
+#define PNG_TRANSFORM_SHIFT 0x0040 /* read and write */
+#define PNG_TRANSFORM_BGR 0x0080 /* read and write */
+#define PNG_TRANSFORM_SWAP_ALPHA 0x0100 /* read and write */
+#define PNG_TRANSFORM_SWAP_ENDIAN 0x0200 /* read and write */
+#define PNG_TRANSFORM_INVERT_ALPHA 0x0400 /* read and write */
+#define PNG_TRANSFORM_STRIP_FILLER 0x0800 /* write only, deprecated */
+/* Added to libpng-1.2.34 */
+#define PNG_TRANSFORM_STRIP_FILLER_BEFORE 0x0800 /* write only */
+#define PNG_TRANSFORM_STRIP_FILLER_AFTER 0x1000 /* write only */
+/* Added to libpng-1.2.41 */
+#define PNG_TRANSFORM_GRAY_TO_RGB 0x2000 /* read only */
+
+/* Flags for MNG supported features */
+#define PNG_FLAG_MNG_EMPTY_PLTE 0x01
+#define PNG_FLAG_MNG_FILTER_64 0x04
+#define PNG_ALL_MNG_FEATURES 0x05
+
+typedef png_voidp (*png_malloc_ptr) PNGARG((png_structp, png_size_t));
+typedef void (*png_free_ptr) PNGARG((png_structp, png_voidp));
+
+/* The structure that holds the information to read and write PNG files.
+ * The only people who need to care about what is inside of this are the
+ * people who will be modifying the library for their own special needs.
+ * It should NOT be accessed directly by an application, except to store
+ * the jmp_buf.
+ */
+
+struct png_struct_def
+{
+#ifdef PNG_SETJMP_SUPPORTED
+ jmp_buf jmpbuf; /* used in png_error */
+#endif
+ png_error_ptr error_fn PNG_DEPSTRUCT; /* function for printing errors and aborting */
+ png_error_ptr warning_fn PNG_DEPSTRUCT; /* function for printing warnings */
+ png_voidp error_ptr PNG_DEPSTRUCT; /* user supplied struct for error functions */
+ png_rw_ptr write_data_fn PNG_DEPSTRUCT; /* function for writing output data */
+ png_rw_ptr read_data_fn PNG_DEPSTRUCT; /* function for reading input data */
+ png_voidp io_ptr PNG_DEPSTRUCT; /* ptr to application struct for I/O functions */
+
+#ifdef PNG_READ_USER_TRANSFORM_SUPPORTED
+ png_user_transform_ptr read_user_transform_fn PNG_DEPSTRUCT; /* user read transform */
+#endif
+
+#ifdef PNG_WRITE_USER_TRANSFORM_SUPPORTED
+ png_user_transform_ptr write_user_transform_fn PNG_DEPSTRUCT; /* user write transform */
+#endif
+
+/* These were added in libpng-1.0.2 */
+#ifdef PNG_USER_TRANSFORM_PTR_SUPPORTED
+#if defined(PNG_READ_USER_TRANSFORM_SUPPORTED) || \
+ defined(PNG_WRITE_USER_TRANSFORM_SUPPORTED)
+ png_voidp user_transform_ptr PNG_DEPSTRUCT; /* user supplied struct for user transform */
+ png_byte user_transform_depth PNG_DEPSTRUCT; /* bit depth of user transformed pixels */
+ png_byte user_transform_channels PNG_DEPSTRUCT; /* channels in user transformed pixels */
+#endif
+#endif
+
+ png_uint_32 mode PNG_DEPSTRUCT; /* tells us where we are in the PNG file */
+ png_uint_32 flags PNG_DEPSTRUCT; /* flags indicating various things to libpng */
+ png_uint_32 transformations PNG_DEPSTRUCT; /* which transformations to perform */
+
+ z_stream zstream PNG_DEPSTRUCT; /* pointer to decompression structure (below) */
+ png_bytep zbuf PNG_DEPSTRUCT; /* buffer for zlib */
+ png_size_t zbuf_size PNG_DEPSTRUCT; /* size of zbuf */
+ int zlib_level PNG_DEPSTRUCT; /* holds zlib compression level */
+ int zlib_method PNG_DEPSTRUCT; /* holds zlib compression method */
+ int zlib_window_bits PNG_DEPSTRUCT; /* holds zlib compression window bits */
+ int zlib_mem_level PNG_DEPSTRUCT; /* holds zlib compression memory level */
+ int zlib_strategy PNG_DEPSTRUCT; /* holds zlib compression strategy */
+
+ png_uint_32 width PNG_DEPSTRUCT; /* width of image in pixels */
+ png_uint_32 height PNG_DEPSTRUCT; /* height of image in pixels */
+ png_uint_32 num_rows PNG_DEPSTRUCT; /* number of rows in current pass */
+ png_uint_32 usr_width PNG_DEPSTRUCT; /* width of row at start of write */
+ png_uint_32 rowbytes PNG_DEPSTRUCT; /* size of row in bytes */
+#if 0 /* Replaced with the following in libpng-1.2.43 */
+ png_size_t irowbytes PNG_DEPSTRUCT;
+#endif
+/* Added in libpng-1.2.43 */
+#ifdef PNG_USER_LIMITS_SUPPORTED
+ /* Added in libpng-1.4.0: Total number of sPLT, text, and unknown
+ * chunks that can be stored (0 means unlimited).
+ */
+ png_uint_32 user_chunk_cache_max PNG_DEPSTRUCT;
+#endif
+ png_uint_32 iwidth PNG_DEPSTRUCT; /* width of current interlaced row in pixels */
+ png_uint_32 row_number PNG_DEPSTRUCT; /* current row in interlace pass */
+ png_bytep prev_row PNG_DEPSTRUCT; /* buffer to save previous (unfiltered) row */
+ png_bytep row_buf PNG_DEPSTRUCT; /* buffer to save current (unfiltered) row */
+#ifndef PNG_NO_WRITE_FILTER
+ png_bytep sub_row PNG_DEPSTRUCT; /* buffer to save "sub" row when filtering */
+ png_bytep up_row PNG_DEPSTRUCT; /* buffer to save "up" row when filtering */
+ png_bytep avg_row PNG_DEPSTRUCT; /* buffer to save "avg" row when filtering */
+ png_bytep paeth_row PNG_DEPSTRUCT; /* buffer to save "Paeth" row when filtering */
+#endif
+ png_row_info row_info PNG_DEPSTRUCT; /* used for transformation routines */
+
+ png_uint_32 idat_size PNG_DEPSTRUCT; /* current IDAT size for read */
+ png_uint_32 crc PNG_DEPSTRUCT; /* current chunk CRC value */
+ png_colorp palette PNG_DEPSTRUCT; /* palette from the input file */
+ png_uint_16 num_palette PNG_DEPSTRUCT; /* number of color entries in palette */
+ png_uint_16 num_trans PNG_DEPSTRUCT; /* number of transparency values */
+ png_byte chunk_name[5] PNG_DEPSTRUCT; /* null-terminated name of current chunk */
+ png_byte compression PNG_DEPSTRUCT; /* file compression type (always 0) */
+ png_byte filter PNG_DEPSTRUCT; /* file filter type (always 0) */
+ png_byte interlaced PNG_DEPSTRUCT; /* PNG_INTERLACE_NONE, PNG_INTERLACE_ADAM7 */
+ png_byte pass PNG_DEPSTRUCT; /* current interlace pass (0 - 6) */
+ png_byte do_filter PNG_DEPSTRUCT; /* row filter flags (see PNG_FILTER_ below ) */
+ png_byte color_type PNG_DEPSTRUCT; /* color type of file */
+ png_byte bit_depth PNG_DEPSTRUCT; /* bit depth of file */
+ png_byte usr_bit_depth PNG_DEPSTRUCT; /* bit depth of users row */
+ png_byte pixel_depth PNG_DEPSTRUCT; /* number of bits per pixel */
+ png_byte channels PNG_DEPSTRUCT; /* number of channels in file */
+ png_byte usr_channels PNG_DEPSTRUCT; /* channels at start of write */
+ png_byte sig_bytes PNG_DEPSTRUCT; /* magic bytes read/written from start of file */
+
+#if defined(PNG_READ_FILLER_SUPPORTED) || defined(PNG_WRITE_FILLER_SUPPORTED)
+#ifdef PNG_LEGACY_SUPPORTED
+ png_byte filler PNG_DEPSTRUCT; /* filler byte for pixel expansion */
+#else
+ png_uint_16 filler PNG_DEPSTRUCT; /* filler bytes for pixel expansion */
+#endif
+#endif
+
+#ifdef PNG_bKGD_SUPPORTED
+ png_byte background_gamma_type PNG_DEPSTRUCT;
+# ifdef PNG_FLOATING_POINT_SUPPORTED
+ float background_gamma PNG_DEPSTRUCT;
+# endif
+ png_color_16 background PNG_DEPSTRUCT; /* background color in screen gamma space */
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ png_color_16 background_1 PNG_DEPSTRUCT; /* background normalized to gamma 1.0 */
+#endif
+#endif /* PNG_bKGD_SUPPORTED */
+
+#ifdef PNG_WRITE_FLUSH_SUPPORTED
+ png_flush_ptr output_flush_fn PNG_DEPSTRUCT; /* Function for flushing output */
+ png_uint_32 flush_dist PNG_DEPSTRUCT; /* how many rows apart to flush, 0 - no flush */
+ png_uint_32 flush_rows PNG_DEPSTRUCT; /* number of rows written since last flush */
+#endif
+
+#if defined(PNG_READ_GAMMA_SUPPORTED) || defined(PNG_READ_BACKGROUND_SUPPORTED)
+ int gamma_shift PNG_DEPSTRUCT; /* number of "insignificant" bits 16-bit gamma */
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ float gamma PNG_DEPSTRUCT; /* file gamma value */
+ float screen_gamma PNG_DEPSTRUCT; /* screen gamma value (display_exponent) */
+#endif
+#endif
+
+#if defined(PNG_READ_GAMMA_SUPPORTED) || defined(PNG_READ_BACKGROUND_SUPPORTED)
+ png_bytep gamma_table PNG_DEPSTRUCT; /* gamma table for 8-bit depth files */
+ png_bytep gamma_from_1 PNG_DEPSTRUCT; /* converts from 1.0 to screen */
+ png_bytep gamma_to_1 PNG_DEPSTRUCT; /* converts from file to 1.0 */
+ png_uint_16pp gamma_16_table PNG_DEPSTRUCT; /* gamma table for 16-bit depth files */
+ png_uint_16pp gamma_16_from_1 PNG_DEPSTRUCT; /* converts from 1.0 to screen */
+ png_uint_16pp gamma_16_to_1 PNG_DEPSTRUCT; /* converts from file to 1.0 */
+#endif
+
+#if defined(PNG_READ_GAMMA_SUPPORTED) || defined(PNG_sBIT_SUPPORTED)
+ png_color_8 sig_bit PNG_DEPSTRUCT; /* significant bits in each available channel */
+#endif
+
+#if defined(PNG_READ_SHIFT_SUPPORTED) || defined(PNG_WRITE_SHIFT_SUPPORTED)
+ png_color_8 shift PNG_DEPSTRUCT; /* shift for significant bit tranformation */
+#endif
+
+#if defined(PNG_tRNS_SUPPORTED) || defined(PNG_READ_BACKGROUND_SUPPORTED) \
+ || defined(PNG_READ_EXPAND_SUPPORTED) || defined(PNG_READ_BACKGROUND_SUPPORTED)
+ png_bytep trans PNG_DEPSTRUCT; /* transparency values for paletted files */
+ png_color_16 trans_values PNG_DEPSTRUCT; /* transparency values for non-paletted files */
+#endif
+
+ png_read_status_ptr read_row_fn PNG_DEPSTRUCT; /* called after each row is decoded */
+ png_write_status_ptr write_row_fn PNG_DEPSTRUCT; /* called after each row is encoded */
+#ifdef PNG_PROGRESSIVE_READ_SUPPORTED
+ png_progressive_info_ptr info_fn PNG_DEPSTRUCT; /* called after header data fully read */
+ png_progressive_row_ptr row_fn PNG_DEPSTRUCT; /* called after each prog. row is decoded */
+ png_progressive_end_ptr end_fn PNG_DEPSTRUCT; /* called after image is complete */
+ png_bytep save_buffer_ptr PNG_DEPSTRUCT; /* current location in save_buffer */
+ png_bytep save_buffer PNG_DEPSTRUCT; /* buffer for previously read data */
+ png_bytep current_buffer_ptr PNG_DEPSTRUCT; /* current location in current_buffer */
+ png_bytep current_buffer PNG_DEPSTRUCT; /* buffer for recently used data */
+ png_uint_32 push_length PNG_DEPSTRUCT; /* size of current input chunk */
+ png_uint_32 skip_length PNG_DEPSTRUCT; /* bytes to skip in input data */
+ png_size_t save_buffer_size PNG_DEPSTRUCT; /* amount of data now in save_buffer */
+ png_size_t save_buffer_max PNG_DEPSTRUCT; /* total size of save_buffer */
+ png_size_t buffer_size PNG_DEPSTRUCT; /* total amount of available input data */
+ png_size_t current_buffer_size PNG_DEPSTRUCT; /* amount of data now in current_buffer */
+ int process_mode PNG_DEPSTRUCT; /* what push library is currently doing */
+ int cur_palette PNG_DEPSTRUCT; /* current push library palette index */
+
+# ifdef PNG_TEXT_SUPPORTED
+ png_size_t current_text_size PNG_DEPSTRUCT; /* current size of text input data */
+ png_size_t current_text_left PNG_DEPSTRUCT; /* how much text left to read in input */
+ png_charp current_text PNG_DEPSTRUCT; /* current text chunk buffer */
+ png_charp current_text_ptr PNG_DEPSTRUCT; /* current location in current_text */
+# endif /* PNG_TEXT_SUPPORTED */
+#endif /* PNG_PROGRESSIVE_READ_SUPPORTED */
+
+#if defined(__TURBOC__) && !defined(_Windows) && !defined(__FLAT__)
+/* for the Borland special 64K segment handler */
+ png_bytepp offset_table_ptr PNG_DEPSTRUCT;
+ png_bytep offset_table PNG_DEPSTRUCT;
+ png_uint_16 offset_table_number PNG_DEPSTRUCT;
+ png_uint_16 offset_table_count PNG_DEPSTRUCT;
+ png_uint_16 offset_table_count_free PNG_DEPSTRUCT;
+#endif
+
+#ifdef PNG_READ_DITHER_SUPPORTED
+ png_bytep palette_lookup PNG_DEPSTRUCT; /* lookup table for dithering */
+ png_bytep dither_index PNG_DEPSTRUCT; /* index translation for palette files */
+#endif
+
+#if defined(PNG_READ_DITHER_SUPPORTED) || defined(PNG_hIST_SUPPORTED)
+ png_uint_16p hist PNG_DEPSTRUCT; /* histogram */
+#endif
+
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ png_byte heuristic_method PNG_DEPSTRUCT; /* heuristic for row filter selection */
+ png_byte num_prev_filters PNG_DEPSTRUCT; /* number of weights for previous rows */
+ png_bytep prev_filters PNG_DEPSTRUCT; /* filter type(s) of previous row(s) */
+ png_uint_16p filter_weights PNG_DEPSTRUCT; /* weight(s) for previous line(s) */
+ png_uint_16p inv_filter_weights PNG_DEPSTRUCT; /* 1/weight(s) for previous line(s) */
+ png_uint_16p filter_costs PNG_DEPSTRUCT; /* relative filter calculation cost */
+ png_uint_16p inv_filter_costs PNG_DEPSTRUCT; /* 1/relative filter calculation cost */
+#endif
+
+#ifdef PNG_TIME_RFC1123_SUPPORTED
+ png_charp time_buffer PNG_DEPSTRUCT; /* String to hold RFC 1123 time text */
+#endif
+
+/* New members added in libpng-1.0.6 */
+
+#ifdef PNG_FREE_ME_SUPPORTED
+ png_uint_32 free_me PNG_DEPSTRUCT; /* flags items libpng is responsible for freeing */
+#endif
+
+#ifdef PNG_USER_CHUNKS_SUPPORTED
+ png_voidp user_chunk_ptr PNG_DEPSTRUCT;
+ png_user_chunk_ptr read_user_chunk_fn PNG_DEPSTRUCT; /* user read chunk handler */
+#endif
+
+#ifdef PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+ int num_chunk_list PNG_DEPSTRUCT;
+ png_bytep chunk_list PNG_DEPSTRUCT;
+#endif
+
+/* New members added in libpng-1.0.3 */
+#ifdef PNG_READ_RGB_TO_GRAY_SUPPORTED
+ png_byte rgb_to_gray_status PNG_DEPSTRUCT;
+ /* These were changed from png_byte in libpng-1.0.6 */
+ png_uint_16 rgb_to_gray_red_coeff PNG_DEPSTRUCT;
+ png_uint_16 rgb_to_gray_green_coeff PNG_DEPSTRUCT;
+ png_uint_16 rgb_to_gray_blue_coeff PNG_DEPSTRUCT;
+#endif
+
+/* New member added in libpng-1.0.4 (renamed in 1.0.9) */
+#if defined(PNG_MNG_FEATURES_SUPPORTED) || \
+ defined(PNG_READ_EMPTY_PLTE_SUPPORTED) || \
+ defined(PNG_WRITE_EMPTY_PLTE_SUPPORTED)
+/* Changed from png_byte to png_uint_32 at version 1.2.0 */
+#ifdef PNG_1_0_X
+ png_byte mng_features_permitted PNG_DEPSTRUCT;
+#else
+ png_uint_32 mng_features_permitted PNG_DEPSTRUCT;
+#endif /* PNG_1_0_X */
+#endif
+
+/* New member added in libpng-1.0.7 */
+#if defined(PNG_READ_GAMMA_SUPPORTED) || defined(PNG_READ_BACKGROUND_SUPPORTED)
+ png_fixed_point int_gamma PNG_DEPSTRUCT;
+#endif
+
+/* New member added in libpng-1.0.9, ifdef'ed out in 1.0.12, enabled in 1.2.0 */
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+ png_byte filter_type PNG_DEPSTRUCT;
+#endif
+
+#ifdef PNG_1_0_X
+/* New member added in libpng-1.0.10, ifdef'ed out in 1.2.0 */
+ png_uint_32 row_buf_size PNG_DEPSTRUCT;
+#endif
+
+/* New members added in libpng-1.2.0 */
+#ifdef PNG_ASSEMBLER_CODE_SUPPORTED
+# ifndef PNG_1_0_X
+# ifdef PNG_MMX_CODE_SUPPORTED
+ png_byte mmx_bitdepth_threshold PNG_DEPSTRUCT;
+ png_uint_32 mmx_rowbytes_threshold PNG_DEPSTRUCT;
+# endif
+ png_uint_32 asm_flags PNG_DEPSTRUCT;
+# endif
+#endif
+
+/* New members added in libpng-1.0.2 but first enabled by default in 1.2.0 */
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_voidp mem_ptr PNG_DEPSTRUCT; /* user supplied struct for mem functions */
+ png_malloc_ptr malloc_fn PNG_DEPSTRUCT; /* function for allocating memory */
+ png_free_ptr free_fn PNG_DEPSTRUCT; /* function for freeing memory */
+#endif
+
+/* New member added in libpng-1.0.13 and 1.2.0 */
+ png_bytep big_row_buf PNG_DEPSTRUCT; /* buffer to save current (unfiltered) row */
+
+#ifdef PNG_READ_DITHER_SUPPORTED
+/* The following three members were added at version 1.0.14 and 1.2.4 */
+ png_bytep dither_sort PNG_DEPSTRUCT; /* working sort array */
+ png_bytep index_to_palette PNG_DEPSTRUCT; /* where the original index currently is */
+ /* in the palette */
+ png_bytep palette_to_index PNG_DEPSTRUCT; /* which original index points to this */
+ /* palette color */
+#endif
+
+/* New members added in libpng-1.0.16 and 1.2.6 */
+ png_byte compression_type PNG_DEPSTRUCT;
+
+#ifdef PNG_USER_LIMITS_SUPPORTED
+ png_uint_32 user_width_max PNG_DEPSTRUCT;
+ png_uint_32 user_height_max PNG_DEPSTRUCT;
+#endif
+
+/* New member added in libpng-1.0.25 and 1.2.17 */
+#ifdef PNG_UNKNOWN_CHUNKS_SUPPORTED
+ /* Storage for unknown chunk that the library doesn't recognize. */
+ png_unknown_chunk unknown_chunk PNG_DEPSTRUCT;
+#endif
+
+/* New members added in libpng-1.2.26 */
+ png_uint_32 old_big_row_buf_size PNG_DEPSTRUCT;
+ png_uint_32 old_prev_row_size PNG_DEPSTRUCT;
+
+/* New member added in libpng-1.2.30 */
+ png_charp chunkdata PNG_DEPSTRUCT; /* buffer for reading chunk data */
+
+
+};
+
+
+/* This triggers a compiler error in png.c, if png.c and png.h
+ * do not agree upon the version number.
+ */
+typedef png_structp version_1_2_44;
+
+typedef png_struct FAR * FAR * png_structpp;
+
+/* Here are the function definitions most commonly used. This is not
+ * the place to find out how to use libpng. See libpng.txt for the
+ * full explanation, see example.c for the summary. This just provides
+ * a simple one line description of the use of each function.
+ */
+
+/* Returns the version number of the library */
+extern PNG_EXPORT(png_uint_32,png_access_version_number) PNGARG((void));
+
+/* Tell lib we have already handled the first <num_bytes> magic bytes.
+ * Handling more than 8 bytes from the beginning of the file is an error.
+ */
+extern PNG_EXPORT(void,png_set_sig_bytes) PNGARG((png_structp png_ptr,
+ int num_bytes));
+
+/* Check sig[start] through sig[start + num_to_check - 1] to see if it's a
+ * PNG file. Returns zero if the supplied bytes match the 8-byte PNG
+ * signature, and non-zero otherwise. Having num_to_check == 0 or
+ * start > 7 will always fail (ie return non-zero).
+ */
+extern PNG_EXPORT(int,png_sig_cmp) PNGARG((png_bytep sig, png_size_t start,
+ png_size_t num_to_check));
+
+/* Simple signature checking function. This is the same as calling
+ * png_check_sig(sig, n) := !png_sig_cmp(sig, 0, n).
+ */
+extern PNG_EXPORT(int,png_check_sig) PNGARG((png_bytep sig, int num)) PNG_DEPRECATED;
+
+/* Allocate and initialize png_ptr struct for reading, and any other memory. */
+extern PNG_EXPORT(png_structp,png_create_read_struct)
+ PNGARG((png_const_charp user_png_ver, png_voidp error_ptr,
+ png_error_ptr error_fn, png_error_ptr warn_fn)) PNG_ALLOCATED;
+
+/* Allocate and initialize png_ptr struct for writing, and any other memory */
+extern PNG_EXPORT(png_structp,png_create_write_struct)
+ PNGARG((png_const_charp user_png_ver, png_voidp error_ptr,
+ png_error_ptr error_fn, png_error_ptr warn_fn)) PNG_ALLOCATED;
+
+#ifdef PNG_WRITE_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_compression_buffer_size)
+ PNGARG((png_structp png_ptr));
+#endif
+
+#ifdef PNG_WRITE_SUPPORTED
+extern PNG_EXPORT(void,png_set_compression_buffer_size)
+ PNGARG((png_structp png_ptr, png_uint_32 size));
+#endif
+
+/* Reset the compression stream */
+extern PNG_EXPORT(int,png_reset_zstream) PNGARG((png_structp png_ptr));
+
+/* New functions added in libpng-1.0.2 (not enabled by default until 1.2.0) */
+#ifdef PNG_USER_MEM_SUPPORTED
+extern PNG_EXPORT(png_structp,png_create_read_struct_2)
+ PNGARG((png_const_charp user_png_ver, png_voidp error_ptr,
+ png_error_ptr error_fn, png_error_ptr warn_fn, png_voidp mem_ptr,
+ png_malloc_ptr malloc_fn, png_free_ptr free_fn)) PNG_ALLOCATED;
+extern PNG_EXPORT(png_structp,png_create_write_struct_2)
+ PNGARG((png_const_charp user_png_ver, png_voidp error_ptr,
+ png_error_ptr error_fn, png_error_ptr warn_fn, png_voidp mem_ptr,
+ png_malloc_ptr malloc_fn, png_free_ptr free_fn)) PNG_ALLOCATED;
+#endif
+
+/* Write a PNG chunk - size, type, (optional) data, CRC. */
+extern PNG_EXPORT(void,png_write_chunk) PNGARG((png_structp png_ptr,
+ png_bytep chunk_name, png_bytep data, png_size_t length));
+
+/* Write the start of a PNG chunk - length and chunk name. */
+extern PNG_EXPORT(void,png_write_chunk_start) PNGARG((png_structp png_ptr,
+ png_bytep chunk_name, png_uint_32 length));
+
+/* Write the data of a PNG chunk started with png_write_chunk_start(). */
+extern PNG_EXPORT(void,png_write_chunk_data) PNGARG((png_structp png_ptr,
+ png_bytep data, png_size_t length));
+
+/* Finish a chunk started with png_write_chunk_start() (includes CRC). */
+extern PNG_EXPORT(void,png_write_chunk_end) PNGARG((png_structp png_ptr));
+
+/* Allocate and initialize the info structure */
+extern PNG_EXPORT(png_infop,png_create_info_struct)
+ PNGARG((png_structp png_ptr)) PNG_ALLOCATED;
+
+#if defined(PNG_1_0_X) || defined (PNG_1_2_X)
+/* Initialize the info structure (old interface - DEPRECATED) */
+extern PNG_EXPORT(void,png_info_init) PNGARG((png_infop info_ptr))
+ PNG_DEPRECATED;
+#undef png_info_init
+#define png_info_init(info_ptr) png_info_init_3(&info_ptr,\
+ png_sizeof(png_info));
+#endif
+
+extern PNG_EXPORT(void,png_info_init_3) PNGARG((png_infopp info_ptr,
+ png_size_t png_info_struct_size));
+
+/* Writes all the PNG information before the image. */
+extern PNG_EXPORT(void,png_write_info_before_PLTE) PNGARG((png_structp png_ptr,
+ png_infop info_ptr));
+extern PNG_EXPORT(void,png_write_info) PNGARG((png_structp png_ptr,
+ png_infop info_ptr));
+
+#ifdef PNG_SEQUENTIAL_READ_SUPPORTED
+/* Read the information before the actual image data. */
+extern PNG_EXPORT(void,png_read_info) PNGARG((png_structp png_ptr,
+ png_infop info_ptr));
+#endif
+
+#ifdef PNG_TIME_RFC1123_SUPPORTED
+extern PNG_EXPORT(png_charp,png_convert_to_rfc1123)
+ PNGARG((png_structp png_ptr, png_timep ptime));
+#endif
+
+#ifdef PNG_CONVERT_tIME_SUPPORTED
+/* Convert from a struct tm to png_time */
+extern PNG_EXPORT(void,png_convert_from_struct_tm) PNGARG((png_timep ptime,
+ struct tm FAR * ttime));
+
+/* Convert from time_t to png_time. Uses gmtime() */
+extern PNG_EXPORT(void,png_convert_from_time_t) PNGARG((png_timep ptime,
+ time_t ttime));
+#endif /* PNG_CONVERT_tIME_SUPPORTED */
+
+#ifdef PNG_READ_EXPAND_SUPPORTED
+/* Expand data to 24-bit RGB, or 8-bit grayscale, with alpha if available. */
+extern PNG_EXPORT(void,png_set_expand) PNGARG((png_structp png_ptr));
+#ifndef PNG_1_0_X
+extern PNG_EXPORT(void,png_set_expand_gray_1_2_4_to_8) PNGARG((png_structp
+ png_ptr));
+#endif
+extern PNG_EXPORT(void,png_set_palette_to_rgb) PNGARG((png_structp png_ptr));
+extern PNG_EXPORT(void,png_set_tRNS_to_alpha) PNGARG((png_structp png_ptr));
+#if defined(PNG_1_0_X) || defined (PNG_1_2_X)
+/* Deprecated */
+extern PNG_EXPORT(void,png_set_gray_1_2_4_to_8) PNGARG((png_structp
+ png_ptr)) PNG_DEPRECATED;
+#endif
+#endif
+
+#if defined(PNG_READ_BGR_SUPPORTED) || defined(PNG_WRITE_BGR_SUPPORTED)
+/* Use blue, green, red order for pixels. */
+extern PNG_EXPORT(void,png_set_bgr) PNGARG((png_structp png_ptr));
+#endif
+
+#ifdef PNG_READ_GRAY_TO_RGB_SUPPORTED
+/* Expand the grayscale to 24-bit RGB if necessary. */
+extern PNG_EXPORT(void,png_set_gray_to_rgb) PNGARG((png_structp png_ptr));
+#endif
+
+#ifdef PNG_READ_RGB_TO_GRAY_SUPPORTED
+/* Reduce RGB to grayscale. */
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+extern PNG_EXPORT(void,png_set_rgb_to_gray) PNGARG((png_structp png_ptr,
+ int error_action, double red, double green ));
+#endif
+extern PNG_EXPORT(void,png_set_rgb_to_gray_fixed) PNGARG((png_structp png_ptr,
+ int error_action, png_fixed_point red, png_fixed_point green ));
+extern PNG_EXPORT(png_byte,png_get_rgb_to_gray_status) PNGARG((png_structp
+ png_ptr));
+#endif
+
+extern PNG_EXPORT(void,png_build_grayscale_palette) PNGARG((int bit_depth,
+ png_colorp palette));
+
+#ifdef PNG_READ_STRIP_ALPHA_SUPPORTED
+extern PNG_EXPORT(void,png_set_strip_alpha) PNGARG((png_structp png_ptr));
+#endif
+
+#if defined(PNG_READ_SWAP_ALPHA_SUPPORTED) || \
+ defined(PNG_WRITE_SWAP_ALPHA_SUPPORTED)
+extern PNG_EXPORT(void,png_set_swap_alpha) PNGARG((png_structp png_ptr));
+#endif
+
+#if defined(PNG_READ_INVERT_ALPHA_SUPPORTED) || \
+ defined(PNG_WRITE_INVERT_ALPHA_SUPPORTED)
+extern PNG_EXPORT(void,png_set_invert_alpha) PNGARG((png_structp png_ptr));
+#endif
+
+#if defined(PNG_READ_FILLER_SUPPORTED) || defined(PNG_WRITE_FILLER_SUPPORTED)
+/* Add a filler byte to 8-bit Gray or 24-bit RGB images. */
+extern PNG_EXPORT(void,png_set_filler) PNGARG((png_structp png_ptr,
+ png_uint_32 filler, int flags));
+/* The values of the PNG_FILLER_ defines should NOT be changed */
+#define PNG_FILLER_BEFORE 0
+#define PNG_FILLER_AFTER 1
+/* Add an alpha byte to 8-bit Gray or 24-bit RGB images. */
+#ifndef PNG_1_0_X
+extern PNG_EXPORT(void,png_set_add_alpha) PNGARG((png_structp png_ptr,
+ png_uint_32 filler, int flags));
+#endif
+#endif /* PNG_READ_FILLER_SUPPORTED || PNG_WRITE_FILLER_SUPPORTED */
+
+#if defined(PNG_READ_SWAP_SUPPORTED) || defined(PNG_WRITE_SWAP_SUPPORTED)
+/* Swap bytes in 16-bit depth files. */
+extern PNG_EXPORT(void,png_set_swap) PNGARG((png_structp png_ptr));
+#endif
+
+#if defined(PNG_READ_PACK_SUPPORTED) || defined(PNG_WRITE_PACK_SUPPORTED)
+/* Use 1 byte per pixel in 1, 2, or 4-bit depth files. */
+extern PNG_EXPORT(void,png_set_packing) PNGARG((png_structp png_ptr));
+#endif
+
+#if defined(PNG_READ_PACKSWAP_SUPPORTED) || defined(PNG_WRITE_PACKSWAP_SUPPORTED)
+/* Swap packing order of pixels in bytes. */
+extern PNG_EXPORT(void,png_set_packswap) PNGARG((png_structp png_ptr));
+#endif
+
+#if defined(PNG_READ_SHIFT_SUPPORTED) || defined(PNG_WRITE_SHIFT_SUPPORTED)
+/* Converts files to legal bit depths. */
+extern PNG_EXPORT(void,png_set_shift) PNGARG((png_structp png_ptr,
+ png_color_8p true_bits));
+#endif
+
+#if defined(PNG_READ_INTERLACING_SUPPORTED) || \
+ defined(PNG_WRITE_INTERLACING_SUPPORTED)
+/* Have the code handle the interlacing. Returns the number of passes. */
+extern PNG_EXPORT(int,png_set_interlace_handling) PNGARG((png_structp png_ptr));
+#endif
+
+#if defined(PNG_READ_INVERT_SUPPORTED) || defined(PNG_WRITE_INVERT_SUPPORTED)
+/* Invert monochrome files */
+extern PNG_EXPORT(void,png_set_invert_mono) PNGARG((png_structp png_ptr));
+#endif
+
+#ifdef PNG_READ_BACKGROUND_SUPPORTED
+/* Handle alpha and tRNS by replacing with a background color. */
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+extern PNG_EXPORT(void,png_set_background) PNGARG((png_structp png_ptr,
+ png_color_16p background_color, int background_gamma_code,
+ int need_expand, double background_gamma));
+#endif
+#define PNG_BACKGROUND_GAMMA_UNKNOWN 0
+#define PNG_BACKGROUND_GAMMA_SCREEN 1
+#define PNG_BACKGROUND_GAMMA_FILE 2
+#define PNG_BACKGROUND_GAMMA_UNIQUE 3
+#endif
+
+#ifdef PNG_READ_16_TO_8_SUPPORTED
+/* Strip the second byte of information from a 16-bit depth file. */
+extern PNG_EXPORT(void,png_set_strip_16) PNGARG((png_structp png_ptr));
+#endif
+
+#ifdef PNG_READ_DITHER_SUPPORTED
+/* Turn on dithering, and reduce the palette to the number of colors available. */
+extern PNG_EXPORT(void,png_set_dither) PNGARG((png_structp png_ptr,
+ png_colorp palette, int num_palette, int maximum_colors,
+ png_uint_16p histogram, int full_dither));
+#endif
+
+#ifdef PNG_READ_GAMMA_SUPPORTED
+/* Handle gamma correction. Screen_gamma=(display_exponent) */
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+extern PNG_EXPORT(void,png_set_gamma) PNGARG((png_structp png_ptr,
+ double screen_gamma, double default_file_gamma));
+#endif
+#endif
+
+#if defined(PNG_1_0_X) || defined (PNG_1_2_X)
+#if defined(PNG_READ_EMPTY_PLTE_SUPPORTED) || \
+ defined(PNG_WRITE_EMPTY_PLTE_SUPPORTED)
+/* Permit or disallow empty PLTE (0: not permitted, 1: permitted) */
+/* Deprecated and will be removed. Use png_permit_mng_features() instead. */
+extern PNG_EXPORT(void,png_permit_empty_plte) PNGARG((png_structp png_ptr,
+ int empty_plte_permitted)) PNG_DEPRECATED;
+#endif
+#endif
+
+#ifdef PNG_WRITE_FLUSH_SUPPORTED
+/* Set how many lines between output flushes - 0 for no flushing */
+extern PNG_EXPORT(void,png_set_flush) PNGARG((png_structp png_ptr, int nrows));
+/* Flush the current PNG output buffer */
+extern PNG_EXPORT(void,png_write_flush) PNGARG((png_structp png_ptr));
+#endif
+
+/* Optional update palette with requested transformations */
+extern PNG_EXPORT(void,png_start_read_image) PNGARG((png_structp png_ptr));
+
+/* Optional call to update the users info structure */
+extern PNG_EXPORT(void,png_read_update_info) PNGARG((png_structp png_ptr,
+ png_infop info_ptr));
+
+#ifndef PNG_NO_SEQUENTIAL_READ_SUPPORTED
+/* Read one or more rows of image data. */
+extern PNG_EXPORT(void,png_read_rows) PNGARG((png_structp png_ptr,
+ png_bytepp row, png_bytepp display_row, png_uint_32 num_rows));
+#endif
+
+#ifndef PNG_NO_SEQUENTIAL_READ_SUPPORTED
+/* Read a row of data. */
+extern PNG_EXPORT(void,png_read_row) PNGARG((png_structp png_ptr,
+ png_bytep row,
+ png_bytep display_row));
+#endif
+
+#ifndef PNG_NO_SEQUENTIAL_READ_SUPPORTED
+/* Read the whole image into memory at once. */
+extern PNG_EXPORT(void,png_read_image) PNGARG((png_structp png_ptr,
+ png_bytepp image));
+#endif
+
+/* Write a row of image data */
+extern PNG_EXPORT(void,png_write_row) PNGARG((png_structp png_ptr,
+ png_bytep row));
+
+/* Write a few rows of image data */
+extern PNG_EXPORT(void,png_write_rows) PNGARG((png_structp png_ptr,
+ png_bytepp row, png_uint_32 num_rows));
+
+/* Write the image data */
+extern PNG_EXPORT(void,png_write_image) PNGARG((png_structp png_ptr,
+ png_bytepp image));
+
+/* Writes the end of the PNG file. */
+extern PNG_EXPORT(void,png_write_end) PNGARG((png_structp png_ptr,
+ png_infop info_ptr));
+
+#ifndef PNG_NO_SEQUENTIAL_READ_SUPPORTED
+/* Read the end of the PNG file. */
+extern PNG_EXPORT(void,png_read_end) PNGARG((png_structp png_ptr,
+ png_infop info_ptr));
+#endif
+
+/* Free any memory associated with the png_info_struct */
+extern PNG_EXPORT(void,png_destroy_info_struct) PNGARG((png_structp png_ptr,
+ png_infopp info_ptr_ptr));
+
+/* Free any memory associated with the png_struct and the png_info_structs */
+extern PNG_EXPORT(void,png_destroy_read_struct) PNGARG((png_structpp
+ png_ptr_ptr, png_infopp info_ptr_ptr, png_infopp end_info_ptr_ptr));
+
+/* Free all memory used by the read (old method - NOT DLL EXPORTED) */
+extern void png_read_destroy PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_infop end_info_ptr)) PNG_DEPRECATED;
+
+/* Free any memory associated with the png_struct and the png_info_structs */
+extern PNG_EXPORT(void,png_destroy_write_struct)
+ PNGARG((png_structpp png_ptr_ptr, png_infopp info_ptr_ptr));
+
+/* Free any memory used in png_ptr struct (old method - NOT DLL EXPORTED) */
+extern void png_write_destroy PNGARG((png_structp png_ptr)) PNG_DEPRECATED;
+
+/* Set the libpng method of handling chunk CRC errors */
+extern PNG_EXPORT(void,png_set_crc_action) PNGARG((png_structp png_ptr,
+ int crit_action, int ancil_action));
+
+/* Values for png_set_crc_action() to say how to handle CRC errors in
+ * ancillary and critical chunks, and whether to use the data contained
+ * therein. Note that it is impossible to "discard" data in a critical
+ * chunk. For versions prior to 0.90, the action was always error/quit,
+ * whereas in version 0.90 and later, the action for CRC errors in ancillary
+ * chunks is warn/discard. These values should NOT be changed.
+ *
+ * value action:critical action:ancillary
+ */
+#define PNG_CRC_DEFAULT 0 /* error/quit warn/discard data */
+#define PNG_CRC_ERROR_QUIT 1 /* error/quit error/quit */
+#define PNG_CRC_WARN_DISCARD 2 /* (INVALID) warn/discard data */
+#define PNG_CRC_WARN_USE 3 /* warn/use data warn/use data */
+#define PNG_CRC_QUIET_USE 4 /* quiet/use data quiet/use data */
+#define PNG_CRC_NO_CHANGE 5 /* use current value use current value */
+
+/* These functions give the user control over the scan-line filtering in
+ * libpng and the compression methods used by zlib. These functions are
+ * mainly useful for testing, as the defaults should work with most users.
+ * Those users who are tight on memory or want faster performance at the
+ * expense of compression can modify them. See the compression library
+ * header file (zlib.h) for an explination of the compression functions.
+ */
+
+/* Set the filtering method(s) used by libpng. Currently, the only valid
+ * value for "method" is 0.
+ */
+extern PNG_EXPORT(void,png_set_filter) PNGARG((png_structp png_ptr, int method,
+ int filters));
+
+/* Flags for png_set_filter() to say which filters to use. The flags
+ * are chosen so that they don't conflict with real filter types
+ * below, in case they are supplied instead of the #defined constants.
+ * These values should NOT be changed.
+ */
+#define PNG_NO_FILTERS 0x00
+#define PNG_FILTER_NONE 0x08
+#define PNG_FILTER_SUB 0x10
+#define PNG_FILTER_UP 0x20
+#define PNG_FILTER_AVG 0x40
+#define PNG_FILTER_PAETH 0x80
+#define PNG_ALL_FILTERS (PNG_FILTER_NONE | PNG_FILTER_SUB | PNG_FILTER_UP | \
+ PNG_FILTER_AVG | PNG_FILTER_PAETH)
+
+/* Filter values (not flags) - used in pngwrite.c, pngwutil.c for now.
+ * These defines should NOT be changed.
+ */
+#define PNG_FILTER_VALUE_NONE 0
+#define PNG_FILTER_VALUE_SUB 1
+#define PNG_FILTER_VALUE_UP 2
+#define PNG_FILTER_VALUE_AVG 3
+#define PNG_FILTER_VALUE_PAETH 4
+#define PNG_FILTER_VALUE_LAST 5
+
+#if defined(PNG_WRITE_WEIGHTED_FILTER_SUPPORTED) /* EXPERIMENTAL */
+/* The "heuristic_method" is given by one of the PNG_FILTER_HEURISTIC_
+ * defines, either the default (minimum-sum-of-absolute-differences), or
+ * the experimental method (weighted-minimum-sum-of-absolute-differences).
+ *
+ * Weights are factors >= 1.0, indicating how important it is to keep the
+ * filter type consistent between rows. Larger numbers mean the current
+ * filter is that many times as likely to be the same as the "num_weights"
+ * previous filters. This is cumulative for each previous row with a weight.
+ * There needs to be "num_weights" values in "filter_weights", or it can be
+ * NULL if the weights aren't being specified. Weights have no influence on
+ * the selection of the first row filter. Well chosen weights can (in theory)
+ * improve the compression for a given image.
+ *
+ * Costs are factors >= 1.0 indicating the relative decoding costs of a
+ * filter type. Higher costs indicate more decoding expense, and are
+ * therefore less likely to be selected over a filter with lower computational
+ * costs. There needs to be a value in "filter_costs" for each valid filter
+ * type (given by PNG_FILTER_VALUE_LAST), or it can be NULL if you aren't
+ * setting the costs. Costs try to improve the speed of decompression without
+ * unduly increasing the compressed image size.
+ *
+ * A negative weight or cost indicates the default value is to be used, and
+ * values in the range [0.0, 1.0) indicate the value is to remain unchanged.
+ * The default values for both weights and costs are currently 1.0, but may
+ * change if good general weighting/cost heuristics can be found. If both
+ * the weights and costs are set to 1.0, this degenerates the WEIGHTED method
+ * to the UNWEIGHTED method, but with added encoding time/computation.
+ */
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+extern PNG_EXPORT(void,png_set_filter_heuristics) PNGARG((png_structp png_ptr,
+ int heuristic_method, int num_weights, png_doublep filter_weights,
+ png_doublep filter_costs));
+#endif
+#endif /* PNG_WRITE_WEIGHTED_FILTER_SUPPORTED */
+
+/* Heuristic used for row filter selection. These defines should NOT be
+ * changed.
+ */
+#define PNG_FILTER_HEURISTIC_DEFAULT 0 /* Currently "UNWEIGHTED" */
+#define PNG_FILTER_HEURISTIC_UNWEIGHTED 1 /* Used by libpng < 0.95 */
+#define PNG_FILTER_HEURISTIC_WEIGHTED 2 /* Experimental feature */
+#define PNG_FILTER_HEURISTIC_LAST 3 /* Not a valid value */
+
+/* Set the library compression level. Currently, valid values range from
+ * 0 - 9, corresponding directly to the zlib compression levels 0 - 9
+ * (0 - no compression, 9 - "maximal" compression). Note that tests have
+ * shown that zlib compression levels 3-6 usually perform as well as level 9
+ * for PNG images, and do considerably fewer caclulations. In the future,
+ * these values may not correspond directly to the zlib compression levels.
+ */
+extern PNG_EXPORT(void,png_set_compression_level) PNGARG((png_structp png_ptr,
+ int level));
+
+extern PNG_EXPORT(void,png_set_compression_mem_level)
+ PNGARG((png_structp png_ptr, int mem_level));
+
+extern PNG_EXPORT(void,png_set_compression_strategy)
+ PNGARG((png_structp png_ptr, int strategy));
+
+extern PNG_EXPORT(void,png_set_compression_window_bits)
+ PNGARG((png_structp png_ptr, int window_bits));
+
+extern PNG_EXPORT(void,png_set_compression_method) PNGARG((png_structp png_ptr,
+ int method));
+
+/* These next functions are called for input/output, memory, and error
+ * handling. They are in the file pngrio.c, pngwio.c, and pngerror.c,
+ * and call standard C I/O routines such as fread(), fwrite(), and
+ * fprintf(). These functions can be made to use other I/O routines
+ * at run time for those applications that need to handle I/O in a
+ * different manner by calling png_set_???_fn(). See libpng.txt for
+ * more information.
+ */
+
+#ifdef PNG_STDIO_SUPPORTED
+/* Initialize the input/output for the PNG file to the default functions. */
+extern PNG_EXPORT(void,png_init_io) PNGARG((png_structp png_ptr, png_FILE_p fp));
+#endif
+
+/* Replace the (error and abort), and warning functions with user
+ * supplied functions. If no messages are to be printed you must still
+ * write and use replacement functions. The replacement error_fn should
+ * still do a longjmp to the last setjmp location if you are using this
+ * method of error handling. If error_fn or warning_fn is NULL, the
+ * default function will be used.
+ */
+
+extern PNG_EXPORT(void,png_set_error_fn) PNGARG((png_structp png_ptr,
+ png_voidp error_ptr, png_error_ptr error_fn, png_error_ptr warning_fn));
+
+/* Return the user pointer associated with the error functions */
+extern PNG_EXPORT(png_voidp,png_get_error_ptr) PNGARG((png_structp png_ptr));
+
+/* Replace the default data output functions with a user supplied one(s).
+ * If buffered output is not used, then output_flush_fn can be set to NULL.
+ * If PNG_WRITE_FLUSH_SUPPORTED is not defined at libpng compile time
+ * output_flush_fn will be ignored (and thus can be NULL).
+ * It is probably a mistake to use NULL for output_flush_fn if
+ * write_data_fn is not also NULL unless you have built libpng with
+ * PNG_WRITE_FLUSH_SUPPORTED undefined, because in this case libpng's
+ * default flush function, which uses the standard *FILE structure, will
+ * be used.
+ */
+extern PNG_EXPORT(void,png_set_write_fn) PNGARG((png_structp png_ptr,
+ png_voidp io_ptr, png_rw_ptr write_data_fn, png_flush_ptr output_flush_fn));
+
+/* Replace the default data input function with a user supplied one. */
+extern PNG_EXPORT(void,png_set_read_fn) PNGARG((png_structp png_ptr,
+ png_voidp io_ptr, png_rw_ptr read_data_fn));
+
+/* Return the user pointer associated with the I/O functions */
+extern PNG_EXPORT(png_voidp,png_get_io_ptr) PNGARG((png_structp png_ptr));
+
+extern PNG_EXPORT(void,png_set_read_status_fn) PNGARG((png_structp png_ptr,
+ png_read_status_ptr read_row_fn));
+
+extern PNG_EXPORT(void,png_set_write_status_fn) PNGARG((png_structp png_ptr,
+ png_write_status_ptr write_row_fn));
+
+#ifdef PNG_USER_MEM_SUPPORTED
+/* Replace the default memory allocation functions with user supplied one(s). */
+extern PNG_EXPORT(void,png_set_mem_fn) PNGARG((png_structp png_ptr,
+ png_voidp mem_ptr, png_malloc_ptr malloc_fn, png_free_ptr free_fn));
+/* Return the user pointer associated with the memory functions */
+extern PNG_EXPORT(png_voidp,png_get_mem_ptr) PNGARG((png_structp png_ptr));
+#endif
+
+#if defined(PNG_READ_USER_TRANSFORM_SUPPORTED) || \
+ defined(PNG_LEGACY_SUPPORTED)
+extern PNG_EXPORT(void,png_set_read_user_transform_fn) PNGARG((png_structp
+ png_ptr, png_user_transform_ptr read_user_transform_fn));
+#endif
+
+#if defined(PNG_WRITE_USER_TRANSFORM_SUPPORTED) || \
+ defined(PNG_LEGACY_SUPPORTED)
+extern PNG_EXPORT(void,png_set_write_user_transform_fn) PNGARG((png_structp
+ png_ptr, png_user_transform_ptr write_user_transform_fn));
+#endif
+
+#if defined(PNG_READ_USER_TRANSFORM_SUPPORTED) || \
+ defined(PNG_WRITE_USER_TRANSFORM_SUPPORTED) || \
+ defined(PNG_LEGACY_SUPPORTED)
+extern PNG_EXPORT(void,png_set_user_transform_info) PNGARG((png_structp
+ png_ptr, png_voidp user_transform_ptr, int user_transform_depth,
+ int user_transform_channels));
+/* Return the user pointer associated with the user transform functions */
+extern PNG_EXPORT(png_voidp,png_get_user_transform_ptr)
+ PNGARG((png_structp png_ptr));
+#endif
+
+#ifdef PNG_USER_CHUNKS_SUPPORTED
+extern PNG_EXPORT(void,png_set_read_user_chunk_fn) PNGARG((png_structp png_ptr,
+ png_voidp user_chunk_ptr, png_user_chunk_ptr read_user_chunk_fn));
+extern PNG_EXPORT(png_voidp,png_get_user_chunk_ptr) PNGARG((png_structp
+ png_ptr));
+#endif
+
+#ifdef PNG_PROGRESSIVE_READ_SUPPORTED
+/* Sets the function callbacks for the push reader, and a pointer to a
+ * user-defined structure available to the callback functions.
+ */
+extern PNG_EXPORT(void,png_set_progressive_read_fn) PNGARG((png_structp png_ptr,
+ png_voidp progressive_ptr,
+ png_progressive_info_ptr info_fn, png_progressive_row_ptr row_fn,
+ png_progressive_end_ptr end_fn));
+
+/* Returns the user pointer associated with the push read functions */
+extern PNG_EXPORT(png_voidp,png_get_progressive_ptr)
+ PNGARG((png_structp png_ptr));
+
+/* Function to be called when data becomes available */
+extern PNG_EXPORT(void,png_process_data) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_bytep buffer, png_size_t buffer_size));
+
+/* Function that combines rows. Not very much different than the
+ * png_combine_row() call. Is this even used?????
+ */
+extern PNG_EXPORT(void,png_progressive_combine_row) PNGARG((png_structp png_ptr,
+ png_bytep old_row, png_bytep new_row));
+#endif /* PNG_PROGRESSIVE_READ_SUPPORTED */
+
+extern PNG_EXPORT(png_voidp,png_malloc) PNGARG((png_structp png_ptr,
+ png_uint_32 size)) PNG_ALLOCATED;
+
+#ifdef PNG_1_0_X
+# define png_malloc_warn png_malloc
+#else
+/* Added at libpng version 1.2.4 */
+extern PNG_EXPORT(png_voidp,png_malloc_warn) PNGARG((png_structp png_ptr,
+ png_uint_32 size)) PNG_ALLOCATED;
+#endif
+
+/* Frees a pointer allocated by png_malloc() */
+extern PNG_EXPORT(void,png_free) PNGARG((png_structp png_ptr, png_voidp ptr));
+
+#ifdef PNG_1_0_X
+/* Function to allocate memory for zlib. */
+extern PNG_EXPORT(voidpf,png_zalloc) PNGARG((voidpf png_ptr, uInt items,
+ uInt size));
+
+/* Function to free memory for zlib */
+extern PNG_EXPORT(void,png_zfree) PNGARG((voidpf png_ptr, voidpf ptr));
+#endif
+
+/* Free data that was allocated internally */
+extern PNG_EXPORT(void,png_free_data) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_uint_32 free_me, int num));
+#ifdef PNG_FREE_ME_SUPPORTED
+/* Reassign responsibility for freeing existing data, whether allocated
+ * by libpng or by the application
+ */
+extern PNG_EXPORT(void,png_data_freer) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, int freer, png_uint_32 mask));
+#endif
+/* Assignments for png_data_freer */
+#define PNG_DESTROY_WILL_FREE_DATA 1
+#define PNG_SET_WILL_FREE_DATA 1
+#define PNG_USER_WILL_FREE_DATA 2
+/* Flags for png_ptr->free_me and info_ptr->free_me */
+#define PNG_FREE_HIST 0x0008
+#define PNG_FREE_ICCP 0x0010
+#define PNG_FREE_SPLT 0x0020
+#define PNG_FREE_ROWS 0x0040
+#define PNG_FREE_PCAL 0x0080
+#define PNG_FREE_SCAL 0x0100
+#define PNG_FREE_UNKN 0x0200
+#define PNG_FREE_LIST 0x0400
+#define PNG_FREE_PLTE 0x1000
+#define PNG_FREE_TRNS 0x2000
+#define PNG_FREE_TEXT 0x4000
+#define PNG_FREE_ALL 0x7fff
+#define PNG_FREE_MUL 0x4220 /* PNG_FREE_SPLT|PNG_FREE_TEXT|PNG_FREE_UNKN */
+
+#ifdef PNG_USER_MEM_SUPPORTED
+extern PNG_EXPORT(png_voidp,png_malloc_default) PNGARG((png_structp png_ptr,
+ png_uint_32 size)) PNG_ALLOCATED;
+extern PNG_EXPORT(void,png_free_default) PNGARG((png_structp png_ptr,
+ png_voidp ptr));
+#endif
+
+extern PNG_EXPORT(png_voidp,png_memcpy_check) PNGARG((png_structp png_ptr,
+ png_voidp s1, png_voidp s2, png_uint_32 size)) PNG_DEPRECATED;
+
+extern PNG_EXPORT(png_voidp,png_memset_check) PNGARG((png_structp png_ptr,
+ png_voidp s1, int value, png_uint_32 size)) PNG_DEPRECATED;
+
+#if defined(USE_FAR_KEYWORD) /* memory model conversion function */
+extern void *png_far_to_near PNGARG((png_structp png_ptr,png_voidp ptr,
+ int check));
+#endif /* USE_FAR_KEYWORD */
+
+#ifndef PNG_NO_ERROR_TEXT
+/* Fatal error in PNG image of libpng - can't continue */
+extern PNG_EXPORT(void,png_error) PNGARG((png_structp png_ptr,
+ png_const_charp error_message)) PNG_NORETURN;
+
+/* The same, but the chunk name is prepended to the error string. */
+extern PNG_EXPORT(void,png_chunk_error) PNGARG((png_structp png_ptr,
+ png_const_charp error_message)) PNG_NORETURN;
+#else
+/* Fatal error in PNG image of libpng - can't continue */
+extern PNG_EXPORT(void,png_err) PNGARG((png_structp png_ptr)) PNG_NORETURN;
+#endif
+
+#ifndef PNG_NO_WARNINGS
+/* Non-fatal error in libpng. Can continue, but may have a problem. */
+extern PNG_EXPORT(void,png_warning) PNGARG((png_structp png_ptr,
+ png_const_charp warning_message));
+
+#ifdef PNG_READ_SUPPORTED
+/* Non-fatal error in libpng, chunk name is prepended to message. */
+extern PNG_EXPORT(void,png_chunk_warning) PNGARG((png_structp png_ptr,
+ png_const_charp warning_message));
+#endif /* PNG_READ_SUPPORTED */
+#endif /* PNG_NO_WARNINGS */
+
+/* The png_set_<chunk> functions are for storing values in the png_info_struct.
+ * Similarly, the png_get_<chunk> calls are used to read values from the
+ * png_info_struct, either storing the parameters in the passed variables, or
+ * setting pointers into the png_info_struct where the data is stored. The
+ * png_get_<chunk> functions return a non-zero value if the data was available
+ * in info_ptr, or return zero and do not change any of the parameters if the
+ * data was not available.
+ *
+ * These functions should be used instead of directly accessing png_info
+ * to avoid problems with future changes in the size and internal layout of
+ * png_info_struct.
+ */
+/* Returns "flag" if chunk data is valid in info_ptr. */
+extern PNG_EXPORT(png_uint_32,png_get_valid) PNGARG((png_structp png_ptr,
+png_infop info_ptr, png_uint_32 flag));
+
+/* Returns number of bytes needed to hold a transformed row. */
+extern PNG_EXPORT(png_uint_32,png_get_rowbytes) PNGARG((png_structp png_ptr,
+png_infop info_ptr));
+
+#ifdef PNG_INFO_IMAGE_SUPPORTED
+/* Returns row_pointers, which is an array of pointers to scanlines that was
+ * returned from png_read_png().
+ */
+extern PNG_EXPORT(png_bytepp,png_get_rows) PNGARG((png_structp png_ptr,
+png_infop info_ptr));
+/* Set row_pointers, which is an array of pointers to scanlines for use
+ * by png_write_png().
+ */
+extern PNG_EXPORT(void,png_set_rows) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_bytepp row_pointers));
+#endif
+
+/* Returns number of color channels in image. */
+extern PNG_EXPORT(png_byte,png_get_channels) PNGARG((png_structp png_ptr,
+png_infop info_ptr));
+
+#ifdef PNG_EASY_ACCESS_SUPPORTED
+/* Returns image width in pixels. */
+extern PNG_EXPORT(png_uint_32, png_get_image_width) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+
+/* Returns image height in pixels. */
+extern PNG_EXPORT(png_uint_32, png_get_image_height) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+
+/* Returns image bit_depth. */
+extern PNG_EXPORT(png_byte, png_get_bit_depth) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+
+/* Returns image color_type. */
+extern PNG_EXPORT(png_byte, png_get_color_type) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+
+/* Returns image filter_type. */
+extern PNG_EXPORT(png_byte, png_get_filter_type) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+
+/* Returns image interlace_type. */
+extern PNG_EXPORT(png_byte, png_get_interlace_type) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+
+/* Returns image compression_type. */
+extern PNG_EXPORT(png_byte, png_get_compression_type) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+
+/* Returns image resolution in pixels per meter, from pHYs chunk data. */
+extern PNG_EXPORT(png_uint_32, png_get_pixels_per_meter) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+extern PNG_EXPORT(png_uint_32, png_get_x_pixels_per_meter) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+extern PNG_EXPORT(png_uint_32, png_get_y_pixels_per_meter) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+
+/* Returns pixel aspect ratio, computed from pHYs chunk data. */
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+extern PNG_EXPORT(float, png_get_pixel_aspect_ratio) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+#endif
+
+/* Returns image x, y offset in pixels or microns, from oFFs chunk data. */
+extern PNG_EXPORT(png_int_32, png_get_x_offset_pixels) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+extern PNG_EXPORT(png_int_32, png_get_y_offset_pixels) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+extern PNG_EXPORT(png_int_32, png_get_x_offset_microns) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+extern PNG_EXPORT(png_int_32, png_get_y_offset_microns) PNGARG((png_structp
+png_ptr, png_infop info_ptr));
+
+#endif /* PNG_EASY_ACCESS_SUPPORTED */
+
+/* Returns pointer to signature string read from PNG header */
+extern PNG_EXPORT(png_bytep,png_get_signature) PNGARG((png_structp png_ptr,
+png_infop info_ptr));
+
+#ifdef PNG_bKGD_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_bKGD) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_color_16p *background));
+#endif
+
+#ifdef PNG_bKGD_SUPPORTED
+extern PNG_EXPORT(void,png_set_bKGD) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_color_16p background));
+#endif
+
+#ifdef PNG_cHRM_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_cHRM) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, double *white_x, double *white_y, double *red_x,
+ double *red_y, double *green_x, double *green_y, double *blue_x,
+ double *blue_y));
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_cHRM_fixed) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_fixed_point *int_white_x, png_fixed_point
+ *int_white_y, png_fixed_point *int_red_x, png_fixed_point *int_red_y,
+ png_fixed_point *int_green_x, png_fixed_point *int_green_y, png_fixed_point
+ *int_blue_x, png_fixed_point *int_blue_y));
+#endif
+#endif
+
+#ifdef PNG_cHRM_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+extern PNG_EXPORT(void,png_set_cHRM) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, double white_x, double white_y, double red_x,
+ double red_y, double green_x, double green_y, double blue_x, double blue_y));
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+extern PNG_EXPORT(void,png_set_cHRM_fixed) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_fixed_point int_white_x, png_fixed_point int_white_y,
+ png_fixed_point int_red_x, png_fixed_point int_red_y, png_fixed_point
+ int_green_x, png_fixed_point int_green_y, png_fixed_point int_blue_x,
+ png_fixed_point int_blue_y));
+#endif
+#endif
+
+#ifdef PNG_gAMA_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_gAMA) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, double *file_gamma));
+#endif
+extern PNG_EXPORT(png_uint_32,png_get_gAMA_fixed) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_fixed_point *int_file_gamma));
+#endif
+
+#ifdef PNG_gAMA_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+extern PNG_EXPORT(void,png_set_gAMA) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, double file_gamma));
+#endif
+extern PNG_EXPORT(void,png_set_gAMA_fixed) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_fixed_point int_file_gamma));
+#endif
+
+#ifdef PNG_hIST_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_hIST) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_uint_16p *hist));
+#endif
+
+#ifdef PNG_hIST_SUPPORTED
+extern PNG_EXPORT(void,png_set_hIST) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_uint_16p hist));
+#endif
+
+extern PNG_EXPORT(png_uint_32,png_get_IHDR) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_uint_32 *width, png_uint_32 *height,
+ int *bit_depth, int *color_type, int *interlace_method,
+ int *compression_method, int *filter_method));
+
+extern PNG_EXPORT(void,png_set_IHDR) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_uint_32 width, png_uint_32 height, int bit_depth,
+ int color_type, int interlace_method, int compression_method,
+ int filter_method));
+
+#ifdef PNG_oFFs_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_oFFs) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_int_32 *offset_x, png_int_32 *offset_y,
+ int *unit_type));
+#endif
+
+#ifdef PNG_oFFs_SUPPORTED
+extern PNG_EXPORT(void,png_set_oFFs) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_int_32 offset_x, png_int_32 offset_y,
+ int unit_type));
+#endif
+
+#ifdef PNG_pCAL_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_pCAL) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_charp *purpose, png_int_32 *X0, png_int_32 *X1,
+ int *type, int *nparams, png_charp *units, png_charpp *params));
+#endif
+
+#ifdef PNG_pCAL_SUPPORTED
+extern PNG_EXPORT(void,png_set_pCAL) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_charp purpose, png_int_32 X0, png_int_32 X1,
+ int type, int nparams, png_charp units, png_charpp params));
+#endif
+
+#ifdef PNG_pHYs_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_pHYs) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_uint_32 *res_x, png_uint_32 *res_y, int *unit_type));
+#endif
+
+#ifdef PNG_pHYs_SUPPORTED
+extern PNG_EXPORT(void,png_set_pHYs) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_uint_32 res_x, png_uint_32 res_y, int unit_type));
+#endif
+
+extern PNG_EXPORT(png_uint_32,png_get_PLTE) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_colorp *palette, int *num_palette));
+
+extern PNG_EXPORT(void,png_set_PLTE) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_colorp palette, int num_palette));
+
+#ifdef PNG_sBIT_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_sBIT) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_color_8p *sig_bit));
+#endif
+
+#ifdef PNG_sBIT_SUPPORTED
+extern PNG_EXPORT(void,png_set_sBIT) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_color_8p sig_bit));
+#endif
+
+#ifdef PNG_sRGB_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_sRGB) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, int *intent));
+#endif
+
+#ifdef PNG_sRGB_SUPPORTED
+extern PNG_EXPORT(void,png_set_sRGB) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, int intent));
+extern PNG_EXPORT(void,png_set_sRGB_gAMA_and_cHRM) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, int intent));
+#endif
+
+#ifdef PNG_iCCP_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_iCCP) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_charpp name, int *compression_type,
+ png_charpp profile, png_uint_32 *proflen));
+ /* Note to maintainer: profile should be png_bytepp */
+#endif
+
+#ifdef PNG_iCCP_SUPPORTED
+extern PNG_EXPORT(void,png_set_iCCP) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_charp name, int compression_type,
+ png_charp profile, png_uint_32 proflen));
+ /* Note to maintainer: profile should be png_bytep */
+#endif
+
+#ifdef PNG_sPLT_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_sPLT) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_sPLT_tpp entries));
+#endif
+
+#ifdef PNG_sPLT_SUPPORTED
+extern PNG_EXPORT(void,png_set_sPLT) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_sPLT_tp entries, int nentries));
+#endif
+
+#ifdef PNG_TEXT_SUPPORTED
+/* png_get_text also returns the number of text chunks in *num_text */
+extern PNG_EXPORT(png_uint_32,png_get_text) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_textp *text_ptr, int *num_text));
+#endif
+
+/*
+ * Note while png_set_text() will accept a structure whose text,
+ * language, and translated keywords are NULL pointers, the structure
+ * returned by png_get_text will always contain regular
+ * zero-terminated C strings. They might be empty strings but
+ * they will never be NULL pointers.
+ */
+
+#ifdef PNG_TEXT_SUPPORTED
+extern PNG_EXPORT(void,png_set_text) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_textp text_ptr, int num_text));
+#endif
+
+#ifdef PNG_tIME_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_tIME) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_timep *mod_time));
+#endif
+
+#ifdef PNG_tIME_SUPPORTED
+extern PNG_EXPORT(void,png_set_tIME) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_timep mod_time));
+#endif
+
+#ifdef PNG_tRNS_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_tRNS) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_bytep *trans, int *num_trans,
+ png_color_16p *trans_values));
+#endif
+
+#ifdef PNG_tRNS_SUPPORTED
+extern PNG_EXPORT(void,png_set_tRNS) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_bytep trans, int num_trans,
+ png_color_16p trans_values));
+#endif
+
+#ifdef PNG_tRNS_SUPPORTED
+#endif
+
+#ifdef PNG_sCAL_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_sCAL) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, int *unit, double *width, double *height));
+#else
+#ifdef PNG_FIXED_POINT_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_get_sCAL_s) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, int *unit, png_charpp swidth, png_charpp sheight));
+#endif
+#endif
+#endif /* PNG_sCAL_SUPPORTED */
+
+#ifdef PNG_sCAL_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+extern PNG_EXPORT(void,png_set_sCAL) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, int unit, double width, double height));
+#else
+#ifdef PNG_FIXED_POINT_SUPPORTED
+extern PNG_EXPORT(void,png_set_sCAL_s) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, int unit, png_charp swidth, png_charp sheight));
+#endif
+#endif
+#endif /* PNG_sCAL_SUPPORTED || PNG_WRITE_sCAL_SUPPORTED */
+
+#ifdef PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+/* Provide a list of chunks and how they are to be handled, if the built-in
+ handling or default unknown chunk handling is not desired. Any chunks not
+ listed will be handled in the default manner. The IHDR and IEND chunks
+ must not be listed.
+ keep = 0: follow default behaviour
+ = 1: do not keep
+ = 2: keep only if safe-to-copy
+ = 3: keep even if unsafe-to-copy
+*/
+extern PNG_EXPORT(void, png_set_keep_unknown_chunks) PNGARG((png_structp
+ png_ptr, int keep, png_bytep chunk_list, int num_chunks));
+PNG_EXPORT(int,png_handle_as_unknown) PNGARG((png_structp png_ptr, png_bytep
+ chunk_name));
+#endif
+#ifdef PNG_UNKNOWN_CHUNKS_SUPPORTED
+extern PNG_EXPORT(void, png_set_unknown_chunks) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_unknown_chunkp unknowns, int num_unknowns));
+extern PNG_EXPORT(void, png_set_unknown_chunk_location)
+ PNGARG((png_structp png_ptr, png_infop info_ptr, int chunk, int location));
+extern PNG_EXPORT(png_uint_32,png_get_unknown_chunks) PNGARG((png_structp
+ png_ptr, png_infop info_ptr, png_unknown_chunkpp entries));
+#endif
+
+/* Png_free_data() will turn off the "valid" flag for anything it frees.
+ * If you need to turn it off for a chunk that your application has freed,
+ * you can use png_set_invalid(png_ptr, info_ptr, PNG_INFO_CHNK);
+ */
+extern PNG_EXPORT(void, png_set_invalid) PNGARG((png_structp png_ptr,
+ png_infop info_ptr, int mask));
+
+#ifdef PNG_INFO_IMAGE_SUPPORTED
+/* The "params" pointer is currently not used and is for future expansion. */
+extern PNG_EXPORT(void, png_read_png) PNGARG((png_structp png_ptr,
+ png_infop info_ptr,
+ int transforms,
+ png_voidp params));
+extern PNG_EXPORT(void, png_write_png) PNGARG((png_structp png_ptr,
+ png_infop info_ptr,
+ int transforms,
+ png_voidp params));
+#endif
+
+/* Define PNG_DEBUG at compile time for debugging information. Higher
+ * numbers for PNG_DEBUG mean more debugging information. This has
+ * only been added since version 0.95 so it is not implemented throughout
+ * libpng yet, but more support will be added as needed.
+ */
+#ifdef PNG_DEBUG
+#if (PNG_DEBUG > 0)
+#if !defined(PNG_DEBUG_FILE) && defined(_MSC_VER)
+#include <crtdbg.h>
+#if (PNG_DEBUG > 1)
+#ifndef _DEBUG
+# define _DEBUG
+#endif
+#ifndef png_debug
+#define png_debug(l,m) _RPT0(_CRT_WARN,m PNG_STRING_NEWLINE)
+#endif
+#ifndef png_debug1
+#define png_debug1(l,m,p1) _RPT1(_CRT_WARN,m PNG_STRING_NEWLINE,p1)
+#endif
+#ifndef png_debug2
+#define png_debug2(l,m,p1,p2) _RPT2(_CRT_WARN,m PNG_STRING_NEWLINE,p1,p2)
+#endif
+#endif
+#else /* PNG_DEBUG_FILE || !_MSC_VER */
+#ifndef PNG_DEBUG_FILE
+#define PNG_DEBUG_FILE stderr
+#endif /* PNG_DEBUG_FILE */
+
+#if (PNG_DEBUG > 1)
+/* Note: ["%s"m PNG_STRING_NEWLINE] probably does not work on non-ISO
+ * compilers.
+ */
+# ifdef __STDC__
+# ifndef png_debug
+# define png_debug(l,m) \
+ { \
+ int num_tabs=l; \
+ fprintf(PNG_DEBUG_FILE,"%s"m PNG_STRING_NEWLINE,(num_tabs==1 ? "\t" : \
+ (num_tabs==2 ? "\t\t":(num_tabs>2 ? "\t\t\t":"")))); \
+ }
+# endif
+# ifndef png_debug1
+# define png_debug1(l,m,p1) \
+ { \
+ int num_tabs=l; \
+ fprintf(PNG_DEBUG_FILE,"%s"m PNG_STRING_NEWLINE,(num_tabs==1 ? "\t" : \
+ (num_tabs==2 ? "\t\t":(num_tabs>2 ? "\t\t\t":""))),p1); \
+ }
+# endif
+# ifndef png_debug2
+# define png_debug2(l,m,p1,p2) \
+ { \
+ int num_tabs=l; \
+ fprintf(PNG_DEBUG_FILE,"%s"m PNG_STRING_NEWLINE,(num_tabs==1 ? "\t" : \
+ (num_tabs==2 ? "\t\t":(num_tabs>2 ? "\t\t\t":""))),p1,p2); \
+ }
+# endif
+# else /* __STDC __ */
+# ifndef png_debug
+# define png_debug(l,m) \
+ { \
+ int num_tabs=l; \
+ char format[256]; \
+ snprintf(format,256,"%s%s%s",(num_tabs==1 ? "\t" : \
+ (num_tabs==2 ? "\t\t":(num_tabs>2 ? "\t\t\t":""))), \
+ m,PNG_STRING_NEWLINE); \
+ fprintf(PNG_DEBUG_FILE,format); \
+ }
+# endif
+# ifndef png_debug1
+# define png_debug1(l,m,p1) \
+ { \
+ int num_tabs=l; \
+ char format[256]; \
+ snprintf(format,256,"%s%s%s",(num_tabs==1 ? "\t" : \
+ (num_tabs==2 ? "\t\t":(num_tabs>2 ? "\t\t\t":""))), \
+ m,PNG_STRING_NEWLINE); \
+ fprintf(PNG_DEBUG_FILE,format,p1); \
+ }
+# endif
+# ifndef png_debug2
+# define png_debug2(l,m,p1,p2) \
+ { \
+ int num_tabs=l; \
+ char format[256]; \
+ snprintf(format,256,"%s%s%s",(num_tabs==1 ? "\t" : \
+ (num_tabs==2 ? "\t\t":(num_tabs>2 ? "\t\t\t":""))), \
+ m,PNG_STRING_NEWLINE); \
+ fprintf(PNG_DEBUG_FILE,format,p1,p2); \
+ }
+# endif
+# endif /* __STDC __ */
+#endif /* (PNG_DEBUG > 1) */
+
+#endif /* _MSC_VER */
+#endif /* (PNG_DEBUG > 0) */
+#endif /* PNG_DEBUG */
+#ifndef png_debug
+#define png_debug(l, m)
+#endif
+#ifndef png_debug1
+#define png_debug1(l, m, p1)
+#endif
+#ifndef png_debug2
+#define png_debug2(l, m, p1, p2)
+#endif
+
+extern PNG_EXPORT(png_charp,png_get_copyright) PNGARG((png_structp png_ptr));
+extern PNG_EXPORT(png_charp,png_get_header_ver) PNGARG((png_structp png_ptr));
+extern PNG_EXPORT(png_charp,png_get_header_version) PNGARG((png_structp png_ptr));
+extern PNG_EXPORT(png_charp,png_get_libpng_ver) PNGARG((png_structp png_ptr));
+
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+extern PNG_EXPORT(png_uint_32,png_permit_mng_features) PNGARG((png_structp
+ png_ptr, png_uint_32 mng_features_permitted));
+#endif
+
+/* For use in png_set_keep_unknown, added to version 1.2.6 */
+#define PNG_HANDLE_CHUNK_AS_DEFAULT 0
+#define PNG_HANDLE_CHUNK_NEVER 1
+#define PNG_HANDLE_CHUNK_IF_SAFE 2
+#define PNG_HANDLE_CHUNK_ALWAYS 3
+
+/* Added to version 1.2.0 */
+#ifdef PNG_ASSEMBLER_CODE_SUPPORTED
+#ifdef PNG_MMX_CODE_SUPPORTED
+#define PNG_ASM_FLAG_MMX_SUPPORT_COMPILED 0x01 /* not user-settable */
+#define PNG_ASM_FLAG_MMX_SUPPORT_IN_CPU 0x02 /* not user-settable */
+#define PNG_ASM_FLAG_MMX_READ_COMBINE_ROW 0x04
+#define PNG_ASM_FLAG_MMX_READ_INTERLACE 0x08
+#define PNG_ASM_FLAG_MMX_READ_FILTER_SUB 0x10
+#define PNG_ASM_FLAG_MMX_READ_FILTER_UP 0x20
+#define PNG_ASM_FLAG_MMX_READ_FILTER_AVG 0x40
+#define PNG_ASM_FLAG_MMX_READ_FILTER_PAETH 0x80
+#define PNG_ASM_FLAGS_INITIALIZED 0x80000000 /* not user-settable */
+
+#define PNG_MMX_READ_FLAGS ( PNG_ASM_FLAG_MMX_READ_COMBINE_ROW \
+ | PNG_ASM_FLAG_MMX_READ_INTERLACE \
+ | PNG_ASM_FLAG_MMX_READ_FILTER_SUB \
+ | PNG_ASM_FLAG_MMX_READ_FILTER_UP \
+ | PNG_ASM_FLAG_MMX_READ_FILTER_AVG \
+ | PNG_ASM_FLAG_MMX_READ_FILTER_PAETH )
+#define PNG_MMX_WRITE_FLAGS ( 0 )
+
+#define PNG_MMX_FLAGS ( PNG_ASM_FLAG_MMX_SUPPORT_COMPILED \
+ | PNG_ASM_FLAG_MMX_SUPPORT_IN_CPU \
+ | PNG_MMX_READ_FLAGS \
+ | PNG_MMX_WRITE_FLAGS )
+
+#define PNG_SELECT_READ 1
+#define PNG_SELECT_WRITE 2
+#endif /* PNG_MMX_CODE_SUPPORTED */
+
+#ifndef PNG_1_0_X
+/* pngget.c */
+extern PNG_EXPORT(png_uint_32,png_get_mmx_flagmask)
+ PNGARG((int flag_select, int *compilerID));
+
+/* pngget.c */
+extern PNG_EXPORT(png_uint_32,png_get_asm_flagmask)
+ PNGARG((int flag_select));
+
+/* pngget.c */
+extern PNG_EXPORT(png_uint_32,png_get_asm_flags)
+ PNGARG((png_structp png_ptr));
+
+/* pngget.c */
+extern PNG_EXPORT(png_byte,png_get_mmx_bitdepth_threshold)
+ PNGARG((png_structp png_ptr));
+
+/* pngget.c */
+extern PNG_EXPORT(png_uint_32,png_get_mmx_rowbytes_threshold)
+ PNGARG((png_structp png_ptr));
+
+/* pngset.c */
+extern PNG_EXPORT(void,png_set_asm_flags)
+ PNGARG((png_structp png_ptr, png_uint_32 asm_flags));
+
+/* pngset.c */
+extern PNG_EXPORT(void,png_set_mmx_thresholds)
+ PNGARG((png_structp png_ptr, png_byte mmx_bitdepth_threshold,
+ png_uint_32 mmx_rowbytes_threshold));
+
+#endif /* PNG_1_0_X */
+
+#ifndef PNG_1_0_X
+/* png.c, pnggccrd.c, or pngvcrd.c */
+extern PNG_EXPORT(int,png_mmx_support) PNGARG((void));
+#endif /* PNG_1_0_X */
+#endif /* PNG_ASSEMBLER_CODE_SUPPORTED */
+
+/* Strip the prepended error numbers ("#nnn ") from error and warning
+ * messages before passing them to the error or warning handler.
+ */
+#ifdef PNG_ERROR_NUMBERS_SUPPORTED
+extern PNG_EXPORT(void,png_set_strip_error_numbers) PNGARG((png_structp
+ png_ptr, png_uint_32 strip_mode));
+#endif
+
+/* Added at libpng-1.2.6 */
+#ifdef PNG_SET_USER_LIMITS_SUPPORTED
+extern PNG_EXPORT(void,png_set_user_limits) PNGARG((png_structp
+ png_ptr, png_uint_32 user_width_max, png_uint_32 user_height_max));
+extern PNG_EXPORT(png_uint_32,png_get_user_width_max) PNGARG((png_structp
+ png_ptr));
+extern PNG_EXPORT(png_uint_32,png_get_user_height_max) PNGARG((png_structp
+ png_ptr));
+#endif
+/* Maintainer: Put new public prototypes here ^, in libpng.3, and in
+ * project defs
+ */
+
+#ifdef PNG_READ_COMPOSITE_NODIV_SUPPORTED
+/* With these routines we avoid an integer divide, which will be slower on
+ * most machines. However, it does take more operations than the corresponding
+ * divide method, so it may be slower on a few RISC systems. There are two
+ * shifts (by 8 or 16 bits) and an addition, versus a single integer divide.
+ *
+ * Note that the rounding factors are NOT supposed to be the same! 128 and
+ * 32768 are correct for the NODIV code; 127 and 32767 are correct for the
+ * standard method.
+ *
+ * [Optimized code by Greg Roelofs and Mark Adler...blame us for bugs. :-) ]
+ */
+
+ /* fg and bg should be in `gamma 1.0' space; alpha is the opacity */
+
+# define png_composite(composite, fg, alpha, bg) \
+ { png_uint_16 temp = (png_uint_16)((png_uint_16)(fg) * (png_uint_16)(alpha) \
+ + (png_uint_16)(bg)*(png_uint_16)(255 - \
+ (png_uint_16)(alpha)) + (png_uint_16)128); \
+ (composite) = (png_byte)((temp + (temp >> 8)) >> 8); }
+
+# define png_composite_16(composite, fg, alpha, bg) \
+ { png_uint_32 temp = (png_uint_32)((png_uint_32)(fg) * (png_uint_32)(alpha) \
+ + (png_uint_32)(bg)*(png_uint_32)(65535L - \
+ (png_uint_32)(alpha)) + (png_uint_32)32768L); \
+ (composite) = (png_uint_16)((temp + (temp >> 16)) >> 16); }
+
+#else /* Standard method using integer division */
+
+# define png_composite(composite, fg, alpha, bg) \
+ (composite) = (png_byte)(((png_uint_16)(fg) * (png_uint_16)(alpha) + \
+ (png_uint_16)(bg) * (png_uint_16)(255 - (png_uint_16)(alpha)) + \
+ (png_uint_16)127) / 255)
+
+# define png_composite_16(composite, fg, alpha, bg) \
+ (composite) = (png_uint_16)(((png_uint_32)(fg) * (png_uint_32)(alpha) + \
+ (png_uint_32)(bg)*(png_uint_32)(65535L - (png_uint_32)(alpha)) + \
+ (png_uint_32)32767) / (png_uint_32)65535L)
+
+#endif /* PNG_READ_COMPOSITE_NODIV_SUPPORTED */
+
+/* Inline macros to do direct reads of bytes from the input buffer. These
+ * require that you are using an architecture that uses PNG byte ordering
+ * (MSB first) and supports unaligned data storage. I think that PowerPC
+ * in big-endian mode and 680x0 are the only ones that will support this.
+ * The x86 line of processors definitely do not. The png_get_int_32()
+ * routine also assumes we are using two's complement format for negative
+ * values, which is almost certainly true.
+ */
+#ifdef PNG_READ_BIG_ENDIAN_SUPPORTED
+# define png_get_uint_32(buf) ( *((png_uint_32p) (buf)))
+# define png_get_uint_16(buf) ( *((png_uint_16p) (buf)))
+# define png_get_int_32(buf) ( *((png_int_32p) (buf)))
+#else
+extern PNG_EXPORT(png_uint_32,png_get_uint_32) PNGARG((png_bytep buf));
+extern PNG_EXPORT(png_uint_16,png_get_uint_16) PNGARG((png_bytep buf));
+extern PNG_EXPORT(png_int_32,png_get_int_32) PNGARG((png_bytep buf));
+#endif /* !PNG_READ_BIG_ENDIAN_SUPPORTED */
+extern PNG_EXPORT(png_uint_32,png_get_uint_31)
+ PNGARG((png_structp png_ptr, png_bytep buf));
+/* No png_get_int_16 -- may be added if there's a real need for it. */
+
+/* Place a 32-bit number into a buffer in PNG byte order (big-endian).
+ */
+extern PNG_EXPORT(void,png_save_uint_32)
+ PNGARG((png_bytep buf, png_uint_32 i));
+extern PNG_EXPORT(void,png_save_int_32)
+ PNGARG((png_bytep buf, png_int_32 i));
+
+/* Place a 16-bit number into a buffer in PNG byte order.
+ * The parameter is declared unsigned int, not png_uint_16,
+ * just to avoid potential problems on pre-ANSI C compilers.
+ */
+extern PNG_EXPORT(void,png_save_uint_16)
+ PNGARG((png_bytep buf, unsigned int i));
+/* No png_save_int_16 -- may be added if there's a real need for it. */
+
+/* ************************************************************************* */
+
+/* These next functions are used internally in the code. They generally
+ * shouldn't be used unless you are writing code to add or replace some
+ * functionality in libpng. More information about most functions can
+ * be found in the files where the functions are located.
+ */
+
+
+/* Various modes of operation, that are visible to applications because
+ * they are used for unknown chunk location.
+ */
+#define PNG_HAVE_IHDR 0x01
+#define PNG_HAVE_PLTE 0x02
+#define PNG_HAVE_IDAT 0x04
+#define PNG_AFTER_IDAT 0x08 /* Have complete zlib datastream */
+#define PNG_HAVE_IEND 0x10
+
+#ifdef PNG_INTERNAL
+
+/* More modes of operation. Note that after an init, mode is set to
+ * zero automatically when the structure is created.
+ */
+#define PNG_HAVE_gAMA 0x20
+#define PNG_HAVE_cHRM 0x40
+#define PNG_HAVE_sRGB 0x80
+#define PNG_HAVE_CHUNK_HEADER 0x100
+#define PNG_WROTE_tIME 0x200
+#define PNG_WROTE_INFO_BEFORE_PLTE 0x400
+#define PNG_BACKGROUND_IS_GRAY 0x800
+#define PNG_HAVE_PNG_SIGNATURE 0x1000
+#define PNG_HAVE_CHUNK_AFTER_IDAT 0x2000 /* Have another chunk after IDAT */
+
+/* Flags for the transformations the PNG library does on the image data */
+#define PNG_BGR 0x0001
+#define PNG_INTERLACE 0x0002
+#define PNG_PACK 0x0004
+#define PNG_SHIFT 0x0008
+#define PNG_SWAP_BYTES 0x0010
+#define PNG_INVERT_MONO 0x0020
+#define PNG_DITHER 0x0040
+#define PNG_BACKGROUND 0x0080
+#define PNG_BACKGROUND_EXPAND 0x0100
+ /* 0x0200 unused */
+#define PNG_16_TO_8 0x0400
+#define PNG_RGBA 0x0800
+#define PNG_EXPAND 0x1000
+#define PNG_GAMMA 0x2000
+#define PNG_GRAY_TO_RGB 0x4000
+#define PNG_FILLER 0x8000L
+#define PNG_PACKSWAP 0x10000L
+#define PNG_SWAP_ALPHA 0x20000L
+#define PNG_STRIP_ALPHA 0x40000L
+#define PNG_INVERT_ALPHA 0x80000L
+#define PNG_USER_TRANSFORM 0x100000L
+#define PNG_RGB_TO_GRAY_ERR 0x200000L
+#define PNG_RGB_TO_GRAY_WARN 0x400000L
+#define PNG_RGB_TO_GRAY 0x600000L /* two bits, RGB_TO_GRAY_ERR|WARN */
+ /* 0x800000L Unused */
+#define PNG_ADD_ALPHA 0x1000000L /* Added to libpng-1.2.7 */
+#define PNG_EXPAND_tRNS 0x2000000L /* Added to libpng-1.2.9 */
+#define PNG_PREMULTIPLY_ALPHA 0x4000000L /* Added to libpng-1.2.41 */
+ /* by volker */
+ /* 0x8000000L unused */
+ /* 0x10000000L unused */
+ /* 0x20000000L unused */
+ /* 0x40000000L unused */
+
+/* Flags for png_create_struct */
+#define PNG_STRUCT_PNG 0x0001
+#define PNG_STRUCT_INFO 0x0002
+
+/* Scaling factor for filter heuristic weighting calculations */
+#define PNG_WEIGHT_SHIFT 8
+#define PNG_WEIGHT_FACTOR (1<<(PNG_WEIGHT_SHIFT))
+#define PNG_COST_SHIFT 3
+#define PNG_COST_FACTOR (1<<(PNG_COST_SHIFT))
+
+/* Flags for the png_ptr->flags rather than declaring a byte for each one */
+#define PNG_FLAG_ZLIB_CUSTOM_STRATEGY 0x0001
+#define PNG_FLAG_ZLIB_CUSTOM_LEVEL 0x0002
+#define PNG_FLAG_ZLIB_CUSTOM_MEM_LEVEL 0x0004
+#define PNG_FLAG_ZLIB_CUSTOM_WINDOW_BITS 0x0008
+#define PNG_FLAG_ZLIB_CUSTOM_METHOD 0x0010
+#define PNG_FLAG_ZLIB_FINISHED 0x0020
+#define PNG_FLAG_ROW_INIT 0x0040
+#define PNG_FLAG_FILLER_AFTER 0x0080
+#define PNG_FLAG_CRC_ANCILLARY_USE 0x0100
+#define PNG_FLAG_CRC_ANCILLARY_NOWARN 0x0200
+#define PNG_FLAG_CRC_CRITICAL_USE 0x0400
+#define PNG_FLAG_CRC_CRITICAL_IGNORE 0x0800
+#define PNG_FLAG_FREE_PLTE 0x1000
+#define PNG_FLAG_FREE_TRNS 0x2000
+#define PNG_FLAG_FREE_HIST 0x4000
+#define PNG_FLAG_KEEP_UNKNOWN_CHUNKS 0x8000L
+#define PNG_FLAG_KEEP_UNSAFE_CHUNKS 0x10000L
+#define PNG_FLAG_LIBRARY_MISMATCH 0x20000L
+#define PNG_FLAG_STRIP_ERROR_NUMBERS 0x40000L
+#define PNG_FLAG_STRIP_ERROR_TEXT 0x80000L
+#define PNG_FLAG_MALLOC_NULL_MEM_OK 0x100000L
+#define PNG_FLAG_ADD_ALPHA 0x200000L /* Added to libpng-1.2.8 */
+#define PNG_FLAG_STRIP_ALPHA 0x400000L /* Added to libpng-1.2.8 */
+ /* 0x800000L unused */
+ /* 0x1000000L unused */
+ /* 0x2000000L unused */
+ /* 0x4000000L unused */
+ /* 0x8000000L unused */
+ /* 0x10000000L unused */
+ /* 0x20000000L unused */
+ /* 0x40000000L unused */
+
+#define PNG_FLAG_CRC_ANCILLARY_MASK (PNG_FLAG_CRC_ANCILLARY_USE | \
+ PNG_FLAG_CRC_ANCILLARY_NOWARN)
+
+#define PNG_FLAG_CRC_CRITICAL_MASK (PNG_FLAG_CRC_CRITICAL_USE | \
+ PNG_FLAG_CRC_CRITICAL_IGNORE)
+
+#define PNG_FLAG_CRC_MASK (PNG_FLAG_CRC_ANCILLARY_MASK | \
+ PNG_FLAG_CRC_CRITICAL_MASK)
+
+/* Save typing and make code easier to understand */
+
+#define PNG_COLOR_DIST(c1, c2) (abs((int)((c1).red) - (int)((c2).red)) + \
+ abs((int)((c1).green) - (int)((c2).green)) + \
+ abs((int)((c1).blue) - (int)((c2).blue)))
+
+/* Added to libpng-1.2.6 JB */
+#define PNG_ROWBYTES(pixel_bits, width) \
+ ((pixel_bits) >= 8 ? \
+ ((width) * (((png_uint_32)(pixel_bits)) >> 3)) : \
+ (( ((width) * ((png_uint_32)(pixel_bits))) + 7) >> 3) )
+
+/* PNG_OUT_OF_RANGE returns true if value is outside the range
+ * ideal-delta..ideal+delta. Each argument is evaluated twice.
+ * "ideal" and "delta" should be constants, normally simple
+ * integers, "value" a variable. Added to libpng-1.2.6 JB
+ */
+#define PNG_OUT_OF_RANGE(value, ideal, delta) \
+ ( (value) < (ideal)-(delta) || (value) > (ideal)+(delta) )
+
+/* Variables declared in png.c - only it needs to define PNG_NO_EXTERN */
+#if !defined(PNG_NO_EXTERN) || defined(PNG_ALWAYS_EXTERN)
+/* Place to hold the signature string for a PNG file. */
+#ifdef PNG_USE_GLOBAL_ARRAYS
+ PNG_EXPORT_VAR (PNG_CONST png_byte FARDATA) png_sig[8];
+#else
+#endif
+#endif /* PNG_NO_EXTERN */
+
+/* Constant strings for known chunk types. If you need to add a chunk,
+ * define the name here, and add an invocation of the macro in png.c and
+ * wherever it's needed.
+ */
+#define PNG_IHDR png_byte png_IHDR[5] = { 73, 72, 68, 82, '\0'}
+#define PNG_IDAT png_byte png_IDAT[5] = { 73, 68, 65, 84, '\0'}
+#define PNG_IEND png_byte png_IEND[5] = { 73, 69, 78, 68, '\0'}
+#define PNG_PLTE png_byte png_PLTE[5] = { 80, 76, 84, 69, '\0'}
+#define PNG_bKGD png_byte png_bKGD[5] = { 98, 75, 71, 68, '\0'}
+#define PNG_cHRM png_byte png_cHRM[5] = { 99, 72, 82, 77, '\0'}
+#define PNG_gAMA png_byte png_gAMA[5] = {103, 65, 77, 65, '\0'}
+#define PNG_hIST png_byte png_hIST[5] = {104, 73, 83, 84, '\0'}
+#define PNG_iCCP png_byte png_iCCP[5] = {105, 67, 67, 80, '\0'}
+#define PNG_iTXt png_byte png_iTXt[5] = {105, 84, 88, 116, '\0'}
+#define PNG_oFFs png_byte png_oFFs[5] = {111, 70, 70, 115, '\0'}
+#define PNG_pCAL png_byte png_pCAL[5] = {112, 67, 65, 76, '\0'}
+#define PNG_sCAL png_byte png_sCAL[5] = {115, 67, 65, 76, '\0'}
+#define PNG_pHYs png_byte png_pHYs[5] = {112, 72, 89, 115, '\0'}
+#define PNG_sBIT png_byte png_sBIT[5] = {115, 66, 73, 84, '\0'}
+#define PNG_sPLT png_byte png_sPLT[5] = {115, 80, 76, 84, '\0'}
+#define PNG_sRGB png_byte png_sRGB[5] = {115, 82, 71, 66, '\0'}
+#define PNG_tEXt png_byte png_tEXt[5] = {116, 69, 88, 116, '\0'}
+#define PNG_tIME png_byte png_tIME[5] = {116, 73, 77, 69, '\0'}
+#define PNG_tRNS png_byte png_tRNS[5] = {116, 82, 78, 83, '\0'}
+#define PNG_zTXt png_byte png_zTXt[5] = {122, 84, 88, 116, '\0'}
+
+#ifdef PNG_USE_GLOBAL_ARRAYS
+PNG_EXPORT_VAR (png_byte FARDATA) png_IHDR[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_IDAT[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_IEND[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_PLTE[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_bKGD[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_cHRM[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_gAMA[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_hIST[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_iCCP[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_iTXt[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_oFFs[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_pCAL[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_sCAL[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_pHYs[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_sBIT[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_sPLT[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_sRGB[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_tEXt[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_tIME[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_tRNS[5];
+PNG_EXPORT_VAR (png_byte FARDATA) png_zTXt[5];
+#endif /* PNG_USE_GLOBAL_ARRAYS */
+
+#if defined(PNG_1_0_X) || defined (PNG_1_2_X)
+/* Initialize png_ptr struct for reading, and allocate any other memory.
+ * (old interface - DEPRECATED - use png_create_read_struct instead).
+ */
+extern PNG_EXPORT(void,png_read_init) PNGARG((png_structp png_ptr))
+ PNG_DEPRECATED;
+#undef png_read_init
+#define png_read_init(png_ptr) png_read_init_3(&png_ptr, \
+ PNG_LIBPNG_VER_STRING, png_sizeof(png_struct));
+#endif
+
+extern PNG_EXPORT(void,png_read_init_3) PNGARG((png_structpp ptr_ptr,
+ png_const_charp user_png_ver, png_size_t png_struct_size));
+#if defined(PNG_1_0_X) || defined (PNG_1_2_X)
+extern PNG_EXPORT(void,png_read_init_2) PNGARG((png_structp png_ptr,
+ png_const_charp user_png_ver, png_size_t png_struct_size, png_size_t
+ png_info_size));
+#endif
+
+#if defined(PNG_1_0_X) || defined (PNG_1_2_X)
+/* Initialize png_ptr struct for writing, and allocate any other memory.
+ * (old interface - DEPRECATED - use png_create_write_struct instead).
+ */
+extern PNG_EXPORT(void,png_write_init) PNGARG((png_structp png_ptr))
+ PNG_DEPRECATED;
+#undef png_write_init
+#define png_write_init(png_ptr) png_write_init_3(&png_ptr, \
+ PNG_LIBPNG_VER_STRING, png_sizeof(png_struct));
+#endif
+
+extern PNG_EXPORT(void,png_write_init_3) PNGARG((png_structpp ptr_ptr,
+ png_const_charp user_png_ver, png_size_t png_struct_size));
+extern PNG_EXPORT(void,png_write_init_2) PNGARG((png_structp png_ptr,
+ png_const_charp user_png_ver, png_size_t png_struct_size, png_size_t
+ png_info_size));
+
+/* Allocate memory for an internal libpng struct */
+PNG_EXTERN png_voidp png_create_struct PNGARG((int type)) PNG_PRIVATE;
+
+/* Free memory from internal libpng struct */
+PNG_EXTERN void png_destroy_struct PNGARG((png_voidp struct_ptr)) PNG_PRIVATE;
+
+PNG_EXTERN png_voidp png_create_struct_2 PNGARG((int type, png_malloc_ptr
+ malloc_fn, png_voidp mem_ptr)) PNG_PRIVATE;
+PNG_EXTERN void png_destroy_struct_2 PNGARG((png_voidp struct_ptr,
+ png_free_ptr free_fn, png_voidp mem_ptr)) PNG_PRIVATE;
+
+/* Free any memory that info_ptr points to and reset struct. */
+PNG_EXTERN void png_info_destroy PNGARG((png_structp png_ptr,
+ png_infop info_ptr)) PNG_PRIVATE;
+
+#ifndef PNG_1_0_X
+/* Function to allocate memory for zlib. */
+PNG_EXTERN voidpf png_zalloc PNGARG((voidpf png_ptr, uInt items,
+ uInt size)) PNG_PRIVATE;
+
+/* Function to free memory for zlib */
+PNG_EXTERN void png_zfree PNGARG((voidpf png_ptr, voidpf ptr)) PNG_PRIVATE;
+
+#ifdef PNG_SIZE_T
+/* Function to convert a sizeof an item to png_sizeof item */
+ PNG_EXTERN png_size_t PNGAPI png_convert_size PNGARG((size_t size))
+ PNG_PRIVATE;
+#endif
+
+/* Next four functions are used internally as callbacks. PNGAPI is required
+ * but not PNG_EXPORT. PNGAPI added at libpng version 1.2.3.
+ */
+
+PNG_EXTERN void PNGAPI png_default_read_data PNGARG((png_structp png_ptr,
+ png_bytep data, png_size_t length)) PNG_PRIVATE;
+
+#ifdef PNG_PROGRESSIVE_READ_SUPPORTED
+PNG_EXTERN void PNGAPI png_push_fill_buffer PNGARG((png_structp png_ptr,
+ png_bytep buffer, png_size_t length)) PNG_PRIVATE;
+#endif
+
+PNG_EXTERN void PNGAPI png_default_write_data PNGARG((png_structp png_ptr,
+ png_bytep data, png_size_t length)) PNG_PRIVATE;
+
+#ifdef PNG_WRITE_FLUSH_SUPPORTED
+#ifdef PNG_STDIO_SUPPORTED
+PNG_EXTERN void PNGAPI png_default_flush PNGARG((png_structp png_ptr))
+ PNG_PRIVATE;
+#endif
+#endif
+#else /* PNG_1_0_X */
+#ifdef PNG_PROGRESSIVE_READ_SUPPORTED
+PNG_EXTERN void png_push_fill_buffer PNGARG((png_structp png_ptr,
+ png_bytep buffer, png_size_t length)) PNG_PRIVATE;
+#endif
+#endif /* PNG_1_0_X */
+
+/* Reset the CRC variable */
+PNG_EXTERN void png_reset_crc PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+
+/* Write the "data" buffer to whatever output you are using. */
+PNG_EXTERN void png_write_data PNGARG((png_structp png_ptr, png_bytep data,
+ png_size_t length)) PNG_PRIVATE;
+
+/* Read data from whatever input you are using into the "data" buffer */
+PNG_EXTERN void png_read_data PNGARG((png_structp png_ptr, png_bytep data,
+ png_size_t length)) PNG_PRIVATE;
+
+/* Read bytes into buf, and update png_ptr->crc */
+PNG_EXTERN void png_crc_read PNGARG((png_structp png_ptr, png_bytep buf,
+ png_size_t length)) PNG_PRIVATE;
+
+/* Decompress data in a chunk that uses compression */
+#if defined(PNG_zTXt_SUPPORTED) || defined(PNG_iTXt_SUPPORTED) || \
+ defined(PNG_iCCP_SUPPORTED) || defined(PNG_sPLT_SUPPORTED)
+PNG_EXTERN void png_decompress_chunk PNGARG((png_structp png_ptr,
+ int comp_type, png_size_t chunklength,
+ png_size_t prefix_length, png_size_t *data_length)) PNG_PRIVATE;
+#endif
+
+/* Read "skip" bytes, read the file crc, and (optionally) verify png_ptr->crc */
+PNG_EXTERN int png_crc_finish PNGARG((png_structp png_ptr, png_uint_32 skip)
+ PNG_PRIVATE);
+
+/* Read the CRC from the file and compare it to the libpng calculated CRC */
+PNG_EXTERN int png_crc_error PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+
+/* Calculate the CRC over a section of data. Note that we are only
+ * passing a maximum of 64K on systems that have this as a memory limit,
+ * since this is the maximum buffer size we can specify.
+ */
+PNG_EXTERN void png_calculate_crc PNGARG((png_structp png_ptr, png_bytep ptr,
+ png_size_t length)) PNG_PRIVATE;
+
+#ifdef PNG_WRITE_FLUSH_SUPPORTED
+PNG_EXTERN void png_flush PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+#endif
+
+/* Simple function to write the signature */
+PNG_EXTERN void png_write_sig PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+
+/* Write various chunks */
+
+/* Write the IHDR chunk, and update the png_struct with the necessary
+ * information.
+ */
+PNG_EXTERN void png_write_IHDR PNGARG((png_structp png_ptr, png_uint_32 width,
+ png_uint_32 height,
+ int bit_depth, int color_type, int compression_method, int filter_method,
+ int interlace_method)) PNG_PRIVATE;
+
+PNG_EXTERN void png_write_PLTE PNGARG((png_structp png_ptr, png_colorp palette,
+ png_uint_32 num_pal)) PNG_PRIVATE;
+
+PNG_EXTERN void png_write_IDAT PNGARG((png_structp png_ptr, png_bytep data,
+ png_size_t length)) PNG_PRIVATE;
+
+PNG_EXTERN void png_write_IEND PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+
+#ifdef PNG_WRITE_gAMA_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+PNG_EXTERN void png_write_gAMA PNGARG((png_structp png_ptr, double file_gamma))
+ PNG_PRIVATE;
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+PNG_EXTERN void png_write_gAMA_fixed PNGARG((png_structp png_ptr,
+ png_fixed_point file_gamma)) PNG_PRIVATE;
+#endif
+#endif
+
+#ifdef PNG_WRITE_sBIT_SUPPORTED
+PNG_EXTERN void png_write_sBIT PNGARG((png_structp png_ptr, png_color_8p sbit,
+ int color_type)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_cHRM_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+PNG_EXTERN void png_write_cHRM PNGARG((png_structp png_ptr,
+ double white_x, double white_y,
+ double red_x, double red_y, double green_x, double green_y,
+ double blue_x, double blue_y)) PNG_PRIVATE;
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+PNG_EXTERN void png_write_cHRM_fixed PNGARG((png_structp png_ptr,
+ png_fixed_point int_white_x, png_fixed_point int_white_y,
+ png_fixed_point int_red_x, png_fixed_point int_red_y, png_fixed_point
+ int_green_x, png_fixed_point int_green_y, png_fixed_point int_blue_x,
+ png_fixed_point int_blue_y)) PNG_PRIVATE;
+#endif
+#endif
+
+#ifdef PNG_WRITE_sRGB_SUPPORTED
+PNG_EXTERN void png_write_sRGB PNGARG((png_structp png_ptr,
+ int intent)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_iCCP_SUPPORTED
+PNG_EXTERN void png_write_iCCP PNGARG((png_structp png_ptr,
+ png_charp name, int compression_type,
+ png_charp profile, int proflen)) PNG_PRIVATE;
+ /* Note to maintainer: profile should be png_bytep */
+#endif
+
+#ifdef PNG_WRITE_sPLT_SUPPORTED
+PNG_EXTERN void png_write_sPLT PNGARG((png_structp png_ptr,
+ png_sPLT_tp palette)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_tRNS_SUPPORTED
+PNG_EXTERN void png_write_tRNS PNGARG((png_structp png_ptr, png_bytep trans,
+ png_color_16p values, int number, int color_type)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_bKGD_SUPPORTED
+PNG_EXTERN void png_write_bKGD PNGARG((png_structp png_ptr,
+ png_color_16p values, int color_type)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_hIST_SUPPORTED
+PNG_EXTERN void png_write_hIST PNGARG((png_structp png_ptr, png_uint_16p hist,
+ int num_hist)) PNG_PRIVATE;
+#endif
+
+#if defined(PNG_WRITE_TEXT_SUPPORTED) || defined(PNG_WRITE_pCAL_SUPPORTED) || \
+ defined(PNG_WRITE_iCCP_SUPPORTED) || defined(PNG_WRITE_sPLT_SUPPORTED)
+PNG_EXTERN png_size_t png_check_keyword PNGARG((png_structp png_ptr,
+ png_charp key, png_charpp new_key)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_tEXt_SUPPORTED
+PNG_EXTERN void png_write_tEXt PNGARG((png_structp png_ptr, png_charp key,
+ png_charp text, png_size_t text_len)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_zTXt_SUPPORTED
+PNG_EXTERN void png_write_zTXt PNGARG((png_structp png_ptr, png_charp key,
+ png_charp text, png_size_t text_len, int compression)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_iTXt_SUPPORTED
+PNG_EXTERN void png_write_iTXt PNGARG((png_structp png_ptr,
+ int compression, png_charp key, png_charp lang, png_charp lang_key,
+ png_charp text)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_TEXT_SUPPORTED /* Added at version 1.0.14 and 1.2.4 */
+PNG_EXTERN int png_set_text_2 PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_textp text_ptr, int num_text)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_oFFs_SUPPORTED
+PNG_EXTERN void png_write_oFFs PNGARG((png_structp png_ptr,
+ png_int_32 x_offset, png_int_32 y_offset, int unit_type)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_pCAL_SUPPORTED
+PNG_EXTERN void png_write_pCAL PNGARG((png_structp png_ptr, png_charp purpose,
+ png_int_32 X0, png_int_32 X1, int type, int nparams,
+ png_charp units, png_charpp params)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_pHYs_SUPPORTED
+PNG_EXTERN void png_write_pHYs PNGARG((png_structp png_ptr,
+ png_uint_32 x_pixels_per_unit, png_uint_32 y_pixels_per_unit,
+ int unit_type)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_tIME_SUPPORTED
+PNG_EXTERN void png_write_tIME PNGARG((png_structp png_ptr,
+ png_timep mod_time)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_sCAL_SUPPORTED
+#if defined(PNG_FLOATING_POINT_SUPPORTED) && !defined(PNG_NO_STDIO)
+PNG_EXTERN void png_write_sCAL PNGARG((png_structp png_ptr,
+ int unit, double width, double height)) PNG_PRIVATE;
+#else
+#ifdef PNG_FIXED_POINT_SUPPORTED
+PNG_EXTERN void png_write_sCAL_s PNGARG((png_structp png_ptr,
+ int unit, png_charp width, png_charp height)) PNG_PRIVATE;
+#endif
+#endif
+#endif
+
+/* Called when finished processing a row of data */
+PNG_EXTERN void png_write_finish_row PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+
+/* Internal use only. Called before first row of data */
+PNG_EXTERN void png_write_start_row PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+
+#ifdef PNG_READ_GAMMA_SUPPORTED
+PNG_EXTERN void png_build_gamma_table PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+#endif
+
+/* Combine a row of data, dealing with alpha, etc. if requested */
+PNG_EXTERN void png_combine_row PNGARG((png_structp png_ptr, png_bytep row,
+ int mask)) PNG_PRIVATE;
+
+#ifdef PNG_READ_INTERLACING_SUPPORTED
+/* Expand an interlaced row */
+/* OLD pre-1.0.9 interface:
+PNG_EXTERN void png_do_read_interlace PNGARG((png_row_infop row_info,
+ png_bytep row, int pass, png_uint_32 transformations)) PNG_PRIVATE;
+ */
+PNG_EXTERN void png_do_read_interlace PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+#endif
+
+/* GRR TO DO (2.0 or whenever): simplify other internal calling interfaces */
+
+#ifdef PNG_WRITE_INTERLACING_SUPPORTED
+/* Grab pixels out of a row for an interlaced pass */
+PNG_EXTERN void png_do_write_interlace PNGARG((png_row_infop row_info,
+ png_bytep row, int pass)) PNG_PRIVATE;
+#endif
+
+/* Unfilter a row */
+PNG_EXTERN void png_read_filter_row PNGARG((png_structp png_ptr,
+ png_row_infop row_info, png_bytep row, png_bytep prev_row,
+ int filter)) PNG_PRIVATE;
+
+/* Choose the best filter to use and filter the row data */
+PNG_EXTERN void png_write_find_filter PNGARG((png_structp png_ptr,
+ png_row_infop row_info)) PNG_PRIVATE;
+
+/* Write out the filtered row. */
+PNG_EXTERN void png_write_filtered_row PNGARG((png_structp png_ptr,
+ png_bytep filtered_row)) PNG_PRIVATE;
+/* Finish a row while reading, dealing with interlacing passes, etc. */
+PNG_EXTERN void png_read_finish_row PNGARG((png_structp png_ptr));
+
+/* Initialize the row buffers, etc. */
+PNG_EXTERN void png_read_start_row PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+/* Optional call to update the users info structure */
+PNG_EXTERN void png_read_transform_info PNGARG((png_structp png_ptr,
+ png_infop info_ptr)) PNG_PRIVATE;
+
+/* These are the functions that do the transformations */
+#ifdef PNG_READ_FILLER_SUPPORTED
+PNG_EXTERN void png_do_read_filler PNGARG((png_row_infop row_info,
+ png_bytep row, png_uint_32 filler, png_uint_32 flags)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_SWAP_ALPHA_SUPPORTED
+PNG_EXTERN void png_do_read_swap_alpha PNGARG((png_row_infop row_info,
+ png_bytep row)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_SWAP_ALPHA_SUPPORTED
+PNG_EXTERN void png_do_write_swap_alpha PNGARG((png_row_infop row_info,
+ png_bytep row)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_INVERT_ALPHA_SUPPORTED
+PNG_EXTERN void png_do_read_invert_alpha PNGARG((png_row_infop row_info,
+ png_bytep row)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_INVERT_ALPHA_SUPPORTED
+PNG_EXTERN void png_do_write_invert_alpha PNGARG((png_row_infop row_info,
+ png_bytep row)) PNG_PRIVATE;
+#endif
+
+#if defined(PNG_WRITE_FILLER_SUPPORTED) || \
+ defined(PNG_READ_STRIP_ALPHA_SUPPORTED)
+PNG_EXTERN void png_do_strip_filler PNGARG((png_row_infop row_info,
+ png_bytep row, png_uint_32 flags)) PNG_PRIVATE;
+#endif
+
+#if defined(PNG_READ_SWAP_SUPPORTED) || defined(PNG_WRITE_SWAP_SUPPORTED)
+PNG_EXTERN void png_do_swap PNGARG((png_row_infop row_info,
+ png_bytep row)) PNG_PRIVATE;
+#endif
+
+#if defined(PNG_READ_PACKSWAP_SUPPORTED) || defined(PNG_WRITE_PACKSWAP_SUPPORTED)
+PNG_EXTERN void png_do_packswap PNGARG((png_row_infop row_info,
+ png_bytep row)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_RGB_TO_GRAY_SUPPORTED
+PNG_EXTERN int png_do_rgb_to_gray PNGARG((png_structp png_ptr, png_row_infop
+ row_info, png_bytep row)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_GRAY_TO_RGB_SUPPORTED
+PNG_EXTERN void png_do_gray_to_rgb PNGARG((png_row_infop row_info,
+ png_bytep row)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_PACK_SUPPORTED
+PNG_EXTERN void png_do_unpack PNGARG((png_row_infop row_info,
+ png_bytep row)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_SHIFT_SUPPORTED
+PNG_EXTERN void png_do_unshift PNGARG((png_row_infop row_info, png_bytep row,
+ png_color_8p sig_bits)) PNG_PRIVATE;
+#endif
+
+#if defined(PNG_READ_INVERT_SUPPORTED) || defined(PNG_WRITE_INVERT_SUPPORTED)
+PNG_EXTERN void png_do_invert PNGARG((png_row_infop row_info,
+ png_bytep row)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_16_TO_8_SUPPORTED
+PNG_EXTERN void png_do_chop PNGARG((png_row_infop row_info,
+ png_bytep row)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_DITHER_SUPPORTED
+PNG_EXTERN void png_do_dither PNGARG((png_row_infop row_info,
+ png_bytep row, png_bytep palette_lookup,
+ png_bytep dither_lookup)) PNG_PRIVATE;
+
+# ifdef PNG_CORRECT_PALETTE_SUPPORTED
+PNG_EXTERN void png_correct_palette PNGARG((png_structp png_ptr,
+ png_colorp palette, int num_palette)) PNG_PRIVATE;
+# endif
+#endif
+
+#if defined(PNG_READ_BGR_SUPPORTED) || defined(PNG_WRITE_BGR_SUPPORTED)
+PNG_EXTERN void png_do_bgr PNGARG((png_row_infop row_info,
+ png_bytep row)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_PACK_SUPPORTED
+PNG_EXTERN void png_do_pack PNGARG((png_row_infop row_info,
+ png_bytep row, png_uint_32 bit_depth)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_WRITE_SHIFT_SUPPORTED
+PNG_EXTERN void png_do_shift PNGARG((png_row_infop row_info, png_bytep row,
+ png_color_8p bit_depth)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_BACKGROUND_SUPPORTED
+#ifdef PNG_READ_GAMMA_SUPPORTED
+PNG_EXTERN void png_do_background PNGARG((png_row_infop row_info, png_bytep row,
+ png_color_16p trans_values, png_color_16p background,
+ png_color_16p background_1,
+ png_bytep gamma_table, png_bytep gamma_from_1, png_bytep gamma_to_1,
+ png_uint_16pp gamma_16, png_uint_16pp gamma_16_from_1,
+ png_uint_16pp gamma_16_to_1, int gamma_shift)) PNG_PRIVATE;
+#else
+PNG_EXTERN void png_do_background PNGARG((png_row_infop row_info, png_bytep row,
+ png_color_16p trans_values, png_color_16p background)) PNG_PRIVATE;
+#endif
+#endif
+
+#ifdef PNG_READ_GAMMA_SUPPORTED
+PNG_EXTERN void png_do_gamma PNGARG((png_row_infop row_info, png_bytep row,
+ png_bytep gamma_table, png_uint_16pp gamma_16_table,
+ int gamma_shift)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_EXPAND_SUPPORTED
+PNG_EXTERN void png_do_expand_palette PNGARG((png_row_infop row_info,
+ png_bytep row, png_colorp palette, png_bytep trans,
+ int num_trans)) PNG_PRIVATE;
+PNG_EXTERN void png_do_expand PNGARG((png_row_infop row_info,
+ png_bytep row, png_color_16p trans_value)) PNG_PRIVATE;
+#endif
+
+/* The following decodes the appropriate chunks, and does error correction,
+ * then calls the appropriate callback for the chunk if it is valid.
+ */
+
+/* Decode the IHDR chunk */
+PNG_EXTERN void png_handle_IHDR PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+PNG_EXTERN void png_handle_PLTE PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length));
+PNG_EXTERN void png_handle_IEND PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length));
+
+#ifdef PNG_READ_bKGD_SUPPORTED
+PNG_EXTERN void png_handle_bKGD PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_cHRM_SUPPORTED
+PNG_EXTERN void png_handle_cHRM PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_gAMA_SUPPORTED
+PNG_EXTERN void png_handle_gAMA PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_hIST_SUPPORTED
+PNG_EXTERN void png_handle_hIST PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_iCCP_SUPPORTED
+extern void png_handle_iCCP PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length));
+#endif /* PNG_READ_iCCP_SUPPORTED */
+
+#ifdef PNG_READ_iTXt_SUPPORTED
+PNG_EXTERN void png_handle_iTXt PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_oFFs_SUPPORTED
+PNG_EXTERN void png_handle_oFFs PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_pCAL_SUPPORTED
+PNG_EXTERN void png_handle_pCAL PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_pHYs_SUPPORTED
+PNG_EXTERN void png_handle_pHYs PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_sBIT_SUPPORTED
+PNG_EXTERN void png_handle_sBIT PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_sCAL_SUPPORTED
+PNG_EXTERN void png_handle_sCAL PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_sPLT_SUPPORTED
+extern void png_handle_sPLT PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif /* PNG_READ_sPLT_SUPPORTED */
+
+#ifdef PNG_READ_sRGB_SUPPORTED
+PNG_EXTERN void png_handle_sRGB PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_tEXt_SUPPORTED
+PNG_EXTERN void png_handle_tEXt PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_tIME_SUPPORTED
+PNG_EXTERN void png_handle_tIME PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_tRNS_SUPPORTED
+PNG_EXTERN void png_handle_tRNS PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_READ_zTXt_SUPPORTED
+PNG_EXTERN void png_handle_zTXt PNGARG((png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+#endif
+
+PNG_EXTERN void png_handle_unknown PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_uint_32 length)) PNG_PRIVATE;
+
+PNG_EXTERN void png_check_chunk_name PNGARG((png_structp png_ptr,
+ png_bytep chunk_name)) PNG_PRIVATE;
+
+/* Handle the transformations for reading and writing */
+PNG_EXTERN void png_do_read_transformations
+ PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+PNG_EXTERN void png_do_write_transformations
+ PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+
+PNG_EXTERN void png_init_read_transformations
+ PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+
+#ifdef PNG_PROGRESSIVE_READ_SUPPORTED
+PNG_EXTERN void png_push_read_chunk PNGARG((png_structp png_ptr,
+ png_infop info_ptr)) PNG_PRIVATE;
+PNG_EXTERN void png_push_read_sig PNGARG((png_structp png_ptr,
+ png_infop info_ptr)) PNG_PRIVATE;
+PNG_EXTERN void png_push_check_crc PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+PNG_EXTERN void png_push_crc_skip PNGARG((png_structp png_ptr,
+ png_uint_32 length)) PNG_PRIVATE;
+PNG_EXTERN void png_push_crc_finish PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+PNG_EXTERN void png_push_save_buffer PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+PNG_EXTERN void png_push_restore_buffer PNGARG((png_structp png_ptr,
+ png_bytep buffer, png_size_t buffer_length)) PNG_PRIVATE;
+PNG_EXTERN void png_push_read_IDAT PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+PNG_EXTERN void png_process_IDAT_data PNGARG((png_structp png_ptr,
+ png_bytep buffer, png_size_t buffer_length)) PNG_PRIVATE;
+PNG_EXTERN void png_push_process_row PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+PNG_EXTERN void png_push_handle_unknown PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_uint_32 length)) PNG_PRIVATE;
+PNG_EXTERN void png_push_have_info PNGARG((png_structp png_ptr,
+ png_infop info_ptr)) PNG_PRIVATE;
+PNG_EXTERN void png_push_have_end PNGARG((png_structp png_ptr,
+ png_infop info_ptr)) PNG_PRIVATE;
+PNG_EXTERN void png_push_have_row PNGARG((png_structp png_ptr,
+ png_bytep row)) PNG_PRIVATE;
+PNG_EXTERN void png_push_read_end PNGARG((png_structp png_ptr,
+ png_infop info_ptr)) PNG_PRIVATE;
+PNG_EXTERN void png_process_some_data PNGARG((png_structp png_ptr,
+ png_infop info_ptr)) PNG_PRIVATE;
+PNG_EXTERN void png_read_push_finish_row
+ PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+#ifdef PNG_READ_tEXt_SUPPORTED
+PNG_EXTERN void png_push_handle_tEXt PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_uint_32 length)) PNG_PRIVATE;
+PNG_EXTERN void png_push_read_tEXt PNGARG((png_structp png_ptr,
+ png_infop info_ptr)) PNG_PRIVATE;
+#endif
+#ifdef PNG_READ_zTXt_SUPPORTED
+PNG_EXTERN void png_push_handle_zTXt PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_uint_32 length)) PNG_PRIVATE;
+PNG_EXTERN void png_push_read_zTXt PNGARG((png_structp png_ptr,
+ png_infop info_ptr)) PNG_PRIVATE;
+#endif
+#ifdef PNG_READ_iTXt_SUPPORTED
+PNG_EXTERN void png_push_handle_iTXt PNGARG((png_structp png_ptr,
+ png_infop info_ptr, png_uint_32 length)) PNG_PRIVATE;
+PNG_EXTERN void png_push_read_iTXt PNGARG((png_structp png_ptr,
+ png_infop info_ptr)) PNG_PRIVATE;
+#endif
+
+#endif /* PNG_PROGRESSIVE_READ_SUPPORTED */
+
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+PNG_EXTERN void png_do_read_intrapixel PNGARG((png_row_infop row_info,
+ png_bytep row)) PNG_PRIVATE;
+PNG_EXTERN void png_do_write_intrapixel PNGARG((png_row_infop row_info,
+ png_bytep row)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_ASSEMBLER_CODE_SUPPORTED
+#ifdef PNG_MMX_CODE_SUPPORTED
+/* png.c */ /* PRIVATE */
+PNG_EXTERN void png_init_mmx_flags PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+#endif
+#endif
+
+
+/* The following six functions will be exported in libpng-1.4.0. */
+#if defined(PNG_INCH_CONVERSIONS) && defined(PNG_FLOATING_POINT_SUPPORTED)
+PNG_EXTERN png_uint_32 png_get_pixels_per_inch PNGARG((png_structp png_ptr,
+png_infop info_ptr));
+
+PNG_EXTERN png_uint_32 png_get_x_pixels_per_inch PNGARG((png_structp png_ptr,
+png_infop info_ptr));
+
+PNG_EXTERN png_uint_32 png_get_y_pixels_per_inch PNGARG((png_structp png_ptr,
+png_infop info_ptr));
+
+PNG_EXTERN float png_get_x_offset_inches PNGARG((png_structp png_ptr,
+png_infop info_ptr));
+
+PNG_EXTERN float png_get_y_offset_inches PNGARG((png_structp png_ptr,
+png_infop info_ptr));
+
+#ifdef PNG_pHYs_SUPPORTED
+PNG_EXTERN png_uint_32 png_get_pHYs_dpi PNGARG((png_structp png_ptr,
+png_infop info_ptr, png_uint_32 *res_x, png_uint_32 *res_y, int *unit_type));
+#endif /* PNG_pHYs_SUPPORTED */
+#endif /* PNG_INCH_CONVERSIONS && PNG_FLOATING_POINT_SUPPORTED */
+
+/* Read the chunk header (length + type name) */
+PNG_EXTERN png_uint_32 png_read_chunk_header
+ PNGARG((png_structp png_ptr)) PNG_PRIVATE;
+
+/* Added at libpng version 1.2.34 */
+#ifdef PNG_cHRM_SUPPORTED
+PNG_EXTERN int png_check_cHRM_fixed PNGARG((png_structp png_ptr,
+ png_fixed_point int_white_x, png_fixed_point int_white_y,
+ png_fixed_point int_red_x, png_fixed_point int_red_y, png_fixed_point
+ int_green_x, png_fixed_point int_green_y, png_fixed_point int_blue_x,
+ png_fixed_point int_blue_y)) PNG_PRIVATE;
+#endif
+
+#ifdef PNG_cHRM_SUPPORTED
+#ifdef PNG_CHECK_cHRM_SUPPORTED
+/* Added at libpng version 1.2.34 */
+PNG_EXTERN void png_64bit_product PNGARG((long v1, long v2,
+ unsigned long *hi_product, unsigned long *lo_product)) PNG_PRIVATE;
+#endif
+#endif
+
+/* Added at libpng version 1.2.41 */
+PNG_EXTERN void png_check_IHDR PNGARG((png_structp png_ptr,
+ png_uint_32 width, png_uint_32 height, int bit_depth,
+ int color_type, int interlace_type, int compression_type,
+ int filter_type)) PNG_PRIVATE;
+
+/* Added at libpng version 1.2.41 */
+PNG_EXTERN png_voidp png_calloc PNGARG((png_structp png_ptr,
+ png_uint_32 size));
+
+/* Maintainer: Put new private prototypes here ^ and in libpngpf.3 */
+
+#endif /* PNG_INTERNAL */
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* PNG_VERSION_INFO_ONLY */
+/* Do not put anything past this line */
+#endif /* PNG_H */
diff --git a/trunk/src/third_party/libpng/pngconf.h b/trunk/src/third_party/libpng/pngconf.h
new file mode 100644
index 0000000..defc16d
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngconf.h
@@ -0,0 +1,1665 @@
+
+/* pngconf.h - machine configurable file for libpng
+ *
+ * libpng version 1.2.44 - June 26, 2010
+ * Copyright (c) 1998-2010 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ */
+
+/* Any machine specific code is near the front of this file, so if you
+ * are configuring libpng for a machine, you may want to read the section
+ * starting here down to where it starts to typedef png_color, png_text,
+ * and png_info.
+ */
+
+#ifndef PNGCONF_H
+#define PNGCONF_H
+
+#define PNG_1_2_X
+
+/*
+ * PNG_USER_CONFIG has to be defined on the compiler command line. This
+ * includes the resource compiler for Windows DLL configurations.
+ */
+#ifdef PNG_USER_CONFIG
+# ifndef PNG_USER_PRIVATEBUILD
+# define PNG_USER_PRIVATEBUILD
+# endif
+#include "pngusr.h"
+#endif
+
+/* PNG_CONFIGURE_LIBPNG is set by the "configure" script. */
+#ifdef PNG_CONFIGURE_LIBPNG
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+#endif
+
+/*
+ * Added at libpng-1.2.8
+ *
+ * If you create a private DLL you need to define in "pngusr.h" the followings:
+ * #define PNG_USER_PRIVATEBUILD <Describes by whom and why this version of
+ * the DLL was built>
+ * e.g. #define PNG_USER_PRIVATEBUILD "Build by MyCompany for xyz reasons."
+ * #define PNG_USER_DLLFNAME_POSTFIX <two-letter postfix that serve to
+ * distinguish your DLL from those of the official release. These
+ * correspond to the trailing letters that come after the version
+ * number and must match your private DLL name>
+ * e.g. // private DLL "libpng13gx.dll"
+ * #define PNG_USER_DLLFNAME_POSTFIX "gx"
+ *
+ * The following macros are also at your disposal if you want to complete the
+ * DLL VERSIONINFO structure.
+ * - PNG_USER_VERSIONINFO_COMMENTS
+ * - PNG_USER_VERSIONINFO_COMPANYNAME
+ * - PNG_USER_VERSIONINFO_LEGALTRADEMARKS
+ */
+
+#ifdef __STDC__
+#ifdef SPECIALBUILD
+# pragma message("PNG_LIBPNG_SPECIALBUILD (and deprecated SPECIALBUILD)\
+ are now LIBPNG reserved macros. Use PNG_USER_PRIVATEBUILD instead.")
+#endif
+
+#ifdef PRIVATEBUILD
+# pragma message("PRIVATEBUILD is deprecated.\
+ Use PNG_USER_PRIVATEBUILD instead.")
+# define PNG_USER_PRIVATEBUILD PRIVATEBUILD
+#endif
+#endif /* __STDC__ */
+
+#ifndef PNG_VERSION_INFO_ONLY
+
+/* End of material added to libpng-1.2.8 */
+
+/* Added at libpng-1.2.19, removed at libpng-1.2.20 because it caused trouble
+ Restored at libpng-1.2.21 */
+#if !defined(PNG_NO_WARN_UNINITIALIZED_ROW) && \
+ !defined(PNG_WARN_UNINITIALIZED_ROW)
+# define PNG_WARN_UNINITIALIZED_ROW 1
+#endif
+/* End of material added at libpng-1.2.19/1.2.21 */
+
+/* This is the size of the compression buffer, and thus the size of
+ * an IDAT chunk. Make this whatever size you feel is best for your
+ * machine. One of these will be allocated per png_struct. When this
+ * is full, it writes the data to the disk, and does some other
+ * calculations. Making this an extremely small size will slow
+ * the library down, but you may want to experiment to determine
+ * where it becomes significant, if you are concerned with memory
+ * usage. Note that zlib allocates at least 32Kb also. For readers,
+ * this describes the size of the buffer available to read the data in.
+ * Unless this gets smaller than the size of a row (compressed),
+ * it should not make much difference how big this is.
+ */
+
+#ifndef PNG_ZBUF_SIZE
+# define PNG_ZBUF_SIZE 8192
+#endif
+
+/* Enable if you want a write-only libpng */
+
+#ifndef PNG_NO_READ_SUPPORTED
+# define PNG_READ_SUPPORTED
+#endif
+
+/* Enable if you want a read-only libpng */
+
+#ifndef PNG_NO_WRITE_SUPPORTED
+# define PNG_WRITE_SUPPORTED
+#endif
+
+/* Enabled in 1.2.41. */
+#ifdef PNG_ALLOW_BENIGN_ERRORS
+# define png_benign_error png_warning
+# define png_chunk_benign_error png_chunk_warning
+#else
+# ifndef PNG_BENIGN_ERRORS_SUPPORTED
+# define png_benign_error png_error
+# define png_chunk_benign_error png_chunk_error
+# endif
+#endif
+
+/* Added in libpng-1.2.41 */
+#if !defined(PNG_NO_WARNINGS) && !defined(PNG_WARNINGS_SUPPORTED)
+# define PNG_WARNINGS_SUPPORTED
+#endif
+
+#if !defined(PNG_NO_ERROR_TEXT) && !defined(PNG_ERROR_TEXT_SUPPORTED)
+# define PNG_ERROR_TEXT_SUPPORTED
+#endif
+
+#if !defined(PNG_NO_CHECK_cHRM) && !defined(PNG_CHECK_cHRM_SUPPORTED)
+# define PNG_CHECK_cHRM_SUPPORTED
+#endif
+
+/* Enabled by default in 1.2.0. You can disable this if you don't need to
+ * support PNGs that are embedded in MNG datastreams
+ */
+#if !defined(PNG_1_0_X) && !defined(PNG_NO_MNG_FEATURES)
+# ifndef PNG_MNG_FEATURES_SUPPORTED
+# define PNG_MNG_FEATURES_SUPPORTED
+# endif
+#endif
+
+#ifndef PNG_NO_FLOATING_POINT_SUPPORTED
+# ifndef PNG_FLOATING_POINT_SUPPORTED
+# define PNG_FLOATING_POINT_SUPPORTED
+# endif
+#endif
+
+/* If you are running on a machine where you cannot allocate more
+ * than 64K of memory at once, uncomment this. While libpng will not
+ * normally need that much memory in a chunk (unless you load up a very
+ * large file), zlib needs to know how big of a chunk it can use, and
+ * libpng thus makes sure to check any memory allocation to verify it
+ * will fit into memory.
+#define PNG_MAX_MALLOC_64K
+ */
+#if defined(MAXSEG_64K) && !defined(PNG_MAX_MALLOC_64K)
+# define PNG_MAX_MALLOC_64K
+#endif
+
+/* Special munging to support doing things the 'cygwin' way:
+ * 'Normal' png-on-win32 defines/defaults:
+ * PNG_BUILD_DLL -- building dll
+ * PNG_USE_DLL -- building an application, linking to dll
+ * (no define) -- building static library, or building an
+ * application and linking to the static lib
+ * 'Cygwin' defines/defaults:
+ * PNG_BUILD_DLL -- (ignored) building the dll
+ * (no define) -- (ignored) building an application, linking to the dll
+ * PNG_STATIC -- (ignored) building the static lib, or building an
+ * application that links to the static lib.
+ * ALL_STATIC -- (ignored) building various static libs, or building an
+ * application that links to the static libs.
+ * Thus,
+ * a cygwin user should define either PNG_BUILD_DLL or PNG_STATIC, and
+ * this bit of #ifdefs will define the 'correct' config variables based on
+ * that. If a cygwin user *wants* to define 'PNG_USE_DLL' that's okay, but
+ * unnecessary.
+ *
+ * Also, the precedence order is:
+ * ALL_STATIC (since we can't #undef something outside our namespace)
+ * PNG_BUILD_DLL
+ * PNG_STATIC
+ * (nothing) == PNG_USE_DLL
+ *
+ * CYGWIN (2002-01-20): The preceding is now obsolete. With the advent
+ * of auto-import in binutils, we no longer need to worry about
+ * __declspec(dllexport) / __declspec(dllimport) and friends. Therefore,
+ * we don't need to worry about PNG_STATIC or ALL_STATIC when it comes
+ * to __declspec() stuff. However, we DO need to worry about
+ * PNG_BUILD_DLL and PNG_STATIC because those change some defaults
+ * such as CONSOLE_IO and whether GLOBAL_ARRAYS are allowed.
+ */
+#ifdef __CYGWIN__
+# ifdef ALL_STATIC
+# ifdef PNG_BUILD_DLL
+# undef PNG_BUILD_DLL
+# endif
+# ifdef PNG_USE_DLL
+# undef PNG_USE_DLL
+# endif
+# ifdef PNG_DLL
+# undef PNG_DLL
+# endif
+# ifndef PNG_STATIC
+# define PNG_STATIC
+# endif
+# else
+# ifdef PNG_BUILD_DLL
+# ifdef PNG_STATIC
+# undef PNG_STATIC
+# endif
+# ifdef PNG_USE_DLL
+# undef PNG_USE_DLL
+# endif
+# ifndef PNG_DLL
+# define PNG_DLL
+# endif
+# else
+# ifdef PNG_STATIC
+# ifdef PNG_USE_DLL
+# undef PNG_USE_DLL
+# endif
+# ifdef PNG_DLL
+# undef PNG_DLL
+# endif
+# else
+# ifndef PNG_USE_DLL
+# define PNG_USE_DLL
+# endif
+# ifndef PNG_DLL
+# define PNG_DLL
+# endif
+# endif
+# endif
+# endif
+#endif
+
+/* This protects us against compilers that run on a windowing system
+ * and thus don't have or would rather us not use the stdio types:
+ * stdin, stdout, and stderr. The only one currently used is stderr
+ * in png_error() and png_warning(). #defining PNG_NO_CONSOLE_IO will
+ * prevent these from being compiled and used. #defining PNG_NO_STDIO
+ * will also prevent these, plus will prevent the entire set of stdio
+ * macros and functions (FILE *, printf, etc.) from being compiled and used,
+ * unless (PNG_DEBUG > 0) has been #defined.
+ *
+ * #define PNG_NO_CONSOLE_IO
+ * #define PNG_NO_STDIO
+ */
+
+#if !defined(PNG_NO_STDIO) && !defined(PNG_STDIO_SUPPORTED)
+# define PNG_STDIO_SUPPORTED
+#endif
+
+#ifdef _WIN32_WCE
+# include <windows.h>
+ /* Console I/O functions are not supported on WindowsCE */
+# define PNG_NO_CONSOLE_IO
+ /* abort() may not be supported on some/all Windows CE platforms */
+# define PNG_ABORT() exit(-1)
+# ifdef PNG_DEBUG
+# undef PNG_DEBUG
+# endif
+#endif
+
+#ifdef PNG_BUILD_DLL
+# ifndef PNG_CONSOLE_IO_SUPPORTED
+# ifndef PNG_NO_CONSOLE_IO
+# define PNG_NO_CONSOLE_IO
+# endif
+# endif
+#endif
+
+# ifdef PNG_NO_STDIO
+# ifndef PNG_NO_CONSOLE_IO
+# define PNG_NO_CONSOLE_IO
+# endif
+# ifdef PNG_DEBUG
+# if (PNG_DEBUG > 0)
+# include <stdio.h>
+# endif
+# endif
+# else
+# ifndef _WIN32_WCE
+/* "stdio.h" functions are not supported on WindowsCE */
+# include <stdio.h>
+# endif
+# endif
+
+#if !(defined PNG_NO_CONSOLE_IO) && !defined(PNG_CONSOLE_IO_SUPPORTED)
+# define PNG_CONSOLE_IO_SUPPORTED
+#endif
+
+/* This macro protects us against machines that don't have function
+ * prototypes (ie K&R style headers). If your compiler does not handle
+ * function prototypes, define this macro and use the included ansi2knr.
+ * I've always been able to use _NO_PROTO as the indicator, but you may
+ * need to drag the empty declaration out in front of here, or change the
+ * ifdef to suit your own needs.
+ */
+#ifndef PNGARG
+
+#ifdef OF /* zlib prototype munger */
+# define PNGARG(arglist) OF(arglist)
+#else
+
+#ifdef _NO_PROTO
+# define PNGARG(arglist) ()
+# ifndef PNG_TYPECAST_NULL
+# define PNG_TYPECAST_NULL
+# endif
+#else
+# define PNGARG(arglist) arglist
+#endif /* _NO_PROTO */
+
+
+#endif /* OF */
+
+#endif /* PNGARG */
+
+/* Try to determine if we are compiling on a Mac. Note that testing for
+ * just __MWERKS__ is not good enough, because the Codewarrior is now used
+ * on non-Mac platforms.
+ */
+#ifndef MACOS
+# if (defined(__MWERKS__) && defined(macintosh)) || defined(applec) || \
+ defined(THINK_C) || defined(__SC__) || defined(TARGET_OS_MAC)
+# define MACOS
+# endif
+#endif
+
+/* enough people need this for various reasons to include it here */
+#if !defined(MACOS) && !defined(RISCOS) && !defined(_WIN32_WCE)
+# include <sys/types.h>
+#endif
+
+#if !defined(PNG_SETJMP_NOT_SUPPORTED) && !defined(PNG_NO_SETJMP_SUPPORTED)
+# define PNG_SETJMP_SUPPORTED
+#endif
+
+#ifdef PNG_SETJMP_SUPPORTED
+/* This is an attempt to force a single setjmp behaviour on Linux. If
+ * the X config stuff didn't define _BSD_SOURCE we wouldn't need this.
+ *
+ * You can bypass this test if you know that your application uses exactly
+ * the same setjmp.h that was included when libpng was built. Only define
+ * PNG_SKIP_SETJMP_CHECK while building your application, prior to the
+ * application's '#include "png.h"'. Don't define PNG_SKIP_SETJMP_CHECK
+ * while building a separate libpng library for general use.
+ */
+
+# ifndef PNG_SKIP_SETJMP_CHECK
+# ifdef __linux__
+# ifdef _BSD_SOURCE
+# define PNG_SAVE_BSD_SOURCE
+# undef _BSD_SOURCE
+# endif
+# ifdef _SETJMP_H
+ /* If you encounter a compiler error here, see the explanation
+ * near the end of INSTALL.
+ */
+ __pngconf.h__ in libpng already includes setjmp.h;
+ __dont__ include it again.;
+# endif
+# endif /* __linux__ */
+# endif /* PNG_SKIP_SETJMP_CHECK */
+
+ /* include setjmp.h for error handling */
+# include <setjmp.h>
+
+# ifdef __linux__
+# ifdef PNG_SAVE_BSD_SOURCE
+# ifndef _BSD_SOURCE
+# define _BSD_SOURCE
+# endif
+# undef PNG_SAVE_BSD_SOURCE
+# endif
+# endif /* __linux__ */
+#endif /* PNG_SETJMP_SUPPORTED */
+
+#ifdef BSD
+# include <strings.h>
+#else
+# include <string.h>
+#endif
+
+/* Other defines for things like memory and the like can go here. */
+#ifdef PNG_INTERNAL
+
+#include <stdlib.h>
+
+/* The functions exported by PNG_EXTERN are PNG_INTERNAL functions, which
+ * aren't usually used outside the library (as far as I know), so it is
+ * debatable if they should be exported at all. In the future, when it is
+ * possible to have run-time registry of chunk-handling functions, some of
+ * these will be made available again.
+#define PNG_EXTERN extern
+ */
+#define PNG_EXTERN
+
+/* Other defines specific to compilers can go here. Try to keep
+ * them inside an appropriate ifdef/endif pair for portability.
+ */
+
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+# ifdef MACOS
+ /* We need to check that <math.h> hasn't already been included earlier
+ * as it seems it doesn't agree with <fp.h>, yet we should really use
+ * <fp.h> if possible.
+ */
+# if !defined(__MATH_H__) && !defined(__MATH_H) && !defined(__cmath__)
+# include <fp.h>
+# endif
+# else
+# include <math.h>
+# endif
+# if defined(_AMIGA) && defined(__SASC) && defined(_M68881)
+ /* Amiga SAS/C: We must include builtin FPU functions when compiling using
+ * MATH=68881
+ */
+# include <m68881.h>
+# endif
+#endif
+
+/* Codewarrior on NT has linking problems without this. */
+#if (defined(__MWERKS__) && defined(WIN32)) || defined(__STDC__)
+# define PNG_ALWAYS_EXTERN
+#endif
+
+/* This provides the non-ANSI (far) memory allocation routines. */
+#if defined(__TURBOC__) && defined(__MSDOS__)
+# include <mem.h>
+# include <alloc.h>
+#endif
+
+/* I have no idea why is this necessary... */
+#if defined(_MSC_VER) && (defined(WIN32) || defined(_Windows) || \
+ defined(_WINDOWS) || defined(_WIN32) || defined(__WIN32__))
+# include <malloc.h>
+#endif
+
+/* This controls how fine the dithering gets. As this allocates
+ * a largish chunk of memory (32K), those who are not as concerned
+ * with dithering quality can decrease some or all of these.
+ */
+#ifndef PNG_DITHER_RED_BITS
+# define PNG_DITHER_RED_BITS 5
+#endif
+#ifndef PNG_DITHER_GREEN_BITS
+# define PNG_DITHER_GREEN_BITS 5
+#endif
+#ifndef PNG_DITHER_BLUE_BITS
+# define PNG_DITHER_BLUE_BITS 5
+#endif
+
+/* This controls how fine the gamma correction becomes when you
+ * are only interested in 8 bits anyway. Increasing this value
+ * results in more memory being used, and more pow() functions
+ * being called to fill in the gamma tables. Don't set this value
+ * less then 8, and even that may not work (I haven't tested it).
+ */
+
+#ifndef PNG_MAX_GAMMA_8
+# define PNG_MAX_GAMMA_8 11
+#endif
+
+/* This controls how much a difference in gamma we can tolerate before
+ * we actually start doing gamma conversion.
+ */
+#ifndef PNG_GAMMA_THRESHOLD
+# define PNG_GAMMA_THRESHOLD 0.05
+#endif
+
+#endif /* PNG_INTERNAL */
+
+/* The following uses const char * instead of char * for error
+ * and warning message functions, so some compilers won't complain.
+ * If you do not want to use const, define PNG_NO_CONST here.
+ */
+
+#ifndef PNG_NO_CONST
+# define PNG_CONST const
+#else
+# define PNG_CONST
+#endif
+
+/* The following defines give you the ability to remove code from the
+ * library that you will not be using. I wish I could figure out how to
+ * automate this, but I can't do that without making it seriously hard
+ * on the users. So if you are not using an ability, change the #define
+ * to and #undef, and that part of the library will not be compiled. If
+ * your linker can't find a function, you may want to make sure the
+ * ability is defined here. Some of these depend upon some others being
+ * defined. I haven't figured out all the interactions here, so you may
+ * have to experiment awhile to get everything to compile. If you are
+ * creating or using a shared library, you probably shouldn't touch this,
+ * as it will affect the size of the structures, and this will cause bad
+ * things to happen if the library and/or application ever change.
+ */
+
+/* Any features you will not be using can be undef'ed here */
+
+/* GR-P, 0.96a: Set "*TRANSFORMS_SUPPORTED as default but allow user
+ * to turn it off with "*TRANSFORMS_NOT_SUPPORTED" or *PNG_NO_*_TRANSFORMS
+ * on the compile line, then pick and choose which ones to define without
+ * having to edit this file. It is safe to use the *TRANSFORMS_NOT_SUPPORTED
+ * if you only want to have a png-compliant reader/writer but don't need
+ * any of the extra transformations. This saves about 80 kbytes in a
+ * typical installation of the library. (PNG_NO_* form added in version
+ * 1.0.1c, for consistency)
+ */
+
+/* The size of the png_text structure changed in libpng-1.0.6 when
+ * iTXt support was added. iTXt support was turned off by default through
+ * libpng-1.2.x, to support old apps that malloc the png_text structure
+ * instead of calling png_set_text() and letting libpng malloc it. It
+ * will be turned on by default in libpng-1.4.0.
+ */
+
+#if defined(PNG_1_0_X) || defined (PNG_1_2_X)
+# ifndef PNG_NO_iTXt_SUPPORTED
+# define PNG_NO_iTXt_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_iTXt
+# define PNG_NO_READ_iTXt
+# endif
+# ifndef PNG_NO_WRITE_iTXt
+# define PNG_NO_WRITE_iTXt
+# endif
+#endif
+
+#if !defined(PNG_NO_iTXt_SUPPORTED)
+# if !defined(PNG_READ_iTXt_SUPPORTED) && !defined(PNG_NO_READ_iTXt)
+# define PNG_READ_iTXt
+# endif
+# if !defined(PNG_WRITE_iTXt_SUPPORTED) && !defined(PNG_NO_WRITE_iTXt)
+# define PNG_WRITE_iTXt
+# endif
+#endif
+
+/* The following support, added after version 1.0.0, can be turned off here en
+ * masse by defining PNG_LEGACY_SUPPORTED in case you need binary compatibility
+ * with old applications that require the length of png_struct and png_info
+ * to remain unchanged.
+ */
+
+#ifdef PNG_LEGACY_SUPPORTED
+# define PNG_NO_FREE_ME
+# define PNG_NO_READ_UNKNOWN_CHUNKS
+# define PNG_NO_WRITE_UNKNOWN_CHUNKS
+# define PNG_NO_HANDLE_AS_UNKNOWN
+# define PNG_NO_READ_USER_CHUNKS
+# define PNG_NO_READ_iCCP
+# define PNG_NO_WRITE_iCCP
+# define PNG_NO_READ_iTXt
+# define PNG_NO_WRITE_iTXt
+# define PNG_NO_READ_sCAL
+# define PNG_NO_WRITE_sCAL
+# define PNG_NO_READ_sPLT
+# define PNG_NO_WRITE_sPLT
+# define PNG_NO_INFO_IMAGE
+# define PNG_NO_READ_RGB_TO_GRAY
+# define PNG_NO_READ_USER_TRANSFORM
+# define PNG_NO_WRITE_USER_TRANSFORM
+# define PNG_NO_USER_MEM
+# define PNG_NO_READ_EMPTY_PLTE
+# define PNG_NO_MNG_FEATURES
+# define PNG_NO_FIXED_POINT_SUPPORTED
+#endif
+
+/* Ignore attempt to turn off both floating and fixed point support */
+#if !defined(PNG_FLOATING_POINT_SUPPORTED) || \
+ !defined(PNG_NO_FIXED_POINT_SUPPORTED)
+# define PNG_FIXED_POINT_SUPPORTED
+#endif
+
+#ifndef PNG_NO_FREE_ME
+# define PNG_FREE_ME_SUPPORTED
+#endif
+
+#ifdef PNG_READ_SUPPORTED
+
+#if !defined(PNG_READ_TRANSFORMS_NOT_SUPPORTED) && \
+ !defined(PNG_NO_READ_TRANSFORMS)
+# define PNG_READ_TRANSFORMS_SUPPORTED
+#endif
+
+#ifdef PNG_READ_TRANSFORMS_SUPPORTED
+# ifndef PNG_NO_READ_EXPAND
+# define PNG_READ_EXPAND_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_SHIFT
+# define PNG_READ_SHIFT_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_PACK
+# define PNG_READ_PACK_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_BGR
+# define PNG_READ_BGR_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_SWAP
+# define PNG_READ_SWAP_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_PACKSWAP
+# define PNG_READ_PACKSWAP_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_INVERT
+# define PNG_READ_INVERT_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_DITHER
+# define PNG_READ_DITHER_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_BACKGROUND
+# define PNG_READ_BACKGROUND_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_16_TO_8
+# define PNG_READ_16_TO_8_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_FILLER
+# define PNG_READ_FILLER_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_GAMMA
+# define PNG_READ_GAMMA_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_GRAY_TO_RGB
+# define PNG_READ_GRAY_TO_RGB_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_SWAP_ALPHA
+# define PNG_READ_SWAP_ALPHA_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_INVERT_ALPHA
+# define PNG_READ_INVERT_ALPHA_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_STRIP_ALPHA
+# define PNG_READ_STRIP_ALPHA_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_USER_TRANSFORM
+# define PNG_READ_USER_TRANSFORM_SUPPORTED
+# endif
+# ifndef PNG_NO_READ_RGB_TO_GRAY
+# define PNG_READ_RGB_TO_GRAY_SUPPORTED
+# endif
+#endif /* PNG_READ_TRANSFORMS_SUPPORTED */
+
+/* PNG_PROGRESSIVE_READ_NOT_SUPPORTED is deprecated. */
+#if !defined(PNG_NO_PROGRESSIVE_READ) && \
+ !defined(PNG_PROGRESSIVE_READ_NOT_SUPPORTED) /* if you don't do progressive */
+# define PNG_PROGRESSIVE_READ_SUPPORTED /* reading. This is not talking */
+#endif /* about interlacing capability! You'll */
+ /* still have interlacing unless you change the following define: */
+#define PNG_READ_INTERLACING_SUPPORTED /* required for PNG-compliant decoders */
+
+/* PNG_NO_SEQUENTIAL_READ_SUPPORTED is deprecated. */
+#if !defined(PNG_NO_SEQUENTIAL_READ) && \
+ !defined(PNG_SEQUENTIAL_READ_SUPPORTED) && \
+ !defined(PNG_NO_SEQUENTIAL_READ_SUPPORTED)
+# define PNG_SEQUENTIAL_READ_SUPPORTED
+#endif
+
+#define PNG_READ_INTERLACING_SUPPORTED /* required in PNG-compliant decoders */
+
+#ifndef PNG_NO_READ_COMPOSITE_NODIV
+# ifndef PNG_NO_READ_COMPOSITED_NODIV /* libpng-1.0.x misspelling */
+# define PNG_READ_COMPOSITE_NODIV_SUPPORTED /* well tested on Intel, SGI */
+# endif
+#endif
+
+#if defined(PNG_1_0_X) || defined (PNG_1_2_X)
+/* Deprecated, will be removed from version 2.0.0.
+ Use PNG_MNG_FEATURES_SUPPORTED instead. */
+#ifndef PNG_NO_READ_EMPTY_PLTE
+# define PNG_READ_EMPTY_PLTE_SUPPORTED
+#endif
+#endif
+
+#endif /* PNG_READ_SUPPORTED */
+
+#ifdef PNG_WRITE_SUPPORTED
+
+# if !defined(PNG_WRITE_TRANSFORMS_NOT_SUPPORTED) && \
+ !defined(PNG_NO_WRITE_TRANSFORMS)
+# define PNG_WRITE_TRANSFORMS_SUPPORTED
+#endif
+
+#ifdef PNG_WRITE_TRANSFORMS_SUPPORTED
+# ifndef PNG_NO_WRITE_SHIFT
+# define PNG_WRITE_SHIFT_SUPPORTED
+# endif
+# ifndef PNG_NO_WRITE_PACK
+# define PNG_WRITE_PACK_SUPPORTED
+# endif
+# ifndef PNG_NO_WRITE_BGR
+# define PNG_WRITE_BGR_SUPPORTED
+# endif
+# ifndef PNG_NO_WRITE_SWAP
+# define PNG_WRITE_SWAP_SUPPORTED
+# endif
+# ifndef PNG_NO_WRITE_PACKSWAP
+# define PNG_WRITE_PACKSWAP_SUPPORTED
+# endif
+# ifndef PNG_NO_WRITE_INVERT
+# define PNG_WRITE_INVERT_SUPPORTED
+# endif
+# ifndef PNG_NO_WRITE_FILLER
+# define PNG_WRITE_FILLER_SUPPORTED /* same as WRITE_STRIP_ALPHA */
+# endif
+# ifndef PNG_NO_WRITE_SWAP_ALPHA
+# define PNG_WRITE_SWAP_ALPHA_SUPPORTED
+# endif
+#ifndef PNG_1_0_X
+# ifndef PNG_NO_WRITE_INVERT_ALPHA
+# define PNG_WRITE_INVERT_ALPHA_SUPPORTED
+# endif
+#endif
+# ifndef PNG_NO_WRITE_USER_TRANSFORM
+# define PNG_WRITE_USER_TRANSFORM_SUPPORTED
+# endif
+#endif /* PNG_WRITE_TRANSFORMS_SUPPORTED */
+
+#if !defined(PNG_NO_WRITE_INTERLACING_SUPPORTED) && \
+ !defined(PNG_WRITE_INTERLACING_SUPPORTED)
+#define PNG_WRITE_INTERLACING_SUPPORTED /* not required for PNG-compliant
+ encoders, but can cause trouble
+ if left undefined */
+#endif
+
+#if !defined(PNG_NO_WRITE_WEIGHTED_FILTER) && \
+ !defined(PNG_WRITE_WEIGHTED_FILTER) && \
+ defined(PNG_FLOATING_POINT_SUPPORTED)
+# define PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+#endif
+
+#ifndef PNG_NO_WRITE_FLUSH
+# define PNG_WRITE_FLUSH_SUPPORTED
+#endif
+
+#if defined(PNG_1_0_X) || defined (PNG_1_2_X)
+/* Deprecated, see PNG_MNG_FEATURES_SUPPORTED, above */
+#ifndef PNG_NO_WRITE_EMPTY_PLTE
+# define PNG_WRITE_EMPTY_PLTE_SUPPORTED
+#endif
+#endif
+
+#endif /* PNG_WRITE_SUPPORTED */
+
+#ifndef PNG_1_0_X
+# ifndef PNG_NO_ERROR_NUMBERS
+# define PNG_ERROR_NUMBERS_SUPPORTED
+# endif
+#endif /* PNG_1_0_X */
+
+#if defined(PNG_READ_USER_TRANSFORM_SUPPORTED) || \
+ defined(PNG_WRITE_USER_TRANSFORM_SUPPORTED)
+# ifndef PNG_NO_USER_TRANSFORM_PTR
+# define PNG_USER_TRANSFORM_PTR_SUPPORTED
+# endif
+#endif
+
+#ifndef PNG_NO_STDIO
+# define PNG_TIME_RFC1123_SUPPORTED
+#endif
+
+/* This adds extra functions in pngget.c for accessing data from the
+ * info pointer (added in version 0.99)
+ * png_get_image_width()
+ * png_get_image_height()
+ * png_get_bit_depth()
+ * png_get_color_type()
+ * png_get_compression_type()
+ * png_get_filter_type()
+ * png_get_interlace_type()
+ * png_get_pixel_aspect_ratio()
+ * png_get_pixels_per_meter()
+ * png_get_x_offset_pixels()
+ * png_get_y_offset_pixels()
+ * png_get_x_offset_microns()
+ * png_get_y_offset_microns()
+ */
+#if !defined(PNG_NO_EASY_ACCESS) && !defined(PNG_EASY_ACCESS_SUPPORTED)
+# define PNG_EASY_ACCESS_SUPPORTED
+#endif
+
+/* PNG_ASSEMBLER_CODE was enabled by default in version 1.2.0
+ * and removed from version 1.2.20. The following will be removed
+ * from libpng-1.4.0
+*/
+
+#if defined(PNG_READ_SUPPORTED) && !defined(PNG_NO_OPTIMIZED_CODE)
+# ifndef PNG_OPTIMIZED_CODE_SUPPORTED
+# define PNG_OPTIMIZED_CODE_SUPPORTED
+# endif
+#endif
+
+#if defined(PNG_READ_SUPPORTED) && !defined(PNG_NO_ASSEMBLER_CODE)
+# ifndef PNG_ASSEMBLER_CODE_SUPPORTED
+# define PNG_ASSEMBLER_CODE_SUPPORTED
+# endif
+
+# if defined(__GNUC__) && defined(__x86_64__) && (__GNUC__ < 4)
+ /* work around 64-bit gcc compiler bugs in gcc-3.x */
+# if !defined(PNG_MMX_CODE_SUPPORTED) && !defined(PNG_NO_MMX_CODE)
+# define PNG_NO_MMX_CODE
+# endif
+# endif
+
+# ifdef __APPLE__
+# if !defined(PNG_MMX_CODE_SUPPORTED) && !defined(PNG_NO_MMX_CODE)
+# define PNG_NO_MMX_CODE
+# endif
+# endif
+
+# if (defined(__MWERKS__) && ((__MWERKS__ < 0x0900) || macintosh))
+# if !defined(PNG_MMX_CODE_SUPPORTED) && !defined(PNG_NO_MMX_CODE)
+# define PNG_NO_MMX_CODE
+# endif
+# endif
+
+# if !defined(PNG_MMX_CODE_SUPPORTED) && !defined(PNG_NO_MMX_CODE)
+# define PNG_MMX_CODE_SUPPORTED
+# endif
+
+#endif
+/* end of obsolete code to be removed from libpng-1.4.0 */
+
+/* Added at libpng-1.2.0 */
+#ifndef PNG_1_0_X
+#if !defined(PNG_NO_USER_MEM) && !defined(PNG_USER_MEM_SUPPORTED)
+# define PNG_USER_MEM_SUPPORTED
+#endif
+#endif /* PNG_1_0_X */
+
+/* Added at libpng-1.2.6 */
+#ifndef PNG_1_0_X
+# ifndef PNG_SET_USER_LIMITS_SUPPORTED
+# ifndef PNG_NO_SET_USER_LIMITS
+# define PNG_SET_USER_LIMITS_SUPPORTED
+# endif
+# endif
+#endif /* PNG_1_0_X */
+
+/* Added at libpng-1.0.53 and 1.2.43 */
+#ifndef PNG_USER_LIMITS_SUPPORTED
+# ifndef PNG_NO_USER_LIMITS
+# define PNG_USER_LIMITS_SUPPORTED
+# endif
+#endif
+
+/* Added at libpng-1.0.16 and 1.2.6. To accept all valid PNGS no matter
+ * how large, set these limits to 0x7fffffffL
+ */
+#ifndef PNG_USER_WIDTH_MAX
+# define PNG_USER_WIDTH_MAX 1000000L
+#endif
+#ifndef PNG_USER_HEIGHT_MAX
+# define PNG_USER_HEIGHT_MAX 1000000L
+#endif
+
+/* Added at libpng-1.2.43. To accept all valid PNGs no matter
+ * how large, set these two limits to 0.
+ */
+#ifndef PNG_USER_CHUNK_CACHE_MAX
+# define PNG_USER_CHUNK_CACHE_MAX 0
+#endif
+
+/* Added at libpng-1.2.43 */
+#ifndef PNG_USER_CHUNK_MALLOC_MAX
+# define PNG_USER_CHUNK_MALLOC_MAX 0
+#endif
+
+#ifndef PNG_LITERAL_SHARP
+# define PNG_LITERAL_SHARP 0x23
+#endif
+#ifndef PNG_LITERAL_LEFT_SQUARE_BRACKET
+# define PNG_LITERAL_LEFT_SQUARE_BRACKET 0x5b
+#endif
+#ifndef PNG_LITERAL_RIGHT_SQUARE_BRACKET
+# define PNG_LITERAL_RIGHT_SQUARE_BRACKET 0x5d
+#endif
+
+/* Added at libpng-1.2.34 */
+#ifndef PNG_STRING_NEWLINE
+#define PNG_STRING_NEWLINE "\n"
+#endif
+
+/* These are currently experimental features, define them if you want */
+
+/* very little testing */
+/*
+#ifdef PNG_READ_SUPPORTED
+# ifndef PNG_READ_16_TO_8_ACCURATE_SCALE_SUPPORTED
+# define PNG_READ_16_TO_8_ACCURATE_SCALE_SUPPORTED
+# endif
+#endif
+*/
+
+/* This is only for PowerPC big-endian and 680x0 systems */
+/* some testing */
+/*
+#ifndef PNG_READ_BIG_ENDIAN_SUPPORTED
+# define PNG_READ_BIG_ENDIAN_SUPPORTED
+#endif
+*/
+
+/* Buggy compilers (e.g., gcc 2.7.2.2) need this */
+/*
+#define PNG_NO_POINTER_INDEXING
+*/
+
+#if !defined(PNG_NO_POINTER_INDEXING) && \
+ !defined(PNG_POINTER_INDEXING_SUPPORTED)
+# define PNG_POINTER_INDEXING_SUPPORTED
+#endif
+
+/* These functions are turned off by default, as they will be phased out. */
+/*
+#define PNG_USELESS_TESTS_SUPPORTED
+#define PNG_CORRECT_PALETTE_SUPPORTED
+*/
+
+/* Any chunks you are not interested in, you can undef here. The
+ * ones that allocate memory may be expecially important (hIST,
+ * tEXt, zTXt, tRNS, pCAL). Others will just save time and make png_info
+ * a bit smaller.
+ */
+
+#if defined(PNG_READ_SUPPORTED) && \
+ !defined(PNG_READ_ANCILLARY_CHUNKS_NOT_SUPPORTED) && \
+ !defined(PNG_NO_READ_ANCILLARY_CHUNKS)
+# define PNG_READ_ANCILLARY_CHUNKS_SUPPORTED
+#endif
+
+#if defined(PNG_WRITE_SUPPORTED) && \
+ !defined(PNG_WRITE_ANCILLARY_CHUNKS_NOT_SUPPORTED) && \
+ !defined(PNG_NO_WRITE_ANCILLARY_CHUNKS)
+# define PNG_WRITE_ANCILLARY_CHUNKS_SUPPORTED
+#endif
+
+#ifdef PNG_READ_ANCILLARY_CHUNKS_SUPPORTED
+
+#ifdef PNG_NO_READ_TEXT
+# define PNG_NO_READ_iTXt
+# define PNG_NO_READ_tEXt
+# define PNG_NO_READ_zTXt
+#endif
+#ifndef PNG_NO_READ_bKGD
+# define PNG_READ_bKGD_SUPPORTED
+# define PNG_bKGD_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_cHRM
+# define PNG_READ_cHRM_SUPPORTED
+# define PNG_cHRM_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_gAMA
+# define PNG_READ_gAMA_SUPPORTED
+# define PNG_gAMA_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_hIST
+# define PNG_READ_hIST_SUPPORTED
+# define PNG_hIST_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_iCCP
+# define PNG_READ_iCCP_SUPPORTED
+# define PNG_iCCP_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_iTXt
+# ifndef PNG_READ_iTXt_SUPPORTED
+# define PNG_READ_iTXt_SUPPORTED
+# endif
+# ifndef PNG_iTXt_SUPPORTED
+# define PNG_iTXt_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_READ_oFFs
+# define PNG_READ_oFFs_SUPPORTED
+# define PNG_oFFs_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_pCAL
+# define PNG_READ_pCAL_SUPPORTED
+# define PNG_pCAL_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_sCAL
+# define PNG_READ_sCAL_SUPPORTED
+# define PNG_sCAL_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_pHYs
+# define PNG_READ_pHYs_SUPPORTED
+# define PNG_pHYs_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_sBIT
+# define PNG_READ_sBIT_SUPPORTED
+# define PNG_sBIT_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_sPLT
+# define PNG_READ_sPLT_SUPPORTED
+# define PNG_sPLT_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_sRGB
+# define PNG_READ_sRGB_SUPPORTED
+# define PNG_sRGB_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_tEXt
+# define PNG_READ_tEXt_SUPPORTED
+# define PNG_tEXt_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_tIME
+# define PNG_READ_tIME_SUPPORTED
+# define PNG_tIME_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_tRNS
+# define PNG_READ_tRNS_SUPPORTED
+# define PNG_tRNS_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_zTXt
+# define PNG_READ_zTXt_SUPPORTED
+# define PNG_zTXt_SUPPORTED
+#endif
+#ifndef PNG_NO_READ_OPT_PLTE
+# define PNG_READ_OPT_PLTE_SUPPORTED /* only affects support of the */
+#endif /* optional PLTE chunk in RGB and RGBA images */
+#if defined(PNG_READ_iTXt_SUPPORTED) || defined(PNG_READ_tEXt_SUPPORTED) || \
+ defined(PNG_READ_zTXt_SUPPORTED)
+# define PNG_READ_TEXT_SUPPORTED
+# define PNG_TEXT_SUPPORTED
+#endif
+
+#endif /* PNG_READ_ANCILLARY_CHUNKS_SUPPORTED */
+
+#ifndef PNG_NO_READ_UNKNOWN_CHUNKS
+# define PNG_READ_UNKNOWN_CHUNKS_SUPPORTED
+# ifndef PNG_UNKNOWN_CHUNKS_SUPPORTED
+# define PNG_UNKNOWN_CHUNKS_SUPPORTED
+# endif
+#endif
+#if !defined(PNG_NO_READ_USER_CHUNKS) && \
+ defined(PNG_READ_UNKNOWN_CHUNKS_SUPPORTED)
+# define PNG_READ_USER_CHUNKS_SUPPORTED
+# define PNG_USER_CHUNKS_SUPPORTED
+# ifdef PNG_NO_READ_UNKNOWN_CHUNKS
+# undef PNG_NO_READ_UNKNOWN_CHUNKS
+# endif
+# ifdef PNG_NO_HANDLE_AS_UNKNOWN
+# undef PNG_NO_HANDLE_AS_UNKNOWN
+# endif
+#endif
+
+#ifndef PNG_NO_HANDLE_AS_UNKNOWN
+# ifndef PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+# define PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+# endif
+#endif
+
+#ifdef PNG_WRITE_SUPPORTED
+#ifdef PNG_WRITE_ANCILLARY_CHUNKS_SUPPORTED
+
+#ifdef PNG_NO_WRITE_TEXT
+# define PNG_NO_WRITE_iTXt
+# define PNG_NO_WRITE_tEXt
+# define PNG_NO_WRITE_zTXt
+#endif
+#ifndef PNG_NO_WRITE_bKGD
+# define PNG_WRITE_bKGD_SUPPORTED
+# ifndef PNG_bKGD_SUPPORTED
+# define PNG_bKGD_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_cHRM
+# define PNG_WRITE_cHRM_SUPPORTED
+# ifndef PNG_cHRM_SUPPORTED
+# define PNG_cHRM_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_gAMA
+# define PNG_WRITE_gAMA_SUPPORTED
+# ifndef PNG_gAMA_SUPPORTED
+# define PNG_gAMA_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_hIST
+# define PNG_WRITE_hIST_SUPPORTED
+# ifndef PNG_hIST_SUPPORTED
+# define PNG_hIST_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_iCCP
+# define PNG_WRITE_iCCP_SUPPORTED
+# ifndef PNG_iCCP_SUPPORTED
+# define PNG_iCCP_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_iTXt
+# ifndef PNG_WRITE_iTXt_SUPPORTED
+# define PNG_WRITE_iTXt_SUPPORTED
+# endif
+# ifndef PNG_iTXt_SUPPORTED
+# define PNG_iTXt_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_oFFs
+# define PNG_WRITE_oFFs_SUPPORTED
+# ifndef PNG_oFFs_SUPPORTED
+# define PNG_oFFs_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_pCAL
+# define PNG_WRITE_pCAL_SUPPORTED
+# ifndef PNG_pCAL_SUPPORTED
+# define PNG_pCAL_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_sCAL
+# define PNG_WRITE_sCAL_SUPPORTED
+# ifndef PNG_sCAL_SUPPORTED
+# define PNG_sCAL_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_pHYs
+# define PNG_WRITE_pHYs_SUPPORTED
+# ifndef PNG_pHYs_SUPPORTED
+# define PNG_pHYs_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_sBIT
+# define PNG_WRITE_sBIT_SUPPORTED
+# ifndef PNG_sBIT_SUPPORTED
+# define PNG_sBIT_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_sPLT
+# define PNG_WRITE_sPLT_SUPPORTED
+# ifndef PNG_sPLT_SUPPORTED
+# define PNG_sPLT_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_sRGB
+# define PNG_WRITE_sRGB_SUPPORTED
+# ifndef PNG_sRGB_SUPPORTED
+# define PNG_sRGB_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_tEXt
+# define PNG_WRITE_tEXt_SUPPORTED
+# ifndef PNG_tEXt_SUPPORTED
+# define PNG_tEXt_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_tIME
+# define PNG_WRITE_tIME_SUPPORTED
+# ifndef PNG_tIME_SUPPORTED
+# define PNG_tIME_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_tRNS
+# define PNG_WRITE_tRNS_SUPPORTED
+# ifndef PNG_tRNS_SUPPORTED
+# define PNG_tRNS_SUPPORTED
+# endif
+#endif
+#ifndef PNG_NO_WRITE_zTXt
+# define PNG_WRITE_zTXt_SUPPORTED
+# ifndef PNG_zTXt_SUPPORTED
+# define PNG_zTXt_SUPPORTED
+# endif
+#endif
+#if defined(PNG_WRITE_iTXt_SUPPORTED) || defined(PNG_WRITE_tEXt_SUPPORTED) || \
+ defined(PNG_WRITE_zTXt_SUPPORTED)
+# define PNG_WRITE_TEXT_SUPPORTED
+# ifndef PNG_TEXT_SUPPORTED
+# define PNG_TEXT_SUPPORTED
+# endif
+#endif
+
+#ifdef PNG_WRITE_tIME_SUPPORTED
+# ifndef PNG_NO_CONVERT_tIME
+# ifndef _WIN32_WCE
+/* The "tm" structure is not supported on WindowsCE */
+# ifndef PNG_CONVERT_tIME_SUPPORTED
+# define PNG_CONVERT_tIME_SUPPORTED
+# endif
+# endif
+# endif
+#endif
+
+#endif /* PNG_WRITE_ANCILLARY_CHUNKS_SUPPORTED */
+
+#if !defined(PNG_NO_WRITE_FILTER) && !defined(PNG_WRITE_FILTER_SUPPORTED)
+# define PNG_WRITE_FILTER_SUPPORTED
+#endif
+
+#ifndef PNG_NO_WRITE_UNKNOWN_CHUNKS
+# define PNG_WRITE_UNKNOWN_CHUNKS_SUPPORTED
+# ifndef PNG_UNKNOWN_CHUNKS_SUPPORTED
+# define PNG_UNKNOWN_CHUNKS_SUPPORTED
+# endif
+#endif
+
+#ifndef PNG_NO_HANDLE_AS_UNKNOWN
+# ifndef PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+# define PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+# endif
+#endif
+#endif /* PNG_WRITE_SUPPORTED */
+
+/* Turn this off to disable png_read_png() and
+ * png_write_png() and leave the row_pointers member
+ * out of the info structure.
+ */
+#ifndef PNG_NO_INFO_IMAGE
+# define PNG_INFO_IMAGE_SUPPORTED
+#endif
+
+/* Need the time information for converting tIME chunks */
+#ifdef PNG_CONVERT_tIME_SUPPORTED
+ /* "time.h" functions are not supported on WindowsCE */
+# include <time.h>
+#endif
+
+/* Some typedefs to get us started. These should be safe on most of the
+ * common platforms. The typedefs should be at least as large as the
+ * numbers suggest (a png_uint_32 must be at least 32 bits long), but they
+ * don't have to be exactly that size. Some compilers dislike passing
+ * unsigned shorts as function parameters, so you may be better off using
+ * unsigned int for png_uint_16. Likewise, for 64-bit systems, you may
+ * want to have unsigned int for png_uint_32 instead of unsigned long.
+ */
+
+typedef unsigned long png_uint_32;
+typedef long png_int_32;
+typedef unsigned short png_uint_16;
+typedef short png_int_16;
+typedef unsigned char png_byte;
+
+/* This is usually size_t. It is typedef'ed just in case you need it to
+ change (I'm not sure if you will or not, so I thought I'd be safe) */
+#ifdef PNG_SIZE_T
+ typedef PNG_SIZE_T png_size_t;
+# define png_sizeof(x) png_convert_size(sizeof(x))
+#else
+ typedef size_t png_size_t;
+# define png_sizeof(x) sizeof(x)
+#endif
+
+/* The following is needed for medium model support. It cannot be in the
+ * PNG_INTERNAL section. Needs modification for other compilers besides
+ * MSC. Model independent support declares all arrays and pointers to be
+ * large using the far keyword. The zlib version used must also support
+ * model independent data. As of version zlib 1.0.4, the necessary changes
+ * have been made in zlib. The USE_FAR_KEYWORD define triggers other
+ * changes that are needed. (Tim Wegner)
+ */
+
+/* Separate compiler dependencies (problem here is that zlib.h always
+ defines FAR. (SJT) */
+#ifdef __BORLANDC__
+# if defined(__LARGE__) || defined(__HUGE__) || defined(__COMPACT__)
+# define LDATA 1
+# else
+# define LDATA 0
+# endif
+ /* GRR: why is Cygwin in here? Cygwin is not Borland C... */
+# if !defined(__WIN32__) && !defined(__FLAT__) && !defined(__CYGWIN__)
+# define PNG_MAX_MALLOC_64K
+# if (LDATA != 1)
+# ifndef FAR
+# define FAR __far
+# endif
+# define USE_FAR_KEYWORD
+# endif /* LDATA != 1 */
+ /* Possibly useful for moving data out of default segment.
+ * Uncomment it if you want. Could also define FARDATA as
+ * const if your compiler supports it. (SJT)
+# define FARDATA FAR
+ */
+# endif /* __WIN32__, __FLAT__, __CYGWIN__ */
+#endif /* __BORLANDC__ */
+
+
+/* Suggest testing for specific compiler first before testing for
+ * FAR. The Watcom compiler defines both __MEDIUM__ and M_I86MM,
+ * making reliance oncertain keywords suspect. (SJT)
+ */
+
+/* MSC Medium model */
+#ifdef FAR
+# ifdef M_I86MM
+# define USE_FAR_KEYWORD
+# define FARDATA FAR
+# include <dos.h>
+# endif
+#endif
+
+/* SJT: default case */
+#ifndef FAR
+# define FAR
+#endif
+
+/* At this point FAR is always defined */
+#ifndef FARDATA
+# define FARDATA
+#endif
+
+/* Typedef for floating-point numbers that are converted
+ to fixed-point with a multiple of 100,000, e.g., int_gamma */
+typedef png_int_32 png_fixed_point;
+
+/* Add typedefs for pointers */
+typedef void FAR * png_voidp;
+typedef png_byte FAR * png_bytep;
+typedef png_uint_32 FAR * png_uint_32p;
+typedef png_int_32 FAR * png_int_32p;
+typedef png_uint_16 FAR * png_uint_16p;
+typedef png_int_16 FAR * png_int_16p;
+typedef PNG_CONST char FAR * png_const_charp;
+typedef char FAR * png_charp;
+typedef png_fixed_point FAR * png_fixed_point_p;
+
+#ifndef PNG_NO_STDIO
+#ifdef _WIN32_WCE
+typedef HANDLE png_FILE_p;
+#else
+typedef FILE * png_FILE_p;
+#endif
+#endif
+
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+typedef double FAR * png_doublep;
+#endif
+
+/* Pointers to pointers; i.e. arrays */
+typedef png_byte FAR * FAR * png_bytepp;
+typedef png_uint_32 FAR * FAR * png_uint_32pp;
+typedef png_int_32 FAR * FAR * png_int_32pp;
+typedef png_uint_16 FAR * FAR * png_uint_16pp;
+typedef png_int_16 FAR * FAR * png_int_16pp;
+typedef PNG_CONST char FAR * FAR * png_const_charpp;
+typedef char FAR * FAR * png_charpp;
+typedef png_fixed_point FAR * FAR * png_fixed_point_pp;
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+typedef double FAR * FAR * png_doublepp;
+#endif
+
+/* Pointers to pointers to pointers; i.e., pointer to array */
+typedef char FAR * FAR * FAR * png_charppp;
+
+#if defined(PNG_1_0_X) || defined(PNG_1_2_X)
+/* SPC - Is this stuff deprecated? */
+/* It'll be removed as of libpng-1.4.0 - GR-P */
+/* libpng typedefs for types in zlib. If zlib changes
+ * or another compression library is used, then change these.
+ * Eliminates need to change all the source files.
+ */
+typedef charf * png_zcharp;
+typedef charf * FAR * png_zcharpp;
+typedef z_stream FAR * png_zstreamp;
+#endif /* (PNG_1_0_X) || defined(PNG_1_2_X) */
+
+/*
+ * Define PNG_BUILD_DLL if the module being built is a Windows
+ * LIBPNG DLL.
+ *
+ * Define PNG_USE_DLL if you want to *link* to the Windows LIBPNG DLL.
+ * It is equivalent to Microsoft predefined macro _DLL that is
+ * automatically defined when you compile using the share
+ * version of the CRT (C Run-Time library)
+ *
+ * The cygwin mods make this behavior a little different:
+ * Define PNG_BUILD_DLL if you are building a dll for use with cygwin
+ * Define PNG_STATIC if you are building a static library for use with cygwin,
+ * -or- if you are building an application that you want to link to the
+ * static library.
+ * PNG_USE_DLL is defined by default (no user action needed) unless one of
+ * the other flags is defined.
+ */
+
+#if !defined(PNG_DLL) && (defined(PNG_BUILD_DLL) || defined(PNG_USE_DLL))
+# define PNG_DLL
+#endif
+/* If CYGWIN, then disallow GLOBAL ARRAYS unless building a static lib.
+ * When building a static lib, default to no GLOBAL ARRAYS, but allow
+ * command-line override
+ */
+#ifdef __CYGWIN__
+# ifndef PNG_STATIC
+# ifdef PNG_USE_GLOBAL_ARRAYS
+# undef PNG_USE_GLOBAL_ARRAYS
+# endif
+# ifndef PNG_USE_LOCAL_ARRAYS
+# define PNG_USE_LOCAL_ARRAYS
+# endif
+# else
+# if defined(PNG_USE_LOCAL_ARRAYS) || defined(PNG_NO_GLOBAL_ARRAYS)
+# ifdef PNG_USE_GLOBAL_ARRAYS
+# undef PNG_USE_GLOBAL_ARRAYS
+# endif
+# endif
+# endif
+# if !defined(PNG_USE_LOCAL_ARRAYS) && !defined(PNG_USE_GLOBAL_ARRAYS)
+# define PNG_USE_LOCAL_ARRAYS
+# endif
+#endif
+
+/* Do not use global arrays (helps with building DLL's)
+ * They are no longer used in libpng itself, since version 1.0.5c,
+ * but might be required for some pre-1.0.5c applications.
+ */
+#if !defined(PNG_USE_LOCAL_ARRAYS) && !defined(PNG_USE_GLOBAL_ARRAYS)
+# if defined(PNG_NO_GLOBAL_ARRAYS) || \
+ (defined(__GNUC__) && defined(PNG_DLL)) || defined(_MSC_VER)
+# define PNG_USE_LOCAL_ARRAYS
+# else
+# define PNG_USE_GLOBAL_ARRAYS
+# endif
+#endif
+
+#ifdef __CYGWIN__
+# undef PNGAPI
+# define PNGAPI __cdecl
+# undef PNG_IMPEXP
+# define PNG_IMPEXP
+#endif
+
+/* If you define PNGAPI, e.g., with compiler option "-DPNGAPI=__stdcall",
+ * you may get warnings regarding the linkage of png_zalloc and png_zfree.
+ * Don't ignore those warnings; you must also reset the default calling
+ * convention in your compiler to match your PNGAPI, and you must build
+ * zlib and your applications the same way you build libpng.
+ */
+
+#if defined(__MINGW32__) && !defined(PNG_MODULEDEF)
+# ifndef PNG_NO_MODULEDEF
+# define PNG_NO_MODULEDEF
+# endif
+#endif
+
+#if !defined(PNG_IMPEXP) && defined(PNG_BUILD_DLL) && !defined(PNG_NO_MODULEDEF)
+# define PNG_IMPEXP
+#endif
+
+#if defined(PNG_DLL) || defined(_DLL) || defined(__DLL__ ) || \
+ (( defined(_Windows) || defined(_WINDOWS) || \
+ defined(WIN32) || defined(_WIN32) || defined(__WIN32__) ))
+
+# ifndef PNGAPI
+# if defined(__GNUC__) || (defined (_MSC_VER) && (_MSC_VER >= 800))
+# define PNGAPI __cdecl
+# else
+# define PNGAPI _cdecl
+# endif
+# endif
+
+# if !defined(PNG_IMPEXP) && (!defined(PNG_DLL) || \
+ 0 /* WINCOMPILER_WITH_NO_SUPPORT_FOR_DECLIMPEXP */)
+# define PNG_IMPEXP
+# endif
+
+# ifndef PNG_IMPEXP
+
+# define PNG_EXPORT_TYPE1(type,symbol) PNG_IMPEXP type PNGAPI symbol
+# define PNG_EXPORT_TYPE2(type,symbol) type PNG_IMPEXP PNGAPI symbol
+
+ /* Borland/Microsoft */
+# if defined(_MSC_VER) || defined(__BORLANDC__)
+# if (_MSC_VER >= 800) || (__BORLANDC__ >= 0x500)
+# define PNG_EXPORT PNG_EXPORT_TYPE1
+# else
+# define PNG_EXPORT PNG_EXPORT_TYPE2
+# ifdef PNG_BUILD_DLL
+# define PNG_IMPEXP __export
+# else
+# define PNG_IMPEXP /*__import */ /* doesn't exist AFAIK in
+ VC++ */
+# endif /* Exists in Borland C++ for
+ C++ classes (== huge) */
+# endif
+# endif
+
+# ifndef PNG_IMPEXP
+# ifdef PNG_BUILD_DLL
+# define PNG_IMPEXP __declspec(dllexport)
+# else
+# define PNG_IMPEXP __declspec(dllimport)
+# endif
+# endif
+# endif /* PNG_IMPEXP */
+#else /* !(DLL || non-cygwin WINDOWS) */
+# if (defined(__IBMC__) || defined(__IBMCPP__)) && defined(__OS2__)
+# ifndef PNGAPI
+# define PNGAPI _System
+# endif
+# else
+# if 0 /* ... other platforms, with other meanings */
+# endif
+# endif
+#endif
+
+#ifndef PNGAPI
+# define PNGAPI
+#endif
+#ifndef PNG_IMPEXP
+# define PNG_IMPEXP
+#endif
+
+#ifdef PNG_BUILDSYMS
+# ifndef PNG_EXPORT
+# define PNG_EXPORT(type,symbol) PNG_FUNCTION_EXPORT symbol END
+# endif
+# ifdef PNG_USE_GLOBAL_ARRAYS
+# ifndef PNG_EXPORT_VAR
+# define PNG_EXPORT_VAR(type) PNG_DATA_EXPORT
+# endif
+# endif
+#endif
+
+#ifndef PNG_EXPORT
+# define PNG_EXPORT(type,symbol) PNG_IMPEXP type PNGAPI symbol
+#endif
+
+#ifdef PNG_USE_GLOBAL_ARRAYS
+# ifndef PNG_EXPORT_VAR
+# define PNG_EXPORT_VAR(type) extern PNG_IMPEXP type
+# endif
+#endif
+
+#ifdef PNG_PEDANTIC_WARNINGS
+# ifndef PNG_PEDANTIC_WARNINGS_SUPPORTED
+# define PNG_PEDANTIC_WARNINGS_SUPPORTED
+# endif
+#endif
+
+#ifdef PNG_PEDANTIC_WARNINGS_SUPPORTED
+/* Support for compiler specific function attributes. These are used
+ * so that where compiler support is available incorrect use of API
+ * functions in png.h will generate compiler warnings. Added at libpng
+ * version 1.2.41.
+ */
+# ifdef __GNUC__
+# ifndef PNG_USE_RESULT
+# define PNG_USE_RESULT __attribute__((__warn_unused_result__))
+# endif
+# ifndef PNG_NORETURN
+# define PNG_NORETURN __attribute__((__noreturn__))
+# endif
+# ifndef PNG_ALLOCATED
+# define PNG_ALLOCATED __attribute__((__malloc__))
+# endif
+
+ /* This specifically protects structure members that should only be
+ * accessed from within the library, therefore should be empty during
+ * a library build.
+ */
+# ifndef PNG_DEPRECATED
+# define PNG_DEPRECATED __attribute__((__deprecated__))
+# endif
+# ifndef PNG_DEPSTRUCT
+# define PNG_DEPSTRUCT __attribute__((__deprecated__))
+# endif
+# ifndef PNG_PRIVATE
+# if 0 /* Doesn't work so we use deprecated instead*/
+# define PNG_PRIVATE \
+ __attribute__((warning("This function is not exported by libpng.")))
+# else
+# define PNG_PRIVATE \
+ __attribute__((__deprecated__))
+# endif
+# endif /* PNG_PRIVATE */
+# endif /* __GNUC__ */
+#endif /* PNG_PEDANTIC_WARNINGS */
+
+#ifndef PNG_DEPRECATED
+# define PNG_DEPRECATED /* Use of this function is deprecated */
+#endif
+#ifndef PNG_USE_RESULT
+# define PNG_USE_RESULT /* The result of this function must be checked */
+#endif
+#ifndef PNG_NORETURN
+# define PNG_NORETURN /* This function does not return */
+#endif
+#ifndef PNG_ALLOCATED
+# define PNG_ALLOCATED /* The result of the function is new memory */
+#endif
+#ifndef PNG_DEPSTRUCT
+# define PNG_DEPSTRUCT /* Access to this struct member is deprecated */
+#endif
+#ifndef PNG_PRIVATE
+# define PNG_PRIVATE /* This is a private libpng function */
+#endif
+
+/* User may want to use these so they are not in PNG_INTERNAL. Any library
+ * functions that are passed far data must be model independent.
+ */
+
+#ifndef PNG_ABORT
+# define PNG_ABORT() abort()
+#endif
+
+#ifdef PNG_SETJMP_SUPPORTED
+# define png_jmpbuf(png_ptr) ((png_ptr)->jmpbuf)
+#else
+# define png_jmpbuf(png_ptr) \
+ (LIBPNG_WAS_COMPILED_WITH__PNG_SETJMP_NOT_SUPPORTED)
+#endif
+
+#ifdef USE_FAR_KEYWORD /* memory model independent fns */
+/* Use this to make far-to-near assignments */
+# define CHECK 1
+# define NOCHECK 0
+# define CVT_PTR(ptr) (png_far_to_near(png_ptr,ptr,CHECK))
+# define CVT_PTR_NOCHECK(ptr) (png_far_to_near(png_ptr,ptr,NOCHECK))
+# define png_snprintf _fsnprintf /* Added to v 1.2.19 */
+# define png_strlen _fstrlen
+# define png_memcmp _fmemcmp /* SJT: added */
+# define png_memcpy _fmemcpy
+# define png_memset _fmemset
+#else /* Use the usual functions */
+# define CVT_PTR(ptr) (ptr)
+# define CVT_PTR_NOCHECK(ptr) (ptr)
+# ifndef PNG_NO_SNPRINTF
+# ifdef _MSC_VER
+# define png_snprintf _snprintf /* Added to v 1.2.19 */
+# define png_snprintf2 _snprintf
+# define png_snprintf6 _snprintf
+# else
+# define png_snprintf snprintf /* Added to v 1.2.19 */
+# define png_snprintf2 snprintf
+# define png_snprintf6 snprintf
+# endif
+# else
+ /* You don't have or don't want to use snprintf(). Caution: Using
+ * sprintf instead of snprintf exposes your application to accidental
+ * or malevolent buffer overflows. If you don't have snprintf()
+ * as a general rule you should provide one (you can get one from
+ * Portable OpenSSH).
+ */
+# define png_snprintf(s1,n,fmt,x1) sprintf(s1,fmt,x1)
+# define png_snprintf2(s1,n,fmt,x1,x2) sprintf(s1,fmt,x1,x2)
+# define png_snprintf6(s1,n,fmt,x1,x2,x3,x4,x5,x6) \
+ sprintf(s1,fmt,x1,x2,x3,x4,x5,x6)
+# endif
+# define png_strlen strlen
+# define png_memcmp memcmp /* SJT: added */
+# define png_memcpy memcpy
+# define png_memset memset
+#endif
+/* End of memory model independent support */
+
+/* Just a little check that someone hasn't tried to define something
+ * contradictory.
+ */
+#if (PNG_ZBUF_SIZE > 65536L) && defined(PNG_MAX_MALLOC_64K)
+# undef PNG_ZBUF_SIZE
+# define PNG_ZBUF_SIZE 65536L
+#endif
+
+/* Added at libpng-1.2.8 */
+#endif /* PNG_VERSION_INFO_ONLY */
+
+#endif /* PNGCONF_H */
diff --git a/trunk/src/third_party/libpng/pngerror.c b/trunk/src/third_party/libpng/pngerror.c
new file mode 100644
index 0000000..7bc98fb
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngerror.c
@@ -0,0 +1,386 @@
+
+/* pngerror.c - stub functions for i/o and memory allocation
+ *
+ * Last changed in libpng 1.2.41 [December 3, 2009]
+ * Copyright (c) 1998-2009 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ *
+ * This file provides a location for all error handling. Users who
+ * need special error handling are expected to write replacement functions
+ * and use png_set_error_fn() to use those functions. See the instructions
+ * at each function.
+ */
+
+#define PNG_INTERNAL
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+#if defined(PNG_READ_SUPPORTED) || defined(PNG_WRITE_SUPPORTED)
+
+static void /* PRIVATE */
+png_default_error PNGARG((png_structp png_ptr,
+ png_const_charp error_message)) PNG_NORETURN;
+#ifdef PNG_WARNINGS_SUPPORTED
+static void /* PRIVATE */
+png_default_warning PNGARG((png_structp png_ptr,
+ png_const_charp warning_message));
+#endif /* PNG_WARNINGS_SUPPORTED */
+
+/* This function is called whenever there is a fatal error. This function
+ * should not be changed. If there is a need to handle errors differently,
+ * you should supply a replacement error function and use png_set_error_fn()
+ * to replace the error function at run-time.
+ */
+#ifdef PNG_ERROR_TEXT_SUPPORTED
+void PNGAPI
+png_error(png_structp png_ptr, png_const_charp error_message)
+{
+#ifdef PNG_ERROR_NUMBERS_SUPPORTED
+ char msg[16];
+ if (png_ptr != NULL)
+ {
+ if (png_ptr->flags&
+ (PNG_FLAG_STRIP_ERROR_NUMBERS|PNG_FLAG_STRIP_ERROR_TEXT))
+ {
+ if (*error_message == PNG_LITERAL_SHARP)
+ {
+ /* Strip "#nnnn " from beginning of error message. */
+ int offset;
+ for (offset = 1; offset<15; offset++)
+ if (error_message[offset] == ' ')
+ break;
+ if (png_ptr->flags&PNG_FLAG_STRIP_ERROR_TEXT)
+ {
+ int i;
+ for (i = 0; i < offset - 1; i++)
+ msg[i] = error_message[i + 1];
+ msg[i - 1] = '\0';
+ error_message = msg;
+ }
+ else
+ error_message += offset;
+ }
+ else
+ {
+ if (png_ptr->flags&PNG_FLAG_STRIP_ERROR_TEXT)
+ {
+ msg[0] = '0';
+ msg[1] = '\0';
+ error_message = msg;
+ }
+ }
+ }
+ }
+#endif
+ if (png_ptr != NULL && png_ptr->error_fn != NULL)
+ (*(png_ptr->error_fn))(png_ptr, error_message);
+
+ /* If the custom handler doesn't exist, or if it returns,
+ use the default handler, which will not return. */
+ png_default_error(png_ptr, error_message);
+}
+#else
+void PNGAPI
+png_err(png_structp png_ptr)
+{
+ if (png_ptr != NULL && png_ptr->error_fn != NULL)
+ (*(png_ptr->error_fn))(png_ptr, '\0');
+
+ /* If the custom handler doesn't exist, or if it returns,
+ use the default handler, which will not return. */
+ png_default_error(png_ptr, '\0');
+}
+#endif /* PNG_ERROR_TEXT_SUPPORTED */
+
+#ifdef PNG_WARNINGS_SUPPORTED
+/* This function is called whenever there is a non-fatal error. This function
+ * should not be changed. If there is a need to handle warnings differently,
+ * you should supply a replacement warning function and use
+ * png_set_error_fn() to replace the warning function at run-time.
+ */
+void PNGAPI
+png_warning(png_structp png_ptr, png_const_charp warning_message)
+{
+ int offset = 0;
+ if (png_ptr != NULL)
+ {
+#ifdef PNG_ERROR_NUMBERS_SUPPORTED
+ if (png_ptr->flags&
+ (PNG_FLAG_STRIP_ERROR_NUMBERS|PNG_FLAG_STRIP_ERROR_TEXT))
+#endif
+ {
+ if (*warning_message == PNG_LITERAL_SHARP)
+ {
+ for (offset = 1; offset < 15; offset++)
+ if (warning_message[offset] == ' ')
+ break;
+ }
+ }
+ }
+ if (png_ptr != NULL && png_ptr->warning_fn != NULL)
+ (*(png_ptr->warning_fn))(png_ptr, warning_message + offset);
+ else
+ png_default_warning(png_ptr, warning_message + offset);
+}
+#endif /* PNG_WARNINGS_SUPPORTED */
+
+#ifdef PNG_BENIGN_ERRORS_SUPPORTED
+void PNGAPI
+png_benign_error(png_structp png_ptr, png_const_charp error_message)
+{
+ if (png_ptr->flags & PNG_FLAG_BENIGN_ERRORS_WARN)
+ png_warning(png_ptr, error_message);
+ else
+ png_error(png_ptr, error_message);
+}
+#endif
+
+/* These utilities are used internally to build an error message that relates
+ * to the current chunk. The chunk name comes from png_ptr->chunk_name,
+ * this is used to prefix the message. The message is limited in length
+ * to 63 bytes, the name characters are output as hex digits wrapped in []
+ * if the character is invalid.
+ */
+#define isnonalpha(c) ((c) < 65 || (c) > 122 || ((c) > 90 && (c) < 97))
+static PNG_CONST char png_digit[16] = {
+ '0', '1', '2', '3', '4', '5', '6', '7', '8', '9',
+ 'A', 'B', 'C', 'D', 'E', 'F'
+};
+
+#define PNG_MAX_ERROR_TEXT 64
+#if defined(PNG_WARNINGS_SUPPORTED) || defined(PNG_ERROR_TEXT_SUPPORTED)
+static void /* PRIVATE */
+png_format_buffer(png_structp png_ptr, png_charp buffer, png_const_charp
+ error_message)
+{
+ int iout = 0, iin = 0;
+
+ while (iin < 4)
+ {
+ int c = png_ptr->chunk_name[iin++];
+ if (isnonalpha(c))
+ {
+ buffer[iout++] = PNG_LITERAL_LEFT_SQUARE_BRACKET;
+ buffer[iout++] = png_digit[(c & 0xf0) >> 4];
+ buffer[iout++] = png_digit[c & 0x0f];
+ buffer[iout++] = PNG_LITERAL_RIGHT_SQUARE_BRACKET;
+ }
+ else
+ {
+ buffer[iout++] = (png_byte)c;
+ }
+ }
+
+ if (error_message == NULL)
+ buffer[iout] = '\0';
+ else
+ {
+ buffer[iout++] = ':';
+ buffer[iout++] = ' ';
+ png_memcpy(buffer + iout, error_message, PNG_MAX_ERROR_TEXT);
+ buffer[iout + PNG_MAX_ERROR_TEXT - 1] = '\0';
+ }
+}
+
+#ifdef PNG_READ_SUPPORTED
+void PNGAPI
+png_chunk_error(png_structp png_ptr, png_const_charp error_message)
+{
+ char msg[18+PNG_MAX_ERROR_TEXT];
+ if (png_ptr == NULL)
+ png_error(png_ptr, error_message);
+ else
+ {
+ png_format_buffer(png_ptr, msg, error_message);
+ png_error(png_ptr, msg);
+ }
+}
+#endif /* PNG_READ_SUPPORTED */
+#endif /* PNG_WARNINGS_SUPPORTED || PNG_ERROR_TEXT_SUPPORTED */
+
+#ifdef PNG_WARNINGS_SUPPORTED
+void PNGAPI
+png_chunk_warning(png_structp png_ptr, png_const_charp warning_message)
+{
+ char msg[18+PNG_MAX_ERROR_TEXT];
+ if (png_ptr == NULL)
+ png_warning(png_ptr, warning_message);
+ else
+ {
+ png_format_buffer(png_ptr, msg, warning_message);
+ png_warning(png_ptr, msg);
+ }
+}
+#endif /* PNG_WARNINGS_SUPPORTED */
+
+#ifdef PNG_READ_SUPPORTED
+#ifdef PNG_BENIGN_ERRORS_SUPPORTED
+void PNGAPI
+png_chunk_benign_error(png_structp png_ptr, png_const_charp error_message)
+{
+ if (png_ptr->flags & PNG_FLAG_BENIGN_ERRORS_WARN)
+ png_chunk_warning(png_ptr, error_message);
+ else
+ png_chunk_error(png_ptr, error_message);
+}
+#endif
+#endif /* PNG_READ_SUPPORTED */
+
+/* This is the default error handling function. Note that replacements for
+ * this function MUST NOT RETURN, or the program will likely crash. This
+ * function is used by default, or if the program supplies NULL for the
+ * error function pointer in png_set_error_fn().
+ */
+static void /* PRIVATE */
+png_default_error(png_structp png_ptr, png_const_charp error_message)
+{
+#ifdef PNG_CONSOLE_IO_SUPPORTED
+#ifdef PNG_ERROR_NUMBERS_SUPPORTED
+ if (*error_message == PNG_LITERAL_SHARP)
+ {
+ /* Strip "#nnnn " from beginning of error message. */
+ int offset;
+ char error_number[16];
+ for (offset = 0; offset<15; offset++)
+ {
+ error_number[offset] = error_message[offset + 1];
+ if (error_message[offset] == ' ')
+ break;
+ }
+ if ((offset > 1) && (offset < 15))
+ {
+ error_number[offset - 1] = '\0';
+ fprintf(stderr, "libpng error no. %s: %s",
+ error_number, error_message + offset + 1);
+ fprintf(stderr, PNG_STRING_NEWLINE);
+ }
+ else
+ {
+ fprintf(stderr, "libpng error: %s, offset=%d",
+ error_message, offset);
+ fprintf(stderr, PNG_STRING_NEWLINE);
+ }
+ }
+ else
+#endif
+ {
+ fprintf(stderr, "libpng error: %s", error_message);
+ fprintf(stderr, PNG_STRING_NEWLINE);
+ }
+#endif
+
+#ifdef PNG_SETJMP_SUPPORTED
+ if (png_ptr)
+ {
+# ifdef USE_FAR_KEYWORD
+ {
+ jmp_buf jmpbuf;
+ png_memcpy(jmpbuf, png_ptr->jmpbuf, png_sizeof(jmp_buf));
+ longjmp(jmpbuf,1);
+ }
+# else
+ longjmp(png_ptr->jmpbuf, 1);
+# endif
+ }
+#endif
+ /* Here if not setjmp support or if png_ptr is null. */
+ PNG_ABORT();
+#ifndef PNG_CONSOLE_IO_SUPPORTED
+ error_message = error_message; /* Make compiler happy */
+#endif
+}
+
+#ifdef PNG_WARNINGS_SUPPORTED
+/* This function is called when there is a warning, but the library thinks
+ * it can continue anyway. Replacement functions don't have to do anything
+ * here if you don't want them to. In the default configuration, png_ptr is
+ * not used, but it is passed in case it may be useful.
+ */
+static void /* PRIVATE */
+png_default_warning(png_structp png_ptr, png_const_charp warning_message)
+{
+#ifdef PNG_CONSOLE_IO_SUPPORTED
+# ifdef PNG_ERROR_NUMBERS_SUPPORTED
+ if (*warning_message == PNG_LITERAL_SHARP)
+ {
+ int offset;
+ char warning_number[16];
+ for (offset = 0; offset < 15; offset++)
+ {
+ warning_number[offset] = warning_message[offset + 1];
+ if (warning_message[offset] == ' ')
+ break;
+ }
+ if ((offset > 1) && (offset < 15))
+ {
+ warning_number[offset + 1] = '\0';
+ fprintf(stderr, "libpng warning no. %s: %s",
+ warning_number, warning_message + offset);
+ fprintf(stderr, PNG_STRING_NEWLINE);
+ }
+ else
+ {
+ fprintf(stderr, "libpng warning: %s",
+ warning_message);
+ fprintf(stderr, PNG_STRING_NEWLINE);
+ }
+ }
+ else
+# endif
+ {
+ fprintf(stderr, "libpng warning: %s", warning_message);
+ fprintf(stderr, PNG_STRING_NEWLINE);
+ }
+#else
+ warning_message = warning_message; /* Make compiler happy */
+#endif
+ png_ptr = png_ptr; /* Make compiler happy */
+}
+#endif /* PNG_WARNINGS_SUPPORTED */
+
+/* This function is called when the application wants to use another method
+ * of handling errors and warnings. Note that the error function MUST NOT
+ * return to the calling routine or serious problems will occur. The return
+ * method used in the default routine calls longjmp(png_ptr->jmpbuf, 1)
+ */
+void PNGAPI
+png_set_error_fn(png_structp png_ptr, png_voidp error_ptr,
+ png_error_ptr error_fn, png_error_ptr warning_fn)
+{
+ if (png_ptr == NULL)
+ return;
+ png_ptr->error_ptr = error_ptr;
+ png_ptr->error_fn = error_fn;
+ png_ptr->warning_fn = warning_fn;
+}
+
+
+/* This function returns a pointer to the error_ptr associated with the user
+ * functions. The application should free any memory associated with this
+ * pointer before png_write_destroy and png_read_destroy are called.
+ */
+png_voidp PNGAPI
+png_get_error_ptr(png_structp png_ptr)
+{
+ if (png_ptr == NULL)
+ return NULL;
+ return ((png_voidp)png_ptr->error_ptr);
+}
+
+
+#ifdef PNG_ERROR_NUMBERS_SUPPORTED
+void PNGAPI
+png_set_strip_error_numbers(png_structp png_ptr, png_uint_32 strip_mode)
+{
+ if (png_ptr != NULL)
+ {
+ png_ptr->flags &=
+ ((~(PNG_FLAG_STRIP_ERROR_NUMBERS|PNG_FLAG_STRIP_ERROR_TEXT))&strip_mode);
+ }
+}
+#endif
+#endif /* PNG_READ_SUPPORTED || PNG_WRITE_SUPPORTED */
diff --git a/trunk/src/third_party/libpng/pnggccrd.c b/trunk/src/third_party/libpng/pnggccrd.c
new file mode 100644
index 0000000..78b8a7e
--- /dev/null
+++ b/trunk/src/third_party/libpng/pnggccrd.c
@@ -0,0 +1,103 @@
+/* pnggccrd.c was removed from libpng-1.2.20. */
+
+/* This code snippet is for use by configure's compilation test. */
+
+#if (!defined _MSC_VER) && \
+ defined(PNG_ASSEMBLER_CODE_SUPPORTED) && \
+ defined(PNG_MMX_CODE_SUPPORTED)
+
+int PNGAPI png_dummy_mmx_support(void);
+
+static int _mmx_supported = 2; // 0: no MMX; 1: MMX supported; 2: not tested
+
+int PNGAPI
+png_dummy_mmx_support(void) __attribute__((noinline));
+
+int PNGAPI
+png_dummy_mmx_support(void)
+{
+ int result;
+#ifdef PNG_MMX_CODE_SUPPORTED // superfluous, but what the heck
+ __asm__ __volatile__ (
+#ifdef __x86_64__
+ "pushq %%rbx \n\t" // rbx gets clobbered by CPUID instruction
+ "pushq %%rcx \n\t" // so does rcx...
+ "pushq %%rdx \n\t" // ...and rdx (but rcx & rdx safe on Linux)
+ "pushfq \n\t" // save Eflag to stack
+ "popq %%rax \n\t" // get Eflag from stack into rax
+ "movq %%rax, %%rcx \n\t" // make another copy of Eflag in rcx
+ "xorl $0x200000, %%eax \n\t" // toggle ID bit in Eflag (i.e., bit 21)
+ "pushq %%rax \n\t" // save modified Eflag back to stack
+ "popfq \n\t" // restore modified value to Eflag reg
+ "pushfq \n\t" // save Eflag to stack
+ "popq %%rax \n\t" // get Eflag from stack
+ "pushq %%rcx \n\t" // save original Eflag to stack
+ "popfq \n\t" // restore original Eflag
+#else
+ "pushl %%ebx \n\t" // ebx gets clobbered by CPUID instruction
+ "pushl %%ecx \n\t" // so does ecx...
+ "pushl %%edx \n\t" // ...and edx (but ecx & edx safe on Linux)
+ "pushfl \n\t" // save Eflag to stack
+ "popl %%eax \n\t" // get Eflag from stack into eax
+ "movl %%eax, %%ecx \n\t" // make another copy of Eflag in ecx
+ "xorl $0x200000, %%eax \n\t" // toggle ID bit in Eflag (i.e., bit 21)
+ "pushl %%eax \n\t" // save modified Eflag back to stack
+ "popfl \n\t" // restore modified value to Eflag reg
+ "pushfl \n\t" // save Eflag to stack
+ "popl %%eax \n\t" // get Eflag from stack
+ "pushl %%ecx \n\t" // save original Eflag to stack
+ "popfl \n\t" // restore original Eflag
+#endif
+ "xorl %%ecx, %%eax \n\t" // compare new Eflag with original Eflag
+ "jz 0f \n\t" // if same, CPUID instr. is not supported
+
+ "xorl %%eax, %%eax \n\t" // set eax to zero
+// ".byte 0x0f, 0xa2 \n\t" // CPUID instruction (two-byte opcode)
+ "cpuid \n\t" // get the CPU identification info
+ "cmpl $1, %%eax \n\t" // make sure eax return non-zero value
+ "jl 0f \n\t" // if eax is zero, MMX is not supported
+
+ "xorl %%eax, %%eax \n\t" // set eax to zero and...
+ "incl %%eax \n\t" // ...increment eax to 1. This pair is
+ // faster than the instruction "mov eax, 1"
+ "cpuid \n\t" // get the CPU identification info again
+ "andl $0x800000, %%edx \n\t" // mask out all bits but MMX bit (23)
+ "cmpl $0, %%edx \n\t" // 0 = MMX not supported
+ "jz 0f \n\t" // non-zero = yes, MMX IS supported
+
+ "movl $1, %%eax \n\t" // set return value to 1
+ "jmp 1f \n\t" // DONE: have MMX support
+
+ "0: \n\t" // .NOT_SUPPORTED: target label for jump instructions
+ "movl $0, %%eax \n\t" // set return value to 0
+ "1: \n\t" // .RETURN: target label for jump instructions
+#ifdef __x86_64__
+ "popq %%rdx \n\t" // restore rdx
+ "popq %%rcx \n\t" // restore rcx
+ "popq %%rbx \n\t" // restore rbx
+#else
+ "popl %%edx \n\t" // restore edx
+ "popl %%ecx \n\t" // restore ecx
+ "popl %%ebx \n\t" // restore ebx
+#endif
+
+// "ret \n\t" // DONE: no MMX support
+ // (fall through to standard C "ret")
+
+ : "=a" (result) // output list
+
+ : // any variables used on input (none)
+
+ // no clobber list
+// , "%ebx", "%ecx", "%edx" // GRR: we handle these manually
+// , "memory" // if write to a variable gcc thought was in a reg
+// , "cc" // "condition codes" (flag bits)
+ );
+ _mmx_supported = result;
+#else
+ _mmx_supported = 0;
+#endif /* PNG_MMX_CODE_SUPPORTED */
+
+ return _mmx_supported;
+}
+#endif
diff --git a/trunk/src/third_party/libpng/pngget.c b/trunk/src/third_party/libpng/pngget.c
new file mode 100644
index 0000000..d397329
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngget.c
@@ -0,0 +1,944 @@
+
+/* pngget.c - retrieval of values from info struct
+ *
+ * Last changed in libpng 1.2.43 [February 25, 2010]
+ * Copyright (c) 1998-2010 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ *
+ */
+
+#define PNG_INTERNAL
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+#if defined(PNG_READ_SUPPORTED) || defined(PNG_WRITE_SUPPORTED)
+
+png_uint_32 PNGAPI
+png_get_valid(png_structp png_ptr, png_infop info_ptr, png_uint_32 flag)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+ return(info_ptr->valid & flag);
+
+ else
+ return(0);
+}
+
+png_uint_32 PNGAPI
+png_get_rowbytes(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+ return(info_ptr->rowbytes);
+
+ else
+ return(0);
+}
+
+#ifdef PNG_INFO_IMAGE_SUPPORTED
+png_bytepp PNGAPI
+png_get_rows(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+ return(info_ptr->row_pointers);
+
+ else
+ return(0);
+}
+#endif
+
+#ifdef PNG_EASY_ACCESS_SUPPORTED
+/* Easy access to info, added in libpng-0.99 */
+png_uint_32 PNGAPI
+png_get_image_width(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+ return info_ptr->width;
+
+ return (0);
+}
+
+png_uint_32 PNGAPI
+png_get_image_height(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+ return info_ptr->height;
+
+ return (0);
+}
+
+png_byte PNGAPI
+png_get_bit_depth(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+ return info_ptr->bit_depth;
+
+ return (0);
+}
+
+png_byte PNGAPI
+png_get_color_type(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+ return info_ptr->color_type;
+
+ return (0);
+}
+
+png_byte PNGAPI
+png_get_filter_type(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+ return info_ptr->filter_type;
+
+ return (0);
+}
+
+png_byte PNGAPI
+png_get_interlace_type(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+ return info_ptr->interlace_type;
+
+ return (0);
+}
+
+png_byte PNGAPI
+png_get_compression_type(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+ return info_ptr->compression_type;
+
+ return (0);
+}
+
+png_uint_32 PNGAPI
+png_get_x_pixels_per_meter(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+#ifdef PNG_pHYs_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_pHYs)
+ {
+ png_debug1(1, "in %s retrieval function", "png_get_x_pixels_per_meter");
+
+ if (info_ptr->phys_unit_type != PNG_RESOLUTION_METER)
+ return (0);
+
+ else
+ return (info_ptr->x_pixels_per_unit);
+ }
+#else
+ return (0);
+#endif
+ return (0);
+}
+
+png_uint_32 PNGAPI
+png_get_y_pixels_per_meter(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+#ifdef PNG_pHYs_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_pHYs)
+ {
+ png_debug1(1, "in %s retrieval function", "png_get_y_pixels_per_meter");
+
+ if (info_ptr->phys_unit_type != PNG_RESOLUTION_METER)
+ return (0);
+
+ else
+ return (info_ptr->y_pixels_per_unit);
+ }
+#else
+ return (0);
+#endif
+ return (0);
+}
+
+png_uint_32 PNGAPI
+png_get_pixels_per_meter(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+#ifdef PNG_pHYs_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_pHYs)
+ {
+ png_debug1(1, "in %s retrieval function", "png_get_pixels_per_meter");
+
+ if (info_ptr->phys_unit_type != PNG_RESOLUTION_METER ||
+ info_ptr->x_pixels_per_unit != info_ptr->y_pixels_per_unit)
+ return (0);
+
+ else
+ return (info_ptr->x_pixels_per_unit);
+ }
+#else
+ return (0);
+#endif
+ return (0);
+}
+
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+float PNGAPI
+png_get_pixel_aspect_ratio(png_structp png_ptr, png_infop info_ptr)
+ {
+ if (png_ptr != NULL && info_ptr != NULL)
+#ifdef PNG_pHYs_SUPPORTED
+
+ if (info_ptr->valid & PNG_INFO_pHYs)
+ {
+ png_debug1(1, "in %s retrieval function", "png_get_aspect_ratio");
+
+ if (info_ptr->x_pixels_per_unit == 0)
+ return ((float)0.0);
+
+ else
+ return ((float)((float)info_ptr->y_pixels_per_unit
+ /(float)info_ptr->x_pixels_per_unit));
+ }
+#else
+ return (0.0);
+#endif
+ return ((float)0.0);
+}
+#endif
+
+png_int_32 PNGAPI
+png_get_x_offset_microns(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+#ifdef PNG_oFFs_SUPPORTED
+
+ if (info_ptr->valid & PNG_INFO_oFFs)
+ {
+ png_debug1(1, "in %s retrieval function", "png_get_x_offset_microns");
+
+ if (info_ptr->offset_unit_type != PNG_OFFSET_MICROMETER)
+ return (0);
+
+ else
+ return (info_ptr->x_offset);
+ }
+#else
+ return (0);
+#endif
+ return (0);
+}
+
+png_int_32 PNGAPI
+png_get_y_offset_microns(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+
+#ifdef PNG_oFFs_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_oFFs)
+ {
+ png_debug1(1, "in %s retrieval function", "png_get_y_offset_microns");
+
+ if (info_ptr->offset_unit_type != PNG_OFFSET_MICROMETER)
+ return (0);
+
+ else
+ return (info_ptr->y_offset);
+ }
+#else
+ return (0);
+#endif
+ return (0);
+}
+
+png_int_32 PNGAPI
+png_get_x_offset_pixels(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+
+#ifdef PNG_oFFs_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_oFFs)
+ {
+ png_debug1(1, "in %s retrieval function", "png_get_x_offset_microns");
+
+ if (info_ptr->offset_unit_type != PNG_OFFSET_PIXEL)
+ return (0);
+
+ else
+ return (info_ptr->x_offset);
+ }
+#else
+ return (0);
+#endif
+ return (0);
+}
+
+png_int_32 PNGAPI
+png_get_y_offset_pixels(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+
+#ifdef PNG_oFFs_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_oFFs)
+ {
+ png_debug1(1, "in %s retrieval function", "png_get_y_offset_microns");
+
+ if (info_ptr->offset_unit_type != PNG_OFFSET_PIXEL)
+ return (0);
+
+ else
+ return (info_ptr->y_offset);
+ }
+#else
+ return (0);
+#endif
+ return (0);
+}
+
+#if defined(PNG_INCH_CONVERSIONS) && defined(PNG_FLOATING_POINT_SUPPORTED)
+png_uint_32 PNGAPI
+png_get_pixels_per_inch(png_structp png_ptr, png_infop info_ptr)
+{
+ return ((png_uint_32)((float)png_get_pixels_per_meter(png_ptr, info_ptr)
+ *.0254 +.5));
+}
+
+png_uint_32 PNGAPI
+png_get_x_pixels_per_inch(png_structp png_ptr, png_infop info_ptr)
+{
+ return ((png_uint_32)((float)png_get_x_pixels_per_meter(png_ptr, info_ptr)
+ *.0254 +.5));
+}
+
+png_uint_32 PNGAPI
+png_get_y_pixels_per_inch(png_structp png_ptr, png_infop info_ptr)
+{
+ return ((png_uint_32)((float)png_get_y_pixels_per_meter(png_ptr, info_ptr)
+ *.0254 +.5));
+}
+
+float PNGAPI
+png_get_x_offset_inches(png_structp png_ptr, png_infop info_ptr)
+{
+ return ((float)png_get_x_offset_microns(png_ptr, info_ptr)
+ *.00003937);
+}
+
+float PNGAPI
+png_get_y_offset_inches(png_structp png_ptr, png_infop info_ptr)
+{
+ return ((float)png_get_y_offset_microns(png_ptr, info_ptr)
+ *.00003937);
+}
+
+#ifdef PNG_pHYs_SUPPORTED
+png_uint_32 PNGAPI
+png_get_pHYs_dpi(png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 *res_x, png_uint_32 *res_y, int *unit_type)
+{
+ png_uint_32 retval = 0;
+
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_pHYs))
+ {
+ png_debug1(1, "in %s retrieval function", "pHYs");
+
+ if (res_x != NULL)
+ {
+ *res_x = info_ptr->x_pixels_per_unit;
+ retval |= PNG_INFO_pHYs;
+ }
+ if (res_y != NULL)
+ {
+ *res_y = info_ptr->y_pixels_per_unit;
+ retval |= PNG_INFO_pHYs;
+ }
+ if (unit_type != NULL)
+ {
+ *unit_type = (int)info_ptr->phys_unit_type;
+ retval |= PNG_INFO_pHYs;
+ if (*unit_type == 1)
+ {
+ if (res_x != NULL) *res_x = (png_uint_32)(*res_x * .0254 + .50);
+ if (res_y != NULL) *res_y = (png_uint_32)(*res_y * .0254 + .50);
+ }
+ }
+ }
+ return (retval);
+}
+#endif /* PNG_pHYs_SUPPORTED */
+#endif /* PNG_INCH_CONVERSIONS && PNG_FLOATING_POINT_SUPPORTED */
+
+/* png_get_channels really belongs in here, too, but it's been around longer */
+
+#endif /* PNG_EASY_ACCESS_SUPPORTED */
+
+png_byte PNGAPI
+png_get_channels(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+ return(info_ptr->channels);
+ else
+ return (0);
+}
+
+png_bytep PNGAPI
+png_get_signature(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr != NULL && info_ptr != NULL)
+ return(info_ptr->signature);
+ else
+ return (NULL);
+}
+
+#ifdef PNG_bKGD_SUPPORTED
+png_uint_32 PNGAPI
+png_get_bKGD(png_structp png_ptr, png_infop info_ptr,
+ png_color_16p *background)
+{
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_bKGD)
+ && background != NULL)
+ {
+ png_debug1(1, "in %s retrieval function", "bKGD");
+
+ *background = &(info_ptr->background);
+ return (PNG_INFO_bKGD);
+ }
+ return (0);
+}
+#endif
+
+#ifdef PNG_cHRM_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+png_uint_32 PNGAPI
+png_get_cHRM(png_structp png_ptr, png_infop info_ptr,
+ double *white_x, double *white_y, double *red_x, double *red_y,
+ double *green_x, double *green_y, double *blue_x, double *blue_y)
+{
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_cHRM))
+ {
+ png_debug1(1, "in %s retrieval function", "cHRM");
+
+ if (white_x != NULL)
+ *white_x = (double)info_ptr->x_white;
+ if (white_y != NULL)
+ *white_y = (double)info_ptr->y_white;
+ if (red_x != NULL)
+ *red_x = (double)info_ptr->x_red;
+ if (red_y != NULL)
+ *red_y = (double)info_ptr->y_red;
+ if (green_x != NULL)
+ *green_x = (double)info_ptr->x_green;
+ if (green_y != NULL)
+ *green_y = (double)info_ptr->y_green;
+ if (blue_x != NULL)
+ *blue_x = (double)info_ptr->x_blue;
+ if (blue_y != NULL)
+ *blue_y = (double)info_ptr->y_blue;
+ return (PNG_INFO_cHRM);
+ }
+ return (0);
+}
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+png_uint_32 PNGAPI
+png_get_cHRM_fixed(png_structp png_ptr, png_infop info_ptr,
+ png_fixed_point *white_x, png_fixed_point *white_y, png_fixed_point *red_x,
+ png_fixed_point *red_y, png_fixed_point *green_x, png_fixed_point *green_y,
+ png_fixed_point *blue_x, png_fixed_point *blue_y)
+{
+ png_debug1(1, "in %s retrieval function", "cHRM");
+
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_cHRM))
+ {
+ if (white_x != NULL)
+ *white_x = info_ptr->int_x_white;
+ if (white_y != NULL)
+ *white_y = info_ptr->int_y_white;
+ if (red_x != NULL)
+ *red_x = info_ptr->int_x_red;
+ if (red_y != NULL)
+ *red_y = info_ptr->int_y_red;
+ if (green_x != NULL)
+ *green_x = info_ptr->int_x_green;
+ if (green_y != NULL)
+ *green_y = info_ptr->int_y_green;
+ if (blue_x != NULL)
+ *blue_x = info_ptr->int_x_blue;
+ if (blue_y != NULL)
+ *blue_y = info_ptr->int_y_blue;
+ return (PNG_INFO_cHRM);
+ }
+ return (0);
+}
+#endif
+#endif
+
+#ifdef PNG_gAMA_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+png_uint_32 PNGAPI
+png_get_gAMA(png_structp png_ptr, png_infop info_ptr, double *file_gamma)
+{
+ png_debug1(1, "in %s retrieval function", "gAMA");
+
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_gAMA)
+ && file_gamma != NULL)
+ {
+ *file_gamma = (double)info_ptr->gamma;
+ return (PNG_INFO_gAMA);
+ }
+ return (0);
+}
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+png_uint_32 PNGAPI
+png_get_gAMA_fixed(png_structp png_ptr, png_infop info_ptr,
+ png_fixed_point *int_file_gamma)
+{
+ png_debug1(1, "in %s retrieval function", "gAMA");
+
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_gAMA)
+ && int_file_gamma != NULL)
+ {
+ *int_file_gamma = info_ptr->int_gamma;
+ return (PNG_INFO_gAMA);
+ }
+ return (0);
+}
+#endif
+#endif
+
+#ifdef PNG_sRGB_SUPPORTED
+png_uint_32 PNGAPI
+png_get_sRGB(png_structp png_ptr, png_infop info_ptr, int *file_srgb_intent)
+{
+ png_debug1(1, "in %s retrieval function", "sRGB");
+
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_sRGB)
+ && file_srgb_intent != NULL)
+ {
+ *file_srgb_intent = (int)info_ptr->srgb_intent;
+ return (PNG_INFO_sRGB);
+ }
+ return (0);
+}
+#endif
+
+#ifdef PNG_iCCP_SUPPORTED
+png_uint_32 PNGAPI
+png_get_iCCP(png_structp png_ptr, png_infop info_ptr,
+ png_charpp name, int *compression_type,
+ png_charpp profile, png_uint_32 *proflen)
+{
+ png_debug1(1, "in %s retrieval function", "iCCP");
+
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_iCCP)
+ && name != NULL && profile != NULL && proflen != NULL)
+ {
+ *name = info_ptr->iccp_name;
+ *profile = info_ptr->iccp_profile;
+ /* Compression_type is a dummy so the API won't have to change
+ * if we introduce multiple compression types later.
+ */
+ *proflen = (int)info_ptr->iccp_proflen;
+ *compression_type = (int)info_ptr->iccp_compression;
+ return (PNG_INFO_iCCP);
+ }
+ return (0);
+}
+#endif
+
+#ifdef PNG_sPLT_SUPPORTED
+png_uint_32 PNGAPI
+png_get_sPLT(png_structp png_ptr, png_infop info_ptr,
+ png_sPLT_tpp spalettes)
+{
+ if (png_ptr != NULL && info_ptr != NULL && spalettes != NULL)
+ {
+ *spalettes = info_ptr->splt_palettes;
+ return ((png_uint_32)info_ptr->splt_palettes_num);
+ }
+ return (0);
+}
+#endif
+
+#ifdef PNG_hIST_SUPPORTED
+png_uint_32 PNGAPI
+png_get_hIST(png_structp png_ptr, png_infop info_ptr, png_uint_16p *hist)
+{
+ png_debug1(1, "in %s retrieval function", "hIST");
+
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_hIST)
+ && hist != NULL)
+ {
+ *hist = info_ptr->hist;
+ return (PNG_INFO_hIST);
+ }
+ return (0);
+}
+#endif
+
+png_uint_32 PNGAPI
+png_get_IHDR(png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 *width, png_uint_32 *height, int *bit_depth,
+ int *color_type, int *interlace_type, int *compression_type,
+ int *filter_type)
+
+{
+ png_debug1(1, "in %s retrieval function", "IHDR");
+
+ if (png_ptr == NULL || info_ptr == NULL || width == NULL ||
+ height == NULL || bit_depth == NULL || color_type == NULL)
+ return (0);
+
+ *width = info_ptr->width;
+ *height = info_ptr->height;
+ *bit_depth = info_ptr->bit_depth;
+ *color_type = info_ptr->color_type;
+
+ if (compression_type != NULL)
+ *compression_type = info_ptr->compression_type;
+
+ if (filter_type != NULL)
+ *filter_type = info_ptr->filter_type;
+
+ if (interlace_type != NULL)
+ *interlace_type = info_ptr->interlace_type;
+
+ /* This is redundant if we can be sure that the info_ptr values were all
+ * assigned in png_set_IHDR(). We do the check anyhow in case an
+ * application has ignored our advice not to mess with the members
+ * of info_ptr directly.
+ */
+ png_check_IHDR (png_ptr, info_ptr->width, info_ptr->height,
+ info_ptr->bit_depth, info_ptr->color_type, info_ptr->interlace_type,
+ info_ptr->compression_type, info_ptr->filter_type);
+
+ return (1);
+}
+
+#ifdef PNG_oFFs_SUPPORTED
+png_uint_32 PNGAPI
+png_get_oFFs(png_structp png_ptr, png_infop info_ptr,
+ png_int_32 *offset_x, png_int_32 *offset_y, int *unit_type)
+{
+ png_debug1(1, "in %s retrieval function", "oFFs");
+
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_oFFs)
+ && offset_x != NULL && offset_y != NULL && unit_type != NULL)
+ {
+ *offset_x = info_ptr->x_offset;
+ *offset_y = info_ptr->y_offset;
+ *unit_type = (int)info_ptr->offset_unit_type;
+ return (PNG_INFO_oFFs);
+ }
+ return (0);
+}
+#endif
+
+#ifdef PNG_pCAL_SUPPORTED
+png_uint_32 PNGAPI
+png_get_pCAL(png_structp png_ptr, png_infop info_ptr,
+ png_charp *purpose, png_int_32 *X0, png_int_32 *X1, int *type, int *nparams,
+ png_charp *units, png_charpp *params)
+{
+ png_debug1(1, "in %s retrieval function", "pCAL");
+
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_pCAL)
+ && purpose != NULL && X0 != NULL && X1 != NULL && type != NULL &&
+ nparams != NULL && units != NULL && params != NULL)
+ {
+ *purpose = info_ptr->pcal_purpose;
+ *X0 = info_ptr->pcal_X0;
+ *X1 = info_ptr->pcal_X1;
+ *type = (int)info_ptr->pcal_type;
+ *nparams = (int)info_ptr->pcal_nparams;
+ *units = info_ptr->pcal_units;
+ *params = info_ptr->pcal_params;
+ return (PNG_INFO_pCAL);
+ }
+ return (0);
+}
+#endif
+
+#ifdef PNG_sCAL_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+png_uint_32 PNGAPI
+png_get_sCAL(png_structp png_ptr, png_infop info_ptr,
+ int *unit, double *width, double *height)
+{
+ if (png_ptr != NULL && info_ptr != NULL &&
+ (info_ptr->valid & PNG_INFO_sCAL))
+ {
+ *unit = info_ptr->scal_unit;
+ *width = info_ptr->scal_pixel_width;
+ *height = info_ptr->scal_pixel_height;
+ return (PNG_INFO_sCAL);
+ }
+ return(0);
+}
+#else
+#ifdef PNG_FIXED_POINT_SUPPORTED
+png_uint_32 PNGAPI
+png_get_sCAL_s(png_structp png_ptr, png_infop info_ptr,
+ int *unit, png_charpp width, png_charpp height)
+{
+ if (png_ptr != NULL && info_ptr != NULL &&
+ (info_ptr->valid & PNG_INFO_sCAL))
+ {
+ *unit = info_ptr->scal_unit;
+ *width = info_ptr->scal_s_width;
+ *height = info_ptr->scal_s_height;
+ return (PNG_INFO_sCAL);
+ }
+ return(0);
+}
+#endif
+#endif
+#endif
+
+#ifdef PNG_pHYs_SUPPORTED
+png_uint_32 PNGAPI
+png_get_pHYs(png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 *res_x, png_uint_32 *res_y, int *unit_type)
+{
+ png_uint_32 retval = 0;
+
+ png_debug1(1, "in %s retrieval function", "pHYs");
+
+ if (png_ptr != NULL && info_ptr != NULL &&
+ (info_ptr->valid & PNG_INFO_pHYs))
+ {
+ if (res_x != NULL)
+ {
+ *res_x = info_ptr->x_pixels_per_unit;
+ retval |= PNG_INFO_pHYs;
+ }
+
+ if (res_y != NULL)
+ {
+ *res_y = info_ptr->y_pixels_per_unit;
+ retval |= PNG_INFO_pHYs;
+ }
+
+ if (unit_type != NULL)
+ {
+ *unit_type = (int)info_ptr->phys_unit_type;
+ retval |= PNG_INFO_pHYs;
+ }
+ }
+ return (retval);
+}
+#endif
+
+png_uint_32 PNGAPI
+png_get_PLTE(png_structp png_ptr, png_infop info_ptr, png_colorp *palette,
+ int *num_palette)
+{
+ png_debug1(1, "in %s retrieval function", "PLTE");
+
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_PLTE)
+ && palette != NULL)
+ {
+ *palette = info_ptr->palette;
+ *num_palette = info_ptr->num_palette;
+ png_debug1(3, "num_palette = %d", *num_palette);
+ return (PNG_INFO_PLTE);
+ }
+ return (0);
+}
+
+#ifdef PNG_sBIT_SUPPORTED
+png_uint_32 PNGAPI
+png_get_sBIT(png_structp png_ptr, png_infop info_ptr, png_color_8p *sig_bit)
+{
+ png_debug1(1, "in %s retrieval function", "sBIT");
+
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_sBIT)
+ && sig_bit != NULL)
+ {
+ *sig_bit = &(info_ptr->sig_bit);
+ return (PNG_INFO_sBIT);
+ }
+ return (0);
+}
+#endif
+
+#ifdef PNG_TEXT_SUPPORTED
+png_uint_32 PNGAPI
+png_get_text(png_structp png_ptr, png_infop info_ptr, png_textp *text_ptr,
+ int *num_text)
+{
+ if (png_ptr != NULL && info_ptr != NULL && info_ptr->num_text > 0)
+ {
+ png_debug1(1, "in %s retrieval function",
+ (png_ptr->chunk_name[0] == '\0' ? "text"
+ : (png_const_charp)png_ptr->chunk_name));
+
+ if (text_ptr != NULL)
+ *text_ptr = info_ptr->text;
+
+ if (num_text != NULL)
+ *num_text = info_ptr->num_text;
+
+ return ((png_uint_32)info_ptr->num_text);
+ }
+ if (num_text != NULL)
+ *num_text = 0;
+ return(0);
+}
+#endif
+
+#ifdef PNG_tIME_SUPPORTED
+png_uint_32 PNGAPI
+png_get_tIME(png_structp png_ptr, png_infop info_ptr, png_timep *mod_time)
+{
+ png_debug1(1, "in %s retrieval function", "tIME");
+
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_tIME)
+ && mod_time != NULL)
+ {
+ *mod_time = &(info_ptr->mod_time);
+ return (PNG_INFO_tIME);
+ }
+ return (0);
+}
+#endif
+
+#ifdef PNG_tRNS_SUPPORTED
+png_uint_32 PNGAPI
+png_get_tRNS(png_structp png_ptr, png_infop info_ptr,
+ png_bytep *trans, int *num_trans, png_color_16p *trans_values)
+{
+ png_uint_32 retval = 0;
+ if (png_ptr != NULL && info_ptr != NULL && (info_ptr->valid & PNG_INFO_tRNS))
+ {
+ png_debug1(1, "in %s retrieval function", "tRNS");
+
+ if (info_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ if (trans != NULL)
+ {
+ *trans = info_ptr->trans;
+ retval |= PNG_INFO_tRNS;
+ }
+
+ if (trans_values != NULL)
+ *trans_values = &(info_ptr->trans_values);
+ }
+ else /* if (info_ptr->color_type != PNG_COLOR_TYPE_PALETTE) */
+ {
+ if (trans_values != NULL)
+ {
+ *trans_values = &(info_ptr->trans_values);
+ retval |= PNG_INFO_tRNS;
+ }
+
+ if (trans != NULL)
+ *trans = NULL;
+ }
+ if (num_trans != NULL)
+ {
+ *num_trans = info_ptr->num_trans;
+ retval |= PNG_INFO_tRNS;
+ }
+ }
+ return (retval);
+}
+#endif
+
+#ifdef PNG_UNKNOWN_CHUNKS_SUPPORTED
+png_uint_32 PNGAPI
+png_get_unknown_chunks(png_structp png_ptr, png_infop info_ptr,
+ png_unknown_chunkpp unknowns)
+{
+ if (png_ptr != NULL && info_ptr != NULL && unknowns != NULL)
+ {
+ *unknowns = info_ptr->unknown_chunks;
+ return ((png_uint_32)info_ptr->unknown_chunks_num);
+ }
+ return (0);
+}
+#endif
+
+#ifdef PNG_READ_RGB_TO_GRAY_SUPPORTED
+png_byte PNGAPI
+png_get_rgb_to_gray_status (png_structp png_ptr)
+{
+ return (png_byte)(png_ptr? png_ptr->rgb_to_gray_status : 0);
+}
+#endif
+
+#ifdef PNG_USER_CHUNKS_SUPPORTED
+png_voidp PNGAPI
+png_get_user_chunk_ptr(png_structp png_ptr)
+{
+ return (png_ptr? png_ptr->user_chunk_ptr : NULL);
+}
+#endif
+
+png_uint_32 PNGAPI
+png_get_compression_buffer_size(png_structp png_ptr)
+{
+ return (png_uint_32)(png_ptr? png_ptr->zbuf_size : 0L);
+}
+
+#ifdef PNG_ASSEMBLER_CODE_SUPPORTED
+#ifndef PNG_1_0_X
+/* This function was added to libpng 1.2.0 and should exist by default */
+png_uint_32 PNGAPI
+png_get_asm_flags (png_structp png_ptr)
+{
+ /* Obsolete, to be removed from libpng-1.4.0 */
+ return (png_ptr? 0L: 0L);
+}
+
+/* This function was added to libpng 1.2.0 and should exist by default */
+png_uint_32 PNGAPI
+png_get_asm_flagmask (int flag_select)
+{
+ /* Obsolete, to be removed from libpng-1.4.0 */
+ flag_select=flag_select;
+ return 0L;
+}
+
+ /* GRR: could add this: && defined(PNG_MMX_CODE_SUPPORTED) */
+/* This function was added to libpng 1.2.0 */
+png_uint_32 PNGAPI
+png_get_mmx_flagmask (int flag_select, int *compilerID)
+{
+ /* Obsolete, to be removed from libpng-1.4.0 */
+ flag_select=flag_select;
+ *compilerID = -1; /* unknown (i.e., no asm/MMX code compiled) */
+ return 0L;
+}
+
+/* This function was added to libpng 1.2.0 */
+png_byte PNGAPI
+png_get_mmx_bitdepth_threshold (png_structp png_ptr)
+{
+ /* Obsolete, to be removed from libpng-1.4.0 */
+ return (png_ptr? 0: 0);
+}
+
+/* This function was added to libpng 1.2.0 */
+png_uint_32 PNGAPI
+png_get_mmx_rowbytes_threshold (png_structp png_ptr)
+{
+ /* Obsolete, to be removed from libpng-1.4.0 */
+ return (png_ptr? 0L: 0L);
+}
+#endif /* ?PNG_1_0_X */
+#endif /* ?PNG_ASSEMBLER_CODE_SUPPORTED */
+
+#ifdef PNG_SET_USER_LIMITS_SUPPORTED
+/* These functions were added to libpng 1.2.6 but not enabled
+* by default. They will be enabled in libpng-1.4.0 */
+png_uint_32 PNGAPI
+png_get_user_width_max (png_structp png_ptr)
+{
+ return (png_ptr? png_ptr->user_width_max : 0);
+}
+png_uint_32 PNGAPI
+png_get_user_height_max (png_structp png_ptr)
+{
+ return (png_ptr? png_ptr->user_height_max : 0);
+}
+#endif /* ?PNG_SET_USER_LIMITS_SUPPORTED */
+
+#endif /* PNG_READ_SUPPORTED || PNG_WRITE_SUPPORTED */
diff --git a/trunk/src/third_party/libpng/pngmem.c b/trunk/src/third_party/libpng/pngmem.c
new file mode 100644
index 0000000..91f2765
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngmem.c
@@ -0,0 +1,641 @@
+
+/* pngmem.c - stub functions for memory allocation
+ *
+ * Last changed in libpng 1.2.41 [February 25, 2010]
+ * Copyright (c) 1998-2010 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ *
+ * This file provides a location for all memory allocation. Users who
+ * need special memory handling are expected to supply replacement
+ * functions for png_malloc() and png_free(), and to use
+ * png_create_read_struct_2() and png_create_write_struct_2() to
+ * identify the replacement functions.
+ */
+
+#define PNG_INTERNAL
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+#if defined(PNG_READ_SUPPORTED) || defined(PNG_WRITE_SUPPORTED)
+
+/* Borland DOS special memory handler */
+#if defined(__TURBOC__) && !defined(_Windows) && !defined(__FLAT__)
+/* If you change this, be sure to change the one in png.h also */
+
+/* Allocate memory for a png_struct. The malloc and memset can be replaced
+ by a single call to calloc() if this is thought to improve performance. */
+png_voidp /* PRIVATE */
+png_create_struct(int type)
+{
+#ifdef PNG_USER_MEM_SUPPORTED
+ return (png_create_struct_2(type, png_malloc_ptr_NULL, png_voidp_NULL));
+}
+
+/* Alternate version of png_create_struct, for use with user-defined malloc. */
+png_voidp /* PRIVATE */
+png_create_struct_2(int type, png_malloc_ptr malloc_fn, png_voidp mem_ptr)
+{
+#endif /* PNG_USER_MEM_SUPPORTED */
+ png_size_t size;
+ png_voidp struct_ptr;
+
+ if (type == PNG_STRUCT_INFO)
+ size = png_sizeof(png_info);
+ else if (type == PNG_STRUCT_PNG)
+ size = png_sizeof(png_struct);
+ else
+ return (png_get_copyright(NULL));
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ if (malloc_fn != NULL)
+ {
+ png_struct dummy_struct;
+ png_structp png_ptr = &dummy_struct;
+ png_ptr->mem_ptr=mem_ptr;
+ struct_ptr = (*(malloc_fn))(png_ptr, (png_uint_32)size);
+ }
+ else
+#endif /* PNG_USER_MEM_SUPPORTED */
+ struct_ptr = (png_voidp)farmalloc(size);
+ if (struct_ptr != NULL)
+ png_memset(struct_ptr, 0, size);
+ return (struct_ptr);
+}
+
+/* Free memory allocated by a png_create_struct() call */
+void /* PRIVATE */
+png_destroy_struct(png_voidp struct_ptr)
+{
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_destroy_struct_2(struct_ptr, png_free_ptr_NULL, png_voidp_NULL);
+}
+
+/* Free memory allocated by a png_create_struct() call */
+void /* PRIVATE */
+png_destroy_struct_2(png_voidp struct_ptr, png_free_ptr free_fn,
+ png_voidp mem_ptr)
+{
+#endif
+ if (struct_ptr != NULL)
+ {
+#ifdef PNG_USER_MEM_SUPPORTED
+ if (free_fn != NULL)
+ {
+ png_struct dummy_struct;
+ png_structp png_ptr = &dummy_struct;
+ png_ptr->mem_ptr=mem_ptr;
+ (*(free_fn))(png_ptr, struct_ptr);
+ return;
+ }
+#endif /* PNG_USER_MEM_SUPPORTED */
+ farfree (struct_ptr);
+ }
+}
+
+/* Allocate memory. For reasonable files, size should never exceed
+ * 64K. However, zlib may allocate more then 64K if you don't tell
+ * it not to. See zconf.h and png.h for more information. zlib does
+ * need to allocate exactly 64K, so whatever you call here must
+ * have the ability to do that.
+ *
+ * Borland seems to have a problem in DOS mode for exactly 64K.
+ * It gives you a segment with an offset of 8 (perhaps to store its
+ * memory stuff). zlib doesn't like this at all, so we have to
+ * detect and deal with it. This code should not be needed in
+ * Windows or OS/2 modes, and only in 16 bit mode. This code has
+ * been updated by Alexander Lehmann for version 0.89 to waste less
+ * memory.
+ *
+ * Note that we can't use png_size_t for the "size" declaration,
+ * since on some systems a png_size_t is a 16-bit quantity, and as a
+ * result, we would be truncating potentially larger memory requests
+ * (which should cause a fatal error) and introducing major problems.
+ */
+png_voidp /* PRIVATE */
+png_calloc(png_structp png_ptr, png_uint_32 size)
+{
+ png_voidp ret;
+
+ ret = (png_malloc(png_ptr, size));
+ if (ret != NULL)
+ png_memset(ret,0,(png_size_t)size);
+ return (ret);
+}
+
+png_voidp PNGAPI
+png_malloc(png_structp png_ptr, png_uint_32 size)
+{
+ png_voidp ret;
+
+ if (png_ptr == NULL || size == 0)
+ return (NULL);
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ if (png_ptr->malloc_fn != NULL)
+ ret = ((png_voidp)(*(png_ptr->malloc_fn))(png_ptr, (png_size_t)size));
+ else
+ ret = (png_malloc_default(png_ptr, size));
+ if (ret == NULL && (png_ptr->flags&PNG_FLAG_MALLOC_NULL_MEM_OK) == 0)
+ png_error(png_ptr, "Out of memory!");
+ return (ret);
+}
+
+png_voidp PNGAPI
+png_malloc_default(png_structp png_ptr, png_uint_32 size)
+{
+ png_voidp ret;
+#endif /* PNG_USER_MEM_SUPPORTED */
+
+ if (png_ptr == NULL || size == 0)
+ return (NULL);
+
+#ifdef PNG_MAX_MALLOC_64K
+ if (size > (png_uint_32)65536L)
+ {
+ png_warning(png_ptr, "Cannot Allocate > 64K");
+ ret = NULL;
+ }
+ else
+#endif
+
+ if (size != (size_t)size)
+ ret = NULL;
+ else if (size == (png_uint_32)65536L)
+ {
+ if (png_ptr->offset_table == NULL)
+ {
+ /* Try to see if we need to do any of this fancy stuff */
+ ret = farmalloc(size);
+ if (ret == NULL || ((png_size_t)ret & 0xffff))
+ {
+ int num_blocks;
+ png_uint_32 total_size;
+ png_bytep table;
+ int i;
+ png_byte huge * hptr;
+
+ if (ret != NULL)
+ {
+ farfree(ret);
+ ret = NULL;
+ }
+
+ if (png_ptr->zlib_window_bits > 14)
+ num_blocks = (int)(1 << (png_ptr->zlib_window_bits - 14));
+ else
+ num_blocks = 1;
+ if (png_ptr->zlib_mem_level >= 7)
+ num_blocks += (int)(1 << (png_ptr->zlib_mem_level - 7));
+ else
+ num_blocks++;
+
+ total_size = ((png_uint_32)65536L) * (png_uint_32)num_blocks+16;
+
+ table = farmalloc(total_size);
+
+ if (table == NULL)
+ {
+#ifndef PNG_USER_MEM_SUPPORTED
+ if ((png_ptr->flags&PNG_FLAG_MALLOC_NULL_MEM_OK) == 0)
+ png_error(png_ptr, "Out Of Memory."); /* Note "O", "M" */
+ else
+ png_warning(png_ptr, "Out Of Memory.");
+#endif
+ return (NULL);
+ }
+
+ if ((png_size_t)table & 0xfff0)
+ {
+#ifndef PNG_USER_MEM_SUPPORTED
+ if ((png_ptr->flags&PNG_FLAG_MALLOC_NULL_MEM_OK) == 0)
+ png_error(png_ptr,
+ "Farmalloc didn't return normalized pointer");
+ else
+ png_warning(png_ptr,
+ "Farmalloc didn't return normalized pointer");
+#endif
+ return (NULL);
+ }
+
+ png_ptr->offset_table = table;
+ png_ptr->offset_table_ptr = farmalloc(num_blocks *
+ png_sizeof(png_bytep));
+
+ if (png_ptr->offset_table_ptr == NULL)
+ {
+#ifndef PNG_USER_MEM_SUPPORTED
+ if ((png_ptr->flags&PNG_FLAG_MALLOC_NULL_MEM_OK) == 0)
+ png_error(png_ptr, "Out Of memory."); /* Note "O", "m" */
+ else
+ png_warning(png_ptr, "Out Of memory.");
+#endif
+ return (NULL);
+ }
+
+ hptr = (png_byte huge *)table;
+ if ((png_size_t)hptr & 0xf)
+ {
+ hptr = (png_byte huge *)((long)(hptr) & 0xfffffff0L);
+ hptr = hptr + 16L; /* "hptr += 16L" fails on Turbo C++ 3.0 */
+ }
+ for (i = 0; i < num_blocks; i++)
+ {
+ png_ptr->offset_table_ptr[i] = (png_bytep)hptr;
+ hptr = hptr + (png_uint_32)65536L; /* "+=" fails on TC++3.0 */
+ }
+
+ png_ptr->offset_table_number = num_blocks;
+ png_ptr->offset_table_count = 0;
+ png_ptr->offset_table_count_free = 0;
+ }
+ }
+
+ if (png_ptr->offset_table_count >= png_ptr->offset_table_number)
+ {
+#ifndef PNG_USER_MEM_SUPPORTED
+ if ((png_ptr->flags&PNG_FLAG_MALLOC_NULL_MEM_OK) == 0)
+ png_error(png_ptr, "Out of Memory."); /* Note "o" and "M" */
+ else
+ png_warning(png_ptr, "Out of Memory.");
+#endif
+ return (NULL);
+ }
+
+ ret = png_ptr->offset_table_ptr[png_ptr->offset_table_count++];
+ }
+ else
+ ret = farmalloc(size);
+
+#ifndef PNG_USER_MEM_SUPPORTED
+ if (ret == NULL)
+ {
+ if ((png_ptr->flags&PNG_FLAG_MALLOC_NULL_MEM_OK) == 0)
+ png_error(png_ptr, "Out of memory."); /* Note "o" and "m" */
+ else
+ png_warning(png_ptr, "Out of memory."); /* Note "o" and "m" */
+ }
+#endif
+
+ return (ret);
+}
+
+/* Free a pointer allocated by png_malloc(). In the default
+ * configuration, png_ptr is not used, but is passed in case it
+ * is needed. If ptr is NULL, return without taking any action.
+ */
+void PNGAPI
+png_free(png_structp png_ptr, png_voidp ptr)
+{
+ if (png_ptr == NULL || ptr == NULL)
+ return;
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ if (png_ptr->free_fn != NULL)
+ {
+ (*(png_ptr->free_fn))(png_ptr, ptr);
+ return;
+ }
+ else
+ png_free_default(png_ptr, ptr);
+}
+
+void PNGAPI
+png_free_default(png_structp png_ptr, png_voidp ptr)
+{
+#endif /* PNG_USER_MEM_SUPPORTED */
+
+ if (png_ptr == NULL || ptr == NULL)
+ return;
+
+ if (png_ptr->offset_table != NULL)
+ {
+ int i;
+
+ for (i = 0; i < png_ptr->offset_table_count; i++)
+ {
+ if (ptr == png_ptr->offset_table_ptr[i])
+ {
+ ptr = NULL;
+ png_ptr->offset_table_count_free++;
+ break;
+ }
+ }
+ if (png_ptr->offset_table_count_free == png_ptr->offset_table_count)
+ {
+ farfree(png_ptr->offset_table);
+ farfree(png_ptr->offset_table_ptr);
+ png_ptr->offset_table = NULL;
+ png_ptr->offset_table_ptr = NULL;
+ }
+ }
+
+ if (ptr != NULL)
+ {
+ farfree(ptr);
+ }
+}
+
+#else /* Not the Borland DOS special memory handler */
+
+/* Allocate memory for a png_struct or a png_info. The malloc and
+ memset can be replaced by a single call to calloc() if this is thought
+ to improve performance noticably. */
+png_voidp /* PRIVATE */
+png_create_struct(int type)
+{
+#ifdef PNG_USER_MEM_SUPPORTED
+ return (png_create_struct_2(type, png_malloc_ptr_NULL, png_voidp_NULL));
+}
+
+/* Allocate memory for a png_struct or a png_info. The malloc and
+ memset can be replaced by a single call to calloc() if this is thought
+ to improve performance noticably. */
+png_voidp /* PRIVATE */
+png_create_struct_2(int type, png_malloc_ptr malloc_fn, png_voidp mem_ptr)
+{
+#endif /* PNG_USER_MEM_SUPPORTED */
+ png_size_t size;
+ png_voidp struct_ptr;
+
+ if (type == PNG_STRUCT_INFO)
+ size = png_sizeof(png_info);
+ else if (type == PNG_STRUCT_PNG)
+ size = png_sizeof(png_struct);
+ else
+ return (NULL);
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ if (malloc_fn != NULL)
+ {
+ png_struct dummy_struct;
+ png_structp png_ptr = &dummy_struct;
+ png_ptr->mem_ptr=mem_ptr;
+ struct_ptr = (*(malloc_fn))(png_ptr, size);
+ if (struct_ptr != NULL)
+ png_memset(struct_ptr, 0, size);
+ return (struct_ptr);
+ }
+#endif /* PNG_USER_MEM_SUPPORTED */
+
+#if defined(__TURBOC__) && !defined(__FLAT__)
+ struct_ptr = (png_voidp)farmalloc(size);
+#else
+# if defined(_MSC_VER) && defined(MAXSEG_64K)
+ struct_ptr = (png_voidp)halloc(size, 1);
+# else
+ struct_ptr = (png_voidp)malloc(size);
+# endif
+#endif
+ if (struct_ptr != NULL)
+ png_memset(struct_ptr, 0, size);
+
+ return (struct_ptr);
+}
+
+
+/* Free memory allocated by a png_create_struct() call */
+void /* PRIVATE */
+png_destroy_struct(png_voidp struct_ptr)
+{
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_destroy_struct_2(struct_ptr, png_free_ptr_NULL, png_voidp_NULL);
+}
+
+/* Free memory allocated by a png_create_struct() call */
+void /* PRIVATE */
+png_destroy_struct_2(png_voidp struct_ptr, png_free_ptr free_fn,
+ png_voidp mem_ptr)
+{
+#endif /* PNG_USER_MEM_SUPPORTED */
+ if (struct_ptr != NULL)
+ {
+#ifdef PNG_USER_MEM_SUPPORTED
+ if (free_fn != NULL)
+ {
+ png_struct dummy_struct;
+ png_structp png_ptr = &dummy_struct;
+ png_ptr->mem_ptr=mem_ptr;
+ (*(free_fn))(png_ptr, struct_ptr);
+ return;
+ }
+#endif /* PNG_USER_MEM_SUPPORTED */
+#if defined(__TURBOC__) && !defined(__FLAT__)
+ farfree(struct_ptr);
+#else
+# if defined(_MSC_VER) && defined(MAXSEG_64K)
+ hfree(struct_ptr);
+# else
+ free(struct_ptr);
+# endif
+#endif
+ }
+}
+
+/* Allocate memory. For reasonable files, size should never exceed
+ * 64K. However, zlib may allocate more then 64K if you don't tell
+ * it not to. See zconf.h and png.h for more information. zlib does
+ * need to allocate exactly 64K, so whatever you call here must
+ * have the ability to do that.
+ */
+
+png_voidp PNGAPI
+png_calloc(png_structp png_ptr, png_uint_32 size)
+{
+ png_voidp ret;
+
+ ret = (png_malloc(png_ptr, size));
+ if (ret != NULL)
+ png_memset(ret,0,(png_size_t)size);
+ return (ret);
+}
+
+png_voidp PNGAPI
+png_malloc(png_structp png_ptr, png_uint_32 size)
+{
+ png_voidp ret;
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ if (png_ptr == NULL || size == 0)
+ return (NULL);
+
+ if (png_ptr->malloc_fn != NULL)
+ ret = ((png_voidp)(*(png_ptr->malloc_fn))(png_ptr, (png_size_t)size));
+ else
+ ret = (png_malloc_default(png_ptr, size));
+ if (ret == NULL && (png_ptr->flags&PNG_FLAG_MALLOC_NULL_MEM_OK) == 0)
+ png_error(png_ptr, "Out of Memory!");
+ return (ret);
+}
+
+png_voidp PNGAPI
+png_malloc_default(png_structp png_ptr, png_uint_32 size)
+{
+ png_voidp ret;
+#endif /* PNG_USER_MEM_SUPPORTED */
+
+ if (png_ptr == NULL || size == 0)
+ return (NULL);
+
+#ifdef PNG_MAX_MALLOC_64K
+ if (size > (png_uint_32)65536L)
+ {
+#ifndef PNG_USER_MEM_SUPPORTED
+ if ((png_ptr->flags&PNG_FLAG_MALLOC_NULL_MEM_OK) == 0)
+ png_error(png_ptr, "Cannot Allocate > 64K");
+ else
+#endif
+ return NULL;
+ }
+#endif
+
+ /* Check for overflow */
+#if defined(__TURBOC__) && !defined(__FLAT__)
+ if (size != (unsigned long)size)
+ ret = NULL;
+ else
+ ret = farmalloc(size);
+#else
+# if defined(_MSC_VER) && defined(MAXSEG_64K)
+ if (size != (unsigned long)size)
+ ret = NULL;
+ else
+ ret = halloc(size, 1);
+# else
+ if (size != (size_t)size)
+ ret = NULL;
+ else
+ ret = malloc((size_t)size);
+# endif
+#endif
+
+#ifndef PNG_USER_MEM_SUPPORTED
+ if (ret == NULL && (png_ptr->flags&PNG_FLAG_MALLOC_NULL_MEM_OK) == 0)
+ png_error(png_ptr, "Out of Memory");
+#endif
+
+ return (ret);
+}
+
+/* Free a pointer allocated by png_malloc(). If ptr is NULL, return
+ * without taking any action.
+ */
+void PNGAPI
+png_free(png_structp png_ptr, png_voidp ptr)
+{
+ if (png_ptr == NULL || ptr == NULL)
+ return;
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ if (png_ptr->free_fn != NULL)
+ {
+ (*(png_ptr->free_fn))(png_ptr, ptr);
+ return;
+ }
+ else
+ png_free_default(png_ptr, ptr);
+}
+void PNGAPI
+png_free_default(png_structp png_ptr, png_voidp ptr)
+{
+ if (png_ptr == NULL || ptr == NULL)
+ return;
+
+#endif /* PNG_USER_MEM_SUPPORTED */
+
+#if defined(__TURBOC__) && !defined(__FLAT__)
+ farfree(ptr);
+#else
+# if defined(_MSC_VER) && defined(MAXSEG_64K)
+ hfree(ptr);
+# else
+ free(ptr);
+# endif
+#endif
+}
+
+#endif /* Not Borland DOS special memory handler */
+
+#ifdef PNG_1_0_X
+# define png_malloc_warn png_malloc
+#else
+/* This function was added at libpng version 1.2.3. The png_malloc_warn()
+ * function will set up png_malloc() to issue a png_warning and return NULL
+ * instead of issuing a png_error, if it fails to allocate the requested
+ * memory.
+ */
+png_voidp PNGAPI
+png_malloc_warn(png_structp png_ptr, png_uint_32 size)
+{
+ png_voidp ptr;
+ png_uint_32 save_flags;
+ if (png_ptr == NULL)
+ return (NULL);
+
+ save_flags = png_ptr->flags;
+ png_ptr->flags|=PNG_FLAG_MALLOC_NULL_MEM_OK;
+ ptr = (png_voidp)png_malloc((png_structp)png_ptr, size);
+ png_ptr->flags=save_flags;
+ return(ptr);
+}
+#endif
+
+png_voidp PNGAPI
+png_memcpy_check (png_structp png_ptr, png_voidp s1, png_voidp s2,
+ png_uint_32 length)
+{
+ png_size_t size;
+
+ size = (png_size_t)length;
+ if ((png_uint_32)size != length)
+ png_error(png_ptr, "Overflow in png_memcpy_check.");
+
+ return(png_memcpy (s1, s2, size));
+}
+
+png_voidp PNGAPI
+png_memset_check (png_structp png_ptr, png_voidp s1, int value,
+ png_uint_32 length)
+{
+ png_size_t size;
+
+ size = (png_size_t)length;
+ if ((png_uint_32)size != length)
+ png_error(png_ptr, "Overflow in png_memset_check.");
+
+ return (png_memset (s1, value, size));
+
+}
+
+#ifdef PNG_USER_MEM_SUPPORTED
+/* This function is called when the application wants to use another method
+ * of allocating and freeing memory.
+ */
+void PNGAPI
+png_set_mem_fn(png_structp png_ptr, png_voidp mem_ptr, png_malloc_ptr
+ malloc_fn, png_free_ptr free_fn)
+{
+ if (png_ptr != NULL)
+ {
+ png_ptr->mem_ptr = mem_ptr;
+ png_ptr->malloc_fn = malloc_fn;
+ png_ptr->free_fn = free_fn;
+ }
+}
+
+/* This function returns a pointer to the mem_ptr associated with the user
+ * functions. The application should free any memory associated with this
+ * pointer before png_write_destroy and png_read_destroy are called.
+ */
+png_voidp PNGAPI
+png_get_mem_ptr(png_structp png_ptr)
+{
+ if (png_ptr == NULL)
+ return (NULL);
+ return ((png_voidp)png_ptr->mem_ptr);
+}
+#endif /* PNG_USER_MEM_SUPPORTED */
+#endif /* PNG_READ_SUPPORTED || PNG_WRITE_SUPPORTED */
diff --git a/trunk/src/third_party/libpng/pngpread.c b/trunk/src/third_party/libpng/pngpread.c
new file mode 100644
index 0000000..d066944
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngpread.c
@@ -0,0 +1,1774 @@
+
+/* pngpread.c - read a png file in push mode
+ *
+ * Last changed in libpng 1.2.44 [June 26, 2010]
+ * Copyright (c) 1998-2010 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ */
+
+#define PNG_INTERNAL
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+#ifdef PNG_PROGRESSIVE_READ_SUPPORTED
+
+/* Push model modes */
+#define PNG_READ_SIG_MODE 0
+#define PNG_READ_CHUNK_MODE 1
+#define PNG_READ_IDAT_MODE 2
+#define PNG_SKIP_MODE 3
+#define PNG_READ_tEXt_MODE 4
+#define PNG_READ_zTXt_MODE 5
+#define PNG_READ_DONE_MODE 6
+#define PNG_READ_iTXt_MODE 7
+#define PNG_ERROR_MODE 8
+
+void PNGAPI
+png_process_data(png_structp png_ptr, png_infop info_ptr,
+ png_bytep buffer, png_size_t buffer_size)
+{
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ png_push_restore_buffer(png_ptr, buffer, buffer_size);
+
+ while (png_ptr->buffer_size)
+ {
+ png_process_some_data(png_ptr, info_ptr);
+ }
+}
+
+/* What we do with the incoming data depends on what we were previously
+ * doing before we ran out of data...
+ */
+void /* PRIVATE */
+png_process_some_data(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr == NULL)
+ return;
+
+ switch (png_ptr->process_mode)
+ {
+ case PNG_READ_SIG_MODE:
+ {
+ png_push_read_sig(png_ptr, info_ptr);
+ break;
+ }
+
+ case PNG_READ_CHUNK_MODE:
+ {
+ png_push_read_chunk(png_ptr, info_ptr);
+ break;
+ }
+
+ case PNG_READ_IDAT_MODE:
+ {
+ png_push_read_IDAT(png_ptr);
+ break;
+ }
+
+#ifdef PNG_READ_tEXt_SUPPORTED
+ case PNG_READ_tEXt_MODE:
+ {
+ png_push_read_tEXt(png_ptr, info_ptr);
+ break;
+ }
+
+#endif
+#ifdef PNG_READ_zTXt_SUPPORTED
+ case PNG_READ_zTXt_MODE:
+ {
+ png_push_read_zTXt(png_ptr, info_ptr);
+ break;
+ }
+
+#endif
+#ifdef PNG_READ_iTXt_SUPPORTED
+ case PNG_READ_iTXt_MODE:
+ {
+ png_push_read_iTXt(png_ptr, info_ptr);
+ break;
+ }
+
+#endif
+ case PNG_SKIP_MODE:
+ {
+ png_push_crc_finish(png_ptr);
+ break;
+ }
+
+ default:
+ {
+ png_ptr->buffer_size = 0;
+ break;
+ }
+ }
+}
+
+/* Read any remaining signature bytes from the stream and compare them with
+ * the correct PNG signature. It is possible that this routine is called
+ * with bytes already read from the signature, either because they have been
+ * checked by the calling application, or because of multiple calls to this
+ * routine.
+ */
+void /* PRIVATE */
+png_push_read_sig(png_structp png_ptr, png_infop info_ptr)
+{
+ png_size_t num_checked = png_ptr->sig_bytes,
+ num_to_check = 8 - num_checked;
+
+ if (png_ptr->buffer_size < num_to_check)
+ {
+ num_to_check = png_ptr->buffer_size;
+ }
+
+ png_push_fill_buffer(png_ptr, &(info_ptr->signature[num_checked]),
+ num_to_check);
+ png_ptr->sig_bytes = (png_byte)(png_ptr->sig_bytes + num_to_check);
+
+ if (png_sig_cmp(info_ptr->signature, num_checked, num_to_check))
+ {
+ if (num_checked < 4 &&
+ png_sig_cmp(info_ptr->signature, num_checked, num_to_check - 4))
+ png_error(png_ptr, "Not a PNG file");
+ else
+ png_error(png_ptr, "PNG file corrupted by ASCII conversion");
+ }
+ else
+ {
+ if (png_ptr->sig_bytes >= 8)
+ {
+ png_ptr->process_mode = PNG_READ_CHUNK_MODE;
+ }
+ }
+}
+
+void /* PRIVATE */
+png_push_read_chunk(png_structp png_ptr, png_infop info_ptr)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_CONST PNG_IHDR;
+ PNG_CONST PNG_IDAT;
+ PNG_CONST PNG_IEND;
+ PNG_CONST PNG_PLTE;
+#ifdef PNG_READ_bKGD_SUPPORTED
+ PNG_CONST PNG_bKGD;
+#endif
+#ifdef PNG_READ_cHRM_SUPPORTED
+ PNG_CONST PNG_cHRM;
+#endif
+#ifdef PNG_READ_gAMA_SUPPORTED
+ PNG_CONST PNG_gAMA;
+#endif
+#ifdef PNG_READ_hIST_SUPPORTED
+ PNG_CONST PNG_hIST;
+#endif
+#ifdef PNG_READ_iCCP_SUPPORTED
+ PNG_CONST PNG_iCCP;
+#endif
+#ifdef PNG_READ_iTXt_SUPPORTED
+ PNG_CONST PNG_iTXt;
+#endif
+#ifdef PNG_READ_oFFs_SUPPORTED
+ PNG_CONST PNG_oFFs;
+#endif
+#ifdef PNG_READ_pCAL_SUPPORTED
+ PNG_CONST PNG_pCAL;
+#endif
+#ifdef PNG_READ_pHYs_SUPPORTED
+ PNG_CONST PNG_pHYs;
+#endif
+#ifdef PNG_READ_sBIT_SUPPORTED
+ PNG_CONST PNG_sBIT;
+#endif
+#ifdef PNG_READ_sCAL_SUPPORTED
+ PNG_CONST PNG_sCAL;
+#endif
+#ifdef PNG_READ_sRGB_SUPPORTED
+ PNG_CONST PNG_sRGB;
+#endif
+#ifdef PNG_READ_sPLT_SUPPORTED
+ PNG_CONST PNG_sPLT;
+#endif
+#ifdef PNG_READ_tEXt_SUPPORTED
+ PNG_CONST PNG_tEXt;
+#endif
+#ifdef PNG_READ_tIME_SUPPORTED
+ PNG_CONST PNG_tIME;
+#endif
+#ifdef PNG_READ_tRNS_SUPPORTED
+ PNG_CONST PNG_tRNS;
+#endif
+#ifdef PNG_READ_zTXt_SUPPORTED
+ PNG_CONST PNG_zTXt;
+#endif
+#endif /* PNG_USE_LOCAL_ARRAYS */
+
+ /* First we make sure we have enough data for the 4 byte chunk name
+ * and the 4 byte chunk length before proceeding with decoding the
+ * chunk data. To fully decode each of these chunks, we also make
+ * sure we have enough data in the buffer for the 4 byte CRC at the
+ * end of every chunk (except IDAT, which is handled separately).
+ */
+ if (!(png_ptr->mode & PNG_HAVE_CHUNK_HEADER))
+ {
+ png_byte chunk_length[4];
+
+ if (png_ptr->buffer_size < 8)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_push_fill_buffer(png_ptr, chunk_length, 4);
+ png_ptr->push_length = png_get_uint_31(png_ptr, chunk_length);
+ png_reset_crc(png_ptr);
+ png_crc_read(png_ptr, png_ptr->chunk_name, 4);
+ png_check_chunk_name(png_ptr, png_ptr->chunk_name);
+ png_ptr->mode |= PNG_HAVE_CHUNK_HEADER;
+ }
+
+ if (!png_memcmp(png_ptr->chunk_name, png_IDAT, 4))
+ if (png_ptr->mode & PNG_AFTER_IDAT)
+ png_ptr->mode |= PNG_HAVE_CHUNK_AFTER_IDAT;
+
+ if (!png_memcmp(png_ptr->chunk_name, png_IHDR, 4))
+ {
+ if (png_ptr->push_length != 13)
+ png_error(png_ptr, "Invalid IHDR length");
+
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_IHDR(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+ else if (!png_memcmp(png_ptr->chunk_name, png_IEND, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_IEND(png_ptr, info_ptr, png_ptr->push_length);
+
+ png_ptr->process_mode = PNG_READ_DONE_MODE;
+ png_push_have_end(png_ptr, info_ptr);
+ }
+
+#ifdef PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+ else if (png_handle_as_unknown(png_ptr, png_ptr->chunk_name))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ if (!png_memcmp(png_ptr->chunk_name, png_IDAT, 4))
+ png_ptr->mode |= PNG_HAVE_IDAT;
+
+ png_handle_unknown(png_ptr, info_ptr, png_ptr->push_length);
+
+ if (!png_memcmp(png_ptr->chunk_name, png_PLTE, 4))
+ png_ptr->mode |= PNG_HAVE_PLTE;
+
+ else if (!png_memcmp(png_ptr->chunk_name, png_IDAT, 4))
+ {
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before IDAT");
+
+ else if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE &&
+ !(png_ptr->mode & PNG_HAVE_PLTE))
+ png_error(png_ptr, "Missing PLTE before IDAT");
+ }
+ }
+
+#endif
+ else if (!png_memcmp(png_ptr->chunk_name, png_PLTE, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+ png_handle_PLTE(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+ else if (!png_memcmp(png_ptr->chunk_name, png_IDAT, 4))
+ {
+ /* If we reach an IDAT chunk, this means we have read all of the
+ * header chunks, and we can start reading the image (or if this
+ * is called after the image has been read - we have an error).
+ */
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before IDAT");
+
+ else if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE &&
+ !(png_ptr->mode & PNG_HAVE_PLTE))
+ png_error(png_ptr, "Missing PLTE before IDAT");
+
+ if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ if (!(png_ptr->mode & PNG_HAVE_CHUNK_AFTER_IDAT))
+ if (png_ptr->push_length == 0)
+ return;
+
+ if (png_ptr->mode & PNG_AFTER_IDAT)
+ png_error(png_ptr, "Too many IDAT's found");
+ }
+
+ png_ptr->idat_size = png_ptr->push_length;
+ png_ptr->mode |= PNG_HAVE_IDAT;
+ png_ptr->process_mode = PNG_READ_IDAT_MODE;
+ png_push_have_info(png_ptr, info_ptr);
+ png_ptr->zstream.avail_out =
+ (uInt) PNG_ROWBYTES(png_ptr->pixel_depth,
+ png_ptr->iwidth) + 1;
+ png_ptr->zstream.next_out = png_ptr->row_buf;
+ return;
+ }
+
+#ifdef PNG_READ_gAMA_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_gAMA, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_gAMA(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_sBIT_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_sBIT, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_sBIT(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_cHRM_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_cHRM, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_cHRM(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_sRGB_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_sRGB, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_sRGB(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_iCCP_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_iCCP, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_iCCP(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_sPLT_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_sPLT, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_sPLT(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_tRNS_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_tRNS, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_tRNS(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_bKGD_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_bKGD, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_bKGD(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_hIST_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_hIST, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_hIST(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_pHYs_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_pHYs, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_pHYs(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_oFFs_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_oFFs, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_oFFs(png_ptr, info_ptr, png_ptr->push_length);
+ }
+#endif
+
+#ifdef PNG_READ_pCAL_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_pCAL, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_pCAL(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_sCAL_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_sCAL, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_sCAL(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_tIME_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_tIME, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_handle_tIME(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_tEXt_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_tEXt, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_push_handle_tEXt(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_zTXt_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_zTXt, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_push_handle_zTXt(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+#ifdef PNG_READ_iTXt_SUPPORTED
+ else if (!png_memcmp(png_ptr->chunk_name, png_iTXt, 4))
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_push_handle_iTXt(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+#endif
+ else
+ {
+ if (png_ptr->push_length + 4 > png_ptr->buffer_size)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+ png_push_handle_unknown(png_ptr, info_ptr, png_ptr->push_length);
+ }
+
+ png_ptr->mode &= ~PNG_HAVE_CHUNK_HEADER;
+}
+
+void /* PRIVATE */
+png_push_crc_skip(png_structp png_ptr, png_uint_32 skip)
+{
+ png_ptr->process_mode = PNG_SKIP_MODE;
+ png_ptr->skip_length = skip;
+}
+
+void /* PRIVATE */
+png_push_crc_finish(png_structp png_ptr)
+{
+ if (png_ptr->skip_length && png_ptr->save_buffer_size)
+ {
+ png_size_t save_size;
+
+ if (png_ptr->skip_length < (png_uint_32)png_ptr->save_buffer_size)
+ save_size = (png_size_t)png_ptr->skip_length;
+ else
+ save_size = png_ptr->save_buffer_size;
+
+ png_calculate_crc(png_ptr, png_ptr->save_buffer_ptr, save_size);
+
+ png_ptr->skip_length -= save_size;
+ png_ptr->buffer_size -= save_size;
+ png_ptr->save_buffer_size -= save_size;
+ png_ptr->save_buffer_ptr += save_size;
+ }
+ if (png_ptr->skip_length && png_ptr->current_buffer_size)
+ {
+ png_size_t save_size;
+
+ if (png_ptr->skip_length < (png_uint_32)png_ptr->current_buffer_size)
+ save_size = (png_size_t)png_ptr->skip_length;
+ else
+ save_size = png_ptr->current_buffer_size;
+
+ png_calculate_crc(png_ptr, png_ptr->current_buffer_ptr, save_size);
+
+ png_ptr->skip_length -= save_size;
+ png_ptr->buffer_size -= save_size;
+ png_ptr->current_buffer_size -= save_size;
+ png_ptr->current_buffer_ptr += save_size;
+ }
+ if (!png_ptr->skip_length)
+ {
+ if (png_ptr->buffer_size < 4)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_crc_finish(png_ptr, 0);
+ png_ptr->process_mode = PNG_READ_CHUNK_MODE;
+ }
+}
+
+void PNGAPI
+png_push_fill_buffer(png_structp png_ptr, png_bytep buffer, png_size_t length)
+{
+ png_bytep ptr;
+
+ if (png_ptr == NULL)
+ return;
+
+ ptr = buffer;
+ if (png_ptr->save_buffer_size)
+ {
+ png_size_t save_size;
+
+ if (length < png_ptr->save_buffer_size)
+ save_size = length;
+ else
+ save_size = png_ptr->save_buffer_size;
+
+ png_memcpy(ptr, png_ptr->save_buffer_ptr, save_size);
+ length -= save_size;
+ ptr += save_size;
+ png_ptr->buffer_size -= save_size;
+ png_ptr->save_buffer_size -= save_size;
+ png_ptr->save_buffer_ptr += save_size;
+ }
+ if (length && png_ptr->current_buffer_size)
+ {
+ png_size_t save_size;
+
+ if (length < png_ptr->current_buffer_size)
+ save_size = length;
+
+ else
+ save_size = png_ptr->current_buffer_size;
+
+ png_memcpy(ptr, png_ptr->current_buffer_ptr, save_size);
+ png_ptr->buffer_size -= save_size;
+ png_ptr->current_buffer_size -= save_size;
+ png_ptr->current_buffer_ptr += save_size;
+ }
+}
+
+void /* PRIVATE */
+png_push_save_buffer(png_structp png_ptr)
+{
+ if (png_ptr->save_buffer_size)
+ {
+ if (png_ptr->save_buffer_ptr != png_ptr->save_buffer)
+ {
+ png_size_t i, istop;
+ png_bytep sp;
+ png_bytep dp;
+
+ istop = png_ptr->save_buffer_size;
+ for (i = 0, sp = png_ptr->save_buffer_ptr, dp = png_ptr->save_buffer;
+ i < istop; i++, sp++, dp++)
+ {
+ *dp = *sp;
+ }
+ }
+ }
+ if (png_ptr->save_buffer_size + png_ptr->current_buffer_size >
+ png_ptr->save_buffer_max)
+ {
+ png_size_t new_max;
+ png_bytep old_buffer;
+
+ if (png_ptr->save_buffer_size > PNG_SIZE_MAX -
+ (png_ptr->current_buffer_size + 256))
+ {
+ png_error(png_ptr, "Potential overflow of save_buffer");
+ }
+
+ new_max = png_ptr->save_buffer_size + png_ptr->current_buffer_size + 256;
+ old_buffer = png_ptr->save_buffer;
+ png_ptr->save_buffer = (png_bytep)png_malloc_warn(png_ptr,
+ (png_uint_32)new_max);
+ if (png_ptr->save_buffer == NULL)
+ {
+ png_free(png_ptr, old_buffer);
+ png_error(png_ptr, "Insufficient memory for save_buffer");
+ }
+ png_memcpy(png_ptr->save_buffer, old_buffer, png_ptr->save_buffer_size);
+ png_free(png_ptr, old_buffer);
+ png_ptr->save_buffer_max = new_max;
+ }
+ if (png_ptr->current_buffer_size)
+ {
+ png_memcpy(png_ptr->save_buffer + png_ptr->save_buffer_size,
+ png_ptr->current_buffer_ptr, png_ptr->current_buffer_size);
+ png_ptr->save_buffer_size += png_ptr->current_buffer_size;
+ png_ptr->current_buffer_size = 0;
+ }
+ png_ptr->save_buffer_ptr = png_ptr->save_buffer;
+ png_ptr->buffer_size = 0;
+}
+
+void /* PRIVATE */
+png_push_restore_buffer(png_structp png_ptr, png_bytep buffer,
+ png_size_t buffer_length)
+{
+ png_ptr->current_buffer = buffer;
+ png_ptr->current_buffer_size = buffer_length;
+ png_ptr->buffer_size = buffer_length + png_ptr->save_buffer_size;
+ png_ptr->current_buffer_ptr = png_ptr->current_buffer;
+}
+
+void /* PRIVATE */
+png_push_read_IDAT(png_structp png_ptr)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_CONST PNG_IDAT;
+#endif
+ if (!(png_ptr->mode & PNG_HAVE_CHUNK_HEADER))
+ {
+ png_byte chunk_length[4];
+
+ if (png_ptr->buffer_size < 8)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_push_fill_buffer(png_ptr, chunk_length, 4);
+ png_ptr->push_length = png_get_uint_31(png_ptr, chunk_length);
+ png_reset_crc(png_ptr);
+ png_crc_read(png_ptr, png_ptr->chunk_name, 4);
+ png_ptr->mode |= PNG_HAVE_CHUNK_HEADER;
+
+ if (png_memcmp(png_ptr->chunk_name, png_IDAT, 4))
+ {
+ png_ptr->process_mode = PNG_READ_CHUNK_MODE;
+ if (!(png_ptr->flags & PNG_FLAG_ZLIB_FINISHED))
+ png_error(png_ptr, "Not enough compressed data");
+ return;
+ }
+
+ png_ptr->idat_size = png_ptr->push_length;
+ }
+ if (png_ptr->idat_size && png_ptr->save_buffer_size)
+ {
+ png_size_t save_size;
+
+ if (png_ptr->idat_size < (png_uint_32)png_ptr->save_buffer_size)
+ {
+ save_size = (png_size_t)png_ptr->idat_size;
+
+ /* Check for overflow */
+ if ((png_uint_32)save_size != png_ptr->idat_size)
+ png_error(png_ptr, "save_size overflowed in pngpread");
+ }
+ else
+ save_size = png_ptr->save_buffer_size;
+
+ png_calculate_crc(png_ptr, png_ptr->save_buffer_ptr, save_size);
+
+ png_process_IDAT_data(png_ptr, png_ptr->save_buffer_ptr, save_size);
+
+ png_ptr->idat_size -= save_size;
+ png_ptr->buffer_size -= save_size;
+ png_ptr->save_buffer_size -= save_size;
+ png_ptr->save_buffer_ptr += save_size;
+ }
+ if (png_ptr->idat_size && png_ptr->current_buffer_size)
+ {
+ png_size_t save_size;
+
+ if (png_ptr->idat_size < (png_uint_32)png_ptr->current_buffer_size)
+ {
+ save_size = (png_size_t)png_ptr->idat_size;
+
+ /* Check for overflow */
+ if ((png_uint_32)save_size != png_ptr->idat_size)
+ png_error(png_ptr, "save_size overflowed in pngpread");
+ }
+ else
+ save_size = png_ptr->current_buffer_size;
+
+ png_calculate_crc(png_ptr, png_ptr->current_buffer_ptr, save_size);
+
+ png_process_IDAT_data(png_ptr, png_ptr->current_buffer_ptr, save_size);
+
+ png_ptr->idat_size -= save_size;
+ png_ptr->buffer_size -= save_size;
+ png_ptr->current_buffer_size -= save_size;
+ png_ptr->current_buffer_ptr += save_size;
+ }
+ if (!png_ptr->idat_size)
+ {
+ if (png_ptr->buffer_size < 4)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_crc_finish(png_ptr, 0);
+ png_ptr->mode &= ~PNG_HAVE_CHUNK_HEADER;
+ png_ptr->mode |= PNG_AFTER_IDAT;
+ }
+}
+
+void /* PRIVATE */
+png_process_IDAT_data(png_structp png_ptr, png_bytep buffer,
+ png_size_t buffer_length)
+{
+ /* The caller checks for a non-zero buffer length. */
+ if (!(buffer_length > 0) || buffer == NULL)
+ png_error(png_ptr, "No IDAT data (internal error)");
+
+ /* This routine must process all the data it has been given
+ * before returning, calling the row callback as required to
+ * handle the uncompressed results.
+ */
+ png_ptr->zstream.next_in = buffer;
+ png_ptr->zstream.avail_in = (uInt)buffer_length;
+
+ /* Keep going until the decompressed data is all processed
+ * or the stream marked as finished.
+ */
+ while (png_ptr->zstream.avail_in > 0 &&
+ !(png_ptr->flags & PNG_FLAG_ZLIB_FINISHED))
+ {
+ int ret;
+
+ /* We have data for zlib, but we must check that zlib
+ * has somewhere to put the results. It doesn't matter
+ * if we don't expect any results -- it may be the input
+ * data is just the LZ end code.
+ */
+ if (!(png_ptr->zstream.avail_out > 0))
+ {
+ png_ptr->zstream.avail_out =
+ (uInt) PNG_ROWBYTES(png_ptr->pixel_depth,
+ png_ptr->iwidth) + 1;
+ png_ptr->zstream.next_out = png_ptr->row_buf;
+ }
+
+ /* Using Z_SYNC_FLUSH here means that an unterminated
+ * LZ stream can still be handled (a stream with a missing
+ * end code), otherwise (Z_NO_FLUSH) a future zlib
+ * implementation might defer output and, therefore,
+ * change the current behavior. (See comments in inflate.c
+ * for why this doesn't happen at present with zlib 1.2.5.)
+ */
+ ret = inflate(&png_ptr->zstream, Z_SYNC_FLUSH);
+
+ /* Check for any failure before proceeding. */
+ if (ret != Z_OK && ret != Z_STREAM_END)
+ {
+ /* Terminate the decompression. */
+ png_ptr->flags |= PNG_FLAG_ZLIB_FINISHED;
+
+ /* This may be a truncated stream (missing or
+ * damaged end code). Treat that as a warning.
+ */
+ if (png_ptr->row_number >= png_ptr->num_rows ||
+ png_ptr->pass > 6)
+ png_warning(png_ptr, "Truncated compressed data in IDAT");
+ else
+ png_error(png_ptr, "Decompression error in IDAT");
+
+ /* Skip the check on unprocessed input */
+ return;
+ }
+
+ /* Did inflate output any data? */
+ if (png_ptr->zstream.next_out != png_ptr->row_buf)
+ {
+ /* Is this unexpected data after the last row?
+ * If it is, artificially terminate the LZ output
+ * here.
+ */
+ if (png_ptr->row_number >= png_ptr->num_rows ||
+ png_ptr->pass > 6)
+ {
+ /* Extra data. */
+ png_warning(png_ptr, "Extra compressed data in IDAT");
+ png_ptr->flags |= PNG_FLAG_ZLIB_FINISHED;
+ /* Do no more processing; skip the unprocessed
+ * input check below.
+ */
+ return;
+ }
+
+ /* Do we have a complete row? */
+ if (png_ptr->zstream.avail_out == 0)
+ png_push_process_row(png_ptr);
+ }
+
+ /* And check for the end of the stream. */
+ if (ret == Z_STREAM_END)
+ png_ptr->flags |= PNG_FLAG_ZLIB_FINISHED;
+ }
+
+ /* All the data should have been processed, if anything
+ * is left at this point we have bytes of IDAT data
+ * after the zlib end code.
+ */
+ if (png_ptr->zstream.avail_in > 0)
+ png_warning(png_ptr, "Extra compression data");
+}
+
+void /* PRIVATE */
+png_push_process_row(png_structp png_ptr)
+{
+ png_ptr->row_info.color_type = png_ptr->color_type;
+ png_ptr->row_info.width = png_ptr->iwidth;
+ png_ptr->row_info.channels = png_ptr->channels;
+ png_ptr->row_info.bit_depth = png_ptr->bit_depth;
+ png_ptr->row_info.pixel_depth = png_ptr->pixel_depth;
+
+ png_ptr->row_info.rowbytes = PNG_ROWBYTES(png_ptr->row_info.pixel_depth,
+ png_ptr->row_info.width);
+
+ png_read_filter_row(png_ptr, &(png_ptr->row_info),
+ png_ptr->row_buf + 1, png_ptr->prev_row + 1,
+ (int)(png_ptr->row_buf[0]));
+
+ png_memcpy_check(png_ptr, png_ptr->prev_row, png_ptr->row_buf,
+ png_ptr->rowbytes + 1);
+
+ if (png_ptr->transformations || (png_ptr->flags&PNG_FLAG_STRIP_ALPHA))
+ png_do_read_transformations(png_ptr);
+
+#ifdef PNG_READ_INTERLACING_SUPPORTED
+ /* Blow up interlaced rows to full size */
+ if (png_ptr->interlaced && (png_ptr->transformations & PNG_INTERLACE))
+ {
+ if (png_ptr->pass < 6)
+/* old interface (pre-1.0.9):
+ png_do_read_interlace(&(png_ptr->row_info),
+ png_ptr->row_buf + 1, png_ptr->pass, png_ptr->transformations);
+ */
+ png_do_read_interlace(png_ptr);
+
+ switch (png_ptr->pass)
+ {
+ case 0:
+ {
+ int i;
+ for (i = 0; i < 8 && png_ptr->pass == 0; i++)
+ {
+ png_push_have_row(png_ptr, png_ptr->row_buf + 1);
+ png_read_push_finish_row(png_ptr); /* Updates png_ptr->pass */
+ }
+
+ if (png_ptr->pass == 2) /* Pass 1 might be empty */
+ {
+ for (i = 0; i < 4 && png_ptr->pass == 2; i++)
+ {
+ png_push_have_row(png_ptr, png_bytep_NULL);
+ png_read_push_finish_row(png_ptr);
+ }
+ }
+
+ if (png_ptr->pass == 4 && png_ptr->height <= 4)
+ {
+ for (i = 0; i < 2 && png_ptr->pass == 4; i++)
+ {
+ png_push_have_row(png_ptr, png_bytep_NULL);
+ png_read_push_finish_row(png_ptr);
+ }
+ }
+
+ if (png_ptr->pass == 6 && png_ptr->height <= 4)
+ {
+ png_push_have_row(png_ptr, png_bytep_NULL);
+ png_read_push_finish_row(png_ptr);
+ }
+
+ break;
+ }
+
+ case 1:
+ {
+ int i;
+ for (i = 0; i < 8 && png_ptr->pass == 1; i++)
+ {
+ png_push_have_row(png_ptr, png_ptr->row_buf + 1);
+ png_read_push_finish_row(png_ptr);
+ }
+
+ if (png_ptr->pass == 2) /* Skip top 4 generated rows */
+ {
+ for (i = 0; i < 4 && png_ptr->pass == 2; i++)
+ {
+ png_push_have_row(png_ptr, png_bytep_NULL);
+ png_read_push_finish_row(png_ptr);
+ }
+ }
+
+ break;
+ }
+
+ case 2:
+ {
+ int i;
+
+ for (i = 0; i < 4 && png_ptr->pass == 2; i++)
+ {
+ png_push_have_row(png_ptr, png_ptr->row_buf + 1);
+ png_read_push_finish_row(png_ptr);
+ }
+
+ for (i = 0; i < 4 && png_ptr->pass == 2; i++)
+ {
+ png_push_have_row(png_ptr, png_bytep_NULL);
+ png_read_push_finish_row(png_ptr);
+ }
+
+ if (png_ptr->pass == 4) /* Pass 3 might be empty */
+ {
+ for (i = 0; i < 2 && png_ptr->pass == 4; i++)
+ {
+ png_push_have_row(png_ptr, png_bytep_NULL);
+ png_read_push_finish_row(png_ptr);
+ }
+ }
+
+ break;
+ }
+
+ case 3:
+ {
+ int i;
+
+ for (i = 0; i < 4 && png_ptr->pass == 3; i++)
+ {
+ png_push_have_row(png_ptr, png_ptr->row_buf + 1);
+ png_read_push_finish_row(png_ptr);
+ }
+
+ if (png_ptr->pass == 4) /* Skip top two generated rows */
+ {
+ for (i = 0; i < 2 && png_ptr->pass == 4; i++)
+ {
+ png_push_have_row(png_ptr, png_bytep_NULL);
+ png_read_push_finish_row(png_ptr);
+ }
+ }
+
+ break;
+ }
+
+ case 4:
+ {
+ int i;
+
+ for (i = 0; i < 2 && png_ptr->pass == 4; i++)
+ {
+ png_push_have_row(png_ptr, png_ptr->row_buf + 1);
+ png_read_push_finish_row(png_ptr);
+ }
+
+ for (i = 0; i < 2 && png_ptr->pass == 4; i++)
+ {
+ png_push_have_row(png_ptr, png_bytep_NULL);
+ png_read_push_finish_row(png_ptr);
+ }
+
+ if (png_ptr->pass == 6) /* Pass 5 might be empty */
+ {
+ png_push_have_row(png_ptr, png_bytep_NULL);
+ png_read_push_finish_row(png_ptr);
+ }
+
+ break;
+ }
+
+ case 5:
+ {
+ int i;
+
+ for (i = 0; i < 2 && png_ptr->pass == 5; i++)
+ {
+ png_push_have_row(png_ptr, png_ptr->row_buf + 1);
+ png_read_push_finish_row(png_ptr);
+ }
+
+ if (png_ptr->pass == 6) /* Skip top generated row */
+ {
+ png_push_have_row(png_ptr, png_bytep_NULL);
+ png_read_push_finish_row(png_ptr);
+ }
+
+ break;
+ }
+ case 6:
+ {
+ png_push_have_row(png_ptr, png_ptr->row_buf + 1);
+ png_read_push_finish_row(png_ptr);
+
+ if (png_ptr->pass != 6)
+ break;
+
+ png_push_have_row(png_ptr, png_bytep_NULL);
+ png_read_push_finish_row(png_ptr);
+ }
+ }
+ }
+ else
+#endif
+ {
+ png_push_have_row(png_ptr, png_ptr->row_buf + 1);
+ png_read_push_finish_row(png_ptr);
+ }
+}
+
+void /* PRIVATE */
+png_read_push_finish_row(png_structp png_ptr)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ /* Arrays to facilitate easy interlacing - use pass (0 - 6) as index */
+
+ /* Start of interlace block */
+ PNG_CONST int FARDATA png_pass_start[] = {0, 4, 0, 2, 0, 1, 0};
+
+ /* Offset to next interlace block */
+ PNG_CONST int FARDATA png_pass_inc[] = {8, 8, 4, 4, 2, 2, 1};
+
+ /* Start of interlace block in the y direction */
+ PNG_CONST int FARDATA png_pass_ystart[] = {0, 0, 4, 0, 2, 0, 1};
+
+ /* Offset to next interlace block in the y direction */
+ PNG_CONST int FARDATA png_pass_yinc[] = {8, 8, 8, 4, 4, 2, 2};
+
+ /* Height of interlace block. This is not currently used - if you need
+ * it, uncomment it here and in png.h
+ PNG_CONST int FARDATA png_pass_height[] = {8, 8, 4, 4, 2, 2, 1};
+ */
+#endif
+
+ png_ptr->row_number++;
+ if (png_ptr->row_number < png_ptr->num_rows)
+ return;
+
+#ifdef PNG_READ_INTERLACING_SUPPORTED
+ if (png_ptr->interlaced)
+ {
+ png_ptr->row_number = 0;
+ png_memset_check(png_ptr, png_ptr->prev_row, 0,
+ png_ptr->rowbytes + 1);
+ do
+ {
+ png_ptr->pass++;
+ if ((png_ptr->pass == 1 && png_ptr->width < 5) ||
+ (png_ptr->pass == 3 && png_ptr->width < 3) ||
+ (png_ptr->pass == 5 && png_ptr->width < 2))
+ png_ptr->pass++;
+
+ if (png_ptr->pass > 7)
+ png_ptr->pass--;
+
+ if (png_ptr->pass >= 7)
+ break;
+
+ png_ptr->iwidth = (png_ptr->width +
+ png_pass_inc[png_ptr->pass] - 1 -
+ png_pass_start[png_ptr->pass]) /
+ png_pass_inc[png_ptr->pass];
+
+ if (png_ptr->transformations & PNG_INTERLACE)
+ break;
+
+ png_ptr->num_rows = (png_ptr->height +
+ png_pass_yinc[png_ptr->pass] - 1 -
+ png_pass_ystart[png_ptr->pass]) /
+ png_pass_yinc[png_ptr->pass];
+
+ } while (png_ptr->iwidth == 0 || png_ptr->num_rows == 0);
+ }
+#endif /* PNG_READ_INTERLACING_SUPPORTED */
+}
+
+#ifdef PNG_READ_tEXt_SUPPORTED
+void /* PRIVATE */
+png_push_handle_tEXt(png_structp png_ptr, png_infop info_ptr, png_uint_32
+ length)
+{
+ if (!(png_ptr->mode & PNG_HAVE_IHDR) || (png_ptr->mode & PNG_HAVE_IEND))
+ {
+ png_error(png_ptr, "Out of place tEXt");
+ info_ptr = info_ptr; /* To quiet some compiler warnings */
+ }
+
+#ifdef PNG_MAX_MALLOC_64K
+ png_ptr->skip_length = 0; /* This may not be necessary */
+
+ if (length > (png_uint_32)65535L) /* Can't hold entire string in memory */
+ {
+ png_warning(png_ptr, "tEXt chunk too large to fit in memory");
+ png_ptr->skip_length = length - (png_uint_32)65535L;
+ length = (png_uint_32)65535L;
+ }
+#endif
+
+ png_ptr->current_text = (png_charp)png_malloc(png_ptr,
+ (png_uint_32)(length + 1));
+ png_ptr->current_text[length] = '\0';
+ png_ptr->current_text_ptr = png_ptr->current_text;
+ png_ptr->current_text_size = (png_size_t)length;
+ png_ptr->current_text_left = (png_size_t)length;
+ png_ptr->process_mode = PNG_READ_tEXt_MODE;
+}
+
+void /* PRIVATE */
+png_push_read_tEXt(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr->buffer_size && png_ptr->current_text_left)
+ {
+ png_size_t text_size;
+
+ if (png_ptr->buffer_size < png_ptr->current_text_left)
+ text_size = png_ptr->buffer_size;
+
+ else
+ text_size = png_ptr->current_text_left;
+
+ png_crc_read(png_ptr, (png_bytep)png_ptr->current_text_ptr, text_size);
+ png_ptr->current_text_left -= text_size;
+ png_ptr->current_text_ptr += text_size;
+ }
+ if (!(png_ptr->current_text_left))
+ {
+ png_textp text_ptr;
+ png_charp text;
+ png_charp key;
+ int ret;
+
+ if (png_ptr->buffer_size < 4)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_push_crc_finish(png_ptr);
+
+#ifdef PNG_MAX_MALLOC_64K
+ if (png_ptr->skip_length)
+ return;
+#endif
+
+ key = png_ptr->current_text;
+
+ for (text = key; *text; text++)
+ /* Empty loop */ ;
+
+ if (text < key + png_ptr->current_text_size)
+ text++;
+
+ text_ptr = (png_textp)png_malloc(png_ptr,
+ (png_uint_32)png_sizeof(png_text));
+ text_ptr->compression = PNG_TEXT_COMPRESSION_NONE;
+ text_ptr->key = key;
+#ifdef PNG_iTXt_SUPPORTED
+ text_ptr->lang = NULL;
+ text_ptr->lang_key = NULL;
+#endif
+ text_ptr->text = text;
+
+ ret = png_set_text_2(png_ptr, info_ptr, text_ptr, 1);
+
+ png_free(png_ptr, key);
+ png_free(png_ptr, text_ptr);
+ png_ptr->current_text = NULL;
+
+ if (ret)
+ png_warning(png_ptr, "Insufficient memory to store text chunk.");
+ }
+}
+#endif
+
+#ifdef PNG_READ_zTXt_SUPPORTED
+void /* PRIVATE */
+png_push_handle_zTXt(png_structp png_ptr, png_infop info_ptr, png_uint_32
+ length)
+{
+ if (!(png_ptr->mode & PNG_HAVE_IHDR) || (png_ptr->mode & PNG_HAVE_IEND))
+ {
+ png_error(png_ptr, "Out of place zTXt");
+ info_ptr = info_ptr; /* To quiet some compiler warnings */
+ }
+
+#ifdef PNG_MAX_MALLOC_64K
+ /* We can't handle zTXt chunks > 64K, since we don't have enough space
+ * to be able to store the uncompressed data. Actually, the threshold
+ * is probably around 32K, but it isn't as definite as 64K is.
+ */
+ if (length > (png_uint_32)65535L)
+ {
+ png_warning(png_ptr, "zTXt chunk too large to fit in memory");
+ png_push_crc_skip(png_ptr, length);
+ return;
+ }
+#endif
+
+ png_ptr->current_text = (png_charp)png_malloc(png_ptr,
+ (png_uint_32)(length + 1));
+ png_ptr->current_text[length] = '\0';
+ png_ptr->current_text_ptr = png_ptr->current_text;
+ png_ptr->current_text_size = (png_size_t)length;
+ png_ptr->current_text_left = (png_size_t)length;
+ png_ptr->process_mode = PNG_READ_zTXt_MODE;
+}
+
+void /* PRIVATE */
+png_push_read_zTXt(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr->buffer_size && png_ptr->current_text_left)
+ {
+ png_size_t text_size;
+
+ if (png_ptr->buffer_size < (png_uint_32)png_ptr->current_text_left)
+ text_size = png_ptr->buffer_size;
+
+ else
+ text_size = png_ptr->current_text_left;
+
+ png_crc_read(png_ptr, (png_bytep)png_ptr->current_text_ptr, text_size);
+ png_ptr->current_text_left -= text_size;
+ png_ptr->current_text_ptr += text_size;
+ }
+ if (!(png_ptr->current_text_left))
+ {
+ png_textp text_ptr;
+ png_charp text;
+ png_charp key;
+ int ret;
+ png_size_t text_size, key_size;
+
+ if (png_ptr->buffer_size < 4)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_push_crc_finish(png_ptr);
+
+ key = png_ptr->current_text;
+
+ for (text = key; *text; text++)
+ /* Empty loop */ ;
+
+ /* zTXt can't have zero text */
+ if (text >= key + png_ptr->current_text_size)
+ {
+ png_ptr->current_text = NULL;
+ png_free(png_ptr, key);
+ return;
+ }
+
+ text++;
+
+ if (*text != PNG_TEXT_COMPRESSION_zTXt) /* Check compression byte */
+ {
+ png_ptr->current_text = NULL;
+ png_free(png_ptr, key);
+ return;
+ }
+
+ text++;
+
+ png_ptr->zstream.next_in = (png_bytep )text;
+ png_ptr->zstream.avail_in = (uInt)(png_ptr->current_text_size -
+ (text - key));
+ png_ptr->zstream.next_out = png_ptr->zbuf;
+ png_ptr->zstream.avail_out = (uInt)png_ptr->zbuf_size;
+
+ key_size = text - key;
+ text_size = 0;
+ text = NULL;
+ ret = Z_STREAM_END;
+
+ while (png_ptr->zstream.avail_in)
+ {
+ ret = inflate(&png_ptr->zstream, Z_PARTIAL_FLUSH);
+ if (ret != Z_OK && ret != Z_STREAM_END)
+ {
+ inflateReset(&png_ptr->zstream);
+ png_ptr->zstream.avail_in = 0;
+ png_ptr->current_text = NULL;
+ png_free(png_ptr, key);
+ png_free(png_ptr, text);
+ return;
+ }
+ if (!(png_ptr->zstream.avail_out) || ret == Z_STREAM_END)
+ {
+ if (text == NULL)
+ {
+ text = (png_charp)png_malloc(png_ptr,
+ (png_uint_32)(png_ptr->zbuf_size
+ - png_ptr->zstream.avail_out + key_size + 1));
+
+ png_memcpy(text + key_size, png_ptr->zbuf,
+ png_ptr->zbuf_size - png_ptr->zstream.avail_out);
+
+ png_memcpy(text, key, key_size);
+
+ text_size = key_size + png_ptr->zbuf_size -
+ png_ptr->zstream.avail_out;
+
+ *(text + text_size) = '\0';
+ }
+ else
+ {
+ png_charp tmp;
+
+ tmp = text;
+ text = (png_charp)png_malloc(png_ptr, text_size +
+ (png_uint_32)(png_ptr->zbuf_size
+ - png_ptr->zstream.avail_out + 1));
+
+ png_memcpy(text, tmp, text_size);
+ png_free(png_ptr, tmp);
+
+ png_memcpy(text + text_size, png_ptr->zbuf,
+ png_ptr->zbuf_size - png_ptr->zstream.avail_out);
+
+ text_size += png_ptr->zbuf_size - png_ptr->zstream.avail_out;
+ *(text + text_size) = '\0';
+ }
+ if (ret != Z_STREAM_END)
+ {
+ png_ptr->zstream.next_out = png_ptr->zbuf;
+ png_ptr->zstream.avail_out = (uInt)png_ptr->zbuf_size;
+ }
+ }
+ else
+ {
+ break;
+ }
+
+ if (ret == Z_STREAM_END)
+ break;
+ }
+
+ inflateReset(&png_ptr->zstream);
+ png_ptr->zstream.avail_in = 0;
+
+ if (ret != Z_STREAM_END)
+ {
+ png_ptr->current_text = NULL;
+ png_free(png_ptr, key);
+ png_free(png_ptr, text);
+ return;
+ }
+
+ png_ptr->current_text = NULL;
+ png_free(png_ptr, key);
+ key = text;
+ text += key_size;
+
+ text_ptr = (png_textp)png_malloc(png_ptr,
+ (png_uint_32)png_sizeof(png_text));
+ text_ptr->compression = PNG_TEXT_COMPRESSION_zTXt;
+ text_ptr->key = key;
+#ifdef PNG_iTXt_SUPPORTED
+ text_ptr->lang = NULL;
+ text_ptr->lang_key = NULL;
+#endif
+ text_ptr->text = text;
+
+ ret = png_set_text_2(png_ptr, info_ptr, text_ptr, 1);
+
+ png_free(png_ptr, key);
+ png_free(png_ptr, text_ptr);
+
+ if (ret)
+ png_warning(png_ptr, "Insufficient memory to store text chunk.");
+ }
+}
+#endif
+
+#ifdef PNG_READ_iTXt_SUPPORTED
+void /* PRIVATE */
+png_push_handle_iTXt(png_structp png_ptr, png_infop info_ptr, png_uint_32
+ length)
+{
+ if (!(png_ptr->mode & PNG_HAVE_IHDR) || (png_ptr->mode & PNG_HAVE_IEND))
+ {
+ png_error(png_ptr, "Out of place iTXt");
+ info_ptr = info_ptr; /* To quiet some compiler warnings */
+ }
+
+#ifdef PNG_MAX_MALLOC_64K
+ png_ptr->skip_length = 0; /* This may not be necessary */
+
+ if (length > (png_uint_32)65535L) /* Can't hold entire string in memory */
+ {
+ png_warning(png_ptr, "iTXt chunk too large to fit in memory");
+ png_ptr->skip_length = length - (png_uint_32)65535L;
+ length = (png_uint_32)65535L;
+ }
+#endif
+
+ png_ptr->current_text = (png_charp)png_malloc(png_ptr,
+ (png_uint_32)(length + 1));
+ png_ptr->current_text[length] = '\0';
+ png_ptr->current_text_ptr = png_ptr->current_text;
+ png_ptr->current_text_size = (png_size_t)length;
+ png_ptr->current_text_left = (png_size_t)length;
+ png_ptr->process_mode = PNG_READ_iTXt_MODE;
+}
+
+void /* PRIVATE */
+png_push_read_iTXt(png_structp png_ptr, png_infop info_ptr)
+{
+
+ if (png_ptr->buffer_size && png_ptr->current_text_left)
+ {
+ png_size_t text_size;
+
+ if (png_ptr->buffer_size < png_ptr->current_text_left)
+ text_size = png_ptr->buffer_size;
+
+ else
+ text_size = png_ptr->current_text_left;
+
+ png_crc_read(png_ptr, (png_bytep)png_ptr->current_text_ptr, text_size);
+ png_ptr->current_text_left -= text_size;
+ png_ptr->current_text_ptr += text_size;
+ }
+ if (!(png_ptr->current_text_left))
+ {
+ png_textp text_ptr;
+ png_charp key;
+ int comp_flag;
+ png_charp lang;
+ png_charp lang_key;
+ png_charp text;
+ int ret;
+
+ if (png_ptr->buffer_size < 4)
+ {
+ png_push_save_buffer(png_ptr);
+ return;
+ }
+
+ png_push_crc_finish(png_ptr);
+
+#ifdef PNG_MAX_MALLOC_64K
+ if (png_ptr->skip_length)
+ return;
+#endif
+
+ key = png_ptr->current_text;
+
+ for (lang = key; *lang; lang++)
+ /* Empty loop */ ;
+
+ if (lang < key + png_ptr->current_text_size - 3)
+ lang++;
+
+ comp_flag = *lang++;
+ lang++; /* Skip comp_type, always zero */
+
+ for (lang_key = lang; *lang_key; lang_key++)
+ /* Empty loop */ ;
+
+ lang_key++; /* Skip NUL separator */
+
+ text=lang_key;
+
+ if (lang_key < key + png_ptr->current_text_size - 1)
+ {
+ for (; *text; text++)
+ /* Empty loop */ ;
+ }
+
+ if (text < key + png_ptr->current_text_size)
+ text++;
+
+ text_ptr = (png_textp)png_malloc(png_ptr,
+ (png_uint_32)png_sizeof(png_text));
+
+ text_ptr->compression = comp_flag + 2;
+ text_ptr->key = key;
+ text_ptr->lang = lang;
+ text_ptr->lang_key = lang_key;
+ text_ptr->text = text;
+ text_ptr->text_length = 0;
+ text_ptr->itxt_length = png_strlen(text);
+
+ ret = png_set_text_2(png_ptr, info_ptr, text_ptr, 1);
+
+ png_ptr->current_text = NULL;
+
+ png_free(png_ptr, text_ptr);
+ if (ret)
+ png_warning(png_ptr, "Insufficient memory to store iTXt chunk.");
+ }
+}
+#endif
+
+/* This function is called when we haven't found a handler for this
+ * chunk. If there isn't a problem with the chunk itself (ie a bad chunk
+ * name or a critical chunk), the chunk is (currently) silently ignored.
+ */
+void /* PRIVATE */
+png_push_handle_unknown(png_structp png_ptr, png_infop info_ptr, png_uint_32
+ length)
+{
+ png_uint_32 skip = 0;
+
+ if (!(png_ptr->chunk_name[0] & 0x20))
+ {
+#ifdef PNG_READ_UNKNOWN_CHUNKS_SUPPORTED
+ if (png_handle_as_unknown(png_ptr, png_ptr->chunk_name) !=
+ PNG_HANDLE_CHUNK_ALWAYS
+#ifdef PNG_READ_USER_CHUNKS_SUPPORTED
+ && png_ptr->read_user_chunk_fn == NULL
+#endif
+ )
+#endif
+ png_chunk_error(png_ptr, "unknown critical chunk");
+
+ info_ptr = info_ptr; /* To quiet some compiler warnings */
+ }
+
+#ifdef PNG_READ_UNKNOWN_CHUNKS_SUPPORTED
+ if (png_ptr->flags & PNG_FLAG_KEEP_UNKNOWN_CHUNKS)
+ {
+#ifdef PNG_MAX_MALLOC_64K
+ if (length > (png_uint_32)65535L)
+ {
+ png_warning(png_ptr, "unknown chunk too large to fit in memory");
+ skip = length - (png_uint_32)65535L;
+ length = (png_uint_32)65535L;
+ }
+#endif
+ png_memcpy((png_charp)png_ptr->unknown_chunk.name,
+ (png_charp)png_ptr->chunk_name,
+ png_sizeof(png_ptr->unknown_chunk.name));
+ png_ptr->unknown_chunk.name[png_sizeof(png_ptr->unknown_chunk.name) - 1]
+ = '\0';
+
+ png_ptr->unknown_chunk.size = (png_size_t)length;
+
+ if (length == 0)
+ png_ptr->unknown_chunk.data = NULL;
+
+ else
+ {
+ png_ptr->unknown_chunk.data = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)length);
+ png_crc_read(png_ptr, (png_bytep)png_ptr->unknown_chunk.data, length);
+ }
+
+#ifdef PNG_READ_USER_CHUNKS_SUPPORTED
+ if (png_ptr->read_user_chunk_fn != NULL)
+ {
+ /* Callback to user unknown chunk handler */
+ int ret;
+ ret = (*(png_ptr->read_user_chunk_fn))
+ (png_ptr, &png_ptr->unknown_chunk);
+
+ if (ret < 0)
+ png_chunk_error(png_ptr, "error in user chunk");
+
+ if (ret == 0)
+ {
+ if (!(png_ptr->chunk_name[0] & 0x20))
+ if (png_handle_as_unknown(png_ptr, png_ptr->chunk_name) !=
+ PNG_HANDLE_CHUNK_ALWAYS)
+ png_chunk_error(png_ptr, "unknown critical chunk");
+ png_set_unknown_chunks(png_ptr, info_ptr,
+ &png_ptr->unknown_chunk, 1);
+ }
+ }
+
+ else
+#endif
+ png_set_unknown_chunks(png_ptr, info_ptr, &png_ptr->unknown_chunk, 1);
+ png_free(png_ptr, png_ptr->unknown_chunk.data);
+ png_ptr->unknown_chunk.data = NULL;
+ }
+
+ else
+#endif
+ skip=length;
+ png_push_crc_skip(png_ptr, skip);
+}
+
+void /* PRIVATE */
+png_push_have_info(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr->info_fn != NULL)
+ (*(png_ptr->info_fn))(png_ptr, info_ptr);
+}
+
+void /* PRIVATE */
+png_push_have_end(png_structp png_ptr, png_infop info_ptr)
+{
+ if (png_ptr->end_fn != NULL)
+ (*(png_ptr->end_fn))(png_ptr, info_ptr);
+}
+
+void /* PRIVATE */
+png_push_have_row(png_structp png_ptr, png_bytep row)
+{
+ if (png_ptr->row_fn != NULL)
+ (*(png_ptr->row_fn))(png_ptr, row, png_ptr->row_number,
+ (int)png_ptr->pass);
+}
+
+void PNGAPI
+png_progressive_combine_row (png_structp png_ptr,
+ png_bytep old_row, png_bytep new_row)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_CONST int FARDATA png_pass_dsp_mask[7] =
+ {0xff, 0x0f, 0xff, 0x33, 0xff, 0x55, 0xff};
+#endif
+
+ if (png_ptr == NULL)
+ return;
+
+ if (new_row != NULL) /* new_row must == png_ptr->row_buf here. */
+ png_combine_row(png_ptr, old_row, png_pass_dsp_mask[png_ptr->pass]);
+}
+
+void PNGAPI
+png_set_progressive_read_fn(png_structp png_ptr, png_voidp progressive_ptr,
+ png_progressive_info_ptr info_fn, png_progressive_row_ptr row_fn,
+ png_progressive_end_ptr end_fn)
+{
+ if (png_ptr == NULL)
+ return;
+
+ png_ptr->info_fn = info_fn;
+ png_ptr->row_fn = row_fn;
+ png_ptr->end_fn = end_fn;
+
+ png_set_read_fn(png_ptr, progressive_ptr, png_push_fill_buffer);
+}
+
+png_voidp PNGAPI
+png_get_progressive_ptr(png_structp png_ptr)
+{
+ if (png_ptr == NULL)
+ return (NULL);
+
+ return png_ptr->io_ptr;
+}
+#endif /* PNG_PROGRESSIVE_READ_SUPPORTED */
diff --git a/trunk/src/third_party/libpng/pngread.c b/trunk/src/third_party/libpng/pngread.c
new file mode 100644
index 0000000..6207624
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngread.c
@@ -0,0 +1,1528 @@
+
+/* pngread.c - read a PNG file
+ *
+ * Last changed in libpng 1.2.44 [June 26, 2010]
+ * Copyright (c) 1998-2010 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ *
+ * This file contains routines that an application calls directly to
+ * read a PNG file or stream.
+ */
+
+#define PNG_INTERNAL
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+#ifdef PNG_READ_SUPPORTED
+
+
+/* Create a PNG structure for reading, and allocate any memory needed. */
+png_structp PNGAPI
+png_create_read_struct(png_const_charp user_png_ver, png_voidp error_ptr,
+ png_error_ptr error_fn, png_error_ptr warn_fn)
+{
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ return (png_create_read_struct_2(user_png_ver, error_ptr, error_fn,
+ warn_fn, png_voidp_NULL, png_malloc_ptr_NULL, png_free_ptr_NULL));
+}
+
+/* Alternate create PNG structure for reading, and allocate any memory
+ * needed.
+ */
+png_structp PNGAPI
+png_create_read_struct_2(png_const_charp user_png_ver, png_voidp error_ptr,
+ png_error_ptr error_fn, png_error_ptr warn_fn, png_voidp mem_ptr,
+ png_malloc_ptr malloc_fn, png_free_ptr free_fn)
+{
+#endif /* PNG_USER_MEM_SUPPORTED */
+
+#ifdef PNG_SETJMP_SUPPORTED
+ volatile
+#endif
+ png_structp png_ptr;
+
+#ifdef PNG_SETJMP_SUPPORTED
+#ifdef USE_FAR_KEYWORD
+ jmp_buf jmpbuf;
+#endif
+#endif
+
+ int i;
+
+ png_debug(1, "in png_create_read_struct");
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_ptr = (png_structp)png_create_struct_2(PNG_STRUCT_PNG,
+ (png_malloc_ptr)malloc_fn, (png_voidp)mem_ptr);
+#else
+ png_ptr = (png_structp)png_create_struct(PNG_STRUCT_PNG);
+#endif
+ if (png_ptr == NULL)
+ return (NULL);
+
+ /* Added at libpng-1.2.6 */
+#ifdef PNG_USER_LIMITS_SUPPORTED
+ png_ptr->user_width_max = PNG_USER_WIDTH_MAX;
+ png_ptr->user_height_max = PNG_USER_HEIGHT_MAX;
+# ifdef PNG_USER_CHUNK_CACHE_MAX
+ /* Added at libpng-1.2.43 and 1.4.0 */
+ png_ptr->user_chunk_cache_max = PNG_USER_CHUNK_CACHE_MAX;
+# endif
+# ifdef PNG_SET_USER_CHUNK_MALLOC_MAX
+ /* Added at libpng-1.2.43 and 1.4.1 */
+ png_ptr->user_chunk_malloc_max = PNG_USER_CHUNK_MALLOC_MAX;
+# endif
+#endif
+
+#ifdef PNG_SETJMP_SUPPORTED
+#ifdef USE_FAR_KEYWORD
+ if (setjmp(jmpbuf))
+#else
+ if (setjmp(png_ptr->jmpbuf))
+#endif
+ {
+ png_free(png_ptr, png_ptr->zbuf);
+ png_ptr->zbuf = NULL;
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_destroy_struct_2((png_voidp)png_ptr,
+ (png_free_ptr)free_fn, (png_voidp)mem_ptr);
+#else
+ png_destroy_struct((png_voidp)png_ptr);
+#endif
+ return (NULL);
+ }
+#ifdef USE_FAR_KEYWORD
+ png_memcpy(png_ptr->jmpbuf, jmpbuf, png_sizeof(jmp_buf));
+#endif
+#endif /* PNG_SETJMP_SUPPORTED */
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_set_mem_fn(png_ptr, mem_ptr, malloc_fn, free_fn);
+#endif
+
+ png_set_error_fn(png_ptr, error_ptr, error_fn, warn_fn);
+
+ if (user_png_ver)
+ {
+ i = 0;
+ do
+ {
+ if (user_png_ver[i] != png_libpng_ver[i])
+ png_ptr->flags |= PNG_FLAG_LIBRARY_MISMATCH;
+ } while (png_libpng_ver[i++]);
+ }
+ else
+ png_ptr->flags |= PNG_FLAG_LIBRARY_MISMATCH;
+
+
+ if (png_ptr->flags & PNG_FLAG_LIBRARY_MISMATCH)
+ {
+ /* Libpng 0.90 and later are binary incompatible with libpng 0.89, so
+ * we must recompile any applications that use any older library version.
+ * For versions after libpng 1.0, we will be compatible, so we need
+ * only check the first digit.
+ */
+ if (user_png_ver == NULL || user_png_ver[0] != png_libpng_ver[0] ||
+ (user_png_ver[0] == '1' && user_png_ver[2] != png_libpng_ver[2]) ||
+ (user_png_ver[0] == '0' && user_png_ver[2] < '9'))
+ {
+#if defined(PNG_STDIO_SUPPORTED) && !defined(_WIN32_WCE)
+ char msg[80];
+ if (user_png_ver)
+ {
+ png_snprintf(msg, 80,
+ "Application was compiled with png.h from libpng-%.20s",
+ user_png_ver);
+ png_warning(png_ptr, msg);
+ }
+ png_snprintf(msg, 80,
+ "Application is running with png.c from libpng-%.20s",
+ png_libpng_ver);
+ png_warning(png_ptr, msg);
+#endif
+#ifdef PNG_ERROR_NUMBERS_SUPPORTED
+ png_ptr->flags = 0;
+#endif
+ png_error(png_ptr,
+ "Incompatible libpng version in application and library");
+ }
+ }
+
+ /* Initialize zbuf - compression buffer */
+ png_ptr->zbuf_size = PNG_ZBUF_SIZE;
+ png_ptr->zbuf = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)png_ptr->zbuf_size);
+ png_ptr->zstream.zalloc = png_zalloc;
+ png_ptr->zstream.zfree = png_zfree;
+ png_ptr->zstream.opaque = (voidpf)png_ptr;
+
+ switch (inflateInit(&png_ptr->zstream))
+ {
+ case Z_OK: /* Do nothing */ break;
+ case Z_MEM_ERROR:
+ case Z_STREAM_ERROR: png_error(png_ptr, "zlib memory error");
+ break;
+ case Z_VERSION_ERROR: png_error(png_ptr, "zlib version error");
+ break;
+ default: png_error(png_ptr, "Unknown zlib error");
+ }
+
+
+ png_ptr->zstream.next_out = png_ptr->zbuf;
+ png_ptr->zstream.avail_out = (uInt)png_ptr->zbuf_size;
+
+ png_set_read_fn(png_ptr, png_voidp_NULL, png_rw_ptr_NULL);
+
+#ifdef PNG_SETJMP_SUPPORTED
+/* Applications that neglect to set up their own setjmp() and then
+ encounter a png_error() will longjmp here. Since the jmpbuf is
+ then meaningless we abort instead of returning. */
+#ifdef USE_FAR_KEYWORD
+ if (setjmp(jmpbuf))
+ PNG_ABORT();
+ png_memcpy(png_ptr->jmpbuf, jmpbuf, png_sizeof(jmp_buf));
+#else
+ if (setjmp(png_ptr->jmpbuf))
+ PNG_ABORT();
+#endif
+#endif /* PNG_SETJMP_SUPPORTED */
+
+ return (png_ptr);
+}
+
+#if defined(PNG_1_0_X) || defined(PNG_1_2_X)
+/* Initialize PNG structure for reading, and allocate any memory needed.
+ * This interface is deprecated in favour of the png_create_read_struct(),
+ * and it will disappear as of libpng-1.3.0.
+ */
+#undef png_read_init
+void PNGAPI
+png_read_init(png_structp png_ptr)
+{
+ /* We only come here via pre-1.0.7-compiled applications */
+ png_read_init_2(png_ptr, "1.0.6 or earlier", 0, 0);
+}
+
+void PNGAPI
+png_read_init_2(png_structp png_ptr, png_const_charp user_png_ver,
+ png_size_t png_struct_size, png_size_t png_info_size)
+{
+ /* We only come here via pre-1.0.12-compiled applications */
+ if (png_ptr == NULL)
+ return;
+#if defined(PNG_STDIO_SUPPORTED) && !defined(_WIN32_WCE)
+ if (png_sizeof(png_struct) > png_struct_size ||
+ png_sizeof(png_info) > png_info_size)
+ {
+ char msg[80];
+ png_ptr->warning_fn = NULL;
+ if (user_png_ver)
+ {
+ png_snprintf(msg, 80,
+ "Application was compiled with png.h from libpng-%.20s",
+ user_png_ver);
+ png_warning(png_ptr, msg);
+ }
+ png_snprintf(msg, 80,
+ "Application is running with png.c from libpng-%.20s",
+ png_libpng_ver);
+ png_warning(png_ptr, msg);
+ }
+#endif
+ if (png_sizeof(png_struct) > png_struct_size)
+ {
+ png_ptr->error_fn = NULL;
+#ifdef PNG_ERROR_NUMBERS_SUPPORTED
+ png_ptr->flags = 0;
+#endif
+ png_error(png_ptr,
+ "The png struct allocated by the application for reading is"
+ " too small.");
+ }
+ if (png_sizeof(png_info) > png_info_size)
+ {
+ png_ptr->error_fn = NULL;
+#ifdef PNG_ERROR_NUMBERS_SUPPORTED
+ png_ptr->flags = 0;
+#endif
+ png_error(png_ptr,
+ "The info struct allocated by application for reading is"
+ " too small.");
+ }
+ png_read_init_3(&png_ptr, user_png_ver, png_struct_size);
+}
+#endif /* PNG_1_0_X || PNG_1_2_X */
+
+void PNGAPI
+png_read_init_3(png_structpp ptr_ptr, png_const_charp user_png_ver,
+ png_size_t png_struct_size)
+{
+#ifdef PNG_SETJMP_SUPPORTED
+ jmp_buf tmp_jmp; /* to save current jump buffer */
+#endif
+
+ int i = 0;
+
+ png_structp png_ptr=*ptr_ptr;
+
+ if (png_ptr == NULL)
+ return;
+
+ do
+ {
+ if (user_png_ver[i] != png_libpng_ver[i])
+ {
+#ifdef PNG_LEGACY_SUPPORTED
+ png_ptr->flags |= PNG_FLAG_LIBRARY_MISMATCH;
+#else
+ png_ptr->warning_fn = NULL;
+ png_warning(png_ptr,
+ "Application uses deprecated png_read_init() and should be"
+ " recompiled.");
+ break;
+#endif
+ }
+ } while (png_libpng_ver[i++]);
+
+ png_debug(1, "in png_read_init_3");
+
+#ifdef PNG_SETJMP_SUPPORTED
+ /* Save jump buffer and error functions */
+ png_memcpy(tmp_jmp, png_ptr->jmpbuf, png_sizeof(jmp_buf));
+#endif
+
+ if (png_sizeof(png_struct) > png_struct_size)
+ {
+ png_destroy_struct(png_ptr);
+ *ptr_ptr = (png_structp)png_create_struct(PNG_STRUCT_PNG);
+ png_ptr = *ptr_ptr;
+ }
+
+ /* Reset all variables to 0 */
+ png_memset(png_ptr, 0, png_sizeof(png_struct));
+
+#ifdef PNG_SETJMP_SUPPORTED
+ /* Restore jump buffer */
+ png_memcpy(png_ptr->jmpbuf, tmp_jmp, png_sizeof(jmp_buf));
+#endif
+
+ /* Added at libpng-1.2.6 */
+#ifdef PNG_SET_USER_LIMITS_SUPPORTED
+ png_ptr->user_width_max = PNG_USER_WIDTH_MAX;
+ png_ptr->user_height_max = PNG_USER_HEIGHT_MAX;
+#endif
+
+ /* Initialize zbuf - compression buffer */
+ png_ptr->zbuf_size = PNG_ZBUF_SIZE;
+ png_ptr->zstream.zalloc = png_zalloc;
+ png_ptr->zbuf = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)png_ptr->zbuf_size);
+ png_ptr->zstream.zalloc = png_zalloc;
+ png_ptr->zstream.zfree = png_zfree;
+ png_ptr->zstream.opaque = (voidpf)png_ptr;
+
+ switch (inflateInit(&png_ptr->zstream))
+ {
+ case Z_OK: /* Do nothing */ break;
+ case Z_STREAM_ERROR: png_error(png_ptr, "zlib memory error"); break;
+ case Z_VERSION_ERROR: png_error(png_ptr, "zlib version error");
+ break;
+ default: png_error(png_ptr, "Unknown zlib error");
+ }
+
+ png_ptr->zstream.next_out = png_ptr->zbuf;
+ png_ptr->zstream.avail_out = (uInt)png_ptr->zbuf_size;
+
+ png_set_read_fn(png_ptr, png_voidp_NULL, png_rw_ptr_NULL);
+}
+
+#ifdef PNG_SEQUENTIAL_READ_SUPPORTED
+/* Read the information before the actual image data. This has been
+ * changed in v0.90 to allow reading a file that already has the magic
+ * bytes read from the stream. You can tell libpng how many bytes have
+ * been read from the beginning of the stream (up to the maximum of 8)
+ * via png_set_sig_bytes(), and we will only check the remaining bytes
+ * here. The application can then have access to the signature bytes we
+ * read if it is determined that this isn't a valid PNG file.
+ */
+void PNGAPI
+png_read_info(png_structp png_ptr, png_infop info_ptr)
+{
+ png_debug(1, "in png_read_info");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ /* If we haven't checked all of the PNG signature bytes, do so now. */
+ if (png_ptr->sig_bytes < 8)
+ {
+ png_size_t num_checked = png_ptr->sig_bytes,
+ num_to_check = 8 - num_checked;
+
+ png_read_data(png_ptr, &(info_ptr->signature[num_checked]), num_to_check);
+ png_ptr->sig_bytes = 8;
+
+ if (png_sig_cmp(info_ptr->signature, num_checked, num_to_check))
+ {
+ if (num_checked < 4 &&
+ png_sig_cmp(info_ptr->signature, num_checked, num_to_check - 4))
+ png_error(png_ptr, "Not a PNG file");
+ else
+ png_error(png_ptr, "PNG file corrupted by ASCII conversion");
+ }
+ if (num_checked < 3)
+ png_ptr->mode |= PNG_HAVE_PNG_SIGNATURE;
+ }
+
+ for (;;)
+ {
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_CONST PNG_IHDR;
+ PNG_CONST PNG_IDAT;
+ PNG_CONST PNG_IEND;
+ PNG_CONST PNG_PLTE;
+#ifdef PNG_READ_bKGD_SUPPORTED
+ PNG_CONST PNG_bKGD;
+#endif
+#ifdef PNG_READ_cHRM_SUPPORTED
+ PNG_CONST PNG_cHRM;
+#endif
+#ifdef PNG_READ_gAMA_SUPPORTED
+ PNG_CONST PNG_gAMA;
+#endif
+#ifdef PNG_READ_hIST_SUPPORTED
+ PNG_CONST PNG_hIST;
+#endif
+#ifdef PNG_READ_iCCP_SUPPORTED
+ PNG_CONST PNG_iCCP;
+#endif
+#ifdef PNG_READ_iTXt_SUPPORTED
+ PNG_CONST PNG_iTXt;
+#endif
+#ifdef PNG_READ_oFFs_SUPPORTED
+ PNG_CONST PNG_oFFs;
+#endif
+#ifdef PNG_READ_pCAL_SUPPORTED
+ PNG_CONST PNG_pCAL;
+#endif
+#ifdef PNG_READ_pHYs_SUPPORTED
+ PNG_CONST PNG_pHYs;
+#endif
+#ifdef PNG_READ_sBIT_SUPPORTED
+ PNG_CONST PNG_sBIT;
+#endif
+#ifdef PNG_READ_sCAL_SUPPORTED
+ PNG_CONST PNG_sCAL;
+#endif
+#ifdef PNG_READ_sPLT_SUPPORTED
+ PNG_CONST PNG_sPLT;
+#endif
+#ifdef PNG_READ_sRGB_SUPPORTED
+ PNG_CONST PNG_sRGB;
+#endif
+#ifdef PNG_READ_tEXt_SUPPORTED
+ PNG_CONST PNG_tEXt;
+#endif
+#ifdef PNG_READ_tIME_SUPPORTED
+ PNG_CONST PNG_tIME;
+#endif
+#ifdef PNG_READ_tRNS_SUPPORTED
+ PNG_CONST PNG_tRNS;
+#endif
+#ifdef PNG_READ_zTXt_SUPPORTED
+ PNG_CONST PNG_zTXt;
+#endif
+#endif /* PNG_USE_LOCAL_ARRAYS */
+ png_uint_32 length = png_read_chunk_header(png_ptr);
+ PNG_CONST png_bytep chunk_name = png_ptr->chunk_name;
+
+ /* This should be a binary subdivision search or a hash for
+ * matching the chunk name rather than a linear search.
+ */
+ if (!png_memcmp(chunk_name, png_IDAT, 4))
+ if (png_ptr->mode & PNG_AFTER_IDAT)
+ png_ptr->mode |= PNG_HAVE_CHUNK_AFTER_IDAT;
+
+ if (!png_memcmp(chunk_name, png_IHDR, 4))
+ png_handle_IHDR(png_ptr, info_ptr, length);
+ else if (!png_memcmp(chunk_name, png_IEND, 4))
+ png_handle_IEND(png_ptr, info_ptr, length);
+#ifdef PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+ else if (png_handle_as_unknown(png_ptr, chunk_name))
+ {
+ if (!png_memcmp(chunk_name, png_IDAT, 4))
+ png_ptr->mode |= PNG_HAVE_IDAT;
+ png_handle_unknown(png_ptr, info_ptr, length);
+ if (!png_memcmp(chunk_name, png_PLTE, 4))
+ png_ptr->mode |= PNG_HAVE_PLTE;
+ else if (!png_memcmp(chunk_name, png_IDAT, 4))
+ {
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before IDAT");
+ else if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE &&
+ !(png_ptr->mode & PNG_HAVE_PLTE))
+ png_error(png_ptr, "Missing PLTE before IDAT");
+ break;
+ }
+ }
+#endif
+ else if (!png_memcmp(chunk_name, png_PLTE, 4))
+ png_handle_PLTE(png_ptr, info_ptr, length);
+ else if (!png_memcmp(chunk_name, png_IDAT, 4))
+ {
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before IDAT");
+ else if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE &&
+ !(png_ptr->mode & PNG_HAVE_PLTE))
+ png_error(png_ptr, "Missing PLTE before IDAT");
+
+ png_ptr->idat_size = length;
+ png_ptr->mode |= PNG_HAVE_IDAT;
+ break;
+ }
+#ifdef PNG_READ_bKGD_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_bKGD, 4))
+ png_handle_bKGD(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_cHRM_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_cHRM, 4))
+ png_handle_cHRM(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_gAMA_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_gAMA, 4))
+ png_handle_gAMA(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_hIST_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_hIST, 4))
+ png_handle_hIST(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_oFFs_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_oFFs, 4))
+ png_handle_oFFs(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_pCAL_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_pCAL, 4))
+ png_handle_pCAL(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_sCAL_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_sCAL, 4))
+ png_handle_sCAL(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_pHYs_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_pHYs, 4))
+ png_handle_pHYs(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_sBIT_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_sBIT, 4))
+ png_handle_sBIT(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_sRGB_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_sRGB, 4))
+ png_handle_sRGB(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_iCCP_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_iCCP, 4))
+ png_handle_iCCP(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_sPLT_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_sPLT, 4))
+ png_handle_sPLT(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_tEXt_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_tEXt, 4))
+ png_handle_tEXt(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_tIME_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_tIME, 4))
+ png_handle_tIME(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_tRNS_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_tRNS, 4))
+ png_handle_tRNS(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_zTXt_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_zTXt, 4))
+ png_handle_zTXt(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_iTXt_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_iTXt, 4))
+ png_handle_iTXt(png_ptr, info_ptr, length);
+#endif
+ else
+ png_handle_unknown(png_ptr, info_ptr, length);
+ }
+}
+#endif /* PNG_SEQUENTIAL_READ_SUPPORTED */
+
+/* Optional call to update the users info_ptr structure */
+void PNGAPI
+png_read_update_info(png_structp png_ptr, png_infop info_ptr)
+{
+ png_debug(1, "in png_read_update_info");
+
+ if (png_ptr == NULL)
+ return;
+ if (!(png_ptr->flags & PNG_FLAG_ROW_INIT))
+ png_read_start_row(png_ptr);
+ else
+ png_warning(png_ptr,
+ "Ignoring extra png_read_update_info() call; row buffer not reallocated");
+
+ png_read_transform_info(png_ptr, info_ptr);
+}
+
+#ifdef PNG_SEQUENTIAL_READ_SUPPORTED
+/* Initialize palette, background, etc, after transformations
+ * are set, but before any reading takes place. This allows
+ * the user to obtain a gamma-corrected palette, for example.
+ * If the user doesn't call this, we will do it ourselves.
+ */
+void PNGAPI
+png_start_read_image(png_structp png_ptr)
+{
+ png_debug(1, "in png_start_read_image");
+
+ if (png_ptr == NULL)
+ return;
+ if (!(png_ptr->flags & PNG_FLAG_ROW_INIT))
+ png_read_start_row(png_ptr);
+}
+#endif /* PNG_SEQUENTIAL_READ_SUPPORTED */
+
+#ifdef PNG_SEQUENTIAL_READ_SUPPORTED
+void PNGAPI
+png_read_row(png_structp png_ptr, png_bytep row, png_bytep dsp_row)
+{
+ PNG_CONST PNG_IDAT;
+ PNG_CONST int png_pass_dsp_mask[7] = {0xff, 0x0f, 0xff, 0x33, 0xff, 0x55,
+ 0xff};
+ PNG_CONST int png_pass_mask[7] = {0x80, 0x08, 0x88, 0x22, 0xaa, 0x55, 0xff};
+ int ret;
+
+ if (png_ptr == NULL)
+ return;
+
+ png_debug2(1, "in png_read_row (row %lu, pass %d)",
+ png_ptr->row_number, png_ptr->pass);
+
+ if (!(png_ptr->flags & PNG_FLAG_ROW_INIT))
+ png_read_start_row(png_ptr);
+ if (png_ptr->row_number == 0 && png_ptr->pass == 0)
+ {
+ /* Check for transforms that have been set but were defined out */
+#if defined(PNG_WRITE_INVERT_SUPPORTED) && !defined(PNG_READ_INVERT_SUPPORTED)
+ if (png_ptr->transformations & PNG_INVERT_MONO)
+ png_warning(png_ptr, "PNG_READ_INVERT_SUPPORTED is not defined.");
+#endif
+#if defined(PNG_WRITE_FILLER_SUPPORTED) && !defined(PNG_READ_FILLER_SUPPORTED)
+ if (png_ptr->transformations & PNG_FILLER)
+ png_warning(png_ptr, "PNG_READ_FILLER_SUPPORTED is not defined.");
+#endif
+#if defined(PNG_WRITE_PACKSWAP_SUPPORTED) && \
+ !defined(PNG_READ_PACKSWAP_SUPPORTED)
+ if (png_ptr->transformations & PNG_PACKSWAP)
+ png_warning(png_ptr, "PNG_READ_PACKSWAP_SUPPORTED is not defined.");
+#endif
+#if defined(PNG_WRITE_PACK_SUPPORTED) && !defined(PNG_READ_PACK_SUPPORTED)
+ if (png_ptr->transformations & PNG_PACK)
+ png_warning(png_ptr, "PNG_READ_PACK_SUPPORTED is not defined.");
+#endif
+#if defined(PNG_WRITE_SHIFT_SUPPORTED) && !defined(PNG_READ_SHIFT_SUPPORTED)
+ if (png_ptr->transformations & PNG_SHIFT)
+ png_warning(png_ptr, "PNG_READ_SHIFT_SUPPORTED is not defined.");
+#endif
+#if defined(PNG_WRITE_BGR_SUPPORTED) && !defined(PNG_READ_BGR_SUPPORTED)
+ if (png_ptr->transformations & PNG_BGR)
+ png_warning(png_ptr, "PNG_READ_BGR_SUPPORTED is not defined.");
+#endif
+#if defined(PNG_WRITE_SWAP_SUPPORTED) && !defined(PNG_READ_SWAP_SUPPORTED)
+ if (png_ptr->transformations & PNG_SWAP_BYTES)
+ png_warning(png_ptr, "PNG_READ_SWAP_SUPPORTED is not defined.");
+#endif
+ }
+
+#ifdef PNG_READ_INTERLACING_SUPPORTED
+ /* If interlaced and we do not need a new row, combine row and return */
+ if (png_ptr->interlaced && (png_ptr->transformations & PNG_INTERLACE))
+ {
+ switch (png_ptr->pass)
+ {
+ case 0:
+ if (png_ptr->row_number & 0x07)
+ {
+ if (dsp_row != NULL)
+ png_combine_row(png_ptr, dsp_row,
+ png_pass_dsp_mask[png_ptr->pass]);
+ png_read_finish_row(png_ptr);
+ return;
+ }
+ break;
+ case 1:
+ if ((png_ptr->row_number & 0x07) || png_ptr->width < 5)
+ {
+ if (dsp_row != NULL)
+ png_combine_row(png_ptr, dsp_row,
+ png_pass_dsp_mask[png_ptr->pass]);
+ png_read_finish_row(png_ptr);
+ return;
+ }
+ break;
+ case 2:
+ if ((png_ptr->row_number & 0x07) != 4)
+ {
+ if (dsp_row != NULL && (png_ptr->row_number & 4))
+ png_combine_row(png_ptr, dsp_row,
+ png_pass_dsp_mask[png_ptr->pass]);
+ png_read_finish_row(png_ptr);
+ return;
+ }
+ break;
+ case 3:
+ if ((png_ptr->row_number & 3) || png_ptr->width < 3)
+ {
+ if (dsp_row != NULL)
+ png_combine_row(png_ptr, dsp_row,
+ png_pass_dsp_mask[png_ptr->pass]);
+ png_read_finish_row(png_ptr);
+ return;
+ }
+ break;
+ case 4:
+ if ((png_ptr->row_number & 3) != 2)
+ {
+ if (dsp_row != NULL && (png_ptr->row_number & 2))
+ png_combine_row(png_ptr, dsp_row,
+ png_pass_dsp_mask[png_ptr->pass]);
+ png_read_finish_row(png_ptr);
+ return;
+ }
+ break;
+ case 5:
+ if ((png_ptr->row_number & 1) || png_ptr->width < 2)
+ {
+ if (dsp_row != NULL)
+ png_combine_row(png_ptr, dsp_row,
+ png_pass_dsp_mask[png_ptr->pass]);
+ png_read_finish_row(png_ptr);
+ return;
+ }
+ break;
+ case 6:
+ if (!(png_ptr->row_number & 1))
+ {
+ png_read_finish_row(png_ptr);
+ return;
+ }
+ break;
+ }
+ }
+#endif
+
+ if (!(png_ptr->mode & PNG_HAVE_IDAT))
+ png_error(png_ptr, "Invalid attempt to read row data");
+
+ png_ptr->zstream.next_out = png_ptr->row_buf;
+ png_ptr->zstream.avail_out =
+ (uInt)(PNG_ROWBYTES(png_ptr->pixel_depth,
+ png_ptr->iwidth) + 1);
+ do
+ {
+ if (!(png_ptr->zstream.avail_in))
+ {
+ while (!png_ptr->idat_size)
+ {
+ png_crc_finish(png_ptr, 0);
+
+ png_ptr->idat_size = png_read_chunk_header(png_ptr);
+ if (png_memcmp(png_ptr->chunk_name, png_IDAT, 4))
+ png_error(png_ptr, "Not enough image data");
+ }
+ png_ptr->zstream.avail_in = (uInt)png_ptr->zbuf_size;
+ png_ptr->zstream.next_in = png_ptr->zbuf;
+ if (png_ptr->zbuf_size > png_ptr->idat_size)
+ png_ptr->zstream.avail_in = (uInt)png_ptr->idat_size;
+ png_crc_read(png_ptr, png_ptr->zbuf,
+ (png_size_t)png_ptr->zstream.avail_in);
+ png_ptr->idat_size -= png_ptr->zstream.avail_in;
+ }
+ ret = inflate(&png_ptr->zstream, Z_PARTIAL_FLUSH);
+ if (ret == Z_STREAM_END)
+ {
+ if (png_ptr->zstream.avail_out || png_ptr->zstream.avail_in ||
+ png_ptr->idat_size)
+ png_error(png_ptr, "Extra compressed data");
+ png_ptr->mode |= PNG_AFTER_IDAT;
+ png_ptr->flags |= PNG_FLAG_ZLIB_FINISHED;
+ break;
+ }
+ if (ret != Z_OK)
+ png_error(png_ptr, png_ptr->zstream.msg ? png_ptr->zstream.msg :
+ "Decompression error");
+
+ } while (png_ptr->zstream.avail_out);
+
+ png_ptr->row_info.color_type = png_ptr->color_type;
+ png_ptr->row_info.width = png_ptr->iwidth;
+ png_ptr->row_info.channels = png_ptr->channels;
+ png_ptr->row_info.bit_depth = png_ptr->bit_depth;
+ png_ptr->row_info.pixel_depth = png_ptr->pixel_depth;
+ png_ptr->row_info.rowbytes = PNG_ROWBYTES(png_ptr->row_info.pixel_depth,
+ png_ptr->row_info.width);
+
+ if (png_ptr->row_buf[0])
+ png_read_filter_row(png_ptr, &(png_ptr->row_info),
+ png_ptr->row_buf + 1, png_ptr->prev_row + 1,
+ (int)(png_ptr->row_buf[0]));
+
+ png_memcpy_check(png_ptr, png_ptr->prev_row, png_ptr->row_buf,
+ png_ptr->rowbytes + 1);
+
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+ if ((png_ptr->mng_features_permitted & PNG_FLAG_MNG_FILTER_64) &&
+ (png_ptr->filter_type == PNG_INTRAPIXEL_DIFFERENCING))
+ {
+ /* Intrapixel differencing */
+ png_do_read_intrapixel(&(png_ptr->row_info), png_ptr->row_buf + 1);
+ }
+#endif
+
+
+ if (png_ptr->transformations || (png_ptr->flags&PNG_FLAG_STRIP_ALPHA))
+ png_do_read_transformations(png_ptr);
+
+#ifdef PNG_READ_INTERLACING_SUPPORTED
+ /* Blow up interlaced rows to full size */
+ if (png_ptr->interlaced &&
+ (png_ptr->transformations & PNG_INTERLACE))
+ {
+ if (png_ptr->pass < 6)
+ /* Old interface (pre-1.0.9):
+ * png_do_read_interlace(&(png_ptr->row_info),
+ * png_ptr->row_buf + 1, png_ptr->pass, png_ptr->transformations);
+ */
+ png_do_read_interlace(png_ptr);
+
+ if (dsp_row != NULL)
+ png_combine_row(png_ptr, dsp_row,
+ png_pass_dsp_mask[png_ptr->pass]);
+ if (row != NULL)
+ png_combine_row(png_ptr, row,
+ png_pass_mask[png_ptr->pass]);
+ }
+ else
+#endif
+ {
+ if (row != NULL)
+ png_combine_row(png_ptr, row, 0xff);
+ if (dsp_row != NULL)
+ png_combine_row(png_ptr, dsp_row, 0xff);
+ }
+ png_read_finish_row(png_ptr);
+
+ if (png_ptr->read_row_fn != NULL)
+ (*(png_ptr->read_row_fn))(png_ptr, png_ptr->row_number, png_ptr->pass);
+}
+#endif /* PNG_SEQUENTIAL_READ_SUPPORTED */
+
+#ifdef PNG_SEQUENTIAL_READ_SUPPORTED
+/* Read one or more rows of image data. If the image is interlaced,
+ * and png_set_interlace_handling() has been called, the rows need to
+ * contain the contents of the rows from the previous pass. If the
+ * image has alpha or transparency, and png_handle_alpha()[*] has been
+ * called, the rows contents must be initialized to the contents of the
+ * screen.
+ *
+ * "row" holds the actual image, and pixels are placed in it
+ * as they arrive. If the image is displayed after each pass, it will
+ * appear to "sparkle" in. "display_row" can be used to display a
+ * "chunky" progressive image, with finer detail added as it becomes
+ * available. If you do not want this "chunky" display, you may pass
+ * NULL for display_row. If you do not want the sparkle display, and
+ * you have not called png_handle_alpha(), you may pass NULL for rows.
+ * If you have called png_handle_alpha(), and the image has either an
+ * alpha channel or a transparency chunk, you must provide a buffer for
+ * rows. In this case, you do not have to provide a display_row buffer
+ * also, but you may. If the image is not interlaced, or if you have
+ * not called png_set_interlace_handling(), the display_row buffer will
+ * be ignored, so pass NULL to it.
+ *
+ * [*] png_handle_alpha() does not exist yet, as of this version of libpng
+ */
+
+void PNGAPI
+png_read_rows(png_structp png_ptr, png_bytepp row,
+ png_bytepp display_row, png_uint_32 num_rows)
+{
+ png_uint_32 i;
+ png_bytepp rp;
+ png_bytepp dp;
+
+ png_debug(1, "in png_read_rows");
+
+ if (png_ptr == NULL)
+ return;
+ rp = row;
+ dp = display_row;
+ if (rp != NULL && dp != NULL)
+ for (i = 0; i < num_rows; i++)
+ {
+ png_bytep rptr = *rp++;
+ png_bytep dptr = *dp++;
+
+ png_read_row(png_ptr, rptr, dptr);
+ }
+ else if (rp != NULL)
+ for (i = 0; i < num_rows; i++)
+ {
+ png_bytep rptr = *rp;
+ png_read_row(png_ptr, rptr, png_bytep_NULL);
+ rp++;
+ }
+ else if (dp != NULL)
+ for (i = 0; i < num_rows; i++)
+ {
+ png_bytep dptr = *dp;
+ png_read_row(png_ptr, png_bytep_NULL, dptr);
+ dp++;
+ }
+}
+#endif /* PNG_SEQUENTIAL_READ_SUPPORTED */
+
+#ifdef PNG_SEQUENTIAL_READ_SUPPORTED
+/* Read the entire image. If the image has an alpha channel or a tRNS
+ * chunk, and you have called png_handle_alpha()[*], you will need to
+ * initialize the image to the current image that PNG will be overlaying.
+ * We set the num_rows again here, in case it was incorrectly set in
+ * png_read_start_row() by a call to png_read_update_info() or
+ * png_start_read_image() if png_set_interlace_handling() wasn't called
+ * prior to either of these functions like it should have been. You can
+ * only call this function once. If you desire to have an image for
+ * each pass of a interlaced image, use png_read_rows() instead.
+ *
+ * [*] png_handle_alpha() does not exist yet, as of this version of libpng
+ */
+void PNGAPI
+png_read_image(png_structp png_ptr, png_bytepp image)
+{
+ png_uint_32 i, image_height;
+ int pass, j;
+ png_bytepp rp;
+
+ png_debug(1, "in png_read_image");
+
+ if (png_ptr == NULL)
+ return;
+
+#ifdef PNG_READ_INTERLACING_SUPPORTED
+ pass = png_set_interlace_handling(png_ptr);
+#else
+ if (png_ptr->interlaced)
+ png_error(png_ptr,
+ "Cannot read interlaced image -- interlace handler disabled.");
+ pass = 1;
+#endif
+
+
+ image_height=png_ptr->height;
+ png_ptr->num_rows = image_height; /* Make sure this is set correctly */
+
+ for (j = 0; j < pass; j++)
+ {
+ rp = image;
+ for (i = 0; i < image_height; i++)
+ {
+ png_read_row(png_ptr, *rp, png_bytep_NULL);
+ rp++;
+ }
+ }
+}
+#endif /* PNG_SEQUENTIAL_READ_SUPPORTED */
+
+#ifdef PNG_SEQUENTIAL_READ_SUPPORTED
+/* Read the end of the PNG file. Will not read past the end of the
+ * file, will verify the end is accurate, and will read any comments
+ * or time information at the end of the file, if info is not NULL.
+ */
+void PNGAPI
+png_read_end(png_structp png_ptr, png_infop info_ptr)
+{
+ png_debug(1, "in png_read_end");
+
+ if (png_ptr == NULL)
+ return;
+ png_crc_finish(png_ptr, 0); /* Finish off CRC from last IDAT chunk */
+
+ do
+ {
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_CONST PNG_IHDR;
+ PNG_CONST PNG_IDAT;
+ PNG_CONST PNG_IEND;
+ PNG_CONST PNG_PLTE;
+#ifdef PNG_READ_bKGD_SUPPORTED
+ PNG_CONST PNG_bKGD;
+#endif
+#ifdef PNG_READ_cHRM_SUPPORTED
+ PNG_CONST PNG_cHRM;
+#endif
+#ifdef PNG_READ_gAMA_SUPPORTED
+ PNG_CONST PNG_gAMA;
+#endif
+#ifdef PNG_READ_hIST_SUPPORTED
+ PNG_CONST PNG_hIST;
+#endif
+#ifdef PNG_READ_iCCP_SUPPORTED
+ PNG_CONST PNG_iCCP;
+#endif
+#ifdef PNG_READ_iTXt_SUPPORTED
+ PNG_CONST PNG_iTXt;
+#endif
+#ifdef PNG_READ_oFFs_SUPPORTED
+ PNG_CONST PNG_oFFs;
+#endif
+#ifdef PNG_READ_pCAL_SUPPORTED
+ PNG_CONST PNG_pCAL;
+#endif
+#ifdef PNG_READ_pHYs_SUPPORTED
+ PNG_CONST PNG_pHYs;
+#endif
+#ifdef PNG_READ_sBIT_SUPPORTED
+ PNG_CONST PNG_sBIT;
+#endif
+#ifdef PNG_READ_sCAL_SUPPORTED
+ PNG_CONST PNG_sCAL;
+#endif
+#ifdef PNG_READ_sPLT_SUPPORTED
+ PNG_CONST PNG_sPLT;
+#endif
+#ifdef PNG_READ_sRGB_SUPPORTED
+ PNG_CONST PNG_sRGB;
+#endif
+#ifdef PNG_READ_tEXt_SUPPORTED
+ PNG_CONST PNG_tEXt;
+#endif
+#ifdef PNG_READ_tIME_SUPPORTED
+ PNG_CONST PNG_tIME;
+#endif
+#ifdef PNG_READ_tRNS_SUPPORTED
+ PNG_CONST PNG_tRNS;
+#endif
+#ifdef PNG_READ_zTXt_SUPPORTED
+ PNG_CONST PNG_zTXt;
+#endif
+#endif /* PNG_USE_LOCAL_ARRAYS */
+ png_uint_32 length = png_read_chunk_header(png_ptr);
+ PNG_CONST png_bytep chunk_name = png_ptr->chunk_name;
+
+ if (!png_memcmp(chunk_name, png_IHDR, 4))
+ png_handle_IHDR(png_ptr, info_ptr, length);
+ else if (!png_memcmp(chunk_name, png_IEND, 4))
+ png_handle_IEND(png_ptr, info_ptr, length);
+#ifdef PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+ else if (png_handle_as_unknown(png_ptr, chunk_name))
+ {
+ if (!png_memcmp(chunk_name, png_IDAT, 4))
+ {
+ if ((length > 0) || (png_ptr->mode & PNG_HAVE_CHUNK_AFTER_IDAT))
+ png_error(png_ptr, "Too many IDAT's found");
+ }
+ png_handle_unknown(png_ptr, info_ptr, length);
+ if (!png_memcmp(chunk_name, png_PLTE, 4))
+ png_ptr->mode |= PNG_HAVE_PLTE;
+ }
+#endif
+ else if (!png_memcmp(chunk_name, png_IDAT, 4))
+ {
+ /* Zero length IDATs are legal after the last IDAT has been
+ * read, but not after other chunks have been read.
+ */
+ if ((length > 0) || (png_ptr->mode & PNG_HAVE_CHUNK_AFTER_IDAT))
+ png_error(png_ptr, "Too many IDAT's found");
+ png_crc_finish(png_ptr, length);
+ }
+ else if (!png_memcmp(chunk_name, png_PLTE, 4))
+ png_handle_PLTE(png_ptr, info_ptr, length);
+#ifdef PNG_READ_bKGD_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_bKGD, 4))
+ png_handle_bKGD(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_cHRM_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_cHRM, 4))
+ png_handle_cHRM(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_gAMA_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_gAMA, 4))
+ png_handle_gAMA(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_hIST_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_hIST, 4))
+ png_handle_hIST(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_oFFs_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_oFFs, 4))
+ png_handle_oFFs(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_pCAL_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_pCAL, 4))
+ png_handle_pCAL(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_sCAL_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_sCAL, 4))
+ png_handle_sCAL(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_pHYs_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_pHYs, 4))
+ png_handle_pHYs(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_sBIT_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_sBIT, 4))
+ png_handle_sBIT(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_sRGB_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_sRGB, 4))
+ png_handle_sRGB(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_iCCP_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_iCCP, 4))
+ png_handle_iCCP(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_sPLT_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_sPLT, 4))
+ png_handle_sPLT(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_tEXt_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_tEXt, 4))
+ png_handle_tEXt(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_tIME_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_tIME, 4))
+ png_handle_tIME(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_tRNS_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_tRNS, 4))
+ png_handle_tRNS(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_zTXt_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_zTXt, 4))
+ png_handle_zTXt(png_ptr, info_ptr, length);
+#endif
+#ifdef PNG_READ_iTXt_SUPPORTED
+ else if (!png_memcmp(chunk_name, png_iTXt, 4))
+ png_handle_iTXt(png_ptr, info_ptr, length);
+#endif
+ else
+ png_handle_unknown(png_ptr, info_ptr, length);
+ } while (!(png_ptr->mode & PNG_HAVE_IEND));
+}
+#endif /* PNG_SEQUENTIAL_READ_SUPPORTED */
+
+/* Free all memory used by the read */
+void PNGAPI
+png_destroy_read_struct(png_structpp png_ptr_ptr, png_infopp info_ptr_ptr,
+ png_infopp end_info_ptr_ptr)
+{
+ png_structp png_ptr = NULL;
+ png_infop info_ptr = NULL, end_info_ptr = NULL;
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_free_ptr free_fn = NULL;
+ png_voidp mem_ptr = NULL;
+#endif
+
+ png_debug(1, "in png_destroy_read_struct");
+
+ if (png_ptr_ptr != NULL)
+ png_ptr = *png_ptr_ptr;
+ if (png_ptr == NULL)
+ return;
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ free_fn = png_ptr->free_fn;
+ mem_ptr = png_ptr->mem_ptr;
+#endif
+
+ if (info_ptr_ptr != NULL)
+ info_ptr = *info_ptr_ptr;
+
+ if (end_info_ptr_ptr != NULL)
+ end_info_ptr = *end_info_ptr_ptr;
+
+ png_read_destroy(png_ptr, info_ptr, end_info_ptr);
+
+ if (info_ptr != NULL)
+ {
+#ifdef PNG_TEXT_SUPPORTED
+ png_free_data(png_ptr, info_ptr, PNG_FREE_TEXT, -1);
+#endif
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_destroy_struct_2((png_voidp)info_ptr, (png_free_ptr)free_fn,
+ (png_voidp)mem_ptr);
+#else
+ png_destroy_struct((png_voidp)info_ptr);
+#endif
+ *info_ptr_ptr = NULL;
+ }
+
+ if (end_info_ptr != NULL)
+ {
+#ifdef PNG_READ_TEXT_SUPPORTED
+ png_free_data(png_ptr, end_info_ptr, PNG_FREE_TEXT, -1);
+#endif
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_destroy_struct_2((png_voidp)end_info_ptr, (png_free_ptr)free_fn,
+ (png_voidp)mem_ptr);
+#else
+ png_destroy_struct((png_voidp)end_info_ptr);
+#endif
+ *end_info_ptr_ptr = NULL;
+ }
+
+ if (png_ptr != NULL)
+ {
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_destroy_struct_2((png_voidp)png_ptr, (png_free_ptr)free_fn,
+ (png_voidp)mem_ptr);
+#else
+ png_destroy_struct((png_voidp)png_ptr);
+#endif
+ *png_ptr_ptr = NULL;
+ }
+}
+
+/* Free all memory used by the read (old method) */
+void /* PRIVATE */
+png_read_destroy(png_structp png_ptr, png_infop info_ptr,
+ png_infop end_info_ptr)
+{
+#ifdef PNG_SETJMP_SUPPORTED
+ jmp_buf tmp_jmp;
+#endif
+ png_error_ptr error_fn;
+ png_error_ptr warning_fn;
+ png_voidp error_ptr;
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_free_ptr free_fn;
+#endif
+
+ png_debug(1, "in png_read_destroy");
+
+ if (info_ptr != NULL)
+ png_info_destroy(png_ptr, info_ptr);
+
+ if (end_info_ptr != NULL)
+ png_info_destroy(png_ptr, end_info_ptr);
+
+ png_free(png_ptr, png_ptr->zbuf);
+ png_free(png_ptr, png_ptr->big_row_buf);
+ png_free(png_ptr, png_ptr->prev_row);
+ png_free(png_ptr, png_ptr->chunkdata);
+#ifdef PNG_READ_DITHER_SUPPORTED
+ png_free(png_ptr, png_ptr->palette_lookup);
+ png_free(png_ptr, png_ptr->dither_index);
+#endif
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ png_free(png_ptr, png_ptr->gamma_table);
+#endif
+#ifdef PNG_READ_BACKGROUND_SUPPORTED
+ png_free(png_ptr, png_ptr->gamma_from_1);
+ png_free(png_ptr, png_ptr->gamma_to_1);
+#endif
+#ifdef PNG_FREE_ME_SUPPORTED
+ if (png_ptr->free_me & PNG_FREE_PLTE)
+ png_zfree(png_ptr, png_ptr->palette);
+ png_ptr->free_me &= ~PNG_FREE_PLTE;
+#else
+ if (png_ptr->flags & PNG_FLAG_FREE_PLTE)
+ png_zfree(png_ptr, png_ptr->palette);
+ png_ptr->flags &= ~PNG_FLAG_FREE_PLTE;
+#endif
+#if defined(PNG_tRNS_SUPPORTED) || \
+ defined(PNG_READ_EXPAND_SUPPORTED) || defined(PNG_READ_BACKGROUND_SUPPORTED)
+#ifdef PNG_FREE_ME_SUPPORTED
+ if (png_ptr->free_me & PNG_FREE_TRNS)
+ png_free(png_ptr, png_ptr->trans);
+ png_ptr->free_me &= ~PNG_FREE_TRNS;
+#else
+ if (png_ptr->flags & PNG_FLAG_FREE_TRNS)
+ png_free(png_ptr, png_ptr->trans);
+ png_ptr->flags &= ~PNG_FLAG_FREE_TRNS;
+#endif
+#endif
+#ifdef PNG_READ_hIST_SUPPORTED
+#ifdef PNG_FREE_ME_SUPPORTED
+ if (png_ptr->free_me & PNG_FREE_HIST)
+ png_free(png_ptr, png_ptr->hist);
+ png_ptr->free_me &= ~PNG_FREE_HIST;
+#else
+ if (png_ptr->flags & PNG_FLAG_FREE_HIST)
+ png_free(png_ptr, png_ptr->hist);
+ png_ptr->flags &= ~PNG_FLAG_FREE_HIST;
+#endif
+#endif
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ if (png_ptr->gamma_16_table != NULL)
+ {
+ int i;
+ int istop = (1 << (8 - png_ptr->gamma_shift));
+ for (i = 0; i < istop; i++)
+ {
+ png_free(png_ptr, png_ptr->gamma_16_table[i]);
+ }
+ png_free(png_ptr, png_ptr->gamma_16_table);
+ }
+#ifdef PNG_READ_BACKGROUND_SUPPORTED
+ if (png_ptr->gamma_16_from_1 != NULL)
+ {
+ int i;
+ int istop = (1 << (8 - png_ptr->gamma_shift));
+ for (i = 0; i < istop; i++)
+ {
+ png_free(png_ptr, png_ptr->gamma_16_from_1[i]);
+ }
+ png_free(png_ptr, png_ptr->gamma_16_from_1);
+ }
+ if (png_ptr->gamma_16_to_1 != NULL)
+ {
+ int i;
+ int istop = (1 << (8 - png_ptr->gamma_shift));
+ for (i = 0; i < istop; i++)
+ {
+ png_free(png_ptr, png_ptr->gamma_16_to_1[i]);
+ }
+ png_free(png_ptr, png_ptr->gamma_16_to_1);
+ }
+#endif
+#endif
+#ifdef PNG_TIME_RFC1123_SUPPORTED
+ png_free(png_ptr, png_ptr->time_buffer);
+#endif
+
+ inflateEnd(&png_ptr->zstream);
+#ifdef PNG_PROGRESSIVE_READ_SUPPORTED
+ png_free(png_ptr, png_ptr->save_buffer);
+#endif
+
+#ifdef PNG_PROGRESSIVE_READ_SUPPORTED
+#ifdef PNG_TEXT_SUPPORTED
+ png_free(png_ptr, png_ptr->current_text);
+#endif /* PNG_TEXT_SUPPORTED */
+#endif /* PNG_PROGRESSIVE_READ_SUPPORTED */
+
+ /* Save the important info out of the png_struct, in case it is
+ * being used again.
+ */
+#ifdef PNG_SETJMP_SUPPORTED
+ png_memcpy(tmp_jmp, png_ptr->jmpbuf, png_sizeof(jmp_buf));
+#endif
+
+ error_fn = png_ptr->error_fn;
+ warning_fn = png_ptr->warning_fn;
+ error_ptr = png_ptr->error_ptr;
+#ifdef PNG_USER_MEM_SUPPORTED
+ free_fn = png_ptr->free_fn;
+#endif
+
+ png_memset(png_ptr, 0, png_sizeof(png_struct));
+
+ png_ptr->error_fn = error_fn;
+ png_ptr->warning_fn = warning_fn;
+ png_ptr->error_ptr = error_ptr;
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_ptr->free_fn = free_fn;
+#endif
+
+#ifdef PNG_SETJMP_SUPPORTED
+ png_memcpy(png_ptr->jmpbuf, tmp_jmp, png_sizeof(jmp_buf));
+#endif
+
+}
+
+void PNGAPI
+png_set_read_status_fn(png_structp png_ptr, png_read_status_ptr read_row_fn)
+{
+ if (png_ptr == NULL)
+ return;
+ png_ptr->read_row_fn = read_row_fn;
+}
+
+
+#ifdef PNG_SEQUENTIAL_READ_SUPPORTED
+#ifdef PNG_INFO_IMAGE_SUPPORTED
+void PNGAPI
+png_read_png(png_structp png_ptr, png_infop info_ptr,
+ int transforms,
+ voidp params)
+{
+ int row;
+
+ if (png_ptr == NULL)
+ return;
+#ifdef PNG_READ_INVERT_ALPHA_SUPPORTED
+ /* Invert the alpha channel from opacity to transparency
+ */
+ if (transforms & PNG_TRANSFORM_INVERT_ALPHA)
+ png_set_invert_alpha(png_ptr);
+#endif
+
+ /* png_read_info() gives us all of the information from the
+ * PNG file before the first IDAT (image data chunk).
+ */
+ png_read_info(png_ptr, info_ptr);
+ if (info_ptr->height > PNG_UINT_32_MAX/png_sizeof(png_bytep))
+ png_error(png_ptr, "Image is too high to process with png_read_png()");
+
+ /* -------------- image transformations start here ------------------- */
+
+#ifdef PNG_READ_16_TO_8_SUPPORTED
+ /* Tell libpng to strip 16 bit/color files down to 8 bits per color.
+ */
+ if (transforms & PNG_TRANSFORM_STRIP_16)
+ png_set_strip_16(png_ptr);
+#endif
+
+#ifdef PNG_READ_STRIP_ALPHA_SUPPORTED
+ /* Strip alpha bytes from the input data without combining with
+ * the background (not recommended).
+ */
+ if (transforms & PNG_TRANSFORM_STRIP_ALPHA)
+ png_set_strip_alpha(png_ptr);
+#endif
+
+#if defined(PNG_READ_PACK_SUPPORTED) && !defined(PNG_READ_EXPAND_SUPPORTED)
+ /* Extract multiple pixels with bit depths of 1, 2, or 4 from a single
+ * byte into separate bytes (useful for paletted and grayscale images).
+ */
+ if (transforms & PNG_TRANSFORM_PACKING)
+ png_set_packing(png_ptr);
+#endif
+
+#ifdef PNG_READ_PACKSWAP_SUPPORTED
+ /* Change the order of packed pixels to least significant bit first
+ * (not useful if you are using png_set_packing).
+ */
+ if (transforms & PNG_TRANSFORM_PACKSWAP)
+ png_set_packswap(png_ptr);
+#endif
+
+#ifdef PNG_READ_EXPAND_SUPPORTED
+ /* Expand paletted colors into true RGB triplets
+ * Expand grayscale images to full 8 bits from 1, 2, or 4 bits/pixel
+ * Expand paletted or RGB images with transparency to full alpha
+ * channels so the data will be available as RGBA quartets.
+ */
+ if (transforms & PNG_TRANSFORM_EXPAND)
+ if ((png_ptr->bit_depth < 8) ||
+ (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE) ||
+ (png_get_valid(png_ptr, info_ptr, PNG_INFO_tRNS)))
+ png_set_expand(png_ptr);
+#endif
+
+ /* We don't handle background color or gamma transformation or dithering.
+ */
+
+#ifdef PNG_READ_INVERT_SUPPORTED
+ /* Invert monochrome files to have 0 as white and 1 as black
+ */
+ if (transforms & PNG_TRANSFORM_INVERT_MONO)
+ png_set_invert_mono(png_ptr);
+#endif
+
+#ifdef PNG_READ_SHIFT_SUPPORTED
+ /* If you want to shift the pixel values from the range [0,255] or
+ * [0,65535] to the original [0,7] or [0,31], or whatever range the
+ * colors were originally in:
+ */
+ if ((transforms & PNG_TRANSFORM_SHIFT)
+ && png_get_valid(png_ptr, info_ptr, PNG_INFO_sBIT))
+ {
+ png_color_8p sig_bit;
+
+ png_get_sBIT(png_ptr, info_ptr, &sig_bit);
+ png_set_shift(png_ptr, sig_bit);
+ }
+#endif
+
+#ifdef PNG_READ_BGR_SUPPORTED
+ /* Flip the RGB pixels to BGR (or RGBA to BGRA)
+ */
+ if (transforms & PNG_TRANSFORM_BGR)
+ png_set_bgr(png_ptr);
+#endif
+
+#ifdef PNG_READ_SWAP_ALPHA_SUPPORTED
+ /* Swap the RGBA or GA data to ARGB or AG (or BGRA to ABGR)
+ */
+ if (transforms & PNG_TRANSFORM_SWAP_ALPHA)
+ png_set_swap_alpha(png_ptr);
+#endif
+
+#ifdef PNG_READ_SWAP_SUPPORTED
+ /* Swap bytes of 16 bit files to least significant byte first
+ */
+ if (transforms & PNG_TRANSFORM_SWAP_ENDIAN)
+ png_set_swap(png_ptr);
+#endif
+
+/* Added at libpng-1.2.41 */
+#ifdef PNG_READ_INVERT_ALPHA_SUPPORTED
+ /* Invert the alpha channel from opacity to transparency
+ */
+ if (transforms & PNG_TRANSFORM_INVERT_ALPHA)
+ png_set_invert_alpha(png_ptr);
+#endif
+
+/* Added at libpng-1.2.41 */
+#ifdef PNG_READ_GRAY_TO_RGB_SUPPORTED
+ /* Expand grayscale image to RGB
+ */
+ if (transforms & PNG_TRANSFORM_GRAY_TO_RGB)
+ png_set_gray_to_rgb(png_ptr);
+#endif
+
+ /* We don't handle adding filler bytes */
+
+ /* Optional call to gamma correct and add the background to the palette
+ * and update info structure. REQUIRED if you are expecting libpng to
+ * update the palette for you (i.e., you selected such a transform above).
+ */
+ png_read_update_info(png_ptr, info_ptr);
+
+ /* -------------- image transformations end here ------------------- */
+
+#ifdef PNG_FREE_ME_SUPPORTED
+ png_free_data(png_ptr, info_ptr, PNG_FREE_ROWS, 0);
+#endif
+ if (info_ptr->row_pointers == NULL)
+ {
+ info_ptr->row_pointers = (png_bytepp)png_malloc(png_ptr,
+ info_ptr->height * png_sizeof(png_bytep));
+ png_memset(info_ptr->row_pointers, 0, info_ptr->height
+ * png_sizeof(png_bytep));
+
+#ifdef PNG_FREE_ME_SUPPORTED
+ info_ptr->free_me |= PNG_FREE_ROWS;
+#endif
+
+ for (row = 0; row < (int)info_ptr->height; row++)
+ info_ptr->row_pointers[row] = (png_bytep)png_malloc(png_ptr,
+ png_get_rowbytes(png_ptr, info_ptr));
+ }
+
+ png_read_image(png_ptr, info_ptr->row_pointers);
+ info_ptr->valid |= PNG_INFO_IDAT;
+
+ /* Read rest of file, and get additional chunks in info_ptr - REQUIRED */
+ png_read_end(png_ptr, info_ptr);
+
+ transforms = transforms; /* Quiet compiler warnings */
+ params = params;
+
+}
+#endif /* PNG_INFO_IMAGE_SUPPORTED */
+#endif /* PNG_SEQUENTIAL_READ_SUPPORTED */
+#endif /* PNG_READ_SUPPORTED */
diff --git a/trunk/src/third_party/libpng/pngrio.c b/trunk/src/third_party/libpng/pngrio.c
new file mode 100644
index 0000000..6978682
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngrio.c
@@ -0,0 +1,180 @@
+
+/* pngrio.c - functions for data input
+ *
+ * Last changed in libpng 1.2.43 [February 25, 2010]
+ * Copyright (c) 1998-2010 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ *
+ * This file provides a location for all input. Users who need
+ * special handling are expected to write a function that has the same
+ * arguments as this and performs a similar function, but that possibly
+ * has a different input method. Note that you shouldn't change this
+ * function, but rather write a replacement function and then make
+ * libpng use it at run time with png_set_read_fn(...).
+ */
+
+#define PNG_INTERNAL
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+#ifdef PNG_READ_SUPPORTED
+
+/* Read the data from whatever input you are using. The default routine
+ * reads from a file pointer. Note that this routine sometimes gets called
+ * with very small lengths, so you should implement some kind of simple
+ * buffering if you are using unbuffered reads. This should never be asked
+ * to read more then 64K on a 16 bit machine.
+ */
+void /* PRIVATE */
+png_read_data(png_structp png_ptr, png_bytep data, png_size_t length)
+{
+ png_debug1(4, "reading %d bytes", (int)length);
+
+ if (png_ptr->read_data_fn != NULL)
+ (*(png_ptr->read_data_fn))(png_ptr, data, length);
+ else
+ png_error(png_ptr, "Call to NULL read function");
+}
+
+#ifdef PNG_STDIO_SUPPORTED
+/* This is the function that does the actual reading of data. If you are
+ * not reading from a standard C stream, you should create a replacement
+ * read_data function and use it at run time with png_set_read_fn(), rather
+ * than changing the library.
+ */
+#ifndef USE_FAR_KEYWORD
+void PNGAPI
+png_default_read_data(png_structp png_ptr, png_bytep data, png_size_t length)
+{
+ png_size_t check;
+
+ if (png_ptr == NULL)
+ return;
+ /* fread() returns 0 on error, so it is OK to store this in a png_size_t
+ * instead of an int, which is what fread() actually returns.
+ */
+#ifdef _WIN32_WCE
+ if ( !ReadFile((HANDLE)(png_ptr->io_ptr), data, length, &check, NULL) )
+ check = 0;
+#else
+ check = (png_size_t)fread(data, (png_size_t)1, length,
+ (png_FILE_p)png_ptr->io_ptr);
+#endif
+
+ if (check != length)
+ png_error(png_ptr, "Read Error");
+}
+#else
+/* This is the model-independent version. Since the standard I/O library
+ can't handle far buffers in the medium and small models, we have to copy
+ the data.
+*/
+
+#define NEAR_BUF_SIZE 1024
+#define MIN(a,b) (a <= b ? a : b)
+
+static void PNGAPI
+png_default_read_data(png_structp png_ptr, png_bytep data, png_size_t length)
+{
+ int check;
+ png_byte *n_data;
+ png_FILE_p io_ptr;
+
+ if (png_ptr == NULL)
+ return;
+ /* Check if data really is near. If so, use usual code. */
+ n_data = (png_byte *)CVT_PTR_NOCHECK(data);
+ io_ptr = (png_FILE_p)CVT_PTR(png_ptr->io_ptr);
+ if ((png_bytep)n_data == data)
+ {
+#ifdef _WIN32_WCE
+ if ( !ReadFile((HANDLE)(png_ptr->io_ptr), data, length, &check,
+ NULL) )
+ check = 0;
+#else
+ check = fread(n_data, 1, length, io_ptr);
+#endif
+ }
+ else
+ {
+ png_byte buf[NEAR_BUF_SIZE];
+ png_size_t read, remaining, err;
+ check = 0;
+ remaining = length;
+ do
+ {
+ read = MIN(NEAR_BUF_SIZE, remaining);
+#ifdef _WIN32_WCE
+ if ( !ReadFile((HANDLE)(io_ptr), buf, read, &err, NULL) )
+ err = 0;
+#else
+ err = fread(buf, (png_size_t)1, read, io_ptr);
+#endif
+ png_memcpy(data, buf, read); /* copy far buffer to near buffer */
+ if (err != read)
+ break;
+ else
+ check += err;
+ data += read;
+ remaining -= read;
+ }
+ while (remaining != 0);
+ }
+ if ((png_uint_32)check != (png_uint_32)length)
+ png_error(png_ptr, "read Error");
+}
+#endif
+#endif
+
+/* This function allows the application to supply a new input function
+ * for libpng if standard C streams aren't being used.
+ *
+ * This function takes as its arguments:
+ * png_ptr - pointer to a png input data structure
+ * io_ptr - pointer to user supplied structure containing info about
+ * the input functions. May be NULL.
+ * read_data_fn - pointer to a new input function that takes as its
+ * arguments a pointer to a png_struct, a pointer to
+ * a location where input data can be stored, and a 32-bit
+ * unsigned int that is the number of bytes to be read.
+ * To exit and output any fatal error messages the new write
+ * function should call png_error(png_ptr, "Error msg").
+ * May be NULL, in which case libpng's default function will
+ * be used.
+ */
+void PNGAPI
+png_set_read_fn(png_structp png_ptr, png_voidp io_ptr,
+ png_rw_ptr read_data_fn)
+{
+ if (png_ptr == NULL)
+ return;
+ png_ptr->io_ptr = io_ptr;
+
+#ifdef PNG_STDIO_SUPPORTED
+ if (read_data_fn != NULL)
+ png_ptr->read_data_fn = read_data_fn;
+ else
+ png_ptr->read_data_fn = png_default_read_data;
+#else
+ png_ptr->read_data_fn = read_data_fn;
+#endif
+
+ /* It is an error to write to a read device */
+ if (png_ptr->write_data_fn != NULL)
+ {
+ png_ptr->write_data_fn = NULL;
+ png_warning(png_ptr,
+ "It's an error to set both read_data_fn and write_data_fn in the ");
+ png_warning(png_ptr,
+ "same structure. Resetting write_data_fn to NULL.");
+ }
+
+#ifdef PNG_WRITE_FLUSH_SUPPORTED
+ png_ptr->output_flush_fn = NULL;
+#endif
+}
+#endif /* PNG_READ_SUPPORTED */
diff --git a/trunk/src/third_party/libpng/pngrtran.c b/trunk/src/third_party/libpng/pngrtran.c
new file mode 100644
index 0000000..af1aa8e
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngrtran.c
@@ -0,0 +1,4457 @@
+
+/* pngrtran.c - transforms the data in a row for PNG readers
+ *
+ * Last changed in libpng 1.2.43 [February 25, 2010]
+ * Copyright (c) 1998-2010 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ *
+ * This file contains functions optionally called by an application
+ * in order to tell libpng how to handle data when reading a PNG.
+ * Transformations that are used in both reading and writing are
+ * in pngtrans.c.
+ */
+
+#define PNG_INTERNAL
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+#ifdef PNG_READ_SUPPORTED
+
+/* Set the action on getting a CRC error for an ancillary or critical chunk. */
+void PNGAPI
+png_set_crc_action(png_structp png_ptr, int crit_action, int ancil_action)
+{
+ png_debug(1, "in png_set_crc_action");
+
+ if (png_ptr == NULL)
+ return;
+
+ /* Tell libpng how we react to CRC errors in critical chunks */
+ switch (crit_action)
+ {
+ case PNG_CRC_NO_CHANGE: /* Leave setting as is */
+ break;
+
+ case PNG_CRC_WARN_USE: /* Warn/use data */
+ png_ptr->flags &= ~PNG_FLAG_CRC_CRITICAL_MASK;
+ png_ptr->flags |= PNG_FLAG_CRC_CRITICAL_USE;
+ break;
+
+ case PNG_CRC_QUIET_USE: /* Quiet/use data */
+ png_ptr->flags &= ~PNG_FLAG_CRC_CRITICAL_MASK;
+ png_ptr->flags |= PNG_FLAG_CRC_CRITICAL_USE |
+ PNG_FLAG_CRC_CRITICAL_IGNORE;
+ break;
+
+ case PNG_CRC_WARN_DISCARD: /* Not a valid action for critical data */
+ png_warning(png_ptr,
+ "Can't discard critical data on CRC error.");
+ case PNG_CRC_ERROR_QUIT: /* Error/quit */
+
+ case PNG_CRC_DEFAULT:
+ default:
+ png_ptr->flags &= ~PNG_FLAG_CRC_CRITICAL_MASK;
+ break;
+ }
+
+ /* Tell libpng how we react to CRC errors in ancillary chunks */
+ switch (ancil_action)
+ {
+ case PNG_CRC_NO_CHANGE: /* Leave setting as is */
+ break;
+
+ case PNG_CRC_WARN_USE: /* Warn/use data */
+ png_ptr->flags &= ~PNG_FLAG_CRC_ANCILLARY_MASK;
+ png_ptr->flags |= PNG_FLAG_CRC_ANCILLARY_USE;
+ break;
+
+ case PNG_CRC_QUIET_USE: /* Quiet/use data */
+ png_ptr->flags &= ~PNG_FLAG_CRC_ANCILLARY_MASK;
+ png_ptr->flags |= PNG_FLAG_CRC_ANCILLARY_USE |
+ PNG_FLAG_CRC_ANCILLARY_NOWARN;
+ break;
+
+ case PNG_CRC_ERROR_QUIT: /* Error/quit */
+ png_ptr->flags &= ~PNG_FLAG_CRC_ANCILLARY_MASK;
+ png_ptr->flags |= PNG_FLAG_CRC_ANCILLARY_NOWARN;
+ break;
+
+ case PNG_CRC_WARN_DISCARD: /* Warn/discard data */
+
+ case PNG_CRC_DEFAULT:
+ default:
+ png_ptr->flags &= ~PNG_FLAG_CRC_ANCILLARY_MASK;
+ break;
+ }
+}
+
+#if defined(PNG_READ_BACKGROUND_SUPPORTED) && \
+ defined(PNG_FLOATING_POINT_SUPPORTED)
+/* Handle alpha and tRNS via a background color */
+void PNGAPI
+png_set_background(png_structp png_ptr,
+ png_color_16p background_color, int background_gamma_code,
+ int need_expand, double background_gamma)
+{
+ png_debug(1, "in png_set_background");
+
+ if (png_ptr == NULL)
+ return;
+ if (background_gamma_code == PNG_BACKGROUND_GAMMA_UNKNOWN)
+ {
+ png_warning(png_ptr, "Application must supply a known background gamma");
+ return;
+ }
+
+ png_ptr->transformations |= PNG_BACKGROUND;
+ png_memcpy(&(png_ptr->background), background_color,
+ png_sizeof(png_color_16));
+ png_ptr->background_gamma = (float)background_gamma;
+ png_ptr->background_gamma_type = (png_byte)(background_gamma_code);
+ png_ptr->transformations |= (need_expand ? PNG_BACKGROUND_EXPAND : 0);
+}
+#endif
+
+#ifdef PNG_READ_16_TO_8_SUPPORTED
+/* Strip 16 bit depth files to 8 bit depth */
+void PNGAPI
+png_set_strip_16(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_strip_16");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->transformations |= PNG_16_TO_8;
+}
+#endif
+
+#ifdef PNG_READ_STRIP_ALPHA_SUPPORTED
+void PNGAPI
+png_set_strip_alpha(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_strip_alpha");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->flags |= PNG_FLAG_STRIP_ALPHA;
+}
+#endif
+
+#ifdef PNG_READ_DITHER_SUPPORTED
+/* Dither file to 8 bit. Supply a palette, the current number
+ * of elements in the palette, the maximum number of elements
+ * allowed, and a histogram if possible. If the current number
+ * of colors is greater then the maximum number, the palette will be
+ * modified to fit in the maximum number. "full_dither" indicates
+ * whether we need a dithering cube set up for RGB images, or if we
+ * simply are reducing the number of colors in a paletted image.
+ */
+
+typedef struct png_dsort_struct
+{
+ struct png_dsort_struct FAR * next;
+ png_byte left;
+ png_byte right;
+} png_dsort;
+typedef png_dsort FAR * png_dsortp;
+typedef png_dsort FAR * FAR * png_dsortpp;
+
+void PNGAPI
+png_set_dither(png_structp png_ptr, png_colorp palette,
+ int num_palette, int maximum_colors, png_uint_16p histogram,
+ int full_dither)
+{
+ png_debug(1, "in png_set_dither");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->transformations |= PNG_DITHER;
+
+ if (!full_dither)
+ {
+ int i;
+
+ png_ptr->dither_index = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)(num_palette * png_sizeof(png_byte)));
+ for (i = 0; i < num_palette; i++)
+ png_ptr->dither_index[i] = (png_byte)i;
+ }
+
+ if (num_palette > maximum_colors)
+ {
+ if (histogram != NULL)
+ {
+ /* This is easy enough, just throw out the least used colors.
+ * Perhaps not the best solution, but good enough.
+ */
+
+ int i;
+
+ /* Initialize an array to sort colors */
+ png_ptr->dither_sort = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)(num_palette * png_sizeof(png_byte)));
+
+ /* Initialize the dither_sort array */
+ for (i = 0; i < num_palette; i++)
+ png_ptr->dither_sort[i] = (png_byte)i;
+
+ /* Find the least used palette entries by starting a
+ * bubble sort, and running it until we have sorted
+ * out enough colors. Note that we don't care about
+ * sorting all the colors, just finding which are
+ * least used.
+ */
+
+ for (i = num_palette - 1; i >= maximum_colors; i--)
+ {
+ int done; /* To stop early if the list is pre-sorted */
+ int j;
+
+ done = 1;
+ for (j = 0; j < i; j++)
+ {
+ if (histogram[png_ptr->dither_sort[j]]
+ < histogram[png_ptr->dither_sort[j + 1]])
+ {
+ png_byte t;
+
+ t = png_ptr->dither_sort[j];
+ png_ptr->dither_sort[j] = png_ptr->dither_sort[j + 1];
+ png_ptr->dither_sort[j + 1] = t;
+ done = 0;
+ }
+ }
+ if (done)
+ break;
+ }
+
+ /* Swap the palette around, and set up a table, if necessary */
+ if (full_dither)
+ {
+ int j = num_palette;
+
+ /* Put all the useful colors within the max, but don't
+ * move the others.
+ */
+ for (i = 0; i < maximum_colors; i++)
+ {
+ if ((int)png_ptr->dither_sort[i] >= maximum_colors)
+ {
+ do
+ j--;
+ while ((int)png_ptr->dither_sort[j] >= maximum_colors);
+ palette[i] = palette[j];
+ }
+ }
+ }
+ else
+ {
+ int j = num_palette;
+
+ /* Move all the used colors inside the max limit, and
+ * develop a translation table.
+ */
+ for (i = 0; i < maximum_colors; i++)
+ {
+ /* Only move the colors we need to */
+ if ((int)png_ptr->dither_sort[i] >= maximum_colors)
+ {
+ png_color tmp_color;
+
+ do
+ j--;
+ while ((int)png_ptr->dither_sort[j] >= maximum_colors);
+
+ tmp_color = palette[j];
+ palette[j] = palette[i];
+ palette[i] = tmp_color;
+ /* Indicate where the color went */
+ png_ptr->dither_index[j] = (png_byte)i;
+ png_ptr->dither_index[i] = (png_byte)j;
+ }
+ }
+
+ /* Find closest color for those colors we are not using */
+ for (i = 0; i < num_palette; i++)
+ {
+ if ((int)png_ptr->dither_index[i] >= maximum_colors)
+ {
+ int min_d, k, min_k, d_index;
+
+ /* Find the closest color to one we threw out */
+ d_index = png_ptr->dither_index[i];
+ min_d = PNG_COLOR_DIST(palette[d_index], palette[0]);
+ for (k = 1, min_k = 0; k < maximum_colors; k++)
+ {
+ int d;
+
+ d = PNG_COLOR_DIST(palette[d_index], palette[k]);
+
+ if (d < min_d)
+ {
+ min_d = d;
+ min_k = k;
+ }
+ }
+ /* Point to closest color */
+ png_ptr->dither_index[i] = (png_byte)min_k;
+ }
+ }
+ }
+ png_free(png_ptr, png_ptr->dither_sort);
+ png_ptr->dither_sort = NULL;
+ }
+ else
+ {
+ /* This is much harder to do simply (and quickly). Perhaps
+ * we need to go through a median cut routine, but those
+ * don't always behave themselves with only a few colors
+ * as input. So we will just find the closest two colors,
+ * and throw out one of them (chosen somewhat randomly).
+ * [We don't understand this at all, so if someone wants to
+ * work on improving it, be our guest - AED, GRP]
+ */
+ int i;
+ int max_d;
+ int num_new_palette;
+ png_dsortp t;
+ png_dsortpp hash;
+
+ t = NULL;
+
+ /* Initialize palette index arrays */
+ png_ptr->index_to_palette = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)(num_palette * png_sizeof(png_byte)));
+ png_ptr->palette_to_index = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)(num_palette * png_sizeof(png_byte)));
+
+ /* Initialize the sort array */
+ for (i = 0; i < num_palette; i++)
+ {
+ png_ptr->index_to_palette[i] = (png_byte)i;
+ png_ptr->palette_to_index[i] = (png_byte)i;
+ }
+
+ hash = (png_dsortpp)png_calloc(png_ptr, (png_uint_32)(769 *
+ png_sizeof(png_dsortp)));
+
+ num_new_palette = num_palette;
+
+ /* Initial wild guess at how far apart the farthest pixel
+ * pair we will be eliminating will be. Larger
+ * numbers mean more areas will be allocated, Smaller
+ * numbers run the risk of not saving enough data, and
+ * having to do this all over again.
+ *
+ * I have not done extensive checking on this number.
+ */
+ max_d = 96;
+
+ while (num_new_palette > maximum_colors)
+ {
+ for (i = 0; i < num_new_palette - 1; i++)
+ {
+ int j;
+
+ for (j = i + 1; j < num_new_palette; j++)
+ {
+ int d;
+
+ d = PNG_COLOR_DIST(palette[i], palette[j]);
+
+ if (d <= max_d)
+ {
+
+ t = (png_dsortp)png_malloc_warn(png_ptr,
+ (png_uint_32)(png_sizeof(png_dsort)));
+ if (t == NULL)
+ break;
+ t->next = hash[d];
+ t->left = (png_byte)i;
+ t->right = (png_byte)j;
+ hash[d] = t;
+ }
+ }
+ if (t == NULL)
+ break;
+ }
+
+ if (t != NULL)
+ for (i = 0; i <= max_d; i++)
+ {
+ if (hash[i] != NULL)
+ {
+ png_dsortp p;
+
+ for (p = hash[i]; p; p = p->next)
+ {
+ if ((int)png_ptr->index_to_palette[p->left]
+ < num_new_palette &&
+ (int)png_ptr->index_to_palette[p->right]
+ < num_new_palette)
+ {
+ int j, next_j;
+
+ if (num_new_palette & 0x01)
+ {
+ j = p->left;
+ next_j = p->right;
+ }
+ else
+ {
+ j = p->right;
+ next_j = p->left;
+ }
+
+ num_new_palette--;
+ palette[png_ptr->index_to_palette[j]]
+ = palette[num_new_palette];
+ if (!full_dither)
+ {
+ int k;
+
+ for (k = 0; k < num_palette; k++)
+ {
+ if (png_ptr->dither_index[k] ==
+ png_ptr->index_to_palette[j])
+ png_ptr->dither_index[k] =
+ png_ptr->index_to_palette[next_j];
+ if ((int)png_ptr->dither_index[k] ==
+ num_new_palette)
+ png_ptr->dither_index[k] =
+ png_ptr->index_to_palette[j];
+ }
+ }
+
+ png_ptr->index_to_palette[png_ptr->palette_to_index
+ [num_new_palette]] = png_ptr->index_to_palette[j];
+ png_ptr->palette_to_index[png_ptr->index_to_palette[j]]
+ = png_ptr->palette_to_index[num_new_palette];
+
+ png_ptr->index_to_palette[j] =
+ (png_byte)num_new_palette;
+ png_ptr->palette_to_index[num_new_palette] =
+ (png_byte)j;
+ }
+ if (num_new_palette <= maximum_colors)
+ break;
+ }
+ if (num_new_palette <= maximum_colors)
+ break;
+ }
+ }
+
+ for (i = 0; i < 769; i++)
+ {
+ if (hash[i] != NULL)
+ {
+ png_dsortp p = hash[i];
+ while (p)
+ {
+ t = p->next;
+ png_free(png_ptr, p);
+ p = t;
+ }
+ }
+ hash[i] = 0;
+ }
+ max_d += 96;
+ }
+ png_free(png_ptr, hash);
+ png_free(png_ptr, png_ptr->palette_to_index);
+ png_free(png_ptr, png_ptr->index_to_palette);
+ png_ptr->palette_to_index = NULL;
+ png_ptr->index_to_palette = NULL;
+ }
+ num_palette = maximum_colors;
+ }
+ if (png_ptr->palette == NULL)
+ {
+ png_ptr->palette = palette;
+ }
+ png_ptr->num_palette = (png_uint_16)num_palette;
+
+ if (full_dither)
+ {
+ int i;
+ png_bytep distance;
+ int total_bits = PNG_DITHER_RED_BITS + PNG_DITHER_GREEN_BITS +
+ PNG_DITHER_BLUE_BITS;
+ int num_red = (1 << PNG_DITHER_RED_BITS);
+ int num_green = (1 << PNG_DITHER_GREEN_BITS);
+ int num_blue = (1 << PNG_DITHER_BLUE_BITS);
+ png_size_t num_entries = ((png_size_t)1 << total_bits);
+
+ png_ptr->palette_lookup = (png_bytep )png_calloc(png_ptr,
+ (png_uint_32)(num_entries * png_sizeof(png_byte)));
+
+ distance = (png_bytep)png_malloc(png_ptr, (png_uint_32)(num_entries *
+ png_sizeof(png_byte)));
+ png_memset(distance, 0xff, num_entries * png_sizeof(png_byte));
+
+ for (i = 0; i < num_palette; i++)
+ {
+ int ir, ig, ib;
+ int r = (palette[i].red >> (8 - PNG_DITHER_RED_BITS));
+ int g = (palette[i].green >> (8 - PNG_DITHER_GREEN_BITS));
+ int b = (palette[i].blue >> (8 - PNG_DITHER_BLUE_BITS));
+
+ for (ir = 0; ir < num_red; ir++)
+ {
+ /* int dr = abs(ir - r); */
+ int dr = ((ir > r) ? ir - r : r - ir);
+ int index_r = (ir << (PNG_DITHER_BLUE_BITS +
+ PNG_DITHER_GREEN_BITS));
+
+ for (ig = 0; ig < num_green; ig++)
+ {
+ /* int dg = abs(ig - g); */
+ int dg = ((ig > g) ? ig - g : g - ig);
+ int dt = dr + dg;
+ int dm = ((dr > dg) ? dr : dg);
+ int index_g = index_r | (ig << PNG_DITHER_BLUE_BITS);
+
+ for (ib = 0; ib < num_blue; ib++)
+ {
+ int d_index = index_g | ib;
+ /* int db = abs(ib - b); */
+ int db = ((ib > b) ? ib - b : b - ib);
+ int dmax = ((dm > db) ? dm : db);
+ int d = dmax + dt + db;
+
+ if (d < (int)distance[d_index])
+ {
+ distance[d_index] = (png_byte)d;
+ png_ptr->palette_lookup[d_index] = (png_byte)i;
+ }
+ }
+ }
+ }
+ }
+
+ png_free(png_ptr, distance);
+ }
+}
+#endif
+
+#if defined(PNG_READ_GAMMA_SUPPORTED) && defined(PNG_FLOATING_POINT_SUPPORTED)
+/* Transform the image from the file_gamma to the screen_gamma. We
+ * only do transformations on images where the file_gamma and screen_gamma
+ * are not close reciprocals, otherwise it slows things down slightly, and
+ * also needlessly introduces small errors.
+ *
+ * We will turn off gamma transformation later if no semitransparent entries
+ * are present in the tRNS array for palette images. We can't do it here
+ * because we don't necessarily have the tRNS chunk yet.
+ */
+void PNGAPI
+png_set_gamma(png_structp png_ptr, double scrn_gamma, double file_gamma)
+{
+ png_debug(1, "in png_set_gamma");
+
+ if (png_ptr == NULL)
+ return;
+
+ if ((fabs(scrn_gamma * file_gamma - 1.0) > PNG_GAMMA_THRESHOLD) ||
+ (png_ptr->color_type & PNG_COLOR_MASK_ALPHA) ||
+ (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE))
+ png_ptr->transformations |= PNG_GAMMA;
+ png_ptr->gamma = (float)file_gamma;
+ png_ptr->screen_gamma = (float)scrn_gamma;
+}
+#endif
+
+#ifdef PNG_READ_EXPAND_SUPPORTED
+/* Expand paletted images to RGB, expand grayscale images of
+ * less than 8-bit depth to 8-bit depth, and expand tRNS chunks
+ * to alpha channels.
+ */
+void PNGAPI
+png_set_expand(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_expand");
+
+ if (png_ptr == NULL)
+ return;
+
+ png_ptr->transformations |= (PNG_EXPAND | PNG_EXPAND_tRNS);
+ png_ptr->flags &= ~PNG_FLAG_ROW_INIT;
+}
+
+/* GRR 19990627: the following three functions currently are identical
+ * to png_set_expand(). However, it is entirely reasonable that someone
+ * might wish to expand an indexed image to RGB but *not* expand a single,
+ * fully transparent palette entry to a full alpha channel--perhaps instead
+ * convert tRNS to the grayscale/RGB format (16-bit RGB value), or replace
+ * the transparent color with a particular RGB value, or drop tRNS entirely.
+ * IOW, a future version of the library may make the transformations flag
+ * a bit more fine-grained, with separate bits for each of these three
+ * functions.
+ *
+ * More to the point, these functions make it obvious what libpng will be
+ * doing, whereas "expand" can (and does) mean any number of things.
+ *
+ * GRP 20060307: In libpng-1.2.9, png_set_gray_1_2_4_to_8() was modified
+ * to expand only the sample depth but not to expand the tRNS to alpha
+ * and its name was changed to png_set_expand_gray_1_2_4_to_8().
+ */
+
+/* Expand paletted images to RGB. */
+void PNGAPI
+png_set_palette_to_rgb(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_palette_to_rgb");
+
+ if (png_ptr == NULL)
+ return;
+
+ png_ptr->transformations |= (PNG_EXPAND | PNG_EXPAND_tRNS);
+ png_ptr->flags &= ~PNG_FLAG_ROW_INIT;
+}
+
+#ifndef PNG_1_0_X
+/* Expand grayscale images of less than 8-bit depth to 8 bits. */
+void PNGAPI
+png_set_expand_gray_1_2_4_to_8(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_expand_gray_1_2_4_to_8");
+
+ if (png_ptr == NULL)
+ return;
+
+ png_ptr->transformations |= PNG_EXPAND;
+ png_ptr->flags &= ~PNG_FLAG_ROW_INIT;
+}
+#endif
+
+#if defined(PNG_1_0_X) || defined(PNG_1_2_X)
+/* Expand grayscale images of less than 8-bit depth to 8 bits. */
+/* Deprecated as of libpng-1.2.9 */
+void PNGAPI
+png_set_gray_1_2_4_to_8(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_gray_1_2_4_to_8");
+
+ if (png_ptr == NULL)
+ return;
+
+ png_ptr->transformations |= (PNG_EXPAND | PNG_EXPAND_tRNS);
+}
+#endif
+
+
+/* Expand tRNS chunks to alpha channels. */
+void PNGAPI
+png_set_tRNS_to_alpha(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_tRNS_to_alpha");
+
+ png_ptr->transformations |= (PNG_EXPAND | PNG_EXPAND_tRNS);
+ png_ptr->flags &= ~PNG_FLAG_ROW_INIT;
+}
+#endif /* defined(PNG_READ_EXPAND_SUPPORTED) */
+
+#ifdef PNG_READ_GRAY_TO_RGB_SUPPORTED
+void PNGAPI
+png_set_gray_to_rgb(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_gray_to_rgb");
+
+ png_ptr->transformations |= PNG_GRAY_TO_RGB;
+ png_ptr->flags &= ~PNG_FLAG_ROW_INIT;
+}
+#endif
+
+#ifdef PNG_READ_RGB_TO_GRAY_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+/* Convert a RGB image to a grayscale of the same width. This allows us,
+ * for example, to convert a 24 bpp RGB image into an 8 bpp grayscale image.
+ */
+
+void PNGAPI
+png_set_rgb_to_gray(png_structp png_ptr, int error_action, double red,
+ double green)
+{
+ int red_fixed = (int)((float)red*100000.0 + 0.5);
+ int green_fixed = (int)((float)green*100000.0 + 0.5);
+ if (png_ptr == NULL)
+ return;
+ png_set_rgb_to_gray_fixed(png_ptr, error_action, red_fixed, green_fixed);
+}
+#endif
+
+void PNGAPI
+png_set_rgb_to_gray_fixed(png_structp png_ptr, int error_action,
+ png_fixed_point red, png_fixed_point green)
+{
+ png_debug(1, "in png_set_rgb_to_gray");
+
+ if (png_ptr == NULL)
+ return;
+
+ switch(error_action)
+ {
+ case 1: png_ptr->transformations |= PNG_RGB_TO_GRAY;
+ break;
+
+ case 2: png_ptr->transformations |= PNG_RGB_TO_GRAY_WARN;
+ break;
+
+ case 3: png_ptr->transformations |= PNG_RGB_TO_GRAY_ERR;
+ }
+ if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+#ifdef PNG_READ_EXPAND_SUPPORTED
+ png_ptr->transformations |= PNG_EXPAND;
+#else
+ {
+ png_warning(png_ptr,
+ "Cannot do RGB_TO_GRAY without EXPAND_SUPPORTED.");
+ png_ptr->transformations &= ~PNG_RGB_TO_GRAY;
+ }
+#endif
+ {
+ png_uint_16 red_int, green_int;
+ if (red < 0 || green < 0)
+ {
+ red_int = 6968; /* .212671 * 32768 + .5 */
+ green_int = 23434; /* .715160 * 32768 + .5 */
+ }
+ else if (red + green < 100000L)
+ {
+ red_int = (png_uint_16)(((png_uint_32)red*32768L)/100000L);
+ green_int = (png_uint_16)(((png_uint_32)green*32768L)/100000L);
+ }
+ else
+ {
+ png_warning(png_ptr, "ignoring out of range rgb_to_gray coefficients");
+ red_int = 6968;
+ green_int = 23434;
+ }
+ png_ptr->rgb_to_gray_red_coeff = red_int;
+ png_ptr->rgb_to_gray_green_coeff = green_int;
+ png_ptr->rgb_to_gray_blue_coeff =
+ (png_uint_16)(32768 - red_int - green_int);
+ }
+}
+#endif
+
+#if defined(PNG_READ_USER_TRANSFORM_SUPPORTED) || \
+ defined(PNG_LEGACY_SUPPORTED) || \
+ defined(PNG_WRITE_USER_TRANSFORM_SUPPORTED)
+void PNGAPI
+png_set_read_user_transform_fn(png_structp png_ptr, png_user_transform_ptr
+ read_user_transform_fn)
+{
+ png_debug(1, "in png_set_read_user_transform_fn");
+
+ if (png_ptr == NULL)
+ return;
+
+#ifdef PNG_READ_USER_TRANSFORM_SUPPORTED
+ png_ptr->transformations |= PNG_USER_TRANSFORM;
+ png_ptr->read_user_transform_fn = read_user_transform_fn;
+#endif
+#ifdef PNG_LEGACY_SUPPORTED
+ if (read_user_transform_fn)
+ png_warning(png_ptr,
+ "This version of libpng does not support user transforms");
+#endif
+}
+#endif
+
+/* Initialize everything needed for the read. This includes modifying
+ * the palette.
+ */
+void /* PRIVATE */
+png_init_read_transformations(png_structp png_ptr)
+{
+ png_debug(1, "in png_init_read_transformations");
+
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ if (png_ptr != NULL)
+#endif
+ {
+#if defined(PNG_READ_BACKGROUND_SUPPORTED) || \
+ defined(PNG_READ_SHIFT_SUPPORTED) || \
+ defined(PNG_READ_GAMMA_SUPPORTED)
+ int color_type = png_ptr->color_type;
+#endif
+
+#if defined(PNG_READ_EXPAND_SUPPORTED) && defined(PNG_READ_BACKGROUND_SUPPORTED)
+
+#ifdef PNG_READ_GRAY_TO_RGB_SUPPORTED
+ /* Detect gray background and attempt to enable optimization
+ * for gray --> RGB case
+ *
+ * Note: if PNG_BACKGROUND_EXPAND is set and color_type is either RGB or
+ * RGB_ALPHA (in which case need_expand is superfluous anyway), the
+ * background color might actually be gray yet not be flagged as such.
+ * This is not a problem for the current code, which uses
+ * PNG_BACKGROUND_IS_GRAY only to decide when to do the
+ * png_do_gray_to_rgb() transformation.
+ */
+ if ((png_ptr->transformations & PNG_BACKGROUND_EXPAND) &&
+ !(color_type & PNG_COLOR_MASK_COLOR))
+ {
+ png_ptr->mode |= PNG_BACKGROUND_IS_GRAY;
+ } else if ((png_ptr->transformations & PNG_BACKGROUND) &&
+ !(png_ptr->transformations & PNG_BACKGROUND_EXPAND) &&
+ (png_ptr->transformations & PNG_GRAY_TO_RGB) &&
+ png_ptr->background.red == png_ptr->background.green &&
+ png_ptr->background.red == png_ptr->background.blue)
+ {
+ png_ptr->mode |= PNG_BACKGROUND_IS_GRAY;
+ png_ptr->background.gray = png_ptr->background.red;
+ }
+#endif
+
+ if ((png_ptr->transformations & PNG_BACKGROUND_EXPAND) &&
+ (png_ptr->transformations & PNG_EXPAND))
+ {
+ if (!(color_type & PNG_COLOR_MASK_COLOR)) /* i.e., GRAY or GRAY_ALPHA */
+ {
+ /* Expand background and tRNS chunks */
+ switch (png_ptr->bit_depth)
+ {
+ case 1:
+ png_ptr->background.gray *= (png_uint_16)0xff;
+ png_ptr->background.red = png_ptr->background.green
+ = png_ptr->background.blue = png_ptr->background.gray;
+ if (!(png_ptr->transformations & PNG_EXPAND_tRNS))
+ {
+ png_ptr->trans_values.gray *= (png_uint_16)0xff;
+ png_ptr->trans_values.red = png_ptr->trans_values.green
+ = png_ptr->trans_values.blue = png_ptr->trans_values.gray;
+ }
+ break;
+
+ case 2:
+ png_ptr->background.gray *= (png_uint_16)0x55;
+ png_ptr->background.red = png_ptr->background.green
+ = png_ptr->background.blue = png_ptr->background.gray;
+ if (!(png_ptr->transformations & PNG_EXPAND_tRNS))
+ {
+ png_ptr->trans_values.gray *= (png_uint_16)0x55;
+ png_ptr->trans_values.red = png_ptr->trans_values.green
+ = png_ptr->trans_values.blue = png_ptr->trans_values.gray;
+ }
+ break;
+
+ case 4:
+ png_ptr->background.gray *= (png_uint_16)0x11;
+ png_ptr->background.red = png_ptr->background.green
+ = png_ptr->background.blue = png_ptr->background.gray;
+ if (!(png_ptr->transformations & PNG_EXPAND_tRNS))
+ {
+ png_ptr->trans_values.gray *= (png_uint_16)0x11;
+ png_ptr->trans_values.red = png_ptr->trans_values.green
+ = png_ptr->trans_values.blue = png_ptr->trans_values.gray;
+ }
+ break;
+
+ case 8:
+
+ case 16:
+ png_ptr->background.red = png_ptr->background.green
+ = png_ptr->background.blue = png_ptr->background.gray;
+ break;
+ }
+ }
+ else if (color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ png_ptr->background.red =
+ png_ptr->palette[png_ptr->background.index].red;
+ png_ptr->background.green =
+ png_ptr->palette[png_ptr->background.index].green;
+ png_ptr->background.blue =
+ png_ptr->palette[png_ptr->background.index].blue;
+
+#ifdef PNG_READ_INVERT_ALPHA_SUPPORTED
+ if (png_ptr->transformations & PNG_INVERT_ALPHA)
+ {
+#ifdef PNG_READ_EXPAND_SUPPORTED
+ if (!(png_ptr->transformations & PNG_EXPAND_tRNS))
+#endif
+ {
+ /* Invert the alpha channel (in tRNS) unless the pixels are
+ * going to be expanded, in which case leave it for later
+ */
+ int i, istop;
+ istop=(int)png_ptr->num_trans;
+ for (i=0; i<istop; i++)
+ png_ptr->trans[i] = (png_byte)(255 - png_ptr->trans[i]);
+ }
+ }
+#endif
+
+ }
+ }
+#endif
+
+#if defined(PNG_READ_BACKGROUND_SUPPORTED) && defined(PNG_READ_GAMMA_SUPPORTED)
+ png_ptr->background_1 = png_ptr->background;
+#endif
+#if defined(PNG_READ_GAMMA_SUPPORTED) && defined(PNG_FLOATING_POINT_SUPPORTED)
+
+ if ((color_type == PNG_COLOR_TYPE_PALETTE && png_ptr->num_trans != 0)
+ && (fabs(png_ptr->screen_gamma * png_ptr->gamma - 1.0)
+ < PNG_GAMMA_THRESHOLD))
+ {
+ int i, k;
+ k=0;
+ for (i=0; i<png_ptr->num_trans; i++)
+ {
+ if (png_ptr->trans[i] != 0 && png_ptr->trans[i] != 0xff)
+ k=1; /* Partial transparency is present */
+ }
+ if (k == 0)
+ png_ptr->transformations &= ~PNG_GAMMA;
+ }
+
+ if ((png_ptr->transformations & (PNG_GAMMA | PNG_RGB_TO_GRAY)) &&
+ png_ptr->gamma != 0.0)
+ {
+ png_build_gamma_table(png_ptr);
+
+#ifdef PNG_READ_BACKGROUND_SUPPORTED
+ if (png_ptr->transformations & PNG_BACKGROUND)
+ {
+ if (color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ /* Could skip if no transparency */
+ png_color back, back_1;
+ png_colorp palette = png_ptr->palette;
+ int num_palette = png_ptr->num_palette;
+ int i;
+ if (png_ptr->background_gamma_type == PNG_BACKGROUND_GAMMA_FILE)
+ {
+ back.red = png_ptr->gamma_table[png_ptr->background.red];
+ back.green = png_ptr->gamma_table[png_ptr->background.green];
+ back.blue = png_ptr->gamma_table[png_ptr->background.blue];
+
+ back_1.red = png_ptr->gamma_to_1[png_ptr->background.red];
+ back_1.green = png_ptr->gamma_to_1[png_ptr->background.green];
+ back_1.blue = png_ptr->gamma_to_1[png_ptr->background.blue];
+ }
+ else
+ {
+ double g, gs;
+
+ switch (png_ptr->background_gamma_type)
+ {
+ case PNG_BACKGROUND_GAMMA_SCREEN:
+ g = (png_ptr->screen_gamma);
+ gs = 1.0;
+ break;
+
+ case PNG_BACKGROUND_GAMMA_FILE:
+ g = 1.0 / (png_ptr->gamma);
+ gs = 1.0 / (png_ptr->gamma * png_ptr->screen_gamma);
+ break;
+
+ case PNG_BACKGROUND_GAMMA_UNIQUE:
+ g = 1.0 / (png_ptr->background_gamma);
+ gs = 1.0 / (png_ptr->background_gamma *
+ png_ptr->screen_gamma);
+ break;
+ default:
+ g = 1.0; /* back_1 */
+ gs = 1.0; /* back */
+ }
+
+ if ( fabs(gs - 1.0) < PNG_GAMMA_THRESHOLD)
+ {
+ back.red = (png_byte)png_ptr->background.red;
+ back.green = (png_byte)png_ptr->background.green;
+ back.blue = (png_byte)png_ptr->background.blue;
+ }
+ else
+ {
+ back.red = (png_byte)(pow(
+ (double)png_ptr->background.red/255, gs) * 255.0 + .5);
+ back.green = (png_byte)(pow(
+ (double)png_ptr->background.green/255, gs) * 255.0
+ + .5);
+ back.blue = (png_byte)(pow(
+ (double)png_ptr->background.blue/255, gs) * 255.0 + .5);
+ }
+
+ back_1.red = (png_byte)(pow(
+ (double)png_ptr->background.red/255, g) * 255.0 + .5);
+ back_1.green = (png_byte)(pow(
+ (double)png_ptr->background.green/255, g) * 255.0 + .5);
+ back_1.blue = (png_byte)(pow(
+ (double)png_ptr->background.blue/255, g) * 255.0 + .5);
+ }
+ for (i = 0; i < num_palette; i++)
+ {
+ if (i < (int)png_ptr->num_trans && png_ptr->trans[i] != 0xff)
+ {
+ if (png_ptr->trans[i] == 0)
+ {
+ palette[i] = back;
+ }
+ else /* if (png_ptr->trans[i] != 0xff) */
+ {
+ png_byte v, w;
+
+ v = png_ptr->gamma_to_1[palette[i].red];
+ png_composite(w, v, png_ptr->trans[i], back_1.red);
+ palette[i].red = png_ptr->gamma_from_1[w];
+
+ v = png_ptr->gamma_to_1[palette[i].green];
+ png_composite(w, v, png_ptr->trans[i], back_1.green);
+ palette[i].green = png_ptr->gamma_from_1[w];
+
+ v = png_ptr->gamma_to_1[palette[i].blue];
+ png_composite(w, v, png_ptr->trans[i], back_1.blue);
+ palette[i].blue = png_ptr->gamma_from_1[w];
+ }
+ }
+ else
+ {
+ palette[i].red = png_ptr->gamma_table[palette[i].red];
+ palette[i].green = png_ptr->gamma_table[palette[i].green];
+ palette[i].blue = png_ptr->gamma_table[palette[i].blue];
+ }
+ }
+ /* Prevent the transformations being done again, and make sure
+ * that the now spurious alpha channel is stripped - the code
+ * has just reduced background composition and gamma correction
+ * to a simple alpha channel strip.
+ */
+ png_ptr->transformations &= ~PNG_BACKGROUND;
+ png_ptr->transformations &= ~PNG_GAMMA;
+ png_ptr->transformations |= PNG_STRIP_ALPHA;
+ }
+ /* if (png_ptr->background_gamma_type!=PNG_BACKGROUND_GAMMA_UNKNOWN) */
+ else
+ /* color_type != PNG_COLOR_TYPE_PALETTE */
+ {
+ double m = (double)(((png_uint_32)1 << png_ptr->bit_depth) - 1);
+ double g = 1.0;
+ double gs = 1.0;
+
+ switch (png_ptr->background_gamma_type)
+ {
+ case PNG_BACKGROUND_GAMMA_SCREEN:
+ g = (png_ptr->screen_gamma);
+ gs = 1.0;
+ break;
+
+ case PNG_BACKGROUND_GAMMA_FILE:
+ g = 1.0 / (png_ptr->gamma);
+ gs = 1.0 / (png_ptr->gamma * png_ptr->screen_gamma);
+ break;
+
+ case PNG_BACKGROUND_GAMMA_UNIQUE:
+ g = 1.0 / (png_ptr->background_gamma);
+ gs = 1.0 / (png_ptr->background_gamma *
+ png_ptr->screen_gamma);
+ break;
+ }
+
+ png_ptr->background_1.gray = (png_uint_16)(pow(
+ (double)png_ptr->background.gray / m, g) * m + .5);
+ png_ptr->background.gray = (png_uint_16)(pow(
+ (double)png_ptr->background.gray / m, gs) * m + .5);
+
+ if ((png_ptr->background.red != png_ptr->background.green) ||
+ (png_ptr->background.red != png_ptr->background.blue) ||
+ (png_ptr->background.red != png_ptr->background.gray))
+ {
+ /* RGB or RGBA with color background */
+ png_ptr->background_1.red = (png_uint_16)(pow(
+ (double)png_ptr->background.red / m, g) * m + .5);
+ png_ptr->background_1.green = (png_uint_16)(pow(
+ (double)png_ptr->background.green / m, g) * m + .5);
+ png_ptr->background_1.blue = (png_uint_16)(pow(
+ (double)png_ptr->background.blue / m, g) * m + .5);
+ png_ptr->background.red = (png_uint_16)(pow(
+ (double)png_ptr->background.red / m, gs) * m + .5);
+ png_ptr->background.green = (png_uint_16)(pow(
+ (double)png_ptr->background.green / m, gs) * m + .5);
+ png_ptr->background.blue = (png_uint_16)(pow(
+ (double)png_ptr->background.blue / m, gs) * m + .5);
+ }
+ else
+ {
+ /* GRAY, GRAY ALPHA, RGB, or RGBA with gray background */
+ png_ptr->background_1.red = png_ptr->background_1.green
+ = png_ptr->background_1.blue = png_ptr->background_1.gray;
+ png_ptr->background.red = png_ptr->background.green
+ = png_ptr->background.blue = png_ptr->background.gray;
+ }
+ }
+ }
+ else
+ /* Transformation does not include PNG_BACKGROUND */
+#endif /* PNG_READ_BACKGROUND_SUPPORTED */
+ if (color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ png_colorp palette = png_ptr->palette;
+ int num_palette = png_ptr->num_palette;
+ int i;
+
+ for (i = 0; i < num_palette; i++)
+ {
+ palette[i].red = png_ptr->gamma_table[palette[i].red];
+ palette[i].green = png_ptr->gamma_table[palette[i].green];
+ palette[i].blue = png_ptr->gamma_table[palette[i].blue];
+ }
+
+ /* Done the gamma correction. */
+ png_ptr->transformations &= ~PNG_GAMMA;
+ }
+ }
+#ifdef PNG_READ_BACKGROUND_SUPPORTED
+ else
+#endif
+#endif /* PNG_READ_GAMMA_SUPPORTED && PNG_FLOATING_POINT_SUPPORTED */
+#ifdef PNG_READ_BACKGROUND_SUPPORTED
+ /* No GAMMA transformation */
+ if ((png_ptr->transformations & PNG_BACKGROUND) &&
+ (color_type == PNG_COLOR_TYPE_PALETTE))
+ {
+ int i;
+ int istop = (int)png_ptr->num_trans;
+ png_color back;
+ png_colorp palette = png_ptr->palette;
+
+ back.red = (png_byte)png_ptr->background.red;
+ back.green = (png_byte)png_ptr->background.green;
+ back.blue = (png_byte)png_ptr->background.blue;
+
+ for (i = 0; i < istop; i++)
+ {
+ if (png_ptr->trans[i] == 0)
+ {
+ palette[i] = back;
+ }
+ else if (png_ptr->trans[i] != 0xff)
+ {
+ /* The png_composite() macro is defined in png.h */
+ png_composite(palette[i].red, palette[i].red,
+ png_ptr->trans[i], back.red);
+ png_composite(palette[i].green, palette[i].green,
+ png_ptr->trans[i], back.green);
+ png_composite(palette[i].blue, palette[i].blue,
+ png_ptr->trans[i], back.blue);
+ }
+ }
+
+ /* Handled alpha, still need to strip the channel. */
+ png_ptr->transformations &= ~PNG_BACKGROUND;
+ png_ptr->transformations |= PNG_STRIP_ALPHA;
+ }
+#endif /* PNG_READ_BACKGROUND_SUPPORTED */
+
+#ifdef PNG_READ_SHIFT_SUPPORTED
+ if ((png_ptr->transformations & PNG_SHIFT) &&
+ (color_type == PNG_COLOR_TYPE_PALETTE))
+ {
+ png_uint_16 i;
+ png_uint_16 istop = png_ptr->num_palette;
+ int sr = 8 - png_ptr->sig_bit.red;
+ int sg = 8 - png_ptr->sig_bit.green;
+ int sb = 8 - png_ptr->sig_bit.blue;
+
+ if (sr < 0 || sr > 8)
+ sr = 0;
+ if (sg < 0 || sg > 8)
+ sg = 0;
+ if (sb < 0 || sb > 8)
+ sb = 0;
+ for (i = 0; i < istop; i++)
+ {
+ png_ptr->palette[i].red >>= sr;
+ png_ptr->palette[i].green >>= sg;
+ png_ptr->palette[i].blue >>= sb;
+ }
+ }
+#endif /* PNG_READ_SHIFT_SUPPORTED */
+ }
+#if !defined(PNG_READ_GAMMA_SUPPORTED) && !defined(PNG_READ_SHIFT_SUPPORTED) \
+ && !defined(PNG_READ_BACKGROUND_SUPPORTED)
+ if (png_ptr)
+ return;
+#endif
+}
+
+/* Modify the info structure to reflect the transformations. The
+ * info should be updated so a PNG file could be written with it,
+ * assuming the transformations result in valid PNG data.
+ */
+void /* PRIVATE */
+png_read_transform_info(png_structp png_ptr, png_infop info_ptr)
+{
+ png_debug(1, "in png_read_transform_info");
+
+#ifdef PNG_READ_EXPAND_SUPPORTED
+ if (png_ptr->transformations & PNG_EXPAND)
+ {
+ if (info_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ if (png_ptr->num_trans &&
+ (png_ptr->transformations & PNG_EXPAND_tRNS))
+ info_ptr->color_type = PNG_COLOR_TYPE_RGB_ALPHA;
+ else
+ info_ptr->color_type = PNG_COLOR_TYPE_RGB;
+ info_ptr->bit_depth = 8;
+ info_ptr->num_trans = 0;
+ }
+ else
+ {
+ if (png_ptr->num_trans)
+ {
+ if (png_ptr->transformations & PNG_EXPAND_tRNS)
+ info_ptr->color_type |= PNG_COLOR_MASK_ALPHA;
+ }
+ if (info_ptr->bit_depth < 8)
+ info_ptr->bit_depth = 8;
+ info_ptr->num_trans = 0;
+ }
+ }
+#endif
+
+#ifdef PNG_READ_BACKGROUND_SUPPORTED
+ if (png_ptr->transformations & PNG_BACKGROUND)
+ {
+ info_ptr->color_type &= ~PNG_COLOR_MASK_ALPHA;
+ info_ptr->num_trans = 0;
+ info_ptr->background = png_ptr->background;
+ }
+#endif
+
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ if (png_ptr->transformations & PNG_GAMMA)
+ {
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ info_ptr->gamma = png_ptr->gamma;
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ info_ptr->int_gamma = png_ptr->int_gamma;
+#endif
+ }
+#endif
+
+#ifdef PNG_READ_16_TO_8_SUPPORTED
+ if ((png_ptr->transformations & PNG_16_TO_8) && (info_ptr->bit_depth == 16))
+ info_ptr->bit_depth = 8;
+#endif
+
+#ifdef PNG_READ_GRAY_TO_RGB_SUPPORTED
+ if (png_ptr->transformations & PNG_GRAY_TO_RGB)
+ info_ptr->color_type |= PNG_COLOR_MASK_COLOR;
+#endif
+
+#ifdef PNG_READ_RGB_TO_GRAY_SUPPORTED
+ if (png_ptr->transformations & PNG_RGB_TO_GRAY)
+ info_ptr->color_type &= ~PNG_COLOR_MASK_COLOR;
+#endif
+
+#ifdef PNG_READ_DITHER_SUPPORTED
+ if (png_ptr->transformations & PNG_DITHER)
+ {
+ if (((info_ptr->color_type == PNG_COLOR_TYPE_RGB) ||
+ (info_ptr->color_type == PNG_COLOR_TYPE_RGB_ALPHA)) &&
+ png_ptr->palette_lookup && info_ptr->bit_depth == 8)
+ {
+ info_ptr->color_type = PNG_COLOR_TYPE_PALETTE;
+ }
+ }
+#endif
+
+#ifdef PNG_READ_PACK_SUPPORTED
+ if ((png_ptr->transformations & PNG_PACK) && (info_ptr->bit_depth < 8))
+ info_ptr->bit_depth = 8;
+#endif
+
+ if (info_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ info_ptr->channels = 1;
+ else if (info_ptr->color_type & PNG_COLOR_MASK_COLOR)
+ info_ptr->channels = 3;
+ else
+ info_ptr->channels = 1;
+
+#ifdef PNG_READ_STRIP_ALPHA_SUPPORTED
+ if (png_ptr->flags & PNG_FLAG_STRIP_ALPHA)
+ info_ptr->color_type &= ~PNG_COLOR_MASK_ALPHA;
+#endif
+
+ if (info_ptr->color_type & PNG_COLOR_MASK_ALPHA)
+ info_ptr->channels++;
+
+#ifdef PNG_READ_FILLER_SUPPORTED
+ /* STRIP_ALPHA and FILLER allowed: MASK_ALPHA bit stripped above */
+ if ((png_ptr->transformations & PNG_FILLER) &&
+ ((info_ptr->color_type == PNG_COLOR_TYPE_RGB) ||
+ (info_ptr->color_type == PNG_COLOR_TYPE_GRAY)))
+ {
+ info_ptr->channels++;
+ /* If adding a true alpha channel not just filler */
+#ifndef PNG_1_0_X
+ if (png_ptr->transformations & PNG_ADD_ALPHA)
+ info_ptr->color_type |= PNG_COLOR_MASK_ALPHA;
+#endif
+ }
+#endif
+
+#if defined(PNG_USER_TRANSFORM_PTR_SUPPORTED) && \
+defined(PNG_READ_USER_TRANSFORM_SUPPORTED)
+ if (png_ptr->transformations & PNG_USER_TRANSFORM)
+ {
+ if (info_ptr->bit_depth < png_ptr->user_transform_depth)
+ info_ptr->bit_depth = png_ptr->user_transform_depth;
+ if (info_ptr->channels < png_ptr->user_transform_channels)
+ info_ptr->channels = png_ptr->user_transform_channels;
+ }
+#endif
+
+ info_ptr->pixel_depth = (png_byte)(info_ptr->channels *
+ info_ptr->bit_depth);
+
+ info_ptr->rowbytes = PNG_ROWBYTES(info_ptr->pixel_depth, info_ptr->width);
+
+#ifndef PNG_READ_EXPAND_SUPPORTED
+ if (png_ptr)
+ return;
+#endif
+}
+
+/* Transform the row. The order of transformations is significant,
+ * and is very touchy. If you add a transformation, take care to
+ * decide how it fits in with the other transformations here.
+ */
+void /* PRIVATE */
+png_do_read_transformations(png_structp png_ptr)
+{
+ png_debug(1, "in png_do_read_transformations");
+
+ if (png_ptr->row_buf == NULL)
+ {
+#if defined(PNG_STDIO_SUPPORTED) && !defined(_WIN32_WCE)
+ char msg[50];
+
+ png_snprintf2(msg, 50,
+ "NULL row buffer for row %ld, pass %d", (long)png_ptr->row_number,
+ png_ptr->pass);
+ png_error(png_ptr, msg);
+#else
+ png_error(png_ptr, "NULL row buffer");
+#endif
+ }
+#ifdef PNG_WARN_UNINITIALIZED_ROW
+ if (!(png_ptr->flags & PNG_FLAG_ROW_INIT))
+ /* Application has failed to call either png_read_start_image()
+ * or png_read_update_info() after setting transforms that expand
+ * pixels. This check added to libpng-1.2.19
+ */
+#if (PNG_WARN_UNINITIALIZED_ROW==1)
+ png_error(png_ptr, "Uninitialized row");
+#else
+ png_warning(png_ptr, "Uninitialized row");
+#endif
+#endif
+
+#ifdef PNG_READ_EXPAND_SUPPORTED
+ if (png_ptr->transformations & PNG_EXPAND)
+ {
+ if (png_ptr->row_info.color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ png_do_expand_palette(&(png_ptr->row_info), png_ptr->row_buf + 1,
+ png_ptr->palette, png_ptr->trans, png_ptr->num_trans);
+ }
+ else
+ {
+ if (png_ptr->num_trans &&
+ (png_ptr->transformations & PNG_EXPAND_tRNS))
+ png_do_expand(&(png_ptr->row_info), png_ptr->row_buf + 1,
+ &(png_ptr->trans_values));
+ else
+ png_do_expand(&(png_ptr->row_info), png_ptr->row_buf + 1,
+ NULL);
+ }
+ }
+#endif
+
+#ifdef PNG_READ_STRIP_ALPHA_SUPPORTED
+ if (png_ptr->flags & PNG_FLAG_STRIP_ALPHA)
+ png_do_strip_filler(&(png_ptr->row_info), png_ptr->row_buf + 1,
+ PNG_FLAG_FILLER_AFTER | (png_ptr->flags & PNG_FLAG_STRIP_ALPHA));
+#endif
+
+#ifdef PNG_READ_RGB_TO_GRAY_SUPPORTED
+ if (png_ptr->transformations & PNG_RGB_TO_GRAY)
+ {
+ int rgb_error =
+ png_do_rgb_to_gray(png_ptr, &(png_ptr->row_info),
+ png_ptr->row_buf + 1);
+ if (rgb_error)
+ {
+ png_ptr->rgb_to_gray_status=1;
+ if ((png_ptr->transformations & PNG_RGB_TO_GRAY) ==
+ PNG_RGB_TO_GRAY_WARN)
+ png_warning(png_ptr, "png_do_rgb_to_gray found nongray pixel");
+ if ((png_ptr->transformations & PNG_RGB_TO_GRAY) ==
+ PNG_RGB_TO_GRAY_ERR)
+ png_error(png_ptr, "png_do_rgb_to_gray found nongray pixel");
+ }
+ }
+#endif
+
+/* From Andreas Dilger e-mail to png-implement, 26 March 1998:
+ *
+ * In most cases, the "simple transparency" should be done prior to doing
+ * gray-to-RGB, or you will have to test 3x as many bytes to check if a
+ * pixel is transparent. You would also need to make sure that the
+ * transparency information is upgraded to RGB.
+ *
+ * To summarize, the current flow is:
+ * - Gray + simple transparency -> compare 1 or 2 gray bytes and composite
+ * with background "in place" if transparent,
+ * convert to RGB if necessary
+ * - Gray + alpha -> composite with gray background and remove alpha bytes,
+ * convert to RGB if necessary
+ *
+ * To support RGB backgrounds for gray images we need:
+ * - Gray + simple transparency -> convert to RGB + simple transparency,
+ * compare 3 or 6 bytes and composite with
+ * background "in place" if transparent
+ * (3x compare/pixel compared to doing
+ * composite with gray bkgrnd)
+ * - Gray + alpha -> convert to RGB + alpha, composite with background and
+ * remove alpha bytes (3x float
+ * operations/pixel compared with composite
+ * on gray background)
+ *
+ * Greg's change will do this. The reason it wasn't done before is for
+ * performance, as this increases the per-pixel operations. If we would check
+ * in advance if the background was gray or RGB, and position the gray-to-RGB
+ * transform appropriately, then it would save a lot of work/time.
+ */
+
+#ifdef PNG_READ_GRAY_TO_RGB_SUPPORTED
+ /* If gray -> RGB, do so now only if background is non-gray; else do later
+ * for performance reasons
+ */
+ if ((png_ptr->transformations & PNG_GRAY_TO_RGB) &&
+ !(png_ptr->mode & PNG_BACKGROUND_IS_GRAY))
+ png_do_gray_to_rgb(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+
+#ifdef PNG_READ_BACKGROUND_SUPPORTED
+ if ((png_ptr->transformations & PNG_BACKGROUND) &&
+ ((png_ptr->num_trans != 0 ) ||
+ (png_ptr->color_type & PNG_COLOR_MASK_ALPHA)))
+ png_do_background(&(png_ptr->row_info), png_ptr->row_buf + 1,
+ &(png_ptr->trans_values), &(png_ptr->background)
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ , &(png_ptr->background_1),
+ png_ptr->gamma_table, png_ptr->gamma_from_1,
+ png_ptr->gamma_to_1, png_ptr->gamma_16_table,
+ png_ptr->gamma_16_from_1, png_ptr->gamma_16_to_1,
+ png_ptr->gamma_shift
+#endif
+);
+#endif
+
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ if ((png_ptr->transformations & PNG_GAMMA) &&
+#ifdef PNG_READ_BACKGROUND_SUPPORTED
+ !((png_ptr->transformations & PNG_BACKGROUND) &&
+ ((png_ptr->num_trans != 0) ||
+ (png_ptr->color_type & PNG_COLOR_MASK_ALPHA))) &&
+#endif
+ (png_ptr->color_type != PNG_COLOR_TYPE_PALETTE))
+ png_do_gamma(&(png_ptr->row_info), png_ptr->row_buf + 1,
+ png_ptr->gamma_table, png_ptr->gamma_16_table,
+ png_ptr->gamma_shift);
+#endif
+
+#ifdef PNG_READ_16_TO_8_SUPPORTED
+ if (png_ptr->transformations & PNG_16_TO_8)
+ png_do_chop(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+
+#ifdef PNG_READ_DITHER_SUPPORTED
+ if (png_ptr->transformations & PNG_DITHER)
+ {
+ png_do_dither((png_row_infop)&(png_ptr->row_info), png_ptr->row_buf + 1,
+ png_ptr->palette_lookup, png_ptr->dither_index);
+ if (png_ptr->row_info.rowbytes == (png_uint_32)0)
+ png_error(png_ptr, "png_do_dither returned rowbytes=0");
+ }
+#endif
+
+#ifdef PNG_READ_INVERT_SUPPORTED
+ if (png_ptr->transformations & PNG_INVERT_MONO)
+ png_do_invert(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+
+#ifdef PNG_READ_SHIFT_SUPPORTED
+ if (png_ptr->transformations & PNG_SHIFT)
+ png_do_unshift(&(png_ptr->row_info), png_ptr->row_buf + 1,
+ &(png_ptr->shift));
+#endif
+
+#ifdef PNG_READ_PACK_SUPPORTED
+ if (png_ptr->transformations & PNG_PACK)
+ png_do_unpack(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+
+#ifdef PNG_READ_BGR_SUPPORTED
+ if (png_ptr->transformations & PNG_BGR)
+ png_do_bgr(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+
+#ifdef PNG_READ_PACKSWAP_SUPPORTED
+ if (png_ptr->transformations & PNG_PACKSWAP)
+ png_do_packswap(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+
+#ifdef PNG_READ_GRAY_TO_RGB_SUPPORTED
+ /* If gray -> RGB, do so now only if we did not do so above */
+ if ((png_ptr->transformations & PNG_GRAY_TO_RGB) &&
+ (png_ptr->mode & PNG_BACKGROUND_IS_GRAY))
+ png_do_gray_to_rgb(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+
+#ifdef PNG_READ_FILLER_SUPPORTED
+ if (png_ptr->transformations & PNG_FILLER)
+ png_do_read_filler(&(png_ptr->row_info), png_ptr->row_buf + 1,
+ (png_uint_32)png_ptr->filler, png_ptr->flags);
+#endif
+
+#ifdef PNG_READ_INVERT_ALPHA_SUPPORTED
+ if (png_ptr->transformations & PNG_INVERT_ALPHA)
+ png_do_read_invert_alpha(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+
+#ifdef PNG_READ_SWAP_ALPHA_SUPPORTED
+ if (png_ptr->transformations & PNG_SWAP_ALPHA)
+ png_do_read_swap_alpha(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+
+#ifdef PNG_READ_SWAP_SUPPORTED
+ if (png_ptr->transformations & PNG_SWAP_BYTES)
+ png_do_swap(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+
+#ifdef PNG_READ_USER_TRANSFORM_SUPPORTED
+ if (png_ptr->transformations & PNG_USER_TRANSFORM)
+ {
+ if (png_ptr->read_user_transform_fn != NULL)
+ (*(png_ptr->read_user_transform_fn)) /* User read transform function */
+ (png_ptr, /* png_ptr */
+ &(png_ptr->row_info), /* row_info: */
+ /* png_uint_32 width; width of row */
+ /* png_uint_32 rowbytes; number of bytes in row */
+ /* png_byte color_type; color type of pixels */
+ /* png_byte bit_depth; bit depth of samples */
+ /* png_byte channels; number of channels (1-4) */
+ /* png_byte pixel_depth; bits per pixel (depth*channels) */
+ png_ptr->row_buf + 1); /* start of pixel data for row */
+#ifdef PNG_USER_TRANSFORM_PTR_SUPPORTED
+ if (png_ptr->user_transform_depth)
+ png_ptr->row_info.bit_depth = png_ptr->user_transform_depth;
+ if (png_ptr->user_transform_channels)
+ png_ptr->row_info.channels = png_ptr->user_transform_channels;
+#endif
+ png_ptr->row_info.pixel_depth = (png_byte)(png_ptr->row_info.bit_depth *
+ png_ptr->row_info.channels);
+ png_ptr->row_info.rowbytes = PNG_ROWBYTES(png_ptr->row_info.pixel_depth,
+ png_ptr->row_info.width);
+ }
+#endif
+
+}
+
+#ifdef PNG_READ_PACK_SUPPORTED
+/* Unpack pixels of 1, 2, or 4 bits per pixel into 1 byte per pixel,
+ * without changing the actual values. Thus, if you had a row with
+ * a bit depth of 1, you would end up with bytes that only contained
+ * the numbers 0 or 1. If you would rather they contain 0 and 255, use
+ * png_do_shift() after this.
+ */
+void /* PRIVATE */
+png_do_unpack(png_row_infop row_info, png_bytep row)
+{
+ png_debug(1, "in png_do_unpack");
+
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ if (row != NULL && row_info != NULL && row_info->bit_depth < 8)
+#else
+ if (row_info->bit_depth < 8)
+#endif
+ {
+ png_uint_32 i;
+ png_uint_32 row_width=row_info->width;
+
+ switch (row_info->bit_depth)
+ {
+ case 1:
+ {
+ png_bytep sp = row + (png_size_t)((row_width - 1) >> 3);
+ png_bytep dp = row + (png_size_t)row_width - 1;
+ png_uint_32 shift = 7 - (int)((row_width + 7) & 0x07);
+ for (i = 0; i < row_width; i++)
+ {
+ *dp = (png_byte)((*sp >> shift) & 0x01);
+ if (shift == 7)
+ {
+ shift = 0;
+ sp--;
+ }
+ else
+ shift++;
+
+ dp--;
+ }
+ break;
+ }
+
+ case 2:
+ {
+
+ png_bytep sp = row + (png_size_t)((row_width - 1) >> 2);
+ png_bytep dp = row + (png_size_t)row_width - 1;
+ png_uint_32 shift = (int)((3 - ((row_width + 3) & 0x03)) << 1);
+ for (i = 0; i < row_width; i++)
+ {
+ *dp = (png_byte)((*sp >> shift) & 0x03);
+ if (shift == 6)
+ {
+ shift = 0;
+ sp--;
+ }
+ else
+ shift += 2;
+
+ dp--;
+ }
+ break;
+ }
+
+ case 4:
+ {
+ png_bytep sp = row + (png_size_t)((row_width - 1) >> 1);
+ png_bytep dp = row + (png_size_t)row_width - 1;
+ png_uint_32 shift = (int)((1 - ((row_width + 1) & 0x01)) << 2);
+ for (i = 0; i < row_width; i++)
+ {
+ *dp = (png_byte)((*sp >> shift) & 0x0f);
+ if (shift == 4)
+ {
+ shift = 0;
+ sp--;
+ }
+ else
+ shift = 4;
+
+ dp--;
+ }
+ break;
+ }
+ }
+ row_info->bit_depth = 8;
+ row_info->pixel_depth = (png_byte)(8 * row_info->channels);
+ row_info->rowbytes = row_width * row_info->channels;
+ }
+}
+#endif
+
+#ifdef PNG_READ_SHIFT_SUPPORTED
+/* Reverse the effects of png_do_shift. This routine merely shifts the
+ * pixels back to their significant bits values. Thus, if you have
+ * a row of bit depth 8, but only 5 are significant, this will shift
+ * the values back to 0 through 31.
+ */
+void /* PRIVATE */
+png_do_unshift(png_row_infop row_info, png_bytep row, png_color_8p sig_bits)
+{
+ png_debug(1, "in png_do_unshift");
+
+ if (
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ row != NULL && row_info != NULL && sig_bits != NULL &&
+#endif
+ row_info->color_type != PNG_COLOR_TYPE_PALETTE)
+ {
+ int shift[4];
+ int channels = 0;
+ int c;
+ png_uint_16 value = 0;
+ png_uint_32 row_width = row_info->width;
+
+ if (row_info->color_type & PNG_COLOR_MASK_COLOR)
+ {
+ shift[channels++] = row_info->bit_depth - sig_bits->red;
+ shift[channels++] = row_info->bit_depth - sig_bits->green;
+ shift[channels++] = row_info->bit_depth - sig_bits->blue;
+ }
+ else
+ {
+ shift[channels++] = row_info->bit_depth - sig_bits->gray;
+ }
+ if (row_info->color_type & PNG_COLOR_MASK_ALPHA)
+ {
+ shift[channels++] = row_info->bit_depth - sig_bits->alpha;
+ }
+
+ for (c = 0; c < channels; c++)
+ {
+ if (shift[c] <= 0)
+ shift[c] = 0;
+ else
+ value = 1;
+ }
+
+ if (!value)
+ return;
+
+ switch (row_info->bit_depth)
+ {
+ case 2:
+ {
+ png_bytep bp;
+ png_uint_32 i;
+ png_uint_32 istop = row_info->rowbytes;
+
+ for (bp = row, i = 0; i < istop; i++)
+ {
+ *bp >>= 1;
+ *bp++ &= 0x55;
+ }
+ break;
+ }
+
+ case 4:
+ {
+ png_bytep bp = row;
+ png_uint_32 i;
+ png_uint_32 istop = row_info->rowbytes;
+ png_byte mask = (png_byte)((((int)0xf0 >> shift[0]) & (int)0xf0) |
+ (png_byte)((int)0xf >> shift[0]));
+
+ for (i = 0; i < istop; i++)
+ {
+ *bp >>= shift[0];
+ *bp++ &= mask;
+ }
+ break;
+ }
+
+ case 8:
+ {
+ png_bytep bp = row;
+ png_uint_32 i;
+ png_uint_32 istop = row_width * channels;
+
+ for (i = 0; i < istop; i++)
+ {
+ *bp++ >>= shift[i%channels];
+ }
+ break;
+ }
+
+ case 16:
+ {
+ png_bytep bp = row;
+ png_uint_32 i;
+ png_uint_32 istop = channels * row_width;
+
+ for (i = 0; i < istop; i++)
+ {
+ value = (png_uint_16)((*bp << 8) + *(bp + 1));
+ value >>= shift[i%channels];
+ *bp++ = (png_byte)(value >> 8);
+ *bp++ = (png_byte)(value & 0xff);
+ }
+ break;
+ }
+ }
+ }
+}
+#endif
+
+#ifdef PNG_READ_16_TO_8_SUPPORTED
+/* Chop rows of bit depth 16 down to 8 */
+void /* PRIVATE */
+png_do_chop(png_row_infop row_info, png_bytep row)
+{
+ png_debug(1, "in png_do_chop");
+
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ if (row != NULL && row_info != NULL && row_info->bit_depth == 16)
+#else
+ if (row_info->bit_depth == 16)
+#endif
+ {
+ png_bytep sp = row;
+ png_bytep dp = row;
+ png_uint_32 i;
+ png_uint_32 istop = row_info->width * row_info->channels;
+
+ for (i = 0; i<istop; i++, sp += 2, dp++)
+ {
+#ifdef PNG_READ_16_TO_8_ACCURATE_SCALE_SUPPORTED
+ /* This does a more accurate scaling of the 16-bit color
+ * value, rather than a simple low-byte truncation.
+ *
+ * What the ideal calculation should be:
+ * *dp = (((((png_uint_32)(*sp) << 8) |
+ * (png_uint_32)(*(sp + 1))) * 255 + 127)
+ * / (png_uint_32)65535L;
+ *
+ * GRR: no, I think this is what it really should be:
+ * *dp = (((((png_uint_32)(*sp) << 8) |
+ * (png_uint_32)(*(sp + 1))) + 128L)
+ * / (png_uint_32)257L;
+ *
+ * GRR: here's the exact calculation with shifts:
+ * temp = (((png_uint_32)(*sp) << 8) |
+ * (png_uint_32)(*(sp + 1))) + 128L;
+ * *dp = (temp - (temp >> 8)) >> 8;
+ *
+ * Approximate calculation with shift/add instead of multiply/divide:
+ * *dp = ((((png_uint_32)(*sp) << 8) |
+ * (png_uint_32)((int)(*(sp + 1)) - *sp)) + 128) >> 8;
+ *
+ * What we actually do to avoid extra shifting and conversion:
+ */
+
+ *dp = *sp + ((((int)(*(sp + 1)) - *sp) > 128) ? 1 : 0);
+#else
+ /* Simply discard the low order byte */
+ *dp = *sp;
+#endif
+ }
+ row_info->bit_depth = 8;
+ row_info->pixel_depth = (png_byte)(8 * row_info->channels);
+ row_info->rowbytes = row_info->width * row_info->channels;
+ }
+}
+#endif
+
+#ifdef PNG_READ_SWAP_ALPHA_SUPPORTED
+void /* PRIVATE */
+png_do_read_swap_alpha(png_row_infop row_info, png_bytep row)
+{
+ png_debug(1, "in png_do_read_swap_alpha");
+
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ if (row != NULL && row_info != NULL)
+#endif
+ {
+ png_uint_32 row_width = row_info->width;
+ if (row_info->color_type == PNG_COLOR_TYPE_RGB_ALPHA)
+ {
+ /* This converts from RGBA to ARGB */
+ if (row_info->bit_depth == 8)
+ {
+ png_bytep sp = row + row_info->rowbytes;
+ png_bytep dp = sp;
+ png_byte save;
+ png_uint_32 i;
+
+ for (i = 0; i < row_width; i++)
+ {
+ save = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = save;
+ }
+ }
+ /* This converts from RRGGBBAA to AARRGGBB */
+ else
+ {
+ png_bytep sp = row + row_info->rowbytes;
+ png_bytep dp = sp;
+ png_byte save[2];
+ png_uint_32 i;
+
+ for (i = 0; i < row_width; i++)
+ {
+ save[0] = *(--sp);
+ save[1] = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = save[0];
+ *(--dp) = save[1];
+ }
+ }
+ }
+ else if (row_info->color_type == PNG_COLOR_TYPE_GRAY_ALPHA)
+ {
+ /* This converts from GA to AG */
+ if (row_info->bit_depth == 8)
+ {
+ png_bytep sp = row + row_info->rowbytes;
+ png_bytep dp = sp;
+ png_byte save;
+ png_uint_32 i;
+
+ for (i = 0; i < row_width; i++)
+ {
+ save = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = save;
+ }
+ }
+ /* This converts from GGAA to AAGG */
+ else
+ {
+ png_bytep sp = row + row_info->rowbytes;
+ png_bytep dp = sp;
+ png_byte save[2];
+ png_uint_32 i;
+
+ for (i = 0; i < row_width; i++)
+ {
+ save[0] = *(--sp);
+ save[1] = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = save[0];
+ *(--dp) = save[1];
+ }
+ }
+ }
+ }
+}
+#endif
+
+#ifdef PNG_READ_INVERT_ALPHA_SUPPORTED
+void /* PRIVATE */
+png_do_read_invert_alpha(png_row_infop row_info, png_bytep row)
+{
+ png_debug(1, "in png_do_read_invert_alpha");
+
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ if (row != NULL && row_info != NULL)
+#endif
+ {
+ png_uint_32 row_width = row_info->width;
+ if (row_info->color_type == PNG_COLOR_TYPE_RGB_ALPHA)
+ {
+ /* This inverts the alpha channel in RGBA */
+ if (row_info->bit_depth == 8)
+ {
+ png_bytep sp = row + row_info->rowbytes;
+ png_bytep dp = sp;
+ png_uint_32 i;
+
+ for (i = 0; i < row_width; i++)
+ {
+ *(--dp) = (png_byte)(255 - *(--sp));
+
+/* This does nothing:
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ We can replace it with:
+*/
+ sp-=3;
+ dp=sp;
+ }
+ }
+ /* This inverts the alpha channel in RRGGBBAA */
+ else
+ {
+ png_bytep sp = row + row_info->rowbytes;
+ png_bytep dp = sp;
+ png_uint_32 i;
+
+ for (i = 0; i < row_width; i++)
+ {
+ *(--dp) = (png_byte)(255 - *(--sp));
+ *(--dp) = (png_byte)(255 - *(--sp));
+
+/* This does nothing:
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ We can replace it with:
+*/
+ sp-=6;
+ dp=sp;
+ }
+ }
+ }
+ else if (row_info->color_type == PNG_COLOR_TYPE_GRAY_ALPHA)
+ {
+ /* This inverts the alpha channel in GA */
+ if (row_info->bit_depth == 8)
+ {
+ png_bytep sp = row + row_info->rowbytes;
+ png_bytep dp = sp;
+ png_uint_32 i;
+
+ for (i = 0; i < row_width; i++)
+ {
+ *(--dp) = (png_byte)(255 - *(--sp));
+ *(--dp) = *(--sp);
+ }
+ }
+ /* This inverts the alpha channel in GGAA */
+ else
+ {
+ png_bytep sp = row + row_info->rowbytes;
+ png_bytep dp = sp;
+ png_uint_32 i;
+
+ for (i = 0; i < row_width; i++)
+ {
+ *(--dp) = (png_byte)(255 - *(--sp));
+ *(--dp) = (png_byte)(255 - *(--sp));
+/*
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+*/
+ sp-=2;
+ dp=sp;
+ }
+ }
+ }
+ }
+}
+#endif
+
+#ifdef PNG_READ_FILLER_SUPPORTED
+/* Add filler channel if we have RGB color */
+void /* PRIVATE */
+png_do_read_filler(png_row_infop row_info, png_bytep row,
+ png_uint_32 filler, png_uint_32 flags)
+{
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+
+ png_byte hi_filler = (png_byte)((filler>>8) & 0xff);
+ png_byte lo_filler = (png_byte)(filler & 0xff);
+
+ png_debug(1, "in png_do_read_filler");
+
+ if (
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ row != NULL && row_info != NULL &&
+#endif
+ row_info->color_type == PNG_COLOR_TYPE_GRAY)
+ {
+ if (row_info->bit_depth == 8)
+ {
+ /* This changes the data from G to GX */
+ if (flags & PNG_FLAG_FILLER_AFTER)
+ {
+ png_bytep sp = row + (png_size_t)row_width;
+ png_bytep dp = sp + (png_size_t)row_width;
+ for (i = 1; i < row_width; i++)
+ {
+ *(--dp) = lo_filler;
+ *(--dp) = *(--sp);
+ }
+ *(--dp) = lo_filler;
+ row_info->channels = 2;
+ row_info->pixel_depth = 16;
+ row_info->rowbytes = row_width * 2;
+ }
+ /* This changes the data from G to XG */
+ else
+ {
+ png_bytep sp = row + (png_size_t)row_width;
+ png_bytep dp = sp + (png_size_t)row_width;
+ for (i = 0; i < row_width; i++)
+ {
+ *(--dp) = *(--sp);
+ *(--dp) = lo_filler;
+ }
+ row_info->channels = 2;
+ row_info->pixel_depth = 16;
+ row_info->rowbytes = row_width * 2;
+ }
+ }
+ else if (row_info->bit_depth == 16)
+ {
+ /* This changes the data from GG to GGXX */
+ if (flags & PNG_FLAG_FILLER_AFTER)
+ {
+ png_bytep sp = row + (png_size_t)row_width * 2;
+ png_bytep dp = sp + (png_size_t)row_width * 2;
+ for (i = 1; i < row_width; i++)
+ {
+ *(--dp) = hi_filler;
+ *(--dp) = lo_filler;
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ }
+ *(--dp) = hi_filler;
+ *(--dp) = lo_filler;
+ row_info->channels = 2;
+ row_info->pixel_depth = 32;
+ row_info->rowbytes = row_width * 4;
+ }
+ /* This changes the data from GG to XXGG */
+ else
+ {
+ png_bytep sp = row + (png_size_t)row_width * 2;
+ png_bytep dp = sp + (png_size_t)row_width * 2;
+ for (i = 0; i < row_width; i++)
+ {
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = hi_filler;
+ *(--dp) = lo_filler;
+ }
+ row_info->channels = 2;
+ row_info->pixel_depth = 32;
+ row_info->rowbytes = row_width * 4;
+ }
+ }
+ } /* COLOR_TYPE == GRAY */
+ else if (row_info->color_type == PNG_COLOR_TYPE_RGB)
+ {
+ if (row_info->bit_depth == 8)
+ {
+ /* This changes the data from RGB to RGBX */
+ if (flags & PNG_FLAG_FILLER_AFTER)
+ {
+ png_bytep sp = row + (png_size_t)row_width * 3;
+ png_bytep dp = sp + (png_size_t)row_width;
+ for (i = 1; i < row_width; i++)
+ {
+ *(--dp) = lo_filler;
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ }
+ *(--dp) = lo_filler;
+ row_info->channels = 4;
+ row_info->pixel_depth = 32;
+ row_info->rowbytes = row_width * 4;
+ }
+ /* This changes the data from RGB to XRGB */
+ else
+ {
+ png_bytep sp = row + (png_size_t)row_width * 3;
+ png_bytep dp = sp + (png_size_t)row_width;
+ for (i = 0; i < row_width; i++)
+ {
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = lo_filler;
+ }
+ row_info->channels = 4;
+ row_info->pixel_depth = 32;
+ row_info->rowbytes = row_width * 4;
+ }
+ }
+ else if (row_info->bit_depth == 16)
+ {
+ /* This changes the data from RRGGBB to RRGGBBXX */
+ if (flags & PNG_FLAG_FILLER_AFTER)
+ {
+ png_bytep sp = row + (png_size_t)row_width * 6;
+ png_bytep dp = sp + (png_size_t)row_width * 2;
+ for (i = 1; i < row_width; i++)
+ {
+ *(--dp) = hi_filler;
+ *(--dp) = lo_filler;
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ }
+ *(--dp) = hi_filler;
+ *(--dp) = lo_filler;
+ row_info->channels = 4;
+ row_info->pixel_depth = 64;
+ row_info->rowbytes = row_width * 8;
+ }
+ /* This changes the data from RRGGBB to XXRRGGBB */
+ else
+ {
+ png_bytep sp = row + (png_size_t)row_width * 6;
+ png_bytep dp = sp + (png_size_t)row_width * 2;
+ for (i = 0; i < row_width; i++)
+ {
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = *(--sp);
+ *(--dp) = hi_filler;
+ *(--dp) = lo_filler;
+ }
+ row_info->channels = 4;
+ row_info->pixel_depth = 64;
+ row_info->rowbytes = row_width * 8;
+ }
+ }
+ } /* COLOR_TYPE == RGB */
+}
+#endif
+
+#ifdef PNG_READ_GRAY_TO_RGB_SUPPORTED
+/* Expand grayscale files to RGB, with or without alpha */
+void /* PRIVATE */
+png_do_gray_to_rgb(png_row_infop row_info, png_bytep row)
+{
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+
+ png_debug(1, "in png_do_gray_to_rgb");
+
+ if (row_info->bit_depth >= 8 &&
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ row != NULL && row_info != NULL &&
+#endif
+ !(row_info->color_type & PNG_COLOR_MASK_COLOR))
+ {
+ if (row_info->color_type == PNG_COLOR_TYPE_GRAY)
+ {
+ if (row_info->bit_depth == 8)
+ {
+ png_bytep sp = row + (png_size_t)row_width - 1;
+ png_bytep dp = sp + (png_size_t)row_width * 2;
+ for (i = 0; i < row_width; i++)
+ {
+ *(dp--) = *sp;
+ *(dp--) = *sp;
+ *(dp--) = *(sp--);
+ }
+ }
+ else
+ {
+ png_bytep sp = row + (png_size_t)row_width * 2 - 1;
+ png_bytep dp = sp + (png_size_t)row_width * 4;
+ for (i = 0; i < row_width; i++)
+ {
+ *(dp--) = *sp;
+ *(dp--) = *(sp - 1);
+ *(dp--) = *sp;
+ *(dp--) = *(sp - 1);
+ *(dp--) = *(sp--);
+ *(dp--) = *(sp--);
+ }
+ }
+ }
+ else if (row_info->color_type == PNG_COLOR_TYPE_GRAY_ALPHA)
+ {
+ if (row_info->bit_depth == 8)
+ {
+ png_bytep sp = row + (png_size_t)row_width * 2 - 1;
+ png_bytep dp = sp + (png_size_t)row_width * 2;
+ for (i = 0; i < row_width; i++)
+ {
+ *(dp--) = *(sp--);
+ *(dp--) = *sp;
+ *(dp--) = *sp;
+ *(dp--) = *(sp--);
+ }
+ }
+ else
+ {
+ png_bytep sp = row + (png_size_t)row_width * 4 - 1;
+ png_bytep dp = sp + (png_size_t)row_width * 4;
+ for (i = 0; i < row_width; i++)
+ {
+ *(dp--) = *(sp--);
+ *(dp--) = *(sp--);
+ *(dp--) = *sp;
+ *(dp--) = *(sp - 1);
+ *(dp--) = *sp;
+ *(dp--) = *(sp - 1);
+ *(dp--) = *(sp--);
+ *(dp--) = *(sp--);
+ }
+ }
+ }
+ row_info->channels += (png_byte)2;
+ row_info->color_type |= PNG_COLOR_MASK_COLOR;
+ row_info->pixel_depth = (png_byte)(row_info->channels *
+ row_info->bit_depth);
+ row_info->rowbytes = PNG_ROWBYTES(row_info->pixel_depth, row_width);
+ }
+}
+#endif
+
+#ifdef PNG_READ_RGB_TO_GRAY_SUPPORTED
+/* Reduce RGB files to grayscale, with or without alpha
+ * using the equation given in Poynton's ColorFAQ at
+ * <http://www.inforamp.net/~poynton/> (THIS LINK IS DEAD June 2008)
+ * New link:
+ * <http://www.poynton.com/notes/colour_and_gamma/>
+ * Charles Poynton poynton at poynton.com
+ *
+ * Y = 0.212671 * R + 0.715160 * G + 0.072169 * B
+ *
+ * We approximate this with
+ *
+ * Y = 0.21268 * R + 0.7151 * G + 0.07217 * B
+ *
+ * which can be expressed with integers as
+ *
+ * Y = (6969 * R + 23434 * G + 2365 * B)/32768
+ *
+ * The calculation is to be done in a linear colorspace.
+ *
+ * Other integer coefficents can be used via png_set_rgb_to_gray().
+ */
+int /* PRIVATE */
+png_do_rgb_to_gray(png_structp png_ptr, png_row_infop row_info, png_bytep row)
+
+{
+ png_uint_32 i;
+
+ png_uint_32 row_width = row_info->width;
+ int rgb_error = 0;
+
+ png_debug(1, "in png_do_rgb_to_gray");
+
+ if (
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ row != NULL && row_info != NULL &&
+#endif
+ (row_info->color_type & PNG_COLOR_MASK_COLOR))
+ {
+ png_uint_32 rc = png_ptr->rgb_to_gray_red_coeff;
+ png_uint_32 gc = png_ptr->rgb_to_gray_green_coeff;
+ png_uint_32 bc = png_ptr->rgb_to_gray_blue_coeff;
+
+ if (row_info->color_type == PNG_COLOR_TYPE_RGB)
+ {
+ if (row_info->bit_depth == 8)
+ {
+#if defined(PNG_READ_GAMMA_SUPPORTED) || defined(PNG_READ_BACKGROUND_SUPPORTED)
+ if (png_ptr->gamma_from_1 != NULL && png_ptr->gamma_to_1 != NULL)
+ {
+ png_bytep sp = row;
+ png_bytep dp = row;
+
+ for (i = 0; i < row_width; i++)
+ {
+ png_byte red = png_ptr->gamma_to_1[*(sp++)];
+ png_byte green = png_ptr->gamma_to_1[*(sp++)];
+ png_byte blue = png_ptr->gamma_to_1[*(sp++)];
+ if (red != green || red != blue)
+ {
+ rgb_error |= 1;
+ *(dp++) = png_ptr->gamma_from_1[
+ (rc*red + gc*green + bc*blue)>>15];
+ }
+ else
+ *(dp++) = *(sp - 1);
+ }
+ }
+ else
+#endif
+ {
+ png_bytep sp = row;
+ png_bytep dp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ png_byte red = *(sp++);
+ png_byte green = *(sp++);
+ png_byte blue = *(sp++);
+ if (red != green || red != blue)
+ {
+ rgb_error |= 1;
+ *(dp++) = (png_byte)((rc*red + gc*green + bc*blue)>>15);
+ }
+ else
+ *(dp++) = *(sp - 1);
+ }
+ }
+ }
+
+ else /* RGB bit_depth == 16 */
+ {
+#if defined(PNG_READ_GAMMA_SUPPORTED) || defined(PNG_READ_BACKGROUND_SUPPORTED)
+ if (png_ptr->gamma_16_to_1 != NULL &&
+ png_ptr->gamma_16_from_1 != NULL)
+ {
+ png_bytep sp = row;
+ png_bytep dp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ png_uint_16 red, green, blue, w;
+
+ red = (png_uint_16)(((*(sp))<<8) | *(sp+1)); sp+=2;
+ green = (png_uint_16)(((*(sp))<<8) | *(sp+1)); sp+=2;
+ blue = (png_uint_16)(((*(sp))<<8) | *(sp+1)); sp+=2;
+
+ if (red == green && red == blue)
+ w = red;
+ else
+ {
+ png_uint_16 red_1 = png_ptr->gamma_16_to_1[(red&0xff) >>
+ png_ptr->gamma_shift][red>>8];
+ png_uint_16 green_1 =
+ png_ptr->gamma_16_to_1[(green&0xff) >>
+ png_ptr->gamma_shift][green>>8];
+ png_uint_16 blue_1 = png_ptr->gamma_16_to_1[(blue&0xff) >>
+ png_ptr->gamma_shift][blue>>8];
+ png_uint_16 gray16 = (png_uint_16)((rc*red_1 + gc*green_1
+ + bc*blue_1)>>15);
+ w = png_ptr->gamma_16_from_1[(gray16&0xff) >>
+ png_ptr->gamma_shift][gray16 >> 8];
+ rgb_error |= 1;
+ }
+
+ *(dp++) = (png_byte)((w>>8) & 0xff);
+ *(dp++) = (png_byte)(w & 0xff);
+ }
+ }
+ else
+#endif
+ {
+ png_bytep sp = row;
+ png_bytep dp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ png_uint_16 red, green, blue, gray16;
+
+ red = (png_uint_16)(((*(sp))<<8) | *(sp+1)); sp+=2;
+ green = (png_uint_16)(((*(sp))<<8) | *(sp+1)); sp+=2;
+ blue = (png_uint_16)(((*(sp))<<8) | *(sp+1)); sp+=2;
+
+ if (red != green || red != blue)
+ rgb_error |= 1;
+ gray16 = (png_uint_16)((rc*red + gc*green + bc*blue)>>15);
+ *(dp++) = (png_byte)((gray16>>8) & 0xff);
+ *(dp++) = (png_byte)(gray16 & 0xff);
+ }
+ }
+ }
+ }
+ if (row_info->color_type == PNG_COLOR_TYPE_RGB_ALPHA)
+ {
+ if (row_info->bit_depth == 8)
+ {
+#if defined(PNG_READ_GAMMA_SUPPORTED) || defined(PNG_READ_BACKGROUND_SUPPORTED)
+ if (png_ptr->gamma_from_1 != NULL && png_ptr->gamma_to_1 != NULL)
+ {
+ png_bytep sp = row;
+ png_bytep dp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ png_byte red = png_ptr->gamma_to_1[*(sp++)];
+ png_byte green = png_ptr->gamma_to_1[*(sp++)];
+ png_byte blue = png_ptr->gamma_to_1[*(sp++)];
+ if (red != green || red != blue)
+ rgb_error |= 1;
+ *(dp++) = png_ptr->gamma_from_1
+ [(rc*red + gc*green + bc*blue)>>15];
+ *(dp++) = *(sp++); /* alpha */
+ }
+ }
+ else
+#endif
+ {
+ png_bytep sp = row;
+ png_bytep dp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ png_byte red = *(sp++);
+ png_byte green = *(sp++);
+ png_byte blue = *(sp++);
+ if (red != green || red != blue)
+ rgb_error |= 1;
+ *(dp++) = (png_byte)((rc*red + gc*green + bc*blue)>>15);
+ *(dp++) = *(sp++); /* alpha */
+ }
+ }
+ }
+ else /* RGBA bit_depth == 16 */
+ {
+#if defined(PNG_READ_GAMMA_SUPPORTED) || defined(PNG_READ_BACKGROUND_SUPPORTED)
+ if (png_ptr->gamma_16_to_1 != NULL &&
+ png_ptr->gamma_16_from_1 != NULL)
+ {
+ png_bytep sp = row;
+ png_bytep dp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ png_uint_16 red, green, blue, w;
+
+ red = (png_uint_16)(((*(sp))<<8) | *(sp+1)); sp+=2;
+ green = (png_uint_16)(((*(sp))<<8) | *(sp+1)); sp+=2;
+ blue = (png_uint_16)(((*(sp))<<8) | *(sp+1)); sp+=2;
+
+ if (red == green && red == blue)
+ w = red;
+ else
+ {
+ png_uint_16 red_1 = png_ptr->gamma_16_to_1[(red&0xff) >>
+ png_ptr->gamma_shift][red>>8];
+ png_uint_16 green_1 =
+ png_ptr->gamma_16_to_1[(green&0xff) >>
+ png_ptr->gamma_shift][green>>8];
+ png_uint_16 blue_1 = png_ptr->gamma_16_to_1[(blue&0xff) >>
+ png_ptr->gamma_shift][blue>>8];
+ png_uint_16 gray16 = (png_uint_16)((rc * red_1
+ + gc * green_1 + bc * blue_1)>>15);
+ w = png_ptr->gamma_16_from_1[(gray16&0xff) >>
+ png_ptr->gamma_shift][gray16 >> 8];
+ rgb_error |= 1;
+ }
+
+ *(dp++) = (png_byte)((w>>8) & 0xff);
+ *(dp++) = (png_byte)(w & 0xff);
+ *(dp++) = *(sp++); /* alpha */
+ *(dp++) = *(sp++);
+ }
+ }
+ else
+#endif
+ {
+ png_bytep sp = row;
+ png_bytep dp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ png_uint_16 red, green, blue, gray16;
+ red = (png_uint_16)((*(sp)<<8) | *(sp+1)); sp+=2;
+ green = (png_uint_16)((*(sp)<<8) | *(sp+1)); sp+=2;
+ blue = (png_uint_16)((*(sp)<<8) | *(sp+1)); sp+=2;
+ if (red != green || red != blue)
+ rgb_error |= 1;
+ gray16 = (png_uint_16)((rc*red + gc*green + bc*blue)>>15);
+ *(dp++) = (png_byte)((gray16>>8) & 0xff);
+ *(dp++) = (png_byte)(gray16 & 0xff);
+ *(dp++) = *(sp++); /* alpha */
+ *(dp++) = *(sp++);
+ }
+ }
+ }
+ }
+ row_info->channels -= (png_byte)2;
+ row_info->color_type &= ~PNG_COLOR_MASK_COLOR;
+ row_info->pixel_depth = (png_byte)(row_info->channels *
+ row_info->bit_depth);
+ row_info->rowbytes = PNG_ROWBYTES(row_info->pixel_depth, row_width);
+ }
+ return rgb_error;
+}
+#endif
+
+/* Build a grayscale palette. Palette is assumed to be 1 << bit_depth
+ * large of png_color. This lets grayscale images be treated as
+ * paletted. Most useful for gamma correction and simplification
+ * of code.
+ */
+void PNGAPI
+png_build_grayscale_palette(int bit_depth, png_colorp palette)
+{
+ int num_palette;
+ int color_inc;
+ int i;
+ int v;
+
+ png_debug(1, "in png_do_build_grayscale_palette");
+
+ if (palette == NULL)
+ return;
+
+ switch (bit_depth)
+ {
+ case 1:
+ num_palette = 2;
+ color_inc = 0xff;
+ break;
+
+ case 2:
+ num_palette = 4;
+ color_inc = 0x55;
+ break;
+
+ case 4:
+ num_palette = 16;
+ color_inc = 0x11;
+ break;
+
+ case 8:
+ num_palette = 256;
+ color_inc = 1;
+ break;
+
+ default:
+ num_palette = 0;
+ color_inc = 0;
+ break;
+ }
+
+ for (i = 0, v = 0; i < num_palette; i++, v += color_inc)
+ {
+ palette[i].red = (png_byte)v;
+ palette[i].green = (png_byte)v;
+ palette[i].blue = (png_byte)v;
+ }
+}
+
+/* This function is currently unused. Do we really need it? */
+#if defined(PNG_READ_DITHER_SUPPORTED) && \
+ defined(PNG_CORRECT_PALETTE_SUPPORTED)
+void /* PRIVATE */
+png_correct_palette(png_structp png_ptr, png_colorp palette,
+ int num_palette)
+{
+ png_debug(1, "in png_correct_palette");
+
+#if defined(PNG_READ_BACKGROUND_SUPPORTED) && \
+ defined(PNG_READ_GAMMA_SUPPORTED) && \
+ defined(PNG_FLOATING_POINT_SUPPORTED)
+ if (png_ptr->transformations & (PNG_GAMMA | PNG_BACKGROUND))
+ {
+ png_color back, back_1;
+
+ if (png_ptr->background_gamma_type == PNG_BACKGROUND_GAMMA_FILE)
+ {
+ back.red = png_ptr->gamma_table[png_ptr->background.red];
+ back.green = png_ptr->gamma_table[png_ptr->background.green];
+ back.blue = png_ptr->gamma_table[png_ptr->background.blue];
+
+ back_1.red = png_ptr->gamma_to_1[png_ptr->background.red];
+ back_1.green = png_ptr->gamma_to_1[png_ptr->background.green];
+ back_1.blue = png_ptr->gamma_to_1[png_ptr->background.blue];
+ }
+ else
+ {
+ double g;
+
+ g = 1.0 / (png_ptr->background_gamma * png_ptr->screen_gamma);
+
+ if (png_ptr->background_gamma_type == PNG_BACKGROUND_GAMMA_SCREEN
+ || fabs(g - 1.0) < PNG_GAMMA_THRESHOLD)
+ {
+ back.red = png_ptr->background.red;
+ back.green = png_ptr->background.green;
+ back.blue = png_ptr->background.blue;
+ }
+ else
+ {
+ back.red =
+ (png_byte)(pow((double)png_ptr->background.red/255, g) *
+ 255.0 + 0.5);
+ back.green =
+ (png_byte)(pow((double)png_ptr->background.green/255, g) *
+ 255.0 + 0.5);
+ back.blue =
+ (png_byte)(pow((double)png_ptr->background.blue/255, g) *
+ 255.0 + 0.5);
+ }
+
+ g = 1.0 / png_ptr->background_gamma;
+
+ back_1.red =
+ (png_byte)(pow((double)png_ptr->background.red/255, g) *
+ 255.0 + 0.5);
+ back_1.green =
+ (png_byte)(pow((double)png_ptr->background.green/255, g) *
+ 255.0 + 0.5);
+ back_1.blue =
+ (png_byte)(pow((double)png_ptr->background.blue/255, g) *
+ 255.0 + 0.5);
+ }
+
+ if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ png_uint_32 i;
+
+ for (i = 0; i < (png_uint_32)num_palette; i++)
+ {
+ if (i < png_ptr->num_trans && png_ptr->trans[i] == 0)
+ {
+ palette[i] = back;
+ }
+ else if (i < png_ptr->num_trans && png_ptr->trans[i] != 0xff)
+ {
+ png_byte v, w;
+
+ v = png_ptr->gamma_to_1[png_ptr->palette[i].red];
+ png_composite(w, v, png_ptr->trans[i], back_1.red);
+ palette[i].red = png_ptr->gamma_from_1[w];
+
+ v = png_ptr->gamma_to_1[png_ptr->palette[i].green];
+ png_composite(w, v, png_ptr->trans[i], back_1.green);
+ palette[i].green = png_ptr->gamma_from_1[w];
+
+ v = png_ptr->gamma_to_1[png_ptr->palette[i].blue];
+ png_composite(w, v, png_ptr->trans[i], back_1.blue);
+ palette[i].blue = png_ptr->gamma_from_1[w];
+ }
+ else
+ {
+ palette[i].red = png_ptr->gamma_table[palette[i].red];
+ palette[i].green = png_ptr->gamma_table[palette[i].green];
+ palette[i].blue = png_ptr->gamma_table[palette[i].blue];
+ }
+ }
+ }
+ else
+ {
+ int i;
+
+ for (i = 0; i < num_palette; i++)
+ {
+ if (palette[i].red == (png_byte)png_ptr->trans_values.gray)
+ {
+ palette[i] = back;
+ }
+ else
+ {
+ palette[i].red = png_ptr->gamma_table[palette[i].red];
+ palette[i].green = png_ptr->gamma_table[palette[i].green];
+ palette[i].blue = png_ptr->gamma_table[palette[i].blue];
+ }
+ }
+ }
+ }
+ else
+#endif
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ if (png_ptr->transformations & PNG_GAMMA)
+ {
+ int i;
+
+ for (i = 0; i < num_palette; i++)
+ {
+ palette[i].red = png_ptr->gamma_table[palette[i].red];
+ palette[i].green = png_ptr->gamma_table[palette[i].green];
+ palette[i].blue = png_ptr->gamma_table[palette[i].blue];
+ }
+ }
+#ifdef PNG_READ_BACKGROUND_SUPPORTED
+ else
+#endif
+#endif
+#ifdef PNG_READ_BACKGROUND_SUPPORTED
+ if (png_ptr->transformations & PNG_BACKGROUND)
+ {
+ if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ png_color back;
+
+ back.red = (png_byte)png_ptr->background.red;
+ back.green = (png_byte)png_ptr->background.green;
+ back.blue = (png_byte)png_ptr->background.blue;
+
+ for (i = 0; i < (int)png_ptr->num_trans; i++)
+ {
+ if (png_ptr->trans[i] == 0)
+ {
+ palette[i].red = back.red;
+ palette[i].green = back.green;
+ palette[i].blue = back.blue;
+ }
+ else if (png_ptr->trans[i] != 0xff)
+ {
+ png_composite(palette[i].red, png_ptr->palette[i].red,
+ png_ptr->trans[i], back.red);
+ png_composite(palette[i].green, png_ptr->palette[i].green,
+ png_ptr->trans[i], back.green);
+ png_composite(palette[i].blue, png_ptr->palette[i].blue,
+ png_ptr->trans[i], back.blue);
+ }
+ }
+ }
+ else /* Assume grayscale palette (what else could it be?) */
+ {
+ int i;
+
+ for (i = 0; i < num_palette; i++)
+ {
+ if (i == (png_byte)png_ptr->trans_values.gray)
+ {
+ palette[i].red = (png_byte)png_ptr->background.red;
+ palette[i].green = (png_byte)png_ptr->background.green;
+ palette[i].blue = (png_byte)png_ptr->background.blue;
+ }
+ }
+ }
+ }
+#endif
+}
+#endif
+
+#ifdef PNG_READ_BACKGROUND_SUPPORTED
+/* Replace any alpha or transparency with the supplied background color.
+ * "background" is already in the screen gamma, while "background_1" is
+ * at a gamma of 1.0. Paletted files have already been taken care of.
+ */
+void /* PRIVATE */
+png_do_background(png_row_infop row_info, png_bytep row,
+ png_color_16p trans_values, png_color_16p background
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ , png_color_16p background_1,
+ png_bytep gamma_table, png_bytep gamma_from_1, png_bytep gamma_to_1,
+ png_uint_16pp gamma_16, png_uint_16pp gamma_16_from_1,
+ png_uint_16pp gamma_16_to_1, int gamma_shift
+#endif
+ )
+{
+ png_bytep sp, dp;
+ png_uint_32 i;
+ png_uint_32 row_width=row_info->width;
+ int shift;
+
+ png_debug(1, "in png_do_background");
+
+ if (background != NULL &&
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ row != NULL && row_info != NULL &&
+#endif
+ (!(row_info->color_type & PNG_COLOR_MASK_ALPHA) ||
+ (row_info->color_type != PNG_COLOR_TYPE_PALETTE && trans_values)))
+ {
+ switch (row_info->color_type)
+ {
+ case PNG_COLOR_TYPE_GRAY:
+ {
+ switch (row_info->bit_depth)
+ {
+ case 1:
+ {
+ sp = row;
+ shift = 7;
+ for (i = 0; i < row_width; i++)
+ {
+ if ((png_uint_16)((*sp >> shift) & 0x01)
+ == trans_values->gray)
+ {
+ *sp &= (png_byte)((0x7f7f >> (7 - shift)) & 0xff);
+ *sp |= (png_byte)(background->gray << shift);
+ }
+ if (!shift)
+ {
+ shift = 7;
+ sp++;
+ }
+ else
+ shift--;
+ }
+ break;
+ }
+
+ case 2:
+ {
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ if (gamma_table != NULL)
+ {
+ sp = row;
+ shift = 6;
+ for (i = 0; i < row_width; i++)
+ {
+ if ((png_uint_16)((*sp >> shift) & 0x03)
+ == trans_values->gray)
+ {
+ *sp &= (png_byte)((0x3f3f >> (6 - shift)) & 0xff);
+ *sp |= (png_byte)(background->gray << shift);
+ }
+ else
+ {
+ png_byte p = (png_byte)((*sp >> shift) & 0x03);
+ png_byte g = (png_byte)((gamma_table [p | (p << 2) |
+ (p << 4) | (p << 6)] >> 6) & 0x03);
+ *sp &= (png_byte)((0x3f3f >> (6 - shift)) & 0xff);
+ *sp |= (png_byte)(g << shift);
+ }
+ if (!shift)
+ {
+ shift = 6;
+ sp++;
+ }
+ else
+ shift -= 2;
+ }
+ }
+ else
+#endif
+ {
+ sp = row;
+ shift = 6;
+ for (i = 0; i < row_width; i++)
+ {
+ if ((png_uint_16)((*sp >> shift) & 0x03)
+ == trans_values->gray)
+ {
+ *sp &= (png_byte)((0x3f3f >> (6 - shift)) & 0xff);
+ *sp |= (png_byte)(background->gray << shift);
+ }
+ if (!shift)
+ {
+ shift = 6;
+ sp++;
+ }
+ else
+ shift -= 2;
+ }
+ }
+ break;
+ }
+
+ case 4:
+ {
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ if (gamma_table != NULL)
+ {
+ sp = row;
+ shift = 4;
+ for (i = 0; i < row_width; i++)
+ {
+ if ((png_uint_16)((*sp >> shift) & 0x0f)
+ == trans_values->gray)
+ {
+ *sp &= (png_byte)((0xf0f >> (4 - shift)) & 0xff);
+ *sp |= (png_byte)(background->gray << shift);
+ }
+ else
+ {
+ png_byte p = (png_byte)((*sp >> shift) & 0x0f);
+ png_byte g = (png_byte)((gamma_table[p |
+ (p << 4)] >> 4) & 0x0f);
+ *sp &= (png_byte)((0xf0f >> (4 - shift)) & 0xff);
+ *sp |= (png_byte)(g << shift);
+ }
+ if (!shift)
+ {
+ shift = 4;
+ sp++;
+ }
+ else
+ shift -= 4;
+ }
+ }
+ else
+#endif
+ {
+ sp = row;
+ shift = 4;
+ for (i = 0; i < row_width; i++)
+ {
+ if ((png_uint_16)((*sp >> shift) & 0x0f)
+ == trans_values->gray)
+ {
+ *sp &= (png_byte)((0xf0f >> (4 - shift)) & 0xff);
+ *sp |= (png_byte)(background->gray << shift);
+ }
+ if (!shift)
+ {
+ shift = 4;
+ sp++;
+ }
+ else
+ shift -= 4;
+ }
+ }
+ break;
+ }
+
+ case 8:
+ {
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ if (gamma_table != NULL)
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++, sp++)
+ {
+ if (*sp == trans_values->gray)
+ {
+ *sp = (png_byte)background->gray;
+ }
+ else
+ {
+ *sp = gamma_table[*sp];
+ }
+ }
+ }
+ else
+#endif
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++, sp++)
+ {
+ if (*sp == trans_values->gray)
+ {
+ *sp = (png_byte)background->gray;
+ }
+ }
+ }
+ break;
+ }
+
+ case 16:
+ {
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ if (gamma_16 != NULL)
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++, sp += 2)
+ {
+ png_uint_16 v;
+
+ v = (png_uint_16)(((*sp) << 8) + *(sp + 1));
+ if (v == trans_values->gray)
+ {
+ /* Background is already in screen gamma */
+ *sp = (png_byte)((background->gray >> 8) & 0xff);
+ *(sp + 1) = (png_byte)(background->gray & 0xff);
+ }
+ else
+ {
+ v = gamma_16[*(sp + 1) >> gamma_shift][*sp];
+ *sp = (png_byte)((v >> 8) & 0xff);
+ *(sp + 1) = (png_byte)(v & 0xff);
+ }
+ }
+ }
+ else
+#endif
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++, sp += 2)
+ {
+ png_uint_16 v;
+
+ v = (png_uint_16)(((*sp) << 8) + *(sp + 1));
+ if (v == trans_values->gray)
+ {
+ *sp = (png_byte)((background->gray >> 8) & 0xff);
+ *(sp + 1) = (png_byte)(background->gray & 0xff);
+ }
+ }
+ }
+ break;
+ }
+ }
+ break;
+ }
+
+ case PNG_COLOR_TYPE_RGB:
+ {
+ if (row_info->bit_depth == 8)
+ {
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ if (gamma_table != NULL)
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++, sp += 3)
+ {
+ if (*sp == trans_values->red &&
+ *(sp + 1) == trans_values->green &&
+ *(sp + 2) == trans_values->blue)
+ {
+ *sp = (png_byte)background->red;
+ *(sp + 1) = (png_byte)background->green;
+ *(sp + 2) = (png_byte)background->blue;
+ }
+ else
+ {
+ *sp = gamma_table[*sp];
+ *(sp + 1) = gamma_table[*(sp + 1)];
+ *(sp + 2) = gamma_table[*(sp + 2)];
+ }
+ }
+ }
+ else
+#endif
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++, sp += 3)
+ {
+ if (*sp == trans_values->red &&
+ *(sp + 1) == trans_values->green &&
+ *(sp + 2) == trans_values->blue)
+ {
+ *sp = (png_byte)background->red;
+ *(sp + 1) = (png_byte)background->green;
+ *(sp + 2) = (png_byte)background->blue;
+ }
+ }
+ }
+ }
+ else /* if (row_info->bit_depth == 16) */
+ {
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ if (gamma_16 != NULL)
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++, sp += 6)
+ {
+ png_uint_16 r = (png_uint_16)(((*sp) << 8) + *(sp + 1));
+ png_uint_16 g = (png_uint_16)(((*(sp+2)) << 8) + *(sp+3));
+ png_uint_16 b = (png_uint_16)(((*(sp+4)) << 8) + *(sp+5));
+ if (r == trans_values->red && g == trans_values->green &&
+ b == trans_values->blue)
+ {
+ /* Background is already in screen gamma */
+ *sp = (png_byte)((background->red >> 8) & 0xff);
+ *(sp + 1) = (png_byte)(background->red & 0xff);
+ *(sp + 2) = (png_byte)((background->green >> 8) & 0xff);
+ *(sp + 3) = (png_byte)(background->green & 0xff);
+ *(sp + 4) = (png_byte)((background->blue >> 8) & 0xff);
+ *(sp + 5) = (png_byte)(background->blue & 0xff);
+ }
+ else
+ {
+ png_uint_16 v = gamma_16[*(sp + 1) >> gamma_shift][*sp];
+ *sp = (png_byte)((v >> 8) & 0xff);
+ *(sp + 1) = (png_byte)(v & 0xff);
+ v = gamma_16[*(sp + 3) >> gamma_shift][*(sp + 2)];
+ *(sp + 2) = (png_byte)((v >> 8) & 0xff);
+ *(sp + 3) = (png_byte)(v & 0xff);
+ v = gamma_16[*(sp + 5) >> gamma_shift][*(sp + 4)];
+ *(sp + 4) = (png_byte)((v >> 8) & 0xff);
+ *(sp + 5) = (png_byte)(v & 0xff);
+ }
+ }
+ }
+ else
+#endif
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++, sp += 6)
+ {
+ png_uint_16 r = (png_uint_16)(((*sp) << 8) + *(sp+1));
+ png_uint_16 g = (png_uint_16)(((*(sp+2)) << 8) + *(sp+3));
+ png_uint_16 b = (png_uint_16)(((*(sp+4)) << 8) + *(sp+5));
+
+ if (r == trans_values->red && g == trans_values->green &&
+ b == trans_values->blue)
+ {
+ *sp = (png_byte)((background->red >> 8) & 0xff);
+ *(sp + 1) = (png_byte)(background->red & 0xff);
+ *(sp + 2) = (png_byte)((background->green >> 8) & 0xff);
+ *(sp + 3) = (png_byte)(background->green & 0xff);
+ *(sp + 4) = (png_byte)((background->blue >> 8) & 0xff);
+ *(sp + 5) = (png_byte)(background->blue & 0xff);
+ }
+ }
+ }
+ }
+ break;
+ }
+
+ case PNG_COLOR_TYPE_GRAY_ALPHA:
+ {
+ if (row_info->bit_depth == 8)
+ {
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ if (gamma_to_1 != NULL && gamma_from_1 != NULL &&
+ gamma_table != NULL)
+ {
+ sp = row;
+ dp = row;
+ for (i = 0; i < row_width; i++, sp += 2, dp++)
+ {
+ png_uint_16 a = *(sp + 1);
+
+ if (a == 0xff)
+ {
+ *dp = gamma_table[*sp];
+ }
+ else if (a == 0)
+ {
+ /* Background is already in screen gamma */
+ *dp = (png_byte)background->gray;
+ }
+ else
+ {
+ png_byte v, w;
+
+ v = gamma_to_1[*sp];
+ png_composite(w, v, a, background_1->gray);
+ *dp = gamma_from_1[w];
+ }
+ }
+ }
+ else
+#endif
+ {
+ sp = row;
+ dp = row;
+ for (i = 0; i < row_width; i++, sp += 2, dp++)
+ {
+ png_byte a = *(sp + 1);
+
+ if (a == 0xff)
+ {
+ *dp = *sp;
+ }
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ else if (a == 0)
+ {
+ *dp = (png_byte)background->gray;
+ }
+ else
+ {
+ png_composite(*dp, *sp, a, background_1->gray);
+ }
+#else
+ *dp = (png_byte)background->gray;
+#endif
+ }
+ }
+ }
+ else /* if (png_ptr->bit_depth == 16) */
+ {
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ if (gamma_16 != NULL && gamma_16_from_1 != NULL &&
+ gamma_16_to_1 != NULL)
+ {
+ sp = row;
+ dp = row;
+ for (i = 0; i < row_width; i++, sp += 4, dp += 2)
+ {
+ png_uint_16 a = (png_uint_16)(((*(sp+2)) << 8) + *(sp+3));
+
+ if (a == (png_uint_16)0xffff)
+ {
+ png_uint_16 v;
+
+ v = gamma_16[*(sp + 1) >> gamma_shift][*sp];
+ *dp = (png_byte)((v >> 8) & 0xff);
+ *(dp + 1) = (png_byte)(v & 0xff);
+ }
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ else if (a == 0)
+#else
+ else
+#endif
+ {
+ /* Background is already in screen gamma */
+ *dp = (png_byte)((background->gray >> 8) & 0xff);
+ *(dp + 1) = (png_byte)(background->gray & 0xff);
+ }
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ else
+ {
+ png_uint_16 g, v, w;
+
+ g = gamma_16_to_1[*(sp + 1) >> gamma_shift][*sp];
+ png_composite_16(v, g, a, background_1->gray);
+ w = gamma_16_from_1[(v&0xff) >> gamma_shift][v >> 8];
+ *dp = (png_byte)((w >> 8) & 0xff);
+ *(dp + 1) = (png_byte)(w & 0xff);
+ }
+#endif
+ }
+ }
+ else
+#endif
+ {
+ sp = row;
+ dp = row;
+ for (i = 0; i < row_width; i++, sp += 4, dp += 2)
+ {
+ png_uint_16 a = (png_uint_16)(((*(sp+2)) << 8) + *(sp+3));
+ if (a == (png_uint_16)0xffff)
+ {
+ png_memcpy(dp, sp, 2);
+ }
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ else if (a == 0)
+#else
+ else
+#endif
+ {
+ *dp = (png_byte)((background->gray >> 8) & 0xff);
+ *(dp + 1) = (png_byte)(background->gray & 0xff);
+ }
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ else
+ {
+ png_uint_16 g, v;
+
+ g = (png_uint_16)(((*sp) << 8) + *(sp + 1));
+ png_composite_16(v, g, a, background_1->gray);
+ *dp = (png_byte)((v >> 8) & 0xff);
+ *(dp + 1) = (png_byte)(v & 0xff);
+ }
+#endif
+ }
+ }
+ }
+ break;
+ }
+
+ case PNG_COLOR_TYPE_RGB_ALPHA:
+ {
+ if (row_info->bit_depth == 8)
+ {
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ if (gamma_to_1 != NULL && gamma_from_1 != NULL &&
+ gamma_table != NULL)
+ {
+ sp = row;
+ dp = row;
+ for (i = 0; i < row_width; i++, sp += 4, dp += 3)
+ {
+ png_byte a = *(sp + 3);
+
+ if (a == 0xff)
+ {
+ *dp = gamma_table[*sp];
+ *(dp + 1) = gamma_table[*(sp + 1)];
+ *(dp + 2) = gamma_table[*(sp + 2)];
+ }
+ else if (a == 0)
+ {
+ /* Background is already in screen gamma */
+ *dp = (png_byte)background->red;
+ *(dp + 1) = (png_byte)background->green;
+ *(dp + 2) = (png_byte)background->blue;
+ }
+ else
+ {
+ png_byte v, w;
+
+ v = gamma_to_1[*sp];
+ png_composite(w, v, a, background_1->red);
+ *dp = gamma_from_1[w];
+ v = gamma_to_1[*(sp + 1)];
+ png_composite(w, v, a, background_1->green);
+ *(dp + 1) = gamma_from_1[w];
+ v = gamma_to_1[*(sp + 2)];
+ png_composite(w, v, a, background_1->blue);
+ *(dp + 2) = gamma_from_1[w];
+ }
+ }
+ }
+ else
+#endif
+ {
+ sp = row;
+ dp = row;
+ for (i = 0; i < row_width; i++, sp += 4, dp += 3)
+ {
+ png_byte a = *(sp + 3);
+
+ if (a == 0xff)
+ {
+ *dp = *sp;
+ *(dp + 1) = *(sp + 1);
+ *(dp + 2) = *(sp + 2);
+ }
+ else if (a == 0)
+ {
+ *dp = (png_byte)background->red;
+ *(dp + 1) = (png_byte)background->green;
+ *(dp + 2) = (png_byte)background->blue;
+ }
+ else
+ {
+ png_composite(*dp, *sp, a, background->red);
+ png_composite(*(dp + 1), *(sp + 1), a,
+ background->green);
+ png_composite(*(dp + 2), *(sp + 2), a,
+ background->blue);
+ }
+ }
+ }
+ }
+ else /* if (row_info->bit_depth == 16) */
+ {
+#ifdef PNG_READ_GAMMA_SUPPORTED
+ if (gamma_16 != NULL && gamma_16_from_1 != NULL &&
+ gamma_16_to_1 != NULL)
+ {
+ sp = row;
+ dp = row;
+ for (i = 0; i < row_width; i++, sp += 8, dp += 6)
+ {
+ png_uint_16 a = (png_uint_16)(((png_uint_16)(*(sp + 6))
+ << 8) + (png_uint_16)(*(sp + 7)));
+ if (a == (png_uint_16)0xffff)
+ {
+ png_uint_16 v;
+
+ v = gamma_16[*(sp + 1) >> gamma_shift][*sp];
+ *dp = (png_byte)((v >> 8) & 0xff);
+ *(dp + 1) = (png_byte)(v & 0xff);
+ v = gamma_16[*(sp + 3) >> gamma_shift][*(sp + 2)];
+ *(dp + 2) = (png_byte)((v >> 8) & 0xff);
+ *(dp + 3) = (png_byte)(v & 0xff);
+ v = gamma_16[*(sp + 5) >> gamma_shift][*(sp + 4)];
+ *(dp + 4) = (png_byte)((v >> 8) & 0xff);
+ *(dp + 5) = (png_byte)(v & 0xff);
+ }
+ else if (a == 0)
+ {
+ /* Background is already in screen gamma */
+ *dp = (png_byte)((background->red >> 8) & 0xff);
+ *(dp + 1) = (png_byte)(background->red & 0xff);
+ *(dp + 2) = (png_byte)((background->green >> 8) & 0xff);
+ *(dp + 3) = (png_byte)(background->green & 0xff);
+ *(dp + 4) = (png_byte)((background->blue >> 8) & 0xff);
+ *(dp + 5) = (png_byte)(background->blue & 0xff);
+ }
+ else
+ {
+ png_uint_16 v, w, x;
+
+ v = gamma_16_to_1[*(sp + 1) >> gamma_shift][*sp];
+ png_composite_16(w, v, a, background_1->red);
+ x = gamma_16_from_1[((w&0xff) >> gamma_shift)][w >> 8];
+ *dp = (png_byte)((x >> 8) & 0xff);
+ *(dp + 1) = (png_byte)(x & 0xff);
+ v = gamma_16_to_1[*(sp + 3) >> gamma_shift][*(sp + 2)];
+ png_composite_16(w, v, a, background_1->green);
+ x = gamma_16_from_1[((w&0xff) >> gamma_shift)][w >> 8];
+ *(dp + 2) = (png_byte)((x >> 8) & 0xff);
+ *(dp + 3) = (png_byte)(x & 0xff);
+ v = gamma_16_to_1[*(sp + 5) >> gamma_shift][*(sp + 4)];
+ png_composite_16(w, v, a, background_1->blue);
+ x = gamma_16_from_1[(w & 0xff) >> gamma_shift][w >> 8];
+ *(dp + 4) = (png_byte)((x >> 8) & 0xff);
+ *(dp + 5) = (png_byte)(x & 0xff);
+ }
+ }
+ }
+ else
+#endif
+ {
+ sp = row;
+ dp = row;
+ for (i = 0; i < row_width; i++, sp += 8, dp += 6)
+ {
+ png_uint_16 a = (png_uint_16)(((png_uint_16)(*(sp + 6))
+ << 8) + (png_uint_16)(*(sp + 7)));
+ if (a == (png_uint_16)0xffff)
+ {
+ png_memcpy(dp, sp, 6);
+ }
+ else if (a == 0)
+ {
+ *dp = (png_byte)((background->red >> 8) & 0xff);
+ *(dp + 1) = (png_byte)(background->red & 0xff);
+ *(dp + 2) = (png_byte)((background->green >> 8) & 0xff);
+ *(dp + 3) = (png_byte)(background->green & 0xff);
+ *(dp + 4) = (png_byte)((background->blue >> 8) & 0xff);
+ *(dp + 5) = (png_byte)(background->blue & 0xff);
+ }
+ else
+ {
+ png_uint_16 v;
+
+ png_uint_16 r = (png_uint_16)(((*sp) << 8) + *(sp + 1));
+ png_uint_16 g = (png_uint_16)(((*(sp + 2)) << 8)
+ + *(sp + 3));
+ png_uint_16 b = (png_uint_16)(((*(sp + 4)) << 8)
+ + *(sp + 5));
+
+ png_composite_16(v, r, a, background->red);
+ *dp = (png_byte)((v >> 8) & 0xff);
+ *(dp + 1) = (png_byte)(v & 0xff);
+ png_composite_16(v, g, a, background->green);
+ *(dp + 2) = (png_byte)((v >> 8) & 0xff);
+ *(dp + 3) = (png_byte)(v & 0xff);
+ png_composite_16(v, b, a, background->blue);
+ *(dp + 4) = (png_byte)((v >> 8) & 0xff);
+ *(dp + 5) = (png_byte)(v & 0xff);
+ }
+ }
+ }
+ }
+ break;
+ }
+ }
+
+ if (row_info->color_type & PNG_COLOR_MASK_ALPHA)
+ {
+ row_info->color_type &= ~PNG_COLOR_MASK_ALPHA;
+ row_info->channels--;
+ row_info->pixel_depth = (png_byte)(row_info->channels *
+ row_info->bit_depth);
+ row_info->rowbytes = PNG_ROWBYTES(row_info->pixel_depth, row_width);
+ }
+ }
+}
+#endif
+
+#ifdef PNG_READ_GAMMA_SUPPORTED
+/* Gamma correct the image, avoiding the alpha channel. Make sure
+ * you do this after you deal with the transparency issue on grayscale
+ * or RGB images. If your bit depth is 8, use gamma_table, if it
+ * is 16, use gamma_16_table and gamma_shift. Build these with
+ * build_gamma_table().
+ */
+void /* PRIVATE */
+png_do_gamma(png_row_infop row_info, png_bytep row,
+ png_bytep gamma_table, png_uint_16pp gamma_16_table,
+ int gamma_shift)
+{
+ png_bytep sp;
+ png_uint_32 i;
+ png_uint_32 row_width=row_info->width;
+
+ png_debug(1, "in png_do_gamma");
+
+ if (
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ row != NULL && row_info != NULL &&
+#endif
+ ((row_info->bit_depth <= 8 && gamma_table != NULL) ||
+ (row_info->bit_depth == 16 && gamma_16_table != NULL)))
+ {
+ switch (row_info->color_type)
+ {
+ case PNG_COLOR_TYPE_RGB:
+ {
+ if (row_info->bit_depth == 8)
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ *sp = gamma_table[*sp];
+ sp++;
+ *sp = gamma_table[*sp];
+ sp++;
+ *sp = gamma_table[*sp];
+ sp++;
+ }
+ }
+ else /* if (row_info->bit_depth == 16) */
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ png_uint_16 v;
+
+ v = gamma_16_table[*(sp + 1) >> gamma_shift][*sp];
+ *sp = (png_byte)((v >> 8) & 0xff);
+ *(sp + 1) = (png_byte)(v & 0xff);
+ sp += 2;
+ v = gamma_16_table[*(sp + 1) >> gamma_shift][*sp];
+ *sp = (png_byte)((v >> 8) & 0xff);
+ *(sp + 1) = (png_byte)(v & 0xff);
+ sp += 2;
+ v = gamma_16_table[*(sp + 1) >> gamma_shift][*sp];
+ *sp = (png_byte)((v >> 8) & 0xff);
+ *(sp + 1) = (png_byte)(v & 0xff);
+ sp += 2;
+ }
+ }
+ break;
+ }
+
+ case PNG_COLOR_TYPE_RGB_ALPHA:
+ {
+ if (row_info->bit_depth == 8)
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ *sp = gamma_table[*sp];
+ sp++;
+ *sp = gamma_table[*sp];
+ sp++;
+ *sp = gamma_table[*sp];
+ sp++;
+ sp++;
+ }
+ }
+ else /* if (row_info->bit_depth == 16) */
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ png_uint_16 v = gamma_16_table[*(sp + 1) >> gamma_shift][*sp];
+ *sp = (png_byte)((v >> 8) & 0xff);
+ *(sp + 1) = (png_byte)(v & 0xff);
+ sp += 2;
+ v = gamma_16_table[*(sp + 1) >> gamma_shift][*sp];
+ *sp = (png_byte)((v >> 8) & 0xff);
+ *(sp + 1) = (png_byte)(v & 0xff);
+ sp += 2;
+ v = gamma_16_table[*(sp + 1) >> gamma_shift][*sp];
+ *sp = (png_byte)((v >> 8) & 0xff);
+ *(sp + 1) = (png_byte)(v & 0xff);
+ sp += 4;
+ }
+ }
+ break;
+ }
+
+ case PNG_COLOR_TYPE_GRAY_ALPHA:
+ {
+ if (row_info->bit_depth == 8)
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ *sp = gamma_table[*sp];
+ sp += 2;
+ }
+ }
+ else /* if (row_info->bit_depth == 16) */
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ png_uint_16 v = gamma_16_table[*(sp + 1) >> gamma_shift][*sp];
+ *sp = (png_byte)((v >> 8) & 0xff);
+ *(sp + 1) = (png_byte)(v & 0xff);
+ sp += 4;
+ }
+ }
+ break;
+ }
+
+ case PNG_COLOR_TYPE_GRAY:
+ {
+ if (row_info->bit_depth == 2)
+ {
+ sp = row;
+ for (i = 0; i < row_width; i += 4)
+ {
+ int a = *sp & 0xc0;
+ int b = *sp & 0x30;
+ int c = *sp & 0x0c;
+ int d = *sp & 0x03;
+
+ *sp = (png_byte)(
+ ((((int)gamma_table[a|(a>>2)|(a>>4)|(a>>6)]) ) & 0xc0)|
+ ((((int)gamma_table[(b<<2)|b|(b>>2)|(b>>4)])>>2) & 0x30)|
+ ((((int)gamma_table[(c<<4)|(c<<2)|c|(c>>2)])>>4) & 0x0c)|
+ ((((int)gamma_table[(d<<6)|(d<<4)|(d<<2)|d])>>6) ));
+ sp++;
+ }
+ }
+
+ if (row_info->bit_depth == 4)
+ {
+ sp = row;
+ for (i = 0; i < row_width; i += 2)
+ {
+ int msb = *sp & 0xf0;
+ int lsb = *sp & 0x0f;
+
+ *sp = (png_byte)((((int)gamma_table[msb | (msb >> 4)]) & 0xf0)
+ | (((int)gamma_table[(lsb << 4) | lsb]) >> 4));
+ sp++;
+ }
+ }
+
+ else if (row_info->bit_depth == 8)
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ *sp = gamma_table[*sp];
+ sp++;
+ }
+ }
+
+ else if (row_info->bit_depth == 16)
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ png_uint_16 v = gamma_16_table[*(sp + 1) >> gamma_shift][*sp];
+ *sp = (png_byte)((v >> 8) & 0xff);
+ *(sp + 1) = (png_byte)(v & 0xff);
+ sp += 2;
+ }
+ }
+ break;
+ }
+ }
+ }
+}
+#endif
+
+#ifdef PNG_READ_EXPAND_SUPPORTED
+/* Expands a palette row to an RGB or RGBA row depending
+ * upon whether you supply trans and num_trans.
+ */
+void /* PRIVATE */
+png_do_expand_palette(png_row_infop row_info, png_bytep row,
+ png_colorp palette, png_bytep trans, int num_trans)
+{
+ int shift, value;
+ png_bytep sp, dp;
+ png_uint_32 i;
+ png_uint_32 row_width=row_info->width;
+
+ png_debug(1, "in png_do_expand_palette");
+
+ if (
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ row != NULL && row_info != NULL &&
+#endif
+ row_info->color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ if (row_info->bit_depth < 8)
+ {
+ switch (row_info->bit_depth)
+ {
+ case 1:
+ {
+ sp = row + (png_size_t)((row_width - 1) >> 3);
+ dp = row + (png_size_t)row_width - 1;
+ shift = 7 - (int)((row_width + 7) & 0x07);
+ for (i = 0; i < row_width; i++)
+ {
+ if ((*sp >> shift) & 0x01)
+ *dp = 1;
+ else
+ *dp = 0;
+ if (shift == 7)
+ {
+ shift = 0;
+ sp--;
+ }
+ else
+ shift++;
+
+ dp--;
+ }
+ break;
+ }
+
+ case 2:
+ {
+ sp = row + (png_size_t)((row_width - 1) >> 2);
+ dp = row + (png_size_t)row_width - 1;
+ shift = (int)((3 - ((row_width + 3) & 0x03)) << 1);
+ for (i = 0; i < row_width; i++)
+ {
+ value = (*sp >> shift) & 0x03;
+ *dp = (png_byte)value;
+ if (shift == 6)
+ {
+ shift = 0;
+ sp--;
+ }
+ else
+ shift += 2;
+
+ dp--;
+ }
+ break;
+ }
+
+ case 4:
+ {
+ sp = row + (png_size_t)((row_width - 1) >> 1);
+ dp = row + (png_size_t)row_width - 1;
+ shift = (int)((row_width & 0x01) << 2);
+ for (i = 0; i < row_width; i++)
+ {
+ value = (*sp >> shift) & 0x0f;
+ *dp = (png_byte)value;
+ if (shift == 4)
+ {
+ shift = 0;
+ sp--;
+ }
+ else
+ shift += 4;
+
+ dp--;
+ }
+ break;
+ }
+ }
+ row_info->bit_depth = 8;
+ row_info->pixel_depth = 8;
+ row_info->rowbytes = row_width;
+ }
+ switch (row_info->bit_depth)
+ {
+ case 8:
+ {
+ if (trans != NULL)
+ {
+ sp = row + (png_size_t)row_width - 1;
+ dp = row + (png_size_t)(row_width << 2) - 1;
+
+ for (i = 0; i < row_width; i++)
+ {
+ if ((int)(*sp) >= num_trans)
+ *dp-- = 0xff;
+ else
+ *dp-- = trans[*sp];
+ *dp-- = palette[*sp].blue;
+ *dp-- = palette[*sp].green;
+ *dp-- = palette[*sp].red;
+ sp--;
+ }
+ row_info->bit_depth = 8;
+ row_info->pixel_depth = 32;
+ row_info->rowbytes = row_width * 4;
+ row_info->color_type = 6;
+ row_info->channels = 4;
+ }
+ else
+ {
+ sp = row + (png_size_t)row_width - 1;
+ dp = row + (png_size_t)(row_width * 3) - 1;
+
+ for (i = 0; i < row_width; i++)
+ {
+ *dp-- = palette[*sp].blue;
+ *dp-- = palette[*sp].green;
+ *dp-- = palette[*sp].red;
+ sp--;
+ }
+
+ row_info->bit_depth = 8;
+ row_info->pixel_depth = 24;
+ row_info->rowbytes = row_width * 3;
+ row_info->color_type = 2;
+ row_info->channels = 3;
+ }
+ break;
+ }
+ }
+ }
+}
+
+/* If the bit depth < 8, it is expanded to 8. Also, if the already
+ * expanded transparency value is supplied, an alpha channel is built.
+ */
+void /* PRIVATE */
+png_do_expand(png_row_infop row_info, png_bytep row,
+ png_color_16p trans_value)
+{
+ int shift, value;
+ png_bytep sp, dp;
+ png_uint_32 i;
+ png_uint_32 row_width=row_info->width;
+
+ png_debug(1, "in png_do_expand");
+
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ if (row != NULL && row_info != NULL)
+#endif
+ {
+ if (row_info->color_type == PNG_COLOR_TYPE_GRAY)
+ {
+ png_uint_16 gray = (png_uint_16)(trans_value ? trans_value->gray : 0);
+
+ if (row_info->bit_depth < 8)
+ {
+ switch (row_info->bit_depth)
+ {
+ case 1:
+ {
+ gray = (png_uint_16)((gray&0x01)*0xff);
+ sp = row + (png_size_t)((row_width - 1) >> 3);
+ dp = row + (png_size_t)row_width - 1;
+ shift = 7 - (int)((row_width + 7) & 0x07);
+ for (i = 0; i < row_width; i++)
+ {
+ if ((*sp >> shift) & 0x01)
+ *dp = 0xff;
+ else
+ *dp = 0;
+ if (shift == 7)
+ {
+ shift = 0;
+ sp--;
+ }
+ else
+ shift++;
+
+ dp--;
+ }
+ break;
+ }
+
+ case 2:
+ {
+ gray = (png_uint_16)((gray&0x03)*0x55);
+ sp = row + (png_size_t)((row_width - 1) >> 2);
+ dp = row + (png_size_t)row_width - 1;
+ shift = (int)((3 - ((row_width + 3) & 0x03)) << 1);
+ for (i = 0; i < row_width; i++)
+ {
+ value = (*sp >> shift) & 0x03;
+ *dp = (png_byte)(value | (value << 2) | (value << 4) |
+ (value << 6));
+ if (shift == 6)
+ {
+ shift = 0;
+ sp--;
+ }
+ else
+ shift += 2;
+
+ dp--;
+ }
+ break;
+ }
+
+ case 4:
+ {
+ gray = (png_uint_16)((gray&0x0f)*0x11);
+ sp = row + (png_size_t)((row_width - 1) >> 1);
+ dp = row + (png_size_t)row_width - 1;
+ shift = (int)((1 - ((row_width + 1) & 0x01)) << 2);
+ for (i = 0; i < row_width; i++)
+ {
+ value = (*sp >> shift) & 0x0f;
+ *dp = (png_byte)(value | (value << 4));
+ if (shift == 4)
+ {
+ shift = 0;
+ sp--;
+ }
+ else
+ shift = 4;
+
+ dp--;
+ }
+ break;
+ }
+ }
+
+ row_info->bit_depth = 8;
+ row_info->pixel_depth = 8;
+ row_info->rowbytes = row_width;
+ }
+
+ if (trans_value != NULL)
+ {
+ if (row_info->bit_depth == 8)
+ {
+ gray = gray & 0xff;
+ sp = row + (png_size_t)row_width - 1;
+ dp = row + (png_size_t)(row_width << 1) - 1;
+ for (i = 0; i < row_width; i++)
+ {
+ if (*sp == gray)
+ *dp-- = 0;
+ else
+ *dp-- = 0xff;
+ *dp-- = *sp--;
+ }
+ }
+
+ else if (row_info->bit_depth == 16)
+ {
+ png_byte gray_high = (gray >> 8) & 0xff;
+ png_byte gray_low = gray & 0xff;
+ sp = row + row_info->rowbytes - 1;
+ dp = row + (row_info->rowbytes << 1) - 1;
+ for (i = 0; i < row_width; i++)
+ {
+ if (*(sp - 1) == gray_high && *(sp) == gray_low)
+ {
+ *dp-- = 0;
+ *dp-- = 0;
+ }
+ else
+ {
+ *dp-- = 0xff;
+ *dp-- = 0xff;
+ }
+ *dp-- = *sp--;
+ *dp-- = *sp--;
+ }
+ }
+
+ row_info->color_type = PNG_COLOR_TYPE_GRAY_ALPHA;
+ row_info->channels = 2;
+ row_info->pixel_depth = (png_byte)(row_info->bit_depth << 1);
+ row_info->rowbytes = PNG_ROWBYTES(row_info->pixel_depth,
+ row_width);
+ }
+ }
+ else if (row_info->color_type == PNG_COLOR_TYPE_RGB && trans_value)
+ {
+ if (row_info->bit_depth == 8)
+ {
+ png_byte red = trans_value->red & 0xff;
+ png_byte green = trans_value->green & 0xff;
+ png_byte blue = trans_value->blue & 0xff;
+ sp = row + (png_size_t)row_info->rowbytes - 1;
+ dp = row + (png_size_t)(row_width << 2) - 1;
+ for (i = 0; i < row_width; i++)
+ {
+ if (*(sp - 2) == red && *(sp - 1) == green && *(sp) == blue)
+ *dp-- = 0;
+ else
+ *dp-- = 0xff;
+ *dp-- = *sp--;
+ *dp-- = *sp--;
+ *dp-- = *sp--;
+ }
+ }
+ else if (row_info->bit_depth == 16)
+ {
+ png_byte red_high = (trans_value->red >> 8) & 0xff;
+ png_byte green_high = (trans_value->green >> 8) & 0xff;
+ png_byte blue_high = (trans_value->blue >> 8) & 0xff;
+ png_byte red_low = trans_value->red & 0xff;
+ png_byte green_low = trans_value->green & 0xff;
+ png_byte blue_low = trans_value->blue & 0xff;
+ sp = row + row_info->rowbytes - 1;
+ dp = row + (png_size_t)(row_width << 3) - 1;
+ for (i = 0; i < row_width; i++)
+ {
+ if (*(sp - 5) == red_high &&
+ *(sp - 4) == red_low &&
+ *(sp - 3) == green_high &&
+ *(sp - 2) == green_low &&
+ *(sp - 1) == blue_high &&
+ *(sp ) == blue_low)
+ {
+ *dp-- = 0;
+ *dp-- = 0;
+ }
+ else
+ {
+ *dp-- = 0xff;
+ *dp-- = 0xff;
+ }
+ *dp-- = *sp--;
+ *dp-- = *sp--;
+ *dp-- = *sp--;
+ *dp-- = *sp--;
+ *dp-- = *sp--;
+ *dp-- = *sp--;
+ }
+ }
+ row_info->color_type = PNG_COLOR_TYPE_RGB_ALPHA;
+ row_info->channels = 4;
+ row_info->pixel_depth = (png_byte)(row_info->bit_depth << 2);
+ row_info->rowbytes = PNG_ROWBYTES(row_info->pixel_depth, row_width);
+ }
+ }
+}
+#endif
+
+#ifdef PNG_READ_DITHER_SUPPORTED
+void /* PRIVATE */
+png_do_dither(png_row_infop row_info, png_bytep row,
+ png_bytep palette_lookup, png_bytep dither_lookup)
+{
+ png_bytep sp, dp;
+ png_uint_32 i;
+ png_uint_32 row_width=row_info->width;
+
+ png_debug(1, "in png_do_dither");
+
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ if (row != NULL && row_info != NULL)
+#endif
+ {
+ if (row_info->color_type == PNG_COLOR_TYPE_RGB &&
+ palette_lookup && row_info->bit_depth == 8)
+ {
+ int r, g, b, p;
+ sp = row;
+ dp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ r = *sp++;
+ g = *sp++;
+ b = *sp++;
+
+ /* This looks real messy, but the compiler will reduce
+ * it down to a reasonable formula. For example, with
+ * 5 bits per color, we get:
+ * p = (((r >> 3) & 0x1f) << 10) |
+ * (((g >> 3) & 0x1f) << 5) |
+ * ((b >> 3) & 0x1f);
+ */
+ p = (((r >> (8 - PNG_DITHER_RED_BITS)) &
+ ((1 << PNG_DITHER_RED_BITS) - 1)) <<
+ (PNG_DITHER_GREEN_BITS + PNG_DITHER_BLUE_BITS)) |
+ (((g >> (8 - PNG_DITHER_GREEN_BITS)) &
+ ((1 << PNG_DITHER_GREEN_BITS) - 1)) <<
+ (PNG_DITHER_BLUE_BITS)) |
+ ((b >> (8 - PNG_DITHER_BLUE_BITS)) &
+ ((1 << PNG_DITHER_BLUE_BITS) - 1));
+
+ *dp++ = palette_lookup[p];
+ }
+ row_info->color_type = PNG_COLOR_TYPE_PALETTE;
+ row_info->channels = 1;
+ row_info->pixel_depth = row_info->bit_depth;
+ row_info->rowbytes = PNG_ROWBYTES(row_info->pixel_depth, row_width);
+ }
+ else if (row_info->color_type == PNG_COLOR_TYPE_RGB_ALPHA &&
+ palette_lookup != NULL && row_info->bit_depth == 8)
+ {
+ int r, g, b, p;
+ sp = row;
+ dp = row;
+ for (i = 0; i < row_width; i++)
+ {
+ r = *sp++;
+ g = *sp++;
+ b = *sp++;
+ sp++;
+
+ p = (((r >> (8 - PNG_DITHER_RED_BITS)) &
+ ((1 << PNG_DITHER_RED_BITS) - 1)) <<
+ (PNG_DITHER_GREEN_BITS + PNG_DITHER_BLUE_BITS)) |
+ (((g >> (8 - PNG_DITHER_GREEN_BITS)) &
+ ((1 << PNG_DITHER_GREEN_BITS) - 1)) <<
+ (PNG_DITHER_BLUE_BITS)) |
+ ((b >> (8 - PNG_DITHER_BLUE_BITS)) &
+ ((1 << PNG_DITHER_BLUE_BITS) - 1));
+
+ *dp++ = palette_lookup[p];
+ }
+ row_info->color_type = PNG_COLOR_TYPE_PALETTE;
+ row_info->channels = 1;
+ row_info->pixel_depth = row_info->bit_depth;
+ row_info->rowbytes = PNG_ROWBYTES(row_info->pixel_depth, row_width);
+ }
+ else if (row_info->color_type == PNG_COLOR_TYPE_PALETTE &&
+ dither_lookup && row_info->bit_depth == 8)
+ {
+ sp = row;
+ for (i = 0; i < row_width; i++, sp++)
+ {
+ *sp = dither_lookup[*sp];
+ }
+ }
+ }
+}
+#endif
+
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+#ifdef PNG_READ_GAMMA_SUPPORTED
+static PNG_CONST int png_gamma_shift[] =
+ {0x10, 0x21, 0x42, 0x84, 0x110, 0x248, 0x550, 0xff0, 0x00};
+
+/* We build the 8- or 16-bit gamma tables here. Note that for 16-bit
+ * tables, we don't make a full table if we are reducing to 8-bit in
+ * the future. Note also how the gamma_16 tables are segmented so that
+ * we don't need to allocate > 64K chunks for a full 16-bit table.
+ *
+ * See the PNG extensions document for an integer algorithm for creating
+ * the gamma tables. Maybe we will implement that here someday.
+ *
+ * We should only reach this point if
+ *
+ * the file_gamma is known (i.e., the gAMA or sRGB chunk is present,
+ * or the application has provided a file_gamma)
+ *
+ * AND
+ * {
+ * the screen_gamma is known
+ * OR
+ *
+ * RGB_to_gray transformation is being performed
+ * }
+ *
+ * AND
+ * {
+ * the screen_gamma is different from the reciprocal of the
+ * file_gamma by more than the specified threshold
+ *
+ * OR
+ *
+ * a background color has been specified and the file_gamma
+ * and screen_gamma are not 1.0, within the specified threshold.
+ * }
+ */
+
+void /* PRIVATE */
+png_build_gamma_table(png_structp png_ptr)
+{
+ png_debug(1, "in png_build_gamma_table");
+
+ if (png_ptr->bit_depth <= 8)
+ {
+ int i;
+ double g;
+
+ if (png_ptr->screen_gamma > .000001)
+ g = 1.0 / (png_ptr->gamma * png_ptr->screen_gamma);
+
+ else
+ g = 1.0;
+
+ png_ptr->gamma_table = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)256);
+
+ for (i = 0; i < 256; i++)
+ {
+ png_ptr->gamma_table[i] = (png_byte)(pow((double)i / 255.0,
+ g) * 255.0 + .5);
+ }
+
+#if defined(PNG_READ_BACKGROUND_SUPPORTED) || \
+ defined(PNG_READ_RGB_TO_GRAY_SUPPORTED)
+ if (png_ptr->transformations & ((PNG_BACKGROUND) | PNG_RGB_TO_GRAY))
+ {
+
+ g = 1.0 / (png_ptr->gamma);
+
+ png_ptr->gamma_to_1 = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)256);
+
+ for (i = 0; i < 256; i++)
+ {
+ png_ptr->gamma_to_1[i] = (png_byte)(pow((double)i / 255.0,
+ g) * 255.0 + .5);
+ }
+
+
+ png_ptr->gamma_from_1 = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)256);
+
+ if (png_ptr->screen_gamma > 0.000001)
+ g = 1.0 / png_ptr->screen_gamma;
+
+ else
+ g = png_ptr->gamma; /* Probably doing rgb_to_gray */
+
+ for (i = 0; i < 256; i++)
+ {
+ png_ptr->gamma_from_1[i] = (png_byte)(pow((double)i / 255.0,
+ g) * 255.0 + .5);
+
+ }
+ }
+#endif /* PNG_READ_BACKGROUND_SUPPORTED || PNG_RGB_TO_GRAY_SUPPORTED */
+ }
+ else
+ {
+ double g;
+ int i, j, shift, num;
+ int sig_bit;
+ png_uint_32 ig;
+
+ if (png_ptr->color_type & PNG_COLOR_MASK_COLOR)
+ {
+ sig_bit = (int)png_ptr->sig_bit.red;
+
+ if ((int)png_ptr->sig_bit.green > sig_bit)
+ sig_bit = png_ptr->sig_bit.green;
+
+ if ((int)png_ptr->sig_bit.blue > sig_bit)
+ sig_bit = png_ptr->sig_bit.blue;
+ }
+ else
+ {
+ sig_bit = (int)png_ptr->sig_bit.gray;
+ }
+
+ if (sig_bit > 0)
+ shift = 16 - sig_bit;
+
+ else
+ shift = 0;
+
+ if (png_ptr->transformations & PNG_16_TO_8)
+ {
+ if (shift < (16 - PNG_MAX_GAMMA_8))
+ shift = (16 - PNG_MAX_GAMMA_8);
+ }
+
+ if (shift > 8)
+ shift = 8;
+
+ if (shift < 0)
+ shift = 0;
+
+ png_ptr->gamma_shift = (png_byte)shift;
+
+ num = (1 << (8 - shift));
+
+ if (png_ptr->screen_gamma > .000001)
+ g = 1.0 / (png_ptr->gamma * png_ptr->screen_gamma);
+ else
+ g = 1.0;
+
+ png_ptr->gamma_16_table = (png_uint_16pp)png_calloc(png_ptr,
+ (png_uint_32)(num * png_sizeof(png_uint_16p)));
+
+ if (png_ptr->transformations & (PNG_16_TO_8 | PNG_BACKGROUND))
+ {
+ double fin, fout;
+ png_uint_32 last, max;
+
+ for (i = 0; i < num; i++)
+ {
+ png_ptr->gamma_16_table[i] = (png_uint_16p)png_malloc(png_ptr,
+ (png_uint_32)(256 * png_sizeof(png_uint_16)));
+ }
+
+ g = 1.0 / g;
+ last = 0;
+ for (i = 0; i < 256; i++)
+ {
+ fout = ((double)i + 0.5) / 256.0;
+ fin = pow(fout, g);
+ max = (png_uint_32)(fin * (double)((png_uint_32)num << 8));
+ while (last <= max)
+ {
+ png_ptr->gamma_16_table[(int)(last & (0xff >> shift))]
+ [(int)(last >> (8 - shift))] = (png_uint_16)(
+ (png_uint_16)i | ((png_uint_16)i << 8));
+ last++;
+ }
+ }
+ while (last < ((png_uint_32)num << 8))
+ {
+ png_ptr->gamma_16_table[(int)(last & (0xff >> shift))]
+ [(int)(last >> (8 - shift))] = (png_uint_16)65535L;
+ last++;
+ }
+ }
+ else
+ {
+ for (i = 0; i < num; i++)
+ {
+ png_ptr->gamma_16_table[i] = (png_uint_16p)png_malloc(png_ptr,
+ (png_uint_32)(256 * png_sizeof(png_uint_16)));
+
+ ig = (((png_uint_32)i * (png_uint_32)png_gamma_shift[shift]) >> 4);
+
+ for (j = 0; j < 256; j++)
+ {
+ png_ptr->gamma_16_table[i][j] =
+ (png_uint_16)(pow((double)(ig + ((png_uint_32)j << 8)) /
+ 65535.0, g) * 65535.0 + .5);
+ }
+ }
+ }
+
+#if defined(PNG_READ_BACKGROUND_SUPPORTED) || \
+ defined(PNG_READ_RGB_TO_GRAY_SUPPORTED)
+ if (png_ptr->transformations & (PNG_BACKGROUND | PNG_RGB_TO_GRAY))
+ {
+
+ g = 1.0 / (png_ptr->gamma);
+
+ png_ptr->gamma_16_to_1 = (png_uint_16pp)png_calloc(png_ptr,
+ (png_uint_32)(num * png_sizeof(png_uint_16p )));
+
+ for (i = 0; i < num; i++)
+ {
+ png_ptr->gamma_16_to_1[i] = (png_uint_16p)png_malloc(png_ptr,
+ (png_uint_32)(256 * png_sizeof(png_uint_16)));
+
+ ig = (((png_uint_32)i *
+ (png_uint_32)png_gamma_shift[shift]) >> 4);
+ for (j = 0; j < 256; j++)
+ {
+ png_ptr->gamma_16_to_1[i][j] =
+ (png_uint_16)(pow((double)(ig + ((png_uint_32)j << 8)) /
+ 65535.0, g) * 65535.0 + .5);
+ }
+ }
+
+ if (png_ptr->screen_gamma > 0.000001)
+ g = 1.0 / png_ptr->screen_gamma;
+
+ else
+ g = png_ptr->gamma; /* Probably doing rgb_to_gray */
+
+ png_ptr->gamma_16_from_1 = (png_uint_16pp)png_calloc(png_ptr,
+ (png_uint_32)(num * png_sizeof(png_uint_16p)));
+
+ for (i = 0; i < num; i++)
+ {
+ png_ptr->gamma_16_from_1[i] = (png_uint_16p)png_malloc(png_ptr,
+ (png_uint_32)(256 * png_sizeof(png_uint_16)));
+
+ ig = (((png_uint_32)i *
+ (png_uint_32)png_gamma_shift[shift]) >> 4);
+
+ for (j = 0; j < 256; j++)
+ {
+ png_ptr->gamma_16_from_1[i][j] =
+ (png_uint_16)(pow((double)(ig + ((png_uint_32)j << 8)) /
+ 65535.0, g) * 65535.0 + .5);
+ }
+ }
+ }
+#endif /* PNG_READ_BACKGROUND_SUPPORTED || PNG_RGB_TO_GRAY_SUPPORTED */
+ }
+}
+#endif
+/* To do: install integer version of png_build_gamma_table here */
+#endif
+
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+/* Undoes intrapixel differencing */
+void /* PRIVATE */
+png_do_read_intrapixel(png_row_infop row_info, png_bytep row)
+{
+ png_debug(1, "in png_do_read_intrapixel");
+
+ if (
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ row != NULL && row_info != NULL &&
+#endif
+ (row_info->color_type & PNG_COLOR_MASK_COLOR))
+ {
+ int bytes_per_pixel;
+ png_uint_32 row_width = row_info->width;
+ if (row_info->bit_depth == 8)
+ {
+ png_bytep rp;
+ png_uint_32 i;
+
+ if (row_info->color_type == PNG_COLOR_TYPE_RGB)
+ bytes_per_pixel = 3;
+
+ else if (row_info->color_type == PNG_COLOR_TYPE_RGB_ALPHA)
+ bytes_per_pixel = 4;
+
+ else
+ return;
+
+ for (i = 0, rp = row; i < row_width; i++, rp += bytes_per_pixel)
+ {
+ *(rp) = (png_byte)((256 + *rp + *(rp+1))&0xff);
+ *(rp+2) = (png_byte)((256 + *(rp+2) + *(rp+1))&0xff);
+ }
+ }
+ else if (row_info->bit_depth == 16)
+ {
+ png_bytep rp;
+ png_uint_32 i;
+
+ if (row_info->color_type == PNG_COLOR_TYPE_RGB)
+ bytes_per_pixel = 6;
+
+ else if (row_info->color_type == PNG_COLOR_TYPE_RGB_ALPHA)
+ bytes_per_pixel = 8;
+
+ else
+ return;
+
+ for (i = 0, rp = row; i < row_width; i++, rp += bytes_per_pixel)
+ {
+ png_uint_32 s0 = (*(rp ) << 8) | *(rp + 1);
+ png_uint_32 s1 = (*(rp + 2) << 8) | *(rp + 3);
+ png_uint_32 s2 = (*(rp + 4) << 8) | *(rp + 5);
+ png_uint_32 red = (png_uint_32)((s0 + s1 + 65536L) & 0xffffL);
+ png_uint_32 blue = (png_uint_32)((s2 + s1 + 65536L) & 0xffffL);
+ *(rp ) = (png_byte)((red >> 8) & 0xff);
+ *(rp+1) = (png_byte)(red & 0xff);
+ *(rp+4) = (png_byte)((blue >> 8) & 0xff);
+ *(rp+5) = (png_byte)(blue & 0xff);
+ }
+ }
+ }
+}
+#endif /* PNG_MNG_FEATURES_SUPPORTED */
+#endif /* PNG_READ_SUPPORTED */
diff --git a/trunk/src/third_party/libpng/pngrutil.c b/trunk/src/third_party/libpng/pngrutil.c
new file mode 100644
index 0000000..1e2db31
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngrutil.c
@@ -0,0 +1,3382 @@
+
+/* pngrutil.c - utilities to read a PNG file
+ *
+ * Last changed in libpng 1.2.44 [June 26, 2010]
+ * Copyright (c) 1998-2010 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ *
+ * This file contains routines that are only called from within
+ * libpng itself during the course of reading an image.
+ */
+
+#define PNG_INTERNAL
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+#ifdef PNG_READ_SUPPORTED
+
+#if defined(_WIN32_WCE) && (_WIN32_WCE<0x500)
+# define WIN32_WCE_OLD
+#endif
+
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+# ifdef WIN32_WCE_OLD
+/* The strtod() function is not supported on WindowsCE */
+__inline double png_strtod(png_structp png_ptr, PNG_CONST char *nptr,
+ char **endptr)
+{
+ double result = 0;
+ int len;
+ wchar_t *str, *end;
+
+ len = MultiByteToWideChar(CP_ACP, 0, nptr, -1, NULL, 0);
+ str = (wchar_t *)png_malloc(png_ptr, len * png_sizeof(wchar_t));
+ if ( NULL != str )
+ {
+ MultiByteToWideChar(CP_ACP, 0, nptr, -1, str, len);
+ result = wcstod(str, &end);
+ len = WideCharToMultiByte(CP_ACP, 0, end, -1, NULL, 0, NULL, NULL);
+ *endptr = (char *)nptr + (png_strlen(nptr) - len + 1);
+ png_free(png_ptr, str);
+ }
+ return result;
+}
+# else
+# define png_strtod(p,a,b) strtod(a,b)
+# endif
+#endif
+
+png_uint_32 PNGAPI
+png_get_uint_31(png_structp png_ptr, png_bytep buf)
+{
+#ifdef PNG_READ_BIG_ENDIAN_SUPPORTED
+ png_uint_32 i = png_get_uint_32(buf);
+#else
+ /* Avoid an extra function call by inlining the result. */
+ png_uint_32 i = ((png_uint_32)(*buf) << 24) +
+ ((png_uint_32)(*(buf + 1)) << 16) +
+ ((png_uint_32)(*(buf + 2)) << 8) +
+ (png_uint_32)(*(buf + 3));
+#endif
+ if (i > PNG_UINT_31_MAX)
+ png_error(png_ptr, "PNG unsigned integer out of range.");
+ return (i);
+}
+#ifndef PNG_READ_BIG_ENDIAN_SUPPORTED
+/* Grab an unsigned 32-bit integer from a buffer in big-endian format. */
+png_uint_32 PNGAPI
+png_get_uint_32(png_bytep buf)
+{
+ png_uint_32 i = ((png_uint_32)(*buf) << 24) +
+ ((png_uint_32)(*(buf + 1)) << 16) +
+ ((png_uint_32)(*(buf + 2)) << 8) +
+ (png_uint_32)(*(buf + 3));
+
+ return (i);
+}
+
+/* Grab a signed 32-bit integer from a buffer in big-endian format. The
+ * data is stored in the PNG file in two's complement format, and it is
+ * assumed that the machine format for signed integers is the same.
+ */
+png_int_32 PNGAPI
+png_get_int_32(png_bytep buf)
+{
+ png_int_32 i = ((png_int_32)(*buf) << 24) +
+ ((png_int_32)(*(buf + 1)) << 16) +
+ ((png_int_32)(*(buf + 2)) << 8) +
+ (png_int_32)(*(buf + 3));
+
+ return (i);
+}
+
+/* Grab an unsigned 16-bit integer from a buffer in big-endian format. */
+png_uint_16 PNGAPI
+png_get_uint_16(png_bytep buf)
+{
+ png_uint_16 i = (png_uint_16)(((png_uint_16)(*buf) << 8) +
+ (png_uint_16)(*(buf + 1)));
+
+ return (i);
+}
+#endif /* PNG_READ_BIG_ENDIAN_SUPPORTED */
+
+/* Read the chunk header (length + type name).
+ * Put the type name into png_ptr->chunk_name, and return the length.
+ */
+png_uint_32 /* PRIVATE */
+png_read_chunk_header(png_structp png_ptr)
+{
+ png_byte buf[8];
+ png_uint_32 length;
+
+ /* Read the length and the chunk name */
+ png_read_data(png_ptr, buf, 8);
+ length = png_get_uint_31(png_ptr, buf);
+
+ /* Put the chunk name into png_ptr->chunk_name */
+ png_memcpy(png_ptr->chunk_name, buf + 4, 4);
+
+ png_debug2(0, "Reading %s chunk, length = %lu",
+ png_ptr->chunk_name, length);
+
+ /* Reset the crc and run it over the chunk name */
+ png_reset_crc(png_ptr);
+ png_calculate_crc(png_ptr, png_ptr->chunk_name, 4);
+
+ /* Check to see if chunk name is valid */
+ png_check_chunk_name(png_ptr, png_ptr->chunk_name);
+
+ return length;
+}
+
+/* Read data, and (optionally) run it through the CRC. */
+void /* PRIVATE */
+png_crc_read(png_structp png_ptr, png_bytep buf, png_size_t length)
+{
+ if (png_ptr == NULL)
+ return;
+ png_read_data(png_ptr, buf, length);
+ png_calculate_crc(png_ptr, buf, length);
+}
+
+/* Optionally skip data and then check the CRC. Depending on whether we
+ * are reading a ancillary or critical chunk, and how the program has set
+ * things up, we may calculate the CRC on the data and print a message.
+ * Returns '1' if there was a CRC error, '0' otherwise.
+ */
+int /* PRIVATE */
+png_crc_finish(png_structp png_ptr, png_uint_32 skip)
+{
+ png_size_t i;
+ png_size_t istop = png_ptr->zbuf_size;
+
+ for (i = (png_size_t)skip; i > istop; i -= istop)
+ {
+ png_crc_read(png_ptr, png_ptr->zbuf, png_ptr->zbuf_size);
+ }
+ if (i)
+ {
+ png_crc_read(png_ptr, png_ptr->zbuf, i);
+ }
+
+ if (png_crc_error(png_ptr))
+ {
+ if (((png_ptr->chunk_name[0] & 0x20) && /* Ancillary */
+ !(png_ptr->flags & PNG_FLAG_CRC_ANCILLARY_NOWARN)) ||
+ (!(png_ptr->chunk_name[0] & 0x20) && /* Critical */
+ (png_ptr->flags & PNG_FLAG_CRC_CRITICAL_USE)))
+ {
+ png_chunk_warning(png_ptr, "CRC error");
+ }
+ else
+ {
+ png_chunk_error(png_ptr, "CRC error");
+ }
+ return (1);
+ }
+
+ return (0);
+}
+
+/* Compare the CRC stored in the PNG file with that calculated by libpng from
+ * the data it has read thus far.
+ */
+int /* PRIVATE */
+png_crc_error(png_structp png_ptr)
+{
+ png_byte crc_bytes[4];
+ png_uint_32 crc;
+ int need_crc = 1;
+
+ if (png_ptr->chunk_name[0] & 0x20) /* ancillary */
+ {
+ if ((png_ptr->flags & PNG_FLAG_CRC_ANCILLARY_MASK) ==
+ (PNG_FLAG_CRC_ANCILLARY_USE | PNG_FLAG_CRC_ANCILLARY_NOWARN))
+ need_crc = 0;
+ }
+ else /* critical */
+ {
+ if (png_ptr->flags & PNG_FLAG_CRC_CRITICAL_IGNORE)
+ need_crc = 0;
+ }
+
+ png_read_data(png_ptr, crc_bytes, 4);
+
+ if (need_crc)
+ {
+ crc = png_get_uint_32(crc_bytes);
+ return ((int)(crc != png_ptr->crc));
+ }
+ else
+ return (0);
+}
+
+#if defined(PNG_READ_zTXt_SUPPORTED) || defined(PNG_READ_iTXt_SUPPORTED) || \
+ defined(PNG_READ_iCCP_SUPPORTED)
+static png_size_t
+png_inflate(png_structp png_ptr, const png_byte *data, png_size_t size,
+ png_bytep output, png_size_t output_size)
+{
+ png_size_t count = 0;
+
+ png_ptr->zstream.next_in = (png_bytep)data; /* const_cast: VALID */
+ png_ptr->zstream.avail_in = size;
+
+ while (1)
+ {
+ int ret, avail;
+
+ /* Reset the output buffer each time round - we empty it
+ * after every inflate call.
+ */
+ png_ptr->zstream.next_out = png_ptr->zbuf;
+ png_ptr->zstream.avail_out = png_ptr->zbuf_size;
+
+ ret = inflate(&png_ptr->zstream, Z_NO_FLUSH);
+ avail = png_ptr->zbuf_size - png_ptr->zstream.avail_out;
+
+ /* First copy/count any new output - but only if we didn't
+ * get an error code.
+ */
+ if ((ret == Z_OK || ret == Z_STREAM_END) && avail > 0)
+ {
+ if (output != 0 && output_size > count)
+ {
+ int copy = output_size - count;
+ if (avail < copy) copy = avail;
+ png_memcpy(output + count, png_ptr->zbuf, copy);
+ }
+ count += avail;
+ }
+
+ if (ret == Z_OK)
+ continue;
+
+ /* Termination conditions - always reset the zstream, it
+ * must be left in inflateInit state.
+ */
+ png_ptr->zstream.avail_in = 0;
+ inflateReset(&png_ptr->zstream);
+
+ if (ret == Z_STREAM_END)
+ return count; /* NOTE: may be zero. */
+
+ /* Now handle the error codes - the API always returns 0
+ * and the error message is dumped into the uncompressed
+ * buffer if available.
+ */
+ {
+ PNG_CONST char *msg;
+ if (png_ptr->zstream.msg != 0)
+ msg = png_ptr->zstream.msg;
+ else
+ {
+#if defined(PNG_STDIO_SUPPORTED) && !defined(_WIN32_WCE)
+ char umsg[52];
+
+ switch (ret)
+ {
+ case Z_BUF_ERROR:
+ msg = "Buffer error in compressed datastream in %s chunk";
+ break;
+ case Z_DATA_ERROR:
+ msg = "Data error in compressed datastream in %s chunk";
+ break;
+ default:
+ msg = "Incomplete compressed datastream in %s chunk";
+ break;
+ }
+
+ png_snprintf(umsg, sizeof umsg, msg, png_ptr->chunk_name);
+ msg = umsg;
+#else
+ msg = "Damaged compressed datastream in chunk other than IDAT";
+#endif
+ }
+
+ png_warning(png_ptr, msg);
+ }
+
+ /* 0 means an error - notice that this code simple ignores
+ * zero length compressed chunks as a result.
+ */
+ return 0;
+ }
+}
+
+/*
+ * Decompress trailing data in a chunk. The assumption is that chunkdata
+ * points at an allocated area holding the contents of a chunk with a
+ * trailing compressed part. What we get back is an allocated area
+ * holding the original prefix part and an uncompressed version of the
+ * trailing part (the malloc area passed in is freed).
+ */
+void /* PRIVATE */
+png_decompress_chunk(png_structp png_ptr, int comp_type,
+ png_size_t chunklength,
+ png_size_t prefix_size, png_size_t *newlength)
+{
+ /* The caller should guarantee this */
+ if (prefix_size > chunklength)
+ {
+ /* The recovery is to delete the chunk. */
+ png_warning(png_ptr, "invalid chunklength");
+ prefix_size = 0; /* To delete everything */
+ }
+
+ else if (comp_type == PNG_COMPRESSION_TYPE_BASE)
+ {
+ png_size_t expanded_size = png_inflate(png_ptr,
+ (png_bytep)(png_ptr->chunkdata + prefix_size),
+ chunklength - prefix_size,
+ 0/*output*/, 0/*output size*/);
+
+ /* Now check the limits on this chunk - if the limit fails the
+ * compressed data will be removed, the prefix will remain.
+ */
+#ifdef PNG_SET_CHUNK_MALLOC_LIMIT_SUPPORTED
+ if (png_ptr->user_chunk_malloc_max &&
+ (prefix_size + expanded_size >= png_ptr->user_chunk_malloc_max - 1))
+#else
+# ifdef PNG_USER_CHUNK_MALLOC_MAX
+ if ((PNG_USER_CHUNK_MALLOC_MAX > 0) &&
+ prefix_size + expanded_size >= PNG_USER_CHUNK_MALLOC_MAX - 1)
+# endif
+#endif
+ png_warning(png_ptr, "Exceeded size limit while expanding chunk");
+
+ /* If the size is zero either there was an error and a message
+ * has already been output (warning) or the size really is zero
+ * and we have nothing to do - the code will exit through the
+ * error case below.
+ */
+#if defined(PNG_SET_CHUNK_MALLOC_LIMIT_SUPPORTED) || \
+ defined(PNG_USER_CHUNK_MALLOC_MAX)
+ else
+#endif
+ if (expanded_size > 0)
+ {
+ /* Success (maybe) - really uncompress the chunk. */
+ png_size_t new_size = 0;
+ png_charp text = png_malloc_warn(png_ptr,
+ prefix_size + expanded_size + 1);
+
+ if (text != NULL)
+ {
+ png_memcpy(text, png_ptr->chunkdata, prefix_size);
+ new_size = png_inflate(png_ptr,
+ (png_bytep)(png_ptr->chunkdata + prefix_size),
+ chunklength - prefix_size,
+ (png_bytep)(text + prefix_size), expanded_size);
+ text[prefix_size + expanded_size] = 0; /* just in case */
+
+ if (new_size == expanded_size)
+ {
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = text;
+ *newlength = prefix_size + expanded_size;
+ return; /* The success return! */
+ }
+
+ png_warning(png_ptr, "png_inflate logic error");
+ png_free(png_ptr, text);
+ }
+ else
+ png_warning(png_ptr, "Not enough memory to decompress chunk.");
+ }
+ }
+
+ else /* if (comp_type != PNG_COMPRESSION_TYPE_BASE) */
+ {
+#if defined(PNG_STDIO_SUPPORTED) && !defined(_WIN32_WCE)
+ char umsg[50];
+
+ png_snprintf(umsg, sizeof umsg, "Unknown zTXt compression type %d",
+ comp_type);
+ png_warning(png_ptr, umsg);
+#else
+ png_warning(png_ptr, "Unknown zTXt compression type");
+#endif
+
+ /* The recovery is to simply drop the data. */
+ }
+
+ /* Generic error return - leave the prefix, delete the compressed
+ * data, reallocate the chunkdata to remove the potentially large
+ * amount of compressed data.
+ */
+ {
+ png_charp text = png_malloc_warn(png_ptr, prefix_size + 1);
+ if (text != NULL)
+ {
+ if (prefix_size > 0)
+ png_memcpy(text, png_ptr->chunkdata, prefix_size);
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = text;
+
+ /* This is an extra zero in the 'uncompressed' part. */
+ *(png_ptr->chunkdata + prefix_size) = 0x00;
+ }
+ /* Ignore a malloc error here - it is safe. */
+ }
+
+ *newlength = prefix_size;
+}
+#endif
+
+/* Read and check the IDHR chunk */
+void /* PRIVATE */
+png_handle_IHDR(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_byte buf[13];
+ png_uint_32 width, height;
+ int bit_depth, color_type, compression_type, filter_type;
+ int interlace_type;
+
+ png_debug(1, "in png_handle_IHDR");
+
+ if (png_ptr->mode & PNG_HAVE_IHDR)
+ png_error(png_ptr, "Out of place IHDR");
+
+ /* Check the length */
+ if (length != 13)
+ png_error(png_ptr, "Invalid IHDR chunk");
+
+ png_ptr->mode |= PNG_HAVE_IHDR;
+
+ png_crc_read(png_ptr, buf, 13);
+ png_crc_finish(png_ptr, 0);
+
+ width = png_get_uint_31(png_ptr, buf);
+ height = png_get_uint_31(png_ptr, buf + 4);
+ bit_depth = buf[8];
+ color_type = buf[9];
+ compression_type = buf[10];
+ filter_type = buf[11];
+ interlace_type = buf[12];
+
+ /* Set internal variables */
+ png_ptr->width = width;
+ png_ptr->height = height;
+ png_ptr->bit_depth = (png_byte)bit_depth;
+ png_ptr->interlaced = (png_byte)interlace_type;
+ png_ptr->color_type = (png_byte)color_type;
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+ png_ptr->filter_type = (png_byte)filter_type;
+#endif
+ png_ptr->compression_type = (png_byte)compression_type;
+
+ /* Find number of channels */
+ switch (png_ptr->color_type)
+ {
+ case PNG_COLOR_TYPE_GRAY:
+ case PNG_COLOR_TYPE_PALETTE:
+ png_ptr->channels = 1;
+ break;
+
+ case PNG_COLOR_TYPE_RGB:
+ png_ptr->channels = 3;
+ break;
+
+ case PNG_COLOR_TYPE_GRAY_ALPHA:
+ png_ptr->channels = 2;
+ break;
+
+ case PNG_COLOR_TYPE_RGB_ALPHA:
+ png_ptr->channels = 4;
+ break;
+ }
+
+ /* Set up other useful info */
+ png_ptr->pixel_depth = (png_byte)(png_ptr->bit_depth *
+ png_ptr->channels);
+ png_ptr->rowbytes = PNG_ROWBYTES(png_ptr->pixel_depth, png_ptr->width);
+ png_debug1(3, "bit_depth = %d", png_ptr->bit_depth);
+ png_debug1(3, "channels = %d", png_ptr->channels);
+ png_debug1(3, "rowbytes = %lu", png_ptr->rowbytes);
+ png_set_IHDR(png_ptr, info_ptr, width, height, bit_depth,
+ color_type, interlace_type, compression_type, filter_type);
+}
+
+/* Read and check the palette */
+void /* PRIVATE */
+png_handle_PLTE(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_color palette[PNG_MAX_PALETTE_LENGTH];
+ int num, i;
+#ifdef PNG_POINTER_INDEXING_SUPPORTED
+ png_colorp pal_ptr;
+#endif
+
+ png_debug(1, "in png_handle_PLTE");
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before PLTE");
+
+ else if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ png_warning(png_ptr, "Invalid PLTE after IDAT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ else if (png_ptr->mode & PNG_HAVE_PLTE)
+ png_error(png_ptr, "Duplicate PLTE chunk");
+
+ png_ptr->mode |= PNG_HAVE_PLTE;
+
+ if (!(png_ptr->color_type&PNG_COLOR_MASK_COLOR))
+ {
+ png_warning(png_ptr,
+ "Ignoring PLTE chunk in grayscale PNG");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+#ifndef PNG_READ_OPT_PLTE_SUPPORTED
+ if (png_ptr->color_type != PNG_COLOR_TYPE_PALETTE)
+ {
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+#endif
+
+ if (length > 3*PNG_MAX_PALETTE_LENGTH || length % 3)
+ {
+ if (png_ptr->color_type != PNG_COLOR_TYPE_PALETTE)
+ {
+ png_warning(png_ptr, "Invalid palette chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ else
+ {
+ png_error(png_ptr, "Invalid palette chunk");
+ }
+ }
+
+ num = (int)length / 3;
+
+#ifdef PNG_POINTER_INDEXING_SUPPORTED
+ for (i = 0, pal_ptr = palette; i < num; i++, pal_ptr++)
+ {
+ png_byte buf[3];
+
+ png_crc_read(png_ptr, buf, 3);
+ pal_ptr->red = buf[0];
+ pal_ptr->green = buf[1];
+ pal_ptr->blue = buf[2];
+ }
+#else
+ for (i = 0; i < num; i++)
+ {
+ png_byte buf[3];
+
+ png_crc_read(png_ptr, buf, 3);
+ /* Don't depend upon png_color being any order */
+ palette[i].red = buf[0];
+ palette[i].green = buf[1];
+ palette[i].blue = buf[2];
+ }
+#endif
+
+ /* If we actually NEED the PLTE chunk (ie for a paletted image), we do
+ * whatever the normal CRC configuration tells us. However, if we
+ * have an RGB image, the PLTE can be considered ancillary, so
+ * we will act as though it is.
+ */
+#ifndef PNG_READ_OPT_PLTE_SUPPORTED
+ if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+#endif
+ {
+ png_crc_finish(png_ptr, 0);
+ }
+#ifndef PNG_READ_OPT_PLTE_SUPPORTED
+ else if (png_crc_error(png_ptr)) /* Only if we have a CRC error */
+ {
+ /* If we don't want to use the data from an ancillary chunk,
+ we have two options: an error abort, or a warning and we
+ ignore the data in this chunk (which should be OK, since
+ it's considered ancillary for a RGB or RGBA image). */
+ if (!(png_ptr->flags & PNG_FLAG_CRC_ANCILLARY_USE))
+ {
+ if (png_ptr->flags & PNG_FLAG_CRC_ANCILLARY_NOWARN)
+ {
+ png_chunk_error(png_ptr, "CRC error");
+ }
+ else
+ {
+ png_chunk_warning(png_ptr, "CRC error");
+ return;
+ }
+ }
+ /* Otherwise, we (optionally) emit a warning and use the chunk. */
+ else if (!(png_ptr->flags & PNG_FLAG_CRC_ANCILLARY_NOWARN))
+ {
+ png_chunk_warning(png_ptr, "CRC error");
+ }
+ }
+#endif
+
+ png_set_PLTE(png_ptr, info_ptr, palette, num);
+
+#ifdef PNG_READ_tRNS_SUPPORTED
+ if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_tRNS))
+ {
+ if (png_ptr->num_trans > (png_uint_16)num)
+ {
+ png_warning(png_ptr, "Truncating incorrect tRNS chunk length");
+ png_ptr->num_trans = (png_uint_16)num;
+ }
+ if (info_ptr->num_trans > (png_uint_16)num)
+ {
+ png_warning(png_ptr, "Truncating incorrect info tRNS chunk length");
+ info_ptr->num_trans = (png_uint_16)num;
+ }
+ }
+ }
+#endif
+
+}
+
+void /* PRIVATE */
+png_handle_IEND(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_debug(1, "in png_handle_IEND");
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR) || !(png_ptr->mode & PNG_HAVE_IDAT))
+ {
+ png_error(png_ptr, "No image in file");
+ }
+
+ png_ptr->mode |= (PNG_AFTER_IDAT | PNG_HAVE_IEND);
+
+ if (length != 0)
+ {
+ png_warning(png_ptr, "Incorrect IEND chunk length");
+ }
+ png_crc_finish(png_ptr, length);
+
+ info_ptr = info_ptr; /* Quiet compiler warnings about unused info_ptr */
+}
+
+#ifdef PNG_READ_gAMA_SUPPORTED
+void /* PRIVATE */
+png_handle_gAMA(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_fixed_point igamma;
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ float file_gamma;
+#endif
+ png_byte buf[4];
+
+ png_debug(1, "in png_handle_gAMA");
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before gAMA");
+ else if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ png_warning(png_ptr, "Invalid gAMA after IDAT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ else if (png_ptr->mode & PNG_HAVE_PLTE)
+ /* Should be an error, but we can cope with it */
+ png_warning(png_ptr, "Out of place gAMA chunk");
+
+ if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_gAMA)
+#ifdef PNG_READ_sRGB_SUPPORTED
+ && !(info_ptr->valid & PNG_INFO_sRGB)
+#endif
+ )
+ {
+ png_warning(png_ptr, "Duplicate gAMA chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ if (length != 4)
+ {
+ png_warning(png_ptr, "Incorrect gAMA chunk length");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ png_crc_read(png_ptr, buf, 4);
+ if (png_crc_finish(png_ptr, 0))
+ return;
+
+ igamma = (png_fixed_point)png_get_uint_32(buf);
+ /* Check for zero gamma */
+ if (igamma == 0)
+ {
+ png_warning(png_ptr,
+ "Ignoring gAMA chunk with gamma=0");
+ return;
+ }
+
+#ifdef PNG_READ_sRGB_SUPPORTED
+ if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_sRGB))
+ if (PNG_OUT_OF_RANGE(igamma, 45500L, 500))
+ {
+ png_warning(png_ptr,
+ "Ignoring incorrect gAMA value when sRGB is also present");
+#ifdef PNG_CONSOLE_IO_SUPPORTED
+ fprintf(stderr, "gamma = (%d/100000)", (int)igamma);
+#endif
+ return;
+ }
+#endif /* PNG_READ_sRGB_SUPPORTED */
+
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ file_gamma = (float)igamma / (float)100000.0;
+# ifdef PNG_READ_GAMMA_SUPPORTED
+ png_ptr->gamma = file_gamma;
+# endif
+ png_set_gAMA(png_ptr, info_ptr, file_gamma);
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ png_set_gAMA_fixed(png_ptr, info_ptr, igamma);
+#endif
+}
+#endif
+
+#ifdef PNG_READ_sBIT_SUPPORTED
+void /* PRIVATE */
+png_handle_sBIT(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_size_t truelen;
+ png_byte buf[4];
+
+ png_debug(1, "in png_handle_sBIT");
+
+ buf[0] = buf[1] = buf[2] = buf[3] = 0;
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before sBIT");
+ else if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ png_warning(png_ptr, "Invalid sBIT after IDAT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ else if (png_ptr->mode & PNG_HAVE_PLTE)
+ {
+ /* Should be an error, but we can cope with it */
+ png_warning(png_ptr, "Out of place sBIT chunk");
+ }
+ if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_sBIT))
+ {
+ png_warning(png_ptr, "Duplicate sBIT chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ truelen = 3;
+ else
+ truelen = (png_size_t)png_ptr->channels;
+
+ if (length != truelen || length > 4)
+ {
+ png_warning(png_ptr, "Incorrect sBIT chunk length");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ png_crc_read(png_ptr, buf, truelen);
+ if (png_crc_finish(png_ptr, 0))
+ return;
+
+ if (png_ptr->color_type & PNG_COLOR_MASK_COLOR)
+ {
+ png_ptr->sig_bit.red = buf[0];
+ png_ptr->sig_bit.green = buf[1];
+ png_ptr->sig_bit.blue = buf[2];
+ png_ptr->sig_bit.alpha = buf[3];
+ }
+ else
+ {
+ png_ptr->sig_bit.gray = buf[0];
+ png_ptr->sig_bit.red = buf[0];
+ png_ptr->sig_bit.green = buf[0];
+ png_ptr->sig_bit.blue = buf[0];
+ png_ptr->sig_bit.alpha = buf[1];
+ }
+ png_set_sBIT(png_ptr, info_ptr, &(png_ptr->sig_bit));
+}
+#endif
+
+#ifdef PNG_READ_cHRM_SUPPORTED
+void /* PRIVATE */
+png_handle_cHRM(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_byte buf[32];
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ float white_x, white_y, red_x, red_y, green_x, green_y, blue_x, blue_y;
+#endif
+ png_fixed_point int_x_white, int_y_white, int_x_red, int_y_red, int_x_green,
+ int_y_green, int_x_blue, int_y_blue;
+
+ png_uint_32 uint_x, uint_y;
+
+ png_debug(1, "in png_handle_cHRM");
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before cHRM");
+ else if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ png_warning(png_ptr, "Invalid cHRM after IDAT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ else if (png_ptr->mode & PNG_HAVE_PLTE)
+ /* Should be an error, but we can cope with it */
+ png_warning(png_ptr, "Missing PLTE before cHRM");
+
+ if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_cHRM)
+#ifdef PNG_READ_sRGB_SUPPORTED
+ && !(info_ptr->valid & PNG_INFO_sRGB)
+#endif
+ )
+ {
+ png_warning(png_ptr, "Duplicate cHRM chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ if (length != 32)
+ {
+ png_warning(png_ptr, "Incorrect cHRM chunk length");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ png_crc_read(png_ptr, buf, 32);
+ if (png_crc_finish(png_ptr, 0))
+ return;
+
+ uint_x = png_get_uint_32(buf);
+ uint_y = png_get_uint_32(buf + 4);
+ int_x_white = (png_fixed_point)uint_x;
+ int_y_white = (png_fixed_point)uint_y;
+
+ uint_x = png_get_uint_32(buf + 8);
+ uint_y = png_get_uint_32(buf + 12);
+ int_x_red = (png_fixed_point)uint_x;
+ int_y_red = (png_fixed_point)uint_y;
+
+ uint_x = png_get_uint_32(buf + 16);
+ uint_y = png_get_uint_32(buf + 20);
+ int_x_green = (png_fixed_point)uint_x;
+ int_y_green = (png_fixed_point)uint_y;
+
+ uint_x = png_get_uint_32(buf + 24);
+ uint_y = png_get_uint_32(buf + 28);
+ int_x_blue = (png_fixed_point)uint_x;
+ int_y_blue = (png_fixed_point)uint_y;
+
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ white_x = (float)int_x_white / (float)100000.0;
+ white_y = (float)int_y_white / (float)100000.0;
+ red_x = (float)int_x_red / (float)100000.0;
+ red_y = (float)int_y_red / (float)100000.0;
+ green_x = (float)int_x_green / (float)100000.0;
+ green_y = (float)int_y_green / (float)100000.0;
+ blue_x = (float)int_x_blue / (float)100000.0;
+ blue_y = (float)int_y_blue / (float)100000.0;
+#endif
+
+#ifdef PNG_READ_sRGB_SUPPORTED
+ if ((info_ptr != NULL) && (info_ptr->valid & PNG_INFO_sRGB))
+ {
+ if (PNG_OUT_OF_RANGE(int_x_white, 31270, 1000) ||
+ PNG_OUT_OF_RANGE(int_y_white, 32900, 1000) ||
+ PNG_OUT_OF_RANGE(int_x_red, 64000L, 1000) ||
+ PNG_OUT_OF_RANGE(int_y_red, 33000, 1000) ||
+ PNG_OUT_OF_RANGE(int_x_green, 30000, 1000) ||
+ PNG_OUT_OF_RANGE(int_y_green, 60000L, 1000) ||
+ PNG_OUT_OF_RANGE(int_x_blue, 15000, 1000) ||
+ PNG_OUT_OF_RANGE(int_y_blue, 6000, 1000))
+ {
+ png_warning(png_ptr,
+ "Ignoring incorrect cHRM value when sRGB is also present");
+#ifdef PNG_CONSOLE_IO_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ fprintf(stderr, "wx=%f, wy=%f, rx=%f, ry=%f\n",
+ white_x, white_y, red_x, red_y);
+ fprintf(stderr, "gx=%f, gy=%f, bx=%f, by=%f\n",
+ green_x, green_y, blue_x, blue_y);
+#else
+ fprintf(stderr, "wx=%ld, wy=%ld, rx=%ld, ry=%ld\n",
+ (long)int_x_white, (long)int_y_white,
+ (long)int_x_red, (long)int_y_red);
+ fprintf(stderr, "gx=%ld, gy=%ld, bx=%ld, by=%ld\n",
+ (long)int_x_green, (long)int_y_green,
+ (long)int_x_blue, (long)int_y_blue);
+#endif
+#endif /* PNG_CONSOLE_IO_SUPPORTED */
+ }
+ return;
+ }
+#endif /* PNG_READ_sRGB_SUPPORTED */
+
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ png_set_cHRM(png_ptr, info_ptr,
+ white_x, white_y, red_x, red_y, green_x, green_y, blue_x, blue_y);
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ png_set_cHRM_fixed(png_ptr, info_ptr,
+ int_x_white, int_y_white, int_x_red, int_y_red, int_x_green,
+ int_y_green, int_x_blue, int_y_blue);
+#endif
+}
+#endif
+
+#ifdef PNG_READ_sRGB_SUPPORTED
+void /* PRIVATE */
+png_handle_sRGB(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ int intent;
+ png_byte buf[1];
+
+ png_debug(1, "in png_handle_sRGB");
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before sRGB");
+ else if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ png_warning(png_ptr, "Invalid sRGB after IDAT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ else if (png_ptr->mode & PNG_HAVE_PLTE)
+ /* Should be an error, but we can cope with it */
+ png_warning(png_ptr, "Out of place sRGB chunk");
+
+ if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_sRGB))
+ {
+ png_warning(png_ptr, "Duplicate sRGB chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ if (length != 1)
+ {
+ png_warning(png_ptr, "Incorrect sRGB chunk length");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ png_crc_read(png_ptr, buf, 1);
+ if (png_crc_finish(png_ptr, 0))
+ return;
+
+ intent = buf[0];
+ /* Check for bad intent */
+ if (intent >= PNG_sRGB_INTENT_LAST)
+ {
+ png_warning(png_ptr, "Unknown sRGB intent");
+ return;
+ }
+
+#if defined(PNG_READ_gAMA_SUPPORTED) && defined(PNG_READ_GAMMA_SUPPORTED)
+ if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_gAMA))
+ {
+ png_fixed_point igamma;
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ igamma=info_ptr->int_gamma;
+#else
+# ifdef PNG_FLOATING_POINT_SUPPORTED
+ igamma=(png_fixed_point)(info_ptr->gamma * 100000.);
+# endif
+#endif
+ if (PNG_OUT_OF_RANGE(igamma, 45500L, 500))
+ {
+ png_warning(png_ptr,
+ "Ignoring incorrect gAMA value when sRGB is also present");
+#ifdef PNG_CONSOLE_IO_SUPPORTED
+# ifdef PNG_FIXED_POINT_SUPPORTED
+ fprintf(stderr, "incorrect gamma=(%d/100000)\n",
+ (int)png_ptr->int_gamma);
+# else
+# ifdef PNG_FLOATING_POINT_SUPPORTED
+ fprintf(stderr, "incorrect gamma=%f\n", png_ptr->gamma);
+# endif
+# endif
+#endif
+ }
+ }
+#endif /* PNG_READ_gAMA_SUPPORTED */
+
+#ifdef PNG_READ_cHRM_SUPPORTED
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_cHRM))
+ if (PNG_OUT_OF_RANGE(info_ptr->int_x_white, 31270, 1000) ||
+ PNG_OUT_OF_RANGE(info_ptr->int_y_white, 32900, 1000) ||
+ PNG_OUT_OF_RANGE(info_ptr->int_x_red, 64000L, 1000) ||
+ PNG_OUT_OF_RANGE(info_ptr->int_y_red, 33000, 1000) ||
+ PNG_OUT_OF_RANGE(info_ptr->int_x_green, 30000, 1000) ||
+ PNG_OUT_OF_RANGE(info_ptr->int_y_green, 60000L, 1000) ||
+ PNG_OUT_OF_RANGE(info_ptr->int_x_blue, 15000, 1000) ||
+ PNG_OUT_OF_RANGE(info_ptr->int_y_blue, 6000, 1000))
+ {
+ png_warning(png_ptr,
+ "Ignoring incorrect cHRM value when sRGB is also present");
+ }
+#endif /* PNG_FIXED_POINT_SUPPORTED */
+#endif /* PNG_READ_cHRM_SUPPORTED */
+
+ png_set_sRGB_gAMA_and_cHRM(png_ptr, info_ptr, intent);
+}
+#endif /* PNG_READ_sRGB_SUPPORTED */
+
+#ifdef PNG_READ_iCCP_SUPPORTED
+void /* PRIVATE */
+png_handle_iCCP(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+/* Note: this does not properly handle chunks that are > 64K under DOS */
+{
+ png_byte compression_type;
+ png_bytep pC;
+ png_charp profile;
+ png_uint_32 skip = 0;
+ png_uint_32 profile_size, profile_length;
+ png_size_t slength, prefix_length, data_length;
+
+ png_debug(1, "in png_handle_iCCP");
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before iCCP");
+ else if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ png_warning(png_ptr, "Invalid iCCP after IDAT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ else if (png_ptr->mode & PNG_HAVE_PLTE)
+ /* Should be an error, but we can cope with it */
+ png_warning(png_ptr, "Out of place iCCP chunk");
+
+ if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_iCCP))
+ {
+ png_warning(png_ptr, "Duplicate iCCP chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+#ifdef PNG_MAX_MALLOC_64K
+ if (length > (png_uint_32)65535L)
+ {
+ png_warning(png_ptr, "iCCP chunk too large to fit in memory");
+ skip = length - (png_uint_32)65535L;
+ length = (png_uint_32)65535L;
+ }
+#endif
+
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = (png_charp)png_malloc(png_ptr, length + 1);
+ slength = (png_size_t)length;
+ png_crc_read(png_ptr, (png_bytep)png_ptr->chunkdata, slength);
+
+ if (png_crc_finish(png_ptr, skip))
+ {
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+
+ png_ptr->chunkdata[slength] = 0x00;
+
+ for (profile = png_ptr->chunkdata; *profile; profile++)
+ /* Empty loop to find end of name */ ;
+
+ ++profile;
+
+ /* There should be at least one zero (the compression type byte)
+ * following the separator, and we should be on it
+ */
+ if ( profile >= png_ptr->chunkdata + slength - 1)
+ {
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ png_warning(png_ptr, "Malformed iCCP chunk");
+ return;
+ }
+
+ /* Compression_type should always be zero */
+ compression_type = *profile++;
+ if (compression_type)
+ {
+ png_warning(png_ptr, "Ignoring nonzero compression type in iCCP chunk");
+ compression_type = 0x00; /* Reset it to zero (libpng-1.0.6 through 1.0.8
+ wrote nonzero) */
+ }
+
+ prefix_length = profile - png_ptr->chunkdata;
+ png_decompress_chunk(png_ptr, compression_type,
+ slength, prefix_length, &data_length);
+
+ profile_length = data_length - prefix_length;
+
+ if ( prefix_length > data_length || profile_length < 4)
+ {
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ png_warning(png_ptr, "Profile size field missing from iCCP chunk");
+ return;
+ }
+
+ /* Check the profile_size recorded in the first 32 bits of the ICC profile */
+ pC = (png_bytep)(png_ptr->chunkdata + prefix_length);
+ profile_size = ((*(pC ))<<24) |
+ ((*(pC + 1))<<16) |
+ ((*(pC + 2))<< 8) |
+ ((*(pC + 3)) );
+
+ if (profile_size < profile_length)
+ profile_length = profile_size;
+
+ if (profile_size > profile_length)
+ {
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ png_warning(png_ptr, "Ignoring truncated iCCP profile.");
+ return;
+ }
+
+ png_set_iCCP(png_ptr, info_ptr, png_ptr->chunkdata,
+ compression_type, png_ptr->chunkdata + prefix_length, profile_length);
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+}
+#endif /* PNG_READ_iCCP_SUPPORTED */
+
+#ifdef PNG_READ_sPLT_SUPPORTED
+void /* PRIVATE */
+png_handle_sPLT(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+/* Note: this does not properly handle chunks that are > 64K under DOS */
+{
+ png_bytep entry_start;
+ png_sPLT_t new_palette;
+#ifdef PNG_POINTER_INDEXING_SUPPORTED
+ png_sPLT_entryp pp;
+#endif
+ int data_length, entry_size, i;
+ png_uint_32 skip = 0;
+ png_size_t slength;
+
+ png_debug(1, "in png_handle_sPLT");
+
+#ifdef PNG_USER_LIMITS_SUPPORTED
+
+ if (png_ptr->user_chunk_cache_max != 0)
+ {
+ if (png_ptr->user_chunk_cache_max == 1)
+ {
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ if (--png_ptr->user_chunk_cache_max == 1)
+ {
+ png_warning(png_ptr, "No space in chunk cache for sPLT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ }
+#endif
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before sPLT");
+ else if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ png_warning(png_ptr, "Invalid sPLT after IDAT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+#ifdef PNG_MAX_MALLOC_64K
+ if (length > (png_uint_32)65535L)
+ {
+ png_warning(png_ptr, "sPLT chunk too large to fit in memory");
+ skip = length - (png_uint_32)65535L;
+ length = (png_uint_32)65535L;
+ }
+#endif
+
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = (png_charp)png_malloc(png_ptr, length + 1);
+ slength = (png_size_t)length;
+ png_crc_read(png_ptr, (png_bytep)png_ptr->chunkdata, slength);
+
+ if (png_crc_finish(png_ptr, skip))
+ {
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+
+ png_ptr->chunkdata[slength] = 0x00;
+
+ for (entry_start = (png_bytep)png_ptr->chunkdata; *entry_start;
+ entry_start++)
+ /* Empty loop to find end of name */ ;
+ ++entry_start;
+
+ /* A sample depth should follow the separator, and we should be on it */
+ if (entry_start > (png_bytep)png_ptr->chunkdata + slength - 2)
+ {
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ png_warning(png_ptr, "malformed sPLT chunk");
+ return;
+ }
+
+ new_palette.depth = *entry_start++;
+ entry_size = (new_palette.depth == 8 ? 6 : 10);
+ data_length = (slength - (entry_start - (png_bytep)png_ptr->chunkdata));
+
+ /* Integrity-check the data length */
+ if (data_length % entry_size)
+ {
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ png_warning(png_ptr, "sPLT chunk has bad length");
+ return;
+ }
+
+ new_palette.nentries = (png_int_32) ( data_length / entry_size);
+ if ((png_uint_32) new_palette.nentries >
+ (png_uint_32) (PNG_SIZE_MAX / png_sizeof(png_sPLT_entry)))
+ {
+ png_warning(png_ptr, "sPLT chunk too long");
+ return;
+ }
+ new_palette.entries = (png_sPLT_entryp)png_malloc_warn(
+ png_ptr, new_palette.nentries * png_sizeof(png_sPLT_entry));
+ if (new_palette.entries == NULL)
+ {
+ png_warning(png_ptr, "sPLT chunk requires too much memory");
+ return;
+ }
+
+#ifdef PNG_POINTER_INDEXING_SUPPORTED
+ for (i = 0; i < new_palette.nentries; i++)
+ {
+ pp = new_palette.entries + i;
+
+ if (new_palette.depth == 8)
+ {
+ pp->red = *entry_start++;
+ pp->green = *entry_start++;
+ pp->blue = *entry_start++;
+ pp->alpha = *entry_start++;
+ }
+ else
+ {
+ pp->red = png_get_uint_16(entry_start); entry_start += 2;
+ pp->green = png_get_uint_16(entry_start); entry_start += 2;
+ pp->blue = png_get_uint_16(entry_start); entry_start += 2;
+ pp->alpha = png_get_uint_16(entry_start); entry_start += 2;
+ }
+ pp->frequency = png_get_uint_16(entry_start); entry_start += 2;
+ }
+#else
+ pp = new_palette.entries;
+ for (i = 0; i < new_palette.nentries; i++)
+ {
+
+ if (new_palette.depth == 8)
+ {
+ pp[i].red = *entry_start++;
+ pp[i].green = *entry_start++;
+ pp[i].blue = *entry_start++;
+ pp[i].alpha = *entry_start++;
+ }
+ else
+ {
+ pp[i].red = png_get_uint_16(entry_start); entry_start += 2;
+ pp[i].green = png_get_uint_16(entry_start); entry_start += 2;
+ pp[i].blue = png_get_uint_16(entry_start); entry_start += 2;
+ pp[i].alpha = png_get_uint_16(entry_start); entry_start += 2;
+ }
+ pp->frequency = png_get_uint_16(entry_start); entry_start += 2;
+ }
+#endif
+
+ /* Discard all chunk data except the name and stash that */
+ new_palette.name = png_ptr->chunkdata;
+
+ png_set_sPLT(png_ptr, info_ptr, &new_palette, 1);
+
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ png_free(png_ptr, new_palette.entries);
+}
+#endif /* PNG_READ_sPLT_SUPPORTED */
+
+#ifdef PNG_READ_tRNS_SUPPORTED
+void /* PRIVATE */
+png_handle_tRNS(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_byte readbuf[PNG_MAX_PALETTE_LENGTH];
+
+ png_debug(1, "in png_handle_tRNS");
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before tRNS");
+ else if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ png_warning(png_ptr, "Invalid tRNS after IDAT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ else if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_tRNS))
+ {
+ png_warning(png_ptr, "Duplicate tRNS chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ if (png_ptr->color_type == PNG_COLOR_TYPE_GRAY)
+ {
+ png_byte buf[2];
+
+ if (length != 2)
+ {
+ png_warning(png_ptr, "Incorrect tRNS chunk length");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ png_crc_read(png_ptr, buf, 2);
+ png_ptr->num_trans = 1;
+ png_ptr->trans_values.gray = png_get_uint_16(buf);
+ }
+ else if (png_ptr->color_type == PNG_COLOR_TYPE_RGB)
+ {
+ png_byte buf[6];
+
+ if (length != 6)
+ {
+ png_warning(png_ptr, "Incorrect tRNS chunk length");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ png_crc_read(png_ptr, buf, (png_size_t)length);
+ png_ptr->num_trans = 1;
+ png_ptr->trans_values.red = png_get_uint_16(buf);
+ png_ptr->trans_values.green = png_get_uint_16(buf + 2);
+ png_ptr->trans_values.blue = png_get_uint_16(buf + 4);
+ }
+ else if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ if (!(png_ptr->mode & PNG_HAVE_PLTE))
+ {
+ /* Should be an error, but we can cope with it. */
+ png_warning(png_ptr, "Missing PLTE before tRNS");
+ }
+ if (length > (png_uint_32)png_ptr->num_palette ||
+ length > PNG_MAX_PALETTE_LENGTH)
+ {
+ png_warning(png_ptr, "Incorrect tRNS chunk length");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ if (length == 0)
+ {
+ png_warning(png_ptr, "Zero length tRNS chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ png_crc_read(png_ptr, readbuf, (png_size_t)length);
+ png_ptr->num_trans = (png_uint_16)length;
+ }
+ else
+ {
+ png_warning(png_ptr, "tRNS chunk not allowed with alpha channel");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ if (png_crc_finish(png_ptr, 0))
+ {
+ png_ptr->num_trans = 0;
+ return;
+ }
+
+ png_set_tRNS(png_ptr, info_ptr, readbuf, png_ptr->num_trans,
+ &(png_ptr->trans_values));
+}
+#endif
+
+#ifdef PNG_READ_bKGD_SUPPORTED
+void /* PRIVATE */
+png_handle_bKGD(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_size_t truelen;
+ png_byte buf[6];
+
+ png_debug(1, "in png_handle_bKGD");
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before bKGD");
+ else if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ png_warning(png_ptr, "Invalid bKGD after IDAT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ else if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE &&
+ !(png_ptr->mode & PNG_HAVE_PLTE))
+ {
+ png_warning(png_ptr, "Missing PLTE before bKGD");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ else if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_bKGD))
+ {
+ png_warning(png_ptr, "Duplicate bKGD chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ truelen = 1;
+ else if (png_ptr->color_type & PNG_COLOR_MASK_COLOR)
+ truelen = 6;
+ else
+ truelen = 2;
+
+ if (length != truelen)
+ {
+ png_warning(png_ptr, "Incorrect bKGD chunk length");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ png_crc_read(png_ptr, buf, truelen);
+ if (png_crc_finish(png_ptr, 0))
+ return;
+
+ /* We convert the index value into RGB components so that we can allow
+ * arbitrary RGB values for background when we have transparency, and
+ * so it is easy to determine the RGB values of the background color
+ * from the info_ptr struct. */
+ if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ png_ptr->background.index = buf[0];
+ if (info_ptr && info_ptr->num_palette)
+ {
+ if (buf[0] >= info_ptr->num_palette)
+ {
+ png_warning(png_ptr, "Incorrect bKGD chunk index value");
+ return;
+ }
+ png_ptr->background.red =
+ (png_uint_16)png_ptr->palette[buf[0]].red;
+ png_ptr->background.green =
+ (png_uint_16)png_ptr->palette[buf[0]].green;
+ png_ptr->background.blue =
+ (png_uint_16)png_ptr->palette[buf[0]].blue;
+ }
+ }
+ else if (!(png_ptr->color_type & PNG_COLOR_MASK_COLOR)) /* GRAY */
+ {
+ png_ptr->background.red =
+ png_ptr->background.green =
+ png_ptr->background.blue =
+ png_ptr->background.gray = png_get_uint_16(buf);
+ }
+ else
+ {
+ png_ptr->background.red = png_get_uint_16(buf);
+ png_ptr->background.green = png_get_uint_16(buf + 2);
+ png_ptr->background.blue = png_get_uint_16(buf + 4);
+ }
+
+ png_set_bKGD(png_ptr, info_ptr, &(png_ptr->background));
+}
+#endif
+
+#ifdef PNG_READ_hIST_SUPPORTED
+void /* PRIVATE */
+png_handle_hIST(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ unsigned int num, i;
+ png_uint_16 readbuf[PNG_MAX_PALETTE_LENGTH];
+
+ png_debug(1, "in png_handle_hIST");
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before hIST");
+ else if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ png_warning(png_ptr, "Invalid hIST after IDAT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ else if (!(png_ptr->mode & PNG_HAVE_PLTE))
+ {
+ png_warning(png_ptr, "Missing PLTE before hIST");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ else if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_hIST))
+ {
+ png_warning(png_ptr, "Duplicate hIST chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ num = length / 2 ;
+ if (num != (unsigned int) png_ptr->num_palette || num >
+ (unsigned int) PNG_MAX_PALETTE_LENGTH)
+ {
+ png_warning(png_ptr, "Incorrect hIST chunk length");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ for (i = 0; i < num; i++)
+ {
+ png_byte buf[2];
+
+ png_crc_read(png_ptr, buf, 2);
+ readbuf[i] = png_get_uint_16(buf);
+ }
+
+ if (png_crc_finish(png_ptr, 0))
+ return;
+
+ png_set_hIST(png_ptr, info_ptr, readbuf);
+}
+#endif
+
+#ifdef PNG_READ_pHYs_SUPPORTED
+void /* PRIVATE */
+png_handle_pHYs(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_byte buf[9];
+ png_uint_32 res_x, res_y;
+ int unit_type;
+
+ png_debug(1, "in png_handle_pHYs");
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before pHYs");
+ else if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ png_warning(png_ptr, "Invalid pHYs after IDAT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ else if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_pHYs))
+ {
+ png_warning(png_ptr, "Duplicate pHYs chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ if (length != 9)
+ {
+ png_warning(png_ptr, "Incorrect pHYs chunk length");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ png_crc_read(png_ptr, buf, 9);
+ if (png_crc_finish(png_ptr, 0))
+ return;
+
+ res_x = png_get_uint_32(buf);
+ res_y = png_get_uint_32(buf + 4);
+ unit_type = buf[8];
+ png_set_pHYs(png_ptr, info_ptr, res_x, res_y, unit_type);
+}
+#endif
+
+#ifdef PNG_READ_oFFs_SUPPORTED
+void /* PRIVATE */
+png_handle_oFFs(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_byte buf[9];
+ png_int_32 offset_x, offset_y;
+ int unit_type;
+
+ png_debug(1, "in png_handle_oFFs");
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before oFFs");
+ else if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ png_warning(png_ptr, "Invalid oFFs after IDAT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ else if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_oFFs))
+ {
+ png_warning(png_ptr, "Duplicate oFFs chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ if (length != 9)
+ {
+ png_warning(png_ptr, "Incorrect oFFs chunk length");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ png_crc_read(png_ptr, buf, 9);
+ if (png_crc_finish(png_ptr, 0))
+ return;
+
+ offset_x = png_get_int_32(buf);
+ offset_y = png_get_int_32(buf + 4);
+ unit_type = buf[8];
+ png_set_oFFs(png_ptr, info_ptr, offset_x, offset_y, unit_type);
+}
+#endif
+
+#ifdef PNG_READ_pCAL_SUPPORTED
+/* Read the pCAL chunk (described in the PNG Extensions document) */
+void /* PRIVATE */
+png_handle_pCAL(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_int_32 X0, X1;
+ png_byte type, nparams;
+ png_charp buf, units, endptr;
+ png_charpp params;
+ png_size_t slength;
+ int i;
+
+ png_debug(1, "in png_handle_pCAL");
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before pCAL");
+ else if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ png_warning(png_ptr, "Invalid pCAL after IDAT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ else if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_pCAL))
+ {
+ png_warning(png_ptr, "Duplicate pCAL chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ png_debug1(2, "Allocating and reading pCAL chunk data (%lu bytes)",
+ length + 1);
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = (png_charp)png_malloc_warn(png_ptr, length + 1);
+ if (png_ptr->chunkdata == NULL)
+ {
+ png_warning(png_ptr, "No memory for pCAL purpose.");
+ return;
+ }
+ slength = (png_size_t)length;
+ png_crc_read(png_ptr, (png_bytep)png_ptr->chunkdata, slength);
+
+ if (png_crc_finish(png_ptr, 0))
+ {
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+
+ png_ptr->chunkdata[slength] = 0x00; /* Null terminate the last string */
+
+ png_debug(3, "Finding end of pCAL purpose string");
+ for (buf = png_ptr->chunkdata; *buf; buf++)
+ /* Empty loop */ ;
+
+ endptr = png_ptr->chunkdata + slength;
+
+ /* We need to have at least 12 bytes after the purpose string
+ in order to get the parameter information. */
+ if (endptr <= buf + 12)
+ {
+ png_warning(png_ptr, "Invalid pCAL data");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+
+ png_debug(3, "Reading pCAL X0, X1, type, nparams, and units");
+ X0 = png_get_int_32((png_bytep)buf+1);
+ X1 = png_get_int_32((png_bytep)buf+5);
+ type = buf[9];
+ nparams = buf[10];
+ units = buf + 11;
+
+ png_debug(3, "Checking pCAL equation type and number of parameters");
+ /* Check that we have the right number of parameters for known
+ equation types. */
+ if ((type == PNG_EQUATION_LINEAR && nparams != 2) ||
+ (type == PNG_EQUATION_BASE_E && nparams != 3) ||
+ (type == PNG_EQUATION_ARBITRARY && nparams != 3) ||
+ (type == PNG_EQUATION_HYPERBOLIC && nparams != 4))
+ {
+ png_warning(png_ptr, "Invalid pCAL parameters for equation type");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+ else if (type >= PNG_EQUATION_LAST)
+ {
+ png_warning(png_ptr, "Unrecognized equation type for pCAL chunk");
+ }
+
+ for (buf = units; *buf; buf++)
+ /* Empty loop to move past the units string. */ ;
+
+ png_debug(3, "Allocating pCAL parameters array");
+ params = (png_charpp)png_malloc_warn(png_ptr,
+ (png_uint_32)(nparams * png_sizeof(png_charp))) ;
+ if (params == NULL)
+ {
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ png_warning(png_ptr, "No memory for pCAL params.");
+ return;
+ }
+
+ /* Get pointers to the start of each parameter string. */
+ for (i = 0; i < (int)nparams; i++)
+ {
+ buf++; /* Skip the null string terminator from previous parameter. */
+
+ png_debug1(3, "Reading pCAL parameter %d", i);
+ for (params[i] = buf; buf <= endptr && *buf != 0x00; buf++)
+ /* Empty loop to move past each parameter string */ ;
+
+ /* Make sure we haven't run out of data yet */
+ if (buf > endptr)
+ {
+ png_warning(png_ptr, "Invalid pCAL data");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ png_free(png_ptr, params);
+ return;
+ }
+ }
+
+ png_set_pCAL(png_ptr, info_ptr, png_ptr->chunkdata, X0, X1, type, nparams,
+ units, params);
+
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ png_free(png_ptr, params);
+}
+#endif
+
+#ifdef PNG_READ_sCAL_SUPPORTED
+/* Read the sCAL chunk */
+void /* PRIVATE */
+png_handle_sCAL(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_charp ep;
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ double width, height;
+ png_charp vp;
+#else
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ png_charp swidth, sheight;
+#endif
+#endif
+ png_size_t slength;
+
+ png_debug(1, "in png_handle_sCAL");
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before sCAL");
+ else if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+ png_warning(png_ptr, "Invalid sCAL after IDAT");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ else if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_sCAL))
+ {
+ png_warning(png_ptr, "Duplicate sCAL chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ png_debug1(2, "Allocating and reading sCAL chunk data (%lu bytes)",
+ length + 1);
+ png_ptr->chunkdata = (png_charp)png_malloc_warn(png_ptr, length + 1);
+ if (png_ptr->chunkdata == NULL)
+ {
+ png_warning(png_ptr, "Out of memory while processing sCAL chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ slength = (png_size_t)length;
+ png_crc_read(png_ptr, (png_bytep)png_ptr->chunkdata, slength);
+
+ if (png_crc_finish(png_ptr, 0))
+ {
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+
+ png_ptr->chunkdata[slength] = 0x00; /* Null terminate the last string */
+
+ ep = png_ptr->chunkdata + 1; /* Skip unit byte */
+
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ width = png_strtod(png_ptr, ep, &vp);
+ if (*vp)
+ {
+ png_warning(png_ptr, "malformed width string in sCAL chunk");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+#else
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ swidth = (png_charp)png_malloc_warn(png_ptr, png_strlen(ep) + 1);
+ if (swidth == NULL)
+ {
+ png_warning(png_ptr, "Out of memory while processing sCAL chunk width");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+ png_memcpy(swidth, ep, (png_size_t)png_strlen(ep));
+#endif
+#endif
+
+ for (ep = png_ptr->chunkdata; *ep; ep++)
+ /* Empty loop */ ;
+ ep++;
+
+ if (png_ptr->chunkdata + slength < ep)
+ {
+ png_warning(png_ptr, "Truncated sCAL chunk");
+#if defined(PNG_FIXED_POINT_SUPPORTED) && !defined(PNG_FLOATING_POINT_SUPPORTED)
+ png_free(png_ptr, swidth);
+#endif
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ height = png_strtod(png_ptr, ep, &vp);
+ if (*vp)
+ {
+ png_warning(png_ptr, "malformed height string in sCAL chunk");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+#if defined(PNG_FIXED_POINT_SUPPORTED) && !defined(PNG_FLOATING_POINT_SUPPORTED)
+ png_free(png_ptr, swidth);
+#endif
+ return;
+ }
+#else
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ sheight = (png_charp)png_malloc_warn(png_ptr, png_strlen(ep) + 1);
+ if (sheight == NULL)
+ {
+ png_warning(png_ptr, "Out of memory while processing sCAL chunk height");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+#if defined(PNG_FIXED_POINT_SUPPORTED) && !defined(PNG_FLOATING_POINT_SUPPORTED)
+ png_free(png_ptr, swidth);
+#endif
+ return;
+ }
+ png_memcpy(sheight, ep, (png_size_t)png_strlen(ep));
+#endif
+#endif
+
+ if (png_ptr->chunkdata + slength < ep
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ || width <= 0. || height <= 0.
+#endif
+ )
+ {
+ png_warning(png_ptr, "Invalid sCAL data");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+#if defined(PNG_FIXED_POINT_SUPPORTED) && !defined(PNG_FLOATING_POINT_SUPPORTED)
+ png_free(png_ptr, swidth);
+ png_free(png_ptr, sheight);
+#endif
+ return;
+ }
+
+
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ png_set_sCAL(png_ptr, info_ptr, png_ptr->chunkdata[0], width, height);
+#else
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ png_set_sCAL_s(png_ptr, info_ptr, png_ptr->chunkdata[0], swidth, sheight);
+#endif
+#endif
+
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+#if defined(PNG_FIXED_POINT_SUPPORTED) && !defined(PNG_FLOATING_POINT_SUPPORTED)
+ png_free(png_ptr, swidth);
+ png_free(png_ptr, sheight);
+#endif
+}
+#endif
+
+#ifdef PNG_READ_tIME_SUPPORTED
+void /* PRIVATE */
+png_handle_tIME(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_byte buf[7];
+ png_time mod_time;
+
+ png_debug(1, "in png_handle_tIME");
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Out of place tIME chunk");
+ else if (info_ptr != NULL && (info_ptr->valid & PNG_INFO_tIME))
+ {
+ png_warning(png_ptr, "Duplicate tIME chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ if (png_ptr->mode & PNG_HAVE_IDAT)
+ png_ptr->mode |= PNG_AFTER_IDAT;
+
+ if (length != 7)
+ {
+ png_warning(png_ptr, "Incorrect tIME chunk length");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+
+ png_crc_read(png_ptr, buf, 7);
+ if (png_crc_finish(png_ptr, 0))
+ return;
+
+ mod_time.second = buf[6];
+ mod_time.minute = buf[5];
+ mod_time.hour = buf[4];
+ mod_time.day = buf[3];
+ mod_time.month = buf[2];
+ mod_time.year = png_get_uint_16(buf);
+
+ png_set_tIME(png_ptr, info_ptr, &mod_time);
+}
+#endif
+
+#ifdef PNG_READ_tEXt_SUPPORTED
+/* Note: this does not properly handle chunks that are > 64K under DOS */
+void /* PRIVATE */
+png_handle_tEXt(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_textp text_ptr;
+ png_charp key;
+ png_charp text;
+ png_uint_32 skip = 0;
+ png_size_t slength;
+ int ret;
+
+ png_debug(1, "in png_handle_tEXt");
+
+#ifdef PNG_USER_LIMITS_SUPPORTED
+ if (png_ptr->user_chunk_cache_max != 0)
+ {
+ if (png_ptr->user_chunk_cache_max == 1)
+ {
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ if (--png_ptr->user_chunk_cache_max == 1)
+ {
+ png_warning(png_ptr, "No space in chunk cache for tEXt");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ }
+#endif
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before tEXt");
+
+ if (png_ptr->mode & PNG_HAVE_IDAT)
+ png_ptr->mode |= PNG_AFTER_IDAT;
+
+#ifdef PNG_MAX_MALLOC_64K
+ if (length > (png_uint_32)65535L)
+ {
+ png_warning(png_ptr, "tEXt chunk too large to fit in memory");
+ skip = length - (png_uint_32)65535L;
+ length = (png_uint_32)65535L;
+ }
+#endif
+
+ png_free(png_ptr, png_ptr->chunkdata);
+
+ png_ptr->chunkdata = (png_charp)png_malloc_warn(png_ptr, length + 1);
+ if (png_ptr->chunkdata == NULL)
+ {
+ png_warning(png_ptr, "No memory to process text chunk.");
+ return;
+ }
+ slength = (png_size_t)length;
+ png_crc_read(png_ptr, (png_bytep)png_ptr->chunkdata, slength);
+
+ if (png_crc_finish(png_ptr, skip))
+ {
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+
+ key = png_ptr->chunkdata;
+
+ key[slength] = 0x00;
+
+ for (text = key; *text; text++)
+ /* Empty loop to find end of key */ ;
+
+ if (text != key + slength)
+ text++;
+
+ text_ptr = (png_textp)png_malloc_warn(png_ptr,
+ (png_uint_32)png_sizeof(png_text));
+ if (text_ptr == NULL)
+ {
+ png_warning(png_ptr, "Not enough memory to process text chunk.");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+ text_ptr->compression = PNG_TEXT_COMPRESSION_NONE;
+ text_ptr->key = key;
+#ifdef PNG_iTXt_SUPPORTED
+ text_ptr->lang = NULL;
+ text_ptr->lang_key = NULL;
+ text_ptr->itxt_length = 0;
+#endif
+ text_ptr->text = text;
+ text_ptr->text_length = png_strlen(text);
+
+ ret = png_set_text_2(png_ptr, info_ptr, text_ptr, 1);
+
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ png_free(png_ptr, text_ptr);
+ if (ret)
+ png_warning(png_ptr, "Insufficient memory to process text chunk.");
+}
+#endif
+
+#ifdef PNG_READ_zTXt_SUPPORTED
+/* Note: this does not correctly handle chunks that are > 64K under DOS */
+void /* PRIVATE */
+png_handle_zTXt(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_textp text_ptr;
+ png_charp text;
+ int comp_type;
+ int ret;
+ png_size_t slength, prefix_len, data_len;
+
+ png_debug(1, "in png_handle_zTXt");
+
+#ifdef PNG_USER_LIMITS_SUPPORTED
+ if (png_ptr->user_chunk_cache_max != 0)
+ {
+ if (png_ptr->user_chunk_cache_max == 1)
+ {
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ if (--png_ptr->user_chunk_cache_max == 1)
+ {
+ png_warning(png_ptr, "No space in chunk cache for zTXt");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ }
+#endif
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before zTXt");
+
+ if (png_ptr->mode & PNG_HAVE_IDAT)
+ png_ptr->mode |= PNG_AFTER_IDAT;
+
+#ifdef PNG_MAX_MALLOC_64K
+ /* We will no doubt have problems with chunks even half this size, but
+ there is no hard and fast rule to tell us where to stop. */
+ if (length > (png_uint_32)65535L)
+ {
+ png_warning(png_ptr, "zTXt chunk too large to fit in memory");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+#endif
+
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = (png_charp)png_malloc_warn(png_ptr, length + 1);
+ if (png_ptr->chunkdata == NULL)
+ {
+ png_warning(png_ptr, "Out of memory processing zTXt chunk.");
+ return;
+ }
+ slength = (png_size_t)length;
+ png_crc_read(png_ptr, (png_bytep)png_ptr->chunkdata, slength);
+ if (png_crc_finish(png_ptr, 0))
+ {
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+
+ png_ptr->chunkdata[slength] = 0x00;
+
+ for (text = png_ptr->chunkdata; *text; text++)
+ /* Empty loop */ ;
+
+ /* zTXt must have some text after the chunkdataword */
+ if (text >= png_ptr->chunkdata + slength - 2)
+ {
+ png_warning(png_ptr, "Truncated zTXt chunk");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+ else
+ {
+ comp_type = *(++text);
+ if (comp_type != PNG_TEXT_COMPRESSION_zTXt)
+ {
+ png_warning(png_ptr, "Unknown compression type in zTXt chunk");
+ comp_type = PNG_TEXT_COMPRESSION_zTXt;
+ }
+ text++; /* Skip the compression_method byte */
+ }
+ prefix_len = text - png_ptr->chunkdata;
+
+ png_decompress_chunk(png_ptr, comp_type,
+ (png_size_t)length, prefix_len, &data_len);
+
+ text_ptr = (png_textp)png_malloc_warn(png_ptr,
+ (png_uint_32)png_sizeof(png_text));
+ if (text_ptr == NULL)
+ {
+ png_warning(png_ptr, "Not enough memory to process zTXt chunk.");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+ text_ptr->compression = comp_type;
+ text_ptr->key = png_ptr->chunkdata;
+#ifdef PNG_iTXt_SUPPORTED
+ text_ptr->lang = NULL;
+ text_ptr->lang_key = NULL;
+ text_ptr->itxt_length = 0;
+#endif
+ text_ptr->text = png_ptr->chunkdata + prefix_len;
+ text_ptr->text_length = data_len;
+
+ ret = png_set_text_2(png_ptr, info_ptr, text_ptr, 1);
+
+ png_free(png_ptr, text_ptr);
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ if (ret)
+ png_error(png_ptr, "Insufficient memory to store zTXt chunk.");
+}
+#endif
+
+#ifdef PNG_READ_iTXt_SUPPORTED
+/* Note: this does not correctly handle chunks that are > 64K under DOS */
+void /* PRIVATE */
+png_handle_iTXt(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_textp text_ptr;
+ png_charp key, lang, text, lang_key;
+ int comp_flag;
+ int comp_type = 0;
+ int ret;
+ png_size_t slength, prefix_len, data_len;
+
+ png_debug(1, "in png_handle_iTXt");
+
+#ifdef PNG_USER_LIMITS_SUPPORTED
+ if (png_ptr->user_chunk_cache_max != 0)
+ {
+ if (png_ptr->user_chunk_cache_max == 1)
+ {
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ if (--png_ptr->user_chunk_cache_max == 1)
+ {
+ png_warning(png_ptr, "No space in chunk cache for iTXt");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ }
+#endif
+
+ if (!(png_ptr->mode & PNG_HAVE_IHDR))
+ png_error(png_ptr, "Missing IHDR before iTXt");
+
+ if (png_ptr->mode & PNG_HAVE_IDAT)
+ png_ptr->mode |= PNG_AFTER_IDAT;
+
+#ifdef PNG_MAX_MALLOC_64K
+ /* We will no doubt have problems with chunks even half this size, but
+ there is no hard and fast rule to tell us where to stop. */
+ if (length > (png_uint_32)65535L)
+ {
+ png_warning(png_ptr, "iTXt chunk too large to fit in memory");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+#endif
+
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = (png_charp)png_malloc_warn(png_ptr, length + 1);
+ if (png_ptr->chunkdata == NULL)
+ {
+ png_warning(png_ptr, "No memory to process iTXt chunk.");
+ return;
+ }
+ slength = (png_size_t)length;
+ png_crc_read(png_ptr, (png_bytep)png_ptr->chunkdata, slength);
+ if (png_crc_finish(png_ptr, 0))
+ {
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+
+ png_ptr->chunkdata[slength] = 0x00;
+
+ for (lang = png_ptr->chunkdata; *lang; lang++)
+ /* Empty loop */ ;
+ lang++; /* Skip NUL separator */
+
+ /* iTXt must have a language tag (possibly empty), two compression bytes,
+ * translated keyword (possibly empty), and possibly some text after the
+ * keyword
+ */
+
+ if (lang >= png_ptr->chunkdata + slength - 3)
+ {
+ png_warning(png_ptr, "Truncated iTXt chunk");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+ else
+ {
+ comp_flag = *lang++;
+ comp_type = *lang++;
+ }
+
+ for (lang_key = lang; *lang_key; lang_key++)
+ /* Empty loop */ ;
+ lang_key++; /* Skip NUL separator */
+
+ if (lang_key >= png_ptr->chunkdata + slength)
+ {
+ png_warning(png_ptr, "Truncated iTXt chunk");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+
+ for (text = lang_key; *text; text++)
+ /* Empty loop */ ;
+ text++; /* Skip NUL separator */
+ if (text >= png_ptr->chunkdata + slength)
+ {
+ png_warning(png_ptr, "Malformed iTXt chunk");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+
+ prefix_len = text - png_ptr->chunkdata;
+
+ key=png_ptr->chunkdata;
+ if (comp_flag)
+ png_decompress_chunk(png_ptr, comp_type,
+ (size_t)length, prefix_len, &data_len);
+ else
+ data_len = png_strlen(png_ptr->chunkdata + prefix_len);
+ text_ptr = (png_textp)png_malloc_warn(png_ptr,
+ (png_uint_32)png_sizeof(png_text));
+ if (text_ptr == NULL)
+ {
+ png_warning(png_ptr, "Not enough memory to process iTXt chunk.");
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ return;
+ }
+ text_ptr->compression = (int)comp_flag + 1;
+ text_ptr->lang_key = png_ptr->chunkdata + (lang_key - key);
+ text_ptr->lang = png_ptr->chunkdata + (lang - key);
+ text_ptr->itxt_length = data_len;
+ text_ptr->text_length = 0;
+ text_ptr->key = png_ptr->chunkdata;
+ text_ptr->text = png_ptr->chunkdata + prefix_len;
+
+ ret = png_set_text_2(png_ptr, info_ptr, text_ptr, 1);
+
+ png_free(png_ptr, text_ptr);
+ png_free(png_ptr, png_ptr->chunkdata);
+ png_ptr->chunkdata = NULL;
+ if (ret)
+ png_error(png_ptr, "Insufficient memory to store iTXt chunk.");
+}
+#endif
+
+/* This function is called when we haven't found a handler for a
+ chunk. If there isn't a problem with the chunk itself (ie bad
+ chunk name, CRC, or a critical chunk), the chunk is silently ignored
+ -- unless the PNG_FLAG_UNKNOWN_CHUNKS_SUPPORTED flag is on in which
+ case it will be saved away to be written out later. */
+void /* PRIVATE */
+png_handle_unknown(png_structp png_ptr, png_infop info_ptr, png_uint_32 length)
+{
+ png_uint_32 skip = 0;
+
+ png_debug(1, "in png_handle_unknown");
+
+#ifdef PNG_USER_LIMITS_SUPPORTED
+ if (png_ptr->user_chunk_cache_max != 0)
+ {
+ if (png_ptr->user_chunk_cache_max == 1)
+ {
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ if (--png_ptr->user_chunk_cache_max == 1)
+ {
+ png_warning(png_ptr, "No space in chunk cache for unknown chunk");
+ png_crc_finish(png_ptr, length);
+ return;
+ }
+ }
+#endif
+
+ if (png_ptr->mode & PNG_HAVE_IDAT)
+ {
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_CONST PNG_IDAT;
+#endif
+ if (png_memcmp(png_ptr->chunk_name, png_IDAT, 4)) /* Not an IDAT */
+ png_ptr->mode |= PNG_AFTER_IDAT;
+ }
+
+ if (!(png_ptr->chunk_name[0] & 0x20))
+ {
+#ifdef PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+ if (png_handle_as_unknown(png_ptr, png_ptr->chunk_name) !=
+ PNG_HANDLE_CHUNK_ALWAYS
+#ifdef PNG_READ_USER_CHUNKS_SUPPORTED
+ && png_ptr->read_user_chunk_fn == NULL
+#endif
+ )
+#endif
+ png_chunk_error(png_ptr, "unknown critical chunk");
+ }
+
+#ifdef PNG_READ_UNKNOWN_CHUNKS_SUPPORTED
+ if ((png_ptr->flags & PNG_FLAG_KEEP_UNKNOWN_CHUNKS)
+#ifdef PNG_READ_USER_CHUNKS_SUPPORTED
+ || (png_ptr->read_user_chunk_fn != NULL)
+#endif
+ )
+ {
+#ifdef PNG_MAX_MALLOC_64K
+ if (length > (png_uint_32)65535L)
+ {
+ png_warning(png_ptr, "unknown chunk too large to fit in memory");
+ skip = length - (png_uint_32)65535L;
+ length = (png_uint_32)65535L;
+ }
+#endif
+ png_memcpy((png_charp)png_ptr->unknown_chunk.name,
+ (png_charp)png_ptr->chunk_name,
+ png_sizeof(png_ptr->unknown_chunk.name));
+ png_ptr->unknown_chunk.name[png_sizeof(png_ptr->unknown_chunk.name)-1]
+ = '\0';
+ png_ptr->unknown_chunk.size = (png_size_t)length;
+ if (length == 0)
+ png_ptr->unknown_chunk.data = NULL;
+ else
+ {
+ png_ptr->unknown_chunk.data = (png_bytep)png_malloc(png_ptr, length);
+ png_crc_read(png_ptr, (png_bytep)png_ptr->unknown_chunk.data, length);
+ }
+#ifdef PNG_READ_USER_CHUNKS_SUPPORTED
+ if (png_ptr->read_user_chunk_fn != NULL)
+ {
+ /* Callback to user unknown chunk handler */
+ int ret;
+ ret = (*(png_ptr->read_user_chunk_fn))
+ (png_ptr, &png_ptr->unknown_chunk);
+ if (ret < 0)
+ png_chunk_error(png_ptr, "error in user chunk");
+ if (ret == 0)
+ {
+ if (!(png_ptr->chunk_name[0] & 0x20))
+#ifdef PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+ if (png_handle_as_unknown(png_ptr, png_ptr->chunk_name) !=
+ PNG_HANDLE_CHUNK_ALWAYS)
+#endif
+ png_chunk_error(png_ptr, "unknown critical chunk");
+ png_set_unknown_chunks(png_ptr, info_ptr,
+ &png_ptr->unknown_chunk, 1);
+ }
+ }
+ else
+#endif
+ png_set_unknown_chunks(png_ptr, info_ptr, &png_ptr->unknown_chunk, 1);
+ png_free(png_ptr, png_ptr->unknown_chunk.data);
+ png_ptr->unknown_chunk.data = NULL;
+ }
+ else
+#endif
+ skip = length;
+
+ png_crc_finish(png_ptr, skip);
+
+#ifndef PNG_READ_USER_CHUNKS_SUPPORTED
+ info_ptr = info_ptr; /* Quiet compiler warnings about unused info_ptr */
+#endif
+}
+
+/* This function is called to verify that a chunk name is valid.
+ This function can't have the "critical chunk check" incorporated
+ into it, since in the future we will need to be able to call user
+ functions to handle unknown critical chunks after we check that
+ the chunk name itself is valid. */
+
+#define isnonalpha(c) ((c) < 65 || (c) > 122 || ((c) > 90 && (c) < 97))
+
+void /* PRIVATE */
+png_check_chunk_name(png_structp png_ptr, png_bytep chunk_name)
+{
+ png_debug(1, "in png_check_chunk_name");
+ if (isnonalpha(chunk_name[0]) || isnonalpha(chunk_name[1]) ||
+ isnonalpha(chunk_name[2]) || isnonalpha(chunk_name[3]))
+ {
+ png_chunk_error(png_ptr, "invalid chunk type");
+ }
+}
+
+/* Combines the row recently read in with the existing pixels in the
+ row. This routine takes care of alpha and transparency if requested.
+ This routine also handles the two methods of progressive display
+ of interlaced images, depending on the mask value.
+ The mask value describes which pixels are to be combined with
+ the row. The pattern always repeats every 8 pixels, so just 8
+ bits are needed. A one indicates the pixel is to be combined,
+ a zero indicates the pixel is to be skipped. This is in addition
+ to any alpha or transparency value associated with the pixel. If
+ you want all pixels to be combined, pass 0xff (255) in mask. */
+
+void /* PRIVATE */
+png_combine_row(png_structp png_ptr, png_bytep row, int mask)
+{
+ png_debug(1, "in png_combine_row");
+ if (mask == 0xff)
+ {
+ png_memcpy(row, png_ptr->row_buf + 1,
+ PNG_ROWBYTES(png_ptr->row_info.pixel_depth, png_ptr->width));
+ }
+ else
+ {
+ switch (png_ptr->row_info.pixel_depth)
+ {
+ case 1:
+ {
+ png_bytep sp = png_ptr->row_buf + 1;
+ png_bytep dp = row;
+ int s_inc, s_start, s_end;
+ int m = 0x80;
+ int shift;
+ png_uint_32 i;
+ png_uint_32 row_width = png_ptr->width;
+
+#ifdef PNG_READ_PACKSWAP_SUPPORTED
+ if (png_ptr->transformations & PNG_PACKSWAP)
+ {
+ s_start = 0;
+ s_end = 7;
+ s_inc = 1;
+ }
+ else
+#endif
+ {
+ s_start = 7;
+ s_end = 0;
+ s_inc = -1;
+ }
+
+ shift = s_start;
+
+ for (i = 0; i < row_width; i++)
+ {
+ if (m & mask)
+ {
+ int value;
+
+ value = (*sp >> shift) & 0x01;
+ *dp &= (png_byte)((0x7f7f >> (7 - shift)) & 0xff);
+ *dp |= (png_byte)(value << shift);
+ }
+
+ if (shift == s_end)
+ {
+ shift = s_start;
+ sp++;
+ dp++;
+ }
+ else
+ shift += s_inc;
+
+ if (m == 1)
+ m = 0x80;
+ else
+ m >>= 1;
+ }
+ break;
+ }
+ case 2:
+ {
+ png_bytep sp = png_ptr->row_buf + 1;
+ png_bytep dp = row;
+ int s_start, s_end, s_inc;
+ int m = 0x80;
+ int shift;
+ png_uint_32 i;
+ png_uint_32 row_width = png_ptr->width;
+ int value;
+
+#ifdef PNG_READ_PACKSWAP_SUPPORTED
+ if (png_ptr->transformations & PNG_PACKSWAP)
+ {
+ s_start = 0;
+ s_end = 6;
+ s_inc = 2;
+ }
+ else
+#endif
+ {
+ s_start = 6;
+ s_end = 0;
+ s_inc = -2;
+ }
+
+ shift = s_start;
+
+ for (i = 0; i < row_width; i++)
+ {
+ if (m & mask)
+ {
+ value = (*sp >> shift) & 0x03;
+ *dp &= (png_byte)((0x3f3f >> (6 - shift)) & 0xff);
+ *dp |= (png_byte)(value << shift);
+ }
+
+ if (shift == s_end)
+ {
+ shift = s_start;
+ sp++;
+ dp++;
+ }
+ else
+ shift += s_inc;
+ if (m == 1)
+ m = 0x80;
+ else
+ m >>= 1;
+ }
+ break;
+ }
+ case 4:
+ {
+ png_bytep sp = png_ptr->row_buf + 1;
+ png_bytep dp = row;
+ int s_start, s_end, s_inc;
+ int m = 0x80;
+ int shift;
+ png_uint_32 i;
+ png_uint_32 row_width = png_ptr->width;
+ int value;
+
+#ifdef PNG_READ_PACKSWAP_SUPPORTED
+ if (png_ptr->transformations & PNG_PACKSWAP)
+ {
+ s_start = 0;
+ s_end = 4;
+ s_inc = 4;
+ }
+ else
+#endif
+ {
+ s_start = 4;
+ s_end = 0;
+ s_inc = -4;
+ }
+ shift = s_start;
+
+ for (i = 0; i < row_width; i++)
+ {
+ if (m & mask)
+ {
+ value = (*sp >> shift) & 0xf;
+ *dp &= (png_byte)((0xf0f >> (4 - shift)) & 0xff);
+ *dp |= (png_byte)(value << shift);
+ }
+
+ if (shift == s_end)
+ {
+ shift = s_start;
+ sp++;
+ dp++;
+ }
+ else
+ shift += s_inc;
+ if (m == 1)
+ m = 0x80;
+ else
+ m >>= 1;
+ }
+ break;
+ }
+ default:
+ {
+ png_bytep sp = png_ptr->row_buf + 1;
+ png_bytep dp = row;
+ png_size_t pixel_bytes = (png_ptr->row_info.pixel_depth >> 3);
+ png_uint_32 i;
+ png_uint_32 row_width = png_ptr->width;
+ png_byte m = 0x80;
+
+
+ for (i = 0; i < row_width; i++)
+ {
+ if (m & mask)
+ {
+ png_memcpy(dp, sp, pixel_bytes);
+ }
+
+ sp += pixel_bytes;
+ dp += pixel_bytes;
+
+ if (m == 1)
+ m = 0x80;
+ else
+ m >>= 1;
+ }
+ break;
+ }
+ }
+ }
+}
+
+#ifdef PNG_READ_INTERLACING_SUPPORTED
+/* OLD pre-1.0.9 interface:
+void png_do_read_interlace(png_row_infop row_info, png_bytep row, int pass,
+ png_uint_32 transformations)
+ */
+void /* PRIVATE */
+png_do_read_interlace(png_structp png_ptr)
+{
+ png_row_infop row_info = &(png_ptr->row_info);
+ png_bytep row = png_ptr->row_buf + 1;
+ int pass = png_ptr->pass;
+ png_uint_32 transformations = png_ptr->transformations;
+ /* Arrays to facilitate easy interlacing - use pass (0 - 6) as index */
+ /* Offset to next interlace block */
+ PNG_CONST int png_pass_inc[7] = {8, 8, 4, 4, 2, 2, 1};
+
+ png_debug(1, "in png_do_read_interlace");
+ if (row != NULL && row_info != NULL)
+ {
+ png_uint_32 final_width;
+
+ final_width = row_info->width * png_pass_inc[pass];
+
+ switch (row_info->pixel_depth)
+ {
+ case 1:
+ {
+ png_bytep sp = row + (png_size_t)((row_info->width - 1) >> 3);
+ png_bytep dp = row + (png_size_t)((final_width - 1) >> 3);
+ int sshift, dshift;
+ int s_start, s_end, s_inc;
+ int jstop = png_pass_inc[pass];
+ png_byte v;
+ png_uint_32 i;
+ int j;
+
+#ifdef PNG_READ_PACKSWAP_SUPPORTED
+ if (transformations & PNG_PACKSWAP)
+ {
+ sshift = (int)((row_info->width + 7) & 0x07);
+ dshift = (int)((final_width + 7) & 0x07);
+ s_start = 7;
+ s_end = 0;
+ s_inc = -1;
+ }
+ else
+#endif
+ {
+ sshift = 7 - (int)((row_info->width + 7) & 0x07);
+ dshift = 7 - (int)((final_width + 7) & 0x07);
+ s_start = 0;
+ s_end = 7;
+ s_inc = 1;
+ }
+
+ for (i = 0; i < row_info->width; i++)
+ {
+ v = (png_byte)((*sp >> sshift) & 0x01);
+ for (j = 0; j < jstop; j++)
+ {
+ *dp &= (png_byte)((0x7f7f >> (7 - dshift)) & 0xff);
+ *dp |= (png_byte)(v << dshift);
+ if (dshift == s_end)
+ {
+ dshift = s_start;
+ dp--;
+ }
+ else
+ dshift += s_inc;
+ }
+ if (sshift == s_end)
+ {
+ sshift = s_start;
+ sp--;
+ }
+ else
+ sshift += s_inc;
+ }
+ break;
+ }
+ case 2:
+ {
+ png_bytep sp = row + (png_uint_32)((row_info->width - 1) >> 2);
+ png_bytep dp = row + (png_uint_32)((final_width - 1) >> 2);
+ int sshift, dshift;
+ int s_start, s_end, s_inc;
+ int jstop = png_pass_inc[pass];
+ png_uint_32 i;
+
+#ifdef PNG_READ_PACKSWAP_SUPPORTED
+ if (transformations & PNG_PACKSWAP)
+ {
+ sshift = (int)(((row_info->width + 3) & 0x03) << 1);
+ dshift = (int)(((final_width + 3) & 0x03) << 1);
+ s_start = 6;
+ s_end = 0;
+ s_inc = -2;
+ }
+ else
+#endif
+ {
+ sshift = (int)((3 - ((row_info->width + 3) & 0x03)) << 1);
+ dshift = (int)((3 - ((final_width + 3) & 0x03)) << 1);
+ s_start = 0;
+ s_end = 6;
+ s_inc = 2;
+ }
+
+ for (i = 0; i < row_info->width; i++)
+ {
+ png_byte v;
+ int j;
+
+ v = (png_byte)((*sp >> sshift) & 0x03);
+ for (j = 0; j < jstop; j++)
+ {
+ *dp &= (png_byte)((0x3f3f >> (6 - dshift)) & 0xff);
+ *dp |= (png_byte)(v << dshift);
+ if (dshift == s_end)
+ {
+ dshift = s_start;
+ dp--;
+ }
+ else
+ dshift += s_inc;
+ }
+ if (sshift == s_end)
+ {
+ sshift = s_start;
+ sp--;
+ }
+ else
+ sshift += s_inc;
+ }
+ break;
+ }
+ case 4:
+ {
+ png_bytep sp = row + (png_size_t)((row_info->width - 1) >> 1);
+ png_bytep dp = row + (png_size_t)((final_width - 1) >> 1);
+ int sshift, dshift;
+ int s_start, s_end, s_inc;
+ png_uint_32 i;
+ int jstop = png_pass_inc[pass];
+
+#ifdef PNG_READ_PACKSWAP_SUPPORTED
+ if (transformations & PNG_PACKSWAP)
+ {
+ sshift = (int)(((row_info->width + 1) & 0x01) << 2);
+ dshift = (int)(((final_width + 1) & 0x01) << 2);
+ s_start = 4;
+ s_end = 0;
+ s_inc = -4;
+ }
+ else
+#endif
+ {
+ sshift = (int)((1 - ((row_info->width + 1) & 0x01)) << 2);
+ dshift = (int)((1 - ((final_width + 1) & 0x01)) << 2);
+ s_start = 0;
+ s_end = 4;
+ s_inc = 4;
+ }
+
+ for (i = 0; i < row_info->width; i++)
+ {
+ png_byte v = (png_byte)((*sp >> sshift) & 0xf);
+ int j;
+
+ for (j = 0; j < jstop; j++)
+ {
+ *dp &= (png_byte)((0xf0f >> (4 - dshift)) & 0xff);
+ *dp |= (png_byte)(v << dshift);
+ if (dshift == s_end)
+ {
+ dshift = s_start;
+ dp--;
+ }
+ else
+ dshift += s_inc;
+ }
+ if (sshift == s_end)
+ {
+ sshift = s_start;
+ sp--;
+ }
+ else
+ sshift += s_inc;
+ }
+ break;
+ }
+ default:
+ {
+ png_size_t pixel_bytes = (row_info->pixel_depth >> 3);
+ png_bytep sp = row + (png_size_t)(row_info->width - 1)
+ * pixel_bytes;
+ png_bytep dp = row + (png_size_t)(final_width - 1) * pixel_bytes;
+
+ int jstop = png_pass_inc[pass];
+ png_uint_32 i;
+
+ for (i = 0; i < row_info->width; i++)
+ {
+ png_byte v[8];
+ int j;
+
+ png_memcpy(v, sp, pixel_bytes);
+ for (j = 0; j < jstop; j++)
+ {
+ png_memcpy(dp, v, pixel_bytes);
+ dp -= pixel_bytes;
+ }
+ sp -= pixel_bytes;
+ }
+ break;
+ }
+ }
+ row_info->width = final_width;
+ row_info->rowbytes = PNG_ROWBYTES(row_info->pixel_depth, final_width);
+ }
+#ifndef PNG_READ_PACKSWAP_SUPPORTED
+ transformations = transformations; /* Silence compiler warning */
+#endif
+}
+#endif /* PNG_READ_INTERLACING_SUPPORTED */
+
+void /* PRIVATE */
+png_read_filter_row(png_structp png_ptr, png_row_infop row_info, png_bytep row,
+ png_bytep prev_row, int filter)
+{
+ png_debug(1, "in png_read_filter_row");
+ png_debug2(2, "row = %lu, filter = %d", png_ptr->row_number, filter);
+ switch (filter)
+ {
+ case PNG_FILTER_VALUE_NONE:
+ break;
+ case PNG_FILTER_VALUE_SUB:
+ {
+ png_uint_32 i;
+ png_uint_32 istop = row_info->rowbytes;
+ png_uint_32 bpp = (row_info->pixel_depth + 7) >> 3;
+ png_bytep rp = row + bpp;
+ png_bytep lp = row;
+
+ for (i = bpp; i < istop; i++)
+ {
+ *rp = (png_byte)(((int)(*rp) + (int)(*lp++)) & 0xff);
+ rp++;
+ }
+ break;
+ }
+ case PNG_FILTER_VALUE_UP:
+ {
+ png_uint_32 i;
+ png_uint_32 istop = row_info->rowbytes;
+ png_bytep rp = row;
+ png_bytep pp = prev_row;
+
+ for (i = 0; i < istop; i++)
+ {
+ *rp = (png_byte)(((int)(*rp) + (int)(*pp++)) & 0xff);
+ rp++;
+ }
+ break;
+ }
+ case PNG_FILTER_VALUE_AVG:
+ {
+ png_uint_32 i;
+ png_bytep rp = row;
+ png_bytep pp = prev_row;
+ png_bytep lp = row;
+ png_uint_32 bpp = (row_info->pixel_depth + 7) >> 3;
+ png_uint_32 istop = row_info->rowbytes - bpp;
+
+ for (i = 0; i < bpp; i++)
+ {
+ *rp = (png_byte)(((int)(*rp) +
+ ((int)(*pp++) / 2 )) & 0xff);
+ rp++;
+ }
+
+ for (i = 0; i < istop; i++)
+ {
+ *rp = (png_byte)(((int)(*rp) +
+ (int)(*pp++ + *lp++) / 2 ) & 0xff);
+ rp++;
+ }
+ break;
+ }
+ case PNG_FILTER_VALUE_PAETH:
+ {
+ png_uint_32 i;
+ png_bytep rp = row;
+ png_bytep pp = prev_row;
+ png_bytep lp = row;
+ png_bytep cp = prev_row;
+ png_uint_32 bpp = (row_info->pixel_depth + 7) >> 3;
+ png_uint_32 istop=row_info->rowbytes - bpp;
+
+ for (i = 0; i < bpp; i++)
+ {
+ *rp = (png_byte)(((int)(*rp) + (int)(*pp++)) & 0xff);
+ rp++;
+ }
+
+ for (i = 0; i < istop; i++) /* Use leftover rp,pp */
+ {
+ int a, b, c, pa, pb, pc, p;
+
+ a = *lp++;
+ b = *pp++;
+ c = *cp++;
+
+ p = b - c;
+ pc = a - c;
+
+#ifdef PNG_USE_ABS
+ pa = abs(p);
+ pb = abs(pc);
+ pc = abs(p + pc);
+#else
+ pa = p < 0 ? -p : p;
+ pb = pc < 0 ? -pc : pc;
+ pc = (p + pc) < 0 ? -(p + pc) : p + pc;
+#endif
+
+ /*
+ if (pa <= pb && pa <= pc)
+ p = a;
+ else if (pb <= pc)
+ p = b;
+ else
+ p = c;
+ */
+
+ p = (pa <= pb && pa <= pc) ? a : (pb <= pc) ? b : c;
+
+ *rp = (png_byte)(((int)(*rp) + p) & 0xff);
+ rp++;
+ }
+ break;
+ }
+ default:
+ png_warning(png_ptr, "Ignoring bad adaptive filter type");
+ *row = 0;
+ break;
+ }
+}
+
+#ifdef PNG_SEQUENTIAL_READ_SUPPORTED
+void /* PRIVATE */
+png_read_finish_row(png_structp png_ptr)
+{
+#ifdef PNG_READ_INTERLACING_SUPPORTED
+ /* Arrays to facilitate easy interlacing - use pass (0 - 6) as index */
+
+ /* Start of interlace block */
+ PNG_CONST int png_pass_start[7] = {0, 4, 0, 2, 0, 1, 0};
+
+ /* Offset to next interlace block */
+ PNG_CONST int png_pass_inc[7] = {8, 8, 4, 4, 2, 2, 1};
+
+ /* Start of interlace block in the y direction */
+ PNG_CONST int png_pass_ystart[7] = {0, 0, 4, 0, 2, 0, 1};
+
+ /* Offset to next interlace block in the y direction */
+ PNG_CONST int png_pass_yinc[7] = {8, 8, 8, 4, 4, 2, 2};
+#endif /* PNG_READ_INTERLACING_SUPPORTED */
+
+ png_debug(1, "in png_read_finish_row");
+ png_ptr->row_number++;
+ if (png_ptr->row_number < png_ptr->num_rows)
+ return;
+
+#ifdef PNG_READ_INTERLACING_SUPPORTED
+ if (png_ptr->interlaced)
+ {
+ png_ptr->row_number = 0;
+ png_memset_check(png_ptr, png_ptr->prev_row, 0,
+ png_ptr->rowbytes + 1);
+ do
+ {
+ png_ptr->pass++;
+ if (png_ptr->pass >= 7)
+ break;
+ png_ptr->iwidth = (png_ptr->width +
+ png_pass_inc[png_ptr->pass] - 1 -
+ png_pass_start[png_ptr->pass]) /
+ png_pass_inc[png_ptr->pass];
+
+ if (!(png_ptr->transformations & PNG_INTERLACE))
+ {
+ png_ptr->num_rows = (png_ptr->height +
+ png_pass_yinc[png_ptr->pass] - 1 -
+ png_pass_ystart[png_ptr->pass]) /
+ png_pass_yinc[png_ptr->pass];
+ if (!(png_ptr->num_rows))
+ continue;
+ }
+ else /* if (png_ptr->transformations & PNG_INTERLACE) */
+ break;
+ } while (png_ptr->iwidth == 0);
+
+ if (png_ptr->pass < 7)
+ return;
+ }
+#endif /* PNG_READ_INTERLACING_SUPPORTED */
+
+ if (!(png_ptr->flags & PNG_FLAG_ZLIB_FINISHED))
+ {
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_CONST PNG_IDAT;
+#endif
+ char extra;
+ int ret;
+
+ png_ptr->zstream.next_out = (Byte *)&extra;
+ png_ptr->zstream.avail_out = (uInt)1;
+ for (;;)
+ {
+ if (!(png_ptr->zstream.avail_in))
+ {
+ while (!png_ptr->idat_size)
+ {
+ png_byte chunk_length[4];
+
+ png_crc_finish(png_ptr, 0);
+
+ png_read_data(png_ptr, chunk_length, 4);
+ png_ptr->idat_size = png_get_uint_31(png_ptr, chunk_length);
+ png_reset_crc(png_ptr);
+ png_crc_read(png_ptr, png_ptr->chunk_name, 4);
+ if (png_memcmp(png_ptr->chunk_name, png_IDAT, 4))
+ png_error(png_ptr, "Not enough image data");
+
+ }
+ png_ptr->zstream.avail_in = (uInt)png_ptr->zbuf_size;
+ png_ptr->zstream.next_in = png_ptr->zbuf;
+ if (png_ptr->zbuf_size > png_ptr->idat_size)
+ png_ptr->zstream.avail_in = (uInt)png_ptr->idat_size;
+ png_crc_read(png_ptr, png_ptr->zbuf, png_ptr->zstream.avail_in);
+ png_ptr->idat_size -= png_ptr->zstream.avail_in;
+ }
+ ret = inflate(&png_ptr->zstream, Z_PARTIAL_FLUSH);
+ if (ret == Z_STREAM_END)
+ {
+ if (!(png_ptr->zstream.avail_out) || png_ptr->zstream.avail_in ||
+ png_ptr->idat_size)
+ png_warning(png_ptr, "Extra compressed data.");
+ png_ptr->mode |= PNG_AFTER_IDAT;
+ png_ptr->flags |= PNG_FLAG_ZLIB_FINISHED;
+ break;
+ }
+ if (ret != Z_OK)
+ png_error(png_ptr, png_ptr->zstream.msg ? png_ptr->zstream.msg :
+ "Decompression Error");
+
+ if (!(png_ptr->zstream.avail_out))
+ {
+ png_warning(png_ptr, "Extra compressed data.");
+ png_ptr->mode |= PNG_AFTER_IDAT;
+ png_ptr->flags |= PNG_FLAG_ZLIB_FINISHED;
+ break;
+ }
+
+ }
+ png_ptr->zstream.avail_out = 0;
+ }
+
+ if (png_ptr->idat_size || png_ptr->zstream.avail_in)
+ png_warning(png_ptr, "Extra compression data.");
+
+ inflateReset(&png_ptr->zstream);
+
+ png_ptr->mode |= PNG_AFTER_IDAT;
+}
+#endif /* PNG_SEQUENTIAL_READ_SUPPORTED */
+
+void /* PRIVATE */
+png_read_start_row(png_structp png_ptr)
+{
+#ifdef PNG_READ_INTERLACING_SUPPORTED
+ /* Arrays to facilitate easy interlacing - use pass (0 - 6) as index */
+
+ /* Start of interlace block */
+ PNG_CONST int png_pass_start[7] = {0, 4, 0, 2, 0, 1, 0};
+
+ /* Offset to next interlace block */
+ PNG_CONST int png_pass_inc[7] = {8, 8, 4, 4, 2, 2, 1};
+
+ /* Start of interlace block in the y direction */
+ PNG_CONST int png_pass_ystart[7] = {0, 0, 4, 0, 2, 0, 1};
+
+ /* Offset to next interlace block in the y direction */
+ PNG_CONST int png_pass_yinc[7] = {8, 8, 8, 4, 4, 2, 2};
+#endif
+
+ int max_pixel_depth;
+ png_size_t row_bytes;
+
+ png_debug(1, "in png_read_start_row");
+ png_ptr->zstream.avail_in = 0;
+ png_init_read_transformations(png_ptr);
+#ifdef PNG_READ_INTERLACING_SUPPORTED
+ if (png_ptr->interlaced)
+ {
+ if (!(png_ptr->transformations & PNG_INTERLACE))
+ png_ptr->num_rows = (png_ptr->height + png_pass_yinc[0] - 1 -
+ png_pass_ystart[0]) / png_pass_yinc[0];
+ else
+ png_ptr->num_rows = png_ptr->height;
+
+ png_ptr->iwidth = (png_ptr->width +
+ png_pass_inc[png_ptr->pass] - 1 -
+ png_pass_start[png_ptr->pass]) /
+ png_pass_inc[png_ptr->pass];
+ }
+ else
+#endif /* PNG_READ_INTERLACING_SUPPORTED */
+ {
+ png_ptr->num_rows = png_ptr->height;
+ png_ptr->iwidth = png_ptr->width;
+ }
+ max_pixel_depth = png_ptr->pixel_depth;
+
+#ifdef PNG_READ_PACK_SUPPORTED
+ if ((png_ptr->transformations & PNG_PACK) && png_ptr->bit_depth < 8)
+ max_pixel_depth = 8;
+#endif
+
+#ifdef PNG_READ_EXPAND_SUPPORTED
+ if (png_ptr->transformations & PNG_EXPAND)
+ {
+ if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ if (png_ptr->num_trans)
+ max_pixel_depth = 32;
+ else
+ max_pixel_depth = 24;
+ }
+ else if (png_ptr->color_type == PNG_COLOR_TYPE_GRAY)
+ {
+ if (max_pixel_depth < 8)
+ max_pixel_depth = 8;
+ if (png_ptr->num_trans)
+ max_pixel_depth *= 2;
+ }
+ else if (png_ptr->color_type == PNG_COLOR_TYPE_RGB)
+ {
+ if (png_ptr->num_trans)
+ {
+ max_pixel_depth *= 4;
+ max_pixel_depth /= 3;
+ }
+ }
+ }
+#endif
+
+#ifdef PNG_READ_FILLER_SUPPORTED
+ if (png_ptr->transformations & (PNG_FILLER))
+ {
+ if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ max_pixel_depth = 32;
+ else if (png_ptr->color_type == PNG_COLOR_TYPE_GRAY)
+ {
+ if (max_pixel_depth <= 8)
+ max_pixel_depth = 16;
+ else
+ max_pixel_depth = 32;
+ }
+ else if (png_ptr->color_type == PNG_COLOR_TYPE_RGB)
+ {
+ if (max_pixel_depth <= 32)
+ max_pixel_depth = 32;
+ else
+ max_pixel_depth = 64;
+ }
+ }
+#endif
+
+#ifdef PNG_READ_GRAY_TO_RGB_SUPPORTED
+ if (png_ptr->transformations & PNG_GRAY_TO_RGB)
+ {
+ if (
+#ifdef PNG_READ_EXPAND_SUPPORTED
+ (png_ptr->num_trans && (png_ptr->transformations & PNG_EXPAND)) ||
+#endif
+#ifdef PNG_READ_FILLER_SUPPORTED
+ (png_ptr->transformations & (PNG_FILLER)) ||
+#endif
+ png_ptr->color_type == PNG_COLOR_TYPE_GRAY_ALPHA)
+ {
+ if (max_pixel_depth <= 16)
+ max_pixel_depth = 32;
+ else
+ max_pixel_depth = 64;
+ }
+ else
+ {
+ if (max_pixel_depth <= 8)
+ {
+ if (png_ptr->color_type == PNG_COLOR_TYPE_RGB_ALPHA)
+ max_pixel_depth = 32;
+ else
+ max_pixel_depth = 24;
+ }
+ else if (png_ptr->color_type == PNG_COLOR_TYPE_RGB_ALPHA)
+ max_pixel_depth = 64;
+ else
+ max_pixel_depth = 48;
+ }
+ }
+#endif
+
+#if defined(PNG_READ_USER_TRANSFORM_SUPPORTED) && \
+defined(PNG_USER_TRANSFORM_PTR_SUPPORTED)
+ if (png_ptr->transformations & PNG_USER_TRANSFORM)
+ {
+ int user_pixel_depth = png_ptr->user_transform_depth*
+ png_ptr->user_transform_channels;
+ if (user_pixel_depth > max_pixel_depth)
+ max_pixel_depth=user_pixel_depth;
+ }
+#endif
+
+ /* Align the width on the next larger 8 pixels. Mainly used
+ * for interlacing
+ */
+ row_bytes = ((png_ptr->width + 7) & ~((png_uint_32)7));
+ /* Calculate the maximum bytes needed, adding a byte and a pixel
+ * for safety's sake
+ */
+ row_bytes = PNG_ROWBYTES(max_pixel_depth, row_bytes) +
+ 1 + ((max_pixel_depth + 7) >> 3);
+#ifdef PNG_MAX_MALLOC_64K
+ if (row_bytes > (png_uint_32)65536L)
+ png_error(png_ptr, "This image requires a row greater than 64KB");
+#endif
+
+ if (row_bytes + 64 > png_ptr->old_big_row_buf_size)
+ {
+ png_free(png_ptr, png_ptr->big_row_buf);
+ if (png_ptr->interlaced)
+ png_ptr->big_row_buf = (png_bytep)png_calloc(png_ptr,
+ row_bytes + 64);
+ else
+ png_ptr->big_row_buf = (png_bytep)png_malloc(png_ptr,
+ row_bytes + 64);
+ png_ptr->old_big_row_buf_size = row_bytes + 64;
+
+ /* Use 32 bytes of padding before and after row_buf. */
+ png_ptr->row_buf = png_ptr->big_row_buf + 32;
+ png_ptr->old_big_row_buf_size = row_bytes + 64;
+ }
+
+#ifdef PNG_MAX_MALLOC_64K
+ if ((png_uint_32)row_bytes + 1 > (png_uint_32)65536L)
+ png_error(png_ptr, "This image requires a row greater than 64KB");
+#endif
+ if ((png_uint_32)row_bytes > (png_uint_32)(PNG_SIZE_MAX - 1))
+ png_error(png_ptr, "Row has too many bytes to allocate in memory.");
+
+ if (row_bytes + 1 > png_ptr->old_prev_row_size)
+ {
+ png_free(png_ptr, png_ptr->prev_row);
+ png_ptr->prev_row = (png_bytep)png_malloc(png_ptr, (png_uint_32)(
+ row_bytes + 1));
+ png_memset_check(png_ptr, png_ptr->prev_row, 0, row_bytes + 1);
+ png_ptr->old_prev_row_size = row_bytes + 1;
+ }
+
+ png_ptr->rowbytes = row_bytes;
+
+ png_debug1(3, "width = %lu,", png_ptr->width);
+ png_debug1(3, "height = %lu,", png_ptr->height);
+ png_debug1(3, "iwidth = %lu,", png_ptr->iwidth);
+ png_debug1(3, "num_rows = %lu,", png_ptr->num_rows);
+ png_debug1(3, "rowbytes = %lu,", png_ptr->rowbytes);
+ png_debug1(3, "irowbytes = %lu",
+ PNG_ROWBYTES(png_ptr->pixel_depth, png_ptr->iwidth) + 1);
+
+ png_ptr->flags |= PNG_FLAG_ROW_INIT;
+}
+#endif /* PNG_READ_SUPPORTED */
diff --git a/trunk/src/third_party/libpng/pngset.c b/trunk/src/third_party/libpng/pngset.c
new file mode 100644
index 0000000..717757f
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngset.c
@@ -0,0 +1,1226 @@
+
+/* pngset.c - storage of image information into info struct
+ *
+ * Last changed in libpng 1.2.43 [February 25, 2010]
+ * Copyright (c) 1998-2010 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ *
+ * The functions here are used during reads to store data from the file
+ * into the info struct, and during writes to store application data
+ * into the info struct for writing into the file. This abstracts the
+ * info struct and allows us to change the structure in the future.
+ */
+
+#define PNG_INTERNAL
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+#if defined(PNG_READ_SUPPORTED) || defined(PNG_WRITE_SUPPORTED)
+
+#ifdef PNG_bKGD_SUPPORTED
+void PNGAPI
+png_set_bKGD(png_structp png_ptr, png_infop info_ptr, png_color_16p background)
+{
+ png_debug1(1, "in %s storage function", "bKGD");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ png_memcpy(&(info_ptr->background), background, png_sizeof(png_color_16));
+ info_ptr->valid |= PNG_INFO_bKGD;
+}
+#endif
+
+#ifdef PNG_cHRM_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+void PNGAPI
+png_set_cHRM(png_structp png_ptr, png_infop info_ptr,
+ double white_x, double white_y, double red_x, double red_y,
+ double green_x, double green_y, double blue_x, double blue_y)
+{
+ png_debug1(1, "in %s storage function", "cHRM");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ info_ptr->x_white = (float)white_x;
+ info_ptr->y_white = (float)white_y;
+ info_ptr->x_red = (float)red_x;
+ info_ptr->y_red = (float)red_y;
+ info_ptr->x_green = (float)green_x;
+ info_ptr->y_green = (float)green_y;
+ info_ptr->x_blue = (float)blue_x;
+ info_ptr->y_blue = (float)blue_y;
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ info_ptr->int_x_white = (png_fixed_point)(white_x*100000.+0.5);
+ info_ptr->int_y_white = (png_fixed_point)(white_y*100000.+0.5);
+ info_ptr->int_x_red = (png_fixed_point)( red_x*100000.+0.5);
+ info_ptr->int_y_red = (png_fixed_point)( red_y*100000.+0.5);
+ info_ptr->int_x_green = (png_fixed_point)(green_x*100000.+0.5);
+ info_ptr->int_y_green = (png_fixed_point)(green_y*100000.+0.5);
+ info_ptr->int_x_blue = (png_fixed_point)( blue_x*100000.+0.5);
+ info_ptr->int_y_blue = (png_fixed_point)( blue_y*100000.+0.5);
+#endif
+ info_ptr->valid |= PNG_INFO_cHRM;
+}
+#endif /* PNG_FLOATING_POINT_SUPPORTED */
+
+#ifdef PNG_FIXED_POINT_SUPPORTED
+void PNGAPI
+png_set_cHRM_fixed(png_structp png_ptr, png_infop info_ptr,
+ png_fixed_point white_x, png_fixed_point white_y, png_fixed_point red_x,
+ png_fixed_point red_y, png_fixed_point green_x, png_fixed_point green_y,
+ png_fixed_point blue_x, png_fixed_point blue_y)
+{
+ png_debug1(1, "in %s storage function", "cHRM fixed");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+#ifdef PNG_CHECK_cHRM_SUPPORTED
+ if (png_check_cHRM_fixed(png_ptr,
+ white_x, white_y, red_x, red_y, green_x, green_y, blue_x, blue_y))
+#endif
+ {
+ info_ptr->int_x_white = white_x;
+ info_ptr->int_y_white = white_y;
+ info_ptr->int_x_red = red_x;
+ info_ptr->int_y_red = red_y;
+ info_ptr->int_x_green = green_x;
+ info_ptr->int_y_green = green_y;
+ info_ptr->int_x_blue = blue_x;
+ info_ptr->int_y_blue = blue_y;
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ info_ptr->x_white = (float)(white_x/100000.);
+ info_ptr->y_white = (float)(white_y/100000.);
+ info_ptr->x_red = (float)( red_x/100000.);
+ info_ptr->y_red = (float)( red_y/100000.);
+ info_ptr->x_green = (float)(green_x/100000.);
+ info_ptr->y_green = (float)(green_y/100000.);
+ info_ptr->x_blue = (float)( blue_x/100000.);
+ info_ptr->y_blue = (float)( blue_y/100000.);
+#endif
+ info_ptr->valid |= PNG_INFO_cHRM;
+ }
+}
+#endif /* PNG_FIXED_POINT_SUPPORTED */
+#endif /* PNG_cHRM_SUPPORTED */
+
+#ifdef PNG_gAMA_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+void PNGAPI
+png_set_gAMA(png_structp png_ptr, png_infop info_ptr, double file_gamma)
+{
+ double png_gamma;
+
+ png_debug1(1, "in %s storage function", "gAMA");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ /* Check for overflow */
+ if (file_gamma > 21474.83)
+ {
+ png_warning(png_ptr, "Limiting gamma to 21474.83");
+ png_gamma=21474.83;
+ }
+ else
+ png_gamma = file_gamma;
+ info_ptr->gamma = (float)png_gamma;
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ info_ptr->int_gamma = (int)(png_gamma*100000.+.5);
+#endif
+ info_ptr->valid |= PNG_INFO_gAMA;
+ if (png_gamma == 0.0)
+ png_warning(png_ptr, "Setting gamma=0");
+}
+#endif
+void PNGAPI
+png_set_gAMA_fixed(png_structp png_ptr, png_infop info_ptr, png_fixed_point
+ int_gamma)
+{
+ png_fixed_point png_gamma;
+
+ png_debug1(1, "in %s storage function", "gAMA");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ if (int_gamma > (png_fixed_point)PNG_UINT_31_MAX)
+ {
+ png_warning(png_ptr, "Limiting gamma to 21474.83");
+ png_gamma=PNG_UINT_31_MAX;
+ }
+ else
+ {
+ if (int_gamma < 0)
+ {
+ png_warning(png_ptr, "Setting negative gamma to zero");
+ png_gamma = 0;
+ }
+ else
+ png_gamma = int_gamma;
+ }
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ info_ptr->gamma = (float)(png_gamma/100000.);
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ info_ptr->int_gamma = png_gamma;
+#endif
+ info_ptr->valid |= PNG_INFO_gAMA;
+ if (png_gamma == 0)
+ png_warning(png_ptr, "Setting gamma=0");
+}
+#endif
+
+#ifdef PNG_hIST_SUPPORTED
+void PNGAPI
+png_set_hIST(png_structp png_ptr, png_infop info_ptr, png_uint_16p hist)
+{
+ int i;
+
+ png_debug1(1, "in %s storage function", "hIST");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ if (info_ptr->num_palette == 0 || info_ptr->num_palette
+ > PNG_MAX_PALETTE_LENGTH)
+ {
+ png_warning(png_ptr,
+ "Invalid palette size, hIST allocation skipped.");
+ return;
+ }
+
+#ifdef PNG_FREE_ME_SUPPORTED
+ png_free_data(png_ptr, info_ptr, PNG_FREE_HIST, 0);
+#endif
+ /* Changed from info->num_palette to PNG_MAX_PALETTE_LENGTH in
+ * version 1.2.1
+ */
+ png_ptr->hist = (png_uint_16p)png_malloc_warn(png_ptr,
+ (png_uint_32)(PNG_MAX_PALETTE_LENGTH * png_sizeof(png_uint_16)));
+ if (png_ptr->hist == NULL)
+ {
+ png_warning(png_ptr, "Insufficient memory for hIST chunk data.");
+ return;
+ }
+
+ for (i = 0; i < info_ptr->num_palette; i++)
+ png_ptr->hist[i] = hist[i];
+ info_ptr->hist = png_ptr->hist;
+ info_ptr->valid |= PNG_INFO_hIST;
+
+#ifdef PNG_FREE_ME_SUPPORTED
+ info_ptr->free_me |= PNG_FREE_HIST;
+#else
+ png_ptr->flags |= PNG_FLAG_FREE_HIST;
+#endif
+}
+#endif
+
+void PNGAPI
+png_set_IHDR(png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 width, png_uint_32 height, int bit_depth,
+ int color_type, int interlace_type, int compression_type,
+ int filter_type)
+{
+ png_debug1(1, "in %s storage function", "IHDR");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ info_ptr->width = width;
+ info_ptr->height = height;
+ info_ptr->bit_depth = (png_byte)bit_depth;
+ info_ptr->color_type = (png_byte)color_type;
+ info_ptr->compression_type = (png_byte)compression_type;
+ info_ptr->filter_type = (png_byte)filter_type;
+ info_ptr->interlace_type = (png_byte)interlace_type;
+
+ png_check_IHDR (png_ptr, info_ptr->width, info_ptr->height,
+ info_ptr->bit_depth, info_ptr->color_type, info_ptr->interlace_type,
+ info_ptr->compression_type, info_ptr->filter_type);
+
+ if (info_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ info_ptr->channels = 1;
+ else if (info_ptr->color_type & PNG_COLOR_MASK_COLOR)
+ info_ptr->channels = 3;
+ else
+ info_ptr->channels = 1;
+ if (info_ptr->color_type & PNG_COLOR_MASK_ALPHA)
+ info_ptr->channels++;
+ info_ptr->pixel_depth = (png_byte)(info_ptr->channels * info_ptr->bit_depth);
+
+ /* Check for potential overflow */
+ if (width > (PNG_UINT_32_MAX
+ >> 3) /* 8-byte RGBA pixels */
+ - 64 /* bigrowbuf hack */
+ - 1 /* filter byte */
+ - 7*8 /* rounding of width to multiple of 8 pixels */
+ - 8) /* extra max_pixel_depth pad */
+ info_ptr->rowbytes = (png_size_t)0;
+ else
+ info_ptr->rowbytes = PNG_ROWBYTES(info_ptr->pixel_depth, width);
+}
+
+#ifdef PNG_oFFs_SUPPORTED
+void PNGAPI
+png_set_oFFs(png_structp png_ptr, png_infop info_ptr,
+ png_int_32 offset_x, png_int_32 offset_y, int unit_type)
+{
+ png_debug1(1, "in %s storage function", "oFFs");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ info_ptr->x_offset = offset_x;
+ info_ptr->y_offset = offset_y;
+ info_ptr->offset_unit_type = (png_byte)unit_type;
+ info_ptr->valid |= PNG_INFO_oFFs;
+}
+#endif
+
+#ifdef PNG_pCAL_SUPPORTED
+void PNGAPI
+png_set_pCAL(png_structp png_ptr, png_infop info_ptr,
+ png_charp purpose, png_int_32 X0, png_int_32 X1, int type, int nparams,
+ png_charp units, png_charpp params)
+{
+ png_uint_32 length;
+ int i;
+
+ png_debug1(1, "in %s storage function", "pCAL");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ length = png_strlen(purpose) + 1;
+ png_debug1(3, "allocating purpose for info (%lu bytes)",
+ (unsigned long)length);
+ info_ptr->pcal_purpose = (png_charp)png_malloc_warn(png_ptr, length);
+ if (info_ptr->pcal_purpose == NULL)
+ {
+ png_warning(png_ptr, "Insufficient memory for pCAL purpose.");
+ return;
+ }
+ png_memcpy(info_ptr->pcal_purpose, purpose, (png_size_t)length);
+
+ png_debug(3, "storing X0, X1, type, and nparams in info");
+ info_ptr->pcal_X0 = X0;
+ info_ptr->pcal_X1 = X1;
+ info_ptr->pcal_type = (png_byte)type;
+ info_ptr->pcal_nparams = (png_byte)nparams;
+
+ length = png_strlen(units) + 1;
+ png_debug1(3, "allocating units for info (%lu bytes)",
+ (unsigned long)length);
+ info_ptr->pcal_units = (png_charp)png_malloc_warn(png_ptr, length);
+ if (info_ptr->pcal_units == NULL)
+ {
+ png_warning(png_ptr, "Insufficient memory for pCAL units.");
+ return;
+ }
+ png_memcpy(info_ptr->pcal_units, units, (png_size_t)length);
+
+ info_ptr->pcal_params = (png_charpp)png_malloc_warn(png_ptr,
+ (png_uint_32)((nparams + 1) * png_sizeof(png_charp)));
+ if (info_ptr->pcal_params == NULL)
+ {
+ png_warning(png_ptr, "Insufficient memory for pCAL params.");
+ return;
+ }
+
+ png_memset(info_ptr->pcal_params, 0, (nparams + 1) * png_sizeof(png_charp));
+
+ for (i = 0; i < nparams; i++)
+ {
+ length = png_strlen(params[i]) + 1;
+ png_debug2(3, "allocating parameter %d for info (%lu bytes)", i,
+ (unsigned long)length);
+ info_ptr->pcal_params[i] = (png_charp)png_malloc_warn(png_ptr, length);
+ if (info_ptr->pcal_params[i] == NULL)
+ {
+ png_warning(png_ptr, "Insufficient memory for pCAL parameter.");
+ return;
+ }
+ png_memcpy(info_ptr->pcal_params[i], params[i], (png_size_t)length);
+ }
+
+ info_ptr->valid |= PNG_INFO_pCAL;
+#ifdef PNG_FREE_ME_SUPPORTED
+ info_ptr->free_me |= PNG_FREE_PCAL;
+#endif
+}
+#endif
+
+#if defined(PNG_READ_sCAL_SUPPORTED) || defined(PNG_WRITE_sCAL_SUPPORTED)
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+void PNGAPI
+png_set_sCAL(png_structp png_ptr, png_infop info_ptr,
+ int unit, double width, double height)
+{
+ png_debug1(1, "in %s storage function", "sCAL");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ info_ptr->scal_unit = (png_byte)unit;
+ info_ptr->scal_pixel_width = width;
+ info_ptr->scal_pixel_height = height;
+
+ info_ptr->valid |= PNG_INFO_sCAL;
+}
+#else
+#ifdef PNG_FIXED_POINT_SUPPORTED
+void PNGAPI
+png_set_sCAL_s(png_structp png_ptr, png_infop info_ptr,
+ int unit, png_charp swidth, png_charp sheight)
+{
+ png_uint_32 length;
+
+ png_debug1(1, "in %s storage function", "sCAL");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ info_ptr->scal_unit = (png_byte)unit;
+
+ length = png_strlen(swidth) + 1;
+ png_debug1(3, "allocating unit for info (%u bytes)",
+ (unsigned int)length);
+ info_ptr->scal_s_width = (png_charp)png_malloc_warn(png_ptr, length);
+ if (info_ptr->scal_s_width == NULL)
+ {
+ png_warning(png_ptr,
+ "Memory allocation failed while processing sCAL.");
+ return;
+ }
+ png_memcpy(info_ptr->scal_s_width, swidth, (png_size_t)length);
+
+ length = png_strlen(sheight) + 1;
+ png_debug1(3, "allocating unit for info (%u bytes)",
+ (unsigned int)length);
+ info_ptr->scal_s_height = (png_charp)png_malloc_warn(png_ptr, length);
+ if (info_ptr->scal_s_height == NULL)
+ {
+ png_free (png_ptr, info_ptr->scal_s_width);
+ info_ptr->scal_s_width = NULL;
+ png_warning(png_ptr,
+ "Memory allocation failed while processing sCAL.");
+ return;
+ }
+ png_memcpy(info_ptr->scal_s_height, sheight, (png_size_t)length);
+ info_ptr->valid |= PNG_INFO_sCAL;
+#ifdef PNG_FREE_ME_SUPPORTED
+ info_ptr->free_me |= PNG_FREE_SCAL;
+#endif
+}
+#endif
+#endif
+#endif
+
+#ifdef PNG_pHYs_SUPPORTED
+void PNGAPI
+png_set_pHYs(png_structp png_ptr, png_infop info_ptr,
+ png_uint_32 res_x, png_uint_32 res_y, int unit_type)
+{
+ png_debug1(1, "in %s storage function", "pHYs");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ info_ptr->x_pixels_per_unit = res_x;
+ info_ptr->y_pixels_per_unit = res_y;
+ info_ptr->phys_unit_type = (png_byte)unit_type;
+ info_ptr->valid |= PNG_INFO_pHYs;
+}
+#endif
+
+void PNGAPI
+png_set_PLTE(png_structp png_ptr, png_infop info_ptr,
+ png_colorp palette, int num_palette)
+{
+
+ png_debug1(1, "in %s storage function", "PLTE");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ if (num_palette < 0 || num_palette > PNG_MAX_PALETTE_LENGTH)
+ {
+ if (info_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ png_error(png_ptr, "Invalid palette length");
+ else
+ {
+ png_warning(png_ptr, "Invalid palette length");
+ return;
+ }
+ }
+
+ /* It may not actually be necessary to set png_ptr->palette here;
+ * we do it for backward compatibility with the way the png_handle_tRNS
+ * function used to do the allocation.
+ */
+#ifdef PNG_FREE_ME_SUPPORTED
+ png_free_data(png_ptr, info_ptr, PNG_FREE_PLTE, 0);
+#endif
+
+ /* Changed in libpng-1.2.1 to allocate PNG_MAX_PALETTE_LENGTH instead
+ * of num_palette entries, in case of an invalid PNG file that has
+ * too-large sample values.
+ */
+ png_ptr->palette = (png_colorp)png_calloc(png_ptr,
+ PNG_MAX_PALETTE_LENGTH * png_sizeof(png_color));
+ png_memcpy(png_ptr->palette, palette, num_palette * png_sizeof(png_color));
+ info_ptr->palette = png_ptr->palette;
+ info_ptr->num_palette = png_ptr->num_palette = (png_uint_16)num_palette;
+
+#ifdef PNG_FREE_ME_SUPPORTED
+ info_ptr->free_me |= PNG_FREE_PLTE;
+#else
+ png_ptr->flags |= PNG_FLAG_FREE_PLTE;
+#endif
+
+ info_ptr->valid |= PNG_INFO_PLTE;
+}
+
+#ifdef PNG_sBIT_SUPPORTED
+void PNGAPI
+png_set_sBIT(png_structp png_ptr, png_infop info_ptr,
+ png_color_8p sig_bit)
+{
+ png_debug1(1, "in %s storage function", "sBIT");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ png_memcpy(&(info_ptr->sig_bit), sig_bit, png_sizeof(png_color_8));
+ info_ptr->valid |= PNG_INFO_sBIT;
+}
+#endif
+
+#ifdef PNG_sRGB_SUPPORTED
+void PNGAPI
+png_set_sRGB(png_structp png_ptr, png_infop info_ptr, int intent)
+{
+ png_debug1(1, "in %s storage function", "sRGB");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ info_ptr->srgb_intent = (png_byte)intent;
+ info_ptr->valid |= PNG_INFO_sRGB;
+}
+
+void PNGAPI
+png_set_sRGB_gAMA_and_cHRM(png_structp png_ptr, png_infop info_ptr,
+ int intent)
+{
+#ifdef PNG_gAMA_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ float file_gamma;
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ png_fixed_point int_file_gamma;
+#endif
+#endif
+#ifdef PNG_cHRM_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ float white_x, white_y, red_x, red_y, green_x, green_y, blue_x, blue_y;
+#endif
+ png_fixed_point int_white_x, int_white_y, int_red_x, int_red_y, int_green_x,
+ int_green_y, int_blue_x, int_blue_y;
+#endif
+ png_debug1(1, "in %s storage function", "sRGB_gAMA_and_cHRM");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ png_set_sRGB(png_ptr, info_ptr, intent);
+
+#ifdef PNG_gAMA_SUPPORTED
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ file_gamma = (float).45455;
+ png_set_gAMA(png_ptr, info_ptr, file_gamma);
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ int_file_gamma = 45455L;
+ png_set_gAMA_fixed(png_ptr, info_ptr, int_file_gamma);
+#endif
+#endif
+
+#ifdef PNG_cHRM_SUPPORTED
+ int_white_x = 31270L;
+ int_white_y = 32900L;
+ int_red_x = 64000L;
+ int_red_y = 33000L;
+ int_green_x = 30000L;
+ int_green_y = 60000L;
+ int_blue_x = 15000L;
+ int_blue_y = 6000L;
+
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ white_x = (float).3127;
+ white_y = (float).3290;
+ red_x = (float).64;
+ red_y = (float).33;
+ green_x = (float).30;
+ green_y = (float).60;
+ blue_x = (float).15;
+ blue_y = (float).06;
+#endif
+
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ png_set_cHRM_fixed(png_ptr, info_ptr,
+ int_white_x, int_white_y, int_red_x, int_red_y, int_green_x,
+ int_green_y, int_blue_x, int_blue_y);
+#endif
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ png_set_cHRM(png_ptr, info_ptr,
+ white_x, white_y, red_x, red_y, green_x, green_y, blue_x, blue_y);
+#endif
+#endif /* cHRM */
+}
+#endif /* sRGB */
+
+
+#ifdef PNG_iCCP_SUPPORTED
+void PNGAPI
+png_set_iCCP(png_structp png_ptr, png_infop info_ptr,
+ png_charp name, int compression_type,
+ png_charp profile, png_uint_32 proflen)
+{
+ png_charp new_iccp_name;
+ png_charp new_iccp_profile;
+ png_uint_32 length;
+
+ png_debug1(1, "in %s storage function", "iCCP");
+
+ if (png_ptr == NULL || info_ptr == NULL || name == NULL || profile == NULL)
+ return;
+
+ length = png_strlen(name)+1;
+ new_iccp_name = (png_charp)png_malloc_warn(png_ptr, length);
+ if (new_iccp_name == NULL)
+ {
+ png_warning(png_ptr, "Insufficient memory to process iCCP chunk.");
+ return;
+ }
+ png_memcpy(new_iccp_name, name, length);
+ new_iccp_profile = (png_charp)png_malloc_warn(png_ptr, proflen);
+ if (new_iccp_profile == NULL)
+ {
+ png_free (png_ptr, new_iccp_name);
+ png_warning(png_ptr,
+ "Insufficient memory to process iCCP profile.");
+ return;
+ }
+ png_memcpy(new_iccp_profile, profile, (png_size_t)proflen);
+
+ png_free_data(png_ptr, info_ptr, PNG_FREE_ICCP, 0);
+
+ info_ptr->iccp_proflen = proflen;
+ info_ptr->iccp_name = new_iccp_name;
+ info_ptr->iccp_profile = new_iccp_profile;
+ /* Compression is always zero but is here so the API and info structure
+ * does not have to change if we introduce multiple compression types
+ */
+ info_ptr->iccp_compression = (png_byte)compression_type;
+#ifdef PNG_FREE_ME_SUPPORTED
+ info_ptr->free_me |= PNG_FREE_ICCP;
+#endif
+ info_ptr->valid |= PNG_INFO_iCCP;
+}
+#endif
+
+#ifdef PNG_TEXT_SUPPORTED
+void PNGAPI
+png_set_text(png_structp png_ptr, png_infop info_ptr, png_textp text_ptr,
+ int num_text)
+{
+ int ret;
+ ret = png_set_text_2(png_ptr, info_ptr, text_ptr, num_text);
+ if (ret)
+ png_error(png_ptr, "Insufficient memory to store text");
+}
+
+int /* PRIVATE */
+png_set_text_2(png_structp png_ptr, png_infop info_ptr, png_textp text_ptr,
+ int num_text)
+{
+ int i;
+
+ png_debug1(1, "in %s storage function", ((png_ptr == NULL ||
+ png_ptr->chunk_name[0] == '\0') ?
+ "text" : (png_const_charp)png_ptr->chunk_name));
+
+ if (png_ptr == NULL || info_ptr == NULL || num_text == 0)
+ return(0);
+
+ /* Make sure we have enough space in the "text" array in info_struct
+ * to hold all of the incoming text_ptr objects.
+ */
+ if (info_ptr->num_text + num_text > info_ptr->max_text)
+ {
+ if (info_ptr->text != NULL)
+ {
+ png_textp old_text;
+ int old_max;
+
+ old_max = info_ptr->max_text;
+ info_ptr->max_text = info_ptr->num_text + num_text + 8;
+ old_text = info_ptr->text;
+ info_ptr->text = (png_textp)png_malloc_warn(png_ptr,
+ (png_uint_32)(info_ptr->max_text * png_sizeof(png_text)));
+ if (info_ptr->text == NULL)
+ {
+ png_free(png_ptr, old_text);
+ return(1);
+ }
+ png_memcpy(info_ptr->text, old_text, (png_size_t)(old_max *
+ png_sizeof(png_text)));
+ png_free(png_ptr, old_text);
+ }
+ else
+ {
+ info_ptr->max_text = num_text + 8;
+ info_ptr->num_text = 0;
+ info_ptr->text = (png_textp)png_malloc_warn(png_ptr,
+ (png_uint_32)(info_ptr->max_text * png_sizeof(png_text)));
+ if (info_ptr->text == NULL)
+ return(1);
+#ifdef PNG_FREE_ME_SUPPORTED
+ info_ptr->free_me |= PNG_FREE_TEXT;
+#endif
+ }
+ png_debug1(3, "allocated %d entries for info_ptr->text",
+ info_ptr->max_text);
+ }
+ for (i = 0; i < num_text; i++)
+ {
+ png_size_t text_length, key_len;
+ png_size_t lang_len, lang_key_len;
+ png_textp textp = &(info_ptr->text[info_ptr->num_text]);
+
+ if (text_ptr[i].key == NULL)
+ continue;
+
+ key_len = png_strlen(text_ptr[i].key);
+
+ if (text_ptr[i].compression <= 0)
+ {
+ lang_len = 0;
+ lang_key_len = 0;
+ }
+
+ else
+#ifdef PNG_iTXt_SUPPORTED
+ {
+ /* Set iTXt data */
+
+ if (text_ptr[i].lang != NULL)
+ lang_len = png_strlen(text_ptr[i].lang);
+ else
+ lang_len = 0;
+ if (text_ptr[i].lang_key != NULL)
+ lang_key_len = png_strlen(text_ptr[i].lang_key);
+ else
+ lang_key_len = 0;
+ }
+#else /* PNG_iTXt_SUPPORTED */
+ {
+ png_warning(png_ptr, "iTXt chunk not supported.");
+ continue;
+ }
+#endif
+
+ if (text_ptr[i].text == NULL || text_ptr[i].text[0] == '\0')
+ {
+ text_length = 0;
+#ifdef PNG_iTXt_SUPPORTED
+ if (text_ptr[i].compression > 0)
+ textp->compression = PNG_ITXT_COMPRESSION_NONE;
+ else
+#endif
+ textp->compression = PNG_TEXT_COMPRESSION_NONE;
+ }
+
+ else
+ {
+ text_length = png_strlen(text_ptr[i].text);
+ textp->compression = text_ptr[i].compression;
+ }
+
+ textp->key = (png_charp)png_malloc_warn(png_ptr,
+ (png_uint_32)
+ (key_len + text_length + lang_len + lang_key_len + 4));
+ if (textp->key == NULL)
+ return(1);
+ png_debug2(2, "Allocated %lu bytes at %x in png_set_text",
+ (png_uint_32)
+ (key_len + lang_len + lang_key_len + text_length + 4),
+ (int)textp->key);
+
+ png_memcpy(textp->key, text_ptr[i].key,(png_size_t)(key_len));
+ *(textp->key + key_len) = '\0';
+#ifdef PNG_iTXt_SUPPORTED
+ if (text_ptr[i].compression > 0)
+ {
+ textp->lang = textp->key + key_len + 1;
+ png_memcpy(textp->lang, text_ptr[i].lang, lang_len);
+ *(textp->lang + lang_len) = '\0';
+ textp->lang_key = textp->lang + lang_len + 1;
+ png_memcpy(textp->lang_key, text_ptr[i].lang_key, lang_key_len);
+ *(textp->lang_key + lang_key_len) = '\0';
+ textp->text = textp->lang_key + lang_key_len + 1;
+ }
+ else
+#endif
+ {
+#ifdef PNG_iTXt_SUPPORTED
+ textp->lang=NULL;
+ textp->lang_key=NULL;
+#endif
+ textp->text = textp->key + key_len + 1;
+ }
+ if (text_length)
+ png_memcpy(textp->text, text_ptr[i].text,
+ (png_size_t)(text_length));
+ *(textp->text + text_length) = '\0';
+
+#ifdef PNG_iTXt_SUPPORTED
+ if (textp->compression > 0)
+ {
+ textp->text_length = 0;
+ textp->itxt_length = text_length;
+ }
+ else
+#endif
+
+ {
+ textp->text_length = text_length;
+#ifdef PNG_iTXt_SUPPORTED
+ textp->itxt_length = 0;
+#endif
+ }
+ info_ptr->num_text++;
+ png_debug1(3, "transferred text chunk %d", info_ptr->num_text);
+ }
+ return(0);
+}
+#endif
+
+#ifdef PNG_tIME_SUPPORTED
+void PNGAPI
+png_set_tIME(png_structp png_ptr, png_infop info_ptr, png_timep mod_time)
+{
+ png_debug1(1, "in %s storage function", "tIME");
+
+ if (png_ptr == NULL || info_ptr == NULL ||
+ (png_ptr->mode & PNG_WROTE_tIME))
+ return;
+
+ png_memcpy(&(info_ptr->mod_time), mod_time, png_sizeof(png_time));
+ info_ptr->valid |= PNG_INFO_tIME;
+}
+#endif
+
+#ifdef PNG_tRNS_SUPPORTED
+void PNGAPI
+png_set_tRNS(png_structp png_ptr, png_infop info_ptr,
+ png_bytep trans, int num_trans, png_color_16p trans_values)
+{
+ png_debug1(1, "in %s storage function", "tRNS");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ if (trans != NULL)
+ {
+ /* It may not actually be necessary to set png_ptr->trans here;
+ * we do it for backward compatibility with the way the png_handle_tRNS
+ * function used to do the allocation.
+ */
+
+#ifdef PNG_FREE_ME_SUPPORTED
+ png_free_data(png_ptr, info_ptr, PNG_FREE_TRNS, 0);
+#endif
+
+ /* Changed from num_trans to PNG_MAX_PALETTE_LENGTH in version 1.2.1 */
+ png_ptr->trans = info_ptr->trans = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)PNG_MAX_PALETTE_LENGTH);
+ if (num_trans > 0 && num_trans <= PNG_MAX_PALETTE_LENGTH)
+ png_memcpy(info_ptr->trans, trans, (png_size_t)num_trans);
+ }
+
+ if (trans_values != NULL)
+ {
+ int sample_max = (1 << info_ptr->bit_depth);
+ if ((info_ptr->color_type == PNG_COLOR_TYPE_GRAY &&
+ (int)trans_values->gray > sample_max) ||
+ (info_ptr->color_type == PNG_COLOR_TYPE_RGB &&
+ ((int)trans_values->red > sample_max ||
+ (int)trans_values->green > sample_max ||
+ (int)trans_values->blue > sample_max)))
+ png_warning(png_ptr,
+ "tRNS chunk has out-of-range samples for bit_depth");
+ png_memcpy(&(info_ptr->trans_values), trans_values,
+ png_sizeof(png_color_16));
+ if (num_trans == 0)
+ num_trans = 1;
+ }
+
+ info_ptr->num_trans = (png_uint_16)num_trans;
+ if (num_trans != 0)
+ {
+ info_ptr->valid |= PNG_INFO_tRNS;
+#ifdef PNG_FREE_ME_SUPPORTED
+ info_ptr->free_me |= PNG_FREE_TRNS;
+#else
+ png_ptr->flags |= PNG_FLAG_FREE_TRNS;
+#endif
+ }
+}
+#endif
+
+#ifdef PNG_sPLT_SUPPORTED
+void PNGAPI
+png_set_sPLT(png_structp png_ptr,
+ png_infop info_ptr, png_sPLT_tp entries, int nentries)
+/*
+ * entries - array of png_sPLT_t structures
+ * to be added to the list of palettes
+ * in the info structure.
+ * nentries - number of palette structures to be
+ * added.
+ */
+{
+ png_sPLT_tp np;
+ int i;
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ np = (png_sPLT_tp)png_malloc_warn(png_ptr,
+ (info_ptr->splt_palettes_num + nentries) *
+ (png_uint_32)png_sizeof(png_sPLT_t));
+ if (np == NULL)
+ {
+ png_warning(png_ptr, "No memory for sPLT palettes.");
+ return;
+ }
+
+ png_memcpy(np, info_ptr->splt_palettes,
+ info_ptr->splt_palettes_num * png_sizeof(png_sPLT_t));
+ png_free(png_ptr, info_ptr->splt_palettes);
+ info_ptr->splt_palettes=NULL;
+
+ for (i = 0; i < nentries; i++)
+ {
+ png_sPLT_tp to = np + info_ptr->splt_palettes_num + i;
+ png_sPLT_tp from = entries + i;
+ png_uint_32 length;
+
+ length = png_strlen(from->name) + 1;
+ to->name = (png_charp)png_malloc_warn(png_ptr, length);
+ if (to->name == NULL)
+ {
+ png_warning(png_ptr,
+ "Out of memory while processing sPLT chunk");
+ continue;
+ }
+ png_memcpy(to->name, from->name, length);
+ to->entries = (png_sPLT_entryp)png_malloc_warn(png_ptr,
+ (png_uint_32)(from->nentries * png_sizeof(png_sPLT_entry)));
+ if (to->entries == NULL)
+ {
+ png_warning(png_ptr,
+ "Out of memory while processing sPLT chunk");
+ png_free(png_ptr, to->name);
+ to->name = NULL;
+ continue;
+ }
+ png_memcpy(to->entries, from->entries,
+ from->nentries * png_sizeof(png_sPLT_entry));
+ to->nentries = from->nentries;
+ to->depth = from->depth;
+ }
+
+ info_ptr->splt_palettes = np;
+ info_ptr->splt_palettes_num += nentries;
+ info_ptr->valid |= PNG_INFO_sPLT;
+#ifdef PNG_FREE_ME_SUPPORTED
+ info_ptr->free_me |= PNG_FREE_SPLT;
+#endif
+}
+#endif /* PNG_sPLT_SUPPORTED */
+
+#ifdef PNG_UNKNOWN_CHUNKS_SUPPORTED
+void PNGAPI
+png_set_unknown_chunks(png_structp png_ptr,
+ png_infop info_ptr, png_unknown_chunkp unknowns, int num_unknowns)
+{
+ png_unknown_chunkp np;
+ int i;
+
+ if (png_ptr == NULL || info_ptr == NULL || num_unknowns == 0)
+ return;
+
+ np = (png_unknown_chunkp)png_malloc_warn(png_ptr,
+ (png_uint_32)((info_ptr->unknown_chunks_num + num_unknowns) *
+ png_sizeof(png_unknown_chunk)));
+ if (np == NULL)
+ {
+ png_warning(png_ptr,
+ "Out of memory while processing unknown chunk.");
+ return;
+ }
+
+ png_memcpy(np, info_ptr->unknown_chunks,
+ info_ptr->unknown_chunks_num * png_sizeof(png_unknown_chunk));
+ png_free(png_ptr, info_ptr->unknown_chunks);
+ info_ptr->unknown_chunks = NULL;
+
+ for (i = 0; i < num_unknowns; i++)
+ {
+ png_unknown_chunkp to = np + info_ptr->unknown_chunks_num + i;
+ png_unknown_chunkp from = unknowns + i;
+
+ png_memcpy((png_charp)to->name, (png_charp)from->name,
+ png_sizeof(from->name));
+ to->name[png_sizeof(to->name)-1] = '\0';
+ to->size = from->size;
+ /* Note our location in the read or write sequence */
+ to->location = (png_byte)(png_ptr->mode & 0xff);
+
+ if (from->size == 0)
+ to->data=NULL;
+ else
+ {
+ to->data = (png_bytep)png_malloc_warn(png_ptr,
+ (png_uint_32)from->size);
+ if (to->data == NULL)
+ {
+ png_warning(png_ptr,
+ "Out of memory while processing unknown chunk.");
+ to->size = 0;
+ }
+ else
+ png_memcpy(to->data, from->data, from->size);
+ }
+ }
+
+ info_ptr->unknown_chunks = np;
+ info_ptr->unknown_chunks_num += num_unknowns;
+#ifdef PNG_FREE_ME_SUPPORTED
+ info_ptr->free_me |= PNG_FREE_UNKN;
+#endif
+}
+void PNGAPI
+png_set_unknown_chunk_location(png_structp png_ptr, png_infop info_ptr,
+ int chunk, int location)
+{
+ if (png_ptr != NULL && info_ptr != NULL && chunk >= 0 && chunk <
+ (int)info_ptr->unknown_chunks_num)
+ info_ptr->unknown_chunks[chunk].location = (png_byte)location;
+}
+#endif
+
+#if defined(PNG_1_0_X) || defined(PNG_1_2_X)
+#if defined(PNG_READ_EMPTY_PLTE_SUPPORTED) || \
+ defined(PNG_WRITE_EMPTY_PLTE_SUPPORTED)
+void PNGAPI
+png_permit_empty_plte (png_structp png_ptr, int empty_plte_permitted)
+{
+ /* This function is deprecated in favor of png_permit_mng_features()
+ and will be removed from libpng-1.3.0 */
+
+ png_debug(1, "in png_permit_empty_plte, DEPRECATED.");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->mng_features_permitted = (png_byte)
+ ((png_ptr->mng_features_permitted & (~PNG_FLAG_MNG_EMPTY_PLTE)) |
+ ((empty_plte_permitted & PNG_FLAG_MNG_EMPTY_PLTE)));
+}
+#endif
+#endif
+
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+png_uint_32 PNGAPI
+png_permit_mng_features (png_structp png_ptr, png_uint_32 mng_features)
+{
+ png_debug(1, "in png_permit_mng_features");
+
+ if (png_ptr == NULL)
+ return (png_uint_32)0;
+ png_ptr->mng_features_permitted =
+ (png_byte)(mng_features & PNG_ALL_MNG_FEATURES);
+ return (png_uint_32)png_ptr->mng_features_permitted;
+}
+#endif
+
+#ifdef PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+void PNGAPI
+png_set_keep_unknown_chunks(png_structp png_ptr, int keep, png_bytep
+ chunk_list, int num_chunks)
+{
+ png_bytep new_list, p;
+ int i, old_num_chunks;
+ if (png_ptr == NULL)
+ return;
+ if (num_chunks == 0)
+ {
+ if (keep == PNG_HANDLE_CHUNK_ALWAYS || keep == PNG_HANDLE_CHUNK_IF_SAFE)
+ png_ptr->flags |= PNG_FLAG_KEEP_UNKNOWN_CHUNKS;
+ else
+ png_ptr->flags &= ~PNG_FLAG_KEEP_UNKNOWN_CHUNKS;
+
+ if (keep == PNG_HANDLE_CHUNK_ALWAYS)
+ png_ptr->flags |= PNG_FLAG_KEEP_UNSAFE_CHUNKS;
+ else
+ png_ptr->flags &= ~PNG_FLAG_KEEP_UNSAFE_CHUNKS;
+ return;
+ }
+ if (chunk_list == NULL)
+ return;
+ old_num_chunks = png_ptr->num_chunk_list;
+ new_list=(png_bytep)png_malloc(png_ptr,
+ (png_uint_32)
+ (5*(num_chunks + old_num_chunks)));
+ if (png_ptr->chunk_list != NULL)
+ {
+ png_memcpy(new_list, png_ptr->chunk_list,
+ (png_size_t)(5*old_num_chunks));
+ png_free(png_ptr, png_ptr->chunk_list);
+ png_ptr->chunk_list=NULL;
+ }
+ png_memcpy(new_list + 5*old_num_chunks, chunk_list,
+ (png_size_t)(5*num_chunks));
+ for (p = new_list + 5*old_num_chunks + 4, i = 0; i<num_chunks; i++, p += 5)
+ *p=(png_byte)keep;
+ png_ptr->num_chunk_list = old_num_chunks + num_chunks;
+ png_ptr->chunk_list = new_list;
+#ifdef PNG_FREE_ME_SUPPORTED
+ png_ptr->free_me |= PNG_FREE_LIST;
+#endif
+}
+#endif
+
+#ifdef PNG_READ_USER_CHUNKS_SUPPORTED
+void PNGAPI
+png_set_read_user_chunk_fn(png_structp png_ptr, png_voidp user_chunk_ptr,
+ png_user_chunk_ptr read_user_chunk_fn)
+{
+ png_debug(1, "in png_set_read_user_chunk_fn");
+
+ if (png_ptr == NULL)
+ return;
+
+ png_ptr->read_user_chunk_fn = read_user_chunk_fn;
+ png_ptr->user_chunk_ptr = user_chunk_ptr;
+}
+#endif
+
+#ifdef PNG_INFO_IMAGE_SUPPORTED
+void PNGAPI
+png_set_rows(png_structp png_ptr, png_infop info_ptr, png_bytepp row_pointers)
+{
+ png_debug1(1, "in %s storage function", "rows");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ if (info_ptr->row_pointers && (info_ptr->row_pointers != row_pointers))
+ png_free_data(png_ptr, info_ptr, PNG_FREE_ROWS, 0);
+ info_ptr->row_pointers = row_pointers;
+ if (row_pointers)
+ info_ptr->valid |= PNG_INFO_IDAT;
+}
+#endif
+
+void PNGAPI
+png_set_compression_buffer_size(png_structp png_ptr,
+ png_uint_32 size)
+{
+ if (png_ptr == NULL)
+ return;
+ png_free(png_ptr, png_ptr->zbuf);
+ png_ptr->zbuf_size = (png_size_t)size;
+ png_ptr->zbuf = (png_bytep)png_malloc(png_ptr, size);
+ png_ptr->zstream.next_out = png_ptr->zbuf;
+ png_ptr->zstream.avail_out = (uInt)png_ptr->zbuf_size;
+}
+
+void PNGAPI
+png_set_invalid(png_structp png_ptr, png_infop info_ptr, int mask)
+{
+ if (png_ptr && info_ptr)
+ info_ptr->valid &= ~mask;
+}
+
+
+#ifndef PNG_1_0_X
+#ifdef PNG_ASSEMBLER_CODE_SUPPORTED
+/* Function was added to libpng 1.2.0 and should always exist by default */
+void PNGAPI
+png_set_asm_flags (png_structp png_ptr, png_uint_32 asm_flags)
+{
+/* Obsolete as of libpng-1.2.20 and will be removed from libpng-1.4.0 */
+ if (png_ptr != NULL)
+ png_ptr->asm_flags = 0;
+ asm_flags = asm_flags; /* Quiet the compiler */
+}
+
+/* This function was added to libpng 1.2.0 */
+void PNGAPI
+png_set_mmx_thresholds (png_structp png_ptr,
+ png_byte mmx_bitdepth_threshold,
+ png_uint_32 mmx_rowbytes_threshold)
+{
+/* Obsolete as of libpng-1.2.20 and will be removed from libpng-1.4.0 */
+ if (png_ptr == NULL)
+ return;
+ /* Quiet the compiler */
+ mmx_bitdepth_threshold = mmx_bitdepth_threshold;
+ mmx_rowbytes_threshold = mmx_rowbytes_threshold;
+}
+#endif /* ?PNG_ASSEMBLER_CODE_SUPPORTED */
+
+#ifdef PNG_SET_USER_LIMITS_SUPPORTED
+/* This function was added to libpng 1.2.6 */
+void PNGAPI
+png_set_user_limits (png_structp png_ptr, png_uint_32 user_width_max,
+ png_uint_32 user_height_max)
+{
+ /* Images with dimensions larger than these limits will be
+ * rejected by png_set_IHDR(). To accept any PNG datastream
+ * regardless of dimensions, set both limits to 0x7ffffffL.
+ */
+ if (png_ptr == NULL)
+ return;
+ png_ptr->user_width_max = user_width_max;
+ png_ptr->user_height_max = user_height_max;
+}
+#endif /* ?PNG_SET_USER_LIMITS_SUPPORTED */
+
+
+#ifdef PNG_BENIGN_ERRORS_SUPPORTED
+void PNGAPI
+png_set_benign_errors(png_structp png_ptr, int allowed)
+{
+ png_debug(1, "in png_set_benign_errors");
+
+ if (allowed)
+ png_ptr->flags |= PNG_FLAG_BENIGN_ERRORS_WARN;
+ else
+ png_ptr->flags &= ~PNG_FLAG_BENIGN_ERRORS_WARN;
+}
+#endif /* PNG_BENIGN_ERRORS_SUPPORTED */
+#endif /* ?PNG_1_0_X */
+#endif /* PNG_READ_SUPPORTED || PNG_WRITE_SUPPORTED */
diff --git a/trunk/src/third_party/libpng/pngtrans.c b/trunk/src/third_party/libpng/pngtrans.c
new file mode 100644
index 0000000..6ad9dcf
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngtrans.c
@@ -0,0 +1,699 @@
+
+/* pngtrans.c - transforms the data in a row (used by both readers and writers)
+ *
+ * Last changed in libpng 1.2.41 [December 3, 2009]
+ * Copyright (c) 1998-2009 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ */
+
+#define PNG_INTERNAL
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+#if defined(PNG_READ_SUPPORTED) || defined(PNG_WRITE_SUPPORTED)
+
+#if defined(PNG_READ_BGR_SUPPORTED) || defined(PNG_WRITE_BGR_SUPPORTED)
+/* Turn on BGR-to-RGB mapping */
+void PNGAPI
+png_set_bgr(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_bgr");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->transformations |= PNG_BGR;
+}
+#endif
+
+#if defined(PNG_READ_SWAP_SUPPORTED) || defined(PNG_WRITE_SWAP_SUPPORTED)
+/* Turn on 16 bit byte swapping */
+void PNGAPI
+png_set_swap(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_swap");
+
+ if (png_ptr == NULL)
+ return;
+ if (png_ptr->bit_depth == 16)
+ png_ptr->transformations |= PNG_SWAP_BYTES;
+}
+#endif
+
+#if defined(PNG_READ_PACK_SUPPORTED) || defined(PNG_WRITE_PACK_SUPPORTED)
+/* Turn on pixel packing */
+void PNGAPI
+png_set_packing(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_packing");
+
+ if (png_ptr == NULL)
+ return;
+ if (png_ptr->bit_depth < 8)
+ {
+ png_ptr->transformations |= PNG_PACK;
+ png_ptr->usr_bit_depth = 8;
+ }
+}
+#endif
+
+#if defined(PNG_READ_PACKSWAP_SUPPORTED)||defined(PNG_WRITE_PACKSWAP_SUPPORTED)
+/* Turn on packed pixel swapping */
+void PNGAPI
+png_set_packswap(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_packswap");
+
+ if (png_ptr == NULL)
+ return;
+ if (png_ptr->bit_depth < 8)
+ png_ptr->transformations |= PNG_PACKSWAP;
+}
+#endif
+
+#if defined(PNG_READ_SHIFT_SUPPORTED) || defined(PNG_WRITE_SHIFT_SUPPORTED)
+void PNGAPI
+png_set_shift(png_structp png_ptr, png_color_8p true_bits)
+{
+ png_debug(1, "in png_set_shift");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->transformations |= PNG_SHIFT;
+ png_ptr->shift = *true_bits;
+}
+#endif
+
+#if defined(PNG_READ_INTERLACING_SUPPORTED) || \
+ defined(PNG_WRITE_INTERLACING_SUPPORTED)
+int PNGAPI
+png_set_interlace_handling(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_interlace handling");
+
+ if (png_ptr && png_ptr->interlaced)
+ {
+ png_ptr->transformations |= PNG_INTERLACE;
+ return (7);
+ }
+
+ return (1);
+}
+#endif
+
+#if defined(PNG_READ_FILLER_SUPPORTED) || defined(PNG_WRITE_FILLER_SUPPORTED)
+/* Add a filler byte on read, or remove a filler or alpha byte on write.
+ * The filler type has changed in v0.95 to allow future 2-byte fillers
+ * for 48-bit input data, as well as to avoid problems with some compilers
+ * that don't like bytes as parameters.
+ */
+void PNGAPI
+png_set_filler(png_structp png_ptr, png_uint_32 filler, int filler_loc)
+{
+ png_debug(1, "in png_set_filler");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->transformations |= PNG_FILLER;
+#ifdef PNG_LEGACY_SUPPORTED
+ png_ptr->filler = (png_byte)filler;
+#else
+ png_ptr->filler = (png_uint_16)filler;
+#endif
+ if (filler_loc == PNG_FILLER_AFTER)
+ png_ptr->flags |= PNG_FLAG_FILLER_AFTER;
+ else
+ png_ptr->flags &= ~PNG_FLAG_FILLER_AFTER;
+
+ /* This should probably go in the "do_read_filler" routine.
+ * I attempted to do that in libpng-1.0.1a but that caused problems
+ * so I restored it in libpng-1.0.2a
+ */
+
+ if (png_ptr->color_type == PNG_COLOR_TYPE_RGB)
+ {
+ png_ptr->usr_channels = 4;
+ }
+
+ /* Also I added this in libpng-1.0.2a (what happens when we expand
+ * a less-than-8-bit grayscale to GA? */
+
+ if (png_ptr->color_type == PNG_COLOR_TYPE_GRAY && png_ptr->bit_depth >= 8)
+ {
+ png_ptr->usr_channels = 2;
+ }
+}
+
+#ifndef PNG_1_0_X
+/* Added to libpng-1.2.7 */
+void PNGAPI
+png_set_add_alpha(png_structp png_ptr, png_uint_32 filler, int filler_loc)
+{
+ png_debug(1, "in png_set_add_alpha");
+
+ if (png_ptr == NULL)
+ return;
+ png_set_filler(png_ptr, filler, filler_loc);
+ png_ptr->transformations |= PNG_ADD_ALPHA;
+}
+#endif
+
+#endif
+
+#if defined(PNG_READ_SWAP_ALPHA_SUPPORTED) || \
+ defined(PNG_WRITE_SWAP_ALPHA_SUPPORTED)
+void PNGAPI
+png_set_swap_alpha(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_swap_alpha");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->transformations |= PNG_SWAP_ALPHA;
+}
+#endif
+
+#if defined(PNG_READ_INVERT_ALPHA_SUPPORTED) || \
+ defined(PNG_WRITE_INVERT_ALPHA_SUPPORTED)
+void PNGAPI
+png_set_invert_alpha(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_invert_alpha");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->transformations |= PNG_INVERT_ALPHA;
+}
+#endif
+
+#if defined(PNG_READ_INVERT_SUPPORTED) || defined(PNG_WRITE_INVERT_SUPPORTED)
+void PNGAPI
+png_set_invert_mono(png_structp png_ptr)
+{
+ png_debug(1, "in png_set_invert_mono");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->transformations |= PNG_INVERT_MONO;
+}
+
+/* Invert monochrome grayscale data */
+void /* PRIVATE */
+png_do_invert(png_row_infop row_info, png_bytep row)
+{
+ png_debug(1, "in png_do_invert");
+
+ /* This test removed from libpng version 1.0.13 and 1.2.0:
+ * if (row_info->bit_depth == 1 &&
+ */
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ if (row == NULL || row_info == NULL)
+ return;
+#endif
+ if (row_info->color_type == PNG_COLOR_TYPE_GRAY)
+ {
+ png_bytep rp = row;
+ png_uint_32 i;
+ png_uint_32 istop = row_info->rowbytes;
+
+ for (i = 0; i < istop; i++)
+ {
+ *rp = (png_byte)(~(*rp));
+ rp++;
+ }
+ }
+ else if (row_info->color_type == PNG_COLOR_TYPE_GRAY_ALPHA &&
+ row_info->bit_depth == 8)
+ {
+ png_bytep rp = row;
+ png_uint_32 i;
+ png_uint_32 istop = row_info->rowbytes;
+
+ for (i = 0; i < istop; i+=2)
+ {
+ *rp = (png_byte)(~(*rp));
+ rp+=2;
+ }
+ }
+ else if (row_info->color_type == PNG_COLOR_TYPE_GRAY_ALPHA &&
+ row_info->bit_depth == 16)
+ {
+ png_bytep rp = row;
+ png_uint_32 i;
+ png_uint_32 istop = row_info->rowbytes;
+
+ for (i = 0; i < istop; i+=4)
+ {
+ *rp = (png_byte)(~(*rp));
+ *(rp+1) = (png_byte)(~(*(rp+1)));
+ rp+=4;
+ }
+ }
+}
+#endif
+
+#if defined(PNG_READ_SWAP_SUPPORTED) || defined(PNG_WRITE_SWAP_SUPPORTED)
+/* Swaps byte order on 16 bit depth images */
+void /* PRIVATE */
+png_do_swap(png_row_infop row_info, png_bytep row)
+{
+ png_debug(1, "in png_do_swap");
+
+ if (
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ row != NULL && row_info != NULL &&
+#endif
+ row_info->bit_depth == 16)
+ {
+ png_bytep rp = row;
+ png_uint_32 i;
+ png_uint_32 istop= row_info->width * row_info->channels;
+
+ for (i = 0; i < istop; i++, rp += 2)
+ {
+ png_byte t = *rp;
+ *rp = *(rp + 1);
+ *(rp + 1) = t;
+ }
+ }
+}
+#endif
+
+#if defined(PNG_READ_PACKSWAP_SUPPORTED)||defined(PNG_WRITE_PACKSWAP_SUPPORTED)
+static PNG_CONST png_byte onebppswaptable[256] = {
+ 0x00, 0x80, 0x40, 0xC0, 0x20, 0xA0, 0x60, 0xE0,
+ 0x10, 0x90, 0x50, 0xD0, 0x30, 0xB0, 0x70, 0xF0,
+ 0x08, 0x88, 0x48, 0xC8, 0x28, 0xA8, 0x68, 0xE8,
+ 0x18, 0x98, 0x58, 0xD8, 0x38, 0xB8, 0x78, 0xF8,
+ 0x04, 0x84, 0x44, 0xC4, 0x24, 0xA4, 0x64, 0xE4,
+ 0x14, 0x94, 0x54, 0xD4, 0x34, 0xB4, 0x74, 0xF4,
+ 0x0C, 0x8C, 0x4C, 0xCC, 0x2C, 0xAC, 0x6C, 0xEC,
+ 0x1C, 0x9C, 0x5C, 0xDC, 0x3C, 0xBC, 0x7C, 0xFC,
+ 0x02, 0x82, 0x42, 0xC2, 0x22, 0xA2, 0x62, 0xE2,
+ 0x12, 0x92, 0x52, 0xD2, 0x32, 0xB2, 0x72, 0xF2,
+ 0x0A, 0x8A, 0x4A, 0xCA, 0x2A, 0xAA, 0x6A, 0xEA,
+ 0x1A, 0x9A, 0x5A, 0xDA, 0x3A, 0xBA, 0x7A, 0xFA,
+ 0x06, 0x86, 0x46, 0xC6, 0x26, 0xA6, 0x66, 0xE6,
+ 0x16, 0x96, 0x56, 0xD6, 0x36, 0xB6, 0x76, 0xF6,
+ 0x0E, 0x8E, 0x4E, 0xCE, 0x2E, 0xAE, 0x6E, 0xEE,
+ 0x1E, 0x9E, 0x5E, 0xDE, 0x3E, 0xBE, 0x7E, 0xFE,
+ 0x01, 0x81, 0x41, 0xC1, 0x21, 0xA1, 0x61, 0xE1,
+ 0x11, 0x91, 0x51, 0xD1, 0x31, 0xB1, 0x71, 0xF1,
+ 0x09, 0x89, 0x49, 0xC9, 0x29, 0xA9, 0x69, 0xE9,
+ 0x19, 0x99, 0x59, 0xD9, 0x39, 0xB9, 0x79, 0xF9,
+ 0x05, 0x85, 0x45, 0xC5, 0x25, 0xA5, 0x65, 0xE5,
+ 0x15, 0x95, 0x55, 0xD5, 0x35, 0xB5, 0x75, 0xF5,
+ 0x0D, 0x8D, 0x4D, 0xCD, 0x2D, 0xAD, 0x6D, 0xED,
+ 0x1D, 0x9D, 0x5D, 0xDD, 0x3D, 0xBD, 0x7D, 0xFD,
+ 0x03, 0x83, 0x43, 0xC3, 0x23, 0xA3, 0x63, 0xE3,
+ 0x13, 0x93, 0x53, 0xD3, 0x33, 0xB3, 0x73, 0xF3,
+ 0x0B, 0x8B, 0x4B, 0xCB, 0x2B, 0xAB, 0x6B, 0xEB,
+ 0x1B, 0x9B, 0x5B, 0xDB, 0x3B, 0xBB, 0x7B, 0xFB,
+ 0x07, 0x87, 0x47, 0xC7, 0x27, 0xA7, 0x67, 0xE7,
+ 0x17, 0x97, 0x57, 0xD7, 0x37, 0xB7, 0x77, 0xF7,
+ 0x0F, 0x8F, 0x4F, 0xCF, 0x2F, 0xAF, 0x6F, 0xEF,
+ 0x1F, 0x9F, 0x5F, 0xDF, 0x3F, 0xBF, 0x7F, 0xFF
+};
+
+static PNG_CONST png_byte twobppswaptable[256] = {
+ 0x00, 0x40, 0x80, 0xC0, 0x10, 0x50, 0x90, 0xD0,
+ 0x20, 0x60, 0xA0, 0xE0, 0x30, 0x70, 0xB0, 0xF0,
+ 0x04, 0x44, 0x84, 0xC4, 0x14, 0x54, 0x94, 0xD4,
+ 0x24, 0x64, 0xA4, 0xE4, 0x34, 0x74, 0xB4, 0xF4,
+ 0x08, 0x48, 0x88, 0xC8, 0x18, 0x58, 0x98, 0xD8,
+ 0x28, 0x68, 0xA8, 0xE8, 0x38, 0x78, 0xB8, 0xF8,
+ 0x0C, 0x4C, 0x8C, 0xCC, 0x1C, 0x5C, 0x9C, 0xDC,
+ 0x2C, 0x6C, 0xAC, 0xEC, 0x3C, 0x7C, 0xBC, 0xFC,
+ 0x01, 0x41, 0x81, 0xC1, 0x11, 0x51, 0x91, 0xD1,
+ 0x21, 0x61, 0xA1, 0xE1, 0x31, 0x71, 0xB1, 0xF1,
+ 0x05, 0x45, 0x85, 0xC5, 0x15, 0x55, 0x95, 0xD5,
+ 0x25, 0x65, 0xA5, 0xE5, 0x35, 0x75, 0xB5, 0xF5,
+ 0x09, 0x49, 0x89, 0xC9, 0x19, 0x59, 0x99, 0xD9,
+ 0x29, 0x69, 0xA9, 0xE9, 0x39, 0x79, 0xB9, 0xF9,
+ 0x0D, 0x4D, 0x8D, 0xCD, 0x1D, 0x5D, 0x9D, 0xDD,
+ 0x2D, 0x6D, 0xAD, 0xED, 0x3D, 0x7D, 0xBD, 0xFD,
+ 0x02, 0x42, 0x82, 0xC2, 0x12, 0x52, 0x92, 0xD2,
+ 0x22, 0x62, 0xA2, 0xE2, 0x32, 0x72, 0xB2, 0xF2,
+ 0x06, 0x46, 0x86, 0xC6, 0x16, 0x56, 0x96, 0xD6,
+ 0x26, 0x66, 0xA6, 0xE6, 0x36, 0x76, 0xB6, 0xF6,
+ 0x0A, 0x4A, 0x8A, 0xCA, 0x1A, 0x5A, 0x9A, 0xDA,
+ 0x2A, 0x6A, 0xAA, 0xEA, 0x3A, 0x7A, 0xBA, 0xFA,
+ 0x0E, 0x4E, 0x8E, 0xCE, 0x1E, 0x5E, 0x9E, 0xDE,
+ 0x2E, 0x6E, 0xAE, 0xEE, 0x3E, 0x7E, 0xBE, 0xFE,
+ 0x03, 0x43, 0x83, 0xC3, 0x13, 0x53, 0x93, 0xD3,
+ 0x23, 0x63, 0xA3, 0xE3, 0x33, 0x73, 0xB3, 0xF3,
+ 0x07, 0x47, 0x87, 0xC7, 0x17, 0x57, 0x97, 0xD7,
+ 0x27, 0x67, 0xA7, 0xE7, 0x37, 0x77, 0xB7, 0xF7,
+ 0x0B, 0x4B, 0x8B, 0xCB, 0x1B, 0x5B, 0x9B, 0xDB,
+ 0x2B, 0x6B, 0xAB, 0xEB, 0x3B, 0x7B, 0xBB, 0xFB,
+ 0x0F, 0x4F, 0x8F, 0xCF, 0x1F, 0x5F, 0x9F, 0xDF,
+ 0x2F, 0x6F, 0xAF, 0xEF, 0x3F, 0x7F, 0xBF, 0xFF
+};
+
+static PNG_CONST png_byte fourbppswaptable[256] = {
+ 0x00, 0x10, 0x20, 0x30, 0x40, 0x50, 0x60, 0x70,
+ 0x80, 0x90, 0xA0, 0xB0, 0xC0, 0xD0, 0xE0, 0xF0,
+ 0x01, 0x11, 0x21, 0x31, 0x41, 0x51, 0x61, 0x71,
+ 0x81, 0x91, 0xA1, 0xB1, 0xC1, 0xD1, 0xE1, 0xF1,
+ 0x02, 0x12, 0x22, 0x32, 0x42, 0x52, 0x62, 0x72,
+ 0x82, 0x92, 0xA2, 0xB2, 0xC2, 0xD2, 0xE2, 0xF2,
+ 0x03, 0x13, 0x23, 0x33, 0x43, 0x53, 0x63, 0x73,
+ 0x83, 0x93, 0xA3, 0xB3, 0xC3, 0xD3, 0xE3, 0xF3,
+ 0x04, 0x14, 0x24, 0x34, 0x44, 0x54, 0x64, 0x74,
+ 0x84, 0x94, 0xA4, 0xB4, 0xC4, 0xD4, 0xE4, 0xF4,
+ 0x05, 0x15, 0x25, 0x35, 0x45, 0x55, 0x65, 0x75,
+ 0x85, 0x95, 0xA5, 0xB5, 0xC5, 0xD5, 0xE5, 0xF5,
+ 0x06, 0x16, 0x26, 0x36, 0x46, 0x56, 0x66, 0x76,
+ 0x86, 0x96, 0xA6, 0xB6, 0xC6, 0xD6, 0xE6, 0xF6,
+ 0x07, 0x17, 0x27, 0x37, 0x47, 0x57, 0x67, 0x77,
+ 0x87, 0x97, 0xA7, 0xB7, 0xC7, 0xD7, 0xE7, 0xF7,
+ 0x08, 0x18, 0x28, 0x38, 0x48, 0x58, 0x68, 0x78,
+ 0x88, 0x98, 0xA8, 0xB8, 0xC8, 0xD8, 0xE8, 0xF8,
+ 0x09, 0x19, 0x29, 0x39, 0x49, 0x59, 0x69, 0x79,
+ 0x89, 0x99, 0xA9, 0xB9, 0xC9, 0xD9, 0xE9, 0xF9,
+ 0x0A, 0x1A, 0x2A, 0x3A, 0x4A, 0x5A, 0x6A, 0x7A,
+ 0x8A, 0x9A, 0xAA, 0xBA, 0xCA, 0xDA, 0xEA, 0xFA,
+ 0x0B, 0x1B, 0x2B, 0x3B, 0x4B, 0x5B, 0x6B, 0x7B,
+ 0x8B, 0x9B, 0xAB, 0xBB, 0xCB, 0xDB, 0xEB, 0xFB,
+ 0x0C, 0x1C, 0x2C, 0x3C, 0x4C, 0x5C, 0x6C, 0x7C,
+ 0x8C, 0x9C, 0xAC, 0xBC, 0xCC, 0xDC, 0xEC, 0xFC,
+ 0x0D, 0x1D, 0x2D, 0x3D, 0x4D, 0x5D, 0x6D, 0x7D,
+ 0x8D, 0x9D, 0xAD, 0xBD, 0xCD, 0xDD, 0xED, 0xFD,
+ 0x0E, 0x1E, 0x2E, 0x3E, 0x4E, 0x5E, 0x6E, 0x7E,
+ 0x8E, 0x9E, 0xAE, 0xBE, 0xCE, 0xDE, 0xEE, 0xFE,
+ 0x0F, 0x1F, 0x2F, 0x3F, 0x4F, 0x5F, 0x6F, 0x7F,
+ 0x8F, 0x9F, 0xAF, 0xBF, 0xCF, 0xDF, 0xEF, 0xFF
+};
+
+/* Swaps pixel packing order within bytes */
+void /* PRIVATE */
+png_do_packswap(png_row_infop row_info, png_bytep row)
+{
+ png_debug(1, "in png_do_packswap");
+
+ if (
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ row != NULL && row_info != NULL &&
+#endif
+ row_info->bit_depth < 8)
+ {
+ png_bytep rp, end, table;
+
+ end = row + row_info->rowbytes;
+
+ if (row_info->bit_depth == 1)
+ table = (png_bytep)onebppswaptable;
+ else if (row_info->bit_depth == 2)
+ table = (png_bytep)twobppswaptable;
+ else if (row_info->bit_depth == 4)
+ table = (png_bytep)fourbppswaptable;
+ else
+ return;
+
+ for (rp = row; rp < end; rp++)
+ *rp = table[*rp];
+ }
+}
+#endif /* PNG_READ_PACKSWAP_SUPPORTED or PNG_WRITE_PACKSWAP_SUPPORTED */
+
+#if defined(PNG_WRITE_FILLER_SUPPORTED) || \
+ defined(PNG_READ_STRIP_ALPHA_SUPPORTED)
+/* Remove filler or alpha byte(s) */
+void /* PRIVATE */
+png_do_strip_filler(png_row_infop row_info, png_bytep row, png_uint_32 flags)
+{
+ png_debug(1, "in png_do_strip_filler");
+
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ if (row != NULL && row_info != NULL)
+#endif
+ {
+ png_bytep sp=row;
+ png_bytep dp=row;
+ png_uint_32 row_width=row_info->width;
+ png_uint_32 i;
+
+ if ((row_info->color_type == PNG_COLOR_TYPE_RGB ||
+ (row_info->color_type == PNG_COLOR_TYPE_RGB_ALPHA &&
+ (flags & PNG_FLAG_STRIP_ALPHA))) &&
+ row_info->channels == 4)
+ {
+ if (row_info->bit_depth == 8)
+ {
+ /* This converts from RGBX or RGBA to RGB */
+ if (flags & PNG_FLAG_FILLER_AFTER)
+ {
+ dp+=3; sp+=4;
+ for (i = 1; i < row_width; i++)
+ {
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ sp++;
+ }
+ }
+ /* This converts from XRGB or ARGB to RGB */
+ else
+ {
+ for (i = 0; i < row_width; i++)
+ {
+ sp++;
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ }
+ }
+ row_info->pixel_depth = 24;
+ row_info->rowbytes = row_width * 3;
+ }
+ else /* if (row_info->bit_depth == 16) */
+ {
+ if (flags & PNG_FLAG_FILLER_AFTER)
+ {
+ /* This converts from RRGGBBXX or RRGGBBAA to RRGGBB */
+ sp += 8; dp += 6;
+ for (i = 1; i < row_width; i++)
+ {
+ /* This could be (although png_memcpy is probably slower):
+ png_memcpy(dp, sp, 6);
+ sp += 8;
+ dp += 6;
+ */
+
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ sp += 2;
+ }
+ }
+ else
+ {
+ /* This converts from XXRRGGBB or AARRGGBB to RRGGBB */
+ for (i = 0; i < row_width; i++)
+ {
+ /* This could be (although png_memcpy is probably slower):
+ png_memcpy(dp, sp, 6);
+ sp += 8;
+ dp += 6;
+ */
+
+ sp+=2;
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ }
+ }
+ row_info->pixel_depth = 48;
+ row_info->rowbytes = row_width * 6;
+ }
+ row_info->channels = 3;
+ }
+ else if ((row_info->color_type == PNG_COLOR_TYPE_GRAY ||
+ (row_info->color_type == PNG_COLOR_TYPE_GRAY_ALPHA &&
+ (flags & PNG_FLAG_STRIP_ALPHA))) &&
+ row_info->channels == 2)
+ {
+ if (row_info->bit_depth == 8)
+ {
+ /* This converts from GX or GA to G */
+ if (flags & PNG_FLAG_FILLER_AFTER)
+ {
+ for (i = 0; i < row_width; i++)
+ {
+ *dp++ = *sp++;
+ sp++;
+ }
+ }
+ /* This converts from XG or AG to G */
+ else
+ {
+ for (i = 0; i < row_width; i++)
+ {
+ sp++;
+ *dp++ = *sp++;
+ }
+ }
+ row_info->pixel_depth = 8;
+ row_info->rowbytes = row_width;
+ }
+ else /* if (row_info->bit_depth == 16) */
+ {
+ if (flags & PNG_FLAG_FILLER_AFTER)
+ {
+ /* This converts from GGXX or GGAA to GG */
+ sp += 4; dp += 2;
+ for (i = 1; i < row_width; i++)
+ {
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ sp += 2;
+ }
+ }
+ else
+ {
+ /* This converts from XXGG or AAGG to GG */
+ for (i = 0; i < row_width; i++)
+ {
+ sp += 2;
+ *dp++ = *sp++;
+ *dp++ = *sp++;
+ }
+ }
+ row_info->pixel_depth = 16;
+ row_info->rowbytes = row_width * 2;
+ }
+ row_info->channels = 1;
+ }
+ if (flags & PNG_FLAG_STRIP_ALPHA)
+ row_info->color_type &= ~PNG_COLOR_MASK_ALPHA;
+ }
+}
+#endif
+
+#if defined(PNG_READ_BGR_SUPPORTED) || defined(PNG_WRITE_BGR_SUPPORTED)
+/* Swaps red and blue bytes within a pixel */
+void /* PRIVATE */
+png_do_bgr(png_row_infop row_info, png_bytep row)
+{
+ png_debug(1, "in png_do_bgr");
+
+ if (
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ row != NULL && row_info != NULL &&
+#endif
+ (row_info->color_type & PNG_COLOR_MASK_COLOR))
+ {
+ png_uint_32 row_width = row_info->width;
+ if (row_info->bit_depth == 8)
+ {
+ if (row_info->color_type == PNG_COLOR_TYPE_RGB)
+ {
+ png_bytep rp;
+ png_uint_32 i;
+
+ for (i = 0, rp = row; i < row_width; i++, rp += 3)
+ {
+ png_byte save = *rp;
+ *rp = *(rp + 2);
+ *(rp + 2) = save;
+ }
+ }
+ else if (row_info->color_type == PNG_COLOR_TYPE_RGB_ALPHA)
+ {
+ png_bytep rp;
+ png_uint_32 i;
+
+ for (i = 0, rp = row; i < row_width; i++, rp += 4)
+ {
+ png_byte save = *rp;
+ *rp = *(rp + 2);
+ *(rp + 2) = save;
+ }
+ }
+ }
+ else if (row_info->bit_depth == 16)
+ {
+ if (row_info->color_type == PNG_COLOR_TYPE_RGB)
+ {
+ png_bytep rp;
+ png_uint_32 i;
+
+ for (i = 0, rp = row; i < row_width; i++, rp += 6)
+ {
+ png_byte save = *rp;
+ *rp = *(rp + 4);
+ *(rp + 4) = save;
+ save = *(rp + 1);
+ *(rp + 1) = *(rp + 5);
+ *(rp + 5) = save;
+ }
+ }
+ else if (row_info->color_type == PNG_COLOR_TYPE_RGB_ALPHA)
+ {
+ png_bytep rp;
+ png_uint_32 i;
+
+ for (i = 0, rp = row; i < row_width; i++, rp += 8)
+ {
+ png_byte save = *rp;
+ *rp = *(rp + 4);
+ *(rp + 4) = save;
+ save = *(rp + 1);
+ *(rp + 1) = *(rp + 5);
+ *(rp + 5) = save;
+ }
+ }
+ }
+ }
+}
+#endif /* PNG_READ_BGR_SUPPORTED or PNG_WRITE_BGR_SUPPORTED */
+
+#if defined(PNG_READ_USER_TRANSFORM_SUPPORTED) || \
+ defined(PNG_LEGACY_SUPPORTED) || \
+ defined(PNG_WRITE_USER_TRANSFORM_SUPPORTED)
+void PNGAPI
+png_set_user_transform_info(png_structp png_ptr, png_voidp
+ user_transform_ptr, int user_transform_depth, int user_transform_channels)
+{
+ png_debug(1, "in png_set_user_transform_info");
+
+ if (png_ptr == NULL)
+ return;
+#ifdef PNG_USER_TRANSFORM_PTR_SUPPORTED
+ png_ptr->user_transform_ptr = user_transform_ptr;
+ png_ptr->user_transform_depth = (png_byte)user_transform_depth;
+ png_ptr->user_transform_channels = (png_byte)user_transform_channels;
+#else
+ if (user_transform_ptr || user_transform_depth || user_transform_channels)
+ png_warning(png_ptr,
+ "This version of libpng does not support user transform info");
+#endif
+}
+#endif
+
+/* This function returns a pointer to the user_transform_ptr associated with
+ * the user transform functions. The application should free any memory
+ * associated with this pointer before png_write_destroy and png_read_destroy
+ * are called.
+ */
+png_voidp PNGAPI
+png_get_user_transform_ptr(png_structp png_ptr)
+{
+ if (png_ptr == NULL)
+ return (NULL);
+#ifdef PNG_USER_TRANSFORM_PTR_SUPPORTED
+ return ((png_voidp)png_ptr->user_transform_ptr);
+#else
+ return (NULL);
+#endif
+}
+#endif /* PNG_READ_SUPPORTED || PNG_WRITE_SUPPORTED */
diff --git a/trunk/src/third_party/libpng/pngvcrd.c b/trunk/src/third_party/libpng/pngvcrd.c
new file mode 100644
index 0000000..ce4233e
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngvcrd.c
@@ -0,0 +1 @@
+/* pnggvrd.c was removed from libpng-1.2.20. */
diff --git a/trunk/src/third_party/libpng/pngwio.c b/trunk/src/third_party/libpng/pngwio.c
new file mode 100644
index 0000000..44e5ea9
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngwio.c
@@ -0,0 +1,260 @@
+
+/* pngwio.c - functions for data output
+ *
+ * Last changed in libpng 1.2.41 [December 3, 2009]
+ * Copyright (c) 1998-2009 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ *
+ * This file provides a location for all output. Users who need
+ * special handling are expected to write functions that have the same
+ * arguments as these and perform similar functions, but that possibly
+ * use different output methods. Note that you shouldn't change these
+ * functions, but rather write replacement functions and then change
+ * them at run time with png_set_write_fn(...).
+ */
+
+#define PNG_INTERNAL
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+#ifdef PNG_WRITE_SUPPORTED
+
+/* Write the data to whatever output you are using. The default routine
+ * writes to a file pointer. Note that this routine sometimes gets called
+ * with very small lengths, so you should implement some kind of simple
+ * buffering if you are using unbuffered writes. This should never be asked
+ * to write more than 64K on a 16 bit machine.
+ */
+
+void /* PRIVATE */
+png_write_data(png_structp png_ptr, png_bytep data, png_size_t length)
+{
+ if (png_ptr->write_data_fn != NULL )
+ (*(png_ptr->write_data_fn))(png_ptr, data, length);
+ else
+ png_error(png_ptr, "Call to NULL write function");
+}
+
+#ifdef PNG_STDIO_SUPPORTED
+/* This is the function that does the actual writing of data. If you are
+ * not writing to a standard C stream, you should create a replacement
+ * write_data function and use it at run time with png_set_write_fn(), rather
+ * than changing the library.
+ */
+#ifndef USE_FAR_KEYWORD
+void PNGAPI
+png_default_write_data(png_structp png_ptr, png_bytep data, png_size_t length)
+{
+ png_uint_32 check;
+
+ if (png_ptr == NULL)
+ return;
+#ifdef _WIN32_WCE
+ if ( !WriteFile((HANDLE)(png_ptr->io_ptr), data, length, &check, NULL) )
+ check = 0;
+#else
+ check = fwrite(data, 1, length, (png_FILE_p)(png_ptr->io_ptr));
+#endif
+ if (check != length)
+ png_error(png_ptr, "Write Error");
+}
+#else
+/* This is the model-independent version. Since the standard I/O library
+ * can't handle far buffers in the medium and small models, we have to copy
+ * the data.
+ */
+
+#define NEAR_BUF_SIZE 1024
+#define MIN(a,b) (a <= b ? a : b)
+
+void PNGAPI
+png_default_write_data(png_structp png_ptr, png_bytep data, png_size_t length)
+{
+ png_uint_32 check;
+ png_byte *near_data; /* Needs to be "png_byte *" instead of "png_bytep" */
+ png_FILE_p io_ptr;
+
+ if (png_ptr == NULL)
+ return;
+ /* Check if data really is near. If so, use usual code. */
+ near_data = (png_byte *)CVT_PTR_NOCHECK(data);
+ io_ptr = (png_FILE_p)CVT_PTR(png_ptr->io_ptr);
+ if ((png_bytep)near_data == data)
+ {
+#ifdef _WIN32_WCE
+ if ( !WriteFile(io_ptr, near_data, length, &check, NULL) )
+ check = 0;
+#else
+ check = fwrite(near_data, 1, length, io_ptr);
+#endif
+ }
+ else
+ {
+ png_byte buf[NEAR_BUF_SIZE];
+ png_size_t written, remaining, err;
+ check = 0;
+ remaining = length;
+ do
+ {
+ written = MIN(NEAR_BUF_SIZE, remaining);
+ png_memcpy(buf, data, written); /* Copy far buffer to near buffer */
+#ifdef _WIN32_WCE
+ if ( !WriteFile(io_ptr, buf, written, &err, NULL) )
+ err = 0;
+#else
+ err = fwrite(buf, 1, written, io_ptr);
+#endif
+ if (err != written)
+ break;
+
+ else
+ check += err;
+
+ data += written;
+ remaining -= written;
+ }
+ while (remaining != 0);
+ }
+ if (check != length)
+ png_error(png_ptr, "Write Error");
+}
+
+#endif
+#endif
+
+/* This function is called to output any data pending writing (normally
+ * to disk). After png_flush is called, there should be no data pending
+ * writing in any buffers.
+ */
+#ifdef PNG_WRITE_FLUSH_SUPPORTED
+void /* PRIVATE */
+png_flush(png_structp png_ptr)
+{
+ if (png_ptr->output_flush_fn != NULL)
+ (*(png_ptr->output_flush_fn))(png_ptr);
+}
+
+#ifdef PNG_STDIO_SUPPORTED
+void PNGAPI
+png_default_flush(png_structp png_ptr)
+{
+#ifndef _WIN32_WCE
+ png_FILE_p io_ptr;
+#endif
+ if (png_ptr == NULL)
+ return;
+#ifndef _WIN32_WCE
+ io_ptr = (png_FILE_p)CVT_PTR((png_ptr->io_ptr));
+ fflush(io_ptr);
+#endif
+}
+#endif
+#endif
+
+/* This function allows the application to supply new output functions for
+ * libpng if standard C streams aren't being used.
+ *
+ * This function takes as its arguments:
+ * png_ptr - pointer to a png output data structure
+ * io_ptr - pointer to user supplied structure containing info about
+ * the output functions. May be NULL.
+ * write_data_fn - pointer to a new output function that takes as its
+ * arguments a pointer to a png_struct, a pointer to
+ * data to be written, and a 32-bit unsigned int that is
+ * the number of bytes to be written. The new write
+ * function should call png_error(png_ptr, "Error msg")
+ * to exit and output any fatal error messages. May be
+ * NULL, in which case libpng's default function will
+ * be used.
+ * flush_data_fn - pointer to a new flush function that takes as its
+ * arguments a pointer to a png_struct. After a call to
+ * the flush function, there should be no data in any buffers
+ * or pending transmission. If the output method doesn't do
+ * any buffering of output, a function prototype must still be
+ * supplied although it doesn't have to do anything. If
+ * PNG_WRITE_FLUSH_SUPPORTED is not defined at libpng compile
+ * time, output_flush_fn will be ignored, although it must be
+ * supplied for compatibility. May be NULL, in which case
+ * libpng's default function will be used, if
+ * PNG_WRITE_FLUSH_SUPPORTED is defined. This is not
+ * a good idea if io_ptr does not point to a standard
+ * *FILE structure.
+ */
+void PNGAPI
+png_set_write_fn(png_structp png_ptr, png_voidp io_ptr,
+ png_rw_ptr write_data_fn, png_flush_ptr output_flush_fn)
+{
+ if (png_ptr == NULL)
+ return;
+
+ png_ptr->io_ptr = io_ptr;
+
+#ifdef PNG_STDIO_SUPPORTED
+ if (write_data_fn != NULL)
+ png_ptr->write_data_fn = write_data_fn;
+
+ else
+ png_ptr->write_data_fn = png_default_write_data;
+#else
+ png_ptr->write_data_fn = write_data_fn;
+#endif
+
+#ifdef PNG_WRITE_FLUSH_SUPPORTED
+#ifdef PNG_STDIO_SUPPORTED
+ if (output_flush_fn != NULL)
+ png_ptr->output_flush_fn = output_flush_fn;
+
+ else
+ png_ptr->output_flush_fn = png_default_flush;
+#else
+ png_ptr->output_flush_fn = output_flush_fn;
+#endif
+#endif /* PNG_WRITE_FLUSH_SUPPORTED */
+
+ /* It is an error to read while writing a png file */
+ if (png_ptr->read_data_fn != NULL)
+ {
+ png_ptr->read_data_fn = NULL;
+ png_warning(png_ptr,
+ "Attempted to set both read_data_fn and write_data_fn in");
+ png_warning(png_ptr,
+ "the same structure. Resetting read_data_fn to NULL.");
+ }
+}
+
+#ifdef USE_FAR_KEYWORD
+#ifdef _MSC_VER
+void *png_far_to_near(png_structp png_ptr, png_voidp ptr, int check)
+{
+ void *near_ptr;
+ void FAR *far_ptr;
+ FP_OFF(near_ptr) = FP_OFF(ptr);
+ far_ptr = (void FAR *)near_ptr;
+
+ if (check != 0)
+ if (FP_SEG(ptr) != FP_SEG(far_ptr))
+ png_error(png_ptr, "segment lost in conversion");
+
+ return(near_ptr);
+}
+# else
+void *png_far_to_near(png_structp png_ptr, png_voidp ptr, int check)
+{
+ void *near_ptr;
+ void FAR *far_ptr;
+ near_ptr = (void FAR *)ptr;
+ far_ptr = (void FAR *)near_ptr;
+
+ if (check != 0)
+ if (far_ptr != ptr)
+ png_error(png_ptr, "segment lost in conversion");
+
+ return(near_ptr);
+}
+# endif
+# endif
+#endif /* PNG_WRITE_SUPPORTED */
diff --git a/trunk/src/third_party/libpng/pngwrite.c b/trunk/src/third_party/libpng/pngwrite.c
new file mode 100644
index 0000000..e411e81
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngwrite.c
@@ -0,0 +1,1592 @@
+
+/* pngwrite.c - general routines to write a PNG file
+ *
+ * Last changed in libpng 1.2.42 [January 3, 2010]
+ * Copyright (c) 1998-2010 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ */
+
+/* Get internal access to png.h */
+#define PNG_INTERNAL
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+#ifdef PNG_WRITE_SUPPORTED
+
+/* Writes all the PNG information. This is the suggested way to use the
+ * library. If you have a new chunk to add, make a function to write it,
+ * and put it in the correct location here. If you want the chunk written
+ * after the image data, put it in png_write_end(). I strongly encourage
+ * you to supply a PNG_INFO_ flag, and check info_ptr->valid before writing
+ * the chunk, as that will keep the code from breaking if you want to just
+ * write a plain PNG file. If you have long comments, I suggest writing
+ * them in png_write_end(), and compressing them.
+ */
+void PNGAPI
+png_write_info_before_PLTE(png_structp png_ptr, png_infop info_ptr)
+{
+ png_debug(1, "in png_write_info_before_PLTE");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+ if (!(png_ptr->mode & PNG_WROTE_INFO_BEFORE_PLTE))
+ {
+ /* Write PNG signature */
+ png_write_sig(png_ptr);
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+ if ((png_ptr->mode&PNG_HAVE_PNG_SIGNATURE) && \
+ (png_ptr->mng_features_permitted))
+ {
+ png_warning(png_ptr, "MNG features are not allowed in a PNG datastream");
+ png_ptr->mng_features_permitted = 0;
+ }
+#endif
+ /* Write IHDR information. */
+ png_write_IHDR(png_ptr, info_ptr->width, info_ptr->height,
+ info_ptr->bit_depth, info_ptr->color_type, info_ptr->compression_type,
+ info_ptr->filter_type,
+#ifdef PNG_WRITE_INTERLACING_SUPPORTED
+ info_ptr->interlace_type);
+#else
+ 0);
+#endif
+ /* The rest of these check to see if the valid field has the appropriate
+ * flag set, and if it does, writes the chunk.
+ */
+#ifdef PNG_WRITE_gAMA_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_gAMA)
+ {
+# ifdef PNG_FLOATING_POINT_SUPPORTED
+ png_write_gAMA(png_ptr, info_ptr->gamma);
+#else
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ png_write_gAMA_fixed(png_ptr, info_ptr->int_gamma);
+# endif
+#endif
+ }
+#endif
+#ifdef PNG_WRITE_sRGB_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_sRGB)
+ png_write_sRGB(png_ptr, (int)info_ptr->srgb_intent);
+#endif
+#ifdef PNG_WRITE_iCCP_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_iCCP)
+ png_write_iCCP(png_ptr, info_ptr->iccp_name, PNG_COMPRESSION_TYPE_BASE,
+ info_ptr->iccp_profile, (int)info_ptr->iccp_proflen);
+#endif
+#ifdef PNG_WRITE_sBIT_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_sBIT)
+ png_write_sBIT(png_ptr, &(info_ptr->sig_bit), info_ptr->color_type);
+#endif
+#ifdef PNG_WRITE_cHRM_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_cHRM)
+ {
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+ png_write_cHRM(png_ptr,
+ info_ptr->x_white, info_ptr->y_white,
+ info_ptr->x_red, info_ptr->y_red,
+ info_ptr->x_green, info_ptr->y_green,
+ info_ptr->x_blue, info_ptr->y_blue);
+#else
+# ifdef PNG_FIXED_POINT_SUPPORTED
+ png_write_cHRM_fixed(png_ptr,
+ info_ptr->int_x_white, info_ptr->int_y_white,
+ info_ptr->int_x_red, info_ptr->int_y_red,
+ info_ptr->int_x_green, info_ptr->int_y_green,
+ info_ptr->int_x_blue, info_ptr->int_y_blue);
+# endif
+#endif
+ }
+#endif
+#ifdef PNG_WRITE_UNKNOWN_CHUNKS_SUPPORTED
+ if (info_ptr->unknown_chunks_num)
+ {
+ png_unknown_chunk *up;
+
+ png_debug(5, "writing extra chunks");
+
+ for (up = info_ptr->unknown_chunks;
+ up < info_ptr->unknown_chunks + info_ptr->unknown_chunks_num;
+ up++)
+ {
+ int keep = png_handle_as_unknown(png_ptr, up->name);
+ if (keep != PNG_HANDLE_CHUNK_NEVER &&
+ up->location && !(up->location & PNG_HAVE_PLTE) &&
+ !(up->location & PNG_HAVE_IDAT) &&
+ ((up->name[3] & 0x20) || keep == PNG_HANDLE_CHUNK_ALWAYS ||
+ (png_ptr->flags & PNG_FLAG_KEEP_UNSAFE_CHUNKS)))
+ {
+ if (up->size == 0)
+ png_warning(png_ptr, "Writing zero-length unknown chunk");
+ png_write_chunk(png_ptr, up->name, up->data, up->size);
+ }
+ }
+ }
+#endif
+ png_ptr->mode |= PNG_WROTE_INFO_BEFORE_PLTE;
+ }
+}
+
+void PNGAPI
+png_write_info(png_structp png_ptr, png_infop info_ptr)
+{
+#if defined(PNG_WRITE_TEXT_SUPPORTED) || defined(PNG_WRITE_sPLT_SUPPORTED)
+ int i;
+#endif
+
+ png_debug(1, "in png_write_info");
+
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ png_write_info_before_PLTE(png_ptr, info_ptr);
+
+ if (info_ptr->valid & PNG_INFO_PLTE)
+ png_write_PLTE(png_ptr, info_ptr->palette,
+ (png_uint_32)info_ptr->num_palette);
+ else if (info_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ png_error(png_ptr, "Valid palette required for paletted images");
+
+#ifdef PNG_WRITE_tRNS_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_tRNS)
+ {
+#ifdef PNG_WRITE_INVERT_ALPHA_SUPPORTED
+ /* Invert the alpha channel (in tRNS) */
+ if ((png_ptr->transformations & PNG_INVERT_ALPHA) &&
+ info_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ int j;
+ for (j = 0; j<(int)info_ptr->num_trans; j++)
+ info_ptr->trans[j] = (png_byte)(255 - info_ptr->trans[j]);
+ }
+#endif
+ png_write_tRNS(png_ptr, info_ptr->trans, &(info_ptr->trans_values),
+ info_ptr->num_trans, info_ptr->color_type);
+ }
+#endif
+#ifdef PNG_WRITE_bKGD_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_bKGD)
+ png_write_bKGD(png_ptr, &(info_ptr->background), info_ptr->color_type);
+#endif
+#ifdef PNG_WRITE_hIST_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_hIST)
+ png_write_hIST(png_ptr, info_ptr->hist, info_ptr->num_palette);
+#endif
+#ifdef PNG_WRITE_oFFs_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_oFFs)
+ png_write_oFFs(png_ptr, info_ptr->x_offset, info_ptr->y_offset,
+ info_ptr->offset_unit_type);
+#endif
+#ifdef PNG_WRITE_pCAL_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_pCAL)
+ png_write_pCAL(png_ptr, info_ptr->pcal_purpose, info_ptr->pcal_X0,
+ info_ptr->pcal_X1, info_ptr->pcal_type, info_ptr->pcal_nparams,
+ info_ptr->pcal_units, info_ptr->pcal_params);
+#endif
+
+#ifdef PNG_sCAL_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_sCAL)
+#ifdef PNG_WRITE_sCAL_SUPPORTED
+#if defined(PNG_FLOATING_POINT_SUPPORTED) && defined(PNG_STDIO_SUPPORTED)
+ png_write_sCAL(png_ptr, (int)info_ptr->scal_unit,
+ info_ptr->scal_pixel_width, info_ptr->scal_pixel_height);
+#else /* !FLOATING_POINT */
+#ifdef PNG_FIXED_POINT_SUPPORTED
+ png_write_sCAL_s(png_ptr, (int)info_ptr->scal_unit,
+ info_ptr->scal_s_width, info_ptr->scal_s_height);
+#endif /* FIXED_POINT */
+#endif /* FLOATING_POINT */
+#else /* !WRITE_sCAL */
+ png_warning(png_ptr,
+ "png_write_sCAL not supported; sCAL chunk not written.");
+#endif /* WRITE_sCAL */
+#endif /* sCAL */
+
+#ifdef PNG_WRITE_pHYs_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_pHYs)
+ png_write_pHYs(png_ptr, info_ptr->x_pixels_per_unit,
+ info_ptr->y_pixels_per_unit, info_ptr->phys_unit_type);
+#endif /* pHYs */
+
+#ifdef PNG_WRITE_tIME_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_tIME)
+ {
+ png_write_tIME(png_ptr, &(info_ptr->mod_time));
+ png_ptr->mode |= PNG_WROTE_tIME;
+ }
+#endif /* tIME */
+
+#ifdef PNG_WRITE_sPLT_SUPPORTED
+ if (info_ptr->valid & PNG_INFO_sPLT)
+ for (i = 0; i < (int)info_ptr->splt_palettes_num; i++)
+ png_write_sPLT(png_ptr, info_ptr->splt_palettes + i);
+#endif /* sPLT */
+
+#ifdef PNG_WRITE_TEXT_SUPPORTED
+ /* Check to see if we need to write text chunks */
+ for (i = 0; i < info_ptr->num_text; i++)
+ {
+ png_debug2(2, "Writing header text chunk %d, type %d", i,
+ info_ptr->text[i].compression);
+ /* An internationalized chunk? */
+ if (info_ptr->text[i].compression > 0)
+ {
+#ifdef PNG_WRITE_iTXt_SUPPORTED
+ /* Write international chunk */
+ png_write_iTXt(png_ptr,
+ info_ptr->text[i].compression,
+ info_ptr->text[i].key,
+ info_ptr->text[i].lang,
+ info_ptr->text[i].lang_key,
+ info_ptr->text[i].text);
+#else
+ png_warning(png_ptr, "Unable to write international text");
+#endif
+ /* Mark this chunk as written */
+ info_ptr->text[i].compression = PNG_TEXT_COMPRESSION_NONE_WR;
+ }
+ /* If we want a compressed text chunk */
+ else if (info_ptr->text[i].compression == PNG_TEXT_COMPRESSION_zTXt)
+ {
+#ifdef PNG_WRITE_zTXt_SUPPORTED
+ /* Write compressed chunk */
+ png_write_zTXt(png_ptr, info_ptr->text[i].key,
+ info_ptr->text[i].text, 0,
+ info_ptr->text[i].compression);
+#else
+ png_warning(png_ptr, "Unable to write compressed text");
+#endif
+ /* Mark this chunk as written */
+ info_ptr->text[i].compression = PNG_TEXT_COMPRESSION_zTXt_WR;
+ }
+ else if (info_ptr->text[i].compression == PNG_TEXT_COMPRESSION_NONE)
+ {
+#ifdef PNG_WRITE_tEXt_SUPPORTED
+ /* Write uncompressed chunk */
+ png_write_tEXt(png_ptr, info_ptr->text[i].key,
+ info_ptr->text[i].text,
+ 0);
+ /* Mark this chunk as written */
+ info_ptr->text[i].compression = PNG_TEXT_COMPRESSION_NONE_WR;
+#else
+ /* Can't get here */
+ png_warning(png_ptr, "Unable to write uncompressed text");
+#endif
+ }
+ }
+#endif /* tEXt */
+
+#ifdef PNG_WRITE_UNKNOWN_CHUNKS_SUPPORTED
+ if (info_ptr->unknown_chunks_num)
+ {
+ png_unknown_chunk *up;
+
+ png_debug(5, "writing extra chunks");
+
+ for (up = info_ptr->unknown_chunks;
+ up < info_ptr->unknown_chunks + info_ptr->unknown_chunks_num;
+ up++)
+ {
+ int keep = png_handle_as_unknown(png_ptr, up->name);
+ if (keep != PNG_HANDLE_CHUNK_NEVER &&
+ up->location && (up->location & PNG_HAVE_PLTE) &&
+ !(up->location & PNG_HAVE_IDAT) &&
+ ((up->name[3] & 0x20) || keep == PNG_HANDLE_CHUNK_ALWAYS ||
+ (png_ptr->flags & PNG_FLAG_KEEP_UNSAFE_CHUNKS)))
+ {
+ png_write_chunk(png_ptr, up->name, up->data, up->size);
+ }
+ }
+ }
+#endif
+}
+
+/* Writes the end of the PNG file. If you don't want to write comments or
+ * time information, you can pass NULL for info. If you already wrote these
+ * in png_write_info(), do not write them again here. If you have long
+ * comments, I suggest writing them here, and compressing them.
+ */
+void PNGAPI
+png_write_end(png_structp png_ptr, png_infop info_ptr)
+{
+ png_debug(1, "in png_write_end");
+
+ if (png_ptr == NULL)
+ return;
+ if (!(png_ptr->mode & PNG_HAVE_IDAT))
+ png_error(png_ptr, "No IDATs written into file");
+
+ /* See if user wants us to write information chunks */
+ if (info_ptr != NULL)
+ {
+#ifdef PNG_WRITE_TEXT_SUPPORTED
+ int i; /* local index variable */
+#endif
+#ifdef PNG_WRITE_tIME_SUPPORTED
+ /* Check to see if user has supplied a time chunk */
+ if ((info_ptr->valid & PNG_INFO_tIME) &&
+ !(png_ptr->mode & PNG_WROTE_tIME))
+ png_write_tIME(png_ptr, &(info_ptr->mod_time));
+#endif
+#ifdef PNG_WRITE_TEXT_SUPPORTED
+ /* Loop through comment chunks */
+ for (i = 0; i < info_ptr->num_text; i++)
+ {
+ png_debug2(2, "Writing trailer text chunk %d, type %d", i,
+ info_ptr->text[i].compression);
+ /* An internationalized chunk? */
+ if (info_ptr->text[i].compression > 0)
+ {
+#ifdef PNG_WRITE_iTXt_SUPPORTED
+ /* Write international chunk */
+ png_write_iTXt(png_ptr,
+ info_ptr->text[i].compression,
+ info_ptr->text[i].key,
+ info_ptr->text[i].lang,
+ info_ptr->text[i].lang_key,
+ info_ptr->text[i].text);
+#else
+ png_warning(png_ptr, "Unable to write international text");
+#endif
+ /* Mark this chunk as written */
+ info_ptr->text[i].compression = PNG_TEXT_COMPRESSION_NONE_WR;
+ }
+ else if (info_ptr->text[i].compression >= PNG_TEXT_COMPRESSION_zTXt)
+ {
+#ifdef PNG_WRITE_zTXt_SUPPORTED
+ /* Write compressed chunk */
+ png_write_zTXt(png_ptr, info_ptr->text[i].key,
+ info_ptr->text[i].text, 0,
+ info_ptr->text[i].compression);
+#else
+ png_warning(png_ptr, "Unable to write compressed text");
+#endif
+ /* Mark this chunk as written */
+ info_ptr->text[i].compression = PNG_TEXT_COMPRESSION_zTXt_WR;
+ }
+ else if (info_ptr->text[i].compression == PNG_TEXT_COMPRESSION_NONE)
+ {
+#ifdef PNG_WRITE_tEXt_SUPPORTED
+ /* Write uncompressed chunk */
+ png_write_tEXt(png_ptr, info_ptr->text[i].key,
+ info_ptr->text[i].text, 0);
+#else
+ png_warning(png_ptr, "Unable to write uncompressed text");
+#endif
+
+ /* Mark this chunk as written */
+ info_ptr->text[i].compression = PNG_TEXT_COMPRESSION_NONE_WR;
+ }
+ }
+#endif
+#ifdef PNG_WRITE_UNKNOWN_CHUNKS_SUPPORTED
+ if (info_ptr->unknown_chunks_num)
+ {
+ png_unknown_chunk *up;
+
+ png_debug(5, "writing extra chunks");
+
+ for (up = info_ptr->unknown_chunks;
+ up < info_ptr->unknown_chunks + info_ptr->unknown_chunks_num;
+ up++)
+ {
+ int keep = png_handle_as_unknown(png_ptr, up->name);
+ if (keep != PNG_HANDLE_CHUNK_NEVER &&
+ up->location && (up->location & PNG_AFTER_IDAT) &&
+ ((up->name[3] & 0x20) || keep == PNG_HANDLE_CHUNK_ALWAYS ||
+ (png_ptr->flags & PNG_FLAG_KEEP_UNSAFE_CHUNKS)))
+ {
+ png_write_chunk(png_ptr, up->name, up->data, up->size);
+ }
+ }
+ }
+#endif
+ }
+
+ png_ptr->mode |= PNG_AFTER_IDAT;
+
+ /* Write end of PNG file */
+ png_write_IEND(png_ptr);
+ /* This flush, added in libpng-1.0.8, removed from libpng-1.0.9beta03,
+ * and restored again in libpng-1.2.30, may cause some applications that
+ * do not set png_ptr->output_flush_fn to crash. If your application
+ * experiences a problem, please try building libpng with
+ * PNG_WRITE_FLUSH_AFTER_IEND_SUPPORTED defined, and report the event to
+ * png-mng-implement at lists.sf.net .
+ */
+#ifdef PNG_WRITE_FLUSH_SUPPORTED
+# ifdef PNG_WRITE_FLUSH_AFTER_IEND_SUPPORTED
+ png_flush(png_ptr);
+# endif
+#endif
+}
+
+#ifdef PNG_CONVERT_tIME_SUPPORTED
+/* "tm" structure is not supported on WindowsCE */
+void PNGAPI
+png_convert_from_struct_tm(png_timep ptime, struct tm FAR * ttime)
+{
+ png_debug(1, "in png_convert_from_struct_tm");
+
+ ptime->year = (png_uint_16)(1900 + ttime->tm_year);
+ ptime->month = (png_byte)(ttime->tm_mon + 1);
+ ptime->day = (png_byte)ttime->tm_mday;
+ ptime->hour = (png_byte)ttime->tm_hour;
+ ptime->minute = (png_byte)ttime->tm_min;
+ ptime->second = (png_byte)ttime->tm_sec;
+}
+
+void PNGAPI
+png_convert_from_time_t(png_timep ptime, time_t ttime)
+{
+ struct tm *tbuf;
+
+ png_debug(1, "in png_convert_from_time_t");
+
+ tbuf = gmtime(&ttime);
+ png_convert_from_struct_tm(ptime, tbuf);
+}
+#endif
+
+/* Initialize png_ptr structure, and allocate any memory needed */
+png_structp PNGAPI
+png_create_write_struct(png_const_charp user_png_ver, png_voidp error_ptr,
+ png_error_ptr error_fn, png_error_ptr warn_fn)
+{
+#ifdef PNG_USER_MEM_SUPPORTED
+ return (png_create_write_struct_2(user_png_ver, error_ptr, error_fn,
+ warn_fn, png_voidp_NULL, png_malloc_ptr_NULL, png_free_ptr_NULL));
+}
+
+/* Alternate initialize png_ptr structure, and allocate any memory needed */
+png_structp PNGAPI
+png_create_write_struct_2(png_const_charp user_png_ver, png_voidp error_ptr,
+ png_error_ptr error_fn, png_error_ptr warn_fn, png_voidp mem_ptr,
+ png_malloc_ptr malloc_fn, png_free_ptr free_fn)
+{
+#endif /* PNG_USER_MEM_SUPPORTED */
+#ifdef PNG_SETJMP_SUPPORTED
+ volatile
+#endif
+ png_structp png_ptr;
+#ifdef PNG_SETJMP_SUPPORTED
+#ifdef USE_FAR_KEYWORD
+ jmp_buf jmpbuf;
+#endif
+#endif
+ int i;
+
+ png_debug(1, "in png_create_write_struct");
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_ptr = (png_structp)png_create_struct_2(PNG_STRUCT_PNG,
+ (png_malloc_ptr)malloc_fn, (png_voidp)mem_ptr);
+#else
+ png_ptr = (png_structp)png_create_struct(PNG_STRUCT_PNG);
+#endif /* PNG_USER_MEM_SUPPORTED */
+ if (png_ptr == NULL)
+ return (NULL);
+
+ /* Added at libpng-1.2.6 */
+#ifdef PNG_SET_USER_LIMITS_SUPPORTED
+ png_ptr->user_width_max = PNG_USER_WIDTH_MAX;
+ png_ptr->user_height_max = PNG_USER_HEIGHT_MAX;
+#endif
+
+#ifdef PNG_SETJMP_SUPPORTED
+#ifdef USE_FAR_KEYWORD
+ if (setjmp(jmpbuf))
+#else
+ if (setjmp(png_ptr->jmpbuf))
+#endif
+ {
+ png_free(png_ptr, png_ptr->zbuf);
+ png_ptr->zbuf = NULL;
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_destroy_struct_2((png_voidp)png_ptr,
+ (png_free_ptr)free_fn, (png_voidp)mem_ptr);
+#else
+ png_destroy_struct((png_voidp)png_ptr);
+#endif
+ return (NULL);
+ }
+#ifdef USE_FAR_KEYWORD
+ png_memcpy(png_ptr->jmpbuf, jmpbuf, png_sizeof(jmp_buf));
+#endif
+#endif
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_set_mem_fn(png_ptr, mem_ptr, malloc_fn, free_fn);
+#endif /* PNG_USER_MEM_SUPPORTED */
+ png_set_error_fn(png_ptr, error_ptr, error_fn, warn_fn);
+
+ if (user_png_ver)
+ {
+ i = 0;
+ do
+ {
+ if (user_png_ver[i] != png_libpng_ver[i])
+ png_ptr->flags |= PNG_FLAG_LIBRARY_MISMATCH;
+ } while (png_libpng_ver[i++]);
+ }
+
+ if (png_ptr->flags & PNG_FLAG_LIBRARY_MISMATCH)
+ {
+ /* Libpng 0.90 and later are binary incompatible with libpng 0.89, so
+ * we must recompile any applications that use any older library version.
+ * For versions after libpng 1.0, we will be compatible, so we need
+ * only check the first digit.
+ */
+ if (user_png_ver == NULL || user_png_ver[0] != png_libpng_ver[0] ||
+ (user_png_ver[0] == '1' && user_png_ver[2] != png_libpng_ver[2]) ||
+ (user_png_ver[0] == '0' && user_png_ver[2] < '9'))
+ {
+#if defined(PNG_STDIO_SUPPORTED) && !defined(_WIN32_WCE)
+ char msg[80];
+ if (user_png_ver)
+ {
+ png_snprintf(msg, 80,
+ "Application was compiled with png.h from libpng-%.20s",
+ user_png_ver);
+ png_warning(png_ptr, msg);
+ }
+ png_snprintf(msg, 80,
+ "Application is running with png.c from libpng-%.20s",
+ png_libpng_ver);
+ png_warning(png_ptr, msg);
+#endif
+#ifdef PNG_ERROR_NUMBERS_SUPPORTED
+ png_ptr->flags = 0;
+#endif
+ png_error(png_ptr,
+ "Incompatible libpng version in application and library");
+ }
+ }
+
+ /* Initialize zbuf - compression buffer */
+ png_ptr->zbuf_size = PNG_ZBUF_SIZE;
+ png_ptr->zbuf = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)png_ptr->zbuf_size);
+
+ png_set_write_fn(png_ptr, png_voidp_NULL, png_rw_ptr_NULL,
+ png_flush_ptr_NULL);
+
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ png_set_filter_heuristics(png_ptr, PNG_FILTER_HEURISTIC_DEFAULT,
+ 1, png_doublep_NULL, png_doublep_NULL);
+#endif
+
+#ifdef PNG_SETJMP_SUPPORTED
+ /* Applications that neglect to set up their own setjmp() and then
+ * encounter a png_error() will longjmp here. Since the jmpbuf is
+ * then meaningless we abort instead of returning.
+ */
+#ifdef USE_FAR_KEYWORD
+ if (setjmp(jmpbuf))
+ PNG_ABORT();
+ png_memcpy(png_ptr->jmpbuf, jmpbuf, png_sizeof(jmp_buf));
+#else
+ if (setjmp(png_ptr->jmpbuf))
+ PNG_ABORT();
+#endif
+#endif
+ return (png_ptr);
+}
+
+/* Initialize png_ptr structure, and allocate any memory needed */
+#if defined(PNG_1_0_X) || defined(PNG_1_2_X)
+/* Deprecated. */
+#undef png_write_init
+void PNGAPI
+png_write_init(png_structp png_ptr)
+{
+ /* We only come here via pre-1.0.7-compiled applications */
+ png_write_init_2(png_ptr, "1.0.6 or earlier", 0, 0);
+}
+
+void PNGAPI
+png_write_init_2(png_structp png_ptr, png_const_charp user_png_ver,
+ png_size_t png_struct_size, png_size_t png_info_size)
+{
+ /* We only come here via pre-1.0.12-compiled applications */
+ if (png_ptr == NULL) return;
+#if defined(PNG_STDIO_SUPPORTED) && !defined(_WIN32_WCE)
+ if (png_sizeof(png_struct) > png_struct_size ||
+ png_sizeof(png_info) > png_info_size)
+ {
+ char msg[80];
+ png_ptr->warning_fn = NULL;
+ if (user_png_ver)
+ {
+ png_snprintf(msg, 80,
+ "Application was compiled with png.h from libpng-%.20s",
+ user_png_ver);
+ png_warning(png_ptr, msg);
+ }
+ png_snprintf(msg, 80,
+ "Application is running with png.c from libpng-%.20s",
+ png_libpng_ver);
+ png_warning(png_ptr, msg);
+ }
+#endif
+ if (png_sizeof(png_struct) > png_struct_size)
+ {
+ png_ptr->error_fn = NULL;
+#ifdef PNG_ERROR_NUMBERS_SUPPORTED
+ png_ptr->flags = 0;
+#endif
+ png_error(png_ptr,
+ "The png struct allocated by the application for writing is"
+ " too small.");
+ }
+ if (png_sizeof(png_info) > png_info_size)
+ {
+ png_ptr->error_fn = NULL;
+#ifdef PNG_ERROR_NUMBERS_SUPPORTED
+ png_ptr->flags = 0;
+#endif
+ png_error(png_ptr,
+ "The info struct allocated by the application for writing is"
+ " too small.");
+ }
+ png_write_init_3(&png_ptr, user_png_ver, png_struct_size);
+}
+#endif /* PNG_1_0_X || PNG_1_2_X */
+
+
+void PNGAPI
+png_write_init_3(png_structpp ptr_ptr, png_const_charp user_png_ver,
+ png_size_t png_struct_size)
+{
+ png_structp png_ptr = *ptr_ptr;
+#ifdef PNG_SETJMP_SUPPORTED
+ jmp_buf tmp_jmp; /* to save current jump buffer */
+#endif
+
+ int i = 0;
+
+ if (png_ptr == NULL)
+ return;
+
+ do
+ {
+ if (user_png_ver[i] != png_libpng_ver[i])
+ {
+#ifdef PNG_LEGACY_SUPPORTED
+ png_ptr->flags |= PNG_FLAG_LIBRARY_MISMATCH;
+#else
+ png_ptr->warning_fn = NULL;
+ png_warning(png_ptr,
+ "Application uses deprecated png_write_init() and should be recompiled.");
+#endif
+ }
+ } while (png_libpng_ver[i++]);
+
+ png_debug(1, "in png_write_init_3");
+
+#ifdef PNG_SETJMP_SUPPORTED
+ /* Save jump buffer and error functions */
+ png_memcpy(tmp_jmp, png_ptr->jmpbuf, png_sizeof(jmp_buf));
+#endif
+
+ if (png_sizeof(png_struct) > png_struct_size)
+ {
+ png_destroy_struct(png_ptr);
+ png_ptr = (png_structp)png_create_struct(PNG_STRUCT_PNG);
+ *ptr_ptr = png_ptr;
+ }
+
+ /* Reset all variables to 0 */
+ png_memset(png_ptr, 0, png_sizeof(png_struct));
+
+ /* Added at libpng-1.2.6 */
+#ifdef PNG_SET_USER_LIMITS_SUPPORTED
+ png_ptr->user_width_max = PNG_USER_WIDTH_MAX;
+ png_ptr->user_height_max = PNG_USER_HEIGHT_MAX;
+#endif
+
+#ifdef PNG_SETJMP_SUPPORTED
+ /* Restore jump buffer */
+ png_memcpy(png_ptr->jmpbuf, tmp_jmp, png_sizeof(jmp_buf));
+#endif
+
+ png_set_write_fn(png_ptr, png_voidp_NULL, png_rw_ptr_NULL,
+ png_flush_ptr_NULL);
+
+ /* Initialize zbuf - compression buffer */
+ png_ptr->zbuf_size = PNG_ZBUF_SIZE;
+ png_ptr->zbuf = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)png_ptr->zbuf_size);
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ png_set_filter_heuristics(png_ptr, PNG_FILTER_HEURISTIC_DEFAULT,
+ 1, png_doublep_NULL, png_doublep_NULL);
+#endif
+}
+
+/* Write a few rows of image data. If the image is interlaced,
+ * either you will have to write the 7 sub images, or, if you
+ * have called png_set_interlace_handling(), you will have to
+ * "write" the image seven times.
+ */
+void PNGAPI
+png_write_rows(png_structp png_ptr, png_bytepp row,
+ png_uint_32 num_rows)
+{
+ png_uint_32 i; /* row counter */
+ png_bytepp rp; /* row pointer */
+
+ png_debug(1, "in png_write_rows");
+
+ if (png_ptr == NULL)
+ return;
+
+ /* Loop through the rows */
+ for (i = 0, rp = row; i < num_rows; i++, rp++)
+ {
+ png_write_row(png_ptr, *rp);
+ }
+}
+
+/* Write the image. You only need to call this function once, even
+ * if you are writing an interlaced image.
+ */
+void PNGAPI
+png_write_image(png_structp png_ptr, png_bytepp image)
+{
+ png_uint_32 i; /* row index */
+ int pass, num_pass; /* pass variables */
+ png_bytepp rp; /* points to current row */
+
+ if (png_ptr == NULL)
+ return;
+
+ png_debug(1, "in png_write_image");
+
+#ifdef PNG_WRITE_INTERLACING_SUPPORTED
+ /* Initialize interlace handling. If image is not interlaced,
+ * this will set pass to 1
+ */
+ num_pass = png_set_interlace_handling(png_ptr);
+#else
+ num_pass = 1;
+#endif
+ /* Loop through passes */
+ for (pass = 0; pass < num_pass; pass++)
+ {
+ /* Loop through image */
+ for (i = 0, rp = image; i < png_ptr->height; i++, rp++)
+ {
+ png_write_row(png_ptr, *rp);
+ }
+ }
+}
+
+/* Called by user to write a row of image data */
+void PNGAPI
+png_write_row(png_structp png_ptr, png_bytep row)
+{
+ if (png_ptr == NULL)
+ return;
+
+ png_debug2(1, "in png_write_row (row %ld, pass %d)",
+ png_ptr->row_number, png_ptr->pass);
+
+ /* Initialize transformations and other stuff if first time */
+ if (png_ptr->row_number == 0 && png_ptr->pass == 0)
+ {
+ /* Make sure we wrote the header info */
+ if (!(png_ptr->mode & PNG_WROTE_INFO_BEFORE_PLTE))
+ png_error(png_ptr,
+ "png_write_info was never called before png_write_row.");
+
+ /* Check for transforms that have been set but were defined out */
+#if !defined(PNG_WRITE_INVERT_SUPPORTED) && defined(PNG_READ_INVERT_SUPPORTED)
+ if (png_ptr->transformations & PNG_INVERT_MONO)
+ png_warning(png_ptr,
+ "PNG_WRITE_INVERT_SUPPORTED is not defined.");
+#endif
+#if !defined(PNG_WRITE_FILLER_SUPPORTED) && defined(PNG_READ_FILLER_SUPPORTED)
+ if (png_ptr->transformations & PNG_FILLER)
+ png_warning(png_ptr,
+ "PNG_WRITE_FILLER_SUPPORTED is not defined.");
+#endif
+#if !defined(PNG_WRITE_PACKSWAP_SUPPORTED) && \
+ defined(PNG_READ_PACKSWAP_SUPPORTED)
+ if (png_ptr->transformations & PNG_PACKSWAP)
+ png_warning(png_ptr,
+ "PNG_WRITE_PACKSWAP_SUPPORTED is not defined.");
+#endif
+#if !defined(PNG_WRITE_PACK_SUPPORTED) && defined(PNG_READ_PACK_SUPPORTED)
+ if (png_ptr->transformations & PNG_PACK)
+ png_warning(png_ptr, "PNG_WRITE_PACK_SUPPORTED is not defined.");
+#endif
+#if !defined(PNG_WRITE_SHIFT_SUPPORTED) && defined(PNG_READ_SHIFT_SUPPORTED)
+ if (png_ptr->transformations & PNG_SHIFT)
+ png_warning(png_ptr, "PNG_WRITE_SHIFT_SUPPORTED is not defined.");
+#endif
+#if !defined(PNG_WRITE_BGR_SUPPORTED) && defined(PNG_READ_BGR_SUPPORTED)
+ if (png_ptr->transformations & PNG_BGR)
+ png_warning(png_ptr, "PNG_WRITE_BGR_SUPPORTED is not defined.");
+#endif
+#if !defined(PNG_WRITE_SWAP_SUPPORTED) && defined(PNG_READ_SWAP_SUPPORTED)
+ if (png_ptr->transformations & PNG_SWAP_BYTES)
+ png_warning(png_ptr, "PNG_WRITE_SWAP_SUPPORTED is not defined.");
+#endif
+
+ png_write_start_row(png_ptr);
+ }
+
+#ifdef PNG_WRITE_INTERLACING_SUPPORTED
+ /* If interlaced and not interested in row, return */
+ if (png_ptr->interlaced && (png_ptr->transformations & PNG_INTERLACE))
+ {
+ switch (png_ptr->pass)
+ {
+ case 0:
+ if (png_ptr->row_number & 0x07)
+ {
+ png_write_finish_row(png_ptr);
+ return;
+ }
+ break;
+ case 1:
+ if ((png_ptr->row_number & 0x07) || png_ptr->width < 5)
+ {
+ png_write_finish_row(png_ptr);
+ return;
+ }
+ break;
+ case 2:
+ if ((png_ptr->row_number & 0x07) != 4)
+ {
+ png_write_finish_row(png_ptr);
+ return;
+ }
+ break;
+ case 3:
+ if ((png_ptr->row_number & 0x03) || png_ptr->width < 3)
+ {
+ png_write_finish_row(png_ptr);
+ return;
+ }
+ break;
+ case 4:
+ if ((png_ptr->row_number & 0x03) != 2)
+ {
+ png_write_finish_row(png_ptr);
+ return;
+ }
+ break;
+ case 5:
+ if ((png_ptr->row_number & 0x01) || png_ptr->width < 2)
+ {
+ png_write_finish_row(png_ptr);
+ return;
+ }
+ break;
+ case 6:
+ if (!(png_ptr->row_number & 0x01))
+ {
+ png_write_finish_row(png_ptr);
+ return;
+ }
+ break;
+ }
+ }
+#endif
+
+ /* Set up row info for transformations */
+ png_ptr->row_info.color_type = png_ptr->color_type;
+ png_ptr->row_info.width = png_ptr->usr_width;
+ png_ptr->row_info.channels = png_ptr->usr_channels;
+ png_ptr->row_info.bit_depth = png_ptr->usr_bit_depth;
+ png_ptr->row_info.pixel_depth = (png_byte)(png_ptr->row_info.bit_depth *
+ png_ptr->row_info.channels);
+
+ png_ptr->row_info.rowbytes = PNG_ROWBYTES(png_ptr->row_info.pixel_depth,
+ png_ptr->row_info.width);
+
+ png_debug1(3, "row_info->color_type = %d", png_ptr->row_info.color_type);
+ png_debug1(3, "row_info->width = %lu", png_ptr->row_info.width);
+ png_debug1(3, "row_info->channels = %d", png_ptr->row_info.channels);
+ png_debug1(3, "row_info->bit_depth = %d", png_ptr->row_info.bit_depth);
+ png_debug1(3, "row_info->pixel_depth = %d", png_ptr->row_info.pixel_depth);
+ png_debug1(3, "row_info->rowbytes = %lu", png_ptr->row_info.rowbytes);
+
+ /* Copy user's row into buffer, leaving room for filter byte. */
+ png_memcpy_check(png_ptr, png_ptr->row_buf + 1, row,
+ png_ptr->row_info.rowbytes);
+
+#ifdef PNG_WRITE_INTERLACING_SUPPORTED
+ /* Handle interlacing */
+ if (png_ptr->interlaced && png_ptr->pass < 6 &&
+ (png_ptr->transformations & PNG_INTERLACE))
+ {
+ png_do_write_interlace(&(png_ptr->row_info),
+ png_ptr->row_buf + 1, png_ptr->pass);
+ /* This should always get caught above, but still ... */
+ if (!(png_ptr->row_info.width))
+ {
+ png_write_finish_row(png_ptr);
+ return;
+ }
+ }
+#endif
+
+ /* Handle other transformations */
+ if (png_ptr->transformations)
+ png_do_write_transformations(png_ptr);
+
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+ /* Write filter_method 64 (intrapixel differencing) only if
+ * 1. Libpng was compiled with PNG_MNG_FEATURES_SUPPORTED and
+ * 2. Libpng did not write a PNG signature (this filter_method is only
+ * used in PNG datastreams that are embedded in MNG datastreams) and
+ * 3. The application called png_permit_mng_features with a mask that
+ * included PNG_FLAG_MNG_FILTER_64 and
+ * 4. The filter_method is 64 and
+ * 5. The color_type is RGB or RGBA
+ */
+ if ((png_ptr->mng_features_permitted & PNG_FLAG_MNG_FILTER_64) &&
+ (png_ptr->filter_type == PNG_INTRAPIXEL_DIFFERENCING))
+ {
+ /* Intrapixel differencing */
+ png_do_write_intrapixel(&(png_ptr->row_info), png_ptr->row_buf + 1);
+ }
+#endif
+
+ /* Find a filter if necessary, filter the row and write it out. */
+ png_write_find_filter(png_ptr, &(png_ptr->row_info));
+
+ if (png_ptr->write_row_fn != NULL)
+ (*(png_ptr->write_row_fn))(png_ptr, png_ptr->row_number, png_ptr->pass);
+}
+
+#ifdef PNG_WRITE_FLUSH_SUPPORTED
+/* Set the automatic flush interval or 0 to turn flushing off */
+void PNGAPI
+png_set_flush(png_structp png_ptr, int nrows)
+{
+ png_debug(1, "in png_set_flush");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->flush_dist = (nrows < 0 ? 0 : nrows);
+}
+
+/* Flush the current output buffers now */
+void PNGAPI
+png_write_flush(png_structp png_ptr)
+{
+ int wrote_IDAT;
+
+ png_debug(1, "in png_write_flush");
+
+ if (png_ptr == NULL)
+ return;
+ /* We have already written out all of the data */
+ if (png_ptr->row_number >= png_ptr->num_rows)
+ return;
+
+ do
+ {
+ int ret;
+
+ /* Compress the data */
+ ret = deflate(&png_ptr->zstream, Z_SYNC_FLUSH);
+ wrote_IDAT = 0;
+
+ /* Check for compression errors */
+ if (ret != Z_OK)
+ {
+ if (png_ptr->zstream.msg != NULL)
+ png_error(png_ptr, png_ptr->zstream.msg);
+ else
+ png_error(png_ptr, "zlib error");
+ }
+
+ if (!(png_ptr->zstream.avail_out))
+ {
+ /* Write the IDAT and reset the zlib output buffer */
+ png_write_IDAT(png_ptr, png_ptr->zbuf,
+ png_ptr->zbuf_size);
+ png_ptr->zstream.next_out = png_ptr->zbuf;
+ png_ptr->zstream.avail_out = (uInt)png_ptr->zbuf_size;
+ wrote_IDAT = 1;
+ }
+ } while(wrote_IDAT == 1);
+
+ /* If there is any data left to be output, write it into a new IDAT */
+ if (png_ptr->zbuf_size != png_ptr->zstream.avail_out)
+ {
+ /* Write the IDAT and reset the zlib output buffer */
+ png_write_IDAT(png_ptr, png_ptr->zbuf,
+ png_ptr->zbuf_size - png_ptr->zstream.avail_out);
+ png_ptr->zstream.next_out = png_ptr->zbuf;
+ png_ptr->zstream.avail_out = (uInt)png_ptr->zbuf_size;
+ }
+ png_ptr->flush_rows = 0;
+ png_flush(png_ptr);
+}
+#endif /* PNG_WRITE_FLUSH_SUPPORTED */
+
+/* Free all memory used by the write */
+void PNGAPI
+png_destroy_write_struct(png_structpp png_ptr_ptr, png_infopp info_ptr_ptr)
+{
+ png_structp png_ptr = NULL;
+ png_infop info_ptr = NULL;
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_free_ptr free_fn = NULL;
+ png_voidp mem_ptr = NULL;
+#endif
+
+ png_debug(1, "in png_destroy_write_struct");
+
+ if (png_ptr_ptr != NULL)
+ {
+ png_ptr = *png_ptr_ptr;
+#ifdef PNG_USER_MEM_SUPPORTED
+ free_fn = png_ptr->free_fn;
+ mem_ptr = png_ptr->mem_ptr;
+#endif
+ }
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ if (png_ptr != NULL)
+ {
+ free_fn = png_ptr->free_fn;
+ mem_ptr = png_ptr->mem_ptr;
+ }
+#endif
+
+ if (info_ptr_ptr != NULL)
+ info_ptr = *info_ptr_ptr;
+
+ if (info_ptr != NULL)
+ {
+ if (png_ptr != NULL)
+ {
+ png_free_data(png_ptr, info_ptr, PNG_FREE_ALL, -1);
+
+#ifdef PNG_HANDLE_AS_UNKNOWN_SUPPORTED
+ if (png_ptr->num_chunk_list)
+ {
+ png_free(png_ptr, png_ptr->chunk_list);
+ png_ptr->chunk_list = NULL;
+ png_ptr->num_chunk_list = 0;
+ }
+#endif
+ }
+
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_destroy_struct_2((png_voidp)info_ptr, (png_free_ptr)free_fn,
+ (png_voidp)mem_ptr);
+#else
+ png_destroy_struct((png_voidp)info_ptr);
+#endif
+ *info_ptr_ptr = NULL;
+ }
+
+ if (png_ptr != NULL)
+ {
+ png_write_destroy(png_ptr);
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_destroy_struct_2((png_voidp)png_ptr, (png_free_ptr)free_fn,
+ (png_voidp)mem_ptr);
+#else
+ png_destroy_struct((png_voidp)png_ptr);
+#endif
+ *png_ptr_ptr = NULL;
+ }
+}
+
+
+/* Free any memory used in png_ptr struct (old method) */
+void /* PRIVATE */
+png_write_destroy(png_structp png_ptr)
+{
+#ifdef PNG_SETJMP_SUPPORTED
+ jmp_buf tmp_jmp; /* Save jump buffer */
+#endif
+ png_error_ptr error_fn;
+ png_error_ptr warning_fn;
+ png_voidp error_ptr;
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_free_ptr free_fn;
+#endif
+
+ png_debug(1, "in png_write_destroy");
+
+ /* Free any memory zlib uses */
+ deflateEnd(&png_ptr->zstream);
+
+ /* Free our memory. png_free checks NULL for us. */
+ png_free(png_ptr, png_ptr->zbuf);
+ png_free(png_ptr, png_ptr->row_buf);
+#ifdef PNG_WRITE_FILTER_SUPPORTED
+ png_free(png_ptr, png_ptr->prev_row);
+ png_free(png_ptr, png_ptr->sub_row);
+ png_free(png_ptr, png_ptr->up_row);
+ png_free(png_ptr, png_ptr->avg_row);
+ png_free(png_ptr, png_ptr->paeth_row);
+#endif
+
+#ifdef PNG_TIME_RFC1123_SUPPORTED
+ png_free(png_ptr, png_ptr->time_buffer);
+#endif
+
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ png_free(png_ptr, png_ptr->prev_filters);
+ png_free(png_ptr, png_ptr->filter_weights);
+ png_free(png_ptr, png_ptr->inv_filter_weights);
+ png_free(png_ptr, png_ptr->filter_costs);
+ png_free(png_ptr, png_ptr->inv_filter_costs);
+#endif
+
+#ifdef PNG_SETJMP_SUPPORTED
+ /* Reset structure */
+ png_memcpy(tmp_jmp, png_ptr->jmpbuf, png_sizeof(jmp_buf));
+#endif
+
+ error_fn = png_ptr->error_fn;
+ warning_fn = png_ptr->warning_fn;
+ error_ptr = png_ptr->error_ptr;
+#ifdef PNG_USER_MEM_SUPPORTED
+ free_fn = png_ptr->free_fn;
+#endif
+
+ png_memset(png_ptr, 0, png_sizeof(png_struct));
+
+ png_ptr->error_fn = error_fn;
+ png_ptr->warning_fn = warning_fn;
+ png_ptr->error_ptr = error_ptr;
+#ifdef PNG_USER_MEM_SUPPORTED
+ png_ptr->free_fn = free_fn;
+#endif
+
+#ifdef PNG_SETJMP_SUPPORTED
+ png_memcpy(png_ptr->jmpbuf, tmp_jmp, png_sizeof(jmp_buf));
+#endif
+}
+
+/* Allow the application to select one or more row filters to use. */
+void PNGAPI
+png_set_filter(png_structp png_ptr, int method, int filters)
+{
+ png_debug(1, "in png_set_filter");
+
+ if (png_ptr == NULL)
+ return;
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+ if ((png_ptr->mng_features_permitted & PNG_FLAG_MNG_FILTER_64) &&
+ (method == PNG_INTRAPIXEL_DIFFERENCING))
+ method = PNG_FILTER_TYPE_BASE;
+#endif
+ if (method == PNG_FILTER_TYPE_BASE)
+ {
+ switch (filters & (PNG_ALL_FILTERS | 0x07))
+ {
+#ifdef PNG_WRITE_FILTER_SUPPORTED
+ case 5:
+ case 6:
+ case 7: png_warning(png_ptr, "Unknown row filter for method 0");
+#endif /* PNG_WRITE_FILTER_SUPPORTED */
+ case PNG_FILTER_VALUE_NONE:
+ png_ptr->do_filter = PNG_FILTER_NONE; break;
+#ifdef PNG_WRITE_FILTER_SUPPORTED
+ case PNG_FILTER_VALUE_SUB:
+ png_ptr->do_filter = PNG_FILTER_SUB; break;
+ case PNG_FILTER_VALUE_UP:
+ png_ptr->do_filter = PNG_FILTER_UP; break;
+ case PNG_FILTER_VALUE_AVG:
+ png_ptr->do_filter = PNG_FILTER_AVG; break;
+ case PNG_FILTER_VALUE_PAETH:
+ png_ptr->do_filter = PNG_FILTER_PAETH; break;
+ default: png_ptr->do_filter = (png_byte)filters; break;
+#else
+ default: png_warning(png_ptr, "Unknown row filter for method 0");
+#endif /* PNG_WRITE_FILTER_SUPPORTED */
+ }
+
+ /* If we have allocated the row_buf, this means we have already started
+ * with the image and we should have allocated all of the filter buffers
+ * that have been selected. If prev_row isn't already allocated, then
+ * it is too late to start using the filters that need it, since we
+ * will be missing the data in the previous row. If an application
+ * wants to start and stop using particular filters during compression,
+ * it should start out with all of the filters, and then add and
+ * remove them after the start of compression.
+ */
+ if (png_ptr->row_buf != NULL)
+ {
+#ifdef PNG_WRITE_FILTER_SUPPORTED
+ if ((png_ptr->do_filter & PNG_FILTER_SUB) && png_ptr->sub_row == NULL)
+ {
+ png_ptr->sub_row = (png_bytep)png_malloc(png_ptr,
+ (png_ptr->rowbytes + 1));
+ png_ptr->sub_row[0] = PNG_FILTER_VALUE_SUB;
+ }
+
+ if ((png_ptr->do_filter & PNG_FILTER_UP) && png_ptr->up_row == NULL)
+ {
+ if (png_ptr->prev_row == NULL)
+ {
+ png_warning(png_ptr, "Can't add Up filter after starting");
+ png_ptr->do_filter &= ~PNG_FILTER_UP;
+ }
+ else
+ {
+ png_ptr->up_row = (png_bytep)png_malloc(png_ptr,
+ (png_ptr->rowbytes + 1));
+ png_ptr->up_row[0] = PNG_FILTER_VALUE_UP;
+ }
+ }
+
+ if ((png_ptr->do_filter & PNG_FILTER_AVG) && png_ptr->avg_row == NULL)
+ {
+ if (png_ptr->prev_row == NULL)
+ {
+ png_warning(png_ptr, "Can't add Average filter after starting");
+ png_ptr->do_filter &= ~PNG_FILTER_AVG;
+ }
+ else
+ {
+ png_ptr->avg_row = (png_bytep)png_malloc(png_ptr,
+ (png_ptr->rowbytes + 1));
+ png_ptr->avg_row[0] = PNG_FILTER_VALUE_AVG;
+ }
+ }
+
+ if ((png_ptr->do_filter & PNG_FILTER_PAETH) &&
+ png_ptr->paeth_row == NULL)
+ {
+ if (png_ptr->prev_row == NULL)
+ {
+ png_warning(png_ptr, "Can't add Paeth filter after starting");
+ png_ptr->do_filter &= (png_byte)(~PNG_FILTER_PAETH);
+ }
+ else
+ {
+ png_ptr->paeth_row = (png_bytep)png_malloc(png_ptr,
+ (png_ptr->rowbytes + 1));
+ png_ptr->paeth_row[0] = PNG_FILTER_VALUE_PAETH;
+ }
+ }
+
+ if (png_ptr->do_filter == PNG_NO_FILTERS)
+#endif /* PNG_WRITE_FILTER_SUPPORTED */
+ png_ptr->do_filter = PNG_FILTER_NONE;
+ }
+ }
+ else
+ png_error(png_ptr, "Unknown custom filter method");
+}
+
+/* This allows us to influence the way in which libpng chooses the "best"
+ * filter for the current scanline. While the "minimum-sum-of-absolute-
+ * differences metric is relatively fast and effective, there is some
+ * question as to whether it can be improved upon by trying to keep the
+ * filtered data going to zlib more consistent, hopefully resulting in
+ * better compression.
+ */
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED /* GRR 970116 */
+void PNGAPI
+png_set_filter_heuristics(png_structp png_ptr, int heuristic_method,
+ int num_weights, png_doublep filter_weights,
+ png_doublep filter_costs)
+{
+ int i;
+
+ png_debug(1, "in png_set_filter_heuristics");
+
+ if (png_ptr == NULL)
+ return;
+ if (heuristic_method >= PNG_FILTER_HEURISTIC_LAST)
+ {
+ png_warning(png_ptr, "Unknown filter heuristic method");
+ return;
+ }
+
+ if (heuristic_method == PNG_FILTER_HEURISTIC_DEFAULT)
+ {
+ heuristic_method = PNG_FILTER_HEURISTIC_UNWEIGHTED;
+ }
+
+ if (num_weights < 0 || filter_weights == NULL ||
+ heuristic_method == PNG_FILTER_HEURISTIC_UNWEIGHTED)
+ {
+ num_weights = 0;
+ }
+
+ png_ptr->num_prev_filters = (png_byte)num_weights;
+ png_ptr->heuristic_method = (png_byte)heuristic_method;
+
+ if (num_weights > 0)
+ {
+ if (png_ptr->prev_filters == NULL)
+ {
+ png_ptr->prev_filters = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)(png_sizeof(png_byte) * num_weights));
+
+ /* To make sure that the weighting starts out fairly */
+ for (i = 0; i < num_weights; i++)
+ {
+ png_ptr->prev_filters[i] = 255;
+ }
+ }
+
+ if (png_ptr->filter_weights == NULL)
+ {
+ png_ptr->filter_weights = (png_uint_16p)png_malloc(png_ptr,
+ (png_uint_32)(png_sizeof(png_uint_16) * num_weights));
+
+ png_ptr->inv_filter_weights = (png_uint_16p)png_malloc(png_ptr,
+ (png_uint_32)(png_sizeof(png_uint_16) * num_weights));
+ for (i = 0; i < num_weights; i++)
+ {
+ png_ptr->inv_filter_weights[i] =
+ png_ptr->filter_weights[i] = PNG_WEIGHT_FACTOR;
+ }
+ }
+
+ for (i = 0; i < num_weights; i++)
+ {
+ if (filter_weights[i] < 0.0)
+ {
+ png_ptr->inv_filter_weights[i] =
+ png_ptr->filter_weights[i] = PNG_WEIGHT_FACTOR;
+ }
+ else
+ {
+ png_ptr->inv_filter_weights[i] =
+ (png_uint_16)((double)PNG_WEIGHT_FACTOR*filter_weights[i]+0.5);
+ png_ptr->filter_weights[i] =
+ (png_uint_16)((double)PNG_WEIGHT_FACTOR/filter_weights[i]+0.5);
+ }
+ }
+ }
+
+ /* If, in the future, there are other filter methods, this would
+ * need to be based on png_ptr->filter.
+ */
+ if (png_ptr->filter_costs == NULL)
+ {
+ png_ptr->filter_costs = (png_uint_16p)png_malloc(png_ptr,
+ (png_uint_32)(png_sizeof(png_uint_16) * PNG_FILTER_VALUE_LAST));
+
+ png_ptr->inv_filter_costs = (png_uint_16p)png_malloc(png_ptr,
+ (png_uint_32)(png_sizeof(png_uint_16) * PNG_FILTER_VALUE_LAST));
+
+ for (i = 0; i < PNG_FILTER_VALUE_LAST; i++)
+ {
+ png_ptr->inv_filter_costs[i] =
+ png_ptr->filter_costs[i] = PNG_COST_FACTOR;
+ }
+ }
+
+ /* Here is where we set the relative costs of the different filters. We
+ * should take the desired compression level into account when setting
+ * the costs, so that Paeth, for instance, has a high relative cost at low
+ * compression levels, while it has a lower relative cost at higher
+ * compression settings. The filter types are in order of increasing
+ * relative cost, so it would be possible to do this with an algorithm.
+ */
+ for (i = 0; i < PNG_FILTER_VALUE_LAST; i++)
+ {
+ if (filter_costs == NULL || filter_costs[i] < 0.0)
+ {
+ png_ptr->inv_filter_costs[i] =
+ png_ptr->filter_costs[i] = PNG_COST_FACTOR;
+ }
+ else if (filter_costs[i] >= 1.0)
+ {
+ png_ptr->inv_filter_costs[i] =
+ (png_uint_16)((double)PNG_COST_FACTOR / filter_costs[i] + 0.5);
+ png_ptr->filter_costs[i] =
+ (png_uint_16)((double)PNG_COST_FACTOR * filter_costs[i] + 0.5);
+ }
+ }
+}
+#endif /* PNG_WRITE_WEIGHTED_FILTER_SUPPORTED */
+
+void PNGAPI
+png_set_compression_level(png_structp png_ptr, int level)
+{
+ png_debug(1, "in png_set_compression_level");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->flags |= PNG_FLAG_ZLIB_CUSTOM_LEVEL;
+ png_ptr->zlib_level = level;
+}
+
+void PNGAPI
+png_set_compression_mem_level(png_structp png_ptr, int mem_level)
+{
+ png_debug(1, "in png_set_compression_mem_level");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->flags |= PNG_FLAG_ZLIB_CUSTOM_MEM_LEVEL;
+ png_ptr->zlib_mem_level = mem_level;
+}
+
+void PNGAPI
+png_set_compression_strategy(png_structp png_ptr, int strategy)
+{
+ png_debug(1, "in png_set_compression_strategy");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->flags |= PNG_FLAG_ZLIB_CUSTOM_STRATEGY;
+ png_ptr->zlib_strategy = strategy;
+}
+
+void PNGAPI
+png_set_compression_window_bits(png_structp png_ptr, int window_bits)
+{
+ if (png_ptr == NULL)
+ return;
+ if (window_bits > 15)
+ png_warning(png_ptr, "Only compression windows <= 32k supported by PNG");
+ else if (window_bits < 8)
+ png_warning(png_ptr, "Only compression windows >= 256 supported by PNG");
+#ifndef WBITS_8_OK
+ /* Avoid libpng bug with 256-byte windows */
+ if (window_bits == 8)
+ {
+ png_warning(png_ptr, "Compression window is being reset to 512");
+ window_bits = 9;
+ }
+#endif
+ png_ptr->flags |= PNG_FLAG_ZLIB_CUSTOM_WINDOW_BITS;
+ png_ptr->zlib_window_bits = window_bits;
+}
+
+void PNGAPI
+png_set_compression_method(png_structp png_ptr, int method)
+{
+ png_debug(1, "in png_set_compression_method");
+
+ if (png_ptr == NULL)
+ return;
+ if (method != 8)
+ png_warning(png_ptr, "Only compression method 8 is supported by PNG");
+ png_ptr->flags |= PNG_FLAG_ZLIB_CUSTOM_METHOD;
+ png_ptr->zlib_method = method;
+}
+
+void PNGAPI
+png_set_write_status_fn(png_structp png_ptr, png_write_status_ptr write_row_fn)
+{
+ if (png_ptr == NULL)
+ return;
+ png_ptr->write_row_fn = write_row_fn;
+}
+
+#ifdef PNG_WRITE_USER_TRANSFORM_SUPPORTED
+void PNGAPI
+png_set_write_user_transform_fn(png_structp png_ptr, png_user_transform_ptr
+ write_user_transform_fn)
+{
+ png_debug(1, "in png_set_write_user_transform_fn");
+
+ if (png_ptr == NULL)
+ return;
+ png_ptr->transformations |= PNG_USER_TRANSFORM;
+ png_ptr->write_user_transform_fn = write_user_transform_fn;
+}
+#endif
+
+
+#ifdef PNG_INFO_IMAGE_SUPPORTED
+void PNGAPI
+png_write_png(png_structp png_ptr, png_infop info_ptr,
+ int transforms, voidp params)
+{
+ if (png_ptr == NULL || info_ptr == NULL)
+ return;
+
+ /* Write the file header information. */
+ png_write_info(png_ptr, info_ptr);
+
+ /* ------ these transformations don't touch the info structure ------- */
+
+#ifdef PNG_WRITE_INVERT_SUPPORTED
+ /* Invert monochrome pixels */
+ if (transforms & PNG_TRANSFORM_INVERT_MONO)
+ png_set_invert_mono(png_ptr);
+#endif
+
+#ifdef PNG_WRITE_SHIFT_SUPPORTED
+ /* Shift the pixels up to a legal bit depth and fill in
+ * as appropriate to correctly scale the image.
+ */
+ if ((transforms & PNG_TRANSFORM_SHIFT)
+ && (info_ptr->valid & PNG_INFO_sBIT))
+ png_set_shift(png_ptr, &info_ptr->sig_bit);
+#endif
+
+#ifdef PNG_WRITE_PACK_SUPPORTED
+ /* Pack pixels into bytes */
+ if (transforms & PNG_TRANSFORM_PACKING)
+ png_set_packing(png_ptr);
+#endif
+
+#ifdef PNG_WRITE_SWAP_ALPHA_SUPPORTED
+ /* Swap location of alpha bytes from ARGB to RGBA */
+ if (transforms & PNG_TRANSFORM_SWAP_ALPHA)
+ png_set_swap_alpha(png_ptr);
+#endif
+
+#ifdef PNG_WRITE_FILLER_SUPPORTED
+ /* Pack XRGB/RGBX/ARGB/RGBA into * RGB (4 channels -> 3 channels) */
+ if (transforms & PNG_TRANSFORM_STRIP_FILLER_AFTER)
+ png_set_filler(png_ptr, 0, PNG_FILLER_AFTER);
+ else if (transforms & PNG_TRANSFORM_STRIP_FILLER_BEFORE)
+ png_set_filler(png_ptr, 0, PNG_FILLER_BEFORE);
+#endif
+
+#ifdef PNG_WRITE_BGR_SUPPORTED
+ /* Flip BGR pixels to RGB */
+ if (transforms & PNG_TRANSFORM_BGR)
+ png_set_bgr(png_ptr);
+#endif
+
+#ifdef PNG_WRITE_SWAP_SUPPORTED
+ /* Swap bytes of 16-bit files to most significant byte first */
+ if (transforms & PNG_TRANSFORM_SWAP_ENDIAN)
+ png_set_swap(png_ptr);
+#endif
+
+#ifdef PNG_WRITE_PACKSWAP_SUPPORTED
+ /* Swap bits of 1, 2, 4 bit packed pixel formats */
+ if (transforms & PNG_TRANSFORM_PACKSWAP)
+ png_set_packswap(png_ptr);
+#endif
+
+#ifdef PNG_WRITE_INVERT_ALPHA_SUPPORTED
+ /* Invert the alpha channel from opacity to transparency */
+ if (transforms & PNG_TRANSFORM_INVERT_ALPHA)
+ png_set_invert_alpha(png_ptr);
+#endif
+
+ /* ----------------------- end of transformations ------------------- */
+
+ /* Write the bits */
+ if (info_ptr->valid & PNG_INFO_IDAT)
+ png_write_image(png_ptr, info_ptr->row_pointers);
+
+ /* It is REQUIRED to call this to finish writing the rest of the file */
+ png_write_end(png_ptr, info_ptr);
+
+ transforms = transforms; /* Quiet compiler warnings */
+ params = params;
+}
+#endif
+#endif /* PNG_WRITE_SUPPORTED */
diff --git a/trunk/src/third_party/libpng/pngwtran.c b/trunk/src/third_party/libpng/pngwtran.c
new file mode 100644
index 0000000..0ce9b9b
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngwtran.c
@@ -0,0 +1,582 @@
+
+/* pngwtran.c - transforms the data in a row for PNG writers
+ *
+ * Last changed in libpng 1.2.43 [February 25, 2010]
+ * Copyright (c) 1998-2010 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ */
+
+#define PNG_INTERNAL
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+#ifdef PNG_WRITE_SUPPORTED
+
+/* Transform the data according to the user's wishes. The order of
+ * transformations is significant.
+ */
+void /* PRIVATE */
+png_do_write_transformations(png_structp png_ptr)
+{
+ png_debug(1, "in png_do_write_transformations");
+
+ if (png_ptr == NULL)
+ return;
+
+#ifdef PNG_WRITE_USER_TRANSFORM_SUPPORTED
+ if (png_ptr->transformations & PNG_USER_TRANSFORM)
+ if (png_ptr->write_user_transform_fn != NULL)
+ (*(png_ptr->write_user_transform_fn)) /* User write transform
+ function */
+ (png_ptr, /* png_ptr */
+ &(png_ptr->row_info), /* row_info: */
+ /* png_uint_32 width; width of row */
+ /* png_uint_32 rowbytes; number of bytes in row */
+ /* png_byte color_type; color type of pixels */
+ /* png_byte bit_depth; bit depth of samples */
+ /* png_byte channels; number of channels (1-4) */
+ /* png_byte pixel_depth; bits per pixel (depth*channels) */
+ png_ptr->row_buf + 1); /* start of pixel data for row */
+#endif
+#ifdef PNG_WRITE_FILLER_SUPPORTED
+ if (png_ptr->transformations & PNG_FILLER)
+ png_do_strip_filler(&(png_ptr->row_info), png_ptr->row_buf + 1,
+ png_ptr->flags);
+#endif
+#ifdef PNG_WRITE_PACKSWAP_SUPPORTED
+ if (png_ptr->transformations & PNG_PACKSWAP)
+ png_do_packswap(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+#ifdef PNG_WRITE_PACK_SUPPORTED
+ if (png_ptr->transformations & PNG_PACK)
+ png_do_pack(&(png_ptr->row_info), png_ptr->row_buf + 1,
+ (png_uint_32)png_ptr->bit_depth);
+#endif
+#ifdef PNG_WRITE_SWAP_SUPPORTED
+ if (png_ptr->transformations & PNG_SWAP_BYTES)
+ png_do_swap(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+#ifdef PNG_WRITE_SHIFT_SUPPORTED
+ if (png_ptr->transformations & PNG_SHIFT)
+ png_do_shift(&(png_ptr->row_info), png_ptr->row_buf + 1,
+ &(png_ptr->shift));
+#endif
+#ifdef PNG_WRITE_SWAP_ALPHA_SUPPORTED
+ if (png_ptr->transformations & PNG_SWAP_ALPHA)
+ png_do_write_swap_alpha(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+#ifdef PNG_WRITE_INVERT_ALPHA_SUPPORTED
+ if (png_ptr->transformations & PNG_INVERT_ALPHA)
+ png_do_write_invert_alpha(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+#ifdef PNG_WRITE_BGR_SUPPORTED
+ if (png_ptr->transformations & PNG_BGR)
+ png_do_bgr(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+#ifdef PNG_WRITE_INVERT_SUPPORTED
+ if (png_ptr->transformations & PNG_INVERT_MONO)
+ png_do_invert(&(png_ptr->row_info), png_ptr->row_buf + 1);
+#endif
+}
+
+#ifdef PNG_WRITE_PACK_SUPPORTED
+/* Pack pixels into bytes. Pass the true bit depth in bit_depth. The
+ * row_info bit depth should be 8 (one pixel per byte). The channels
+ * should be 1 (this only happens on grayscale and paletted images).
+ */
+void /* PRIVATE */
+png_do_pack(png_row_infop row_info, png_bytep row, png_uint_32 bit_depth)
+{
+ png_debug(1, "in png_do_pack");
+
+ if (row_info->bit_depth == 8 &&
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ row != NULL && row_info != NULL &&
+#endif
+ row_info->channels == 1)
+ {
+ switch ((int)bit_depth)
+ {
+ case 1:
+ {
+ png_bytep sp, dp;
+ int mask, v;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+
+ sp = row;
+ dp = row;
+ mask = 0x80;
+ v = 0;
+
+ for (i = 0; i < row_width; i++)
+ {
+ if (*sp != 0)
+ v |= mask;
+ sp++;
+ if (mask > 1)
+ mask >>= 1;
+ else
+ {
+ mask = 0x80;
+ *dp = (png_byte)v;
+ dp++;
+ v = 0;
+ }
+ }
+ if (mask != 0x80)
+ *dp = (png_byte)v;
+ break;
+ }
+ case 2:
+ {
+ png_bytep sp, dp;
+ int shift, v;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+
+ sp = row;
+ dp = row;
+ shift = 6;
+ v = 0;
+ for (i = 0; i < row_width; i++)
+ {
+ png_byte value;
+
+ value = (png_byte)(*sp & 0x03);
+ v |= (value << shift);
+ if (shift == 0)
+ {
+ shift = 6;
+ *dp = (png_byte)v;
+ dp++;
+ v = 0;
+ }
+ else
+ shift -= 2;
+ sp++;
+ }
+ if (shift != 6)
+ *dp = (png_byte)v;
+ break;
+ }
+ case 4:
+ {
+ png_bytep sp, dp;
+ int shift, v;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+
+ sp = row;
+ dp = row;
+ shift = 4;
+ v = 0;
+ for (i = 0; i < row_width; i++)
+ {
+ png_byte value;
+
+ value = (png_byte)(*sp & 0x0f);
+ v |= (value << shift);
+
+ if (shift == 0)
+ {
+ shift = 4;
+ *dp = (png_byte)v;
+ dp++;
+ v = 0;
+ }
+ else
+ shift -= 4;
+
+ sp++;
+ }
+ if (shift != 4)
+ *dp = (png_byte)v;
+ break;
+ }
+ }
+ row_info->bit_depth = (png_byte)bit_depth;
+ row_info->pixel_depth = (png_byte)(bit_depth * row_info->channels);
+ row_info->rowbytes = PNG_ROWBYTES(row_info->pixel_depth,
+ row_info->width);
+ }
+}
+#endif
+
+#ifdef PNG_WRITE_SHIFT_SUPPORTED
+/* Shift pixel values to take advantage of whole range. Pass the
+ * true number of bits in bit_depth. The row should be packed
+ * according to row_info->bit_depth. Thus, if you had a row of
+ * bit depth 4, but the pixels only had values from 0 to 7, you
+ * would pass 3 as bit_depth, and this routine would translate the
+ * data to 0 to 15.
+ */
+void /* PRIVATE */
+png_do_shift(png_row_infop row_info, png_bytep row, png_color_8p bit_depth)
+{
+ png_debug(1, "in png_do_shift");
+
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ if (row != NULL && row_info != NULL &&
+#else
+ if (
+#endif
+ row_info->color_type != PNG_COLOR_TYPE_PALETTE)
+ {
+ int shift_start[4], shift_dec[4];
+ int channels = 0;
+
+ if (row_info->color_type & PNG_COLOR_MASK_COLOR)
+ {
+ shift_start[channels] = row_info->bit_depth - bit_depth->red;
+ shift_dec[channels] = bit_depth->red;
+ channels++;
+ shift_start[channels] = row_info->bit_depth - bit_depth->green;
+ shift_dec[channels] = bit_depth->green;
+ channels++;
+ shift_start[channels] = row_info->bit_depth - bit_depth->blue;
+ shift_dec[channels] = bit_depth->blue;
+ channels++;
+ }
+ else
+ {
+ shift_start[channels] = row_info->bit_depth - bit_depth->gray;
+ shift_dec[channels] = bit_depth->gray;
+ channels++;
+ }
+ if (row_info->color_type & PNG_COLOR_MASK_ALPHA)
+ {
+ shift_start[channels] = row_info->bit_depth - bit_depth->alpha;
+ shift_dec[channels] = bit_depth->alpha;
+ channels++;
+ }
+
+ /* With low row depths, could only be grayscale, so one channel */
+ if (row_info->bit_depth < 8)
+ {
+ png_bytep bp = row;
+ png_uint_32 i;
+ png_byte mask;
+ png_uint_32 row_bytes = row_info->rowbytes;
+
+ if (bit_depth->gray == 1 && row_info->bit_depth == 2)
+ mask = 0x55;
+ else if (row_info->bit_depth == 4 && bit_depth->gray == 3)
+ mask = 0x11;
+ else
+ mask = 0xff;
+
+ for (i = 0; i < row_bytes; i++, bp++)
+ {
+ png_uint_16 v;
+ int j;
+
+ v = *bp;
+ *bp = 0;
+ for (j = shift_start[0]; j > -shift_dec[0]; j -= shift_dec[0])
+ {
+ if (j > 0)
+ *bp |= (png_byte)((v << j) & 0xff);
+ else
+ *bp |= (png_byte)((v >> (-j)) & mask);
+ }
+ }
+ }
+ else if (row_info->bit_depth == 8)
+ {
+ png_bytep bp = row;
+ png_uint_32 i;
+ png_uint_32 istop = channels * row_info->width;
+
+ for (i = 0; i < istop; i++, bp++)
+ {
+
+ png_uint_16 v;
+ int j;
+ int c = (int)(i%channels);
+
+ v = *bp;
+ *bp = 0;
+ for (j = shift_start[c]; j > -shift_dec[c]; j -= shift_dec[c])
+ {
+ if (j > 0)
+ *bp |= (png_byte)((v << j) & 0xff);
+ else
+ *bp |= (png_byte)((v >> (-j)) & 0xff);
+ }
+ }
+ }
+ else
+ {
+ png_bytep bp;
+ png_uint_32 i;
+ png_uint_32 istop = channels * row_info->width;
+
+ for (bp = row, i = 0; i < istop; i++)
+ {
+ int c = (int)(i%channels);
+ png_uint_16 value, v;
+ int j;
+
+ v = (png_uint_16)(((png_uint_16)(*bp) << 8) + *(bp + 1));
+ value = 0;
+ for (j = shift_start[c]; j > -shift_dec[c]; j -= shift_dec[c])
+ {
+ if (j > 0)
+ value |= (png_uint_16)((v << j) & (png_uint_16)0xffff);
+ else
+ value |= (png_uint_16)((v >> (-j)) & (png_uint_16)0xffff);
+ }
+ *bp++ = (png_byte)(value >> 8);
+ *bp++ = (png_byte)(value & 0xff);
+ }
+ }
+ }
+}
+#endif
+
+#ifdef PNG_WRITE_SWAP_ALPHA_SUPPORTED
+void /* PRIVATE */
+png_do_write_swap_alpha(png_row_infop row_info, png_bytep row)
+{
+ png_debug(1, "in png_do_write_swap_alpha");
+
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ if (row != NULL && row_info != NULL)
+#endif
+ {
+ if (row_info->color_type == PNG_COLOR_TYPE_RGB_ALPHA)
+ {
+ /* This converts from ARGB to RGBA */
+ if (row_info->bit_depth == 8)
+ {
+ png_bytep sp, dp;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+ for (i = 0, sp = dp = row; i < row_width; i++)
+ {
+ png_byte save = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = save;
+ }
+ }
+ /* This converts from AARRGGBB to RRGGBBAA */
+ else
+ {
+ png_bytep sp, dp;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+
+ for (i = 0, sp = dp = row; i < row_width; i++)
+ {
+ png_byte save[2];
+ save[0] = *(sp++);
+ save[1] = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = save[0];
+ *(dp++) = save[1];
+ }
+ }
+ }
+ else if (row_info->color_type == PNG_COLOR_TYPE_GRAY_ALPHA)
+ {
+ /* This converts from AG to GA */
+ if (row_info->bit_depth == 8)
+ {
+ png_bytep sp, dp;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+
+ for (i = 0, sp = dp = row; i < row_width; i++)
+ {
+ png_byte save = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = save;
+ }
+ }
+ /* This converts from AAGG to GGAA */
+ else
+ {
+ png_bytep sp, dp;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+
+ for (i = 0, sp = dp = row; i < row_width; i++)
+ {
+ png_byte save[2];
+ save[0] = *(sp++);
+ save[1] = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = save[0];
+ *(dp++) = save[1];
+ }
+ }
+ }
+ }
+}
+#endif
+
+#ifdef PNG_WRITE_INVERT_ALPHA_SUPPORTED
+void /* PRIVATE */
+png_do_write_invert_alpha(png_row_infop row_info, png_bytep row)
+{
+ png_debug(1, "in png_do_write_invert_alpha");
+
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ if (row != NULL && row_info != NULL)
+#endif
+ {
+ if (row_info->color_type == PNG_COLOR_TYPE_RGB_ALPHA)
+ {
+ /* This inverts the alpha channel in RGBA */
+ if (row_info->bit_depth == 8)
+ {
+ png_bytep sp, dp;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+ for (i = 0, sp = dp = row; i < row_width; i++)
+ {
+ /* Does nothing
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ */
+ sp+=3; dp = sp;
+ *(dp++) = (png_byte)(255 - *(sp++));
+ }
+ }
+ /* This inverts the alpha channel in RRGGBBAA */
+ else
+ {
+ png_bytep sp, dp;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+
+ for (i = 0, sp = dp = row; i < row_width; i++)
+ {
+ /* Does nothing
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ */
+ sp+=6; dp = sp;
+ *(dp++) = (png_byte)(255 - *(sp++));
+ *(dp++) = (png_byte)(255 - *(sp++));
+ }
+ }
+ }
+ else if (row_info->color_type == PNG_COLOR_TYPE_GRAY_ALPHA)
+ {
+ /* This inverts the alpha channel in GA */
+ if (row_info->bit_depth == 8)
+ {
+ png_bytep sp, dp;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+
+ for (i = 0, sp = dp = row; i < row_width; i++)
+ {
+ *(dp++) = *(sp++);
+ *(dp++) = (png_byte)(255 - *(sp++));
+ }
+ }
+ /* This inverts the alpha channel in GGAA */
+ else
+ {
+ png_bytep sp, dp;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+
+ for (i = 0, sp = dp = row; i < row_width; i++)
+ {
+ /* Does nothing
+ *(dp++) = *(sp++);
+ *(dp++) = *(sp++);
+ */
+ sp+=2; dp = sp;
+ *(dp++) = (png_byte)(255 - *(sp++));
+ *(dp++) = (png_byte)(255 - *(sp++));
+ }
+ }
+ }
+ }
+}
+#endif
+
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+/* Undoes intrapixel differencing */
+void /* PRIVATE */
+png_do_write_intrapixel(png_row_infop row_info, png_bytep row)
+{
+ png_debug(1, "in png_do_write_intrapixel");
+
+ if (
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ row != NULL && row_info != NULL &&
+#endif
+ (row_info->color_type & PNG_COLOR_MASK_COLOR))
+ {
+ int bytes_per_pixel;
+ png_uint_32 row_width = row_info->width;
+ if (row_info->bit_depth == 8)
+ {
+ png_bytep rp;
+ png_uint_32 i;
+
+ if (row_info->color_type == PNG_COLOR_TYPE_RGB)
+ bytes_per_pixel = 3;
+ else if (row_info->color_type == PNG_COLOR_TYPE_RGB_ALPHA)
+ bytes_per_pixel = 4;
+ else
+ return;
+
+ for (i = 0, rp = row; i < row_width; i++, rp += bytes_per_pixel)
+ {
+ *(rp) = (png_byte)((*rp - *(rp+1))&0xff);
+ *(rp+2) = (png_byte)((*(rp+2) - *(rp+1))&0xff);
+ }
+ }
+ else if (row_info->bit_depth == 16)
+ {
+ png_bytep rp;
+ png_uint_32 i;
+
+ if (row_info->color_type == PNG_COLOR_TYPE_RGB)
+ bytes_per_pixel = 6;
+ else if (row_info->color_type == PNG_COLOR_TYPE_RGB_ALPHA)
+ bytes_per_pixel = 8;
+ else
+ return;
+
+ for (i = 0, rp = row; i < row_width; i++, rp += bytes_per_pixel)
+ {
+ png_uint_32 s0 = (*(rp ) << 8) | *(rp+1);
+ png_uint_32 s1 = (*(rp+2) << 8) | *(rp+3);
+ png_uint_32 s2 = (*(rp+4) << 8) | *(rp+5);
+ png_uint_32 red = (png_uint_32)((s0 - s1) & 0xffffL);
+ png_uint_32 blue = (png_uint_32)((s2 - s1) & 0xffffL);
+ *(rp ) = (png_byte)((red >> 8) & 0xff);
+ *(rp+1) = (png_byte)(red & 0xff);
+ *(rp+4) = (png_byte)((blue >> 8) & 0xff);
+ *(rp+5) = (png_byte)(blue & 0xff);
+ }
+ }
+ }
+}
+#endif /* PNG_MNG_FEATURES_SUPPORTED */
+#endif /* PNG_WRITE_SUPPORTED */
diff --git a/trunk/src/third_party/libpng/pngwutil.c b/trunk/src/third_party/libpng/pngwutil.c
new file mode 100644
index 0000000..c75f53e
--- /dev/null
+++ b/trunk/src/third_party/libpng/pngwutil.c
@@ -0,0 +1,2832 @@
+
+/* pngwutil.c - utilities to write a PNG file
+ *
+ * Last changed in libpng 1.2.43 [February 25, 2010]
+ * Copyright (c) 1998-2010 Glenn Randers-Pehrson
+ * (Version 0.96 Copyright (c) 1996, 1997 Andreas Dilger)
+ * (Version 0.88 Copyright (c) 1995, 1996 Guy Eric Schalnat, Group 42, Inc.)
+ *
+ * This code is released under the libpng license.
+ * For conditions of distribution and use, see the disclaimer
+ * and license in png.h
+ */
+
+#define PNG_INTERNAL
+#define PNG_NO_PEDANTIC_WARNINGS
+#include "png.h"
+#ifdef PNG_WRITE_SUPPORTED
+
+/* Place a 32-bit number into a buffer in PNG byte order. We work
+ * with unsigned numbers for convenience, although one supported
+ * ancillary chunk uses signed (two's complement) numbers.
+ */
+void PNGAPI
+png_save_uint_32(png_bytep buf, png_uint_32 i)
+{
+ buf[0] = (png_byte)((i >> 24) & 0xff);
+ buf[1] = (png_byte)((i >> 16) & 0xff);
+ buf[2] = (png_byte)((i >> 8) & 0xff);
+ buf[3] = (png_byte)(i & 0xff);
+}
+
+/* The png_save_int_32 function assumes integers are stored in two's
+ * complement format. If this isn't the case, then this routine needs to
+ * be modified to write data in two's complement format.
+ */
+void PNGAPI
+png_save_int_32(png_bytep buf, png_int_32 i)
+{
+ buf[0] = (png_byte)((i >> 24) & 0xff);
+ buf[1] = (png_byte)((i >> 16) & 0xff);
+ buf[2] = (png_byte)((i >> 8) & 0xff);
+ buf[3] = (png_byte)(i & 0xff);
+}
+
+/* Place a 16-bit number into a buffer in PNG byte order.
+ * The parameter is declared unsigned int, not png_uint_16,
+ * just to avoid potential problems on pre-ANSI C compilers.
+ */
+void PNGAPI
+png_save_uint_16(png_bytep buf, unsigned int i)
+{
+ buf[0] = (png_byte)((i >> 8) & 0xff);
+ buf[1] = (png_byte)(i & 0xff);
+}
+
+/* Simple function to write the signature. If we have already written
+ * the magic bytes of the signature, or more likely, the PNG stream is
+ * being embedded into another stream and doesn't need its own signature,
+ * we should call png_set_sig_bytes() to tell libpng how many of the
+ * bytes have already been written.
+ */
+void /* PRIVATE */
+png_write_sig(png_structp png_ptr)
+{
+ png_byte png_signature[8] = {137, 80, 78, 71, 13, 10, 26, 10};
+
+ /* Write the rest of the 8 byte signature */
+ png_write_data(png_ptr, &png_signature[png_ptr->sig_bytes],
+ (png_size_t)(8 - png_ptr->sig_bytes));
+ if (png_ptr->sig_bytes < 3)
+ png_ptr->mode |= PNG_HAVE_PNG_SIGNATURE;
+}
+
+/* Write a PNG chunk all at once. The type is an array of ASCII characters
+ * representing the chunk name. The array must be at least 4 bytes in
+ * length, and does not need to be null terminated. To be safe, pass the
+ * pre-defined chunk names here, and if you need a new one, define it
+ * where the others are defined. The length is the length of the data.
+ * All the data must be present. If that is not possible, use the
+ * png_write_chunk_start(), png_write_chunk_data(), and png_write_chunk_end()
+ * functions instead.
+ */
+void PNGAPI
+png_write_chunk(png_structp png_ptr, png_bytep chunk_name,
+ png_bytep data, png_size_t length)
+{
+ if (png_ptr == NULL)
+ return;
+ png_write_chunk_start(png_ptr, chunk_name, (png_uint_32)length);
+ png_write_chunk_data(png_ptr, data, (png_size_t)length);
+ png_write_chunk_end(png_ptr);
+}
+
+/* Write the start of a PNG chunk. The type is the chunk type.
+ * The total_length is the sum of the lengths of all the data you will be
+ * passing in png_write_chunk_data().
+ */
+void PNGAPI
+png_write_chunk_start(png_structp png_ptr, png_bytep chunk_name,
+ png_uint_32 length)
+{
+ png_byte buf[8];
+
+ png_debug2(0, "Writing %s chunk, length = %lu", chunk_name,
+ (unsigned long)length);
+
+ if (png_ptr == NULL)
+ return;
+
+
+ /* Write the length and the chunk name */
+ png_save_uint_32(buf, length);
+ png_memcpy(buf + 4, chunk_name, 4);
+ png_write_data(png_ptr, buf, (png_size_t)8);
+ /* Put the chunk name into png_ptr->chunk_name */
+ png_memcpy(png_ptr->chunk_name, chunk_name, 4);
+ /* Reset the crc and run it over the chunk name */
+ png_reset_crc(png_ptr);
+ png_calculate_crc(png_ptr, chunk_name, (png_size_t)4);
+}
+
+/* Write the data of a PNG chunk started with png_write_chunk_start().
+ * Note that multiple calls to this function are allowed, and that the
+ * sum of the lengths from these calls *must* add up to the total_length
+ * given to png_write_chunk_start().
+ */
+void PNGAPI
+png_write_chunk_data(png_structp png_ptr, png_bytep data, png_size_t length)
+{
+ /* Write the data, and run the CRC over it */
+ if (png_ptr == NULL)
+ return;
+ if (data != NULL && length > 0)
+ {
+ png_write_data(png_ptr, data, length);
+ /* Update the CRC after writing the data,
+ * in case that the user I/O routine alters it.
+ */
+ png_calculate_crc(png_ptr, data, length);
+ }
+}
+
+/* Finish a chunk started with png_write_chunk_start(). */
+void PNGAPI
+png_write_chunk_end(png_structp png_ptr)
+{
+ png_byte buf[4];
+
+ if (png_ptr == NULL) return;
+
+ /* Write the crc in a single operation */
+ png_save_uint_32(buf, png_ptr->crc);
+
+ png_write_data(png_ptr, buf, (png_size_t)4);
+}
+
+#if defined(PNG_WRITE_TEXT_SUPPORTED) || defined(PNG_WRITE_iCCP_SUPPORTED)
+/* This pair of functions encapsulates the operation of (a) compressing a
+ * text string, and (b) issuing it later as a series of chunk data writes.
+ * The compression_state structure is shared context for these functions
+ * set up by the caller in order to make the whole mess thread-safe.
+ */
+
+typedef struct
+{
+ char *input; /* The uncompressed input data */
+ int input_len; /* Its length */
+ int num_output_ptr; /* Number of output pointers used */
+ int max_output_ptr; /* Size of output_ptr */
+ png_charpp output_ptr; /* Array of pointers to output */
+} compression_state;
+
+/* Compress given text into storage in the png_ptr structure */
+static int /* PRIVATE */
+png_text_compress(png_structp png_ptr,
+ png_charp text, png_size_t text_len, int compression,
+ compression_state *comp)
+{
+ int ret;
+
+ comp->num_output_ptr = 0;
+ comp->max_output_ptr = 0;
+ comp->output_ptr = NULL;
+ comp->input = NULL;
+ comp->input_len = 0;
+
+ /* We may just want to pass the text right through */
+ if (compression == PNG_TEXT_COMPRESSION_NONE)
+ {
+ comp->input = text;
+ comp->input_len = text_len;
+ return((int)text_len);
+ }
+
+ if (compression >= PNG_TEXT_COMPRESSION_LAST)
+ {
+#if defined(PNG_STDIO_SUPPORTED) && !defined(_WIN32_WCE)
+ char msg[50];
+ png_snprintf(msg, 50, "Unknown compression type %d", compression);
+ png_warning(png_ptr, msg);
+#else
+ png_warning(png_ptr, "Unknown compression type");
+#endif
+ }
+
+ /* We can't write the chunk until we find out how much data we have,
+ * which means we need to run the compressor first and save the
+ * output. This shouldn't be a problem, as the vast majority of
+ * comments should be reasonable, but we will set up an array of
+ * malloc'd pointers to be sure.
+ *
+ * If we knew the application was well behaved, we could simplify this
+ * greatly by assuming we can always malloc an output buffer large
+ * enough to hold the compressed text ((1001 * text_len / 1000) + 12)
+ * and malloc this directly. The only time this would be a bad idea is
+ * if we can't malloc more than 64K and we have 64K of random input
+ * data, or if the input string is incredibly large (although this
+ * wouldn't cause a failure, just a slowdown due to swapping).
+ */
+
+ /* Set up the compression buffers */
+ png_ptr->zstream.avail_in = (uInt)text_len;
+ png_ptr->zstream.next_in = (Bytef *)text;
+ png_ptr->zstream.avail_out = (uInt)png_ptr->zbuf_size;
+ png_ptr->zstream.next_out = (Bytef *)png_ptr->zbuf;
+
+ /* This is the same compression loop as in png_write_row() */
+ do
+ {
+ /* Compress the data */
+ ret = deflate(&png_ptr->zstream, Z_NO_FLUSH);
+ if (ret != Z_OK)
+ {
+ /* Error */
+ if (png_ptr->zstream.msg != NULL)
+ png_error(png_ptr, png_ptr->zstream.msg);
+ else
+ png_error(png_ptr, "zlib error");
+ }
+ /* Check to see if we need more room */
+ if (!(png_ptr->zstream.avail_out))
+ {
+ /* Make sure the output array has room */
+ if (comp->num_output_ptr >= comp->max_output_ptr)
+ {
+ int old_max;
+
+ old_max = comp->max_output_ptr;
+ comp->max_output_ptr = comp->num_output_ptr + 4;
+ if (comp->output_ptr != NULL)
+ {
+ png_charpp old_ptr;
+
+ old_ptr = comp->output_ptr;
+ comp->output_ptr = (png_charpp)png_malloc(png_ptr,
+ (png_uint_32)
+ (comp->max_output_ptr * png_sizeof(png_charpp)));
+ png_memcpy(comp->output_ptr, old_ptr, old_max
+ * png_sizeof(png_charp));
+ png_free(png_ptr, old_ptr);
+ }
+ else
+ comp->output_ptr = (png_charpp)png_malloc(png_ptr,
+ (png_uint_32)
+ (comp->max_output_ptr * png_sizeof(png_charp)));
+ }
+
+ /* Save the data */
+ comp->output_ptr[comp->num_output_ptr] =
+ (png_charp)png_malloc(png_ptr,
+ (png_uint_32)png_ptr->zbuf_size);
+ png_memcpy(comp->output_ptr[comp->num_output_ptr], png_ptr->zbuf,
+ png_ptr->zbuf_size);
+ comp->num_output_ptr++;
+
+ /* and reset the buffer */
+ png_ptr->zstream.avail_out = (uInt)png_ptr->zbuf_size;
+ png_ptr->zstream.next_out = png_ptr->zbuf;
+ }
+ /* Continue until we don't have any more to compress */
+ } while (png_ptr->zstream.avail_in);
+
+ /* Finish the compression */
+ do
+ {
+ /* Tell zlib we are finished */
+ ret = deflate(&png_ptr->zstream, Z_FINISH);
+
+ if (ret == Z_OK)
+ {
+ /* Check to see if we need more room */
+ if (!(png_ptr->zstream.avail_out))
+ {
+ /* Check to make sure our output array has room */
+ if (comp->num_output_ptr >= comp->max_output_ptr)
+ {
+ int old_max;
+
+ old_max = comp->max_output_ptr;
+ comp->max_output_ptr = comp->num_output_ptr + 4;
+ if (comp->output_ptr != NULL)
+ {
+ png_charpp old_ptr;
+
+ old_ptr = comp->output_ptr;
+ /* This could be optimized to realloc() */
+ comp->output_ptr = (png_charpp)png_malloc(png_ptr,
+ (png_uint_32)(comp->max_output_ptr *
+ png_sizeof(png_charp)));
+ png_memcpy(comp->output_ptr, old_ptr,
+ old_max * png_sizeof(png_charp));
+ png_free(png_ptr, old_ptr);
+ }
+ else
+ comp->output_ptr = (png_charpp)png_malloc(png_ptr,
+ (png_uint_32)(comp->max_output_ptr *
+ png_sizeof(png_charp)));
+ }
+
+ /* Save the data */
+ comp->output_ptr[comp->num_output_ptr] =
+ (png_charp)png_malloc(png_ptr,
+ (png_uint_32)png_ptr->zbuf_size);
+ png_memcpy(comp->output_ptr[comp->num_output_ptr], png_ptr->zbuf,
+ png_ptr->zbuf_size);
+ comp->num_output_ptr++;
+
+ /* and reset the buffer pointers */
+ png_ptr->zstream.avail_out = (uInt)png_ptr->zbuf_size;
+ png_ptr->zstream.next_out = png_ptr->zbuf;
+ }
+ }
+ else if (ret != Z_STREAM_END)
+ {
+ /* We got an error */
+ if (png_ptr->zstream.msg != NULL)
+ png_error(png_ptr, png_ptr->zstream.msg);
+ else
+ png_error(png_ptr, "zlib error");
+ }
+ } while (ret != Z_STREAM_END);
+
+ /* Text length is number of buffers plus last buffer */
+ text_len = png_ptr->zbuf_size * comp->num_output_ptr;
+ if (png_ptr->zstream.avail_out < png_ptr->zbuf_size)
+ text_len += png_ptr->zbuf_size - (png_size_t)png_ptr->zstream.avail_out;
+
+ return((int)text_len);
+}
+
+/* Ship the compressed text out via chunk writes */
+static void /* PRIVATE */
+png_write_compressed_data_out(png_structp png_ptr, compression_state *comp)
+{
+ int i;
+
+ /* Handle the no-compression case */
+ if (comp->input)
+ {
+ png_write_chunk_data(png_ptr, (png_bytep)comp->input,
+ (png_size_t)comp->input_len);
+ return;
+ }
+
+ /* Write saved output buffers, if any */
+ for (i = 0; i < comp->num_output_ptr; i++)
+ {
+ png_write_chunk_data(png_ptr, (png_bytep)comp->output_ptr[i],
+ (png_size_t)png_ptr->zbuf_size);
+ png_free(png_ptr, comp->output_ptr[i]);
+ comp->output_ptr[i]=NULL;
+ }
+ if (comp->max_output_ptr != 0)
+ png_free(png_ptr, comp->output_ptr);
+ comp->output_ptr=NULL;
+ /* Write anything left in zbuf */
+ if (png_ptr->zstream.avail_out < (png_uint_32)png_ptr->zbuf_size)
+ png_write_chunk_data(png_ptr, png_ptr->zbuf,
+ (png_size_t)(png_ptr->zbuf_size - png_ptr->zstream.avail_out));
+
+ /* Reset zlib for another zTXt/iTXt or image data */
+ deflateReset(&png_ptr->zstream);
+ png_ptr->zstream.data_type = Z_BINARY;
+}
+#endif
+
+/* Write the IHDR chunk, and update the png_struct with the necessary
+ * information. Note that the rest of this code depends upon this
+ * information being correct.
+ */
+void /* PRIVATE */
+png_write_IHDR(png_structp png_ptr, png_uint_32 width, png_uint_32 height,
+ int bit_depth, int color_type, int compression_type, int filter_type,
+ int interlace_type)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_IHDR;
+#endif
+ int ret;
+
+ png_byte buf[13]; /* Buffer to store the IHDR info */
+
+ png_debug(1, "in png_write_IHDR");
+
+ /* Check that we have valid input data from the application info */
+ switch (color_type)
+ {
+ case PNG_COLOR_TYPE_GRAY:
+ switch (bit_depth)
+ {
+ case 1:
+ case 2:
+ case 4:
+ case 8:
+ case 16: png_ptr->channels = 1; break;
+ default: png_error(png_ptr,
+ "Invalid bit depth for grayscale image");
+ }
+ break;
+ case PNG_COLOR_TYPE_RGB:
+ if (bit_depth != 8 && bit_depth != 16)
+ png_error(png_ptr, "Invalid bit depth for RGB image");
+ png_ptr->channels = 3;
+ break;
+ case PNG_COLOR_TYPE_PALETTE:
+ switch (bit_depth)
+ {
+ case 1:
+ case 2:
+ case 4:
+ case 8: png_ptr->channels = 1; break;
+ default: png_error(png_ptr, "Invalid bit depth for paletted image");
+ }
+ break;
+ case PNG_COLOR_TYPE_GRAY_ALPHA:
+ if (bit_depth != 8 && bit_depth != 16)
+ png_error(png_ptr, "Invalid bit depth for grayscale+alpha image");
+ png_ptr->channels = 2;
+ break;
+ case PNG_COLOR_TYPE_RGB_ALPHA:
+ if (bit_depth != 8 && bit_depth != 16)
+ png_error(png_ptr, "Invalid bit depth for RGBA image");
+ png_ptr->channels = 4;
+ break;
+ default:
+ png_error(png_ptr, "Invalid image color type specified");
+ }
+
+ if (compression_type != PNG_COMPRESSION_TYPE_BASE)
+ {
+ png_warning(png_ptr, "Invalid compression type specified");
+ compression_type = PNG_COMPRESSION_TYPE_BASE;
+ }
+
+ /* Write filter_method 64 (intrapixel differencing) only if
+ * 1. Libpng was compiled with PNG_MNG_FEATURES_SUPPORTED and
+ * 2. Libpng did not write a PNG signature (this filter_method is only
+ * used in PNG datastreams that are embedded in MNG datastreams) and
+ * 3. The application called png_permit_mng_features with a mask that
+ * included PNG_FLAG_MNG_FILTER_64 and
+ * 4. The filter_method is 64 and
+ * 5. The color_type is RGB or RGBA
+ */
+ if (
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+ !((png_ptr->mng_features_permitted & PNG_FLAG_MNG_FILTER_64) &&
+ ((png_ptr->mode&PNG_HAVE_PNG_SIGNATURE) == 0) &&
+ (color_type == PNG_COLOR_TYPE_RGB ||
+ color_type == PNG_COLOR_TYPE_RGB_ALPHA) &&
+ (filter_type == PNG_INTRAPIXEL_DIFFERENCING)) &&
+#endif
+ filter_type != PNG_FILTER_TYPE_BASE)
+ {
+ png_warning(png_ptr, "Invalid filter type specified");
+ filter_type = PNG_FILTER_TYPE_BASE;
+ }
+
+#ifdef PNG_WRITE_INTERLACING_SUPPORTED
+ if (interlace_type != PNG_INTERLACE_NONE &&
+ interlace_type != PNG_INTERLACE_ADAM7)
+ {
+ png_warning(png_ptr, "Invalid interlace type specified");
+ interlace_type = PNG_INTERLACE_ADAM7;
+ }
+#else
+ interlace_type=PNG_INTERLACE_NONE;
+#endif
+
+ /* Save the relevent information */
+ png_ptr->bit_depth = (png_byte)bit_depth;
+ png_ptr->color_type = (png_byte)color_type;
+ png_ptr->interlaced = (png_byte)interlace_type;
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+ png_ptr->filter_type = (png_byte)filter_type;
+#endif
+ png_ptr->compression_type = (png_byte)compression_type;
+ png_ptr->width = width;
+ png_ptr->height = height;
+
+ png_ptr->pixel_depth = (png_byte)(bit_depth * png_ptr->channels);
+ png_ptr->rowbytes = PNG_ROWBYTES(png_ptr->pixel_depth, width);
+ /* Set the usr info, so any transformations can modify it */
+ png_ptr->usr_width = png_ptr->width;
+ png_ptr->usr_bit_depth = png_ptr->bit_depth;
+ png_ptr->usr_channels = png_ptr->channels;
+
+ /* Pack the header information into the buffer */
+ png_save_uint_32(buf, width);
+ png_save_uint_32(buf + 4, height);
+ buf[8] = (png_byte)bit_depth;
+ buf[9] = (png_byte)color_type;
+ buf[10] = (png_byte)compression_type;
+ buf[11] = (png_byte)filter_type;
+ buf[12] = (png_byte)interlace_type;
+
+ /* Write the chunk */
+ png_write_chunk(png_ptr, (png_bytep)png_IHDR, buf, (png_size_t)13);
+
+ /* Initialize zlib with PNG info */
+ png_ptr->zstream.zalloc = png_zalloc;
+ png_ptr->zstream.zfree = png_zfree;
+ png_ptr->zstream.opaque = (voidpf)png_ptr;
+ if (!(png_ptr->do_filter))
+ {
+ if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE ||
+ png_ptr->bit_depth < 8)
+ png_ptr->do_filter = PNG_FILTER_NONE;
+ else
+ png_ptr->do_filter = PNG_ALL_FILTERS;
+ }
+ if (!(png_ptr->flags & PNG_FLAG_ZLIB_CUSTOM_STRATEGY))
+ {
+ if (png_ptr->do_filter != PNG_FILTER_NONE)
+ png_ptr->zlib_strategy = Z_FILTERED;
+ else
+ png_ptr->zlib_strategy = Z_DEFAULT_STRATEGY;
+ }
+ if (!(png_ptr->flags & PNG_FLAG_ZLIB_CUSTOM_LEVEL))
+ png_ptr->zlib_level = Z_DEFAULT_COMPRESSION;
+ if (!(png_ptr->flags & PNG_FLAG_ZLIB_CUSTOM_MEM_LEVEL))
+ png_ptr->zlib_mem_level = 8;
+ if (!(png_ptr->flags & PNG_FLAG_ZLIB_CUSTOM_WINDOW_BITS))
+ png_ptr->zlib_window_bits = 15;
+ if (!(png_ptr->flags & PNG_FLAG_ZLIB_CUSTOM_METHOD))
+ png_ptr->zlib_method = 8;
+ ret = deflateInit2(&png_ptr->zstream, png_ptr->zlib_level,
+ png_ptr->zlib_method, png_ptr->zlib_window_bits,
+ png_ptr->zlib_mem_level, png_ptr->zlib_strategy);
+ if (ret != Z_OK)
+ {
+ if (ret == Z_VERSION_ERROR) png_error(png_ptr,
+ "zlib failed to initialize compressor -- version error");
+ if (ret == Z_STREAM_ERROR) png_error(png_ptr,
+ "zlib failed to initialize compressor -- stream error");
+ if (ret == Z_MEM_ERROR) png_error(png_ptr,
+ "zlib failed to initialize compressor -- mem error");
+ png_error(png_ptr, "zlib failed to initialize compressor");
+ }
+ png_ptr->zstream.next_out = png_ptr->zbuf;
+ png_ptr->zstream.avail_out = (uInt)png_ptr->zbuf_size;
+ /* libpng is not interested in zstream.data_type */
+ /* Set it to a predefined value, to avoid its evaluation inside zlib */
+ png_ptr->zstream.data_type = Z_BINARY;
+
+ png_ptr->mode = PNG_HAVE_IHDR;
+}
+
+/* Write the palette. We are careful not to trust png_color to be in the
+ * correct order for PNG, so people can redefine it to any convenient
+ * structure.
+ */
+void /* PRIVATE */
+png_write_PLTE(png_structp png_ptr, png_colorp palette, png_uint_32 num_pal)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_PLTE;
+#endif
+ png_uint_32 i;
+ png_colorp pal_ptr;
+ png_byte buf[3];
+
+ png_debug(1, "in png_write_PLTE");
+
+ if ((
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+ !(png_ptr->mng_features_permitted & PNG_FLAG_MNG_EMPTY_PLTE) &&
+#endif
+ num_pal == 0) || num_pal > 256)
+ {
+ if (png_ptr->color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ png_error(png_ptr, "Invalid number of colors in palette");
+ }
+ else
+ {
+ png_warning(png_ptr, "Invalid number of colors in palette");
+ return;
+ }
+ }
+
+ if (!(png_ptr->color_type&PNG_COLOR_MASK_COLOR))
+ {
+ png_warning(png_ptr,
+ "Ignoring request to write a PLTE chunk in grayscale PNG");
+ return;
+ }
+
+ png_ptr->num_palette = (png_uint_16)num_pal;
+ png_debug1(3, "num_palette = %d", png_ptr->num_palette);
+
+ png_write_chunk_start(png_ptr, (png_bytep)png_PLTE,
+ (png_uint_32)(num_pal * 3));
+#ifdef PNG_POINTER_INDEXING_SUPPORTED
+ for (i = 0, pal_ptr = palette; i < num_pal; i++, pal_ptr++)
+ {
+ buf[0] = pal_ptr->red;
+ buf[1] = pal_ptr->green;
+ buf[2] = pal_ptr->blue;
+ png_write_chunk_data(png_ptr, buf, (png_size_t)3);
+ }
+#else
+ /* This is a little slower but some buggy compilers need to do this
+ * instead
+ */
+ pal_ptr=palette;
+ for (i = 0; i < num_pal; i++)
+ {
+ buf[0] = pal_ptr[i].red;
+ buf[1] = pal_ptr[i].green;
+ buf[2] = pal_ptr[i].blue;
+ png_write_chunk_data(png_ptr, buf, (png_size_t)3);
+ }
+#endif
+ png_write_chunk_end(png_ptr);
+ png_ptr->mode |= PNG_HAVE_PLTE;
+}
+
+/* Write an IDAT chunk */
+void /* PRIVATE */
+png_write_IDAT(png_structp png_ptr, png_bytep data, png_size_t length)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_IDAT;
+#endif
+
+ png_debug(1, "in png_write_IDAT");
+
+ /* Optimize the CMF field in the zlib stream. */
+ /* This hack of the zlib stream is compliant to the stream specification. */
+ if (!(png_ptr->mode & PNG_HAVE_IDAT) &&
+ png_ptr->compression_type == PNG_COMPRESSION_TYPE_BASE)
+ {
+ unsigned int z_cmf = data[0]; /* zlib compression method and flags */
+ if ((z_cmf & 0x0f) == 8 && (z_cmf & 0xf0) <= 0x70)
+ {
+ /* Avoid memory underflows and multiplication overflows.
+ *
+ * The conditions below are practically always satisfied;
+ * however, they still must be checked.
+ */
+ if (length >= 2 &&
+ png_ptr->height < 16384 && png_ptr->width < 16384)
+ {
+ png_uint_32 uncompressed_idat_size = png_ptr->height *
+ ((png_ptr->width *
+ png_ptr->channels * png_ptr->bit_depth + 15) >> 3);
+ unsigned int z_cinfo = z_cmf >> 4;
+ unsigned int half_z_window_size = 1 << (z_cinfo + 7);
+ while (uncompressed_idat_size <= half_z_window_size &&
+ half_z_window_size >= 256)
+ {
+ z_cinfo--;
+ half_z_window_size >>= 1;
+ }
+ z_cmf = (z_cmf & 0x0f) | (z_cinfo << 4);
+ if (data[0] != (png_byte)z_cmf)
+ {
+ data[0] = (png_byte)z_cmf;
+ data[1] &= 0xe0;
+ data[1] += (png_byte)(0x1f - ((z_cmf << 8) + data[1]) % 0x1f);
+ }
+ }
+ }
+ else
+ png_error(png_ptr,
+ "Invalid zlib compression method or flags in IDAT");
+ }
+
+ png_write_chunk(png_ptr, (png_bytep)png_IDAT, data, length);
+ png_ptr->mode |= PNG_HAVE_IDAT;
+}
+
+/* Write an IEND chunk */
+void /* PRIVATE */
+png_write_IEND(png_structp png_ptr)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_IEND;
+#endif
+
+ png_debug(1, "in png_write_IEND");
+
+ png_write_chunk(png_ptr, (png_bytep)png_IEND, png_bytep_NULL,
+ (png_size_t)0);
+ png_ptr->mode |= PNG_HAVE_IEND;
+}
+
+#ifdef PNG_WRITE_gAMA_SUPPORTED
+/* Write a gAMA chunk */
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+void /* PRIVATE */
+png_write_gAMA(png_structp png_ptr, double file_gamma)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_gAMA;
+#endif
+ png_uint_32 igamma;
+ png_byte buf[4];
+
+ png_debug(1, "in png_write_gAMA");
+
+ /* file_gamma is saved in 1/100,000ths */
+ igamma = (png_uint_32)(file_gamma * 100000.0 + 0.5);
+ png_save_uint_32(buf, igamma);
+ png_write_chunk(png_ptr, (png_bytep)png_gAMA, buf, (png_size_t)4);
+}
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+void /* PRIVATE */
+png_write_gAMA_fixed(png_structp png_ptr, png_fixed_point file_gamma)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_gAMA;
+#endif
+ png_byte buf[4];
+
+ png_debug(1, "in png_write_gAMA");
+
+ /* file_gamma is saved in 1/100,000ths */
+ png_save_uint_32(buf, (png_uint_32)file_gamma);
+ png_write_chunk(png_ptr, (png_bytep)png_gAMA, buf, (png_size_t)4);
+}
+#endif
+#endif
+
+#ifdef PNG_WRITE_sRGB_SUPPORTED
+/* Write a sRGB chunk */
+void /* PRIVATE */
+png_write_sRGB(png_structp png_ptr, int srgb_intent)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_sRGB;
+#endif
+ png_byte buf[1];
+
+ png_debug(1, "in png_write_sRGB");
+
+ if (srgb_intent >= PNG_sRGB_INTENT_LAST)
+ png_warning(png_ptr,
+ "Invalid sRGB rendering intent specified");
+ buf[0]=(png_byte)srgb_intent;
+ png_write_chunk(png_ptr, (png_bytep)png_sRGB, buf, (png_size_t)1);
+}
+#endif
+
+#ifdef PNG_WRITE_iCCP_SUPPORTED
+/* Write an iCCP chunk */
+void /* PRIVATE */
+png_write_iCCP(png_structp png_ptr, png_charp name, int compression_type,
+ png_charp profile, int profile_len)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_iCCP;
+#endif
+ png_size_t name_len;
+ png_charp new_name;
+ compression_state comp;
+ int embedded_profile_len = 0;
+
+ png_debug(1, "in png_write_iCCP");
+
+ comp.num_output_ptr = 0;
+ comp.max_output_ptr = 0;
+ comp.output_ptr = NULL;
+ comp.input = NULL;
+ comp.input_len = 0;
+
+ if ((name_len = png_check_keyword(png_ptr, name,
+ &new_name)) == 0)
+ return;
+
+ if (compression_type != PNG_COMPRESSION_TYPE_BASE)
+ png_warning(png_ptr, "Unknown compression type in iCCP chunk");
+
+ if (profile == NULL)
+ profile_len = 0;
+
+ if (profile_len > 3)
+ embedded_profile_len =
+ ((*( (png_bytep)profile ))<<24) |
+ ((*( (png_bytep)profile + 1))<<16) |
+ ((*( (png_bytep)profile + 2))<< 8) |
+ ((*( (png_bytep)profile + 3)) );
+
+ if (embedded_profile_len < 0)
+ {
+ png_warning(png_ptr,
+ "Embedded profile length in iCCP chunk is negative");
+ png_free(png_ptr, new_name);
+ return;
+ }
+
+ if (profile_len < embedded_profile_len)
+ {
+ png_warning(png_ptr,
+ "Embedded profile length too large in iCCP chunk");
+ png_free(png_ptr, new_name);
+ return;
+ }
+
+ if (profile_len > embedded_profile_len)
+ {
+ png_warning(png_ptr,
+ "Truncating profile to actual length in iCCP chunk");
+ profile_len = embedded_profile_len;
+ }
+
+ if (profile_len)
+ profile_len = png_text_compress(png_ptr, profile,
+ (png_size_t)profile_len, PNG_COMPRESSION_TYPE_BASE, &comp);
+
+ /* Make sure we include the NULL after the name and the compression type */
+ png_write_chunk_start(png_ptr, (png_bytep)png_iCCP,
+ (png_uint_32)(name_len + profile_len + 2));
+ new_name[name_len + 1] = 0x00;
+ png_write_chunk_data(png_ptr, (png_bytep)new_name,
+ (png_size_t)(name_len + 2));
+
+ if (profile_len)
+ png_write_compressed_data_out(png_ptr, &comp);
+
+ png_write_chunk_end(png_ptr);
+ png_free(png_ptr, new_name);
+}
+#endif
+
+#ifdef PNG_WRITE_sPLT_SUPPORTED
+/* Write a sPLT chunk */
+void /* PRIVATE */
+png_write_sPLT(png_structp png_ptr, png_sPLT_tp spalette)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_sPLT;
+#endif
+ png_size_t name_len;
+ png_charp new_name;
+ png_byte entrybuf[10];
+ int entry_size = (spalette->depth == 8 ? 6 : 10);
+ int palette_size = entry_size * spalette->nentries;
+ png_sPLT_entryp ep;
+#ifndef PNG_POINTER_INDEXING_SUPPORTED
+ int i;
+#endif
+
+ png_debug(1, "in png_write_sPLT");
+
+ if ((name_len = png_check_keyword(png_ptr,spalette->name, &new_name))==0)
+ return;
+
+ /* Make sure we include the NULL after the name */
+ png_write_chunk_start(png_ptr, (png_bytep)png_sPLT,
+ (png_uint_32)(name_len + 2 + palette_size));
+ png_write_chunk_data(png_ptr, (png_bytep)new_name,
+ (png_size_t)(name_len + 1));
+ png_write_chunk_data(png_ptr, (png_bytep)&spalette->depth, (png_size_t)1);
+
+ /* Loop through each palette entry, writing appropriately */
+#ifdef PNG_POINTER_INDEXING_SUPPORTED
+ for (ep = spalette->entries; ep<spalette->entries + spalette->nentries; ep++)
+ {
+ if (spalette->depth == 8)
+ {
+ entrybuf[0] = (png_byte)ep->red;
+ entrybuf[1] = (png_byte)ep->green;
+ entrybuf[2] = (png_byte)ep->blue;
+ entrybuf[3] = (png_byte)ep->alpha;
+ png_save_uint_16(entrybuf + 4, ep->frequency);
+ }
+ else
+ {
+ png_save_uint_16(entrybuf + 0, ep->red);
+ png_save_uint_16(entrybuf + 2, ep->green);
+ png_save_uint_16(entrybuf + 4, ep->blue);
+ png_save_uint_16(entrybuf + 6, ep->alpha);
+ png_save_uint_16(entrybuf + 8, ep->frequency);
+ }
+ png_write_chunk_data(png_ptr, entrybuf, (png_size_t)entry_size);
+ }
+#else
+ ep=spalette->entries;
+ for (i=0; i>spalette->nentries; i++)
+ {
+ if (spalette->depth == 8)
+ {
+ entrybuf[0] = (png_byte)ep[i].red;
+ entrybuf[1] = (png_byte)ep[i].green;
+ entrybuf[2] = (png_byte)ep[i].blue;
+ entrybuf[3] = (png_byte)ep[i].alpha;
+ png_save_uint_16(entrybuf + 4, ep[i].frequency);
+ }
+ else
+ {
+ png_save_uint_16(entrybuf + 0, ep[i].red);
+ png_save_uint_16(entrybuf + 2, ep[i].green);
+ png_save_uint_16(entrybuf + 4, ep[i].blue);
+ png_save_uint_16(entrybuf + 6, ep[i].alpha);
+ png_save_uint_16(entrybuf + 8, ep[i].frequency);
+ }
+ png_write_chunk_data(png_ptr, entrybuf, (png_size_t)entry_size);
+ }
+#endif
+
+ png_write_chunk_end(png_ptr);
+ png_free(png_ptr, new_name);
+}
+#endif
+
+#ifdef PNG_WRITE_sBIT_SUPPORTED
+/* Write the sBIT chunk */
+void /* PRIVATE */
+png_write_sBIT(png_structp png_ptr, png_color_8p sbit, int color_type)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_sBIT;
+#endif
+ png_byte buf[4];
+ png_size_t size;
+
+ png_debug(1, "in png_write_sBIT");
+
+ /* Make sure we don't depend upon the order of PNG_COLOR_8 */
+ if (color_type & PNG_COLOR_MASK_COLOR)
+ {
+ png_byte maxbits;
+
+ maxbits = (png_byte)(color_type==PNG_COLOR_TYPE_PALETTE ? 8 :
+ png_ptr->usr_bit_depth);
+ if (sbit->red == 0 || sbit->red > maxbits ||
+ sbit->green == 0 || sbit->green > maxbits ||
+ sbit->blue == 0 || sbit->blue > maxbits)
+ {
+ png_warning(png_ptr, "Invalid sBIT depth specified");
+ return;
+ }
+ buf[0] = sbit->red;
+ buf[1] = sbit->green;
+ buf[2] = sbit->blue;
+ size = 3;
+ }
+ else
+ {
+ if (sbit->gray == 0 || sbit->gray > png_ptr->usr_bit_depth)
+ {
+ png_warning(png_ptr, "Invalid sBIT depth specified");
+ return;
+ }
+ buf[0] = sbit->gray;
+ size = 1;
+ }
+
+ if (color_type & PNG_COLOR_MASK_ALPHA)
+ {
+ if (sbit->alpha == 0 || sbit->alpha > png_ptr->usr_bit_depth)
+ {
+ png_warning(png_ptr, "Invalid sBIT depth specified");
+ return;
+ }
+ buf[size++] = sbit->alpha;
+ }
+
+ png_write_chunk(png_ptr, (png_bytep)png_sBIT, buf, size);
+}
+#endif
+
+#ifdef PNG_WRITE_cHRM_SUPPORTED
+/* Write the cHRM chunk */
+#ifdef PNG_FLOATING_POINT_SUPPORTED
+void /* PRIVATE */
+png_write_cHRM(png_structp png_ptr, double white_x, double white_y,
+ double red_x, double red_y, double green_x, double green_y,
+ double blue_x, double blue_y)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_cHRM;
+#endif
+ png_byte buf[32];
+
+ png_fixed_point int_white_x, int_white_y, int_red_x, int_red_y,
+ int_green_x, int_green_y, int_blue_x, int_blue_y;
+
+ png_debug(1, "in png_write_cHRM");
+
+ int_white_x = (png_uint_32)(white_x * 100000.0 + 0.5);
+ int_white_y = (png_uint_32)(white_y * 100000.0 + 0.5);
+ int_red_x = (png_uint_32)(red_x * 100000.0 + 0.5);
+ int_red_y = (png_uint_32)(red_y * 100000.0 + 0.5);
+ int_green_x = (png_uint_32)(green_x * 100000.0 + 0.5);
+ int_green_y = (png_uint_32)(green_y * 100000.0 + 0.5);
+ int_blue_x = (png_uint_32)(blue_x * 100000.0 + 0.5);
+ int_blue_y = (png_uint_32)(blue_y * 100000.0 + 0.5);
+
+#ifdef PNG_CHECK_cHRM_SUPPORTED
+ if (png_check_cHRM_fixed(png_ptr, int_white_x, int_white_y,
+ int_red_x, int_red_y, int_green_x, int_green_y, int_blue_x, int_blue_y))
+#endif
+ {
+ /* Each value is saved in 1/100,000ths */
+
+ png_save_uint_32(buf, int_white_x);
+ png_save_uint_32(buf + 4, int_white_y);
+
+ png_save_uint_32(buf + 8, int_red_x);
+ png_save_uint_32(buf + 12, int_red_y);
+
+ png_save_uint_32(buf + 16, int_green_x);
+ png_save_uint_32(buf + 20, int_green_y);
+
+ png_save_uint_32(buf + 24, int_blue_x);
+ png_save_uint_32(buf + 28, int_blue_y);
+
+ png_write_chunk(png_ptr, (png_bytep)png_cHRM, buf, (png_size_t)32);
+ }
+}
+#endif
+#ifdef PNG_FIXED_POINT_SUPPORTED
+void /* PRIVATE */
+png_write_cHRM_fixed(png_structp png_ptr, png_fixed_point white_x,
+ png_fixed_point white_y, png_fixed_point red_x, png_fixed_point red_y,
+ png_fixed_point green_x, png_fixed_point green_y, png_fixed_point blue_x,
+ png_fixed_point blue_y)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_cHRM;
+#endif
+ png_byte buf[32];
+
+ png_debug(1, "in png_write_cHRM");
+
+ /* Each value is saved in 1/100,000ths */
+#ifdef PNG_CHECK_cHRM_SUPPORTED
+ if (png_check_cHRM_fixed(png_ptr, white_x, white_y, red_x, red_y,
+ green_x, green_y, blue_x, blue_y))
+#endif
+ {
+ png_save_uint_32(buf, (png_uint_32)white_x);
+ png_save_uint_32(buf + 4, (png_uint_32)white_y);
+
+ png_save_uint_32(buf + 8, (png_uint_32)red_x);
+ png_save_uint_32(buf + 12, (png_uint_32)red_y);
+
+ png_save_uint_32(buf + 16, (png_uint_32)green_x);
+ png_save_uint_32(buf + 20, (png_uint_32)green_y);
+
+ png_save_uint_32(buf + 24, (png_uint_32)blue_x);
+ png_save_uint_32(buf + 28, (png_uint_32)blue_y);
+
+ png_write_chunk(png_ptr, (png_bytep)png_cHRM, buf, (png_size_t)32);
+ }
+}
+#endif
+#endif
+
+#ifdef PNG_WRITE_tRNS_SUPPORTED
+/* Write the tRNS chunk */
+void /* PRIVATE */
+png_write_tRNS(png_structp png_ptr, png_bytep trans, png_color_16p tran,
+ int num_trans, int color_type)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_tRNS;
+#endif
+ png_byte buf[6];
+
+ png_debug(1, "in png_write_tRNS");
+
+ if (color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ if (num_trans <= 0 || num_trans > (int)png_ptr->num_palette)
+ {
+ png_warning(png_ptr, "Invalid number of transparent colors specified");
+ return;
+ }
+ /* Write the chunk out as it is */
+ png_write_chunk(png_ptr, (png_bytep)png_tRNS, trans,
+ (png_size_t)num_trans);
+ }
+ else if (color_type == PNG_COLOR_TYPE_GRAY)
+ {
+ /* One 16 bit value */
+ if (tran->gray >= (1 << png_ptr->bit_depth))
+ {
+ png_warning(png_ptr,
+ "Ignoring attempt to write tRNS chunk out-of-range for bit_depth");
+ return;
+ }
+ png_save_uint_16(buf, tran->gray);
+ png_write_chunk(png_ptr, (png_bytep)png_tRNS, buf, (png_size_t)2);
+ }
+ else if (color_type == PNG_COLOR_TYPE_RGB)
+ {
+ /* Three 16 bit values */
+ png_save_uint_16(buf, tran->red);
+ png_save_uint_16(buf + 2, tran->green);
+ png_save_uint_16(buf + 4, tran->blue);
+ if (png_ptr->bit_depth == 8 && (buf[0] | buf[2] | buf[4]))
+ {
+ png_warning(png_ptr,
+ "Ignoring attempt to write 16-bit tRNS chunk when bit_depth is 8");
+ return;
+ }
+ png_write_chunk(png_ptr, (png_bytep)png_tRNS, buf, (png_size_t)6);
+ }
+ else
+ {
+ png_warning(png_ptr, "Can't write tRNS with an alpha channel");
+ }
+}
+#endif
+
+#ifdef PNG_WRITE_bKGD_SUPPORTED
+/* Write the background chunk */
+void /* PRIVATE */
+png_write_bKGD(png_structp png_ptr, png_color_16p back, int color_type)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_bKGD;
+#endif
+ png_byte buf[6];
+
+ png_debug(1, "in png_write_bKGD");
+
+ if (color_type == PNG_COLOR_TYPE_PALETTE)
+ {
+ if (
+#ifdef PNG_MNG_FEATURES_SUPPORTED
+ (png_ptr->num_palette ||
+ (!(png_ptr->mng_features_permitted & PNG_FLAG_MNG_EMPTY_PLTE))) &&
+#endif
+ back->index >= png_ptr->num_palette)
+ {
+ png_warning(png_ptr, "Invalid background palette index");
+ return;
+ }
+ buf[0] = back->index;
+ png_write_chunk(png_ptr, (png_bytep)png_bKGD, buf, (png_size_t)1);
+ }
+ else if (color_type & PNG_COLOR_MASK_COLOR)
+ {
+ png_save_uint_16(buf, back->red);
+ png_save_uint_16(buf + 2, back->green);
+ png_save_uint_16(buf + 4, back->blue);
+ if (png_ptr->bit_depth == 8 && (buf[0] | buf[2] | buf[4]))
+ {
+ png_warning(png_ptr,
+ "Ignoring attempt to write 16-bit bKGD chunk when bit_depth is 8");
+ return;
+ }
+ png_write_chunk(png_ptr, (png_bytep)png_bKGD, buf, (png_size_t)6);
+ }
+ else
+ {
+ if (back->gray >= (1 << png_ptr->bit_depth))
+ {
+ png_warning(png_ptr,
+ "Ignoring attempt to write bKGD chunk out-of-range for bit_depth");
+ return;
+ }
+ png_save_uint_16(buf, back->gray);
+ png_write_chunk(png_ptr, (png_bytep)png_bKGD, buf, (png_size_t)2);
+ }
+}
+#endif
+
+#ifdef PNG_WRITE_hIST_SUPPORTED
+/* Write the histogram */
+void /* PRIVATE */
+png_write_hIST(png_structp png_ptr, png_uint_16p hist, int num_hist)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_hIST;
+#endif
+ int i;
+ png_byte buf[3];
+
+ png_debug(1, "in png_write_hIST");
+
+ if (num_hist > (int)png_ptr->num_palette)
+ {
+ png_debug2(3, "num_hist = %d, num_palette = %d", num_hist,
+ png_ptr->num_palette);
+ png_warning(png_ptr, "Invalid number of histogram entries specified");
+ return;
+ }
+
+ png_write_chunk_start(png_ptr, (png_bytep)png_hIST,
+ (png_uint_32)(num_hist * 2));
+ for (i = 0; i < num_hist; i++)
+ {
+ png_save_uint_16(buf, hist[i]);
+ png_write_chunk_data(png_ptr, buf, (png_size_t)2);
+ }
+ png_write_chunk_end(png_ptr);
+}
+#endif
+
+#if defined(PNG_WRITE_TEXT_SUPPORTED) || defined(PNG_WRITE_pCAL_SUPPORTED) || \
+ defined(PNG_WRITE_iCCP_SUPPORTED) || defined(PNG_WRITE_sPLT_SUPPORTED)
+/* Check that the tEXt or zTXt keyword is valid per PNG 1.0 specification,
+ * and if invalid, correct the keyword rather than discarding the entire
+ * chunk. The PNG 1.0 specification requires keywords 1-79 characters in
+ * length, forbids leading or trailing whitespace, multiple internal spaces,
+ * and the non-break space (0x80) from ISO 8859-1. Returns keyword length.
+ *
+ * The new_key is allocated to hold the corrected keyword and must be freed
+ * by the calling routine. This avoids problems with trying to write to
+ * static keywords without having to have duplicate copies of the strings.
+ */
+png_size_t /* PRIVATE */
+png_check_keyword(png_structp png_ptr, png_charp key, png_charpp new_key)
+{
+ png_size_t key_len;
+ png_charp kp, dp;
+ int kflag;
+ int kwarn=0;
+
+ png_debug(1, "in png_check_keyword");
+
+ *new_key = NULL;
+
+ if (key == NULL || (key_len = png_strlen(key)) == 0)
+ {
+ png_warning(png_ptr, "zero length keyword");
+ return ((png_size_t)0);
+ }
+
+ png_debug1(2, "Keyword to be checked is '%s'", key);
+
+ *new_key = (png_charp)png_malloc_warn(png_ptr, (png_uint_32)(key_len + 2));
+ if (*new_key == NULL)
+ {
+ png_warning(png_ptr, "Out of memory while procesing keyword");
+ return ((png_size_t)0);
+ }
+
+ /* Replace non-printing characters with a blank and print a warning */
+ for (kp = key, dp = *new_key; *kp != '\0'; kp++, dp++)
+ {
+ if ((png_byte)*kp < 0x20 ||
+ ((png_byte)*kp > 0x7E && (png_byte)*kp < 0xA1))
+ {
+#if defined(PNG_STDIO_SUPPORTED) && !defined(_WIN32_WCE)
+ char msg[40];
+
+ png_snprintf(msg, 40,
+ "invalid keyword character 0x%02X", (png_byte)*kp);
+ png_warning(png_ptr, msg);
+#else
+ png_warning(png_ptr, "invalid character in keyword");
+#endif
+ *dp = ' ';
+ }
+ else
+ {
+ *dp = *kp;
+ }
+ }
+ *dp = '\0';
+
+ /* Remove any trailing white space. */
+ kp = *new_key + key_len - 1;
+ if (*kp == ' ')
+ {
+ png_warning(png_ptr, "trailing spaces removed from keyword");
+
+ while (*kp == ' ')
+ {
+ *(kp--) = '\0';
+ key_len--;
+ }
+ }
+
+ /* Remove any leading white space. */
+ kp = *new_key;
+ if (*kp == ' ')
+ {
+ png_warning(png_ptr, "leading spaces removed from keyword");
+
+ while (*kp == ' ')
+ {
+ kp++;
+ key_len--;
+ }
+ }
+
+ png_debug1(2, "Checking for multiple internal spaces in '%s'", kp);
+
+ /* Remove multiple internal spaces. */
+ for (kflag = 0, dp = *new_key; *kp != '\0'; kp++)
+ {
+ if (*kp == ' ' && kflag == 0)
+ {
+ *(dp++) = *kp;
+ kflag = 1;
+ }
+ else if (*kp == ' ')
+ {
+ key_len--;
+ kwarn=1;
+ }
+ else
+ {
+ *(dp++) = *kp;
+ kflag = 0;
+ }
+ }
+ *dp = '\0';
+ if (kwarn)
+ png_warning(png_ptr, "extra interior spaces removed from keyword");
+
+ if (key_len == 0)
+ {
+ png_free(png_ptr, *new_key);
+ *new_key=NULL;
+ png_warning(png_ptr, "Zero length keyword");
+ }
+
+ if (key_len > 79)
+ {
+ png_warning(png_ptr, "keyword length must be 1 - 79 characters");
+ (*new_key)[79] = '\0';
+ key_len = 79;
+ }
+
+ return (key_len);
+}
+#endif
+
+#ifdef PNG_WRITE_tEXt_SUPPORTED
+/* Write a tEXt chunk */
+void /* PRIVATE */
+png_write_tEXt(png_structp png_ptr, png_charp key, png_charp text,
+ png_size_t text_len)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_tEXt;
+#endif
+ png_size_t key_len;
+ png_charp new_key;
+
+ png_debug(1, "in png_write_tEXt");
+
+ if ((key_len = png_check_keyword(png_ptr, key, &new_key))==0)
+ return;
+
+ if (text == NULL || *text == '\0')
+ text_len = 0;
+ else
+ text_len = png_strlen(text);
+
+ /* Make sure we include the 0 after the key */
+ png_write_chunk_start(png_ptr, (png_bytep)png_tEXt,
+ (png_uint_32)(key_len + text_len + 1));
+ /*
+ * We leave it to the application to meet PNG-1.0 requirements on the
+ * contents of the text. PNG-1.0 through PNG-1.2 discourage the use of
+ * any non-Latin-1 characters except for NEWLINE. ISO PNG will forbid them.
+ * The NUL character is forbidden by PNG-1.0 through PNG-1.2 and ISO PNG.
+ */
+ png_write_chunk_data(png_ptr, (png_bytep)new_key,
+ (png_size_t)(key_len + 1));
+ if (text_len)
+ png_write_chunk_data(png_ptr, (png_bytep)text, (png_size_t)text_len);
+
+ png_write_chunk_end(png_ptr);
+ png_free(png_ptr, new_key);
+}
+#endif
+
+#ifdef PNG_WRITE_zTXt_SUPPORTED
+/* Write a compressed text chunk */
+void /* PRIVATE */
+png_write_zTXt(png_structp png_ptr, png_charp key, png_charp text,
+ png_size_t text_len, int compression)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_zTXt;
+#endif
+ png_size_t key_len;
+ char buf[1];
+ png_charp new_key;
+ compression_state comp;
+
+ png_debug(1, "in png_write_zTXt");
+
+ comp.num_output_ptr = 0;
+ comp.max_output_ptr = 0;
+ comp.output_ptr = NULL;
+ comp.input = NULL;
+ comp.input_len = 0;
+
+ if ((key_len = png_check_keyword(png_ptr, key, &new_key))==0)
+ {
+ png_free(png_ptr, new_key);
+ return;
+ }
+
+ if (text == NULL || *text == '\0' || compression==PNG_TEXT_COMPRESSION_NONE)
+ {
+ png_write_tEXt(png_ptr, new_key, text, (png_size_t)0);
+ png_free(png_ptr, new_key);
+ return;
+ }
+
+ text_len = png_strlen(text);
+
+ /* Compute the compressed data; do it now for the length */
+ text_len = png_text_compress(png_ptr, text, text_len, compression,
+ &comp);
+
+ /* Write start of chunk */
+ png_write_chunk_start(png_ptr, (png_bytep)png_zTXt,
+ (png_uint_32)(key_len+text_len + 2));
+ /* Write key */
+ png_write_chunk_data(png_ptr, (png_bytep)new_key,
+ (png_size_t)(key_len + 1));
+ png_free(png_ptr, new_key);
+
+ buf[0] = (png_byte)compression;
+ /* Write compression */
+ png_write_chunk_data(png_ptr, (png_bytep)buf, (png_size_t)1);
+ /* Write the compressed data */
+ png_write_compressed_data_out(png_ptr, &comp);
+
+ /* Close the chunk */
+ png_write_chunk_end(png_ptr);
+}
+#endif
+
+#ifdef PNG_WRITE_iTXt_SUPPORTED
+/* Write an iTXt chunk */
+void /* PRIVATE */
+png_write_iTXt(png_structp png_ptr, int compression, png_charp key,
+ png_charp lang, png_charp lang_key, png_charp text)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_iTXt;
+#endif
+ png_size_t lang_len, key_len, lang_key_len, text_len;
+ png_charp new_lang;
+ png_charp new_key = NULL;
+ png_byte cbuf[2];
+ compression_state comp;
+
+ png_debug(1, "in png_write_iTXt");
+
+ comp.num_output_ptr = 0;
+ comp.max_output_ptr = 0;
+ comp.output_ptr = NULL;
+ comp.input = NULL;
+
+ if ((key_len = png_check_keyword(png_ptr, key, &new_key))==0)
+ return;
+
+ if ((lang_len = png_check_keyword(png_ptr, lang, &new_lang))==0)
+ {
+ png_warning(png_ptr, "Empty language field in iTXt chunk");
+ new_lang = NULL;
+ lang_len = 0;
+ }
+
+ if (lang_key == NULL)
+ lang_key_len = 0;
+ else
+ lang_key_len = png_strlen(lang_key);
+
+ if (text == NULL)
+ text_len = 0;
+ else
+ text_len = png_strlen(text);
+
+ /* Compute the compressed data; do it now for the length */
+ text_len = png_text_compress(png_ptr, text, text_len, compression-2,
+ &comp);
+
+
+ /* Make sure we include the compression flag, the compression byte,
+ * and the NULs after the key, lang, and lang_key parts */
+
+ png_write_chunk_start(png_ptr, (png_bytep)png_iTXt,
+ (png_uint_32)(
+ 5 /* comp byte, comp flag, terminators for key, lang and lang_key */
+ + key_len
+ + lang_len
+ + lang_key_len
+ + text_len));
+
+ /* We leave it to the application to meet PNG-1.0 requirements on the
+ * contents of the text. PNG-1.0 through PNG-1.2 discourage the use of
+ * any non-Latin-1 characters except for NEWLINE. ISO PNG will forbid them.
+ * The NUL character is forbidden by PNG-1.0 through PNG-1.2 and ISO PNG.
+ */
+ png_write_chunk_data(png_ptr, (png_bytep)new_key,
+ (png_size_t)(key_len + 1));
+
+ /* Set the compression flag */
+ if (compression == PNG_ITXT_COMPRESSION_NONE || \
+ compression == PNG_TEXT_COMPRESSION_NONE)
+ cbuf[0] = 0;
+ else /* compression == PNG_ITXT_COMPRESSION_zTXt */
+ cbuf[0] = 1;
+ /* Set the compression method */
+ cbuf[1] = 0;
+ png_write_chunk_data(png_ptr, cbuf, (png_size_t)2);
+
+ cbuf[0] = 0;
+ png_write_chunk_data(png_ptr, (new_lang ? (png_bytep)new_lang : cbuf),
+ (png_size_t)(lang_len + 1));
+ png_write_chunk_data(png_ptr, (lang_key ? (png_bytep)lang_key : cbuf),
+ (png_size_t)(lang_key_len + 1));
+ png_write_compressed_data_out(png_ptr, &comp);
+
+ png_write_chunk_end(png_ptr);
+ png_free(png_ptr, new_key);
+ png_free(png_ptr, new_lang);
+}
+#endif
+
+#ifdef PNG_WRITE_oFFs_SUPPORTED
+/* Write the oFFs chunk */
+void /* PRIVATE */
+png_write_oFFs(png_structp png_ptr, png_int_32 x_offset, png_int_32 y_offset,
+ int unit_type)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_oFFs;
+#endif
+ png_byte buf[9];
+
+ png_debug(1, "in png_write_oFFs");
+
+ if (unit_type >= PNG_OFFSET_LAST)
+ png_warning(png_ptr, "Unrecognized unit type for oFFs chunk");
+
+ png_save_int_32(buf, x_offset);
+ png_save_int_32(buf + 4, y_offset);
+ buf[8] = (png_byte)unit_type;
+
+ png_write_chunk(png_ptr, (png_bytep)png_oFFs, buf, (png_size_t)9);
+}
+#endif
+#ifdef PNG_WRITE_pCAL_SUPPORTED
+/* Write the pCAL chunk (described in the PNG extensions document) */
+void /* PRIVATE */
+png_write_pCAL(png_structp png_ptr, png_charp purpose, png_int_32 X0,
+ png_int_32 X1, int type, int nparams, png_charp units, png_charpp params)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_pCAL;
+#endif
+ png_size_t purpose_len, units_len, total_len;
+ png_uint_32p params_len;
+ png_byte buf[10];
+ png_charp new_purpose;
+ int i;
+
+ png_debug1(1, "in png_write_pCAL (%d parameters)", nparams);
+
+ if (type >= PNG_EQUATION_LAST)
+ png_warning(png_ptr, "Unrecognized equation type for pCAL chunk");
+
+ purpose_len = png_check_keyword(png_ptr, purpose, &new_purpose) + 1;
+ png_debug1(3, "pCAL purpose length = %d", (int)purpose_len);
+ units_len = png_strlen(units) + (nparams == 0 ? 0 : 1);
+ png_debug1(3, "pCAL units length = %d", (int)units_len);
+ total_len = purpose_len + units_len + 10;
+
+ params_len = (png_uint_32p)png_malloc(png_ptr,
+ (png_uint_32)(nparams * png_sizeof(png_uint_32)));
+
+ /* Find the length of each parameter, making sure we don't count the
+ null terminator for the last parameter. */
+ for (i = 0; i < nparams; i++)
+ {
+ params_len[i] = png_strlen(params[i]) + (i == nparams - 1 ? 0 : 1);
+ png_debug2(3, "pCAL parameter %d length = %lu", i,
+ (unsigned long) params_len[i]);
+ total_len += (png_size_t)params_len[i];
+ }
+
+ png_debug1(3, "pCAL total length = %d", (int)total_len);
+ png_write_chunk_start(png_ptr, (png_bytep)png_pCAL, (png_uint_32)total_len);
+ png_write_chunk_data(png_ptr, (png_bytep)new_purpose,
+ (png_size_t)purpose_len);
+ png_save_int_32(buf, X0);
+ png_save_int_32(buf + 4, X1);
+ buf[8] = (png_byte)type;
+ buf[9] = (png_byte)nparams;
+ png_write_chunk_data(png_ptr, buf, (png_size_t)10);
+ png_write_chunk_data(png_ptr, (png_bytep)units, (png_size_t)units_len);
+
+ png_free(png_ptr, new_purpose);
+
+ for (i = 0; i < nparams; i++)
+ {
+ png_write_chunk_data(png_ptr, (png_bytep)params[i],
+ (png_size_t)params_len[i]);
+ }
+
+ png_free(png_ptr, params_len);
+ png_write_chunk_end(png_ptr);
+}
+#endif
+
+#ifdef PNG_WRITE_sCAL_SUPPORTED
+/* Write the sCAL chunk */
+#if defined(PNG_FLOATING_POINT_SUPPORTED) && defined(PNG_STDIO_SUPPORTED)
+void /* PRIVATE */
+png_write_sCAL(png_structp png_ptr, int unit, double width, double height)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_sCAL;
+#endif
+ char buf[64];
+ png_size_t total_len;
+
+ png_debug(1, "in png_write_sCAL");
+
+ buf[0] = (char)unit;
+#ifdef _WIN32_WCE
+/* sprintf() function is not supported on WindowsCE */
+ {
+ wchar_t wc_buf[32];
+ size_t wc_len;
+ swprintf(wc_buf, TEXT("%12.12e"), width);
+ wc_len = wcslen(wc_buf);
+ WideCharToMultiByte(CP_ACP, 0, wc_buf, -1, buf + 1, wc_len, NULL,
+ NULL);
+ total_len = wc_len + 2;
+ swprintf(wc_buf, TEXT("%12.12e"), height);
+ wc_len = wcslen(wc_buf);
+ WideCharToMultiByte(CP_ACP, 0, wc_buf, -1, buf + total_len, wc_len,
+ NULL, NULL);
+ total_len += wc_len;
+ }
+#else
+ png_snprintf(buf + 1, 63, "%12.12e", width);
+ total_len = 1 + png_strlen(buf + 1) + 1;
+ png_snprintf(buf + total_len, 64-total_len, "%12.12e", height);
+ total_len += png_strlen(buf + total_len);
+#endif
+
+ png_debug1(3, "sCAL total length = %u", (unsigned int)total_len);
+ png_write_chunk(png_ptr, (png_bytep)png_sCAL, (png_bytep)buf, total_len);
+}
+#else
+#ifdef PNG_FIXED_POINT_SUPPORTED
+void /* PRIVATE */
+png_write_sCAL_s(png_structp png_ptr, int unit, png_charp width,
+ png_charp height)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_sCAL;
+#endif
+ png_byte buf[64];
+ png_size_t wlen, hlen, total_len;
+
+ png_debug(1, "in png_write_sCAL_s");
+
+ wlen = png_strlen(width);
+ hlen = png_strlen(height);
+ total_len = wlen + hlen + 2;
+ if (total_len > 64)
+ {
+ png_warning(png_ptr, "Can't write sCAL (buffer too small)");
+ return;
+ }
+
+ buf[0] = (png_byte)unit;
+ png_memcpy(buf + 1, width, wlen + 1); /* Append the '\0' here */
+ png_memcpy(buf + wlen + 2, height, hlen); /* Do NOT append the '\0' here */
+
+ png_debug1(3, "sCAL total length = %u", (unsigned int)total_len);
+ png_write_chunk(png_ptr, (png_bytep)png_sCAL, buf, total_len);
+}
+#endif
+#endif
+#endif
+
+#ifdef PNG_WRITE_pHYs_SUPPORTED
+/* Write the pHYs chunk */
+void /* PRIVATE */
+png_write_pHYs(png_structp png_ptr, png_uint_32 x_pixels_per_unit,
+ png_uint_32 y_pixels_per_unit,
+ int unit_type)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_pHYs;
+#endif
+ png_byte buf[9];
+
+ png_debug(1, "in png_write_pHYs");
+
+ if (unit_type >= PNG_RESOLUTION_LAST)
+ png_warning(png_ptr, "Unrecognized unit type for pHYs chunk");
+
+ png_save_uint_32(buf, x_pixels_per_unit);
+ png_save_uint_32(buf + 4, y_pixels_per_unit);
+ buf[8] = (png_byte)unit_type;
+
+ png_write_chunk(png_ptr, (png_bytep)png_pHYs, buf, (png_size_t)9);
+}
+#endif
+
+#ifdef PNG_WRITE_tIME_SUPPORTED
+/* Write the tIME chunk. Use either png_convert_from_struct_tm()
+ * or png_convert_from_time_t(), or fill in the structure yourself.
+ */
+void /* PRIVATE */
+png_write_tIME(png_structp png_ptr, png_timep mod_time)
+{
+#ifdef PNG_USE_LOCAL_ARRAYS
+ PNG_tIME;
+#endif
+ png_byte buf[7];
+
+ png_debug(1, "in png_write_tIME");
+
+ if (mod_time->month > 12 || mod_time->month < 1 ||
+ mod_time->day > 31 || mod_time->day < 1 ||
+ mod_time->hour > 23 || mod_time->second > 60)
+ {
+ png_warning(png_ptr, "Invalid time specified for tIME chunk");
+ return;
+ }
+
+ png_save_uint_16(buf, mod_time->year);
+ buf[2] = mod_time->month;
+ buf[3] = mod_time->day;
+ buf[4] = mod_time->hour;
+ buf[5] = mod_time->minute;
+ buf[6] = mod_time->second;
+
+ png_write_chunk(png_ptr, (png_bytep)png_tIME, buf, (png_size_t)7);
+}
+#endif
+
+/* Initializes the row writing capability of libpng */
+void /* PRIVATE */
+png_write_start_row(png_structp png_ptr)
+{
+#ifdef PNG_WRITE_INTERLACING_SUPPORTED
+ /* Arrays to facilitate easy interlacing - use pass (0 - 6) as index */
+
+ /* Start of interlace block */
+ int png_pass_start[7] = {0, 4, 0, 2, 0, 1, 0};
+
+ /* Offset to next interlace block */
+ int png_pass_inc[7] = {8, 8, 4, 4, 2, 2, 1};
+
+ /* Start of interlace block in the y direction */
+ int png_pass_ystart[7] = {0, 0, 4, 0, 2, 0, 1};
+
+ /* Offset to next interlace block in the y direction */
+ int png_pass_yinc[7] = {8, 8, 8, 4, 4, 2, 2};
+#endif
+
+ png_size_t buf_size;
+
+ png_debug(1, "in png_write_start_row");
+
+ buf_size = (png_size_t)(PNG_ROWBYTES(
+ png_ptr->usr_channels*png_ptr->usr_bit_depth, png_ptr->width) + 1);
+
+ /* Set up row buffer */
+ png_ptr->row_buf = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)buf_size);
+ png_ptr->row_buf[0] = PNG_FILTER_VALUE_NONE;
+
+#ifdef PNG_WRITE_FILTER_SUPPORTED
+ /* Set up filtering buffer, if using this filter */
+ if (png_ptr->do_filter & PNG_FILTER_SUB)
+ {
+ png_ptr->sub_row = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)(png_ptr->rowbytes + 1));
+ png_ptr->sub_row[0] = PNG_FILTER_VALUE_SUB;
+ }
+
+ /* We only need to keep the previous row if we are using one of these. */
+ if (png_ptr->do_filter & (PNG_FILTER_AVG | PNG_FILTER_UP | PNG_FILTER_PAETH))
+ {
+ /* Set up previous row buffer */
+ png_ptr->prev_row = (png_bytep)png_calloc(png_ptr,
+ (png_uint_32)buf_size);
+
+ if (png_ptr->do_filter & PNG_FILTER_UP)
+ {
+ png_ptr->up_row = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)(png_ptr->rowbytes + 1));
+ png_ptr->up_row[0] = PNG_FILTER_VALUE_UP;
+ }
+
+ if (png_ptr->do_filter & PNG_FILTER_AVG)
+ {
+ png_ptr->avg_row = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)(png_ptr->rowbytes + 1));
+ png_ptr->avg_row[0] = PNG_FILTER_VALUE_AVG;
+ }
+
+ if (png_ptr->do_filter & PNG_FILTER_PAETH)
+ {
+ png_ptr->paeth_row = (png_bytep)png_malloc(png_ptr,
+ (png_uint_32)(png_ptr->rowbytes + 1));
+ png_ptr->paeth_row[0] = PNG_FILTER_VALUE_PAETH;
+ }
+ }
+#endif /* PNG_WRITE_FILTER_SUPPORTED */
+
+#ifdef PNG_WRITE_INTERLACING_SUPPORTED
+ /* If interlaced, we need to set up width and height of pass */
+ if (png_ptr->interlaced)
+ {
+ if (!(png_ptr->transformations & PNG_INTERLACE))
+ {
+ png_ptr->num_rows = (png_ptr->height + png_pass_yinc[0] - 1 -
+ png_pass_ystart[0]) / png_pass_yinc[0];
+ png_ptr->usr_width = (png_ptr->width + png_pass_inc[0] - 1 -
+ png_pass_start[0]) / png_pass_inc[0];
+ }
+ else
+ {
+ png_ptr->num_rows = png_ptr->height;
+ png_ptr->usr_width = png_ptr->width;
+ }
+ }
+ else
+#endif
+ {
+ png_ptr->num_rows = png_ptr->height;
+ png_ptr->usr_width = png_ptr->width;
+ }
+ png_ptr->zstream.avail_out = (uInt)png_ptr->zbuf_size;
+ png_ptr->zstream.next_out = png_ptr->zbuf;
+}
+
+/* Internal use only. Called when finished processing a row of data. */
+void /* PRIVATE */
+png_write_finish_row(png_structp png_ptr)
+{
+#ifdef PNG_WRITE_INTERLACING_SUPPORTED
+ /* Arrays to facilitate easy interlacing - use pass (0 - 6) as index */
+
+ /* Start of interlace block */
+ int png_pass_start[7] = {0, 4, 0, 2, 0, 1, 0};
+
+ /* Offset to next interlace block */
+ int png_pass_inc[7] = {8, 8, 4, 4, 2, 2, 1};
+
+ /* Start of interlace block in the y direction */
+ int png_pass_ystart[7] = {0, 0, 4, 0, 2, 0, 1};
+
+ /* Offset to next interlace block in the y direction */
+ int png_pass_yinc[7] = {8, 8, 8, 4, 4, 2, 2};
+#endif
+
+ int ret;
+
+ png_debug(1, "in png_write_finish_row");
+
+ /* Next row */
+ png_ptr->row_number++;
+
+ /* See if we are done */
+ if (png_ptr->row_number < png_ptr->num_rows)
+ return;
+
+#ifdef PNG_WRITE_INTERLACING_SUPPORTED
+ /* If interlaced, go to next pass */
+ if (png_ptr->interlaced)
+ {
+ png_ptr->row_number = 0;
+ if (png_ptr->transformations & PNG_INTERLACE)
+ {
+ png_ptr->pass++;
+ }
+ else
+ {
+ /* Loop until we find a non-zero width or height pass */
+ do
+ {
+ png_ptr->pass++;
+ if (png_ptr->pass >= 7)
+ break;
+ png_ptr->usr_width = (png_ptr->width +
+ png_pass_inc[png_ptr->pass] - 1 -
+ png_pass_start[png_ptr->pass]) /
+ png_pass_inc[png_ptr->pass];
+ png_ptr->num_rows = (png_ptr->height +
+ png_pass_yinc[png_ptr->pass] - 1 -
+ png_pass_ystart[png_ptr->pass]) /
+ png_pass_yinc[png_ptr->pass];
+ if (png_ptr->transformations & PNG_INTERLACE)
+ break;
+ } while (png_ptr->usr_width == 0 || png_ptr->num_rows == 0);
+
+ }
+
+ /* Reset the row above the image for the next pass */
+ if (png_ptr->pass < 7)
+ {
+ if (png_ptr->prev_row != NULL)
+ png_memset(png_ptr->prev_row, 0,
+ (png_size_t)(PNG_ROWBYTES(png_ptr->usr_channels*
+ png_ptr->usr_bit_depth, png_ptr->width)) + 1);
+ return;
+ }
+ }
+#endif
+
+ /* If we get here, we've just written the last row, so we need
+ to flush the compressor */
+ do
+ {
+ /* Tell the compressor we are done */
+ ret = deflate(&png_ptr->zstream, Z_FINISH);
+ /* Check for an error */
+ if (ret == Z_OK)
+ {
+ /* Check to see if we need more room */
+ if (!(png_ptr->zstream.avail_out))
+ {
+ png_write_IDAT(png_ptr, png_ptr->zbuf, png_ptr->zbuf_size);
+ png_ptr->zstream.next_out = png_ptr->zbuf;
+ png_ptr->zstream.avail_out = (uInt)png_ptr->zbuf_size;
+ }
+ }
+ else if (ret != Z_STREAM_END)
+ {
+ if (png_ptr->zstream.msg != NULL)
+ png_error(png_ptr, png_ptr->zstream.msg);
+ else
+ png_error(png_ptr, "zlib error");
+ }
+ } while (ret != Z_STREAM_END);
+
+ /* Write any extra space */
+ if (png_ptr->zstream.avail_out < png_ptr->zbuf_size)
+ {
+ png_write_IDAT(png_ptr, png_ptr->zbuf, png_ptr->zbuf_size -
+ png_ptr->zstream.avail_out);
+ }
+
+ deflateReset(&png_ptr->zstream);
+ png_ptr->zstream.data_type = Z_BINARY;
+}
+
+#ifdef PNG_WRITE_INTERLACING_SUPPORTED
+/* Pick out the correct pixels for the interlace pass.
+ * The basic idea here is to go through the row with a source
+ * pointer and a destination pointer (sp and dp), and copy the
+ * correct pixels for the pass. As the row gets compacted,
+ * sp will always be >= dp, so we should never overwrite anything.
+ * See the default: case for the easiest code to understand.
+ */
+void /* PRIVATE */
+png_do_write_interlace(png_row_infop row_info, png_bytep row, int pass)
+{
+ /* Arrays to facilitate easy interlacing - use pass (0 - 6) as index */
+
+ /* Start of interlace block */
+ int png_pass_start[7] = {0, 4, 0, 2, 0, 1, 0};
+
+ /* Offset to next interlace block */
+ int png_pass_inc[7] = {8, 8, 4, 4, 2, 2, 1};
+
+ png_debug(1, "in png_do_write_interlace");
+
+ /* We don't have to do anything on the last pass (6) */
+#ifdef PNG_USELESS_TESTS_SUPPORTED
+ if (row != NULL && row_info != NULL && pass < 6)
+#else
+ if (pass < 6)
+#endif
+ {
+ /* Each pixel depth is handled separately */
+ switch (row_info->pixel_depth)
+ {
+ case 1:
+ {
+ png_bytep sp;
+ png_bytep dp;
+ int shift;
+ int d;
+ int value;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+
+ dp = row;
+ d = 0;
+ shift = 7;
+ for (i = png_pass_start[pass]; i < row_width;
+ i += png_pass_inc[pass])
+ {
+ sp = row + (png_size_t)(i >> 3);
+ value = (int)(*sp >> (7 - (int)(i & 0x07))) & 0x01;
+ d |= (value << shift);
+
+ if (shift == 0)
+ {
+ shift = 7;
+ *dp++ = (png_byte)d;
+ d = 0;
+ }
+ else
+ shift--;
+
+ }
+ if (shift != 7)
+ *dp = (png_byte)d;
+ break;
+ }
+ case 2:
+ {
+ png_bytep sp;
+ png_bytep dp;
+ int shift;
+ int d;
+ int value;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+
+ dp = row;
+ shift = 6;
+ d = 0;
+ for (i = png_pass_start[pass]; i < row_width;
+ i += png_pass_inc[pass])
+ {
+ sp = row + (png_size_t)(i >> 2);
+ value = (*sp >> ((3 - (int)(i & 0x03)) << 1)) & 0x03;
+ d |= (value << shift);
+
+ if (shift == 0)
+ {
+ shift = 6;
+ *dp++ = (png_byte)d;
+ d = 0;
+ }
+ else
+ shift -= 2;
+ }
+ if (shift != 6)
+ *dp = (png_byte)d;
+ break;
+ }
+ case 4:
+ {
+ png_bytep sp;
+ png_bytep dp;
+ int shift;
+ int d;
+ int value;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+
+ dp = row;
+ shift = 4;
+ d = 0;
+ for (i = png_pass_start[pass]; i < row_width;
+ i += png_pass_inc[pass])
+ {
+ sp = row + (png_size_t)(i >> 1);
+ value = (*sp >> ((1 - (int)(i & 0x01)) << 2)) & 0x0f;
+ d |= (value << shift);
+
+ if (shift == 0)
+ {
+ shift = 4;
+ *dp++ = (png_byte)d;
+ d = 0;
+ }
+ else
+ shift -= 4;
+ }
+ if (shift != 4)
+ *dp = (png_byte)d;
+ break;
+ }
+ default:
+ {
+ png_bytep sp;
+ png_bytep dp;
+ png_uint_32 i;
+ png_uint_32 row_width = row_info->width;
+ png_size_t pixel_bytes;
+
+ /* Start at the beginning */
+ dp = row;
+ /* Find out how many bytes each pixel takes up */
+ pixel_bytes = (row_info->pixel_depth >> 3);
+ /* Loop through the row, only looking at the pixels that
+ matter */
+ for (i = png_pass_start[pass]; i < row_width;
+ i += png_pass_inc[pass])
+ {
+ /* Find out where the original pixel is */
+ sp = row + (png_size_t)i * pixel_bytes;
+ /* Move the pixel */
+ if (dp != sp)
+ png_memcpy(dp, sp, pixel_bytes);
+ /* Next pixel */
+ dp += pixel_bytes;
+ }
+ break;
+ }
+ }
+ /* Set new row width */
+ row_info->width = (row_info->width +
+ png_pass_inc[pass] - 1 -
+ png_pass_start[pass]) /
+ png_pass_inc[pass];
+ row_info->rowbytes = PNG_ROWBYTES(row_info->pixel_depth,
+ row_info->width);
+ }
+}
+#endif
+
+/* This filters the row, chooses which filter to use, if it has not already
+ * been specified by the application, and then writes the row out with the
+ * chosen filter.
+ */
+#define PNG_MAXSUM (((png_uint_32)(-1)) >> 1)
+#define PNG_HISHIFT 10
+#define PNG_LOMASK ((png_uint_32)0xffffL)
+#define PNG_HIMASK ((png_uint_32)(~PNG_LOMASK >> PNG_HISHIFT))
+void /* PRIVATE */
+png_write_find_filter(png_structp png_ptr, png_row_infop row_info)
+{
+ png_bytep best_row;
+#ifdef PNG_WRITE_FILTER_SUPPORTED
+ png_bytep prev_row, row_buf;
+ png_uint_32 mins, bpp;
+ png_byte filter_to_do = png_ptr->do_filter;
+ png_uint_32 row_bytes = row_info->rowbytes;
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ int num_p_filters = (int)png_ptr->num_prev_filters;
+#endif
+
+ png_debug(1, "in png_write_find_filter");
+
+#ifndef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ if (png_ptr->row_number == 0 && filter_to_do == PNG_ALL_FILTERS)
+ {
+ /* These will never be selected so we need not test them. */
+ filter_to_do &= ~(PNG_FILTER_UP | PNG_FILTER_PAETH);
+ }
+#endif
+
+ /* Find out how many bytes offset each pixel is */
+ bpp = (row_info->pixel_depth + 7) >> 3;
+
+ prev_row = png_ptr->prev_row;
+#endif
+ best_row = png_ptr->row_buf;
+#ifdef PNG_WRITE_FILTER_SUPPORTED
+ row_buf = best_row;
+ mins = PNG_MAXSUM;
+
+ /* The prediction method we use is to find which method provides the
+ * smallest value when summing the absolute values of the distances
+ * from zero, using anything >= 128 as negative numbers. This is known
+ * as the "minimum sum of absolute differences" heuristic. Other
+ * heuristics are the "weighted minimum sum of absolute differences"
+ * (experimental and can in theory improve compression), and the "zlib
+ * predictive" method (not implemented yet), which does test compressions
+ * of lines using different filter methods, and then chooses the
+ * (series of) filter(s) that give minimum compressed data size (VERY
+ * computationally expensive).
+ *
+ * GRR 980525: consider also
+ * (1) minimum sum of absolute differences from running average (i.e.,
+ * keep running sum of non-absolute differences & count of bytes)
+ * [track dispersion, too? restart average if dispersion too large?]
+ * (1b) minimum sum of absolute differences from sliding average, probably
+ * with window size <= deflate window (usually 32K)
+ * (2) minimum sum of squared differences from zero or running average
+ * (i.e., ~ root-mean-square approach)
+ */
+
+
+ /* We don't need to test the 'no filter' case if this is the only filter
+ * that has been chosen, as it doesn't actually do anything to the data.
+ */
+ if ((filter_to_do & PNG_FILTER_NONE) &&
+ filter_to_do != PNG_FILTER_NONE)
+ {
+ png_bytep rp;
+ png_uint_32 sum = 0;
+ png_uint_32 i;
+ int v;
+
+ for (i = 0, rp = row_buf + 1; i < row_bytes; i++, rp++)
+ {
+ v = *rp;
+ sum += (v < 128) ? v : 256 - v;
+ }
+
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ if (png_ptr->heuristic_method == PNG_FILTER_HEURISTIC_WEIGHTED)
+ {
+ png_uint_32 sumhi, sumlo;
+ int j;
+ sumlo = sum & PNG_LOMASK;
+ sumhi = (sum >> PNG_HISHIFT) & PNG_HIMASK; /* Gives us some footroom */
+
+ /* Reduce the sum if we match any of the previous rows */
+ for (j = 0; j < num_p_filters; j++)
+ {
+ if (png_ptr->prev_filters[j] == PNG_FILTER_VALUE_NONE)
+ {
+ sumlo = (sumlo * png_ptr->filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ sumhi = (sumhi * png_ptr->filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ }
+ }
+
+ /* Factor in the cost of this filter (this is here for completeness,
+ * but it makes no sense to have a "cost" for the NONE filter, as
+ * it has the minimum possible computational cost - none).
+ */
+ sumlo = (sumlo * png_ptr->filter_costs[PNG_FILTER_VALUE_NONE]) >>
+ PNG_COST_SHIFT;
+ sumhi = (sumhi * png_ptr->filter_costs[PNG_FILTER_VALUE_NONE]) >>
+ PNG_COST_SHIFT;
+
+ if (sumhi > PNG_HIMASK)
+ sum = PNG_MAXSUM;
+ else
+ sum = (sumhi << PNG_HISHIFT) + sumlo;
+ }
+#endif
+ mins = sum;
+ }
+
+ /* Sub filter */
+ if (filter_to_do == PNG_FILTER_SUB)
+ /* It's the only filter so no testing is needed */
+ {
+ png_bytep rp, lp, dp;
+ png_uint_32 i;
+ for (i = 0, rp = row_buf + 1, dp = png_ptr->sub_row + 1; i < bpp;
+ i++, rp++, dp++)
+ {
+ *dp = *rp;
+ }
+ for (lp = row_buf + 1; i < row_bytes;
+ i++, rp++, lp++, dp++)
+ {
+ *dp = (png_byte)(((int)*rp - (int)*lp) & 0xff);
+ }
+ best_row = png_ptr->sub_row;
+ }
+
+ else if (filter_to_do & PNG_FILTER_SUB)
+ {
+ png_bytep rp, dp, lp;
+ png_uint_32 sum = 0, lmins = mins;
+ png_uint_32 i;
+ int v;
+
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ /* We temporarily increase the "minimum sum" by the factor we
+ * would reduce the sum of this filter, so that we can do the
+ * early exit comparison without scaling the sum each time.
+ */
+ if (png_ptr->heuristic_method == PNG_FILTER_HEURISTIC_WEIGHTED)
+ {
+ int j;
+ png_uint_32 lmhi, lmlo;
+ lmlo = lmins & PNG_LOMASK;
+ lmhi = (lmins >> PNG_HISHIFT) & PNG_HIMASK;
+
+ for (j = 0; j < num_p_filters; j++)
+ {
+ if (png_ptr->prev_filters[j] == PNG_FILTER_VALUE_SUB)
+ {
+ lmlo = (lmlo * png_ptr->inv_filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ lmhi = (lmhi * png_ptr->inv_filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ }
+ }
+
+ lmlo = (lmlo * png_ptr->inv_filter_costs[PNG_FILTER_VALUE_SUB]) >>
+ PNG_COST_SHIFT;
+ lmhi = (lmhi * png_ptr->inv_filter_costs[PNG_FILTER_VALUE_SUB]) >>
+ PNG_COST_SHIFT;
+
+ if (lmhi > PNG_HIMASK)
+ lmins = PNG_MAXSUM;
+ else
+ lmins = (lmhi << PNG_HISHIFT) + lmlo;
+ }
+#endif
+
+ for (i = 0, rp = row_buf + 1, dp = png_ptr->sub_row + 1; i < bpp;
+ i++, rp++, dp++)
+ {
+ v = *dp = *rp;
+
+ sum += (v < 128) ? v : 256 - v;
+ }
+ for (lp = row_buf + 1; i < row_bytes;
+ i++, rp++, lp++, dp++)
+ {
+ v = *dp = (png_byte)(((int)*rp - (int)*lp) & 0xff);
+
+ sum += (v < 128) ? v : 256 - v;
+
+ if (sum > lmins) /* We are already worse, don't continue. */
+ break;
+ }
+
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ if (png_ptr->heuristic_method == PNG_FILTER_HEURISTIC_WEIGHTED)
+ {
+ int j;
+ png_uint_32 sumhi, sumlo;
+ sumlo = sum & PNG_LOMASK;
+ sumhi = (sum >> PNG_HISHIFT) & PNG_HIMASK;
+
+ for (j = 0; j < num_p_filters; j++)
+ {
+ if (png_ptr->prev_filters[j] == PNG_FILTER_VALUE_SUB)
+ {
+ sumlo = (sumlo * png_ptr->inv_filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ sumhi = (sumhi * png_ptr->inv_filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ }
+ }
+
+ sumlo = (sumlo * png_ptr->inv_filter_costs[PNG_FILTER_VALUE_SUB]) >>
+ PNG_COST_SHIFT;
+ sumhi = (sumhi * png_ptr->inv_filter_costs[PNG_FILTER_VALUE_SUB]) >>
+ PNG_COST_SHIFT;
+
+ if (sumhi > PNG_HIMASK)
+ sum = PNG_MAXSUM;
+ else
+ sum = (sumhi << PNG_HISHIFT) + sumlo;
+ }
+#endif
+
+ if (sum < mins)
+ {
+ mins = sum;
+ best_row = png_ptr->sub_row;
+ }
+ }
+
+ /* Up filter */
+ if (filter_to_do == PNG_FILTER_UP)
+ {
+ png_bytep rp, dp, pp;
+ png_uint_32 i;
+
+ for (i = 0, rp = row_buf + 1, dp = png_ptr->up_row + 1,
+ pp = prev_row + 1; i < row_bytes;
+ i++, rp++, pp++, dp++)
+ {
+ *dp = (png_byte)(((int)*rp - (int)*pp) & 0xff);
+ }
+ best_row = png_ptr->up_row;
+ }
+
+ else if (filter_to_do & PNG_FILTER_UP)
+ {
+ png_bytep rp, dp, pp;
+ png_uint_32 sum = 0, lmins = mins;
+ png_uint_32 i;
+ int v;
+
+
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ if (png_ptr->heuristic_method == PNG_FILTER_HEURISTIC_WEIGHTED)
+ {
+ int j;
+ png_uint_32 lmhi, lmlo;
+ lmlo = lmins & PNG_LOMASK;
+ lmhi = (lmins >> PNG_HISHIFT) & PNG_HIMASK;
+
+ for (j = 0; j < num_p_filters; j++)
+ {
+ if (png_ptr->prev_filters[j] == PNG_FILTER_VALUE_UP)
+ {
+ lmlo = (lmlo * png_ptr->inv_filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ lmhi = (lmhi * png_ptr->inv_filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ }
+ }
+
+ lmlo = (lmlo * png_ptr->inv_filter_costs[PNG_FILTER_VALUE_UP]) >>
+ PNG_COST_SHIFT;
+ lmhi = (lmhi * png_ptr->inv_filter_costs[PNG_FILTER_VALUE_UP]) >>
+ PNG_COST_SHIFT;
+
+ if (lmhi > PNG_HIMASK)
+ lmins = PNG_MAXSUM;
+ else
+ lmins = (lmhi << PNG_HISHIFT) + lmlo;
+ }
+#endif
+
+ for (i = 0, rp = row_buf + 1, dp = png_ptr->up_row + 1,
+ pp = prev_row + 1; i < row_bytes; i++)
+ {
+ v = *dp++ = (png_byte)(((int)*rp++ - (int)*pp++) & 0xff);
+
+ sum += (v < 128) ? v : 256 - v;
+
+ if (sum > lmins) /* We are already worse, don't continue. */
+ break;
+ }
+
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ if (png_ptr->heuristic_method == PNG_FILTER_HEURISTIC_WEIGHTED)
+ {
+ int j;
+ png_uint_32 sumhi, sumlo;
+ sumlo = sum & PNG_LOMASK;
+ sumhi = (sum >> PNG_HISHIFT) & PNG_HIMASK;
+
+ for (j = 0; j < num_p_filters; j++)
+ {
+ if (png_ptr->prev_filters[j] == PNG_FILTER_VALUE_UP)
+ {
+ sumlo = (sumlo * png_ptr->filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ sumhi = (sumhi * png_ptr->filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ }
+ }
+
+ sumlo = (sumlo * png_ptr->filter_costs[PNG_FILTER_VALUE_UP]) >>
+ PNG_COST_SHIFT;
+ sumhi = (sumhi * png_ptr->filter_costs[PNG_FILTER_VALUE_UP]) >>
+ PNG_COST_SHIFT;
+
+ if (sumhi > PNG_HIMASK)
+ sum = PNG_MAXSUM;
+ else
+ sum = (sumhi << PNG_HISHIFT) + sumlo;
+ }
+#endif
+
+ if (sum < mins)
+ {
+ mins = sum;
+ best_row = png_ptr->up_row;
+ }
+ }
+
+ /* Avg filter */
+ if (filter_to_do == PNG_FILTER_AVG)
+ {
+ png_bytep rp, dp, pp, lp;
+ png_uint_32 i;
+ for (i = 0, rp = row_buf + 1, dp = png_ptr->avg_row + 1,
+ pp = prev_row + 1; i < bpp; i++)
+ {
+ *dp++ = (png_byte)(((int)*rp++ - ((int)*pp++ / 2)) & 0xff);
+ }
+ for (lp = row_buf + 1; i < row_bytes; i++)
+ {
+ *dp++ = (png_byte)(((int)*rp++ - (((int)*pp++ + (int)*lp++) / 2))
+ & 0xff);
+ }
+ best_row = png_ptr->avg_row;
+ }
+
+ else if (filter_to_do & PNG_FILTER_AVG)
+ {
+ png_bytep rp, dp, pp, lp;
+ png_uint_32 sum = 0, lmins = mins;
+ png_uint_32 i;
+ int v;
+
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ if (png_ptr->heuristic_method == PNG_FILTER_HEURISTIC_WEIGHTED)
+ {
+ int j;
+ png_uint_32 lmhi, lmlo;
+ lmlo = lmins & PNG_LOMASK;
+ lmhi = (lmins >> PNG_HISHIFT) & PNG_HIMASK;
+
+ for (j = 0; j < num_p_filters; j++)
+ {
+ if (png_ptr->prev_filters[j] == PNG_FILTER_VALUE_AVG)
+ {
+ lmlo = (lmlo * png_ptr->inv_filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ lmhi = (lmhi * png_ptr->inv_filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ }
+ }
+
+ lmlo = (lmlo * png_ptr->inv_filter_costs[PNG_FILTER_VALUE_AVG]) >>
+ PNG_COST_SHIFT;
+ lmhi = (lmhi * png_ptr->inv_filter_costs[PNG_FILTER_VALUE_AVG]) >>
+ PNG_COST_SHIFT;
+
+ if (lmhi > PNG_HIMASK)
+ lmins = PNG_MAXSUM;
+ else
+ lmins = (lmhi << PNG_HISHIFT) + lmlo;
+ }
+#endif
+
+ for (i = 0, rp = row_buf + 1, dp = png_ptr->avg_row + 1,
+ pp = prev_row + 1; i < bpp; i++)
+ {
+ v = *dp++ = (png_byte)(((int)*rp++ - ((int)*pp++ / 2)) & 0xff);
+
+ sum += (v < 128) ? v : 256 - v;
+ }
+ for (lp = row_buf + 1; i < row_bytes; i++)
+ {
+ v = *dp++ =
+ (png_byte)(((int)*rp++ - (((int)*pp++ + (int)*lp++) / 2)) & 0xff);
+
+ sum += (v < 128) ? v : 256 - v;
+
+ if (sum > lmins) /* We are already worse, don't continue. */
+ break;
+ }
+
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ if (png_ptr->heuristic_method == PNG_FILTER_HEURISTIC_WEIGHTED)
+ {
+ int j;
+ png_uint_32 sumhi, sumlo;
+ sumlo = sum & PNG_LOMASK;
+ sumhi = (sum >> PNG_HISHIFT) & PNG_HIMASK;
+
+ for (j = 0; j < num_p_filters; j++)
+ {
+ if (png_ptr->prev_filters[j] == PNG_FILTER_VALUE_NONE)
+ {
+ sumlo = (sumlo * png_ptr->filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ sumhi = (sumhi * png_ptr->filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ }
+ }
+
+ sumlo = (sumlo * png_ptr->filter_costs[PNG_FILTER_VALUE_AVG]) >>
+ PNG_COST_SHIFT;
+ sumhi = (sumhi * png_ptr->filter_costs[PNG_FILTER_VALUE_AVG]) >>
+ PNG_COST_SHIFT;
+
+ if (sumhi > PNG_HIMASK)
+ sum = PNG_MAXSUM;
+ else
+ sum = (sumhi << PNG_HISHIFT) + sumlo;
+ }
+#endif
+
+ if (sum < mins)
+ {
+ mins = sum;
+ best_row = png_ptr->avg_row;
+ }
+ }
+
+ /* Paeth filter */
+ if (filter_to_do == PNG_FILTER_PAETH)
+ {
+ png_bytep rp, dp, pp, cp, lp;
+ png_uint_32 i;
+ for (i = 0, rp = row_buf + 1, dp = png_ptr->paeth_row + 1,
+ pp = prev_row + 1; i < bpp; i++)
+ {
+ *dp++ = (png_byte)(((int)*rp++ - (int)*pp++) & 0xff);
+ }
+
+ for (lp = row_buf + 1, cp = prev_row + 1; i < row_bytes; i++)
+ {
+ int a, b, c, pa, pb, pc, p;
+
+ b = *pp++;
+ c = *cp++;
+ a = *lp++;
+
+ p = b - c;
+ pc = a - c;
+
+#ifdef PNG_USE_ABS
+ pa = abs(p);
+ pb = abs(pc);
+ pc = abs(p + pc);
+#else
+ pa = p < 0 ? -p : p;
+ pb = pc < 0 ? -pc : pc;
+ pc = (p + pc) < 0 ? -(p + pc) : p + pc;
+#endif
+
+ p = (pa <= pb && pa <=pc) ? a : (pb <= pc) ? b : c;
+
+ *dp++ = (png_byte)(((int)*rp++ - p) & 0xff);
+ }
+ best_row = png_ptr->paeth_row;
+ }
+
+ else if (filter_to_do & PNG_FILTER_PAETH)
+ {
+ png_bytep rp, dp, pp, cp, lp;
+ png_uint_32 sum = 0, lmins = mins;
+ png_uint_32 i;
+ int v;
+
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ if (png_ptr->heuristic_method == PNG_FILTER_HEURISTIC_WEIGHTED)
+ {
+ int j;
+ png_uint_32 lmhi, lmlo;
+ lmlo = lmins & PNG_LOMASK;
+ lmhi = (lmins >> PNG_HISHIFT) & PNG_HIMASK;
+
+ for (j = 0; j < num_p_filters; j++)
+ {
+ if (png_ptr->prev_filters[j] == PNG_FILTER_VALUE_PAETH)
+ {
+ lmlo = (lmlo * png_ptr->inv_filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ lmhi = (lmhi * png_ptr->inv_filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ }
+ }
+
+ lmlo = (lmlo * png_ptr->inv_filter_costs[PNG_FILTER_VALUE_PAETH]) >>
+ PNG_COST_SHIFT;
+ lmhi = (lmhi * png_ptr->inv_filter_costs[PNG_FILTER_VALUE_PAETH]) >>
+ PNG_COST_SHIFT;
+
+ if (lmhi > PNG_HIMASK)
+ lmins = PNG_MAXSUM;
+ else
+ lmins = (lmhi << PNG_HISHIFT) + lmlo;
+ }
+#endif
+
+ for (i = 0, rp = row_buf + 1, dp = png_ptr->paeth_row + 1,
+ pp = prev_row + 1; i < bpp; i++)
+ {
+ v = *dp++ = (png_byte)(((int)*rp++ - (int)*pp++) & 0xff);
+
+ sum += (v < 128) ? v : 256 - v;
+ }
+
+ for (lp = row_buf + 1, cp = prev_row + 1; i < row_bytes; i++)
+ {
+ int a, b, c, pa, pb, pc, p;
+
+ b = *pp++;
+ c = *cp++;
+ a = *lp++;
+
+#ifndef PNG_SLOW_PAETH
+ p = b - c;
+ pc = a - c;
+#ifdef PNG_USE_ABS
+ pa = abs(p);
+ pb = abs(pc);
+ pc = abs(p + pc);
+#else
+ pa = p < 0 ? -p : p;
+ pb = pc < 0 ? -pc : pc;
+ pc = (p + pc) < 0 ? -(p + pc) : p + pc;
+#endif
+ p = (pa <= pb && pa <=pc) ? a : (pb <= pc) ? b : c;
+#else /* PNG_SLOW_PAETH */
+ p = a + b - c;
+ pa = abs(p - a);
+ pb = abs(p - b);
+ pc = abs(p - c);
+ if (pa <= pb && pa <= pc)
+ p = a;
+ else if (pb <= pc)
+ p = b;
+ else
+ p = c;
+#endif /* PNG_SLOW_PAETH */
+
+ v = *dp++ = (png_byte)(((int)*rp++ - p) & 0xff);
+
+ sum += (v < 128) ? v : 256 - v;
+
+ if (sum > lmins) /* We are already worse, don't continue. */
+ break;
+ }
+
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ if (png_ptr->heuristic_method == PNG_FILTER_HEURISTIC_WEIGHTED)
+ {
+ int j;
+ png_uint_32 sumhi, sumlo;
+ sumlo = sum & PNG_LOMASK;
+ sumhi = (sum >> PNG_HISHIFT) & PNG_HIMASK;
+
+ for (j = 0; j < num_p_filters; j++)
+ {
+ if (png_ptr->prev_filters[j] == PNG_FILTER_VALUE_PAETH)
+ {
+ sumlo = (sumlo * png_ptr->filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ sumhi = (sumhi * png_ptr->filter_weights[j]) >>
+ PNG_WEIGHT_SHIFT;
+ }
+ }
+
+ sumlo = (sumlo * png_ptr->filter_costs[PNG_FILTER_VALUE_PAETH]) >>
+ PNG_COST_SHIFT;
+ sumhi = (sumhi * png_ptr->filter_costs[PNG_FILTER_VALUE_PAETH]) >>
+ PNG_COST_SHIFT;
+
+ if (sumhi > PNG_HIMASK)
+ sum = PNG_MAXSUM;
+ else
+ sum = (sumhi << PNG_HISHIFT) + sumlo;
+ }
+#endif
+
+ if (sum < mins)
+ {
+ best_row = png_ptr->paeth_row;
+ }
+ }
+#endif /* PNG_WRITE_FILTER_SUPPORTED */
+ /* Do the actual writing of the filtered row data from the chosen filter. */
+
+ png_write_filtered_row(png_ptr, best_row);
+
+#ifdef PNG_WRITE_FILTER_SUPPORTED
+#ifdef PNG_WRITE_WEIGHTED_FILTER_SUPPORTED
+ /* Save the type of filter we picked this time for future calculations */
+ if (png_ptr->num_prev_filters > 0)
+ {
+ int j;
+ for (j = 1; j < num_p_filters; j++)
+ {
+ png_ptr->prev_filters[j] = png_ptr->prev_filters[j - 1];
+ }
+ png_ptr->prev_filters[j] = best_row[0];
+ }
+#endif
+#endif /* PNG_WRITE_FILTER_SUPPORTED */
+}
+
+
+/* Do the actual writing of a previously filtered row. */
+void /* PRIVATE */
+png_write_filtered_row(png_structp png_ptr, png_bytep filtered_row)
+{
+ png_debug(1, "in png_write_filtered_row");
+
+ png_debug1(2, "filter = %d", filtered_row[0]);
+ /* Set up the zlib input buffer */
+
+ png_ptr->zstream.next_in = filtered_row;
+ png_ptr->zstream.avail_in = (uInt)png_ptr->row_info.rowbytes + 1;
+ /* Repeat until we have compressed all the data */
+ do
+ {
+ int ret; /* Return of zlib */
+
+ /* Compress the data */
+ ret = deflate(&png_ptr->zstream, Z_NO_FLUSH);
+ /* Check for compression errors */
+ if (ret != Z_OK)
+ {
+ if (png_ptr->zstream.msg != NULL)
+ png_error(png_ptr, png_ptr->zstream.msg);
+ else
+ png_error(png_ptr, "zlib error");
+ }
+
+ /* See if it is time to write another IDAT */
+ if (!(png_ptr->zstream.avail_out))
+ {
+ /* Write the IDAT and reset the zlib output buffer */
+ png_write_IDAT(png_ptr, png_ptr->zbuf, png_ptr->zbuf_size);
+ png_ptr->zstream.next_out = png_ptr->zbuf;
+ png_ptr->zstream.avail_out = (uInt)png_ptr->zbuf_size;
+ }
+ /* Repeat until all data has been compressed */
+ } while (png_ptr->zstream.avail_in);
+
+ /* Swap the current and previous rows */
+ if (png_ptr->prev_row != NULL)
+ {
+ png_bytep tptr;
+
+ tptr = png_ptr->prev_row;
+ png_ptr->prev_row = png_ptr->row_buf;
+ png_ptr->row_buf = tptr;
+ }
+
+ /* Finish row - updates counters and flushes zlib if last row */
+ png_write_finish_row(png_ptr);
+
+#ifdef PNG_WRITE_FLUSH_SUPPORTED
+ png_ptr->flush_rows++;
+
+ if (png_ptr->flush_dist > 0 &&
+ png_ptr->flush_rows >= png_ptr->flush_dist)
+ {
+ png_write_flush(png_ptr);
+ }
+#endif
+}
+#endif /* PNG_WRITE_SUPPORTED */
diff --git a/trunk/src/third_party/opencv/gen/arch/linux/ia32/include/cvconfig.h b/trunk/src/third_party/opencv/gen/arch/linux/ia32/include/cvconfig.h
new file mode 100644
index 0000000..0daa41e
--- /dev/null
+++ b/trunk/src/third_party/opencv/gen/arch/linux/ia32/include/cvconfig.h
@@ -0,0 +1,158 @@
+/* Define to one of `_getb67', `GETB67', `getb67' for Cray-2 and Cray-YMP
+ systems. This function is required for `alloca.c' support on those systems.
+ */
+/* #undef CRAY_STACKSEG_END */
+
+/* Define to 1 if using `alloca.c'. */
+/* #undef C_ALLOCA */
+
+/* Define to 1 if you have `alloca', as a function or macro. */
+/* #undef HAVE_ALLOCA 1 */
+
+/* Define to 1 if you have <alloca.h> and it should be used (not on Ultrix).
+ */
+#define HAVE_ALLOCA_H 1
+
+/* V4L capturing support */
+#define HAVE_CAMV4L
+
+/* V4L2 capturing support */
+#define HAVE_CAMV4L2
+
+/* Carbon windowing environment */
+/* #undef HAVE_CARBON */
+
+/* IEEE1394 capturing support */
+/* #undef HAVE_DC1394 */
+
+/* libdc1394 0.9.4 or 0.9.5 */
+/* #undef HAVE_DC1394_095 */
+
+/* IEEE1394 capturing support - libdc1394 v2.x */
+/* #undef HAVE_DC1394_2 */
+
+/* ffmpeg in Gentoo */
+/* #undef HAVE_GENTOO_FFMPEG */
+
+/* FFMpeg video library */
+/* #undef HAVE_FFMPEG */
+
+/* ffmpeg's libswscale */
+/* #undef HAVE_FFMPEG_SWSCALE */
+
+/* GStreamer multimedia framework */
+/* #undef HAVE_GSTREAMER */
+
+/* GStreamer with gstappsink & gstappsrc */
+/* #undef HAVE_GSTREAMER_APP */
+
+/* GTK+ 2.0 Thread support */
+/* #undef HAVE_GTHREAD */
+
+/* GTK+ 2.x toolkit */
+/* #undef HAVE_GTK */
+
+/* OpenEXR codec */
+/* #undef HAVE_ILMIMF */
+
+/* Apple ImageIO Framework */
+/* #undef HAVE_IMAGEIO */
+
+/* Define to 1 if you have the <inttypes.h> header file. */
+/* #undef HAVE_INTTYPES_H 1 */
+
+/* JPEG-2000 codec */
+/* #undef HAVE_JASPER */
+
+/* IJG JPEG codec */
+#define HAVE_JPEG
+
+/* Define to 1 if you have the `dl' library (-ldl). */
+/* #undef HAVE_LIBDL 1 */
+
+/* Define to 1 if you have the `gomp' library (-lgomp). */
+/* #undef HAVE_LIBGOMP 1 */
+
+/* Define to 1 if you have the `m' library (-lm). */
+#define HAVE_LIBM 1
+
+/* libpng/png.h needs to be included */
+/* #undef HAVE_LIBPNG_PNG_H */
+
+/* Define to 1 if you have the `pthread' library (-lpthread). */
+#define HAVE_LIBPTHREAD 1
+
+/* Define to 1 if you have the `lrint' function. */
+/* #undef HAVE_LRINT 1 */
+
+/* PNG codec */
+#define HAVE_PNG
+
+/* Define to 1 if you have the `png_get_valid' function. */
+#define HAVE_PNG_GET_VALID 1
+
+/* png.h needs to be included */
+#define HAVE_PNG_H
+
+/* Define to 1 if you have the `png_set_tRNS_to_alpha' function. */
+#define HAVE_PNG_SET_TRNS_TO_ALPHA 1
+
+/* QuickTime video libraries */
+/* #undef HAVE_QUICKTIME */
+
+/* TIFF codec */
+/* #undef HAVE_TIFF */
+
+/* Unicap video capture library */
+/* #undef HAVE_UNICAP */
+
+/* Define to 1 if you have the <unistd.h> header file. */
+#define HAVE_UNISTD_H 1
+
+/* Xine video library */
+/* #undef HAVE_XINE */
+
+/* LZ77 compression/decompression library (used for PNG) */
+#define HAVE_ZLIB
+
+/* Intel Integrated Performance Primitives */
+/* #undef HAVE_IPP */
+
+/* OpenCV compiled as static or dynamic libs */
+#define OPENCV_BUILD_SHARED_LIB
+
+/* Name of package */
+#define PACKAGE "opencv"
+
+/* Define to the address where bug reports for this package should be sent. */
+#define PACKAGE_BUGREPORT "opencvlibrary-devel@lists.sourceforge.net"
+
+/* Define to the full name of this package. */
+#define PACKAGE_NAME "opencv"
+
+/* Define to the full name and version of this package. */
+#define PACKAGE_STRING "opencv 2.1.0"
+
+/* Define to the one symbol short name of this package. */
+#define PACKAGE_TARNAME "opencv"
+
+/* Define to the version of this package. */
+#define PACKAGE_VERSION "2.1.0"
+
+/* If using the C implementation of alloca, define if you know the
+ direction of stack growth for your system; otherwise it will be
+ automatically deduced at runtime.
+ STACK_DIRECTION > 0 => grows toward higher addresses
+ STACK_DIRECTION < 0 => grows toward lower addresses
+ STACK_DIRECTION = 0 => direction of growth unknown */
+/* #undef STACK_DIRECTION */
+
+/* Version number of package */
+#define VERSION "2.1.0"
+
+/* Define to 1 if your processor stores words with the most significant byte
+ first (like Motorola and SPARC, unlike Intel and VAX). */
+/* #undef WORDS_BIGENDIAN */
+
+/* Intel Threading Building Blocks */
+/* #undef HAVE_TBB */
diff --git a/trunk/src/third_party/opencv/gen/arch/linux/x64/include/cvconfig.h b/trunk/src/third_party/opencv/gen/arch/linux/x64/include/cvconfig.h
new file mode 100644
index 0000000..a9be56a
--- /dev/null
+++ b/trunk/src/third_party/opencv/gen/arch/linux/x64/include/cvconfig.h
@@ -0,0 +1,158 @@
+/* Define to one of `_getb67', `GETB67', `getb67' for Cray-2 and Cray-YMP
+ systems. This function is required for `alloca.c' support on those systems.
+ */
+/* #undef CRAY_STACKSEG_END */
+
+/* Define to 1 if using `alloca.c'. */
+/* #undef C_ALLOCA */
+
+/* Define to 1 if you have `alloca', as a function or macro. */
+/* #undef HAVE_ALLOCA */
+
+/* Define to 1 if you have <alloca.h> and it should be used (not on Ultrix).
+ */
+#define HAVE_ALLOCA_H 1
+
+/* V4L capturing support */
+#define HAVE_CAMV4L
+
+/* V4L2 capturing support */
+#define HAVE_CAMV4L2
+
+/* Carbon windowing environment */
+/* #undef HAVE_CARBON */
+
+/* IEEE1394 capturing support */
+/* #undef HAVE_DC1394 */
+
+/* libdc1394 0.9.4 or 0.9.5 */
+/* #undef HAVE_DC1394_095 */
+
+/* IEEE1394 capturing support - libdc1394 v2.x */
+/* #undef HAVE_DC1394_2 */
+
+/* ffmpeg in Gentoo */
+/* #undef HAVE_GENTOO_FFMPEG */
+
+/* FFMpeg video library */
+/* #undef HAVE_FFMPEG */
+
+/* ffmpeg's libswscale */
+/* #undef HAVE_FFMPEG_SWSCALE */
+
+/* GStreamer multimedia framework */
+/* #undef HAVE_GSTREAMER */
+
+/* GStreamer with gstappsink & gstappsrc */
+/* #undef HAVE_GSTREAMER_APP */
+
+/* GTK+ 2.0 Thread support */
+/* #undef HAVE_GTHREAD */
+
+/* GTK+ 2.x toolkit */
+/* #undef HAVE_GTK */
+
+/* OpenEXR codec */
+/* #undef HAVE_ILMIMF */
+
+/* Apple ImageIO Framework */
+/* #undef HAVE_IMAGEIO */
+
+/* Define to 1 if you have the <inttypes.h> header file. */
+/* #undef HAVE_INTTYPES_H */
+
+/* JPEG-2000 codec */
+/* #undef HAVE_JASPER */
+
+/* IJG JPEG codec */
+#define HAVE_JPEG
+
+/* Define to 1 if you have the `dl' library (-ldl). */
+/* #undef HAVE_LIBDL */
+
+/* Define to 1 if you have the `gomp' library (-lgomp). */
+/* #undef HAVE_LIBGOMP */
+
+/* Define to 1 if you have the `m' library (-lm). */
+#define HAVE_LIBM 1
+
+/* libpng/png.h needs to be included */
+/* #undef HAVE_LIBPNG_PNG_H */
+
+/* Define to 1 if you have the `pthread' library (-lpthread). */
+#define HAVE_LIBPTHREAD 1
+
+/* Define to 1 if you have the `lrint' function. */
+/* #undef HAVE_LRINT */
+
+/* PNG codec */
+#define HAVE_PNG
+
+/* Define to 1 if you have the `png_get_valid' function. */
+#define HAVE_PNG_GET_VALID 1
+
+/* png.h needs to be included */
+#define HAVE_PNG_H
+
+/* Define to 1 if you have the `png_set_tRNS_to_alpha' function. */
+#define HAVE_PNG_SET_TRNS_TO_ALPHA 1
+
+/* QuickTime video libraries */
+/* #undef HAVE_QUICKTIME */
+
+/* TIFF codec */
+/* #undef HAVE_TIFF */
+
+/* Unicap video capture library */
+/* #undef HAVE_UNICAP */
+
+/* Define to 1 if you have the <unistd.h> header file. */
+#define HAVE_UNISTD_H 1
+
+/* Xine video library */
+/* #undef HAVE_XINE */
+
+/* LZ77 compression/decompression library (used for PNG) */
+#define HAVE_ZLIB
+
+/* Intel Integrated Performance Primitives */
+/* #undef HAVE_IPP */
+
+/* OpenCV compiled as static or dynamic libs */
+#define OPENCV_BUILD_SHARED_LIB
+
+/* Name of package */
+#define PACKAGE "opencv"
+
+/* Define to the address where bug reports for this package should be sent. */
+#define PACKAGE_BUGREPORT "opencvlibrary-devel@lists.sourceforge.net"
+
+/* Define to the full name of this package. */
+#define PACKAGE_NAME "opencv"
+
+/* Define to the full name and version of this package. */
+#define PACKAGE_STRING "opencv 2.1.0"
+
+/* Define to the one symbol short name of this package. */
+#define PACKAGE_TARNAME "opencv"
+
+/* Define to the version of this package. */
+#define PACKAGE_VERSION "2.1.0"
+
+/* If using the C implementation of alloca, define if you know the
+ direction of stack growth for your system; otherwise it will be
+ automatically deduced at runtime.
+ STACK_DIRECTION > 0 => grows toward higher addresses
+ STACK_DIRECTION < 0 => grows toward lower addresses
+ STACK_DIRECTION = 0 => direction of growth unknown */
+/* #undef STACK_DIRECTION */
+
+/* Version number of package */
+#define VERSION "2.1.0"
+
+/* Define to 1 if your processor stores words with the most significant byte
+ first (like Motorola and SPARC, unlike Intel and VAX). */
+/* #undef WORDS_BIGENDIAN */
+
+/* Intel Threading Building Blocks */
+/* #undef HAVE_TBB */
diff --git a/trunk/src/third_party/opencv/gen/arch/mac/ia32/include/cvconfig.h b/trunk/src/third_party/opencv/gen/arch/mac/ia32/include/cvconfig.h
new file mode 100644
index 0000000..ab7c7a6
--- /dev/null
+++ b/trunk/src/third_party/opencv/gen/arch/mac/ia32/include/cvconfig.h
@@ -0,0 +1,158 @@
+/* Define to one of `_getb67', `GETB67', `getb67' for Cray-2 and Cray-YMP
+ systems. This function is required for `alloca.c' support on those systems.
+ */
+/* #undef CRAY_STACKSEG_END */
+
+/* Define to 1 if using `alloca.c'. */
+/* #undef C_ALLOCA */
+
+/* Define to 1 if you have `alloca', as a function or macro. */
+/* #undef HAVE_ALLOCA */
+
+/* Define to 1 if you have <alloca.h> and it should be used (not on Ultrix).
+ */
+/* #undef HAVE_ALLOCA_H */
+
+/* V4L capturing support */
+/* #undef HAVE_CAMV4L */
+
+/* V4L2 capturing support */
+/* #undef HAVE_CAMV4L2 */
+
+/* Carbon windowing environment */
+/* #undef HAVE_CARBON */
+
+/* IEEE1394 capturing support */
+/* #undef HAVE_DC1394 */
+
+/* libdc1394 0.9.4 or 0.9.5 */
+/* #undef HAVE_DC1394_095 */
+
+/* IEEE1394 capturing support - libdc1394 v2.x */
+/* #undef HAVE_DC1394_2 */
+
+/* ffmpeg in Gentoo */
+/* #undef HAVE_GENTOO_FFMPEG */
+
+/* FFMpeg video library */
+/* #undef HAVE_FFMPEG */
+
+/* ffmpeg's libswscale */
+/* #undef HAVE_FFMPEG_SWSCALE */
+
+/* GStreamer multimedia framework */
+/* #undef HAVE_GSTREAMER */
+
+/* GStreamer with gstappsink & gstappsrc */
+/* #undef HAVE_GSTREAMER_APP */
+
+/* GTK+ 2.0 Thread support */
+/* #undef HAVE_GTHREAD */
+
+/* GTK+ 2.x toolkit */
+/* #undef HAVE_GTK */
+
+/* OpenEXR codec */
+/* #undef HAVE_ILMIMF */
+
+/* Apple ImageIO Framework */
+/* #undef HAVE_IMAGEIO */
+
+/* Define to 1 if you have the <inttypes.h> header file. */
+/* #undef HAVE_INTTYPES_H */
+
+/* JPEG-2000 codec */
+/* #undef HAVE_JASPER */
+
+/* IJG JPEG codec */
+/* #undef HAVE_JPEG */
+
+/* Define to 1 if you have the `dl' library (-ldl). */
+/* #undef HAVE_LIBDL */
+
+/* Define to 1 if you have the `gomp' library (-lgomp). */
+/* #undef HAVE_LIBGOMP */
+
+/* Define to 1 if you have the `m' library (-lm). */
+/* #undef HAVE_LIBM */
+
+/* libpng/png.h needs to be included */
+/* #undef HAVE_LIBPNG_PNG_H */
+
+/* Define to 1 if you have the `pthread' library (-lpthread). */
+/* #undef HAVE_LIBPTHREAD */
+
+/* Define to 1 if you have the `lrint' function. */
+/* #undef HAVE_LRINT */
+
+/* PNG codec */
+/* #undef HAVE_PNG */
+
+/* Define to 1 if you have the `png_get_valid' function. */
+/* #undef HAVE_PNG_GET_VALID */
+
+/* png.h needs to be included */
+/* #undef HAVE_PNG_H */
+
+/* Define to 1 if you have the `png_set_tRNS_to_alpha' function. */
+/* #undef HAVE_PNG_SET_TRNS_TO_ALPHA */
+
+/* QuickTime video libraries */
+/* #undef HAVE_QUICKTIME */
+
+/* TIFF codec */
+/* #undef HAVE_TIFF */
+
+/* Unicap video capture library */
+/* #undef HAVE_UNICAP */
+
+/* Define to 1 if you have the <unistd.h> header file. */
+/* #undef HAVE_UNISTD_H */
+
+/* Xine video library */
+/* #undef HAVE_XINE */
+
+/* LZ77 compression/decompression library (used for PNG) */
+/* #undef HAVE_ZLIB */
+
+/* Intel Integrated Performance Primitives */
+/* #undef HAVE_IPP */
+
+/* OpenCV compiled as static or dynamic libs */
+#define OPENCV_BUILD_SHARED_LIB
+
+/* Name of package */
+#define PACKAGE "opencv"
+
+/* Define to the address where bug reports for this package should be sent. */
+#define PACKAGE_BUGREPORT "opencvlibrary-devel@lists.sourceforge.net"
+
+/* Define to the full name of this package. */
+#define PACKAGE_NAME "opencv"
+
+/* Define to the full name and version of this package. */
+#define PACKAGE_STRING "opencv 2.1.0"
+
+/* Define to the one symbol short name of this package. */
+#define PACKAGE_TARNAME "opencv"
+
+/* Define to the version of this package. */
+#define PACKAGE_VERSION "2.1.0"
+
+/* If using the C implementation of alloca, define if you know the
+ direction of stack growth for your system; otherwise it will be
+ automatically deduced at runtime.
+ STACK_DIRECTION > 0 => grows toward higher addresses
+ STACK_DIRECTION < 0 => grows toward lower addresses
+ STACK_DIRECTION = 0 => direction of growth unknown */
+/* #undef STACK_DIRECTION */
+
+/* Version number of package */
+#define VERSION "2.1.0"
+
+/* Define to 1 if your processor stores words with the most significant byte
+ first (like Motorola and SPARC, unlike Intel and VAX). */
+/* #undef WORDS_BIGENDIAN */
+
+/* Intel Threading Building Blocks */
+/* #undef HAVE_TBB */
diff --git a/trunk/src/third_party/opencv/opencv.gyp b/trunk/src/third_party/opencv/opencv.gyp
new file mode 100644
index 0000000..dcbe0ad
--- /dev/null
+++ b/trunk/src/third_party/opencv/opencv.gyp
@@ -0,0 +1,528 @@
+# Copyright 2009 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'variables': {
+ 'opencv_root': '<(DEPTH)/third_party/opencv',
+ 'opencv_src': '<(opencv_root)/src',
+ 'opencv_gen': '<(DEPTH)/third_party/opencv/gen/arch/<(OS)/<(target_arch)',
+ 'opencv_system_include%': '/usr/include/opencv',
+ 'use_system_opencv%': 0,
+ },
+ 'conditions': [
+ ['use_system_opencv==0', {
+ 'targets': [
+ {
+ 'target_name': 'flann',
+ 'type': '<(library)',
+ 'sources': [
+ 'src/opencv/3rdparty/flann/algorithms/dist.cpp',
+ 'src/opencv/3rdparty/flann/flann.cpp',
+ 'src/opencv/3rdparty/flann/nn/index_testing.cpp',
+ 'src/opencv/3rdparty/flann/util/logger.cpp',
+ 'src/opencv/3rdparty/flann/util/random.cpp',
+ 'src/opencv/3rdparty/flann/util/saving.cpp',
+ ],
+ 'include_dirs': [
+ '<(opencv_src)/opencv/3rdparty/flann/algorithms',
+ '<(opencv_src)/opencv/3rdparty/flann/nn',
+ '<(opencv_src)/opencv/3rdparty/flann/util',
+ '<(opencv_src)/opencv/3rdparty/include',
+ '<(opencv_src)/opencv/3rdparty/include/flann',
+ ],
+ },
+ {
+ 'target_name': 'lapack',
+ 'type': '<(library)',
+ 'sources': [
+ 'src/opencv/3rdparty/lapack/dsyevr.c',
+ 'src/opencv/3rdparty/lapack/slalsa.c',
+ 'src/opencv/3rdparty/lapack/sorml2.c',
+ 'src/opencv/3rdparty/lapack/scopy.c',
+ 'src/opencv/3rdparty/lapack/slassq.c',
+ 'src/opencv/3rdparty/lapack/dlarre.c',
+ 'src/opencv/3rdparty/lapack/dlasq2.c',
+ 'src/opencv/3rdparty/lapack/slarfg.c',
+ 'src/opencv/3rdparty/lapack/slaeda.c',
+ 'src/opencv/3rdparty/lapack/ddot.c',
+ 'src/opencv/3rdparty/lapack/dgetri.c',
+ 'src/opencv/3rdparty/lapack/slarft.c',
+ 'src/opencv/3rdparty/lapack/sgemm.c',
+ 'src/opencv/3rdparty/lapack/slarre.c',
+ 'src/opencv/3rdparty/lapack/slasq5.c',
+ 'src/opencv/3rdparty/lapack/strtrs.c',
+ 'src/opencv/3rdparty/lapack/dgetrf.c',
+ 'src/opencv/3rdparty/lapack/spotrf.c',
+ 'src/opencv/3rdparty/lapack/slapy2.c',
+ 'src/opencv/3rdparty/lapack/sormlq.c',
+ 'src/opencv/3rdparty/lapack/dsytri.c',
+ 'src/opencv/3rdparty/lapack/slaed1.c',
+ 'src/opencv/3rdparty/lapack/slaed5.c',
+ 'src/opencv/3rdparty/lapack/dlasda.c',
+ 'src/opencv/3rdparty/lapack/sgemv.c',
+ 'src/opencv/3rdparty/lapack/strti2.c',
+ 'src/opencv/3rdparty/lapack/dsyrk.c',
+ 'src/opencv/3rdparty/lapack/dlarrk.c',
+ 'src/opencv/3rdparty/lapack/sormbr.c',
+ 'src/opencv/3rdparty/lapack/sbdsdc.c',
+ 'src/opencv/3rdparty/lapack/dlascl.c',
+ 'src/opencv/3rdparty/lapack/slanst.c',
+ 'src/opencv/3rdparty/lapack/dlanst.c',
+ 'src/opencv/3rdparty/lapack/dorml2.c',
+ 'src/opencv/3rdparty/lapack/slaset.c',
+ 'src/opencv/3rdparty/lapack/dorm2r.c',
+ 'src/opencv/3rdparty/lapack/dorgbr.c',
+ 'src/opencv/3rdparty/lapack/dstemr.c',
+ 'src/opencv/3rdparty/lapack/dlasdq.c',
+ 'src/opencv/3rdparty/lapack/dlansy.c',
+ 'src/opencv/3rdparty/lapack/dlalsd.c',
+ 'src/opencv/3rdparty/lapack/sdot.c',
+ 'src/opencv/3rdparty/lapack/dsytf2.c',
+ 'src/opencv/3rdparty/lapack/slabrd.c',
+ 'src/opencv/3rdparty/lapack/dormql.c',
+ 'src/opencv/3rdparty/lapack/ilaenv.c',
+ 'src/opencv/3rdparty/lapack/dlaed9.c',
+ 'src/opencv/3rdparty/lapack/sormqr.c',
+ 'src/opencv/3rdparty/lapack/dlange.c',
+ 'src/opencv/3rdparty/lapack/dlabrd.c',
+ 'src/opencv/3rdparty/lapack/dlasd6.c',
+ 'src/opencv/3rdparty/lapack/dpotrs.c',
+ 'src/opencv/3rdparty/lapack/dlazq3.c',
+ 'src/opencv/3rdparty/lapack/idamax.c',
+ 'src/opencv/3rdparty/lapack/dsterf.c',
+ 'src/opencv/3rdparty/lapack/slasd7.c',
+ 'src/opencv/3rdparty/lapack/dlacpy.c',
+ 'src/opencv/3rdparty/lapack/dlasd8.c',
+ 'src/opencv/3rdparty/lapack/ssyr2k.c',
+ 'src/opencv/3rdparty/lapack/slasda.c',
+ 'src/opencv/3rdparty/lapack/dormtr.c',
+ 'src/opencv/3rdparty/lapack/slalsd.c',
+ 'src/opencv/3rdparty/lapack/dlarrj.c',
+ 'src/opencv/3rdparty/lapack/dlarra.c',
+ 'src/opencv/3rdparty/lapack/dlar1v.c',
+ 'src/opencv/3rdparty/lapack/dtrtrs.c',
+ 'src/opencv/3rdparty/lapack/slasd5.c',
+ 'src/opencv/3rdparty/lapack/slaruv.c',
+ 'src/opencv/3rdparty/lapack/dlamrg.c',
+ 'src/opencv/3rdparty/lapack/slaebz.c',
+ 'src/opencv/3rdparty/lapack/dgetf2.c',
+ 'src/opencv/3rdparty/lapack/sgelqf.c',
+ 'src/opencv/3rdparty/lapack/sgebrd.c',
+ 'src/opencv/3rdparty/lapack/slartg.c',
+ 'src/opencv/3rdparty/lapack/sorglq.c',
+ 'src/opencv/3rdparty/lapack/dormlq.c',
+ 'src/opencv/3rdparty/lapack/slaed2.c',
+ 'src/opencv/3rdparty/lapack/ssytrd.c',
+ 'src/opencv/3rdparty/lapack/dlaed5.c',
+ 'src/opencv/3rdparty/lapack/dlagts.c',
+ 'src/opencv/3rdparty/lapack/slarrf.c',
+ 'src/opencv/3rdparty/lapack/sorm2l.c',
+ 'src/opencv/3rdparty/lapack/sorgqr.c',
+ 'src/opencv/3rdparty/lapack/dlasq3.c',
+ 'src/opencv/3rdparty/lapack/slaed6.c',
+ 'src/opencv/3rdparty/lapack/dgesv.c',
+ 'src/opencv/3rdparty/lapack/dlasr.c',
+ 'src/opencv/3rdparty/lapack/s_cat.c',
+ 'src/opencv/3rdparty/lapack/dlasq5.c',
+ 'src/opencv/3rdparty/lapack/dbdsqr.c',
+ 'src/opencv/3rdparty/lapack/slae2.c',
+ 'src/opencv/3rdparty/lapack/srot.c',
+ 'src/opencv/3rdparty/lapack/sormql.c',
+ 'src/opencv/3rdparty/lapack/dlasq4.c',
+ 'src/opencv/3rdparty/lapack/sgeqrf.c',
+ 'src/opencv/3rdparty/lapack/sormtr.c',
+ 'src/opencv/3rdparty/lapack/slarrb.c',
+ 'src/opencv/3rdparty/lapack/slarrd.c',
+ 'src/opencv/3rdparty/lapack/sgetri.c',
+ 'src/opencv/3rdparty/lapack/dlartg.c',
+ 'src/opencv/3rdparty/lapack/dpotf2.c',
+ 'src/opencv/3rdparty/lapack/sorg2r.c',
+ 'src/opencv/3rdparty/lapack/dsytrd.c',
+ 'src/opencv/3rdparty/lapack/dgels.c',
+ 'src/opencv/3rdparty/lapack/slasd4.c',
+ 'src/opencv/3rdparty/lapack/slatrd.c',
+ 'src/opencv/3rdparty/lapack/ieeeck.c',
+ 'src/opencv/3rdparty/lapack/slasq4.c',
+ 'src/opencv/3rdparty/lapack/slarrv.c',
+ 'src/opencv/3rdparty/lapack/dtrmv.c',
+ 'src/opencv/3rdparty/lapack/dlaed1.c',
+ 'src/opencv/3rdparty/lapack/s_cmp.c',
+ 'src/opencv/3rdparty/lapack/ssterf.c',
+ 'src/opencv/3rdparty/lapack/dgelq2.c',
+ 'src/opencv/3rdparty/lapack/dlarrd.c',
+ 'src/opencv/3rdparty/lapack/ssyrk.c',
+ 'src/opencv/3rdparty/lapack/slamrg.c',
+ 'src/opencv/3rdparty/lapack/dlasrt.c',
+ 'src/opencv/3rdparty/lapack/slaed7.c',
+ 'src/opencv/3rdparty/lapack/dlarrv.c',
+ 'src/opencv/3rdparty/lapack/dlae2.c',
+ 'src/opencv/3rdparty/lapack/sgelsd.c',
+ 'src/opencv/3rdparty/lapack/dscal.c',
+ 'src/opencv/3rdparty/lapack/dlabad.c',
+ 'src/opencv/3rdparty/lapack/slaisnan.c',
+ 'src/opencv/3rdparty/lapack/dsytd2.c',
+ 'src/opencv/3rdparty/lapack/dlaed2.c',
+ 'src/opencv/3rdparty/lapack/slarnv.c',
+ 'src/opencv/3rdparty/lapack/dlaneg.c',
+ 'src/opencv/3rdparty/lapack/dorm2l.c',
+ 'src/opencv/3rdparty/lapack/f77_aloc.c',
+ 'src/opencv/3rdparty/lapack/strmm.c',
+ 'src/opencv/3rdparty/lapack/dlaisnan.c',
+ 'src/opencv/3rdparty/lapack/slals0.c',
+ 'src/opencv/3rdparty/lapack/dger.c',
+ 'src/opencv/3rdparty/lapack/slar1v.c',
+ 'src/opencv/3rdparty/lapack/slaed0.c',
+ 'src/opencv/3rdparty/lapack/dlaeda.c',
+ 'src/opencv/3rdparty/lapack/sger.c',
+ 'src/opencv/3rdparty/lapack/dcopy.c',
+ 'src/opencv/3rdparty/lapack/dlazq4.c',
+ 'src/opencv/3rdparty/lapack/daxpy.c',
+ 'src/opencv/3rdparty/lapack/ssyevr.c',
+ 'src/opencv/3rdparty/lapack/sgels.c',
+ 'src/opencv/3rdparty/lapack/dorglq.c',
+ 'src/opencv/3rdparty/lapack/dgelqf.c',
+ 'src/opencv/3rdparty/lapack/slarrc.c',
+ 'src/opencv/3rdparty/lapack/dlaebz.c',
+ 'src/opencv/3rdparty/lapack/sgetrs.c',
+ 'src/opencv/3rdparty/lapack/slasr.c',
+ 'src/opencv/3rdparty/lapack/dsteqr.c',
+ 'src/opencv/3rdparty/lapack/dtrtri.c',
+ 'src/opencv/3rdparty/lapack/dtrmm.c',
+ 'src/opencv/3rdparty/lapack/dstein.c',
+ 'src/opencv/3rdparty/lapack/dlaruv.c',
+ 'src/opencv/3rdparty/lapack/slaneg.c',
+ 'src/opencv/3rdparty/lapack/slarfb.c',
+ 'src/opencv/3rdparty/lapack/dorgqr.c',
+ 'src/opencv/3rdparty/lapack/dlasd3.c',
+ 'src/opencv/3rdparty/lapack/dlarnv.c',
+ 'src/opencv/3rdparty/lapack/precomp.c',
+ 'src/opencv/3rdparty/lapack/dsytrs.c',
+ 'src/opencv/3rdparty/lapack/dlarfg.c',
+ 'src/opencv/3rdparty/lapack/slaed9.c',
+ 'src/opencv/3rdparty/lapack/dgemm.c',
+ 'src/opencv/3rdparty/lapack/dgebrd.c',
+ 'src/opencv/3rdparty/lapack/slas2.c',
+ 'src/opencv/3rdparty/lapack/dlasq1.c',
+ 'src/opencv/3rdparty/lapack/dlarrr.c',
+ 'src/opencv/3rdparty/lapack/dpotri.c',
+ 'src/opencv/3rdparty/lapack/slasrt.c',
+ 'src/opencv/3rdparty/lapack/slagts.c',
+ 'src/opencv/3rdparty/lapack/dlasd4.c',
+ 'src/opencv/3rdparty/lapack/sbdsqr.c',
+ 'src/opencv/3rdparty/lapack/slarrk.c',
+ 'src/opencv/3rdparty/lapack/slaed8.c',
+ 'src/opencv/3rdparty/lapack/sstebz.c',
+ 'src/opencv/3rdparty/lapack/dlarrb.c',
+ 'src/opencv/3rdparty/lapack/saxpy.c',
+ 'src/opencv/3rdparty/lapack/dlauu2.c',
+ 'src/opencv/3rdparty/lapack/dasum.c',
+ 'src/opencv/3rdparty/lapack/pow_ri.c',
+ 'src/opencv/3rdparty/lapack/slange.c',
+ 'src/opencv/3rdparty/lapack/dpotrf.c',
+ 'src/opencv/3rdparty/lapack/dbdsdc.c',
+ 'src/opencv/3rdparty/lapack/sgesv.c',
+ 'src/opencv/3rdparty/lapack/drot.c',
+ 'src/opencv/3rdparty/lapack/sorgbr.c',
+ 'src/opencv/3rdparty/lapack/spotf2.c',
+ 'src/opencv/3rdparty/lapack/ssteqr.c',
+ 'src/opencv/3rdparty/lapack/dstebz.c',
+ 'src/opencv/3rdparty/lapack/dsyr2.c',
+ 'src/opencv/3rdparty/lapack/dlalsa.c',
+ 'src/opencv/3rdparty/lapack/sgebd2.c',
+ 'src/opencv/3rdparty/lapack/dgeqr2.c',
+ 'src/opencv/3rdparty/lapack/ssymv.c',
+ 'src/opencv/3rdparty/lapack/dlasdt.c',
+ 'src/opencv/3rdparty/lapack/dtrti2.c',
+ 'src/opencv/3rdparty/lapack/slasdt.c',
+ 'src/opencv/3rdparty/lapack/sorgl2.c',
+ 'src/opencv/3rdparty/lapack/slazq4.c',
+ 'src/opencv/3rdparty/lapack/dtrsm.c',
+ 'src/opencv/3rdparty/lapack/slascl.c',
+ 'src/opencv/3rdparty/lapack/iparmq.c',
+ 'src/opencv/3rdparty/lapack/sswap.c',
+ 'src/opencv/3rdparty/lapack/dlarrc.c',
+ 'src/opencv/3rdparty/lapack/dlassq.c',
+ 'src/opencv/3rdparty/lapack/dlasd5.c',
+ 'src/opencv/3rdparty/lapack/dlaswp.c',
+ 'src/opencv/3rdparty/lapack/slagtf.c',
+ 'src/opencv/3rdparty/lapack/dormbr.c',
+ 'src/opencv/3rdparty/lapack/dgesdd.c',
+ 'src/opencv/3rdparty/lapack/dlaed8.c',
+ 'src/opencv/3rdparty/lapack/dswap.c',
+ 'src/opencv/3rdparty/lapack/dlaed7.c',
+ 'src/opencv/3rdparty/lapack/sgesdd.c',
+ 'src/opencv/3rdparty/lapack/slasdq.c',
+ 'src/opencv/3rdparty/lapack/slaed3.c',
+ 'src/opencv/3rdparty/lapack/dlapy2.c',
+ 'src/opencv/3rdparty/lapack/dlaed4.c',
+ 'src/opencv/3rdparty/lapack/dsytrf.c',
+ 'src/opencv/3rdparty/lapack/dgeqrf.c',
+ 'src/opencv/3rdparty/lapack/slasd0.c',
+ 'src/opencv/3rdparty/lapack/slasd3.c',
+ 'src/opencv/3rdparty/lapack/dgemv.c',
+ 'src/opencv/3rdparty/lapack/slasd6.c',
+ 'src/opencv/3rdparty/lapack/dlamch.c',
+ 'src/opencv/3rdparty/lapack/dlasd0.c',
+ 'src/opencv/3rdparty/lapack/s_copy.c',
+ 'src/opencv/3rdparty/lapack/snrm2.c',
+ 'src/opencv/3rdparty/lapack/dsymv.c',
+ 'src/opencv/3rdparty/lapack/dgelsd.c',
+ 'src/opencv/3rdparty/lapack/dlarf.c',
+ 'src/opencv/3rdparty/lapack/strsm.c',
+ 'src/opencv/3rdparty/lapack/xerbla.c',
+ 'src/opencv/3rdparty/lapack/slasq1.c',
+ 'src/opencv/3rdparty/lapack/dlasd1.c',
+ 'src/opencv/3rdparty/lapack/slazq3.c',
+ 'src/opencv/3rdparty/lapack/spotri.c',
+ 'src/opencv/3rdparty/lapack/dlarfb.c',
+ 'src/opencv/3rdparty/lapack/sstein.c',
+ 'src/opencv/3rdparty/lapack/strmv.c',
+ 'src/opencv/3rdparty/lapack/pow_ii.c',
+ 'src/opencv/3rdparty/lapack/slarrr.c',
+ 'src/opencv/3rdparty/lapack/slauum.c',
+ 'src/opencv/3rdparty/lapack/dlals0.c',
+ 'src/opencv/3rdparty/lapack/slasq2.c',
+ 'src/opencv/3rdparty/lapack/dgetrs.c',
+ 'src/opencv/3rdparty/lapack/dlatrd.c',
+ 'src/opencv/3rdparty/lapack/slauu2.c',
+ 'src/opencv/3rdparty/lapack/slaev2.c',
+ 'src/opencv/3rdparty/lapack/slacpy.c',
+ 'src/opencv/3rdparty/lapack/dlaed0.c',
+ 'src/opencv/3rdparty/lapack/slasq6.c',
+ 'src/opencv/3rdparty/lapack/dlasd2.c',
+ 'src/opencv/3rdparty/lapack/slarrj.c',
+ 'src/opencv/3rdparty/lapack/sasum.c',
+ 'src/opencv/3rdparty/lapack/dsyr.c',
+ 'src/opencv/3rdparty/lapack/slasd2.c',
+ 'src/opencv/3rdparty/lapack/dlaed3.c',
+ 'src/opencv/3rdparty/lapack/slasd8.c',
+ 'src/opencv/3rdparty/lapack/sstemr.c',
+ 'src/opencv/3rdparty/lapack/slabad.c',
+ 'src/opencv/3rdparty/lapack/slansy.c',
+ 'src/opencv/3rdparty/lapack/spotrs.c',
+ 'src/opencv/3rdparty/lapack/dnrm2.c',
+ 'src/opencv/3rdparty/lapack/sgeqr2.c',
+ 'src/opencv/3rdparty/lapack/slamch.c',
+ 'src/opencv/3rdparty/lapack/sgetf2.c',
+ 'src/opencv/3rdparty/lapack/dlaev2.c',
+ 'src/opencv/3rdparty/lapack/pow_di.c',
+ 'src/opencv/3rdparty/lapack/slarf.c',
+ 'src/opencv/3rdparty/lapack/dlagtf.c',
+ 'src/opencv/3rdparty/lapack/slarra.c',
+ 'src/opencv/3rdparty/lapack/dlaed6.c',
+ 'src/opencv/3rdparty/lapack/dlarrf.c',
+ 'src/opencv/3rdparty/lapack/ssytd2.c',
+ 'src/opencv/3rdparty/lapack/slasv2.c',
+ 'src/opencv/3rdparty/lapack/dsyr2k.c',
+ 'src/opencv/3rdparty/lapack/ssyr2.c',
+ 'src/opencv/3rdparty/lapack/slaswp.c',
+ 'src/opencv/3rdparty/lapack/dormqr.c',
+ 'src/opencv/3rdparty/lapack/isamax.c',
+ 'src/opencv/3rdparty/lapack/dlasq6.c',
+ 'src/opencv/3rdparty/lapack/strtri.c',
+ 'src/opencv/3rdparty/lapack/slasq3.c',
+ 'src/opencv/3rdparty/lapack/dorg2r.c',
+ 'src/opencv/3rdparty/lapack/dlarft.c',
+ 'src/opencv/3rdparty/lapack/dlauum.c',
+ 'src/opencv/3rdparty/lapack/sgetrf.c',
+ 'src/opencv/3rdparty/lapack/dlaset.c',
+ 'src/opencv/3rdparty/lapack/dlas2.c',
+ 'src/opencv/3rdparty/lapack/dlasv2.c',
+ 'src/opencv/3rdparty/lapack/sgelq2.c',
+ 'src/opencv/3rdparty/lapack/dlasd7.c',
+ 'src/opencv/3rdparty/lapack/sscal.c',
+ 'src/opencv/3rdparty/lapack/dgebd2.c',
+ 'src/opencv/3rdparty/lapack/slaed4.c',
+ 'src/opencv/3rdparty/lapack/slasd1.c',
+ 'src/opencv/3rdparty/lapack/sorm2r.c',
+ 'src/opencv/3rdparty/lapack/dorgl2.c',
+ 'src/opencv/3rdparty/lapack/dlasyf.c',
+ ],
+ 'include_dirs': [
+ '<(opencv_src)/opencv/3rdparty/include',
+ ],
+ },
+ {
+ 'target_name': 'cxcore',
+ 'type': '<(library)',
+ 'dependencies': [
+ 'flann',
+ 'lapack',
+ '<(DEPTH)/third_party/zlib/zlib.gyp:zlib',
+ ],
+ 'sources': [
+ 'src/opencv/src/cxcore/cxalloc.cpp',
+ 'src/opencv/src/cxcore/cxarithm.cpp',
+ 'src/opencv/src/cxcore/cxarray.cpp',
+ 'src/opencv/src/cxcore/cxconvert.cpp',
+ 'src/opencv/src/cxcore/cxcopy.cpp',
+ 'src/opencv/src/cxcore/_cxcore.h',
+ 'src/opencv/src/cxcore/cxdatastructs.cpp',
+ 'src/opencv/src/cxcore/cxdrawing.cpp',
+ 'src/opencv/src/cxcore/cxdxt.cpp',
+ 'src/opencv/src/cxcore/cxflann.cpp',
+ 'src/opencv/src/cxcore/cxlapack.cpp',
+ 'src/opencv/src/cxcore/cxmathfuncs.cpp',
+ 'src/opencv/src/cxcore/cxmatmul.cpp',
+ 'src/opencv/src/cxcore/cxmatrix.cpp',
+ 'src/opencv/src/cxcore/cxstat.cpp',
+ 'src/opencv/src/cxcore/cxpersistence.cpp',
+ 'src/opencv/src/cxcore/cxprecomp.cpp',
+ 'src/opencv/src/cxcore/cxrand.cpp',
+ 'src/opencv/src/cxcore/cxsystem.cpp',
+ 'src/opencv/src/cxcore/cxtables.cpp',
+ ],
+ 'include_dirs': [
+ # opencv provides its own copy of zlib.h, but we want to use
+ # our version in third_party/zlib, so third_party/zlib must
+ # appear early in the list of include dirs.
+ '<(DEPTH)/third_party/zlib',
+ '<(opencv_src)/opencv/include/opencv',
+ '<(opencv_src)/opencv/3rdparty/include',
+ ],
+ },
+ {
+ 'target_name': 'cv',
+ 'type': '<(library)',
+ 'dependencies': [
+ 'cxcore',
+ ],
+ 'sources': [
+ 'src/opencv/src/cv/cvaccum.cpp',
+ 'src/opencv/src/cv/cvapprox.cpp',
+ 'src/opencv/src/cv/cvcalibinit.cpp',
+ 'src/opencv/src/cv/cvcalibration.cpp',
+ 'src/opencv/src/cv/cvcamshift.cpp',
+ 'src/opencv/src/cv/cvcanny.cpp',
+ 'src/opencv/src/cv/cvcascadedetect.cpp',
+ 'src/opencv/src/cv/cvcheckchessboard.cpp',
+ 'src/opencv/src/cv/cvcolor.cpp',
+ 'src/opencv/src/cv/cvcontours.cpp',
+ 'src/opencv/src/cv/cvcontourtree.cpp',
+ 'src/opencv/src/cv/cvconvhull.cpp',
+ 'src/opencv/src/cv/cvcorner.cpp',
+ 'src/opencv/src/cv/cvcornersubpix.cpp',
+ 'src/opencv/src/cv/cvderiv.cpp',
+ 'src/opencv/src/cv/cvdistransform.cpp',
+ 'src/opencv/src/cv/cvemd.cpp',
+ 'src/opencv/src/cv/cvfeatureselect.cpp',
+ 'src/opencv/src/cv/cvfloodfill.cpp',
+ 'src/opencv/src/cv/cvfundam.cpp',
+ 'src/opencv/src/cv/cvgeometry.cpp',
+ 'src/opencv/src/cv/cvhaar.cpp',
+ 'src/opencv/src/cv/cvhistogram.cpp',
+ 'src/opencv/src/cv/cvhough.cpp',
+ 'src/opencv/src/cv/cvimgwarp.cpp',
+ 'src/opencv/src/cv/cvkalman.cpp',
+ 'src/opencv/src/cv/cvlinefit.cpp',
+ 'src/opencv/src/cv/cvlkpyramid.cpp',
+ 'src/opencv/src/cv/cvmatchcontours.cpp',
+ 'src/opencv/src/cv/cvmodelest.cpp',
+ 'src/opencv/src/cv/cvmoments.cpp',
+ 'src/opencv/src/cv/cvmorph.cpp',
+ 'src/opencv/src/cv/cvmotempl.cpp',
+ 'src/opencv/src/cv/cvoptflowbm.cpp',
+ 'src/opencv/src/cv/cvoptflowhs.cpp',
+ 'src/opencv/src/cv/cvoptflowlk.cpp',
+ 'src/opencv/src/cv/cvposit.cpp',
+ 'src/opencv/src/cv/cvprecomp.cpp',
+ 'src/opencv/src/cv/cvpyramids.cpp',
+ 'src/opencv/src/cv/cvpyrsegmentation.cpp',
+ 'src/opencv/src/cv/cvrotcalipers.cpp',
+ 'src/opencv/src/cv/cvsamplers.cpp',
+ 'src/opencv/src/cv/cvshapedescr.cpp',
+ 'src/opencv/src/cv/cvsmooth.cpp',
+ 'src/opencv/src/cv/cvsnakes.cpp',
+ 'src/opencv/src/cv/cvsubdivision2d.cpp',
+ 'src/opencv/src/cv/cvsumpixels.cpp',
+ 'src/opencv/src/cv/cvtables.cpp',
+ 'src/opencv/src/cv/cvtemplmatch.cpp',
+ 'src/opencv/src/cv/cvthresh.cpp',
+ 'src/opencv/src/cv/cvundistort.cpp',
+ 'src/opencv/src/cv/cvutils.cpp',
+ 'src/opencv/src/cv/cvfilter.cpp',
+ 'src/opencv/src/cv/cvinpaint.cpp',
+ 'src/opencv/src/cv/cvsegmentation.cpp',
+ ],
+ 'include_dirs': [
+ '<(opencv_src)/opencv/include/opencv',
+ ],
+ },
+ {
+ 'target_name': 'highgui',
+ 'type': '<(library)',
+ 'dependencies': [
+ 'cv',
+ '<(DEPTH)/third_party/zlib/zlib.gyp:zlib',
+ '<(DEPTH)/third_party/libjpeg/libjpeg.gyp:libjpeg',
+ '<(DEPTH)/third_party/libpng/libpng.gyp:libpng',
+ ],
+ 'sources': [
+ 'src/opencv/src/highgui/bitstrm.cpp',
+ 'src/opencv/src/highgui/cvcap.cpp',
+ 'src/opencv/src/highgui/cvcap_dc1394.cpp',
+ 'src/opencv/src/highgui/cvcap_images.cpp',
+ 'src/opencv/src/highgui/cvcap_v4l.cpp',
+ 'src/opencv/src/highgui/grfmt_base.cpp',
+ 'src/opencv/src/highgui/grfmt_bmp.cpp',
+ 'src/opencv/src/highgui/grfmt_jpeg.cpp',
+ 'src/opencv/src/highgui/grfmt_png.cpp',
+ 'src/opencv/src/highgui/grfmt_pxm.cpp',
+ 'src/opencv/src/highgui/grfmt_sunras.cpp',
+ 'src/opencv/src/highgui/grfmt_tiff.cpp',
+ 'src/opencv/src/highgui/image.cpp',
+ 'src/opencv/src/highgui/loadsave.cpp',
+ 'src/opencv/src/highgui/precomp.cpp',
+ 'src/opencv/src/highgui/utils.cpp',
+ 'src/opencv/src/highgui/window.cpp',
+ ],
+ 'include_dirs': [
+ '<(opencv_gen)/include',
+ '<(opencv_src)/opencv/include/opencv',
+ ],
+ },
+ ],
+ },
+ { # use_system_opencv
+ 'targets': [
+ {
+ 'target_name': 'highgui',
+ 'type': 'settings',
+ 'include_dirs': [
+ '<(opencv_system_include)',
+ ],
+ 'direct_dependent_settings': {
+ 'defines': [
+ 'USE_SYSTEM_OPENCV',
+ ],
+ 'include_dirs': [
+ '<(opencv_system_include)',
+ ],
+ },
+ 'defines': [
+ 'USE_SYSTEM_OPENCV',
+ ],
+ 'link_settings': {
+ 'libraries': [
+ '-lcv',
+ '-lcxcore',
+ '-lhighgui',
+ ],
+ },
+ },
+ ],
+ }],
+ ],
+}
+
diff --git a/trunk/src/third_party/serf/instaweb_allocator.c b/trunk/src/third_party/serf/instaweb_allocator.c
new file mode 100644
index 0000000..5064b0d
--- /dev/null
+++ b/trunk/src/third_party/serf/instaweb_allocator.c
@@ -0,0 +1,415 @@
+/* Copyright 2002-2004 Justin Erenkrantz and Greg Stein
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <stdlib.h>
+
+#include <apr_pools.h>
+
+#include "serf.h"
+#include "serf_bucket_util.h"
+
+
+typedef struct node_header_t {
+ apr_size_t size;
+ union {
+ struct node_header_t *next; /* if size == 0 (freed/inactive) */
+ /* no data if size == STANDARD_NODE_SIZE */
+ apr_memnode_t *memnode; /* if size > STANDARD_NODE_SIZE */
+ } u;
+} node_header_t;
+
+/* The size of a node_header_t, properly aligned. Note that (normally)
+ * this macro will round the size to a multiple of 8 bytes. Keep this in
+ * mind when altering the node_header_t structure. Also, keep in mind that
+ * node_header_t is an overhead for every allocation performed through
+ * the serf_bucket_mem_alloc() function.
+ */
+#define SIZEOF_NODE_HEADER_T APR_ALIGN_DEFAULT(sizeof(node_header_t))
+
+
+/* STANDARD_NODE_SIZE is manually set to an allocation size that will
+ * capture most allocators performed via this API. It must be "large
+ * enough" to avoid lots of spillage to allocating directly from the
+ * apr_allocator associated with the bucket allocator. The apr_allocator
+ * has a minimum size of 8k, which can be expensive if you missed the
+ * STANDARD_NODE_SIZE by just a few bytes.
+ */
+/* ### we should define some rules or ways to determine how to derive
+ * ### a "good" value for this. probably log some stats on allocs, then
+ * ### analyze them for size "misses". then find the balance point between
+ * ### wasted space due to min-size allocator, and wasted-space due to
+ * ### size-spill to the 8k minimum.
+ */
+#define STANDARD_NODE_SIZE 128
+
+/* When allocating a block of memory from the allocator, we should go for
+ * an 8k block, minus the overhead that the allocator needs.
+ */
+#define ALLOC_AMT (8192 - APR_MEMNODE_T_SIZE)
+
+/* Define DEBUG_DOUBLE_FREE if you're interested in debugging double-free
+ * calls to serf_bucket_mem_free().
+ */
+#define DEBUG_DOUBLE_FREE
+
+
+typedef struct {
+ const serf_bucket_t *bucket;
+ apr_status_t last;
+} read_status_t;
+
+#define TRACK_BUCKET_COUNT 100 /* track N buckets' status */
+
+typedef struct {
+ int next_index; /* info[] is a ring. next bucket goes at this idx. */
+ int num_used;
+
+ read_status_t info[TRACK_BUCKET_COUNT];
+} track_state_t;
+
+
+struct serf_bucket_alloc_t {
+ apr_pool_t *pool;
+ apr_allocator_t *allocator;
+ int own_allocator;
+
+ serf_unfreed_func_t unfreed;
+ void *unfreed_baton;
+
+ apr_uint32_t num_alloc;
+
+ node_header_t *freelist; /* free STANDARD_NODE_SIZE blocks */
+ apr_memnode_t *blocks; /* blocks we allocated for subdividing */
+
+ track_state_t *track;
+};
+
+/* ==================================================================== */
+
+
+static apr_status_t allocator_cleanup(void *data)
+{
+ serf_bucket_alloc_t *allocator = data;
+
+ /* If we allocated anything, give it back. */
+ if (allocator->blocks) {
+ apr_allocator_free(allocator->allocator, allocator->blocks);
+ }
+ if (allocator->own_allocator == 1) {
+ apr_allocator_destroy(allocator->allocator);
+ }
+ return APR_SUCCESS;
+}
+
+SERF_DECLARE(serf_bucket_alloc_t *) serf_bucket_allocator_create(
+ apr_pool_t *pool,
+ serf_unfreed_func_t unfreed,
+ void *unfreed_baton)
+{
+ serf_bucket_alloc_t *allocator = apr_pcalloc(pool, sizeof(*allocator));
+
+ allocator->pool = pool;
+ allocator->allocator = apr_pool_allocator_get(pool);
+ allocator->own_allocator = 0;
+ if (allocator->allocator == NULL) {
+ /* This most likely means pools are running in debug mode, create our
+ * own allocator to deal with memory ourselves */
+ apr_allocator_create(&allocator->allocator);
+ allocator->own_allocator = 1;
+ }
+ allocator->unfreed = unfreed;
+ allocator->unfreed_baton = unfreed_baton;
+
+#ifdef SERF_DEBUG_BUCKET_USE
+ {
+ track_state_t *track;
+
+ track = allocator->track = apr_palloc(pool, sizeof(*allocator->track));
+ track->next_index = 0;
+ track->num_used = 0;
+ }
+#endif
+
+ /* ### this implies buckets cannot cross a fork/exec. desirable?
+ *
+ * ### hmm. it probably also means that buckets cannot be AROUND
+ * ### during a fork/exec. the new process will try to clean them
+ * ### up and figure out there are unfreed blocks...
+ */
+ apr_pool_cleanup_register(pool, allocator,
+ allocator_cleanup, allocator_cleanup);
+
+ return allocator;
+}
+
+SERF_DECLARE(apr_pool_t *) serf_bucket_allocator_get_pool(
+ const serf_bucket_alloc_t *allocator)
+{
+ return allocator->pool;
+}
+
+SERF_DECLARE(void *) serf_bucket_mem_alloc(
+ serf_bucket_alloc_t *allocator,
+ apr_size_t size)
+{
+ node_header_t *node;
+
+ ++allocator->num_alloc;
+
+ size += SIZEOF_NODE_HEADER_T;
+ if (size <= STANDARD_NODE_SIZE) {
+ if (allocator->freelist) {
+ /* just pull a node off our freelist */
+ node = allocator->freelist;
+ allocator->freelist = node->u.next;
+#ifdef DEBUG_DOUBLE_FREE
+ /* When we free an item, we set its size to zero. Thus, when
+ * we return it to the caller, we must ensure the size is set
+ * properly.
+ */
+ node->size = STANDARD_NODE_SIZE;
+#endif
+ }
+ else {
+ apr_memnode_t *active = allocator->blocks;
+
+ if (active == NULL
+ || active->first_avail + STANDARD_NODE_SIZE >= active->endp) {
+ apr_memnode_t *head = allocator->blocks;
+
+ /* ran out of room. grab another block. */
+ active = apr_allocator_alloc(allocator->allocator, ALLOC_AMT);
+
+ /* link the block into our tracking list */
+ allocator->blocks = active;
+ active->next = head;
+ }
+
+ node = (node_header_t *)active->first_avail;
+ node->size = STANDARD_NODE_SIZE;
+ active->first_avail += STANDARD_NODE_SIZE;
+ }
+ }
+ else {
+ apr_memnode_t *memnode = apr_allocator_alloc(allocator->allocator,
+ size);
+
+ node = (node_header_t *)memnode->first_avail;
+ node->u.memnode = memnode;
+ node->size = size;
+ }
+
+ return ((char *)node) + SIZEOF_NODE_HEADER_T;
+}
+
+SERF_DECLARE(void *) serf_bucket_mem_calloc(
+ serf_bucket_alloc_t *allocator,
+ apr_size_t size)
+{
+ void *mem;
+ mem = serf_bucket_mem_alloc(allocator, size);
+ memset(mem, 0, size);
+ return mem;
+}
+
+SERF_DECLARE(void) serf_bucket_mem_free(
+ serf_bucket_alloc_t *allocator,
+ void *block)
+{
+ node_header_t *node;
+
+ --allocator->num_alloc;
+
+ node = (node_header_t *)((char *)block - SIZEOF_NODE_HEADER_T);
+
+ if (node->size == STANDARD_NODE_SIZE) {
+ /* put the node onto our free list */
+ node->u.next = allocator->freelist;
+ allocator->freelist = node;
+
+#ifdef DEBUG_DOUBLE_FREE
+ /* note that this thing was freed. */
+ node->size = 0;
+ }
+ else if (node->size == 0) {
+ /* damn thing was freed already. */
+ abort();
+#endif
+ }
+ else {
+#ifdef DEBUG_DOUBLE_FREE
+ /* note that this thing was freed. */
+ node->size = 0;
+#endif
+
+ /* now free it */
+ apr_allocator_free(allocator->allocator, node->u.memnode);
+ }
+}
+
+
+/* ==================================================================== */
+
+
+#ifdef SERF_DEBUG_BUCKET_USE
+
+static read_status_t *find_read_status(
+ track_state_t *track,
+ const serf_bucket_t *bucket,
+ int create_rs)
+{
+ read_status_t *rs;
+
+ if (track->num_used) {
+ int count = track->num_used;
+ int idx = track->next_index;
+
+ /* Search backwards. In all likelihood, the bucket which just got
+ * read was read very recently.
+ */
+ while (count-- > 0) {
+ if (!idx--) {
+ /* assert: track->num_used == TRACK_BUCKET_COUNT */
+ idx = track->num_used - 1;
+ }
+ if ((rs = &track->info[idx])->bucket == bucket) {
+ return rs;
+ }
+ }
+ }
+
+ /* Only create a new read_status_t when asked. */
+ if (!create_rs)
+ return NULL;
+
+ if (track->num_used < TRACK_BUCKET_COUNT) {
+ /* We're still filling up the ring. */
+ ++track->num_used;
+ }
+
+ rs = &track->info[track->next_index];
+ rs->bucket = bucket;
+ rs->last = APR_SUCCESS; /* ### the right initial value? */
+
+ if (++track->next_index == TRACK_BUCKET_COUNT)
+ track->next_index = 0;
+
+ return rs;
+}
+
+#endif /* SERF_DEBUG_BUCKET_USE */
+
+
+SERF_DECLARE(apr_status_t) serf_debug__record_read(
+ const serf_bucket_t *bucket,
+ apr_status_t status)
+{
+#ifndef SERF_DEBUG_BUCKET_USE
+ return status;
+#else
+
+ track_state_t *track = bucket->allocator->track;
+ read_status_t *rs = find_read_status(track, bucket, 1);
+
+ /* Validate that the previous status value allowed for another read. */
+ if (APR_STATUS_IS_EAGAIN(rs->last) /* ### or APR_EOF? */) {
+ /* Somebody read when they weren't supposed to. Bail. */
+ abort();
+ }
+
+ /* Save the current status for later. */
+ rs->last = status;
+
+ return status;
+#endif
+}
+
+SERF_DECLARE(void) serf_debug__entered_loop(serf_bucket_alloc_t *allocator)
+{
+#ifdef SERF_DEBUG_BUCKET_USE
+
+ track_state_t *track = allocator->track;
+ read_status_t *rs = &track->info[0];
+
+ for ( ; track->num_used; --track->num_used, ++rs ) {
+ if (rs->last == APR_SUCCESS) {
+ /* Somebody should have read this bucket again. */
+ abort();
+ }
+
+ /* ### other status values? */
+ }
+
+ /* num_used was reset. also need to reset the next index. */
+ track->next_index = 0;
+
+#endif
+}
+
+SERF_DECLARE(void) serf_debug__closed_conn(serf_bucket_alloc_t *allocator)
+{
+#ifdef SERF_DEBUG_BUCKET_USE
+
+ /* Just reset the number used so that we don't examine the info[] */
+ allocator->track->num_used = 0;
+ allocator->track->next_index = 0;
+
+#endif
+}
+
+SERF_DECLARE(void) serf_debug__bucket_destroy(const serf_bucket_t *bucket)
+{
+#ifdef SERF_DEBUG_BUCKET_USE
+
+ track_state_t *track = bucket->allocator->track;
+ read_status_t *rs = find_read_status(track, bucket, 0);
+
+ if (rs != NULL && rs->last != APR_EOF) {
+ /* The bucket was destroyed before it was read to completion. */
+
+ /* Special exception for socket buckets. If a connection remains
+ * open, they are not read to completion.
+ */
+ if (SERF_BUCKET_IS_SOCKET(bucket))
+ return;
+
+ /* Ditto for SSL Decrypt buckets. */
+ if (SERF_BUCKET_IS_SSL_DECRYPT(bucket))
+ return;
+
+ /* Ditto for SSL Encrypt buckets. */
+ if (SERF_BUCKET_IS_SSL_ENCRYPT(bucket))
+ return;
+
+ /* Ditto for barrier buckets. */
+ if (SERF_BUCKET_IS_BARRIER(bucket))
+ return;
+
+
+ abort();
+ }
+
+#endif
+}
+
+SERF_DECLARE(void) serf_debug__bucket_alloc_check(
+ serf_bucket_alloc_t *allocator)
+{
+#ifdef SERF_DEBUG_BUCKET_USE
+ if (allocator->num_alloc != 0) {
+ abort();
+ }
+#endif
+}
diff --git a/trunk/src/third_party/serf/instaweb_context.c b/trunk/src/third_party/serf/instaweb_context.c
new file mode 100644
index 0000000..a2b2ca2
--- /dev/null
+++ b/trunk/src/third_party/serf/instaweb_context.c
@@ -0,0 +1,1467 @@
+/* Copyright 2002-2004 Justin Erenkrantz and Greg Stein
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <apr_pools.h>
+#include <apr_poll.h>
+#include <apr_version.h>
+
+#include "serf.h"
+#include "serf_bucket_util.h"
+
+
+/* ### what the hell? why does the APR interface have a "size" ??
+ ### the implication is that, if we bust this limit, we'd need to
+ ### stop, rebuild a pollset, and repopulate it. what suckage. */
+#define MAX_CONN 16
+
+/* Windows does not define IOV_MAX, so we need to ensure it is defined. */
+#ifndef IOV_MAX
+#define IOV_MAX 16
+#endif
+
+/* Holds all the information corresponding to a request/response pair. */
+struct serf_request_t {
+ serf_connection_t *conn;
+
+ apr_pool_t *respool;
+ serf_bucket_alloc_t *allocator;
+
+ /* The bucket corresponding to the request. Will be NULL once the
+ * bucket has been emptied (for delivery into the socket).
+ */
+ serf_bucket_t *req_bkt;
+
+ serf_request_setup_t setup;
+ void *setup_baton;
+
+ serf_response_acceptor_t acceptor;
+ void *acceptor_baton;
+
+ serf_response_handler_t handler;
+ void *handler_baton;
+
+ serf_bucket_t *resp_bkt;
+
+ struct serf_request_t *next;
+};
+
+typedef struct serf_pollset_t {
+ /* the set of connections to poll */
+ apr_pollset_t *pollset;
+} serf_pollset_t;
+
+struct serf_context_t {
+ /* the pool used for self and for other allocations */
+ apr_pool_t *pool;
+
+ void *pollset_baton;
+ serf_socket_add_t pollset_add;
+ serf_socket_remove_t pollset_rm;
+
+ /* one of our connections has a dirty pollset state. */
+ int dirty_pollset;
+
+ /* the list of active connections */
+ apr_array_header_t *conns;
+#define GET_CONN(ctx, i) (((serf_connection_t **)(ctx)->conns->elts)[i])
+
+ /* Proxy server address */
+ apr_sockaddr_t *proxy_address;
+
+ /* Progress callback */
+ serf_progress_t progress_func;
+ void *progress_baton;
+ apr_off_t progress_read;
+ apr_off_t progress_written;
+};
+
+struct serf_connection_t {
+ serf_context_t *ctx;
+
+ apr_pool_t *pool;
+ serf_bucket_alloc_t *allocator;
+
+ apr_sockaddr_t *address;
+
+ apr_socket_t *skt;
+ apr_pool_t *skt_pool;
+
+ /* the last reqevents we gave to pollset_add */
+ apr_int16_t reqevents;
+
+ /* the events we've seen for this connection in our returned pollset */
+ apr_int16_t seen_in_pollset;
+
+ /* are we a dirty connection that needs its poll status updated? */
+ int dirty_conn;
+
+ /* number of completed requests we've sent */
+ unsigned int completed_requests;
+
+ /* number of completed responses we've got */
+ unsigned int completed_responses;
+
+ /* keepalive */
+ unsigned int probable_keepalive_limit;
+
+ /* someone has told us that the connection is closing
+ * so, let's start a new socket.
+ */
+ int closing;
+
+ /* A bucket wrapped around our socket (for reading responses). */
+ serf_bucket_t *stream;
+
+ /* The list of active requests. */
+ serf_request_t *requests;
+ serf_request_t *requests_tail;
+
+ /* The list of requests we're holding on to because we're going to
+ * reset the connection soon.
+ */
+ serf_request_t *hold_requests;
+ serf_request_t *hold_requests_tail;
+
+ struct iovec vec[IOV_MAX];
+ int vec_len;
+
+ serf_connection_setup_t setup;
+ void *setup_baton;
+ serf_connection_closed_t closed;
+ void *closed_baton;
+
+ /* Max. number of outstanding requests. */
+ unsigned int max_outstanding_requests;
+
+ /* Host info. */
+ const char *host_url;
+ apr_uri_t host_info;
+};
+
+/* cleanup for sockets */
+static apr_status_t clean_skt(void *data)
+{
+ serf_connection_t *conn = data;
+ apr_status_t status = APR_SUCCESS;
+
+ if (conn->skt) {
+ status = apr_socket_close(conn->skt);
+ conn->skt = NULL;
+ }
+
+ return status;
+}
+
+static apr_status_t clean_resp(void *data)
+{
+ serf_request_t *req = data;
+
+ /* This pool just got cleared/destroyed. Don't try to destroy the pool
+ * (again) when the request is canceled.
+ */
+ req->respool = NULL;
+
+ return APR_SUCCESS;
+}
+
+/* cleanup for conns */
+static apr_status_t clean_conn(void *data)
+{
+ serf_connection_t *conn = data;
+
+ serf_connection_close(conn);
+
+ return APR_SUCCESS;
+}
+
+/* Update the pollset for this connection. We tweak the pollset based on
+ * whether we want to read and/or write, given conditions within the
+ * connection. If the connection is not (yet) in the pollset, then it
+ * will be added.
+ */
+static apr_status_t update_pollset(serf_connection_t *conn)
+{
+ serf_context_t *ctx = conn->ctx;
+ apr_status_t status;
+ apr_pollfd_t desc = { 0 };
+
+ if (!conn->skt) {
+ return APR_SUCCESS;
+ }
+
+ /* Remove the socket from the poll set. */
+ desc.desc_type = APR_POLL_SOCKET;
+ desc.desc.s = conn->skt;
+ desc.reqevents = conn->reqevents;
+
+ status = ctx->pollset_rm(ctx->pollset_baton,
+ &desc, conn);
+ if (status && !APR_STATUS_IS_NOTFOUND(status))
+ return status;
+
+ /* Now put it back in with the correct read/write values. */
+ desc.reqevents = APR_POLLHUP | APR_POLLERR;
+ if (conn->requests) {
+ /* If there are any outstanding events, then we want to read. */
+ /* ### not true. we only want to read IF we have sent some data */
+ desc.reqevents |= APR_POLLIN;
+
+ /* If the connection has unwritten data, or there are any requests
+ * that still have buckets to write out, then we want to write.
+ */
+ if (conn->vec_len)
+ desc.reqevents |= APR_POLLOUT;
+ else {
+ serf_request_t *request = conn->requests;
+
+ if ((conn->probable_keepalive_limit &&
+ conn->completed_requests > conn->probable_keepalive_limit) ||
+ (conn->max_outstanding_requests &&
+ conn->completed_requests - conn->completed_responses >=
+ conn->max_outstanding_requests)) {
+ /* we wouldn't try to write any way right now. */
+ }
+ else {
+ while (request != NULL && request->req_bkt == NULL &&
+ request->setup == NULL)
+ request = request->next;
+ if (request != NULL)
+ desc.reqevents |= APR_POLLOUT;
+ }
+ }
+ }
+
+ /* save our reqevents, so we can pass it in to remove later. */
+ conn->reqevents = desc.reqevents;
+
+ /* Note: even if we don't want to read/write this socket, we still
+ * want to poll it for hangups and errors.
+ */
+ return ctx->pollset_add(ctx->pollset_baton,
+ &desc, conn);
+}
+
+#ifdef SERF_DEBUG_BUCKET_USE
+
+/* Make sure all response buckets were drained. */
+static void check_buckets_drained(serf_connection_t *conn)
+{
+ serf_request_t *request = conn->requests;
+
+ for ( ; request ; request = request->next ) {
+ if (request->resp_bkt != NULL) {
+ /* ### crap. can't do this. this allocator may have un-drained
+ * ### REQUEST buckets.
+ */
+ /* serf_debug__entered_loop(request->resp_bkt->allocator); */
+ /* ### for now, pretend we closed the conn (resets the tracking) */
+ serf_debug__closed_conn(request->resp_bkt->allocator);
+ }
+ }
+}
+
+#endif
+
+/* Create and connect sockets for any connections which don't have them
+ * yet. This is the core of our lazy-connect behavior.
+ */
+static apr_status_t open_connections(serf_context_t *ctx)
+{
+ int i;
+
+ for (i = ctx->conns->nelts; i--; ) {
+ serf_connection_t *conn = GET_CONN(ctx, i);
+ apr_status_t status;
+ apr_socket_t *skt;
+ apr_sockaddr_t *serv_addr;
+
+ conn->seen_in_pollset = 0;
+
+ if (conn->skt != NULL) {
+#ifdef SERF_DEBUG_BUCKET_USE
+ check_buckets_drained(conn);
+#endif
+ continue;
+ }
+
+ /* Delay opening until we have something to deliver! */
+ if (conn->requests == NULL) {
+ continue;
+ }
+
+ apr_pool_clear(conn->skt_pool);
+ apr_pool_cleanup_register(conn->skt_pool, conn, clean_skt, clean_skt);
+
+ /* Do we have to connect to a proxy server? */
+ if (ctx->proxy_address)
+ serv_addr = ctx->proxy_address;
+ else
+ serv_addr = conn->address;
+
+ if ((status = apr_socket_create(&skt, serv_addr->family,
+ SOCK_STREAM,
+#if APR_MAJOR_VERSION > 0
+ APR_PROTO_TCP,
+#endif
+ conn->skt_pool)) != APR_SUCCESS)
+ return status;
+
+ /* Set the socket to be non-blocking */
+ if ((status = apr_socket_timeout_set(skt, 0)) != APR_SUCCESS)
+ return status;
+
+ /* Disable Nagle's algorithm */
+ if ((status = apr_socket_opt_set(skt,
+ APR_TCP_NODELAY, 0)) != APR_SUCCESS)
+ return status;
+
+ /* Configured. Store it into the connection now. */
+ conn->skt = skt;
+
+ /* Now that the socket is set up, let's connect it. This should
+ * return immediately.
+ */
+ if ((status = apr_socket_connect(skt,
+ serv_addr)) != APR_SUCCESS) {
+ if (!APR_STATUS_IS_EINPROGRESS(status))
+ return status;
+ }
+
+ /* Flag our pollset as dirty now that we have a new socket. */
+ conn->dirty_conn = 1;
+ ctx->dirty_pollset = 1;
+ }
+
+ return APR_SUCCESS;
+}
+
+static apr_status_t no_more_writes(serf_connection_t *conn,
+ serf_request_t *request)
+{
+ /* Note that we should hold new requests until we open our new socket. */
+ conn->closing = 1;
+
+ /* We can take the *next* request in our list and assume it hasn't
+ * been written yet and 'save' it for the new socket.
+ */
+ conn->hold_requests = request->next;
+ conn->hold_requests_tail = conn->requests_tail;
+ request->next = NULL;
+ conn->requests_tail = request;
+
+ /* Clear our iovec. */
+ conn->vec_len = 0;
+
+ /* Update the pollset to know we don't want to write on this socket any
+ * more.
+ */
+ conn->dirty_conn = 1;
+ conn->ctx->dirty_pollset = 1;
+ return APR_SUCCESS;
+}
+
+/* Read the 'Connection' header from the response. Return SERF_ERROR_CLOSING if
+ * the header contains value 'close' indicating the server is closing the
+ * connection right after this response.
+ * Otherwise returns APR_SUCCESS.
+ */
+static apr_status_t is_conn_closing(serf_bucket_t *response)
+{
+ serf_bucket_t *hdrs;
+ const char *val;
+
+ hdrs = serf_bucket_response_get_headers(response);
+ val = serf_bucket_headers_get(hdrs, "Connection");
+ if (val && strcasecmp("close", val) == 0)
+ {
+ return SERF_ERROR_CLOSING;
+ }
+
+ return APR_SUCCESS;
+}
+
+static void link_requests(serf_request_t **list, serf_request_t **tail,
+ serf_request_t *request)
+{
+ if (*list == NULL) {
+ *list = request;
+ *tail = request;
+ }
+ else {
+ (*tail)->next = request;
+ *tail = request;
+ }
+}
+
+static apr_status_t cancel_request(serf_request_t *request,
+ serf_request_t **list,
+ int notify_request)
+{
+ /* If we haven't run setup, then we won't have a handler to call. */
+ if (request->handler && notify_request) {
+ /* We actually don't care what the handler returns.
+ * We have bigger matters at hand.
+ */
+ (*request->handler)(request, NULL, request->handler_baton,
+ request->respool);
+ }
+
+ if (*list == request) {
+ *list = request->next;
+ }
+ else {
+ serf_request_t *scan = *list;
+
+ while (scan->next && scan->next != request)
+ scan = scan->next;
+
+ if (scan->next) {
+ scan->next = scan->next->next;
+ }
+ }
+
+ if (request->resp_bkt) {
+ serf_debug__closed_conn(request->resp_bkt->allocator);
+ serf_bucket_destroy(request->resp_bkt);
+ }
+ if (request->req_bkt) {
+ serf_debug__closed_conn(request->req_bkt->allocator);
+ serf_bucket_destroy(request->req_bkt);
+ }
+
+ if (request->respool) {
+ apr_pool_destroy(request->respool);
+ }
+
+ serf_bucket_mem_free(request->conn->allocator, request);
+
+ return APR_SUCCESS;
+}
+
+static apr_status_t remove_connection(serf_context_t *ctx,
+ serf_connection_t *conn)
+{
+ apr_pollfd_t desc = { 0 };
+
+ desc.desc_type = APR_POLL_SOCKET;
+ desc.desc.s = conn->skt;
+ desc.reqevents = conn->reqevents;
+
+ return ctx->pollset_rm(ctx->pollset_baton,
+ &desc, conn);
+}
+
+static apr_status_t reset_connection(serf_connection_t *conn,
+ int requeue_requests)
+{
+ serf_context_t *ctx = conn->ctx;
+ apr_status_t status;
+ serf_request_t *old_reqs, *held_reqs, *held_reqs_tail;
+
+ conn->probable_keepalive_limit = conn->completed_responses;
+ conn->completed_requests = 0;
+ conn->completed_responses = 0;
+
+ old_reqs = conn->requests;
+ held_reqs = conn->hold_requests;
+ held_reqs_tail = conn->hold_requests_tail;
+
+ if (conn->closing) {
+ conn->hold_requests = NULL;
+ conn->hold_requests_tail = NULL;
+ conn->closing = 0;
+ }
+
+ conn->requests = NULL;
+ conn->requests_tail = NULL;
+
+ while (old_reqs) {
+ /* If we haven't started to write the connection, bring it over
+ * unchanged to our new socket. Otherwise, call the cancel function.
+ */
+ if (requeue_requests && old_reqs->setup) {
+ serf_request_t *req = old_reqs;
+ old_reqs = old_reqs->next;
+ req->next = NULL;
+ link_requests(&conn->requests, &conn->requests_tail, req);
+ }
+ else {
+ cancel_request(old_reqs, &old_reqs, requeue_requests);
+ }
+ }
+
+ if (conn->requests_tail) {
+ conn->requests_tail->next = held_reqs;
+ }
+ else {
+ conn->requests = held_reqs;
+ }
+ if (held_reqs_tail) {
+ conn->requests_tail = held_reqs_tail;
+ }
+
+ if (conn->skt != NULL) {
+ remove_connection(ctx, conn);
+ status = apr_socket_close(conn->skt);
+ if (conn->closed != NULL) {
+ (*conn->closed)(conn, conn->closed_baton, status,
+ conn->pool);
+ }
+ conn->skt = NULL;
+ }
+
+ if (conn->stream != NULL) {
+ serf_bucket_destroy(conn->stream);
+ conn->stream = NULL;
+ }
+
+ /* Don't try to resume any writes */
+ conn->vec_len = 0;
+
+ conn->dirty_conn = 1;
+ conn->ctx->dirty_pollset = 1;
+
+ /* Let our context know that we've 'reset' the socket already. */
+ conn->seen_in_pollset |= APR_POLLHUP;
+
+ /* Found the connection. Closed it. All done. */
+ return APR_SUCCESS;
+}
+
+/**
+ * Callback function (implements serf_progress_t). Takes a number of bytes
+ * read @a read and bytes written @a written, adds those to the total for this
+ * context and notifies an interested party (if any).
+ */
+static void serf_context_progress_delta(
+ void *progress_baton,
+ apr_off_t read,
+ apr_off_t written)
+{
+ serf_context_t *ctx = progress_baton;
+
+ ctx->progress_read += read;
+ ctx->progress_written += written;
+
+ if (ctx->progress_func)
+ ctx->progress_func(ctx->progress_baton,
+ ctx->progress_read,
+ ctx->progress_written);
+}
+
+static apr_status_t socket_writev(serf_connection_t *conn)
+{
+ apr_size_t written;
+ apr_status_t status;
+
+ status = apr_socket_sendv(conn->skt, conn->vec,
+ conn->vec_len, &written);
+
+ /* did we write everything? */
+ if (written) {
+ apr_size_t len = 0;
+ int i;
+
+ for (i = 0; i < conn->vec_len; i++) {
+ len += conn->vec[i].iov_len;
+ if (written < len) {
+ if (i) {
+ memmove(conn->vec, &conn->vec[i],
+ sizeof(struct iovec) * (conn->vec_len - i));
+ conn->vec_len -= i;
+ }
+ conn->vec[0].iov_base += conn->vec[0].iov_len - (len - written);
+ conn->vec[0].iov_len = len - written;
+ break;
+ }
+ }
+ if (len == written) {
+ conn->vec_len = 0;
+ }
+
+ /* Log progress information */
+ serf_context_progress_delta(conn->ctx, 0, written);
+ }
+
+ return status;
+}
+
+/* write data out to the connection */
+static apr_status_t write_to_connection(serf_connection_t *conn)
+{
+ serf_request_t *request = conn->requests;
+
+ if (conn->probable_keepalive_limit &&
+ conn->completed_requests > conn->probable_keepalive_limit) {
+ /* backoff for now. */
+ return APR_SUCCESS;
+ }
+
+ /* Find a request that has data which needs to be delivered. */
+ while (request != NULL &&
+ request->req_bkt == NULL && request->setup == NULL)
+ request = request->next;
+
+ /* assert: request != NULL || conn->vec_len */
+
+ /* Keep reading and sending until we run out of stuff to read, or
+ * writing would block.
+ */
+ while (1) {
+ int stop_reading = 0;
+ apr_status_t status;
+ apr_status_t read_status;
+
+ if (conn->max_outstanding_requests &&
+ conn->completed_requests -
+ conn->completed_responses >= conn->max_outstanding_requests) {
+ /* backoff for now. */
+ return APR_SUCCESS;
+ }
+
+ /* If we have unwritten data, then write what we can. */
+ while (conn->vec_len) {
+ status = socket_writev(conn);
+
+ /* If the write would have blocked, then we're done. Don't try
+ * to write anything else to the socket.
+ */
+ if (APR_STATUS_IS_EAGAIN(status))
+ return APR_SUCCESS;
+ if (APR_STATUS_IS_EPIPE(status))
+ return no_more_writes(conn, request);
+ if (status)
+ return status;
+ }
+ /* ### can we have a short write, yet no EAGAIN? a short write
+ ### would imply unwritten_len > 0 ... */
+ /* assert: unwritten_len == 0. */
+
+ /* We may need to move forward to a request which has something
+ * to write.
+ */
+ while (request != NULL &&
+ request->req_bkt == NULL && request->setup == NULL)
+ request = request->next;
+
+ if (request == NULL) {
+ /* No more requests (with data) are registered with the
+ * connection. Let's update the pollset so that we don't
+ * try to write to this socket again.
+ */
+ conn->dirty_conn = 1;
+ conn->ctx->dirty_pollset = 1;
+ return APR_SUCCESS;
+ }
+
+ /* If the connection does not have an associated bucket, then
+ * call the setup callback to get one.
+ */
+ if (conn->stream == NULL) {
+ conn->stream = (*conn->setup)(conn->skt,
+ conn->setup_baton,
+ conn->pool);
+ }
+
+ if (request->req_bkt == NULL) {
+ /* Now that we are about to serve the request, allocate a pool. */
+ apr_pool_create(&request->respool, conn->pool);
+ request->allocator = serf_bucket_allocator_create(request->respool,
+ NULL, NULL);
+ apr_pool_cleanup_register(request->respool, request,
+ clean_resp, clean_resp);
+
+ /* Fill in the rest of the values for the request. */
+ read_status = request->setup(request, request->setup_baton,
+ &request->req_bkt,
+ &request->acceptor,
+ &request->acceptor_baton,
+ &request->handler,
+ &request->handler_baton,
+ request->respool);
+
+ if (read_status) {
+ /* Something bad happened. Propagate any errors. */
+ return read_status;
+ }
+
+ request->setup = NULL;
+ }
+
+ /* ### optimize at some point by using read_for_sendfile */
+ read_status = serf_bucket_read_iovec(request->req_bkt,
+ SERF_READ_ALL_AVAIL,
+ IOV_MAX,
+ conn->vec,
+ &conn->vec_len);
+
+ if (APR_STATUS_IS_EAGAIN(read_status)) {
+ /* We read some stuff, but should not try to read again. */
+ stop_reading = 1;
+
+ /* ### we should avoid looking for writability for a while so
+ ### that (hopefully) something will appear in the bucket so
+ ### we can actually write something. otherwise, we could
+ ### end up in a CPU spin: socket wants something, but we
+ ### don't have anything (and keep returning EAGAIN)
+ */
+ }
+ else if (read_status && !APR_STATUS_IS_EOF(read_status)) {
+ /* Something bad happened. Propagate any errors. */
+ return read_status;
+ }
+
+ /* If we got some data, then deliver it. */
+ /* ### what to do if we got no data?? is that a problem? */
+ if (conn->vec_len > 0) {
+ status = socket_writev(conn);
+
+ /* If we can't write any more, or an error occurred, then
+ * we're done here.
+ */
+ if (APR_STATUS_IS_EAGAIN(status))
+ return APR_SUCCESS;
+ if (APR_STATUS_IS_EPIPE(status))
+ return no_more_writes(conn, request);
+ if (APR_STATUS_IS_ECONNRESET(status)) {
+ return no_more_writes(conn, request);
+ }
+ if (status)
+ return status;
+ }
+
+ if (APR_STATUS_IS_EOF(read_status) &&
+ conn->vec_len == 0) {
+ /* If we hit the end of the request bucket and all of its data has
+ * been written, then clear it out to signify that we're done
+ * sending the request. On the next iteration through this loop:
+ * - if there are remaining bytes they will be written, and as the
+ * request bucket will be completely read it will be destroyed then.
+ * - we'll see if there are other requests that need to be sent
+ * ("pipelining").
+ */
+ serf_bucket_destroy(request->req_bkt);
+ request->req_bkt = NULL;
+
+ conn->completed_requests++;
+
+ if (conn->probable_keepalive_limit &&
+ conn->completed_requests > conn->probable_keepalive_limit) {
+ /* backoff for now. */
+ stop_reading = 1;
+ }
+ }
+
+ if (stop_reading) {
+ return APR_SUCCESS;
+ }
+ }
+ /* NOTREACHED */
+}
+
+/* read data from the connection */
+static apr_status_t read_from_connection(serf_connection_t *conn)
+{
+ apr_status_t status;
+ apr_pool_t *tmppool;
+ int close_connection = FALSE;
+
+ /* Whatever is coming in on the socket corresponds to the first request
+ * on our chain.
+ */
+ serf_request_t *request = conn->requests;
+
+ /* assert: request != NULL */
+
+ if ((status = apr_pool_create(&tmppool, conn->pool)) != APR_SUCCESS)
+ goto error;
+
+ /* Invoke response handlers until we have no more work. */
+ while (1) {
+ apr_pool_clear(tmppool);
+
+ /* If the connection does not have an associated bucket, then
+ * call the setup callback to get one.
+ */
+ if (conn->stream == NULL) {
+ conn->stream = (*conn->setup)(conn->skt,
+ conn->setup_baton,
+ conn->pool);
+ }
+
+ /* We are reading a response for a request we haven't
+ * written yet!
+ *
+ * This shouldn't normally happen EXCEPT:
+ *
+ * 1) when the other end has closed the socket and we're
+ * pending an EOF return.
+ * 2) Doing the initial SSL handshake - we'll get EAGAIN
+ * as the SSL buckets will hide the handshake from us
+ * but not return any data.
+ *
+ * In these cases, we should not receive any actual user data.
+ *
+ * If we see an EOF (due to an expired timeout), we'll reset the
+ * connection and open a new one.
+ */
+ if (request->req_bkt || request->setup) {
+ const char *data;
+ apr_size_t len;
+
+ status = serf_bucket_read(conn->stream, SERF_READ_ALL_AVAIL,
+ &data, &len);
+
+ if (!status && len) {
+ status = APR_EGENERAL;
+ }
+ else if (APR_STATUS_IS_EOF(status)) {
+ reset_connection(conn, 1);
+ status = APR_SUCCESS;
+ }
+ else if (APR_STATUS_IS_EAGAIN(status)) {
+ status = APR_SUCCESS;
+ }
+
+ goto error;
+ }
+
+ /* If the request doesn't have a response bucket, then call the
+ * acceptor to get one created.
+ */
+ if (request->resp_bkt == NULL) {
+ request->resp_bkt = (*request->acceptor)(request, conn->stream,
+ request->acceptor_baton,
+ tmppool);
+ apr_pool_clear(tmppool);
+ }
+
+ status = (*request->handler)(request,
+ request->resp_bkt,
+ request->handler_baton,
+ tmppool);
+
+ /* Some systems will not generate a HUP poll event so we have to
+ * handle the ECONNRESET issue here.
+ */
+ if (APR_STATUS_IS_ECONNRESET(status) ||
+ status == SERF_ERROR_REQUEST_LOST) {
+ reset_connection(conn, 1);
+ status = APR_SUCCESS;
+ goto error;
+ }
+
+ /* If our response handler says it can't do anything more, we now
+ * treat that as a success.
+ */
+ if (APR_STATUS_IS_EAGAIN(status)) {
+ status = APR_SUCCESS;
+ goto error;
+ }
+
+ /* If we received APR_SUCCESS, run this loop again. */
+ if (!status) {
+ continue;
+ }
+
+ close_connection = is_conn_closing(request->resp_bkt);
+
+ if (!APR_STATUS_IS_EOF(status) &&
+ close_connection != SERF_ERROR_CLOSING) {
+ /* Whether success, or an error, there is no more to do unless
+ * this request has been completed.
+ */
+ goto error;
+ }
+
+ /* The request has been fully-delivered, and the response has
+ * been fully-read. Remove it from our queue and loop to read
+ * another response.
+ */
+ conn->requests = request->next;
+
+ /* The bucket is no longer needed, nor is the request's pool. */
+ serf_bucket_destroy(request->resp_bkt);
+ if (request->req_bkt) {
+ serf_bucket_destroy(request->req_bkt);
+ }
+
+ serf_debug__bucket_alloc_check(request->allocator);
+ apr_pool_destroy(request->respool);
+ serf_bucket_mem_free(conn->allocator, request);
+
+ request = conn->requests;
+
+ /* If we're truly empty, update our tail. */
+ if (request == NULL) {
+ conn->requests_tail = NULL;
+ }
+
+ conn->completed_responses++;
+
+ /* This means that we're being advised that the connection is done. */
+ if (close_connection == SERF_ERROR_CLOSING) {
+ reset_connection(conn, 1);
+ if (APR_STATUS_IS_EOF(status))
+ status = APR_SUCCESS;
+ goto error;
+ }
+
+ /* The server is suddenly deciding to serve more responses than we've
+ * seen before.
+ *
+ * Let our requests go.
+ */
+ if (conn->probable_keepalive_limit &&
+ conn->completed_responses > conn->probable_keepalive_limit) {
+ conn->probable_keepalive_limit = 0;
+ }
+
+ /* If we just ran out of requests or have unwritten requests, then
+ * update the pollset. We don't want to read from this socket any
+ * more. We are definitely done with this loop, too.
+ */
+ if (request == NULL || request->setup) {
+ conn->dirty_conn = 1;
+ conn->ctx->dirty_pollset = 1;
+ status = APR_SUCCESS;
+ goto error;
+ }
+ }
+
+ error:
+ apr_pool_destroy(tmppool);
+ return status;
+}
+
+/* process all events on the connection */
+static apr_status_t process_connection(serf_connection_t *conn,
+ apr_int16_t events)
+{
+ apr_status_t status;
+
+ /* POLLHUP/ERR should come after POLLIN so if there's an error message or
+ * the like sitting on the connection, we give the app a chance to read
+ * it before we trigger a reset condition.
+ */
+ if ((events & APR_POLLIN) != 0) {
+ if ((status = read_from_connection(conn)) != APR_SUCCESS)
+ return status;
+
+ /* If we decided to reset our connection, return now as we don't
+ * want to write.
+ */
+ if ((conn->seen_in_pollset & APR_POLLHUP) != 0) {
+ return APR_SUCCESS;
+ }
+ }
+ if ((events & APR_POLLHUP) != 0) {
+ return APR_ECONNRESET;
+ }
+ if ((events & APR_POLLERR) != 0) {
+ /* We might be talking to a buggy HTTP server that doesn't
+ * do lingering-close. (httpd < 2.1.8 does this.)
+ *
+ * See:
+ *
+ * http://issues.apache.org/bugzilla/show_bug.cgi?id=35292
+ */
+ if (!conn->probable_keepalive_limit) {
+ return reset_connection(conn, 1);
+ }
+ return APR_EGENERAL;
+ }
+ if ((events & APR_POLLOUT) != 0) {
+ if ((status = write_to_connection(conn)) != APR_SUCCESS)
+ return status;
+ }
+ return APR_SUCCESS;
+}
+
+/* Check for dirty connections and update their pollsets accordingly. */
+static apr_status_t check_dirty_pollsets(serf_context_t *ctx)
+{
+ int i;
+
+ /* if we're not dirty, return now. */
+ if (!ctx->dirty_pollset) {
+ return APR_SUCCESS;
+ }
+
+ for (i = ctx->conns->nelts; i--; ) {
+ serf_connection_t *conn = GET_CONN(ctx, i);
+ apr_status_t status;
+
+ /* if this connection isn't dirty, skip it. */
+ if (!conn->dirty_conn) {
+ continue;
+ }
+
+ /* reset this connection's flag before we update. */
+ conn->dirty_conn = 0;
+
+ if ((status = update_pollset(conn)) != APR_SUCCESS)
+ return status;
+ }
+
+ /* reset our context flag now */
+ ctx->dirty_pollset = 0;
+
+ return APR_SUCCESS;
+}
+
+
+static apr_status_t pollset_add(void *user_baton,
+ apr_pollfd_t *pfd,
+ void *serf_baton)
+{
+ serf_pollset_t *s = (serf_pollset_t*)user_baton;
+ pfd->client_data = serf_baton;
+ return apr_pollset_add(s->pollset, pfd);
+}
+
+static apr_status_t pollset_rm(void *user_baton,
+ apr_pollfd_t *pfd,
+ void *serf_baton)
+{
+ serf_pollset_t *s = (serf_pollset_t*)user_baton;
+ pfd->client_data = serf_baton;
+ return apr_pollset_remove(s->pollset, pfd);
+}
+
+
+SERF_DECLARE(void) serf_config_proxy(serf_context_t *ctx,
+ apr_sockaddr_t *address)
+{
+ ctx->proxy_address = address;
+}
+
+SERF_DECLARE(serf_context_t *) serf_context_create_ex(void *user_baton,
+ serf_socket_add_t addf,
+ serf_socket_remove_t rmf,
+ apr_pool_t *pool)
+{
+ serf_context_t *ctx = apr_pcalloc(pool, sizeof(*ctx));
+
+ ctx->pool = pool;
+
+ if (user_baton != NULL) {
+ ctx->pollset_baton = user_baton;
+ ctx->pollset_add = addf;
+ ctx->pollset_rm = rmf;
+ }
+ else {
+ /* build the pollset with a (default) number of connections */
+ serf_pollset_t *ps = apr_pcalloc(pool, sizeof(*ps));
+ (void) apr_pollset_create(&ps->pollset, MAX_CONN, pool, 0);
+ ctx->pollset_baton = ps;
+ ctx->pollset_add = pollset_add;
+ ctx->pollset_rm = pollset_rm;
+ }
+
+ /* default to a single connection since that is the typical case */
+ ctx->conns = apr_array_make(pool, 1, sizeof(serf_connection_t *));
+
+ /* Initialize progress status */
+ ctx->progress_read = 0;
+ ctx->progress_written = 0;
+
+ return ctx;
+}
+
+SERF_DECLARE(serf_context_t *) serf_context_create(apr_pool_t *pool)
+{
+ return serf_context_create_ex(NULL, NULL, NULL, pool);
+}
+
+SERF_DECLARE(apr_status_t) serf_context_prerun(serf_context_t *ctx)
+{
+ apr_status_t status = APR_SUCCESS;
+ if ((status = open_connections(ctx)) != APR_SUCCESS)
+ return status;
+
+ if ((status = check_dirty_pollsets(ctx)) != APR_SUCCESS)
+ return status;
+ return status;
+}
+
+SERF_DECLARE(apr_status_t) serf_event_trigger(serf_context_t *s,
+ void *serf_baton,
+ const apr_pollfd_t *desc)
+{
+ apr_status_t status = APR_SUCCESS;
+
+ serf_connection_t *conn = serf_baton;
+
+ /* apr_pollset_poll() can return a conn multiple times... */
+ if ((conn->seen_in_pollset & desc->rtnevents) != 0 ||
+ (conn->seen_in_pollset & APR_POLLHUP) != 0) {
+ return APR_SUCCESS;
+ }
+
+ conn->seen_in_pollset |= desc->rtnevents;
+
+ if ((status = process_connection(conn,
+ desc->rtnevents)) != APR_SUCCESS) {
+ return status;
+ }
+
+ return status;
+}
+
+SERF_DECLARE(apr_status_t) serf_context_run(serf_context_t *ctx,
+ apr_short_interval_time_t duration,
+ apr_pool_t *pool)
+{
+ apr_status_t status;
+ apr_int32_t num;
+ const apr_pollfd_t *desc;
+ serf_pollset_t *ps = (serf_pollset_t*)ctx->pollset_baton;
+
+ if ((status = serf_context_prerun(ctx)) != APR_SUCCESS) {
+ return status;
+ }
+
+ if ((status = apr_pollset_poll(ps->pollset, duration, &num,
+ &desc)) != APR_SUCCESS) {
+ /* ### do we still need to dispatch stuff here?
+ ### look at the potential return codes. map to our defined
+ ### return values? ...
+ */
+ return status;
+ }
+
+ while (num--) {
+ serf_connection_t *conn = desc->client_data;
+
+ status = serf_event_trigger(ctx, conn, desc);
+ if (status) {
+ return status;
+ }
+
+ desc++;
+ }
+
+ return APR_SUCCESS;
+}
+
+SERF_DECLARE(void) serf_context_set_progress_cb(
+ serf_context_t *ctx,
+ const serf_progress_t progress_func,
+ void *progress_baton)
+{
+ ctx->progress_func = progress_func;
+ ctx->progress_baton = progress_baton;
+}
+
+SERF_DECLARE(serf_connection_t *) serf_connection_create(
+ serf_context_t *ctx,
+ apr_sockaddr_t *address,
+ serf_connection_setup_t setup,
+ void *setup_baton,
+ serf_connection_closed_t closed,
+ void *closed_baton,
+ apr_pool_t *pool)
+{
+ serf_connection_t *conn = apr_pcalloc(pool, sizeof(*conn));
+
+ conn->ctx = ctx;
+ conn->address = address;
+ conn->setup = setup;
+ conn->setup_baton = setup_baton;
+ conn->closed = closed;
+ conn->closed_baton = closed_baton;
+ conn->pool = pool;
+ conn->allocator = serf_bucket_allocator_create(pool, NULL, NULL);
+ conn->stream = NULL;
+
+ /* Create a subpool for our connection. */
+ apr_pool_create(&conn->skt_pool, conn->pool);
+
+ /* register a cleanup */
+ apr_pool_cleanup_register(conn->pool, conn, clean_conn, apr_pool_cleanup_null);
+
+ /* Add the connection to the context. */
+ *(serf_connection_t **)apr_array_push(ctx->conns) = conn;
+
+ return conn;
+}
+
+SERF_DECLARE(apr_status_t) serf_connection_create2(
+ serf_connection_t **conn,
+ serf_context_t *ctx,
+ apr_uri_t host_info,
+ serf_connection_setup_t setup,
+ void *setup_baton,
+ serf_connection_closed_t closed,
+ void *closed_baton,
+ apr_pool_t *pool)
+{
+ apr_status_t status;
+ serf_connection_t *c;
+ apr_sockaddr_t *host_address;
+
+ /* Parse the url, store the address of the server. */
+ status = apr_sockaddr_info_get(&host_address,
+ host_info.hostname,
+ APR_UNSPEC, host_info.port, 0, pool);
+ if (status)
+ return status;
+
+ c = serf_connection_create(ctx, host_address, setup, setup_baton,
+ closed, closed_baton, pool);
+
+ /* We're not interested in the path following the hostname. */
+ c->host_url = apr_uri_unparse(c->pool,
+ &host_info,
+ APR_URI_UNP_OMITPATHINFO);
+ c->host_info = host_info;
+
+ *conn = c;
+
+ return status;
+}
+
+SERF_DECLARE(apr_status_t) serf_connection_reset(
+ serf_connection_t *conn)
+{
+ return reset_connection(conn, 0);
+}
+
+SERF_DECLARE(apr_status_t) serf_connection_close(
+ serf_connection_t *conn)
+{
+ int i;
+ serf_context_t *ctx = conn->ctx;
+ apr_status_t status;
+
+ for (i = ctx->conns->nelts; i--; ) {
+ serf_connection_t *conn_seq = GET_CONN(ctx, i);
+
+ if (conn_seq == conn) {
+ while (conn->requests) {
+ serf_request_cancel(conn->requests);
+ }
+ if (conn->skt != NULL) {
+ remove_connection(ctx, conn);
+ status = apr_socket_close(conn->skt);
+ if (conn->closed != NULL) {
+ (*conn->closed)(conn, conn->closed_baton, status,
+ conn->pool);
+ }
+ conn->skt = NULL;
+ }
+ if (conn->stream != NULL) {
+ serf_bucket_destroy(conn->stream);
+ conn->stream = NULL;
+ }
+
+ /* Remove the connection from the context. We don't want to
+ * deal with it any more.
+ */
+ if (i < ctx->conns->nelts - 1) {
+ /* move later connections over this one. */
+ memmove(
+ &GET_CONN(ctx, i),
+ &GET_CONN(ctx, i + 1),
+ (ctx->conns->nelts - i - 1) * sizeof(serf_connection_t *));
+ }
+ --ctx->conns->nelts;
+
+ /* Found the connection. Closed it. All done. */
+ return APR_SUCCESS;
+ }
+ }
+
+ /* We didn't find the specified connection. */
+ /* ### doc talks about this w.r.t poll structures. use something else? */
+ return APR_NOTFOUND;
+}
+
+SERF_DECLARE(void)
+serf_connection_set_max_outstanding_requests(serf_connection_t *conn,
+ unsigned int max_requests)
+{
+ conn->max_outstanding_requests = max_requests;
+}
+
+SERF_DECLARE(serf_request_t *) serf_connection_request_create(
+ serf_connection_t *conn,
+ serf_request_setup_t setup,
+ void *setup_baton)
+{
+ serf_request_t *request;
+
+ request = serf_bucket_mem_alloc(conn->allocator, sizeof(*request));
+ request->conn = conn;
+ request->setup = setup;
+ request->setup_baton = setup_baton;
+ request->handler = NULL;
+ request->respool = NULL;
+ request->req_bkt = NULL;
+ request->resp_bkt = NULL;
+ request->next = NULL;
+
+ /* Link the request to the end of the request chain. */
+ if (conn->closing) {
+ link_requests(&conn->hold_requests, &conn->hold_requests_tail, request);
+ }
+ else {
+ link_requests(&conn->requests, &conn->requests_tail, request);
+
+ /* Ensure our pollset becomes writable in context run */
+ conn->ctx->dirty_pollset = 1;
+ conn->dirty_conn = 1;
+ }
+
+ return request;
+}
+
+SERF_DECLARE(serf_request_t *) serf_connection_priority_request_create(
+ serf_connection_t *conn,
+ serf_request_setup_t setup,
+ void *setup_baton)
+{
+ serf_request_t *request;
+ serf_request_t *iter, *prev;
+
+ request = serf_bucket_mem_alloc(conn->allocator, sizeof(*request));
+ request->conn = conn;
+ request->setup = setup;
+ request->setup_baton = setup_baton;
+ request->handler = NULL;
+ request->respool = NULL;
+ request->req_bkt = NULL;
+ request->resp_bkt = NULL;
+ request->next = NULL;
+
+ /* Link the new request after the last written request, but before all
+ upcoming requests. */
+ if (conn->closing) {
+ iter = conn->hold_requests;
+ }
+ else {
+ iter = conn->requests;
+ }
+ prev = NULL;
+
+ /* Find a request that has data which needs to be delivered. */
+ while (iter != NULL && iter->req_bkt == NULL && iter->setup == NULL) {
+ prev = iter;
+ iter = iter->next;
+ }
+
+ if (prev) {
+ request->next = iter;
+ prev->next = request;
+ } else {
+ request->next = iter;
+ if (conn->closing) {
+ conn->hold_requests = request;
+ }
+ else {
+ conn->requests = request;
+ }
+ }
+
+ if (! conn->closing) {
+ /* Ensure our pollset becomes writable in context run */
+ conn->ctx->dirty_pollset = 1;
+ conn->dirty_conn = 1;
+ }
+
+ return request;
+}
+
+SERF_DECLARE(apr_status_t) serf_request_cancel(serf_request_t *request)
+{
+ return cancel_request(request, &request->conn->requests, 0);
+}
+
+SERF_DECLARE(apr_pool_t *) serf_request_get_pool(const serf_request_t *request)
+{
+ return request->respool;
+}
+
+SERF_DECLARE(serf_bucket_alloc_t *) serf_request_get_alloc(
+ const serf_request_t *request)
+{
+ return request->allocator;
+}
+
+SERF_DECLARE(serf_connection_t *) serf_request_get_conn(
+ const serf_request_t *request)
+{
+ return request->conn;
+}
+
+SERF_DECLARE(void) serf_request_set_handler(
+ serf_request_t *request,
+ const serf_response_handler_t handler,
+ const void **handler_baton)
+{
+ request->handler = handler;
+ request->handler_baton = handler_baton;
+}
+
+SERF_DECLARE(serf_bucket_t *) serf_context_bucket_socket_create(
+ serf_context_t *ctx,
+ apr_socket_t *skt,
+ serf_bucket_alloc_t *allocator)
+{
+ serf_bucket_t *bucket = serf_bucket_socket_create(skt, allocator);
+
+ /* Use serf's default bytes read/written callback */
+ serf_bucket_socket_set_read_progress_cb(bucket,
+ serf_context_progress_delta,
+ ctx);
+
+ return bucket;
+}
+
+SERF_DECLARE(serf_bucket_t *) serf_request_bucket_request_create_for_host(
+ serf_request_t *request,
+ const char *method,
+ const char *uri,
+ serf_bucket_t *body,
+ serf_bucket_alloc_t *allocator, const char* host)
+{
+ serf_bucket_t *req_bkt, *hdrs_bkt;
+
+ req_bkt = serf_bucket_request_create(method, uri, body, allocator);
+ hdrs_bkt = serf_bucket_request_get_headers(req_bkt);
+
+ /* Proxy? */
+ if (request->conn->ctx->proxy_address && request->conn->host_url)
+ serf_bucket_request_set_root(req_bkt, request->conn->host_url);
+
+ if (host == NULL)
+ host = request->conn->host_info.hostname;
+ if (host)
+ serf_bucket_headers_setn(hdrs_bkt, "Host", host);
+
+ return req_bkt;
+}
+
+SERF_DECLARE(serf_bucket_t *) serf_request_bucket_request_create(
+ serf_request_t *request,
+ const char *method,
+ const char *uri,
+ serf_bucket_t *body,
+ serf_bucket_alloc_t *allocator)
+{
+ return serf_request_bucket_request_create_for_host(
+ request, method, uri, body, allocator, NULL);
+}
diff --git a/trunk/src/third_party/serf/instaweb_headers_buckets.c b/trunk/src/third_party/serf/instaweb_headers_buckets.c
new file mode 100644
index 0000000..be041a9
--- /dev/null
+++ b/trunk/src/third_party/serf/instaweb_headers_buckets.c
@@ -0,0 +1,431 @@
+/* Copyright 2004 Justin Erenkrantz and Greg Stein
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <stdlib.h>
+
+#include <apr_general.h> /* for strcasecmp() */
+
+#include "serf.h"
+#include "serf_bucket_util.h"
+
+
+typedef struct header_list {
+ const char *header;
+ const char *value;
+
+ apr_size_t header_size;
+ apr_size_t value_size;
+
+ int alloc_flags;
+#define ALLOC_HEADER 0x0001 /* header lives in our allocator */
+#define ALLOC_VALUE 0x0002 /* value lives in our allocator */
+
+ struct header_list *next;
+} header_list_t;
+
+typedef struct {
+ header_list_t *list;
+
+ header_list_t *cur_read;
+ enum {
+ READ_START, /* haven't started reading yet */
+ READ_HEADER, /* reading cur_read->header */
+ READ_SEP, /* reading ": " */
+ READ_VALUE, /* reading cur_read->value */
+ READ_CRLF, /* reading "\r\n" */
+ READ_TERM, /* reading the final "\r\n" */
+ READ_DONE /* no more data to read */
+ } state;
+ apr_size_t amt_read; /* how much of the current state we've read */
+
+} headers_context_t;
+
+
+SERF_DECLARE(serf_bucket_t *) serf_bucket_headers_create(
+ serf_bucket_alloc_t *allocator)
+{
+ headers_context_t *ctx;
+
+ ctx = serf_bucket_mem_alloc(allocator, sizeof(*ctx));
+ ctx->list = NULL;
+ ctx->state = READ_START;
+
+ return serf_bucket_create(&serf_bucket_type_headers, allocator, ctx);
+}
+
+SERF_DECLARE(void) serf_bucket_headers_setx(
+ serf_bucket_t *bkt,
+ const char *header, apr_size_t header_size, int header_copy,
+ const char *value, apr_size_t value_size, int value_copy)
+{
+ headers_context_t *ctx = bkt->data;
+ header_list_t *found = ctx->list;
+ header_list_t *hdr;
+
+ /* Check to see if this header is already present. */
+ while (found) {
+ if (strncasecmp(found->header, header, header_size) == 0)
+ break;
+ found = found->next;
+ }
+
+ if (found) {
+
+ /* The header is already present. RFC 2616, section 4.2
+ indicates that we should append the new value, separated by
+ a comma. Reasoning: for headers whose values are known to
+ be comma-separated, that is clearly the correct behavior;
+ for others, the correct behavior is undefined anyway. */
+
+ /* The "+1" is for the comma; serf_bstrmemdup() will also add
+ one slot for the terminating '\0'. */
+ apr_size_t new_size = found->value_size + value_size + 1;
+ char *new_val = serf_bucket_mem_alloc(bkt->allocator, new_size);
+ memcpy(new_val, found->value, found->value_size);
+ new_val[found->value_size] = ',';
+ memcpy(new_val + found->value_size + 1, value, value_size);
+ new_val[new_size] = '\0';
+
+ // If the previous value was added, we must free it before overwriting
+ if (found->alloc_flags & ALLOC_VALUE) {
+ serf_bucket_mem_free(bkt->allocator, (void *)found->value);
+ }
+
+ found->value = new_val;
+ found->value_size = new_size;
+ found->alloc_flags |= ALLOC_VALUE;
+ return;
+ }
+
+ /* Else the header is not already present. Add it to the bucket. */
+
+#if 0
+ /* ### include this? */
+ if (ctx->cur_read) {
+ /* we started reading. can't change now. */
+ abort();
+ }
+#endif
+
+ hdr = serf_bucket_mem_alloc(bkt->allocator, sizeof(*hdr));
+ hdr->header_size = header_size;
+ hdr->value_size = value_size;
+ hdr->alloc_flags = 0;
+
+ if (header_copy) {
+ hdr->header = serf_bstrmemdup(bkt->allocator, header, header_size);
+ hdr->alloc_flags |= ALLOC_HEADER;
+ }
+ else {
+ hdr->header = header;
+ }
+
+ if (value_copy) {
+ hdr->value = serf_bstrmemdup(bkt->allocator, value, value_size);
+ hdr->alloc_flags |= ALLOC_VALUE;
+ }
+ else {
+ hdr->value = value;
+ }
+
+ hdr->next = ctx->list;
+ ctx->list = hdr;
+}
+
+SERF_DECLARE(void) serf_bucket_headers_set(
+ serf_bucket_t *headers_bucket,
+ const char *header,
+ const char *value)
+{
+ serf_bucket_headers_setx(headers_bucket,
+ header, strlen(header), 0,
+ value, strlen(value), 1);
+}
+
+SERF_DECLARE(void) serf_bucket_headers_setc(
+ serf_bucket_t *headers_bucket,
+ const char *header,
+ const char *value)
+{
+ serf_bucket_headers_setx(headers_bucket,
+ header, strlen(header), 1,
+ value, strlen(value), 1);
+}
+
+SERF_DECLARE(void) serf_bucket_headers_setn(
+ serf_bucket_t *headers_bucket,
+ const char *header,
+ const char *value)
+{
+ serf_bucket_headers_setx(headers_bucket,
+ header, strlen(header), 0,
+ value, strlen(value), 0);
+}
+
+SERF_DECLARE(const char *) serf_bucket_headers_get(
+ serf_bucket_t *headers_bucket,
+ const char *header)
+{
+ headers_context_t *ctx = headers_bucket->data;
+ header_list_t *scan = ctx->list;
+
+ while (scan) {
+ if (strcasecmp(scan->header, header) == 0)
+ return scan->value;
+ scan = scan->next;
+ }
+
+ return NULL;
+}
+
+SERF_DECLARE(void) serf_bucket_headers_do(
+ serf_bucket_t *headers_bucket,
+ serf_bucket_headers_do_callback_fn_t func,
+ void *baton)
+{
+ headers_context_t *ctx = headers_bucket->data;
+ header_list_t *scan = ctx->list;
+
+ while (scan) {
+ if (func(baton, scan->header, scan->value) != 0) {
+ break;
+ }
+ scan = scan->next;
+ }
+}
+
+static void serf_headers_destroy_and_data(serf_bucket_t *bucket)
+{
+ headers_context_t *ctx = bucket->data;
+ header_list_t *scan = ctx->list;
+
+ while (scan) {
+ header_list_t *next_hdr = scan->next;
+
+ if (scan->alloc_flags & ALLOC_HEADER)
+ serf_bucket_mem_free(bucket->allocator, (void *)scan->header);
+ if (scan->alloc_flags & ALLOC_VALUE)
+ serf_bucket_mem_free(bucket->allocator, (void *)scan->value);
+ serf_bucket_mem_free(bucket->allocator, scan);
+
+ scan = next_hdr;
+ }
+
+ serf_default_destroy_and_data(bucket);
+}
+
+static void select_value(
+ headers_context_t *ctx,
+ const char **value,
+ apr_size_t *len)
+{
+ const char *v;
+ apr_size_t l;
+
+ if (ctx->state == READ_START) {
+ if (ctx->list == NULL) {
+ /* No headers. Move straight to the TERM state. */
+ ctx->state = READ_TERM;
+ }
+ else {
+ ctx->state = READ_HEADER;
+ ctx->cur_read = ctx->list;
+ }
+ ctx->amt_read = 0;
+ }
+
+ switch (ctx->state) {
+ case READ_HEADER:
+ v = ctx->cur_read->header;
+ l = ctx->cur_read->header_size;
+ break;
+ case READ_SEP:
+ v = ": ";
+ l = 2;
+ break;
+ case READ_VALUE:
+ v = ctx->cur_read->value;
+ l = ctx->cur_read->value_size;
+ break;
+ case READ_CRLF:
+ case READ_TERM:
+ v = "\r\n";
+ l = 2;
+ break;
+ case READ_DONE:
+ *len = 0;
+ return;
+ default:
+ /* Not reachable */
+ return;
+ }
+
+ *value = v + ctx->amt_read;
+ *len = l - ctx->amt_read;
+}
+
+/* the current data chunk has been read/consumed. move our internal state. */
+static apr_status_t consume_chunk(headers_context_t *ctx)
+{
+ /* move to the next state, resetting the amount read. */
+ ++ctx->state;
+ ctx->amt_read = 0;
+
+ /* just sent the terminator and moved to DONE. signal completion. */
+ if (ctx->state == READ_DONE)
+ return APR_EOF;
+
+ /* end of this header. move to the next one. */
+ if (ctx->state == READ_TERM) {
+ ctx->cur_read = ctx->cur_read->next;
+ if (ctx->cur_read != NULL) {
+ /* We've got another head to send. Reset the read state. */
+ ctx->state = READ_HEADER;
+ }
+ /* else leave in READ_TERM */
+ }
+
+ /* there is more data which can be read immediately. */
+ return APR_SUCCESS;
+}
+
+static apr_status_t serf_headers_peek(serf_bucket_t *bucket,
+ const char **data,
+ apr_size_t *len)
+{
+ headers_context_t *ctx = bucket->data;
+
+ select_value(ctx, data, len);
+
+ /* already done or returning the CRLF terminator? return EOF */
+ if (ctx->state == READ_DONE || ctx->state == READ_TERM)
+ return APR_EOF;
+
+ return APR_SUCCESS;
+}
+
+static apr_status_t serf_headers_read(serf_bucket_t *bucket,
+ apr_size_t requested,
+ const char **data, apr_size_t *len)
+{
+ headers_context_t *ctx = bucket->data;
+ apr_size_t avail;
+
+ select_value(ctx, data, &avail);
+ if (ctx->state == READ_DONE)
+ return APR_EOF;
+
+ if (requested >= avail) {
+ /* return everything from this chunk */
+ *len = avail;
+
+ /* we consumed this chunk. advance the state. */
+ return consume_chunk(ctx);
+ }
+
+ /* return just the amount requested, and advance our pointer */
+ *len = requested;
+ ctx->amt_read += requested;
+
+ /* there is more that can be read immediately */
+ return APR_SUCCESS;
+}
+
+static apr_status_t serf_headers_readline(serf_bucket_t *bucket,
+ int acceptable, int *found,
+ const char **data, apr_size_t *len)
+{
+ headers_context_t *ctx = bucket->data;
+ apr_status_t status;
+
+ /* ### what behavior should we use here? APR_EGENERAL for now */
+ if ((acceptable & SERF_NEWLINE_CRLF) == 0)
+ return APR_EGENERAL;
+
+ /* get whatever is in this chunk */
+ select_value(ctx, data, len);
+ if (ctx->state == READ_DONE)
+ return APR_EOF;
+
+ /* we consumed this chunk. advance the state. */
+ status = consume_chunk(ctx);
+
+ /* the type of newline found is easy... */
+ *found = (ctx->state == READ_CRLF || ctx->state == READ_TERM)
+ ? SERF_NEWLINE_CRLF : SERF_NEWLINE_NONE;
+
+ return status;
+}
+
+static apr_status_t serf_headers_read_iovec(serf_bucket_t *bucket,
+ apr_size_t requested,
+ int vecs_size,
+ struct iovec *vecs,
+ int *vecs_used)
+{
+ apr_size_t avail = requested;
+ int i;
+
+ *vecs_used = 0;
+
+ for (i = 0; i < vecs_size; i++) {
+ const char *data;
+ apr_size_t len;
+ apr_status_t status;
+
+ /* Calling read() would not be a safe opt in the general case, but it
+ * is here for the header bucket as it only frees all of the header
+ * keys and values when the entire bucket goes away - not on a
+ * per-read() basis as is normally the case.
+ */
+ status = serf_headers_read(bucket, avail, &data, &len);
+
+ if (len) {
+ vecs[*vecs_used].iov_base = (char*)data;
+ vecs[*vecs_used].iov_len = len;
+
+ (*vecs_used)++;
+
+ if (avail != SERF_READ_ALL_AVAIL) {
+ avail -= len;
+
+ /* If we reach 0, then read()'s status will suffice. */
+ if (avail == 0) {
+ return status;
+ }
+ }
+ }
+
+ if (status) {
+ return status;
+ }
+ }
+
+ return APR_SUCCESS;
+}
+
+SERF_DECLARE_DATA const serf_bucket_type_t serf_bucket_type_headers = {
+ "HEADERS",
+ serf_headers_read,
+ serf_headers_readline,
+ serf_headers_read_iovec,
+ serf_default_read_for_sendfile,
+ serf_default_read_bucket,
+ serf_headers_peek,
+ serf_headers_destroy_and_data,
+ serf_default_snapshot,
+ serf_default_restore_snapshot,
+ serf_default_is_snapshot_set,
+};
diff --git a/trunk/src/third_party/serf/instaweb_response_buckets.c b/trunk/src/third_party/serf/instaweb_response_buckets.c
new file mode 100644
index 0000000..1c9a2c3
--- /dev/null
+++ b/trunk/src/third_party/serf/instaweb_response_buckets.c
@@ -0,0 +1,440 @@
+/* Copyright 2002-2004 Justin Erenkrantz and Greg Stein
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ * This is an Instaweb-specific modification of
+ * src/third_party/serf/src/buckets/response_buckets.c
+ *
+ * 1. Fix a buffer overrun in fetch_headers() by guarding the call to
+ * serf_bucket_headers_setx.
+ * 2. Removing the block of code responsible for gunzipping zipped content.
+ * If the caller as specified Accept-Encoding:gzip then serf should not
+ * unzip it.
+ */
+
+#include <apr_lib.h>
+#include <apr_strings.h>
+#include <apr_date.h>
+
+#include "serf.h"
+#include "serf_bucket_util.h"
+
+
+typedef struct {
+ serf_bucket_t *stream;
+ serf_bucket_t *body; /* Pointer to the stream wrapping the body. */
+ serf_bucket_t *headers; /* holds parsed headers */
+
+ enum {
+ STATE_STATUS_LINE, /* reading status line */
+ STATE_HEADERS, /* reading headers */
+ STATE_BODY, /* reading body */
+ STATE_TRAILERS, /* reading trailers */
+ STATE_DONE /* we've sent EOF */
+ } state;
+
+ /* Buffer for accumulating a line from the response. */
+ serf_linebuf_t linebuf;
+
+ serf_status_line sl;
+
+ int chunked; /* Do we need to read trailers? */
+ int head_req; /* Was this a HEAD request? */
+} response_context_t;
+
+
+SERF_DECLARE(serf_bucket_t *) serf_bucket_response_create(
+ serf_bucket_t *stream,
+ serf_bucket_alloc_t *allocator)
+{
+ response_context_t *ctx;
+
+ ctx = serf_bucket_mem_alloc(allocator, sizeof(*ctx));
+ ctx->stream = stream;
+ ctx->body = NULL;
+ ctx->headers = serf_bucket_headers_create(allocator);
+ ctx->state = STATE_STATUS_LINE;
+ ctx->chunked = 0;
+ ctx->head_req = 0;
+
+ serf_linebuf_init(&ctx->linebuf);
+
+ return serf_bucket_create(&serf_bucket_type_response, allocator, ctx);
+}
+
+SERF_DECLARE(void) serf_bucket_response_set_head(
+ serf_bucket_t *bucket)
+{
+ response_context_t *ctx = bucket->data;
+
+ ctx->head_req = 1;
+}
+
+SERF_DECLARE(serf_bucket_t *) serf_bucket_response_get_headers(
+ serf_bucket_t *bucket)
+{
+ return ((response_context_t *)bucket->data)->headers;
+}
+
+
+static void serf_response_destroy_and_data(serf_bucket_t *bucket)
+{
+ response_context_t *ctx = bucket->data;
+
+ if (ctx->state != STATE_STATUS_LINE) {
+ serf_bucket_mem_free(bucket->allocator, (void*)ctx->sl.reason);
+ }
+
+ serf_bucket_destroy(ctx->stream);
+ if (ctx->body != NULL)
+ serf_bucket_destroy(ctx->body);
+ serf_bucket_destroy(ctx->headers);
+
+ serf_default_destroy_and_data(bucket);
+}
+
+static apr_status_t fetch_line(response_context_t *ctx, int acceptable)
+{
+ return serf_linebuf_fetch(&ctx->linebuf, ctx->stream, acceptable);
+}
+
+static apr_status_t parse_status_line(response_context_t *ctx,
+ serf_bucket_alloc_t *allocator)
+{
+ int res;
+ char *reason; /* ### stupid APR interface makes this non-const */
+
+ /* ctx->linebuf.line should be of form: HTTP/1.1 200 OK */
+ res = apr_date_checkmask(ctx->linebuf.line, "HTTP/#.# ###*");
+ if (!res) {
+ /* Not an HTTP response? Well, at least we won't understand it. */
+ return APR_EGENERAL;
+ }
+
+ ctx->sl.version = SERF_HTTP_VERSION(ctx->linebuf.line[5] - '0',
+ ctx->linebuf.line[7] - '0');
+ ctx->sl.code = apr_strtoi64(ctx->linebuf.line + 8, &reason, 10);
+
+ /* Skip leading spaces for the reason string. */
+ if (apr_isspace(*reason)) {
+ reason++;
+ }
+
+ /* Copy the reason value out of the line buffer. */
+ ctx->sl.reason = serf_bstrmemdup(allocator, reason,
+ ctx->linebuf.used
+ - (reason - ctx->linebuf.line));
+
+ return APR_SUCCESS;
+}
+
+/* This code should be replaced with header buckets. */
+static apr_status_t fetch_headers(serf_bucket_t *bkt, response_context_t *ctx)
+{
+ apr_status_t status;
+
+ /* RFC 2616 says that CRLF is the only line ending, but we can easily
+ * accept any kind of line ending.
+ */
+ status = fetch_line(ctx, SERF_NEWLINE_ANY);
+ if (SERF_BUCKET_READ_ERROR(status)) {
+ return status;
+ }
+ /* Something was read. Process it. */
+
+ if (ctx->linebuf.state == SERF_LINEBUF_READY && ctx->linebuf.used) {
+ const char *end_key;
+ const char *c;
+
+ end_key = c = memchr(ctx->linebuf.line, ':', ctx->linebuf.used);
+ if (!c) {
+ /* Bad headers? */
+ return APR_EGENERAL;
+ }
+
+ /* Skip over initial : and spaces. */
+ while (apr_isspace(*++c))
+ continue;
+
+ /* Always copy the headers (from the linebuf into new mem). */
+ /* ### we should be able to optimize some mem copies */
+
+ /*
+ * In Instaweb, we noticed that this code needs to be fixed to
+ * avoid risking a buffer overrun.
+ */
+ if (ctx->linebuf.line + ctx->linebuf.used > c) {
+ apr_size_t value_size = ctx->linebuf.line + ctx->linebuf.used - c;
+ serf_bucket_headers_setx(
+ ctx->headers,
+ ctx->linebuf.line, end_key - ctx->linebuf.line, 1,
+ c, value_size, 1);
+ }
+ }
+
+ return status;
+}
+
+/* Perform one iteration of the state machine.
+ *
+ * Will return when one the following conditions occurred:
+ * 1) a state change
+ * 2) an error
+ * 3) the stream is not ready or at EOF
+ * 4) APR_SUCCESS, meaning the machine can be run again immediately
+ */
+static apr_status_t run_machine(serf_bucket_t *bkt, response_context_t *ctx)
+{
+ apr_status_t status = APR_SUCCESS; /* initialize to avoid gcc warnings */
+
+ switch (ctx->state) {
+ case STATE_STATUS_LINE:
+ /* RFC 2616 says that CRLF is the only line ending, but we can easily
+ * accept any kind of line ending.
+ */
+ status = fetch_line(ctx, SERF_NEWLINE_ANY);
+ if (SERF_BUCKET_READ_ERROR(status))
+ return status;
+
+ if (ctx->linebuf.state == SERF_LINEBUF_READY) {
+ /* The Status-Line is in the line buffer. Process it. */
+ status = parse_status_line(ctx, bkt->allocator);
+ if (status)
+ return status;
+
+ /* Okay... move on to reading the headers. */
+ ctx->state = STATE_HEADERS;
+ }
+ else {
+ /* The connection closed before we could get the next
+ * response. Treat the request as lost so that our upper
+ * end knows the server never tried to give us a response.
+ */
+ if (APR_STATUS_IS_EOF(status)) {
+ return SERF_ERROR_REQUEST_LOST;
+ }
+ }
+ break;
+ case STATE_HEADERS:
+ status = fetch_headers(bkt, ctx);
+ if (SERF_BUCKET_READ_ERROR(status))
+ return status;
+
+ /* If an empty line was read, then we hit the end of the headers.
+ * Move on to the body.
+ */
+ if (ctx->linebuf.state == SERF_LINEBUF_READY && !ctx->linebuf.used) {
+ const void *v;
+
+ /* Advance the state. */
+ ctx->state = STATE_BODY;
+
+ ctx->body =
+ serf_bucket_barrier_create(ctx->stream, bkt->allocator);
+
+ /* Are we C-L, chunked, or conn close? */
+ v = serf_bucket_headers_get(ctx->headers, "Content-Length");
+ if (v) {
+ apr_uint64_t length;
+ length = apr_strtoi64(v, NULL, 10);
+ if (errno == ERANGE) {
+ return APR_FROM_OS_ERROR(ERANGE);
+ }
+ ctx->body = serf_bucket_limit_create(ctx->body, length,
+ bkt->allocator);
+ }
+ else {
+ v = serf_bucket_headers_get(ctx->headers, "Transfer-Encoding");
+
+ /* Need to handle multiple transfer-encoding. */
+ if (v && strcasecmp("chunked", v) == 0) {
+ ctx->chunked = 1;
+ ctx->body = serf_bucket_dechunk_create(ctx->body,
+ bkt->allocator);
+ }
+
+ if (!v && (ctx->sl.code == 204 || ctx->sl.code == 304)) {
+ ctx->state = STATE_DONE;
+ }
+ }
+
+ /*
+ * Instaweb would prefer to receive gzipped output if that's what
+ * was asked for.
+ *
+ * v = serf_bucket_headers_get(ctx->headers, "Content-Encoding");
+ * if (v) {
+ * * Need to handle multiple content-encoding. *
+ * if (v && strcasecmp("gzip", v) == 0) {
+ * ctx->body =
+ * serf_bucket_deflate_create(ctx->body, bkt->allocator,
+ * SERF_DEFLATE_GZIP);
+ * }
+ * else if (v && strcasecmp("deflate", v) == 0) {
+ * ctx->body =
+ * serf_bucket_deflate_create(ctx->body, bkt->allocator,
+ * SERF_DEFLATE_DEFLATE);
+ * }
+ * }
+ */
+
+ /* If we're a HEAD request, we don't receive a body. */
+ if (ctx->head_req) {
+ ctx->state = STATE_DONE;
+ }
+ }
+ break;
+ case STATE_BODY:
+ /* Don't do anything. */
+ break;
+ case STATE_TRAILERS:
+ status = fetch_headers(bkt, ctx);
+ if (SERF_BUCKET_READ_ERROR(status))
+ return status;
+
+ /* If an empty line was read, then we're done. */
+ if (ctx->linebuf.state == SERF_LINEBUF_READY && !ctx->linebuf.used) {
+ ctx->state = STATE_DONE;
+ return APR_EOF;
+ }
+ break;
+ case STATE_DONE:
+ return APR_EOF;
+ default:
+ /* Not reachable */
+ return APR_EGENERAL;
+ }
+
+ return status;
+}
+
+static apr_status_t wait_for_body(serf_bucket_t *bkt, response_context_t *ctx)
+{
+ apr_status_t status;
+
+ /* Keep reading and moving through states if we aren't at the BODY */
+ while (ctx->state != STATE_BODY) {
+ status = run_machine(bkt, ctx);
+
+ /* Anything other than APR_SUCCESS means that we cannot immediately
+ * read again (for now).
+ */
+ if (status)
+ return status;
+ }
+ /* in STATE_BODY */
+
+ return APR_SUCCESS;
+}
+
+SERF_DECLARE(apr_status_t) serf_bucket_response_wait_for_headers(
+ serf_bucket_t *bucket)
+{
+ response_context_t *ctx = bucket->data;
+
+ return wait_for_body(bucket, ctx);
+}
+
+SERF_DECLARE(apr_status_t) serf_bucket_response_status(
+ serf_bucket_t *bkt,
+ serf_status_line *sline)
+{
+ response_context_t *ctx = bkt->data;
+ apr_status_t status;
+
+ if (ctx->state != STATE_STATUS_LINE) {
+ /* We already read it and moved on. Just return it. */
+ *sline = ctx->sl;
+ return APR_SUCCESS;
+ }
+
+ /* Running the state machine once will advance the machine, or state
+ * that the stream isn't ready with enough data. There isn't ever a
+ * need to run the machine more than once to try and satisfy this. We
+ * have to look at the state to tell whether it advanced, though, as
+ * it is quite possible to advance *and* to return APR_EAGAIN.
+ */
+ status = run_machine(bkt, ctx);
+ if (ctx->state == STATE_HEADERS) {
+ *sline = ctx->sl;
+ }
+ else {
+ /* Indicate that we don't have the information yet. */
+ sline->version = 0;
+ }
+
+ return status;
+}
+
+static apr_status_t serf_response_read(serf_bucket_t *bucket,
+ apr_size_t requested,
+ const char **data, apr_size_t *len)
+{
+ response_context_t *ctx = bucket->data;
+ apr_status_t rv;
+
+ rv = wait_for_body(bucket, ctx);
+ if (rv) {
+ /* It's not possible to have read anything yet! */
+ if (APR_STATUS_IS_EOF(rv) || APR_STATUS_IS_EAGAIN(rv)) {
+ *len = 0;
+ }
+ return rv;
+ }
+
+ rv = serf_bucket_read(ctx->body, requested, data, len);
+ if (APR_STATUS_IS_EOF(rv)) {
+ if (ctx->chunked) {
+ ctx->state = STATE_TRAILERS;
+ /* Mask the result. */
+ rv = APR_SUCCESS;
+ }
+ else {
+ ctx->state = STATE_DONE;
+ }
+ }
+ return rv;
+}
+
+static apr_status_t serf_response_readline(serf_bucket_t *bucket,
+ int acceptable, int *found,
+ const char **data, apr_size_t *len)
+{
+ response_context_t *ctx = bucket->data;
+ apr_status_t rv;
+
+ rv = wait_for_body(bucket, ctx);
+ if (rv) {
+ return rv;
+ }
+
+ /* Delegate to the stream bucket to do the readline. */
+ return serf_bucket_readline(ctx->body, acceptable, found, data, len);
+}
+
+/* ### need to implement */
+#define serf_response_peek NULL
+
+SERF_DECLARE_DATA const serf_bucket_type_t serf_bucket_type_response = {
+ "RESPONSE",
+ serf_response_read,
+ serf_response_readline,
+ serf_default_read_iovec,
+ serf_default_read_for_sendfile,
+ serf_default_read_bucket,
+ serf_response_peek,
+ serf_response_destroy_and_data,
+ serf_default_snapshot,
+ serf_default_restore_snapshot,
+ serf_default_is_snapshot_set,
+};
diff --git a/trunk/src/third_party/serf/serf.diff b/trunk/src/third_party/serf/serf.diff
new file mode 100644
index 0000000..ca65c7d
--- /dev/null
+++ b/trunk/src/third_party/serf/serf.diff
@@ -0,0 +1,58 @@
+Index: buckets/allocator.c
+===================================================================
+--- buckets/allocator.c (revision 1421)
++++ buckets/allocator.c (working copy)
+@@ -83,6 +83,7 @@
+ struct serf_bucket_alloc_t {
+ apr_pool_t *pool;
+ apr_allocator_t *allocator;
++ int own_allocator;
+
+ serf_unfreed_func_t unfreed;
+ void *unfreed_baton;
+@@ -106,7 +107,9 @@
+ if (allocator->blocks) {
+ apr_allocator_free(allocator->allocator, allocator->blocks);
+ }
+-
++ if (allocator->own_allocator == 1) {
++ apr_allocator_destroy(allocator->allocator);
++ }
+ return APR_SUCCESS;
+ }
+
+@@ -119,10 +122,12 @@
+
+ allocator->pool = pool;
+ allocator->allocator = apr_pool_allocator_get(pool);
++ allocator->own_allocator = 0;
+ if (allocator->allocator == NULL) {
+ /* This most likely means pools are running in debug mode, create our
+ * own allocator to deal with memory ourselves */
+ apr_allocator_create(&allocator->allocator);
++ allocator->own_allocator = 1;
+ }
+ allocator->unfreed = unfreed;
+ allocator->unfreed_baton = unfreed_baton;
+@@ -408,4 +413,3 @@
+ }
+ #endif
+ }
+-
+Index: buckets/headers_buckets.c
+===================================================================
+--- buckets/headers_buckets.c (revision 1421)
++++ buckets/headers_buckets.c (working copy)
+@@ -97,6 +97,12 @@
+ new_val[found->value_size] = ',';
+ memcpy(new_val + found->value_size + 1, value, value_size);
+ new_val[new_size] = '\0';
++
++ // If the previous value was added, we must free it before overwriting
++ if (found->alloc_flags & ALLOC_VALUE) {
++ serf_bucket_mem_free(bkt->allocator, (void *)found->value);
++ }
++
+ found->value = new_val;
+ found->value_size = new_size;
+ found->alloc_flags |= ALLOC_VALUE;
diff --git a/trunk/src/third_party/serf/serf.gyp b/trunk/src/third_party/serf/serf.gyp
new file mode 100644
index 0000000..fbbe74d
--- /dev/null
+++ b/trunk/src/third_party/serf/serf.gyp
@@ -0,0 +1,69 @@
+# Copyright 2009 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{
+ 'variables': {
+ 'serf_root': '<(DEPTH)/third_party/serf',
+ 'serf_src': '<(serf_root)/src',
+ },
+ 'targets': [
+ {
+ 'target_name': 'serf',
+ 'type': '<(library)',
+ 'dependencies': [
+ '<(DEPTH)/third_party/apache/apr/apr.gyp:include',
+ '<(DEPTH)/third_party/apache/aprutil/aprutil.gyp:include',
+ '<(DEPTH)/third_party/zlib/zlib.gyp:zlib',
+ ],
+ 'sources': [
+ '<(serf_root)/instaweb_context.c',
+ '<(serf_src)/context.c',
+ '<(serf_src)/buckets/aggregate_buckets.c',
+ '<(serf_root)/instaweb_allocator.c',
+ '<(serf_src)/buckets/barrier_buckets.c',
+ '<(serf_src)/buckets/buckets.c',
+ '<(serf_src)/buckets/chunk_buckets.c',
+ '<(serf_src)/buckets/dechunk_buckets.c',
+ '<(serf_src)/buckets/deflate_buckets.c',
+ '<(serf_src)/buckets/file_buckets.c',
+ '<(serf_root)/instaweb_headers_buckets.c',
+ '<(serf_src)/buckets/limit_buckets.c',
+ '<(serf_src)/buckets/mmap_buckets.c',
+ '<(serf_src)/buckets/request_buckets.c',
+# There are two bugs in serf 0.3.1.
+# 1. There is a buffer-overrun risk in fetch_headers
+# 2. Serf always unzips zipped content, which is not what we want, and
+# then it leaves the 'gzip' headers in. This is in contrast to the
+# behavior of curl and wget, which provide gzipped output if that's
+# what was requested.
+#
+# We will try to pursue fixes to these issues with the Serf community but
+# in the meantime we will override our own version of response_buckets.c,
+# which will be placed in the root directory.
+ '<(serf_root)/instaweb_response_buckets.c',
+ '<(serf_src)/buckets/simple_buckets.c',
+ '<(serf_src)/buckets/socket_buckets.c',
+ ],
+ 'include_dirs': [
+ '<(serf_src)',
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ '<(serf_src)',
+ ],
+ },
+ }
+ ],
+}
+
