content/apt/guides/mini/guide-reproducible-builds.apt - maven-site - Git at Google

  ------
  Guide to Configuring for Reproducible Builds
  ------
  Hervé Boutemy
  ------
  2019-11-03
  ------

 ~~ Licensed to the Apache Software Foundation (ASF) under one
 ~~ or more contributor license agreements.  See the NOTICE file
 ~~ distributed with this work for additional information
 ~~ regarding copyright ownership.  The ASF licenses this file
 ~~ to you under the Apache License, Version 2.0 (the
 ~~ "License"); you may not use this file except in compliance
 ~~ with the License.  You may obtain a copy of the License at
 ~~
 ~~   http://www.apache.org/licenses/LICENSE-2.0
 ~~
 ~~ Unless required by applicable law or agreed to in writing,
 ~~ software distributed under the License is distributed on an
 ~~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 ~~ KIND, either express or implied.  See the License for the
 ~~ specific language governing permissions and limitations
 ~~ under the License.

 ~~ NOTE: For help with the syntax of this file, see:
 ~~ http://maven.apache.org/doxia/references/apt-format.html

 Configuring for Reproducible Builds

 * What are Reproducible Builds?

  {{{https://reproducible-builds.org/}Reproducible builds}} are a set of software development practices that create an
  independently-verifiable path from source to binary code. A build is <<reproducible>> if given the same source code,
  build environment and build instructions, any party can recreate <<bit-by-bit>> identical copies of all specified artifacts.

 * How do I configure my Maven build?

  There is no Maven version prerequisite. Everything happens at plugin level:

  [[1]] Upgrade your plugins to reproducible versions, particularly <<<maven-jar-plugin>>>, <<<maven-source-plugin>>> and <<<maven-assembly-plugin>>> to version 3.2.0 minimum.

  [[2]] Add a <<<project.build.outputTimestamp>>> property to the project's <<<pom.xml>>> with the timestamp value that will be used in zip/jar/tar archives (see FAQ for future value update strategy):

 +--------+
    <properties>
      <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
    </properties>
 +--------+
  []

  You have the basics configured. The output should be reproducible now.

 * How to test and fix my Maven build reproducibility?

  Using {{{https://github.com/apache/maven-artifact-plugin}<<<maven-artifact-plugin>>>'s <<<buildinfo>>> goal}},
  you can check that the second build of your project produce the same output than an initial build:

  [[1]] build and install your project: <<<mvn clean install -e -DskipTests>>> (don't hesitate to customize arguments to better match your project)

  [[2]] rebuild (without installing) and check against the previous install: <<<mvn clean verify -e -DskipTests artifact:buildinfo -Dreference.repo=central>>>

  []

  The <<<buildinfo>>> goal in the second run will calculate fingerprints for the second build output and for the initial installed output, then compare the 2 build informations and display result.

  If something is still not reproducible:

  [[1]] Use {{{https://diffoscope.org/}diffoscope}} to find the unstable output. The <<<artifact:buildinfo>>> goal proposes a command with path to files: just copy/paste to launch.

  [[2]] Find the plugin that generated this output.

  [[3]] Check if a reproducible version of the plugin is available.
  If not, please open an issue to help plugin maintainers improving Reproducible Builds support at every plugin level.

  []

  <<Notice>>: Reproducible Builds for Maven:

  * Require <<no version ranges>> in dependencies,

  * Generally give <<different results on Windows and Unix>> because of different newlines. (carriage return linefeed on Windows, linefeed on Unixes)

  * Generally depend on the <<major version of the JDK>> used to compile. (Even with source/target defined, each major JDK version changes the generated bytecode)

  []

  For detailed explanations, see {{{https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318}Maven "Reproducible/Verifiable Builds" Wiki page}}.

 * How Many Projects are Reproducible?

  You can have a look at {{{https://github.com/jvm-repo-rebuild/reproducible-central}Reproducible Central}} to see which projects releases
  have been checked as reproducible by rebuilding independently from the reference build published in Central Repository.
  You can also check for yourself that you can rebuild locally and get the same result.

 * FAQ

  * Q. Can the <<<project.build.outputTimestamp>>> property in <<<pom.xml>>> be updated automatically at release time?

    A. Yes.

    Details depend on your release process tooling:

    * if you use {{{/plugins/maven-release-plugin/}maven-release-plugin}}, you'll need <<version 3.0.0-M1 or later>>:
      it will automatically update the timestamp value in <<<pom.xml>>> during the release in the same commit that updates version,

    * if you have a custom release process tooling, you'll need to add the feature to your release tooling.

    []

    Don't hesitate to share your questions or solutions on {{{/mailing-lists.html}user mailing-list}}.

  * Q. Which additional plugins need to be updated for Reproducible Builds?

    A. Here is a simplified list:

 *----------------------------------------------------------------------------+-------+--------------+
 | <<plugin>>                                                   | <<minimum version>> | <<comments>>
 *----------------------------------------------------------------------------+-------+--------------+
 | {{{/plugins/maven-assembly-plugin/}maven-assembly-plugin}}                 | 3.2.0 |
 *----------------------------------------------------------------------------+-------+--------------+
 | {{{/plugins/maven-jar-plugin/}maven-jar-plugin}}                           | 3.2.0 |
 *----------------------------------------------------------------------------+-------+--------------+
 | {{{/plugins/maven-ejb-plugin/}maven-ejb-plugin}}                           | 3.1.0 |
 *----------------------------------------------------------------------------+-------+--------------+
 | {{{/plugins/maven-javadoc-plugin/}maven-javadoc-plugin}}                   | 3.2.0 |
 *----------------------------------------------------------------------------+-------+--------------+
 | {{{/plugin-tools/maven-plugin-plugin/}maven-plugin-plugin}}                | 3.5.1 |
 *----------------------------------------------------------------------------+-------+--------------+
 | {{{/plugins/maven-remote-resources-plugin/}maven-remote-resources-plugin}} | 1.7.0 |
 *----------------------------------------------------------------------------+-------+--------------+
 | {{{/plugins/maven-shade-plugin/}maven-shade-plugin}}                       | 3.2.3 |
 *----------------------------------------------------------------------------+-------+--------------+
 | {{{/plugins/maven-site-plugin/}maven-site-plugin}}                         | 3.9.0 |
 *----------------------------------------------------------------------------+-------+--------------+
 | {{{/plugins/maven-source-plugin/}maven-source-plugin}}                     | 3.2.1 |
 *----------------------------------------------------------------------------+-------+--------------+
 | {{{/plugins/maven-war-plugin/}maven-war-plugin}}                           | 3.3.1 |
 *----------------------------------------------------------------------------+-------+--------------+
 | {{{https://codehaus-plexus.github.io/plexus-containers/plexus-component-metadata/}plexus-component-metadata}} | 2.1.0 |
 *----------------------------------------------------------------------------+-------+--------------+
 | {{{https://github.com/bndtools/bnd/tree/master/maven/bnd-maven-plugin}bnd-maven-plugin}} |   | see {{{https://github.com/bndtools/bnd/tree/master/maven/bnd-maven-plugin#reproducible-builds}configuration instructions}}
 *----------------------------------------------------------------------------+-------+--------------+
 | Apache Felix {{{https://felix.apache.org/documentation/subprojects/apache-felix-maven-bundle-plugin-bnd.html}maven-bundle-plugin}} |   | <<<manifest>>> goal requires <<<\<_removeheaders\>Bnd-LastModified\</_removeheaders\>>>> as configuration <<<instructions>>>
 *----------------------------------------------------------------------------+-------+--------------+
 | {{{https://docs.spring.io/spring-boot/docs/current/maven-plugin/}springboot-maven-plugin}} | 2.3.0-M4 | not yet for war (because maven-war-plugin not yet ok)
 *----------------------------------------------------------------------------+-------+--------------+

   For more details, see {{{https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318#Reproducible/VerifiableBuilds-Whataretheissuestosolve?}Maven "Reproducible/Verifiable Builds" Wiki page}}
	------
	Guide to Configuring for Reproducible Builds
	------
	Hervé Boutemy
	------
	2019-11-03
	------

	~~ Licensed to the Apache Software Foundation (ASF) under one
	~~ or more contributor license agreements. See the NOTICE file
	~~ distributed with this work for additional information
	~~ regarding copyright ownership. The ASF licenses this file
	~~ to you under the Apache License, Version 2.0 (the
	~~ "License"); you may not use this file except in compliance
	~~ with the License. You may obtain a copy of the License at
	~~
	~~ http://www.apache.org/licenses/LICENSE-2.0
	~~
	~~ Unless required by applicable law or agreed to in writing,
	~~ software distributed under the License is distributed on an
	~~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	~~ KIND, either express or implied. See the License for the
	~~ specific language governing permissions and limitations
	~~ under the License.

	~~ NOTE: For help with the syntax of this file, see:
	~~ http://maven.apache.org/doxia/references/apt-format.html

	Configuring for Reproducible Builds

	* What are Reproducible Builds?

	{{{https://reproducible-builds.org/}Reproducible builds}} are a set of software development practices that create an
	independently-verifiable path from source to binary code. A build is <<reproducible>> if given the same source code,
	build environment and build instructions, any party can recreate <<bit-by-bit>> identical copies of all specified artifacts.

	* How do I configure my Maven build?

	There is no Maven version prerequisite. Everything happens at plugin level:

	[[1]] Upgrade your plugins to reproducible versions, particularly <<<maven-jar-plugin>>>, <<<maven-source-plugin>>> and <<<maven-assembly-plugin>>> to version 3.2.0 minimum.

	[[2]] Add a <<<project.build.outputTimestamp>>> property to the project's <<<pom.xml>>> with the timestamp value that will be used in zip/jar/tar archives (see FAQ for future value update strategy):

	+--------+
	<properties>
	<project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
	</properties>
	+--------+
	[]

	You have the basics configured. The output should be reproducible now.

	* How to test and fix my Maven build reproducibility?

	Using {{{https://github.com/apache/maven-artifact-plugin}<<<maven-artifact-plugin>>>'s <<<buildinfo>>> goal}},
	you can check that the second build of your project produce the same output than an initial build:

	[[1]] build and install your project: <<<mvn clean install -e -DskipTests>>> (don't hesitate to customize arguments to better match your project)

	[[2]] rebuild (without installing) and check against the previous install: <<<mvn clean verify -e -DskipTests artifact:buildinfo -Dreference.repo=central>>>

	[]

	The <<<buildinfo>>> goal in the second run will calculate fingerprints for the second build output and for the initial installed output, then compare the 2 build informations and display result.

	If something is still not reproducible:

	[[1]] Use {{{https://diffoscope.org/}diffoscope}} to find the unstable output. The <<<artifact:buildinfo>>> goal proposes a command with path to files: just copy/paste to launch.

	[[2]] Find the plugin that generated this output.

	[[3]] Check if a reproducible version of the plugin is available.
	If not, please open an issue to help plugin maintainers improving Reproducible Builds support at every plugin level.

	[]

	<<Notice>>: Reproducible Builds for Maven:

	* Require <<no version ranges>> in dependencies,

	* Generally give <<different results on Windows and Unix>> because of different newlines. (carriage return linefeed on Windows, linefeed on Unixes)

	* Generally depend on the <<major version of the JDK>> used to compile. (Even with source/target defined, each major JDK version changes the generated bytecode)

	[]

	For detailed explanations, see {{{https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318}Maven "Reproducible/Verifiable Builds" Wiki page}}.

	* How Many Projects are Reproducible?

	You can have a look at {{{https://github.com/jvm-repo-rebuild/reproducible-central}Reproducible Central}} to see which projects releases
	have been checked as reproducible by rebuilding independently from the reference build published in Central Repository.
	You can also check for yourself that you can rebuild locally and get the same result.

	* FAQ

	* Q. Can the <<<project.build.outputTimestamp>>> property in <<<pom.xml>>> be updated automatically at release time?

	A. Yes.

	Details depend on your release process tooling:

	* if you use {{{/plugins/maven-release-plugin/}maven-release-plugin}}, you'll need <<version 3.0.0-M1 or later>>:
	it will automatically update the timestamp value in <<<pom.xml>>> during the release in the same commit that updates version,

	* if you have a custom release process tooling, you'll need to add the feature to your release tooling.

	[]

	Don't hesitate to share your questions or solutions on {{{/mailing-lists.html}user mailing-list}}.

	* Q. Which additional plugins need to be updated for Reproducible Builds?

	A. Here is a simplified list:

	*----------------------------------------------------------------------------+-------+--------------+
	\| <<plugin>> \| <<minimum version>> \| <<comments>>
	*----------------------------------------------------------------------------+-------+--------------+
	\| {{{/plugins/maven-assembly-plugin/}maven-assembly-plugin}} \| 3.2.0 \|
	*----------------------------------------------------------------------------+-------+--------------+
	\| {{{/plugins/maven-jar-plugin/}maven-jar-plugin}} \| 3.2.0 \|
	*----------------------------------------------------------------------------+-------+--------------+
	\| {{{/plugins/maven-ejb-plugin/}maven-ejb-plugin}} \| 3.1.0 \|
	*----------------------------------------------------------------------------+-------+--------------+
	\| {{{/plugins/maven-javadoc-plugin/}maven-javadoc-plugin}} \| 3.2.0 \|
	*----------------------------------------------------------------------------+-------+--------------+
	\| {{{/plugin-tools/maven-plugin-plugin/}maven-plugin-plugin}} \| 3.5.1 \|
	*----------------------------------------------------------------------------+-------+--------------+
	\| {{{/plugins/maven-remote-resources-plugin/}maven-remote-resources-plugin}} \| 1.7.0 \|
	*----------------------------------------------------------------------------+-------+--------------+
	\| {{{/plugins/maven-shade-plugin/}maven-shade-plugin}} \| 3.2.3 \|
	*----------------------------------------------------------------------------+-------+--------------+
	\| {{{/plugins/maven-site-plugin/}maven-site-plugin}} \| 3.9.0 \|
	*----------------------------------------------------------------------------+-------+--------------+
	\| {{{/plugins/maven-source-plugin/}maven-source-plugin}} \| 3.2.1 \|
	*----------------------------------------------------------------------------+-------+--------------+
	\| {{{/plugins/maven-war-plugin/}maven-war-plugin}} \| 3.3.1 \|
	*----------------------------------------------------------------------------+-------+--------------+
	\| {{{https://codehaus-plexus.github.io/plexus-containers/plexus-component-metadata/}plexus-component-metadata}} \| 2.1.0 \|
	*----------------------------------------------------------------------------+-------+--------------+
	\| {{{https://github.com/bndtools/bnd/tree/master/maven/bnd-maven-plugin}bnd-maven-plugin}} \| \| see {{{https://github.com/bndtools/bnd/tree/master/maven/bnd-maven-plugin#reproducible-builds}configuration instructions}}
	*----------------------------------------------------------------------------+-------+--------------+
	\| Apache Felix {{{https://felix.apache.org/documentation/subprojects/apache-felix-maven-bundle-plugin-bnd.html}maven-bundle-plugin}} \| \| <<<manifest>>> goal requires <<<\<_removeheaders\>Bnd-LastModified\</_removeheaders\>>>> as configuration <<<instructions>>>
	*----------------------------------------------------------------------------+-------+--------------+
	\| {{{https://docs.spring.io/spring-boot/docs/current/maven-plugin/}springboot-maven-plugin}} \| 2.3.0-M4 \| not yet for war (because maven-war-plugin not yet ok)
	*----------------------------------------------------------------------------+-------+--------------+

	For more details, see {{{https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318#Reproducible/VerifiableBuilds-Whataretheissuestosolve?}Maven "Reproducible/Verifiable Builds" Wiki page}}