manual/src/docs/tutorials/howto-build-system.txt - polygene-java - Git at Google

 ///////////////////////////////////////////////////////////////
  * Licensed to the Apache Software Foundation (ASF) under one
  * or more contributor license agreements.  See the NOTICE file
  * distributed with this work for additional information
  * regarding copyright ownership.  The ASF licenses this file
  * to you under the Apache License, Version 2.0 (the
  * "License"); you may not use this file except in compliance
  * with the License.  You may obtain a copy of the License at
  *
  *   http://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing,
  * software distributed under the License is distributed on an
  * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
  * KIND, either express or implied.  See the License for the
  * specific language governing permissions and limitations
  * under the License.
 ///////////////////////////////////////////////////////////////

 [[build-system,Build System]]
 = Polygene™ Build System =

 This tutorial is intended for developers who want to build the Polygene™ SDK themselves.
 It describe the Polygene™ SDK Build System from compilation to publication of artifacts for consumption by other
 applications.

 If instead you want to setup your project build system to depend on modules of the Polygene™ SDK see the
 <<howto-depend-on-polygene,dedicated tutorial>>.


 == Gradle ==

 NOTE: All major Java IDEs have great Gradle support.
 Visit the https://www.gradle.org/resources[Gradle] website to learn how to import the Polygene™ SDK build into your
 favorite IDE.

 Polygene™ community migrated away from Maven after several years of frustration, especially around release management,
 versioning and cross-module dependency resolution issues, in Feb 2011.
 The tool of choice is now Gradle, and it doesn't require any installation, there are +gradlew+ and +gradlew.bat+ in
 the root folder of the Polygene™ SDK that will bootstrap Gradle if not done so already.

 If you are new to Gradle, you should keep the https://gradle.org/docs[documentation] at hands.

 Build System configuration is done through Gradle properties.
 This can be done in many ways, see
 https://docs.gradle.org/current/userguide/build_environment.html#sec:gradle_properties_and_system_properties[Gradle properties and system properties].

 == The Wrapper ==

 `gradlew` and `gradlew.bat` scripts that can be found at the root of the Polygene sources is *The Wrapper*.
 Any build invocation starts from this script.
 It will download the Gradle distribution version required by the build.
 See the https://docs.gradle.org/current/userguide/gradle_wrapper.html[Gradle Wrapper] documentation for more details.

 == Main tasks ==

 The Polygene™ SDK project has tasks that work with the whole SDK.

 `./gradlew downloadDependencies`::
 --
 Resolve, download and cache all needed dependencies.
 Useful to go offline.
 --

 `./gradlew`::
 --
 The default build, triggered when running gradle without any command line arguments, compiles the code and run the
 tests, but nothing else. A quick way to check that nothing broke.
 --

 `./gradlew clean`::
 --
 Clean up of all build output and restore the code base to a fresh state.
 --

 `./gradlew assemble`::
 --
 Produces all the archives, javadocs, manuals and website content.
 Global output is generated into +distributions/build+.
 --

 `./gradlew check`::
 --
 Run the tests and other checks like checkstyle.
 Global reports are generated in +reports/build/reports+.
 --

 `./gradlew build`::
 --
 Equivalent to `./gradlew assemble check`
 --

 `./gradlew checkDistributions`::
 --
 Run global checks against the assembled distributions.
 Can take a while.
 --

 `./gradlew install`::
 --
 Is roughly the same as Maven's install goal.
 It produces the test reports, javadocs and installs all the Jars into the local disk repository, for consumption
 by other applications.
 --


 == Other tasks ==

 In addition to that, some submodules have specific tasks.
 To see all available tasks, issue the following command:

 [source,bash]
 ----
 ./gradlew tasks
 ----

 All available tasks from all modules of the SDK are shown.
 If you want to narrow your exploration to submodules use the following:

 [source,bash]
 ----
 ./gradlew :test:performance:tasks
 ./gradlew :release:tasks
 ----

 These examples will respectively output all gradle tasks available in the +:tests:performance+ module where you should find
 the +performanceTest+ task that runs the Polygene™ performance test suite and the +:release+ module tasks.

 +tasks+ itself is a task, in the same way we can target module(s) with tasks, e.g.:

 [source,bash]
 ----
 ./gradlew :core:check :libraries:alarm:check
 ----


 == Versions ==

 By default, the build system produces a "zero build".
 It means that there is no version assigned to the build, and a "0" is used in the produced artifacts.
 This is due to our disagreement (with Maven community) that the "next" version name/number is known prior to
 the release.
 This is in our opinion a delayed decision.
 To build a particular version, you specify a +version+ property on the command-line, like

 [source,bash]
 -----------
 ./gradlew -Dversion=2.0-FLAVOUR install
 -----------

 If a +version+ property is not defined, the build system will refuse to make a release and upload.
 It will also try hard to do less and not get in your way.


 == Tests ==

 NOTE: See the https://builds.apache.org/view/P/view/Polygene/[Polygene™ Continuous Integration] for current tests results

 Unit and integration tests are located near the code under test.
 You'll find theses tests across the whole SDK.

 === Unit tests requiring external services ===

 Among unit tests, some require an external service to be run.
 For example, the Redis EntityStore extension requires an actual Redis server to run its tests.

 NOTE: The HTML test reports generated by Gradle shows skipped tests.

 Testing against external services is automated using Docker and is enabled automatically if a running Docker service
 is reachable.
 The build creates the necessary Docker images and start/stop containers around the tests.

 On Linux it should work out of the box.

 The simplest way to get this running on other systems (macOS and Windows) is to use `docker-machine` to create a
 development Docker virtual machine where all images will be built and containers started:

 [source,bash]
 ----
 docker-machine create dev
 docker-machine start dev
 eval $(docker-machine env dev)
 ----

 The last stanza set environment variables for Docker to use the newly created Docker virtual machine.

 If you want to run the Docker containers in a remote machine, simply set the `DOCKER_HOST` and `DOCKER_CERT_PATH`
 environment variables to something sensible for your setup.

 If you want to forcibly skip all Docker related work, set the `skipDocker` Gradle property by e.g. appending
 `-PskipDocker` to your Gradle command line.

 === Performance tests ===

 Performance tests provide performance measurements for typical Polygene™ use cases.
 They are not part of the default build and are located in the `tests/performance` directory of the SDK.

 They can be run with the following Gradle command:

 [source,bash]
 -----------
 ./gradlew :tests:performance:performanceTest
 -----------

 Results will then be available in the test reports.


 == Documentation generation ==

 The build generates a documentation minisite:

 [source,bash]
 -----------
 ./gradlew :manual:assemble
 -----------

 Output is in `~/manual/build/docs/website`.

 You'll need Asciidoc and docbook-xsl installed.


 == Build for releases ==

 IMPORTANT: Remember that if a +version+ property is not defined, the build system will refuse to make a release and upload.

 The Polygene™ SDK build system is setup for an easy release process.
 This is very useful to the Polygene™ Core Team but can also be useful to third parties that want to cut a in-house release.
 In this regard, we try to make every aspect of the release process usable for such cases.

 The following sections describe various aspects of the release process.
 By default you need to have a proper PGP setup, see below.


 === Release Criteria ===

 The Polygene™ SDK modules are of varying maturity level and we try to maintain a STATUS (+dev-status.xml+) file indicating
 how good the codebase, documentation and unit tests are for each of the modules. This is highly subjective and
 potentially different individuals will judge this differently, but at least it gives a ballpark idea of the situation
 for our users.

 The Polygene™ SDK build system use the values from the +dev-status.xml+ files to filter out non-releasable modules out for
 the +javadocs+ and +uploadArchives+ root project tasks.
 Moreover, the +release+ task ensure that no releasable module depends on module(s) that don't fit the release criteria
 and throw a detailed exception if need be.

 This can be relaxed by adding +-x checkReleaseSpec+ arguments to gradle invocation.


 === Signing ===

 Artifact signing is done using PGP.
 You need to provide Gradle the following properties, `~/.gradle/gradle.properties` is a good place:

     signing.keyId=FB751943
     signing.password=foobar
     signing.secretKeyRingFile=/home/foo/.gnupg/secring.gpg

 You can skip the signing process by adding +-x signArchives+ arguments to gradle invocation.


 === Artifact Upload ===

 Artifact upload behavior depends on the version assigned to the build.

 By default RELEASES are signed, SNAPSHOTS are not.
 Signing can be turned on or off by setting the `uploadSigned` property to false.

 By default RELEASES must satisfy ReleaseSpecification, SNAPSHOT don't.
 ReleaseSpecification usage can be turned on or off by setting the `uploadReleaseSpec` property to false.

 By default RELEASES and SNAPHOTS are uploaded using HTTP.
 Used Wagon can be overriden by setting the `uploadWagon` property.

 By default RELEASES and SNAPSHOTS are uploaded to the Apache Nexus.
 Target repository can be overriden by setting the `uploadRepository` property.

 No username/password is provided by default.
 If needed set them using the `uploadUsername` and `uploadPassword` properties.

 For example here is how to deploy all artifacts as unsigned SNAPSHOTs to a given repository:

 [source,bash]
 -----------
 ./gradlew uploadArchives -Dversion=3.2.1-SNAPSHOT -PuploadReleaseSpec=false \
     -PuploadWagon=what:ever:wagon -PuploadRepository=http://what.ever.repository/url \
     -PuploadUsername=foo -PuploadPassword=bar
 -----------

 And here is how to deploy a signed release to the local filesystem:

 [source,bash]
 -----------
 ./gradlew uploadArchives -Dversion=3.2.1 -PuploadRepository=file:///path/to/local/repository
 -----------

 See the https://docs.gradle.org/current/userguide/maven_plugin.html#wagonLibs[Gradle documentation] about
 supported protocols.
	///////////////////////////////////////////////////////////////
	* Licensed to the Apache Software Foundation (ASF) under one
	* or more contributor license agreements. See the NOTICE file
	* distributed with this work for additional information
	* regarding copyright ownership. The ASF licenses this file
	* to you under the Apache License, Version 2.0 (the
	* "License"); you may not use this file except in compliance
	* with the License. You may obtain a copy of the License at
	*
	* http://www.apache.org/licenses/LICENSE-2.0
	*
	* Unless required by applicable law or agreed to in writing,
	* software distributed under the License is distributed on an
	* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	* KIND, either express or implied. See the License for the
	* specific language governing permissions and limitations
	* under the License.
	///////////////////////////////////////////////////////////////

	[[build-system,Build System]]
	= Polygene™ Build System =

	This tutorial is intended for developers who want to build the Polygene™ SDK themselves.
	It describe the Polygene™ SDK Build System from compilation to publication of artifacts for consumption by other
	applications.

	If instead you want to setup your project build system to depend on modules of the Polygene™ SDK see the
	<<howto-depend-on-polygene,dedicated tutorial>>.


	== Gradle ==

	NOTE: All major Java IDEs have great Gradle support.
	Visit the https://www.gradle.org/resources[Gradle] website to learn how to import the Polygene™ SDK build into your
	favorite IDE.

	Polygene™ community migrated away from Maven after several years of frustration, especially around release management,
	versioning and cross-module dependency resolution issues, in Feb 2011.
	The tool of choice is now Gradle, and it doesn't require any installation, there are +gradlew+ and +gradlew.bat+ in
	the root folder of the Polygene™ SDK that will bootstrap Gradle if not done so already.

	If you are new to Gradle, you should keep the https://gradle.org/docs[documentation] at hands.

	Build System configuration is done through Gradle properties.
	This can be done in many ways, see
	https://docs.gradle.org/current/userguide/build_environment.html#sec:gradle_properties_and_system_properties[Gradle properties and system properties].

	== The Wrapper ==

	`gradlew` and `gradlew.bat` scripts that can be found at the root of the Polygene sources is The Wrapper.
	Any build invocation starts from this script.
	It will download the Gradle distribution version required by the build.
	See the https://docs.gradle.org/current/userguide/gradle_wrapper.html[Gradle Wrapper] documentation for more details.

	== Main tasks ==

	The Polygene™ SDK project has tasks that work with the whole SDK.

	`./gradlew downloadDependencies`::
	--
	Resolve, download and cache all needed dependencies.
	Useful to go offline.
	--

	`./gradlew`::
	--
	The default build, triggered when running gradle without any command line arguments, compiles the code and run the
	tests, but nothing else. A quick way to check that nothing broke.
	--

	`./gradlew clean`::
	--
	Clean up of all build output and restore the code base to a fresh state.
	--

	`./gradlew assemble`::
	--
	Produces all the archives, javadocs, manuals and website content.
	Global output is generated into +distributions/build+.
	--

	`./gradlew check`::
	--
	Run the tests and other checks like checkstyle.
	Global reports are generated in +reports/build/reports+.
	--

	`./gradlew build`::
	--
	Equivalent to `./gradlew assemble check`
	--

	`./gradlew checkDistributions`::
	--
	Run global checks against the assembled distributions.
	Can take a while.
	--

	`./gradlew install`::
	--
	Is roughly the same as Maven's install goal.
	It produces the test reports, javadocs and installs all the Jars into the local disk repository, for consumption
	by other applications.
	--


	== Other tasks ==

	In addition to that, some submodules have specific tasks.
	To see all available tasks, issue the following command:

	[source,bash]
	----
	./gradlew tasks
	----

	All available tasks from all modules of the SDK are shown.
	If you want to narrow your exploration to submodules use the following:

	[source,bash]
	----
	./gradlew :test:performance:tasks
	./gradlew :release:tasks
	----

	These examples will respectively output all gradle tasks available in the +:tests:performance+ module where you should find
	the +performanceTest+ task that runs the Polygene™ performance test suite and the +:release+ module tasks.

	+tasks+ itself is a task, in the same way we can target module(s) with tasks, e.g.:

	[source,bash]
	----
	./gradlew :core:check :libraries:alarm:check
	----


	== Versions ==

	By default, the build system produces a "zero build".
	It means that there is no version assigned to the build, and a "0" is used in the produced artifacts.
	This is due to our disagreement (with Maven community) that the "next" version name/number is known prior to
	the release.
	This is in our opinion a delayed decision.
	To build a particular version, you specify a +version+ property on the command-line, like

	[source,bash]
	-----------
	./gradlew -Dversion=2.0-FLAVOUR install
	-----------

	If a +version+ property is not defined, the build system will refuse to make a release and upload.
	It will also try hard to do less and not get in your way.


	== Tests ==

	NOTE: See the https://builds.apache.org/view/P/view/Polygene/[Polygene™ Continuous Integration] for current tests results

	Unit and integration tests are located near the code under test.
	You'll find theses tests across the whole SDK.

	=== Unit tests requiring external services ===

	Among unit tests, some require an external service to be run.
	For example, the Redis EntityStore extension requires an actual Redis server to run its tests.

	NOTE: The HTML test reports generated by Gradle shows skipped tests.

	Testing against external services is automated using Docker and is enabled automatically if a running Docker service
	is reachable.
	The build creates the necessary Docker images and start/stop containers around the tests.

	On Linux it should work out of the box.

	The simplest way to get this running on other systems (macOS and Windows) is to use `docker-machine` to create a
	development Docker virtual machine where all images will be built and containers started:

	[source,bash]
	----
	docker-machine create dev
	docker-machine start dev
	eval $(docker-machine env dev)
	----

	The last stanza set environment variables for Docker to use the newly created Docker virtual machine.

	If you want to run the Docker containers in a remote machine, simply set the `DOCKER_HOST` and `DOCKER_CERT_PATH`
	environment variables to something sensible for your setup.

	If you want to forcibly skip all Docker related work, set the `skipDocker` Gradle property by e.g. appending
	`-PskipDocker` to your Gradle command line.

	=== Performance tests ===

	Performance tests provide performance measurements for typical Polygene™ use cases.
	They are not part of the default build and are located in the `tests/performance` directory of the SDK.

	They can be run with the following Gradle command:

	[source,bash]
	-----------
	./gradlew :tests:performance:performanceTest
	-----------

	Results will then be available in the test reports.


	== Documentation generation ==

	The build generates a documentation minisite:

	[source,bash]
	-----------
	./gradlew :manual:assemble
	-----------

	Output is in `~/manual/build/docs/website`.

	You'll need Asciidoc and docbook-xsl installed.



	== Build for releases ==

	IMPORTANT: Remember that if a +version+ property is not defined, the build system will refuse to make a release and upload.

	The Polygene™ SDK build system is setup for an easy release process.
	This is very useful to the Polygene™ Core Team but can also be useful to third parties that want to cut a in-house release.
	In this regard, we try to make every aspect of the release process usable for such cases.

	The following sections describe various aspects of the release process.
	By default you need to have a proper PGP setup, see below.


	=== Release Criteria ===

	The Polygene™ SDK modules are of varying maturity level and we try to maintain a STATUS (+dev-status.xml+) file indicating
	how good the codebase, documentation and unit tests are for each of the modules. This is highly subjective and
	potentially different individuals will judge this differently, but at least it gives a ballpark idea of the situation
	for our users.

	The Polygene™ SDK build system use the values from the +dev-status.xml+ files to filter out non-releasable modules out for
	the +javadocs+ and +uploadArchives+ root project tasks.
	Moreover, the +release+ task ensure that no releasable module depends on module(s) that don't fit the release criteria
	and throw a detailed exception if need be.

	This can be relaxed by adding +-x checkReleaseSpec+ arguments to gradle invocation.


	=== Signing ===

	Artifact signing is done using PGP.
	You need to provide Gradle the following properties, `~/.gradle/gradle.properties` is a good place:

	signing.keyId=FB751943
	signing.password=foobar
	signing.secretKeyRingFile=/home/foo/.gnupg/secring.gpg

	You can skip the signing process by adding +-x signArchives+ arguments to gradle invocation.


	=== Artifact Upload ===

	Artifact upload behavior depends on the version assigned to the build.

	By default RELEASES are signed, SNAPSHOTS are not.
	Signing can be turned on or off by setting the `uploadSigned` property to false.

	By default RELEASES must satisfy ReleaseSpecification, SNAPSHOT don't.
	ReleaseSpecification usage can be turned on or off by setting the `uploadReleaseSpec` property to false.

	By default RELEASES and SNAPHOTS are uploaded using HTTP.
	Used Wagon can be overriden by setting the `uploadWagon` property.

	By default RELEASES and SNAPSHOTS are uploaded to the Apache Nexus.
	Target repository can be overriden by setting the `uploadRepository` property.

	No username/password is provided by default.
	If needed set them using the `uploadUsername` and `uploadPassword` properties.

	For example here is how to deploy all artifacts as unsigned SNAPSHOTs to a given repository:

	[source,bash]
	-----------
	./gradlew uploadArchives -Dversion=3.2.1-SNAPSHOT -PuploadReleaseSpec=false \
	-PuploadWagon=what:ever:wagon -PuploadRepository=http://what.ever.repository/url \
	-PuploadUsername=foo -PuploadPassword=bar
	-----------

	And here is how to deploy a signed release to the local filesystem:

	[source,bash]
	-----------
	./gradlew uploadArchives -Dversion=3.2.1 -PuploadRepository=file:///path/to/local/repository
	-----------

	See the https://docs.gradle.org/current/userguide/maven_plugin.html#wagonLibs[Gradle documentation] about
	supported protocols.