asciidoc/tutorial/build-repository/advanced.adoc - ant-ivy - Git at Google

 ////
    Licensed to the Apache Software Foundation (ASF) under one
    or more contributor license agreements.  See the NOTICE file
    distributed with this work for additional information
    regarding copyright ownership.  The ASF licenses this file
    to you under the Apache License, Version 2.0 (the
    "License"); you may not use this file except in compliance
    with the License.  You may obtain a copy of the License at

      https://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing,
    software distributed under the License is distributed on an
    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    KIND, either express or implied.  See the License for the
    specific language governing permissions and limitations
    under the License.
 ////

 Now that you have seen how simple it is to create your own repository from an existing one, you may wonder how you can handle more complex cases, like when the source and destination repositories don't follow the same naming conventions.


 == On the road to a professional repository

 In this section, you will learn how to build a *professional* repository. What is a *professional* repository? Our vision is to say that a good quality repository must follow clear rules about project naming and must offer correct, usable configurations and verified project descriptors. In order to achieve those goals, we believe that you have to build your own repository.
 We have seen in the previous example, that we could use some public repositories to begin building our own repository. Nevertheless, the result is not always the expected one, especially concerning the naming rules used.

 This problem is pretty normal when you have an existing repository, and want to benefit from large public repositories which do not follow the same naming conventions. It also shows up because many public repositories do not use a  consistent naming scheme. For example, why don't all the Apache Commons modules use the org.apache.commons organization? Well... for historical reasons. But if you set up your own repository, you may not want to suffer from the mistakes of history.

 Fortunately, Ivy has a very powerful answer to this problem: link:../../settings/namespaces{outfilesuffix}[namespaces].


 == Using namespaces

 If you look at the repository built with the link:../../tutorial/build-repository/basic.html[previous tutorial], you will see exactly what we were talking about: all Apache Commons modules use their own name as their organization.

 So let's see what Ivy can do using namespaces (we will dig into details later):

 [source,shell]
 ----

 include::asciidoc/tutorial/log/install-namespace.txt[]

 ----


 Now if we look at our repository, it seems to look fine.

 [source,shell]
 ----

 include::asciidoc/tutorial/log/myrepository-content-namespace.txt[]

 ----


 We can even have a look at the commons-lang Ivy file in our repository:

 [source]
 ----

 <?xml version="1.0" encoding="UTF-8"?>
 <ivy-module version="1.0">
     <info organisation="apache"
           module="commons-lang"
           revision="1.0"
           status="integration"
           publication="20051124062021"
           namespace="ibiblio-maven2"/>

 ...

 ----


 Alright, we see that the organisation is now 'apache'. But where did Ivy pick this up?

 === How does this work ?

 Actually, Ivy uses the same repository as before for the source repository, with only one difference: the namespace parameter:

 [source]
 ----

 <ibiblio name="libraries"
          root="${ibiblio-maven2-root}"
          m2compatible="true"
          namespace="maven2"/>

 ----

 A namespace is defined by a set of rules. These rules are based on regular expressions and tell Ivy how to convert data from the repository namespace  to what is called the system namespace, i.e. the namespace in which Ivy runs most of the time (Note: the Ivy cache always uses the system namespace).

 For the namespace we call __maven2__, we have declared several rules. Below is one of the rules:

 ==== rule handling the imported Apache Maven 1 projects


 [source]
 ----
 <rule>  <!-- imported apache maven 1 projects -->
     <fromsystem>
         <src org="apache" module=".+"/>

         <dest org="$m0" module="$m0"/>
     </fromsystem>
     <tosystem>
         <src org="commons-.+" module="commons-.+"/>
         <src org="ant.*" module="ant.*"/>
         ...
         <src org="xmlrpc" module="xmlrpc"/>

         <dest org="apache" module="$m0"/>
     </tosystem>
 </rule>
 ----


 [NOTE]
 ====
 Note about regular expressions usage :
 In order to distinguish matching regular expressions found in organization, module, and revision, the notation Ivy uses prefixes the matching regular expression with the letters 'o', 'm' and 'r'.
 $o0 : the whole matching value in the organization attribute
 $o1 : the first matching expression group that was marked in the organization attribute
 ...
 The same applies for modules : $m0, $m1, ...
 and for revisions : $r0, $r1, ...

 ====

 To understand namespaces,


 * *fromsystem :* we define here that the projects defined in the system namespace under the organization called "apache" are transformed into the destination namespace into projects whose organization is named with the module name, whatever the revision is. For example, the project apache#commons-lang;1.0  in the system namespace will be translated into commons-lang#commons-lang;1.0 in the maven2 resolver namespace. +

 * *tosystem :* we define here the reverse mapping, i.e. how to translate _Apache_ projects from Maven 2 repo into Apache projects in the system namespace. The rule used here tells Ivy that all projects matching `commons-.+` (see it as Java regular expression) for their organization name and module name are transformed into projects whose organisation is `apache` with the module name as it was found. The same kind of rule is defined for other Apache projects like Ant, etc. +


 OK, you should now get the idea behind namespaces, so go ahead and look at the `ivysettings-advanced.xml` file in this example. You can test the installation of a module and its dependencies using namespaces.

 Run

 [source]
 ----
 ant maven2-namespace-deps
 ----

 and you will see the resulting repository is cleaner than the first one we built.

 From our experience, investing in creating a namespace is worth the time it costs if you often need to add new modules or revisions of third party libraries in your own repository, where naming rules already exist or are rather strict.
	////
	Licensed to the Apache Software Foundation (ASF) under one
	or more contributor license agreements. See the NOTICE file
	distributed with this work for additional information
	regarding copyright ownership. The ASF licenses this file
	to you under the Apache License, Version 2.0 (the
	"License"); you may not use this file except in compliance
	with the License. You may obtain a copy of the License at

	https://www.apache.org/licenses/LICENSE-2.0

	Unless required by applicable law or agreed to in writing,
	software distributed under the License is distributed on an
	"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	KIND, either express or implied. See the License for the
	specific language governing permissions and limitations
	under the License.
	////

	Now that you have seen how simple it is to create your own repository from an existing one, you may wonder how you can handle more complex cases, like when the source and destination repositories don't follow the same naming conventions.



	== On the road to a professional repository

	In this section, you will learn how to build a professional repository. What is a professional repository? Our vision is to say that a good quality repository must follow clear rules about project naming and must offer correct, usable configurations and verified project descriptors. In order to achieve those goals, we believe that you have to build your own repository.
	We have seen in the previous example, that we could use some public repositories to begin building our own repository. Nevertheless, the result is not always the expected one, especially concerning the naming rules used.

	This problem is pretty normal when you have an existing repository, and want to benefit from large public repositories which do not follow the same naming conventions. It also shows up because many public repositories do not use a consistent naming scheme. For example, why don't all the Apache Commons modules use the org.apache.commons organization? Well... for historical reasons. But if you set up your own repository, you may not want to suffer from the mistakes of history.

	Fortunately, Ivy has a very powerful answer to this problem: link:../../settings/namespaces{outfilesuffix}[namespaces].


	== Using namespaces

	If you look at the repository built with the link:../../tutorial/build-repository/basic.html[previous tutorial], you will see exactly what we were talking about: all Apache Commons modules use their own name as their organization.

	So let's see what Ivy can do using namespaces (we will dig into details later):

	[source,shell]
	----

	include::asciidoc/tutorial/log/install-namespace.txt[]

	----



	Now if we look at our repository, it seems to look fine.

	[source,shell]
	----

	include::asciidoc/tutorial/log/myrepository-content-namespace.txt[]

	----


	We can even have a look at the commons-lang Ivy file in our repository:

	[source]
	----

	<?xml version="1.0" encoding="UTF-8"?>
	<ivy-module version="1.0">
	<info organisation="apache"
	module="commons-lang"
	revision="1.0"
	status="integration"
	publication="20051124062021"
	namespace="ibiblio-maven2"/>

	...

	----


	Alright, we see that the organisation is now 'apache'. But where did Ivy pick this up?

	=== How does this work ?

	Actually, Ivy uses the same repository as before for the source repository, with only one difference: the namespace parameter:

	[source]
	----

	<ibiblio name="libraries"
	root="${ibiblio-maven2-root}"
	m2compatible="true"
	namespace="maven2"/>

	----

	A namespace is defined by a set of rules. These rules are based on regular expressions and tell Ivy how to convert data from the repository namespace to what is called the system namespace, i.e. the namespace in which Ivy runs most of the time (Note: the Ivy cache always uses the system namespace).

	For the namespace we call __maven2__, we have declared several rules. Below is one of the rules:

	==== rule handling the imported Apache Maven 1 projects


	[source]
	----
	<rule> <!-- imported apache maven 1 projects -->
	<fromsystem>
	<src org="apache" module=".+"/>

	<dest org="$m0" module="$m0"/>
	</fromsystem>
	<tosystem>
	<src org="commons-.+" module="commons-.+"/>
	<src org="ant." module="ant."/>
	...
	<src org="xmlrpc" module="xmlrpc"/>

	<dest org="apache" module="$m0"/>
	</tosystem>
	</rule>
	----


	[NOTE]
	====
	Note about regular expressions usage :
	In order to distinguish matching regular expressions found in organization, module, and revision, the notation Ivy uses prefixes the matching regular expression with the letters 'o', 'm' and 'r'.
	$o0 : the whole matching value in the organization attribute
	$o1 : the first matching expression group that was marked in the organization attribute
	...
	The same applies for modules : $m0, $m1, ...
	and for revisions : $r0, $r1, ...

	====

	To understand namespaces,


	* fromsystem : we define here that the projects defined in the system namespace under the organization called "apache" are transformed into the destination namespace into projects whose organization is named with the module name, whatever the revision is. For example, the project apache#commons-lang;1.0 in the system namespace will be translated into commons-lang#commons-lang;1.0 in the maven2 resolver namespace. +

	* tosystem : we define here the reverse mapping, i.e. how to translate _Apache_ projects from Maven 2 repo into Apache projects in the system namespace. The rule used here tells Ivy that all projects matching `commons-.+` (see it as Java regular expression) for their organization name and module name are transformed into projects whose organisation is `apache` with the module name as it was found. The same kind of rule is defined for other Apache projects like Ant, etc. +


	OK, you should now get the idea behind namespaces, so go ahead and look at the `ivysettings-advanced.xml` file in this example. You can test the installation of a module and its dependencies using namespaces.

	Run

	[source]
	----
	ant maven2-namespace-deps
	----

	and you will see the resulting repository is cleaner than the first one we built.

	From our experience, investing in creating a namespace is worth the time it costs if you often need to add new modules or revisions of third party libraries in your own repository, where naming rules already exist or are rather strict.