content/documentation/extensions/mod_formats.adoc

:jbake-type: page
:jbake-status: published

= Apache Tamaya - Extension: Formats

toc::[]


[[Formats]]
== Tamaya Formats (Extension Module)

Tamaya _Formats_ is an extension module. Refer to the link:../extensions.html[extensions documentation] for further details.


=== What functionality this module provides ?

Tamaya _Formats_ provides an abstraction for configuration formats provding the following benefits:

* Parsing of resources in can be implemented separately from interpreting the different aspects/parts parsed. As an
  example a file format can define different sections. Depending on the company specific semantics of the sections
  a different set of +PropertySource+ instances must be created.
* Similarly the configuration abstraction can also be used as an interface for integrating Tamaya with alternate
  frameworks that provide logic for reading configuration files, such as Apache commons.configuration.

=== Compatibility

The module is based on Java 8.

=== Installation

To use the formats module you only must add the corresponding dependency to your module:

[source, xml]
-----------------------------------------------
<dependency>
  <groupId>org.apache.tamaya.ext</groupId>
  <artifactId>tamaya-formats</artifactId>
  <version>{tamaya_version}</version>
</dependency>
-----------------------------------------------


=== Basic Concept

Formats should be reusable, meaning you should have to write a format parser only once and then be able to map the data read into whatever
data structure (in our cases: property sources). So it is useful to separate concerns into

* an arbitrary configuration format (textual or binary)
* a parser (+ConfigurationFormat+) that transfers a given format into an intermediate
  representation (+ConfigurationData+).
* an optional customization, implemented by a _factory method pattern_ to adapt the mapping of +ConfigurationData+ read
  to a collection of +PropertySources+ (they can have different ordinal semantics).


==== ConfigurationData

Configuration formats can be very different. Some are simple key/value pairs, whereas other also consist of multiple sections (e.g. ini-files) or
hierarchical data (e.g. yaml, xml). This is solved in Tamaya by mapping the configuration read into a normalized intermediary format called
+ConfigurationData+:

[source,java]
.ConfigurationData
-------------------------------------------------------
public final class ConfigurationData {

    public ConfigurationFormat getFormat();
    public String getResource();

    public List<PropertyValue> getData();

    public boolean isEmpty();
}
-------------------------------------------------------

* with +getResource()+ and +getFormat()+ the underlying resource and the format that read this data can be accessed.
* +getData()+ allows access to the data read. Hereby this data can be mapped in different ways:
** as (multiple) simple key-value literal pairs, or
** as field-mapped object type values, or as
** list values
** the children of list and object values can be of any of the three types described above. As a consequence a simple
 `PropertyValue` can be a simple literal value, or a complex tree/list structure, similar to common configuration
 formats.


==== ConfigurationFormat

A ConfigurationFormat is basically an abstraction that reads a configuration resource (modelled by an InputStream) and
creates a corresponding +ConfigurationData+ instance.

[source,java]
-------------------------------------------------------
public interface ConfigurationFormat {

    String getName();
    boolean accepts(URL url);
    ConfigurationData readConfiguration(String resource, InputStream inputStream);
}
-------------------------------------------------------


=== Creating a default PropertySource for a ConfigurationFormat

The module defines a singleton +ConfigurationFormats+ which provides
an easy to use API for creating +ConfigurationData+ and +PropertySources+
using abstract +ConfigurationFormat+ implementations:

[source,java]
-------------------------------------------------------
public final class ConfigurationFormats {

    public static ConfigurationFormats getInstance();
    public static ConfigurationFormats getInstance(ClassLoader classLoader);

    public List<ConfigurationFormat> getFormats();
    public List<ConfigurationFormat> getFormats(String... formatNames);
    public List<ConfigurationFormat> getFormats(final URL url);

    public ConfigurationData readConfigurationData(final URL url)
    throws IOException;
    public ConfigurationData readConfigurationData(URL url, ConfigurationFormat... formats)
    throws IOException;
    public ConfigurationData readConfigurationData(URL url, Collection<ConfigurationFormat> formats)
    throws IOException;
    public Collection<ConfigurationData> readConfigurationData(Collection<URL> urls, ConfigurationFormat... formats);
    public Collection<ConfigurationData> readConfigurationData(Collection<URL> urls, Collection<ConfigurationFormat> formats);
    public ConfigurationData readConfigurationData(String resource, InputStream inputStream,
                                                          ConfigurationFormat... formats)
    throws IOException;
    public ConfigurationData readConfigurationData(String resource, InputStream inputStream,
                                                          Collection<ConfigurationFormat> formats)
    throws IOException;

    public PropertySource createPropertySource(URL url, ConfigurationFormat... formats)
    throws IOException;
    public PropertySource createPropertySource(URL url, Collection<ConfigurationFormat> formats)
    throws IOException;
    public PropertySource createPropertySource(String resource, InputStream inputStream,
                                                      ConfigurationFormat... formats);
    public PropertySource createPropertySource(String resource, InputStream inputStream,
                                                       Collection<ConfigurationFormat> formats);
}
-------------------------------------------------------

* +getFormats()+ returns all registered formats.
* +getFormats(String...)+ allows to access all formats with a given name.
* +getFormats(URL url)+ allows to access all formats that declare that can optionally read an input from
  a given `URL`.
* +readConfigurationData(...)+ reads data from an input and creates a corresponding +ConfigurationData+,
  either trying all known formats that declare its compatibility with the given input or the formats
  passed explicitly.
* +createPropertySource(...)+ allows to create a +PropertySource+ reading a given input and the formats
  to be used or known. Hereby a default property mapping is applied.

So creating a +PropertySource+ from a resource is basically a one liner:

[source,java]
-------------------------------------------------------
URL url = ...;
PropertySource propertySource = ConfigurationFormats.getInstance().createPropertySource(url);

// constraining the formats to be used (assumption: json and yaml extensions are loaded)
PropertySource propertySource = ConfigurationFormats.getInstance().reatePropertySource(
                                    url,
                                    ConfigurationFormats.getFormats("json", "yaml"));
-------------------------------------------------------


=== Customize how ConfigurationData maps to PropertySource

For for the conversion of +ConfigurationData+ into a +PropertySource+ different approaches can be useful.
In most cases the usage of a +MappedConfigurationDataPropertySource+, is a good choice to start. This class
provides a convenient default mapping and also allows to customized the mapping easily: it simply iterates over all
values recursively and adds them using their fully qualified name as single value properties.

This behaviour can be easily adapted by overriding the +popoulateData+ method:

[source,java]
-------------------------------------------------------
ConfigurationData data = ...;
MappedConfigurationDataPropertySource ps =
  new MappedConfigurationDataPropertySource(data){
    protected Map<String, PropertyValue> populateData(ConfigurationData data) {
      ...
    }
  };
-------------------------------------------------------

Nevertheless, depending on the context, where a configuration source was read (classloader, time, source etc.) the
resulting properties can have different semnatics, especially different priorities. Also section
names may be mapped into different ordinals instead of using them as key prefixes (e.g. imagine configuration formats
with a 'default', 'main', and 'overrides' sections). For such use cases no simple mapping
can be defined. Consequently the functionality mapping the normalized +ConfigurationData+ read to the
appropriate collection of +PropertySource+ instances must be implemented by the client code.

For this scenario the +BaseFormatPropertySourceProvider+ can be used, defining the following mapping
function that mus be implemented:

[source,java]
-------------------------------------------------------
/**
 * Method to create a {@link org.apache.tamaya.spi.PropertySource} based on the given entries read.
 *
 * @param data the configuration data, not null.
 * @return the {@link org.apache.tamaya.spi.PropertySource} instance ready to be registered.
 */
protected abstract Collection<PropertySource> getPropertySources(ConfigurationData data);
-------------------------------------------------------

When using Java 8 these mappings can be asily passed as parameters to the +createPropertySource+
methods.


=== Predefined formats

The _formats_ module ships with 3 predefined formats:

* `.ini` files, commonly known from Microsoft based systems, registered as `ini`.
* `.properties` files, as defined by `java.util.Properties`, registered as `properties`.
* `.xml` properties files, as defined by `java.util.Properties`, registered as `xml-properties`.


==== ini Configuration File Mapping

This module implements the ini file format with the class
+org.apache.tamaya.format.formats.IniConfigurationFormat+.

The default mapping is bext illustrated by a small example, so consider the
following `.ini` file:

[source,listing]
-------------------------------------------------------
a=valA
a.b=valB

[section1]
aa=sectionValA
aa.b.c=SectionValC

[section2]
a=val2Section2
-------------------------------------------------------

This file content by default is mapped to the following Tamaya properties:

[source,listing]
-------------------------------------------------------
a=valA
a.b=valB
section1.valA=sectionValA
section1.a.b.c=SectionValC
section2.a=val2Section2
-------------------------------------------------------

Summarizing

* entries without a section are mapped to the _default_ section.
* entries with a section are mapped to a corresponding section, hereby everything between
  the brackets is used as section name (trimmed).
* section names are separated using a double colon (`.`), they are modelled as simple parent `PropertyValue` nodes.

+ConfigurationData+ allows to access all the different parts:

* the _default_ properties (a, a.b)
* the section +section1+, with properties +aa, aa.b.c+
* the section +section2+, with properties +a+


==== XML Property and ordinary Property Files

This module also ships with +ConfigurationFormat+ implementations that reuse the parsing
functionality provided with +java.util.Properties+:

* `org.apache.tamaya.format.formats.PropertiesFormat` uses `Properties.read(InputStream)`.
* `org.apache.tamaya.format.formats.PropertiesXmlFormat` uses `Properties.readFromXml(InputStream)`.