content/documentation/extensions/mod_metamodel.adoc

:jbake-type: page
:jbake-status: published

= Apache Tamaya - Extension: Metamodel (Configuration of Tamaya)

toc::[]


[[Model]]
== Tamaya Metamodel (Configuration of Tamaya) (Extension Module)

Tamaya _metamodel_ is an extension module. Refer to the link:../extensions.html[extensions documentation] for further details.


=== What functionality this module provides ?

The Tamaya _metamodel_ module provides support for configuring the Tamaya system itself. It
allows, like a logging configuration, to configure how your configuration framework should
work, where to find configuration and how it is combined using overrides, filters etc.

By default it uses an XML based configuration format as illustrated below:

[source, xml]
.Extract from `tamaya-config.xml`
-----------------------------------------------
<configuration>
    <!-- Context is evaluated first. -->
    <context>
        <context-entry name="stage">${properties:system:STAGE?default=DEV}</context-entry>
        <context-entry name="configdir">${properties:system:configdir?default=.}</context-entry>
        <context-entry name="app">${properties:system.APP?default=NONE}</context-entry>
        <context-entry name="context">${java:org.apache.tamaya.context.Context#id()}</context-entry>
        <context-entry name="company">Trivadis</context-entry>
        <context-entry name="default-formats">yaml,json</context-entry>
        <context-entry name="default-refresh-period">5 SECOND</context-entry>
    </context>

    <!-- combinationPolicy type="" / -->

    <!-- Configuration definition. -->

    <property-sources>
       <source enabled="${stage=TEST || stage=PTA || stage=PROD}"
           type="env-properties">
           <filter type="PropertyMapping">
               <param name="mapTarget">ENV.</param>
           </filter>
           <filter type="AccessMask">
               <param name="roles">admin,power-user</param>
               <param name="policy">mask</param>
               <param name="mask">*****</param>
               <param name="matchExpression">SEC_</param>
           </filter>
       </source>
       <source type="sys-properties" >
           <filter type="ImmutablePropertySource" />
       </source>
       <source type="file" refreshable="true">
           <name>config.json</name>
           <param name="location">config.json</param>
       </source>
       ...
    </property-sources>
</configuration>
-----------------------------------------------

The module basically provides an XML representation to the +ConfigurationContextBuilder+ API.
It creates a corresponding +ConfigurationContext+ and registers the corresponding +Configuration+
as the system's _default_ configuration (accessible from `ConfigurationProvider.getConfiguration()`.


=== Compatibility

The module is based on Java 8.


=== Installation

To use _metamodel_ features you only must add the corresponding dependency to your module:

[source, xml]
-----------------------------------------------
<dependency>
  <groupId>org.apache.tamaya.ext</groupId>
  <artifactId>tamaya-model</artifactId>
  <version>{tamaya_version}</version>
</dependency>
-----------------------------------------------


=== Creating a Configuration using Meta-Configuration

The basic feature of this module is the capability of creating a +Configuration+ completely
based on a meta-configuration file. For this the +MetaConfiguration+ main singleton
provides different methods:

[source, java)
-----------------------------------------------
public final class MetaConfiguration {
    public static void configure();
    public static void configure(URL metaConfig);
    public static ConfigurationContextBuilder createContextBuilder(URL metaConfig);
    public static Configuration createConfiguration(URL metaConfig);
-----------------------------------------------

* If you have supplied your meta-configuration at `META-INF/tamaya-config.xml` you simply
  call +MetaConfiguration.configure();+. This will read the meta-configuration and
  configure Tamaya's _default_ configuration. Alternatively you can choose your own
  metaconfiguration location by passing an alternate `URL` ro read from.
* With +MetaConfiguration.createContextBuilder()+ you can stop a step earlier: a new
  instance of +ConfigurationContextBuilder+ is created and configured with all the
  entries found in your meta-configuration. Also here you can optionally pass your
  custom location for the meta-configuration resouce.
* Finally +MetaConfiguration.createConfiguration(URL)+ allows you to create an
  arbitrary +Configuration+ instance using a meta-configuration file. The `Configuration`
  instance is completely independent and not registered as _default_ configuration, so
  it's lifecycle and usage is completely under your control.


=== MetaContext

When thinking what are the various input parameters for determining a correct configuration, there
might be different things relevant in different scenarios, especially for developers in different
companies. A good example of such an input parameter is the current `STAGE`. All these kinf od inputs
can be summarized in some sort of meta-configuration, commonly known as a _context_. So
the metamodel extension ships with a +MetaContext+ class that allows to define a common meta-context,
that can be accessed by components as needed to determine the correct settings to be applied:

[source, java)
-----------------------------------------------
public final class MetaContext {

    ...

    public static MetaContext getInstance(String contextName);

    /**
     * Access the default context. Contexts are managed as weak references in this class. If no
     * such context exists, a new instance is created.
     * @return the context instance, never null.
     */
    public static MetaContext getDefaultInstance();

    /**
     * Access a context by name. Contexts are managed as weak references in this class. If no
     * such valid context exists, a new instance is created, using the given {@code validSupplier}.
     * @param contextName the context name, not null.
     * @return the context instance, never null.
     */
    public static MetaContext getInstance(String contextName, Supplier<Boolean> validSupplier);

    /**
     * Access the thread-based context. If no such context
     * exists a new one will be created.
     * @param reinit if true, clear's the thread's context.
     * @return the corresponding context, never null.
     */
    public static MetaContext getThreadInstance(boolean reinit);

    /**
     * Access the current context, which actually is the current context, combined with the thread based
     * context (overriding).
     * @return the corresponding context, never null.
     */
    public MetaContext getCurrentInstance();

     /**
     * Access the current context, which actually is the current context, combined with the thread based
     * context (overriding).
     * @param reinit if true, clear's the thread's context.
     * @return the corresponding context, never null.
     */
    public MetaContext getCurrentInstance(boolean reinit);


    /**
     * Method to evaluate if a context is valid. This basically depends on the
     * {@code validSupplier}, if any is set. If no supplier is present the context is valid.
     *
     * @return true, if this context is valid.
     */
    public boolean isValid();

    /**
     * Combine this context with the other contexts given, hereby only contexts are included
     * which are {@code valid}, see {@link #isValid()}.
     * @param contexts the context to merge with this context.
     * @return the newly created Context.
     */
    public MetaContext combineWith(MetaContext... contexts);

    /**
     * Access the given context property.
     * @param key the key, not null
     * @return the value, or null.
     */
    public String getProperty(String key);

    /**
     * Access the given context property.
     * @param key the key, not the default value.
     * @param defaultValue the default value to be returned, if no value is defined, or the
     *                     stored value's TTL has been reached.
     * @return the value, default value or null.
     */
    public String getProperty(String key, String defaultValue);

    /**
     * Sets the given context property.
     * @param key the key, not null.
     * @param value the value, not null.
     * @return the porevious value, or null.
     */
    public String setProperty(String key, String value);

    /**
     * Sets the given context property.
     * @param key the key, not null.
     * @param value the value, not null.
     * @param ttl the time to live. Zero or less than zero means, no timeout.
     * @param unit the target time unit.
     * @return the porevious value, or null.
     */
    public String setProperty(String key, String value, int ttl, TimeUnit unit);

    /**
     * Sets the given property unless there is already a value defined.
     * @param key the key, not null.
     * @param value the value, not null.
     */
    public void setPropertyIfAbsent(String key, String value);

    /**
     * Sets the given property unless there is already a value defined.
     * @param key the key, not null.
     * @param value the value, not null.
     * @param ttl the time to live. Zero or less than zero means, no timeout.
     * @param unit the target time unit.
     */
    public void setPropertyIfAbsent(String key, String value, long ttl, TimeUnit unit);

    /**
     * Adds all properties given, overriding any existing properties.
     * @param properties the properties, not null.
     */
    public void setProperties(Map<String,String> properties);

    /**
     * Adds all properties given, overriding any existing properties.
     * @param properties the properties, not null.
     * @param ttl the time to live. Zero or less than zero means, no timeout.
     * @param unit the target time unit.
     */
    public void setProperties(Map<String,String> properties, long ttl, TimeUnit unit);

    /**
     * Checks if all the given properties are present.
     * @param keys the keys to check, not null.
     * @return true, if all the given keys are existing.
     */
    public boolean checkProperties(String... keys);

    /**
     * Access all the current context properties.
     * @return the properties, never null.
     */
    public Map<String,String> getProperties();
}
-----------------------------------------------

As you see, a +MetaContext+ has the following aspects:

* there are multiple context's possible, identified by their name.
* Accessing an instance that does not yet exist, will create a new one.
* there is one shared _default_ instance.
* they store ordinary `String,String` key, value pairs.
* they can be _combined_ into a overriging hierarchy
* accessing the _default_ MetaContext returns the global instance combined with
  a threaded override instance. Passing `reinit` will clear the thread instance's
  data.


==== Configuring MetaContexts

`MetaContext` instances can be configured in the _meta-configuration_ in the first
`meta-context` section as illustrated below:

[source, xml]
-----------------------------------------------
<!-- Configuring the default context -->
<context>
    <context-entry name="stage">${properties:system:STAGE?default=DEV}</context-entry>
    <context-entry name="configdir">${properties:system:configdir?default=.}</context-entry>
    <context-entry name="app">${properties:system.APP?default=NONE}</context-entry>
    <context-entry name="context">${java:org.apache.tamaya.context.Context#id()}</context-entry>
    <context-entry name="company">Trivadis</context-entry>
    <context-entry name="default-formats">yaml,json</context-entry>
    <context-entry name="default-refresh-period">5 SECOND</context-entry>
</context>
<!-- Configuring a context named 'APP' -->
<context name="APP">
    <context-entry name="application">someAppName</context-entry>
</context>
-----------------------------------------------

As shown above multiple contexts can be configured. Keys and values are of type `String`.


===== Using Expressions

As shown before, it is possible to add simple expressions, enclosed in `${}`. Hereby the
contents must be formatted as `evaluator:expression`, which then internally must be interpreted by
the +org.apache.tamaya.metamodel.internal.SimpleResolver+, which effectively reads and
applied context entries.

Currently the following placeholders for context entries are provided:

* properties - mapping to system properties (`properties:sys:KEY`) or
  environment properties (`properties:env:KEY`) or other MetaContext
   entries initialized already (`properties:ctx[:CTXNAME]:KEY`)
* java - mapping to a static method or field, returning a `String` value.


=== General Extensions

Working with meta-models requires additional aspects to be generalized to separate
concerns and reuse some of the common functionality. These concepts are shown in the following
subsections.

=== Enabled

Things can be dynamically enabled or disabled, e.g. based on context. This can be
modelled by the +Enabled+ interface:

[source, java]
-----------------------------------------------
public interface Enabled {

    /**
     * Returns the enabled property.
     * @return the enabled value.
     */
    boolean isEnabled();

    /**
     * Enables/disables this property source.
     * @param enabled the enabled value.
     */
    void setEnabled(boolean enabled);
}
-----------------------------------------------

+Enabled+ can be used as a mixin-logic, e.g. for decorating property sources,
property source providers, filters and converters. The decorator can also, if not
set explicitly, evaluate the _enabled_ property based on the current runtime
context.


=== Refreshable

Similar to _Enabled_ things can also be refreshable.

[source, java]
-----------------------------------------------
public interface Refreshable {

    /**
     * Refreshes the given instance.
     */
    void refresh();
}
-----------------------------------------------

This can be used to define a common API for refreshing artifctas. Similar to
_Enabled_ this can be applied as a decorator/mix-in interface to property
sources and property source providers. This property also is supported in the
XML metaconfiguration, e.g.

[source, xml]
-----------------------------------------------
<property-sources>
    <source type="file" refreshable="true">
       <name>config.json</name>
       <param name="location">config.json</param>
    </source>
</property-sources>
-----------------------------------------------


=== The MetaConfiguration XML Structure

In general the `tamaya-config.xml` file does never apply an XML schema or
similar. Nevertheless there is a common DSL structure, which can be extended
as well (see next chapter).

[source, xml]
-----------------------------------------------
<configuration>
    <!-- PART ONE: Contexts initialization. -->
    <context>
        <context-entry name="stage">${properties:system:STAGE?default=DEV}</context-entry>
        <context-entry name="configdir">${properties:system:configdir?default=.}</context-entry>
        ...
    </context>
    <context name="APP">
        <context-entry name="application">someAppName</context-entry>
    </context>

    <!-- PART TWO: Global settings of ConfigurationContext. -->
    <!-- combinationPolicy type="" / -->

    <!-- PART THREE: Configuration definition. -->

    <property-sources>
       <source enabled="${stage=TEST || stage=PTA || stage=PROD}"
           type="env-properties">
           <filter type="PropertyMapping">
               <param name="mapTarget">ENV.</param>
           </filter>
           <filter type="AccessMask">
               <param name="roles">admin,power-user</param>
               <param name="policy">mask</param>
               <param name="mask">*****</param>
               <param name="matchExpression">SEC_</param>
           </filter>
       </source>
       <source type="sys-properties" >
           <filter type="ImmutablePropertySource" />
       </source>
       <source type="file" refreshable="true">
           <name>config.json</name>
           <param name="location">config.json</param>
       </source>
        <source type="file" refreshable="true">
            <name>config.xml</name>
            <param name="location">config.xml</param>
            <param name="formats">xml-properties</param>
        </source>
       <source-provider type="resource">
           <name>classpath:application-config.yml</name>
           <param name="location">/META-INF/application-config.yml</param>
       </source-provider>
       <source type="ch.mypack.MyClassSource" />
       <!--<include enabled="${stage==TEST}">TEST-config.xml</include>-->
       <source-provider type="resource" enabled="${configdir != null}">
           <name>config-dir</name>
           <param name="location">/${configdir}/**/*.json</param>
       </source-provider>
       <source type="url" refreshable="true">
           <name>remote</name>
           <param name="location">https://www.confdrive.com/cfg/customerId=1234</param>
           <param name="formats">json</param>
           <filter type="CachedPropertySource">
               <param name="ttl">30 SECOND</param>
           </filter>
       </source>
    </property-sources>
    <property-filters>
        <filter type="UsageTrackerFilter"/>
        <filter type="AccessControl">
            <param name="roles">admin,power-user</param>
            <param name="policy">hide</param>
            <param name="expression">*.secret</param>
        </filter>
        <filter type="Cache">
            <param name="ttl">30000</param>
            <param name="expression">cached.*</param>
        </filter>
    </property-filters>
    <property-converters>
    <!--<converter type="AllInOneConverter"/>-->
        <default-converters/>
    </property-converters>
</configuration>
-----------------------------------------------

The different parts in fact are not hardcoded, but implemented
as independent components, where each of them gets access to the
XML DOM tree to read the configuration aspects of interest.
Instances related must implement the ++ interface and register it to
the `ServiceContext`. Reading order is mapped using `@Priority`
annotations.
For further details refer to the SPI section in this document.


== Model SPI

=== Extending the XML DSL

The XML DSL can be extended in various ways:

* Basically adding a new feature maps to adding a new section to the
  meta-config XML. This can be easily done, by implementing +MetaConfigurationReader+
  and do whatever is appropriate for your use case.
* For adding new expression capabilities for `MetaContext`entries +SimpleResolver+ must
  be implemented.
* For allowing customized parameterization of artifacts, e.g. property sources,
  property source providers, converters and filters etc. you may implement +ItemFactory+
  instances.

=== MetaConfigurationReader

XML metaconfiguration is effectively processed by instances of
type +org.apache.tamaya.metamodel.spi.MetaConfigurationReader+:

[source,java]
-----------------------------------------------------------
public interface MetaConfigurationReader {

     /**
      * Reads meta-configuration from the given document and configures the current
      * context builder. The priority of readers is determined by the priorization policy
      * implemented by the {@link org.apache.tamaya.spi.ServiceContext},
      * @param document the meta-configuration document
      * @param contextBuilder the context builder to use.
      */
     void read(Document document, ConfigurationContextBuilder contextBuilder);

 }
-----------------------------------------------------------

Hereby we also see that an instance of `ConfigurationContextBuilder` is passed.
Remember, we mentioned earlier that meta-configuration basically is a XML
API to the building a configuration using a +ConfigurationContextBuilder+. So
all you can do with the meta-config XML can also be done programmatically using
the Java API.

This module provides instances of this class for reading of meta-context,
property-sources, property source providers, converters, filters and more.
Look into the +org.apache.tamaya.metamodel.internal+ package for further details.

New instances implementing this interface must be registered into the current
+ServiceContext+, by default the +ServiceLoader+ is used.


=== ItemFactory

Instances of +ItemFactory+ allow to configure artifacts using XML data:

[source, java]
-----------------------------------------------------------
public interface ItemFactory<T> {

    /**
     * Get the factory name.
     * @return the factory name, not null.
     */
    String getName();

    /**
     * Create a new instance.
     * @param parameters the parameters for configuring the instance.
     * @return the new instance, not null.
     */
    T create(Map<String,String> parameters);

    /**
     * Get the target type created by this factory. This can be used to
     * assign the factory to an acording item base type, e.g. a PropertySource,
     * PropertySourceProvider, PropertyFilter etc.
     * @return the target type, not null.
     */
    Class<? extends T> getType();

}
-----------------------------------------------------------

The factory's name hereby is used as a short cut, e.g. have a look at the following
XML snippet defining a `PropertySource` to be added:

[source, xml]
-----------------------------------------------------------
<source type="file" refreshable="true">
   <name>config.json</name>
   <param name="location">config.json</param>
</source>
-----------------------------------------------------------

In the above snippet _file_ equals to the factory name, which provides the user
a simple to use short name, instead of adding the fully qualified classname
(which is always possible).

The _location_ paramter with its value is passed as `Map` to the `create` method.


=== ItemFactoryManager

This singleton class manages the _ItemFactory_ instances found, hereby allowing
accessing and registering instances. This singleton is actually used by the
component parsers (type `MetaConfigurationReader`).

[source, java]
-----------------------------------------------------------
public final class ItemFactoryManager {

    ...

    public static ItemFactoryManager getInstance();

    public <T> List<ItemFactory<T>> getFactories(Class<T> type);
    public <T> ItemFactory<T> getFactory(Class<T> type, String id);

    public <T> void registerItemFactory(ItemFactory<T> factory);

}
-----------------------------------------------------------


=== Extended Implementations

The package +org.apache.tamaya.metamodel.ext+ contains a few useful
implementations that also can be used in your meta-configuration and
show how mixin-functionality can be added without touching property source
implementations.

As of now the package contains

* +EnabledPropertySource+: a decorator for a `PropertySource`
  adding the capability to _enable/disable_ the property source.
* +EnabledPropertySourceProvider+ a decorator for a `PropertySourceProvider`
  adding the capability to _enable/disable_ the property source provider.
* +RefreshablePropertySource+: a decorator for a `PropertySource`
  adding the capability to _refresh_ the property source.
* +EnabledPropertySourceProvider+ a decorator for a `PropertySourceProvider`
  adding the capability to _refresh_ the property source provider.

Not yet implemented but planned are implementations to add the following
functionality:

* _caching_ of entries for a given time.
* _immutability_ of entries, so a configuration data (or parts of it) will
  never change later.