Title: SPI Fly

#SPI Fly#

This page describes the SPI Fly component.
The SPI Fly component is aimed at providing OSGi support for JRE SPI
mechanisms, including the usage of <tt>java.util.ServiceLoader</tt>, 
<tt>META-INF/services</tt> and similar methods.

SPI Fly is the Reference Implementation of the OSGi ServiceLoader Mediator specification, chapter 133 in the [OSGi 
Enterprise Specification][1], available from version 5.

##The Problem##
Java's <tt>java.util.ServiceLoader.load()</tt>, other similar methods such as 
<tt>sun.misc.Service.providers()</tt> and also other static finder methods such as the 
<tt>FactoryFinder.find()</tt> methods try to locate 'service' implementations by looking for 
resources in the META-INF/services directory of all the jars visible to the 
***Thread Context ClassLoader*** (TCCL).

There are a number of issues with the above mechanisms when used in OSGi:

 1. The Thread Context ClassLoader is not defined in general in an OSGi context. It can and has to be set by the caller and OSGi cannot generally enforce that. 
 2. A bundle can't Import-Package META-INF/services as potentially many bundles will contain this pseudo-package and the OSGi framework will only bind a single exporter to an importer for a given package.
 3. Instantiating an SPI provider generally requires access to internal implementation classes, by exporting these classes an implementing bundle would break its encapsulation. 
 4. Even if an implementation class was exported, importing this class in a consumer bundle would bind it to the specific implementation package provided, which violates the principle of loose coupling.
 5. Bundles have a dynamic life-cycle which means that provided services could disappear when a bundle is updated or uninstalled. The java.util.ServiceLoader API does not provide a mechanism to inform service consumers of such an event.

The SPI Fly project makes it possible to use existing code that uses 
<tt>ServiceLoader.load()</tt> and similar mechanisms under OSGi.

##Making it Work##
In order to make ServiceLoader (and other similar SPI or plugin mechanisms) work under OSGi, calling code can be woven
to set the TCCL to the appropriate providers very briefly, only for the duration of the
call. The SPI Fly component does precisely this.

SPI Fly supports two modes of operation:

  - OSGi Specification compliant configuration. This is based on OSGi generic requirements and capabilities and 
provides portability across implementations of this specification. However, it only covers <tt>java.util.ServiceLoader</tt>. [Find it here](#specconf).
  - If you need to handle cases other than <tt>java.util.ServiceLoader</tt>, such as the various FactoryFinders, 
<tt>javax.imageio.spi.ServiceRegistry</tt>, <tt>javax.sound.sampled.AudioSystem</tt> or others that use the TCCL
to find an implementation, you can use the [SPI Fly-specific configuration](#specificconf).

Additionally, services found in the META-INF/services location in opted-in bundles will be registered in the OSGi Service 
Registry so that OSGi-aware consumers can simply find them there. This is supported in both the spec-compliant as 
well as the proprietary configuration modes. Each such service is registered in the OSGi Service Registry with 
the *serviceloader.mediator* service registration property set.

##Getting SPI Fly##

###<a id="releases"></a>Releases###
The latest release of the SPI-Fly components [can be found on Maven Central][2]. It can be obtained from Maven 
Central using the links below.

To use SPI Fly you need to decide whether to use the dynamic or static weaving version. 
More information about this can be found in the [usage section][3].

**For use with dynamic weaving:**

  - [SPI Fly Dynamic Weaving Bundle][4]

Dependencies:

  - [Aries Util 1.1.1][5]
  - [ASM 5.0.4][6]

**For use with static weaving:**

  - [SPI Fly Static Weaving command-line tool][7] (take the executable jar with dependencies)
  - [SPI Fly Static Weaving Bundle][8]

Dependencies:

  - [Aries Util 1.1.1][9]

###Building the code###
The code can be found in
[http://svn.apache.org/repos/asf/aries/trunk/spi-fly][10].

To build, use Maven 3.x and run <tt>mvn install</tt>

#<a id="specconf"></a>Configuration: OSGi Spec-compliant#
All the details surrounding this type of configuration are covered in the 
[OSGi Enterprise Specification][11] (from version 5) chapter 133. This section provides a short overview.

##Providers##
SPI provider bundles opt in to being registered by specifying a requirement on the 
<tt>osgi.serviceloader.registrar</tt> extender. This is done by adding the following Bundle Manifest header. Without this they will not be considered by SPI Fly:

<tt>&nbsp;&nbsp;Require-Capability: osgi.extender; filter:="(osgi.extender=osgi.serviceloader.registrar)"</tt>

Additionally, they need to provide capabilities for all the APIs that are exposed through this mechanism, for example:

<tt>&nbsp;&nbsp;Provide-Capability: osgi.serviceloader; osgi.serviceloader=org.apache.aries.spifly.mysvc.MySPIProvider</tt>

While this example omits it, it is advisable to specify <tt>uses</tt> constraints on the Provide-Capability header, to 
ensure consistent class spaces.

See the [<tt>spi-fly-example-provider2-bundle</tt>][12] for an example of a spec-compliant provider.

##Consumers##

An SPI consumer (i.e. a bundle using the <tt>java.util.ServiceLoader.load()</tt> API) needs to specify required capabilities
in the Required-Capability Manifest header. Two different types of requirements must be specified:

  - A requirement on the SPI Fly processing mechanism. This is stated as follows<br/>
<tt>&nbsp;&nbsp;osgi.extender; filter:="(osgi.extender=osgi.serviceloader.processor)"</tt><br/>
without this requirement the bundle will not be considered for processing.
  - A requirement on the SPI that needs to be provided through this mechanism, for example<br/>
<tt>&nbsp;&nbsp;osgi.serviceloader; filter:="(osgi.serviceloader=org.apache.aries.spifly.mysvc.MySPIProvider)";cardinality:=multiple</tt><br/>
Note that the <tt>cardinality</tt> directive is specified to allow multiple bundles to provide the requested capability, allowing
provided services to come from more than one provider bundle.

All requirements are combined into a single Require-Capability header:

<tt>&nbsp;&nbsp;Require-Capability: osgi.serviceloader; filter:="(osgi.serviceloader=org.apache.aries.spifly.mysvc.MySPIProvider)";cardinality:=multiple,osgi.extender; filter:="(osgi.extender=osgi.serviceloader.processor)"</tt>

See the [<tt>spi-fly-example-client2-bundle</tt>][13] for an example of a spec-compliant consumer.

#<a id="specificconf"></a>Configuration: SPI Fly-specific#
This section describes how to use SPI Fly's proprietary configuration
mechanism. It provides more features, but doesn't provide the 
portability that spec-compliance configuration gives. If you 
are only using SPI Fly with <tt>java.util.ServiceLoader</tt> or you are only using the provided 
services through the OSGi Service Registry, then consider
using the [spec-compliant](#specconf) configuration for portability.

##Providers##
First for all, SPI Fly needs to be made aware of any bundles that provide the services.
These bundles are made visible through the TCCL for the duration of the <tt>ServiceLoader.load()</tt>
(or similar) call.

To mark a bundle as a Provider, set the **<tt>SPI-Provider</tt>** manifest header:

* **SPI-Provider: *** will consider all providers found in the META-INF/services
directory and register them.
* **SPI-Provider: org.acme.MySvc1, org.acme.MySvc2** will only consider MySvc1 and
MySvc2.

Additionally services found in META-INF/services are registered in the OSGi Service 
Registry.

The <tt>SPI-Provider</tt> header can either be set in the providing bundle itself or in a wrapper bundle
that holds the original unmodified jar containing the provider internally as a 
on the <tt>Bundle-ClassPath</tt>.

See the [<tt>spi-fly-example-provider1-bundle</tt>][14] for an example of a provider using this type of configuration.

##Consumers##
Service consumers also need to opt in to the process. 

To specify a consumer, add the <tt>SPI-Consumer</tt> manifest header to the client bundle. This header 
will opt-in the bundle to the weaving process where for the duration of the specified call
the TCCL will be set to the matching provider bundle(s). Some example SPI-Consumer headers are:

* **SPI-Consumer: *** This is a shorthand for 
<tt>java.util.ServiceLoader#load(java.lang.Class)</tt> and will 
automatically weave all <tt>ServiceLoader.load(Class)</tt> calls.
* **SPI-Consumer: java.util.ServiceLoader#load(java.lang.Class[org.apache.aries.mytest.MySPI])**
Only process calls to <tt>ServiceLoader.load(Class)</tt> when it is called with 
<tt>MySPI.class</tt> as argument.
* **SPI-Consumer: javax.xml.parsers.DocumentBuilderFactory#newInstance()** weave clients that
call <tt>DocumentBuilderFactory.newInstance()</tt>. 
* **SPI-Consumer: org.foo.Foo#someMthd(),org.bar.Bar#myMethod()** weave calls to <tt>Foo.someMthd()</tt> and 
<tt>Bar.myMethod()</tt>.

See the [<tt>spi-fly-example-client1-bundle</tt>][15] for an example of a consumer using this type of configuration. 

###Special Cases###
SPI Fly can be used for most SPI provider/lookup systems that use the TCCL pattern to obtain
implementations. However, some cases some *special treatment* is needed. This special treatment is often needed when the API itself does not 
match the name of the resources in META-INF/services, java.util.ServiceLoader is such a case, however SPI-Fly has built-in knowledge of ServiceLoader.
Known APIs that require special treatment are listed below:

  - **javax.imageio.spi.ServiceRegistry**: This class is very much like
java.util.ServiceLoader in that it can load any kind of API implementation.
While SPI Fly knows about ServiceLoader and treats it specially, the ServiceRegistry
class currently does not have special treatment. It can still be made to work
but this requires the following header in the provider bundle:
<tt>SPI-Provider:&nbsp;javax.imageio.spi.ServiceRegistry</tt> on the client
side you can use <tt>SPI-Consumer:&nbsp;javax.imageio.spi.ServiceRegistry#lookupProviders(java.lang.Class)</tt>
 or <tt>SPI-Consumer:&nbsp;javax.imageio.spi.ServiceRegistry#lookupProviders</tt>
  - **javax.sound.sampled.AudioSystem**: This class uses sun.misc.Service under the covers (via com.sun.media.sound.JDK13Services) 
which is a predecessor to java.util.ServiceLoader. There is no special treatment for sun.misc.Service
in SPI Fly (yet), but the AudioSystem.getAudioInputStream() API can be made to work by explicitly listing it in the provider bundle 
(the one that contains the relevant META-INF/services resources):
<tt>SPI-Provider:&nbsp;javax.sound.sampled.AudioSystem</tt> on the consumer side you can use
<tt>SPI-Consumer:&nbsp;javax.sound.sampled.AudioSystem#getAudioInputStream
</tt>

#<a id="usage"></a>Usage#
There are currently two ways to use the SPI Fly component. If you have an OSGi 
4.3 (or higher) compliant framework that supports WeavingHooks you can use the dynamic weaving approach. 

If you have an pre-4.3 OSGi framework or don't want to use bytecode weaving at runtime you 
can use the static weaving approach.

##Use with Dynamic Weaving##
Install and start the [<tt>org.apache.aries.spifly.dynamic.bundle</tt>][16] into the system. This bundle 
has a dependency on <tt>[org.objectweb.asm][17]</tt> version 4.0 or newer and on the Aries 
Util bundle.

<pre>g! lb
START LEVEL 1
   ID|State      |Level|Name
    0|Active     |    0|System Bundle (5.0.1)
... bundles 1-4 are framework/console internal ones ...
    5|Active     |    1|ASM all classes (5.0.4)
    6|Active     |    1|Apache Aries Util (1.1.1)
    7|Active     |    1|Apache Aries SPI Fly Dynamic Weaving Bundle
</pre>

Note that, as with any OSGi Bundle that uses the OSGi 4.3 WeavingHooks, the weaver
bundle (<tt>org.apache.aries.spifly.dynamic.bundle</tt> in the SPI Fly case) needs to 
be active before any bundles that need to be dynamically woven. OSGi Start Levels can
provide a mechanism to control this.

##Use with Static Weaving##
For static use, you need to weave the client bundle before installing it into the system. 
The modification changes the byte code around <tt>java.util.ServiceLoader.load()</tt> or other calls in the 
bundle and inserts calls to set the correct ThreadContextClassLoader around it.
Provider bundles are still handled dynamically.

###To statically weave a bundle###
The easiest way to invoke the static weaver is to take the [<tt>org.apache.aries.spifly.static.tool</tt>][18]
jar with dependencies.

Then run the static tool on any bundle that needs processing:
<pre>
java -jar org.apache.aries.spifly.static.tool-1.0.2-jar-with-dependencies.jar mybundle.jar
</pre>

This will produce a second bundle with the same name with the _spifly suffix appended, so 
in this case the generated bundle will be called mybundle_spifly.jar.

At runtime, install the [<tt>org.apache.aries.spifly.static.bundle</tt>][19] into the system, like
the dynamic weaving bundle it has a dependency on the Aries Util bundle:
<pre>g! lb
START LEVEL 1
   ID|State      |Level|Name
    0|Active     |    0|System Bundle (5.0.1)
... bundles 1-4 are framework/console internal ones ...
    5|Active     |    1|Apache Aries Util (1.1.1)
    6|Active     |    1|Apache Aries SPI Fly Static Weaving Bundle</pre>

Then install and start the statically woven bundle into the system.

##<a id="examples"></a>Examples##
The <tt>spi-fly-examples</tt> directory contains a number of example bundles that can be 
used for testing or experimenting.

The following modules can be found in this directory:

* **spi-fly-example-spi-bundle** - a bundle providing an SPI interface used by the other example bundles. [osgi-bundle][20] [source][21]
* **spi-fly-example-provider1-jar** - a plain jar file providing an implementation of the SPI (via <tt>META-INF/services</tt>). [source][22]
* **spi-fly-example-provider1-bundle** - a bundle that wraps the jar file from the previous bullet and specifies it in its Bundle-ClassPath. This example represents the common case where an existing SPI provider is wrapped as-is in an OSGi bundle. This example uses the SPI Fly proprietary configuration. [osgi-bundle][23] [source][24]
* **spi-fly-example-provider2-bundle** - a bundle that directly provides an SPI service (via <tt>META-INF/services</tt>). This example uses OSGi  specification compliant configuration. [osgi-bundle][25] [source][26] 
* **spi-fly-example-client1-jar** - a plain jar using java.util.ServiceLoader.load() to obtain and invoke all services provided of a certain SPI. [source][27]
* **spi-fly-example-client1-bundle** - a bundle that wraps the jar file from the previous bullet and lists it in its Bundle-ClassPath. This example represents the common case where an existing SPI consumer is wrapped as-is in an OSGi bundle. This example uses SPI Fly proprietary configuration. [osgi-bundle][28] [source][29]
* **spi-fly-example-client2-bundle** - a bundle that has code that invokes <tt>java.util.ServiceLoader.load()</tt> directly. This example uses OSGi specification compliant configuration. [osgi-bundle][30] [source][31]
* **spi-fly-example-provider-consumer-bundle** - a bundle that is both a provider and a consumer at the same time. [source][32]
* **spi-fly-example-resource-provider-bundle** and **spi-fly-example-resource-client-bundle** - these bundles show that SPI Fly can be used to control the TCCL in OSGi for any situation, in this case applied to resource loading via the TCCL. The provider bundle provides a resource used by the Foo.doit() API. The client bundle contains Foo.doit() and in there calls Thread.getContextClassLoader().getResource() to obtain the resource. The TCCL has visibility of the provider bundle because both bundles have the appropriate values set in the SPI-Provider and SPI-Consumer headers. Source [here][33] and [here][34].

##More Information##
More information can be found at the following resources:

  - OSGi Service Loader Mediator specification. [OSGi Enterprise specification][35], Chapter 133.
  - OSGi Blog article: [http://blog.osgi.org/2013/02/javautilserviceloader-in-osgi.html][36]


  [1]: http://www.osgi.org/Download/Release5 "OSGi Enterprise Specification"
  [2]: http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.aries.spifly%22
  [3]: #usage
  [4]: http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22org.apache.aries.spifly.dynamic.bundle%22
  [5]: http://repo1.maven.org/maven2/org/apache/aries/org.apache.aries.util/1.1.1/org.apache.aries.util-1.1.1.jar
  [6]: http://repo1.maven.org/maven2/org/ow2/asm/asm-all/5.0.4/asm-all-5.0.4.jar
  [7]: http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22org.apache.aries.spifly.static.tool%22
  [8]: http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22org.apache.aries.spifly.static.bundle%22
  [9]: http://repo1.maven.org/maven2/org/apache/aries/org.apache.aries.util/1.1.1/org.apache.aries.util-1.1.1.jar
  [10]: http://svn.apache.org/repos/asf/aries/trunk/spi-fly
  [11]: http://www.osgi.org/Download/Release5 "OSGi Enterprise Specification"
  [12]: #examples
  [13]: #examples
  [14]: #examples
  [15]: #examples
  [16]: #releases
  [17]: http://search.maven.org/#artifactdetails%7Corg.ow2.asm%7Casm-all%7C5.0.4%7Cjar
  [18]: #releases
  [19]: #releases
  [20]: http://search.maven.org/remotecontent?filepath=org/apache/aries/spifly/examples/org.apache.aries.spifly.examples.spi.bundle/1.0.0/org.apache.aries.spifly.examples.spi.bundle-1.0.0.jar
  [21]: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-examples/spi-fly-example-spi-bundle/
  [22]: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-examples/spi-fly-example-provider1-jar/
  [23]: http://search.maven.org/remotecontent?filepath=org/apache/aries/spifly/examples/org.apache.aries.spifly.examples.provider1.bundle/1.0.0/org.apache.aries.spifly.examples.provider1.bundle-1.0.0.jar
  [24]: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-examples/spi-fly-example-provider1-bundle/
  [25]: http://search.maven.org/remotecontent?filepath=org/apache/aries/spifly/examples/org.apache.aries.spifly.examples.provider2.bundle/1.0.0/org.apache.aries.spifly.examples.provider2.bundle-1.0.0.jar
  [26]: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-examples/spi-fly-example-provider2-bundle/
  [27]: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-examples/spi-fly-example-client1-jar/
  [28]: http://search.maven.org/remotecontent?filepath=org/apache/aries/spifly/examples/org.apache.aries.spifly.examples.client1.bundle/1.0.0/org.apache.aries.spifly.examples.client1.bundle-1.0.0.jar
  [29]: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-examples/spi-fly-example-client1-bundle/
  [30]: http://search.maven.org/remotecontent?filepath=org/apache/aries/spifly/examples/org.apache.aries.spifly.examples.client2.bundle/1.0.0/org.apache.aries.spifly.examples.client2.bundle-1.0.0.jar
  [31]: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-examples/spi-fly-example-client2-bundle/
  [32]: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-examples/spi-fly-example-provider-consumer-bundle/
  [33]: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-examples/spi-fly-example-resource-provider-bundle/
  [34]: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-examples/spi-fly-example-resource-client-bundle/
  [35]: http://www.osgi.org/Download/Release5
  [36]: http://blog.osgi.org/2013/02/javautilserviceloader-in-osgi.html