h1. Failover Deployments

Karaf provides failover capability using either a simple lock file system or a JDBC locking mechanism. In both cases, a container-level lock system allows bundles to be preloaded into the slave Karaf instance in order to provide faster failover performance.

h2. Simple lock file

The simple lock file mechanism is intended for failover configurations where instances reside on the same host machine.

To use this feature, edit the {{$KARAF_HOME/etc/system.properties}} file as follows on each system in the master/slave setup:

{noformat}
karaf.lock=true
karaf.lock.class=org.apache.karaf.main.SimpleFileLock
karaf.lock.dir=<PathToLockFileDirectory>
karaf.lock.delay=10
{noformat}

*Note*: Ensure that the {{karaf.lock.dir}} property points to the same directory for both the master and slave instance, so that the slave can acquire the lock only when the master releases it.


h2. JDBC locking

The JDBC locking mechanism is intended for failover configurations where instances exist on separate machines. In this deployment, the master instance holds a lock on a Karaf locking table hosted on a database. If the master loses the lock, a waiting slave process gains access to the locking table and fully starts its container. 

To use this feature, do the following on each system in the master/slave setup:

* Update the classpath to include the JDBC driver
* Update the {{$KARAF_HOME/bin/karaf}} script to have a unique JMX remote port set if instances reside on the same host
* Update the {{$KARAF_HOME/etc/system.properties}} file as follows:

{noformat}
karaf.lock=true
karaf.lock.class=org.apache.karaf.main.DefaultJDBCLock
karaf.lock.level=50
karaf.lock.delay=10
karaf.lock.jdbc.url=jdbc:derby://dbserver:1527/sample
karaf.lock.jdbc.driver=org.apache.derby.jdbc.ClientDriver
karaf.lock.jdbc.user=user
karaf.lock.jdbc.password=password
karaf.lock.jdbc.table=KARAF_LOCK
karaf.lock.jdbc.clustername=karaf
karaf.lock.jdbc.timeout=30
{noformat}

*Note*:

* This process will fail if a JDBC driver is not on the classpath.
* The "sample" database referred to above will be created if it does not exist.
* The first Karaf instance to acquire the locking table is the master instance.
* If the connection to the database is lost, the master instance tries to gracefully shutdown, allowing a slave instance to become master when the database service is restored. The former master will require a manual restart.

h3. JDBC locking on Oracle

If you are using Oracle as your database for JDBC locking, the {{karaf.lock.class}} property in the {{$KARAF_HOME/etc/system.properties}} file must point to {{org.apache.karaf.main.OracleJDBCLock}}.

Otherwise, configure the system.properties file as normal for your setup, for example:

{noformat}
karaf.lock=true
karaf.lock.class=org.apache.karaf.main.OracleJDBCLock
karaf.lock.jdbc.url=jdbc:oracle:thin:@hostname:1521:XE
karaf.lock.jdbc.driver=oracle.jdbc.OracleDriver
karaf.lock.jdbc.user=user
karaf.lock.jdbc.password=password
karaf.lock.jdbc.table=KARAF_LOCK
karaf.lock.jdbc.clustername=karaf
karaf.lock.jdbc.timeout=30
{noformat}

As with the default JDBC locking setup, the Oracle JDBC driver JAR file must be in your classpath. You can ensure this by copying the {{ojdbc14.jar}} into Karaf's {{lib}} folder before starting Karaf.

*Note*: The {{karaf.lock.jdbc.url}} requires an active SID, which means you must manually create a database instance before using this particular lock.

h3. Derby

{warning}
TODO
{warning}

h3. MySQL

{warning}
TODO
{warning}


{anchor:locklevel}

h2. Container-level locking

Container-level locking allows bundles to be preloaded into the slave kernel instance in order to provide faster failover performance. Container-level locking is supported in both the simple file and JDBC locking mechanisms.

To implement container-level locking, add the following to the {{$KARAF_HOME/etc/system.properties}} file on each system in the master/slave setup:

{noformat}
karaf.lock=true
karaf.lock.level=50
karaf.lock.delay=10
{noformat}

The {{karaf.lock.level}} property tells the Karaf instance how far into the boot process to bring the OSGi container. All bundles with an ID equal or lower to this start level will be started in that Karaf instance.

Bundle start levels are specified in {{$KARAF_HOME/etc/startup.properties}}, in the format {{jar.name=level}}. The core system bundles have levels below 50, where user bundles have levels greater than 50.

|| Level || Behavior ||
| 1 | A 'cold' standby instance. Core bundles are not loaded into container. Slaves will wait until lock acquired to start server. |
| <50 | A 'hot' standby instance. Core bundles are loaded into the container. Slaves will wait until lock acquired to start user level bundles. The console will be accessible for each slave instance at this level. |
| >50 | This setting is not recommended as user bundles will end up being started. |

*Note*: When using a 'hot' spare on the same host you need to set the JMX remote port to a unique value to avoid bind conflicts. You can edit the Karaf start script to include the following:

{noformat}
DEFAULT_JAVA_OPTS="-server $DEFAULT_JAVA_OPTS -Dcom.sun.management.jmxremote.port=1100 -Dcom.sun.management.jmxremote.authenticate=false"
{noformat}