manual/src/main/asciidoc/user-guide/monitoring.adoc - karaf - Git at Google

 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
 //
 //      http://www.apache.org/licenses/LICENSE-2.0
 //
 // Unless required by applicable law or agreed to in writing, software
 // distributed under the License is distributed on an "AS IS" BASIS,
 // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 // See the License for the specific language governing permissions and
 // limitations under the License.
 //

 === Monitoring and Management using JMX

 Apache Karaf provides a complete JMX layer.

 You can remotely connect to a running Apache Karaf instance using any JMX client (like jconsole).

 The Apache Karaf features provide a set of MBeans, dedicating to monitoring and management.

 ==== Connecting

 Apache Karaf exposes a complete MBean server that you can use remotely.

 The default port number is 1099.

 The JMX URL to use by default is:

 ----
 service:jmx:rmi:///jndi/rmi://localhost:1099/karaf-root
 ----

 If you don't need the remote JMX at all, users can remove

 ----
 -Dcom.sun.management.jmxremote
 ----

 from bin/karaf|bin/karaf.bat to avoid opening the RMI listening port.

 You have to provide an username and password to access the JMX layer.
 The JMX layer uses the security framework, and so, by default, it uses the users defined in `etc/users.properties`.

 You can change the port numbers of the JMX layer in the `etc/org.apache.karaf.management.cfg` configuration file.

 ==== Configuration

 The Apache Karaf JMX management layer is configured in the `etc/org.apache.karaf.management.cfg` configuration file:

 ----
 ################################################################################
 #
 #    Licensed to the Apache Software Foundation (ASF) under one or more
 #    contributor license agreements.  See the NOTICE file distributed with
 #    this work for additional information regarding copyright ownership.
 #    The ASF licenses this file to You under the Apache License, Version 2.0
 #    (the "License"); you may not use this file except in compliance with
 #    the License.  You may obtain a copy of the License at
 #
 #       http://www.apache.org/licenses/LICENSE-2.0
 #
 #    Unless required by applicable law or agreed to in writing, software
 #    distributed under the License is distributed on an "AS IS" BASIS,
 #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 #    See the License for the specific language governing permissions and
 #    limitations under the License.
 #
 ################################################################################

 #
 # The properties in this file define the configuration of Apache Karaf's JMX Management
 #

 #
 # Port number for RMI registry connection
 #
 rmiRegistryPort = 1099

 #
 # Port number for RMI server connection
 #
 rmiServerPort = 44444

 #
 # Name of the JAAS realm used for authentication
 #
 jmxRealm = karaf

 #
 # The service URL for the JMXConnectorServer
 #
 serviceUrl = service:jmx:rmi://0.0.0.0:${rmiServerPort}/jndi/rmi://0.0.0.0:${rmiRegistryPort}/karaf-${karaf.name}

 #
 # Whether any threads started for the JMXConnectorServer should be started as daemon threads
 #
 daemon = true

 #
 # Whether the JMXConnectorServer should be started in a separate thread
 #
 threaded = true

 #
 # The ObjectName used to register the JMXConnectorServer
 #
 objectName = connector:name=rmi

 #
 # Role name used for JMX access authorization
 #
 # jmxRole=admin
 ----

 * `rmiRegistryPort` property contains the port number of the JMX RMI registry. Default is `1099`.
 * `rmiServerPort` property contains the port number of the JMX RMI server. Default is `44444`.
 * `jmxRealm` is the security realm to use as authentication backend. By default it uses the `karaf` realm.

 ==== MBeans

 Apache Karaf provides a bunch of MBeans.

 The MBeans object names have the same format:

 ----
 org.apache.karaf:type=[feature],name=[instance]
 ----

 Installing additional Apache Karaf features and external applications can provide new MBeans.

 The following MBeans list is non exhaustive:

 * `org.apache.karaf:type=bundle,name=*`: management of the OSGi bundles.
 * `org.apache.karaf:type=config,name=*`: management of the configurations.
 * `org.apache.karaf:type=diagnostic,name=*`: creation of dumps containing the current Apache Karaf activity (used for diagnostic).
 * `org.apache.karaf:type=feature,name=*`: management of the Apache Karaf features.
 * `org.apache.karaf:type=http,name=*`: management of the HTTP service (provided by the `http` feature).
 * `org.apache.karaf:type=instance,name=*`: management of the instances.
 * `org.apache.karaf:type=jdbc,name=*`: management of the JDBC service (provided by the `jdbc` feature).
 * `org.apache.karaf:type=jms,name=*`: management of the JMS service (provided by the `jms` feature).
 * `org.apache.karaf:type=jndi,name=*`: management of the JNDI service (provided by the `jndi` feature).
 * `org.apache.karaf:type=kar,name=*`: management of the KAR file.
 * `org.apache.karaf:type=log,name=*`: management of the log service.
 * `org.apache.karaf:type=obr,name=*`: management of the OBR service (provided by the `obr` feature).
 * `org.apache.karaf:type=package,name=*`: details about packages exported/imported.
 * `org.apache.karaf:type=service,name=*`: management of the OSGi services.
 * `org.apache.karaf:type=system,name=*`: management of the Apache Karaf container itself (halt, restart, etc).
 * `org.apache.karaf:type=web,name=*`: management of WebApplications (provided by the `war` feature).
 * `org.apache.karaf:type=wrapper,name=*`: management of the service wrapper (provided by the `wrapper` feature).

 ==== RBAC

 Apache Karaf provides a complete Role-Based Access Control to the JMX MBeans and operations.

 Whenever a JMX operation is invoked, the roles of the user are checked against the required roles for this operation.

 The access lists are defined in configuration file in the `etc` folder.

 The relevant configuration is prefixed with `jmx.acl` and based on the JMX ObjectName that it applies to.

 For instance, specific configuration for a MBean with the object name `foo.bar:type=Test` can be placed in the
 `etc/jmx.acl.foo.bar.Test.cfg` configuration file.

 More generic configurations can be placed in the domain (e.g. jmx.acl.foo.bar.cfg) or at the top level (jmx.acl.cfg).

 A simple configuration file looks like:

 ----
     # operation = role
     test = admin
     getVal = manager, viewer
 ----

 Apache Karaf looks for required roles using the following process
 .
 The most specific configuration file is tried first. It means that in the previous example, the `etc/jmx.acl.foo.bar.Test.cfg` is looked at first.
 In this configuration, Apache Karaf looks for a:

 . Specific match for the invocation, e.g. `test(int)["17"] = role1`

 . Regex match for the invocation, e.g. `test(int)[/[0-9]/] = role2`
  In both cases, the passed argument is converted to a String for the comparison.
  If any of the above match, the search stops and the associated roles are used.

 . Signature match for the invocation, e.g. `test(int) = role3`
  If matched, the search stops and the associated roles are used.

 . Method name match for the invocation, e.g. `test = role4`
  If matched, the search stops and the associated roles are used.

 . A method name wildcard match, e.g. `te* = role5`
  For all the wildcard matches found in the current configuration file, the roles associated with the longest match are used.
  So if you have te* and * and the method invoked is 'test', then the roles defined with te* are used, not the ones defined with *.

 If no matching definition is found, the most specific definition always takes the precedence.

 You can find some configuration examples:

 * Only a `manager` can call GC on the Memory MBean:

 ----
 # etc/jmx.acl.java.lang.Memory.cfg
     gc = manager
 ----

 * Bundles with ID between 0 and 49 can be stopped only by an `admin`, other bundles can be stopped by a `manager`:

 ----
 # etc/jmx.acl.org.apache.karaf.bundles.cfg
     stop(java.lang.String)[/([1-4])?([0-9]/] = admin
     stop = manager
 ----

 The `etc/jmx.acl.cfg` configuration file contains the global configuration for the invocation on any MBean that
 doesn't have a specific configuration:

 ----
 # etc/jmx.acl.cfg
     list* = viewer
     get* = viewer
     is* = viewer
     set* = admin
     * = admin
 ----

 By default, all "read-only" operations (`list*`, `get*`, `is*`) can be performed by a `viewer`, whereas the "read-write" operations can be performed only by an `admin`.

 The `org.apache.karaf:type=security,area=jmx` MBean can be used to check whether the current user can access a certain MBean or invoke a specific operation on it.
 This MBean can be used by management clients (monitoring tools, etc) to decide whether to show certain MBeans or operations to the end user.

 ==== JMX-HTTP bridge with Jolokia

 It's not always easy to use a JMX client with the RMI protocol.

 Some monitoring tools (Nagios, Zabbix, ...) are not native JMX clients.

 But most of them can use HTTP.

 More over, you may want to write your own application/web application. In that case, HTTP and JSON can be very interesting and easy to remotely manage Apache Karaf.

 http://www.jolokia.org/[Jolokia] can be installed in Apache Karaf as a remote JMX-HTTP bridge.

 Karaf provides a jolokia feature, ready to install:

 ----
 karaf@root()> feature:install jolokia
 ----

 By default, Jolokia is listening on the port `8181` (see the link:webcontainer[WebContainer (JSP/Servlet)] page for details about the HTTP configuration).

 If you point a browser on http://localhost:8181/jolokia you will see a JSON output like:

 ----
 {"timestamp":1421765829,"status":200,"request":{"type":"version"},"value":{"protocol":"7.2","config":{"useRestrictorService":"false","canonicalNaming":"true","includeStackTrace":"true","listenForHttpService":"true","historyMaxEntries":"10","agentId":"192.168.134.10-5922-6eb8d517-osgi","debug":"false","realm":"karaf","serializeException":"false","agentContext":"\/jolokia","agentType":"servlet","policyLocation":"classpath:\/jolokia-access.xml","user":"karaf","debugMaxEntries":"100","authMode":"jaas","mimeType":"text\/plain"},"agent":"1.2.4-SNAPSHOT","info":{"product":"felix","vendor":"Apache","version":"4.4.1"`}
 ----

 You can manipulate the Apache Karaf JMX layer via HTTP and JSON, via system tools (like `curl`, `jmx4perl`, monitoring tools (supporting HTTP/JSON), or web applications.

 For instance, you can send a JSON request to get details about the current Apache Karaf heap memory usage.

 The format of the request is:

 ----
 {
     "type":"read",
     "mbean":"java.lang:type=Memory",
     "attribute":"HeapMemoryUsage",
     "path":"used"
 }
 ----

 We can send this JSON request using `curl` and get the result:

 ----
 curl -u karaf -d "{\"type\":\"read\",\"mbean\":\"java.lang:type=Memory\",\"attribute\":\"HeapMemoryUsage\",\"path\":\"used\"}" http://localhost:8181/jolokia/ && echo ""
 Enter host password for user 'karaf':
 {"timestamp":1421765948,"status":200,"request":{"mbean":"java.lang:type=Memory","path":"used","attribute":"HeapMemoryUsage","type":"read"},"value":69121000}
 ----

 You can find details on the http://www.jolokia.org[Jolokia website] and in the http://www.jolokia.org/reference/html/[documentation].

 ==== Apache Karaf Decanter

 Apache Karaf Decanter provides a complete monitoring solution including data history, turnkey dashboards, SLA and alerting support.
	//
	// Licensed under the Apache License, Version 2.0 (the "License");
	// you may not use this file except in compliance with the License.
	// You may obtain a copy of the License at
	//
	// http://www.apache.org/licenses/LICENSE-2.0
	//
	// Unless required by applicable law or agreed to in writing, software
	// distributed under the License is distributed on an "AS IS" BASIS,
	// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
	// See the License for the specific language governing permissions and
	// limitations under the License.
	//

	=== Monitoring and Management using JMX

	Apache Karaf provides a complete JMX layer.

	You can remotely connect to a running Apache Karaf instance using any JMX client (like jconsole).

	The Apache Karaf features provide a set of MBeans, dedicating to monitoring and management.

	==== Connecting

	Apache Karaf exposes a complete MBean server that you can use remotely.

	The default port number is 1099.

	The JMX URL to use by default is:

	----
	service:jmx:rmi:///jndi/rmi://localhost:1099/karaf-root
	----

	If you don't need the remote JMX at all, users can remove

	----
	-Dcom.sun.management.jmxremote
	----

	from bin/karaf\|bin/karaf.bat to avoid opening the RMI listening port.

	You have to provide an username and password to access the JMX layer.
	The JMX layer uses the security framework, and so, by default, it uses the users defined in `etc/users.properties`.

	You can change the port numbers of the JMX layer in the `etc/org.apache.karaf.management.cfg` configuration file.

	==== Configuration

	The Apache Karaf JMX management layer is configured in the `etc/org.apache.karaf.management.cfg` configuration file:

	----
	################################################################################
	#
	# Licensed to the Apache Software Foundation (ASF) under one or more
	# contributor license agreements. See the NOTICE file distributed with
	# this work for additional information regarding copyright ownership.
	# The ASF licenses this file to You under the Apache License, Version 2.0
	# (the "License"); you may not use this file except in compliance with
	# the License. You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#
	# Unless required by applicable law or agreed to in writing, software
	# distributed under the License is distributed on an "AS IS" BASIS,
	# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
	# See the License for the specific language governing permissions and
	# limitations under the License.
	#
	################################################################################

	#
	# The properties in this file define the configuration of Apache Karaf's JMX Management
	#

	#
	# Port number for RMI registry connection
	#
	rmiRegistryPort = 1099

	#
	# Port number for RMI server connection
	#
	rmiServerPort = 44444

	#
	# Name of the JAAS realm used for authentication
	#
	jmxRealm = karaf

	#
	# The service URL for the JMXConnectorServer
	#
	serviceUrl = service:jmx:rmi://0.0.0.0:${rmiServerPort}/jndi/rmi://0.0.0.0:${rmiRegistryPort}/karaf-${karaf.name}

	#
	# Whether any threads started for the JMXConnectorServer should be started as daemon threads
	#
	daemon = true

	#
	# Whether the JMXConnectorServer should be started in a separate thread
	#
	threaded = true

	#
	# The ObjectName used to register the JMXConnectorServer
	#
	objectName = connector:name=rmi

	#
	# Role name used for JMX access authorization
	#
	# jmxRole=admin
	----

	* `rmiRegistryPort` property contains the port number of the JMX RMI registry. Default is `1099`.
	* `rmiServerPort` property contains the port number of the JMX RMI server. Default is `44444`.
	* `jmxRealm` is the security realm to use as authentication backend. By default it uses the `karaf` realm.

	==== MBeans

	Apache Karaf provides a bunch of MBeans.

	The MBeans object names have the same format:

	----
	org.apache.karaf:type=[feature],name=[instance]
	----

	Installing additional Apache Karaf features and external applications can provide new MBeans.

	The following MBeans list is non exhaustive:

	* `org.apache.karaf:type=bundle,name=*`: management of the OSGi bundles.
	* `org.apache.karaf:type=config,name=*`: management of the configurations.
	* `org.apache.karaf:type=diagnostic,name=*`: creation of dumps containing the current Apache Karaf activity (used for diagnostic).
	* `org.apache.karaf:type=feature,name=*`: management of the Apache Karaf features.
	* `org.apache.karaf:type=http,name=*`: management of the HTTP service (provided by the `http` feature).
	* `org.apache.karaf:type=instance,name=*`: management of the instances.
	* `org.apache.karaf:type=jdbc,name=*`: management of the JDBC service (provided by the `jdbc` feature).
	* `org.apache.karaf:type=jms,name=*`: management of the JMS service (provided by the `jms` feature).
	* `org.apache.karaf:type=jndi,name=*`: management of the JNDI service (provided by the `jndi` feature).
	* `org.apache.karaf:type=kar,name=*`: management of the KAR file.
	* `org.apache.karaf:type=log,name=*`: management of the log service.
	* `org.apache.karaf:type=obr,name=*`: management of the OBR service (provided by the `obr` feature).
	* `org.apache.karaf:type=package,name=*`: details about packages exported/imported.
	* `org.apache.karaf:type=service,name=*`: management of the OSGi services.
	* `org.apache.karaf:type=system,name=*`: management of the Apache Karaf container itself (halt, restart, etc).
	* `org.apache.karaf:type=web,name=*`: management of WebApplications (provided by the `war` feature).
	* `org.apache.karaf:type=wrapper,name=*`: management of the service wrapper (provided by the `wrapper` feature).

	==== RBAC

	Apache Karaf provides a complete Role-Based Access Control to the JMX MBeans and operations.

	Whenever a JMX operation is invoked, the roles of the user are checked against the required roles for this operation.

	The access lists are defined in configuration file in the `etc` folder.

	The relevant configuration is prefixed with `jmx.acl` and based on the JMX ObjectName that it applies to.

	For instance, specific configuration for a MBean with the object name `foo.bar:type=Test` can be placed in the
	`etc/jmx.acl.foo.bar.Test.cfg` configuration file.

	More generic configurations can be placed in the domain (e.g. jmx.acl.foo.bar.cfg) or at the top level (jmx.acl.cfg).

	A simple configuration file looks like:

	----
	# operation = role
	test = admin
	getVal = manager, viewer
	----

	Apache Karaf looks for required roles using the following process
	.
	The most specific configuration file is tried first. It means that in the previous example, the `etc/jmx.acl.foo.bar.Test.cfg` is looked at first.
	In this configuration, Apache Karaf looks for a:

	. Specific match for the invocation, e.g. `test(int)["17"] = role1`

	. Regex match for the invocation, e.g. `test(int)[/[0-9]/] = role2`
	In both cases, the passed argument is converted to a String for the comparison.
	If any of the above match, the search stops and the associated roles are used.

	. Signature match for the invocation, e.g. `test(int) = role3`
	If matched, the search stops and the associated roles are used.

	. Method name match for the invocation, e.g. `test = role4`
	If matched, the search stops and the associated roles are used.

	. A method name wildcard match, e.g. `te* = role5`
	For all the wildcard matches found in the current configuration file, the roles associated with the longest match are used.
	So if you have te* and * and the method invoked is 'test', then the roles defined with te* are used, not the ones defined with *.

	If no matching definition is found, the most specific definition always takes the precedence.

	You can find some configuration examples:

	* Only a `manager` can call GC on the Memory MBean:

	----
	# etc/jmx.acl.java.lang.Memory.cfg
	gc = manager
	----

	* Bundles with ID between 0 and 49 can be stopped only by an `admin`, other bundles can be stopped by a `manager`:

	----
	# etc/jmx.acl.org.apache.karaf.bundles.cfg
	stop(java.lang.String)[/([1-4])?([0-9]/] = admin
	stop = manager
	----

	The `etc/jmx.acl.cfg` configuration file contains the global configuration for the invocation on any MBean that
	doesn't have a specific configuration:

	----
	# etc/jmx.acl.cfg
	list* = viewer
	get* = viewer
	is* = viewer
	set* = admin
	* = admin
	----

	By default, all "read-only" operations (`list`, `get`, `is*`) can be performed by a `viewer`, whereas the "read-write" operations can be performed only by an `admin`.

	The `org.apache.karaf:type=security,area=jmx` MBean can be used to check whether the current user can access a certain MBean or invoke a specific operation on it.
	This MBean can be used by management clients (monitoring tools, etc) to decide whether to show certain MBeans or operations to the end user.

	==== JMX-HTTP bridge with Jolokia

	It's not always easy to use a JMX client with the RMI protocol.

	Some monitoring tools (Nagios, Zabbix, ...) are not native JMX clients.

	But most of them can use HTTP.

	More over, you may want to write your own application/web application. In that case, HTTP and JSON can be very interesting and easy to remotely manage Apache Karaf.

	http://www.jolokia.org/[Jolokia] can be installed in Apache Karaf as a remote JMX-HTTP bridge.

	Karaf provides a jolokia feature, ready to install:

	----
	karaf@root()> feature:install jolokia
	----

	By default, Jolokia is listening on the port `8181` (see the link:webcontainer[WebContainer (JSP/Servlet)] page for details about the HTTP configuration).

	If you point a browser on http://localhost:8181/jolokia you will see a JSON output like:

	----
	{"timestamp":1421765829,"status":200,"request":{"type":"version"},"value":{"protocol":"7.2","config":{"useRestrictorService":"false","canonicalNaming":"true","includeStackTrace":"true","listenForHttpService":"true","historyMaxEntries":"10","agentId":"192.168.134.10-5922-6eb8d517-osgi","debug":"false","realm":"karaf","serializeException":"false","agentContext":"\/jolokia","agentType":"servlet","policyLocation":"classpath:\/jolokia-access.xml","user":"karaf","debugMaxEntries":"100","authMode":"jaas","mimeType":"text\/plain"},"agent":"1.2.4-SNAPSHOT","info":{"product":"felix","vendor":"Apache","version":"4.4.1"`}
	----

	You can manipulate the Apache Karaf JMX layer via HTTP and JSON, via system tools (like `curl`, `jmx4perl`, monitoring tools (supporting HTTP/JSON), or web applications.

	For instance, you can send a JSON request to get details about the current Apache Karaf heap memory usage.

	The format of the request is:

	----
	{
	"type":"read",
	"mbean":"java.lang:type=Memory",
	"attribute":"HeapMemoryUsage",
	"path":"used"
	}
	----

	We can send this JSON request using `curl` and get the result:

	----
	curl -u karaf -d "{\"type\":\"read\",\"mbean\":\"java.lang:type=Memory\",\"attribute\":\"HeapMemoryUsage\",\"path\":\"used\"}" http://localhost:8181/jolokia/ && echo ""
	Enter host password for user 'karaf':
	{"timestamp":1421765948,"status":200,"request":{"mbean":"java.lang:type=Memory","path":"used","attribute":"HeapMemoryUsage","type":"read"},"value":69121000}
	----

	You can find details on the http://www.jolokia.org[Jolokia website] and in the http://www.jolokia.org/reference/html/[documentation].

	==== Apache Karaf Decanter

	Apache Karaf Decanter provides a complete monitoring solution including data history, turnkey dashboards, SLA and alerting support.