blob: 957371232a49af62cd5164df66a8126378586cbd [file] [log] [blame] [view]
<!--
{% comment %}
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to you under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
{% endcomment %}
-->
#**A manual for hadoop plugin of collectd**
Collectd website referto [collectd.org](collectd.org)
###Description
The plugin collect information from http://hostname:port/jmx , according to the hadoop role configuration, it support roles as below:
HDFS NameNode
HDFS DataNode
HDFS JournalNode
HBase Master
Hbase RegionServer
Yarn NodeManager
Yarn ResourceManager
###Install
>1)
Deploy hadoop.py in your collectd plugins path , in my environment, it is like : /opt/collectd/lib/collectd/plugins/hadoop.py(It assume that you install collectd in the directory of /opt/collectd), maybe you should create the plugins directory ahead.
>2)
Configure the python plugin in file collectd.conf , and you should trun on the python switch first.
>A snippet of collectd.conf for showing hadoop python plugin configuration:<br/>
>
LoadPlugin python
<Plugin "python">
ModulePath "/opt/collectd/lib/collectd/plugins/"
LogTraces true
Import "hadoop"
<Module "hadoop">
HDFSDatanodeHost "YourHostName"
Port "50075"
Verbose true
Instance "192.168.xxx.xxx"
JsonPath "/xxx/xxx/hadooproleconfig.json"
</Module>
<Module "hadoop">
YARNResourceManager "YourHostName"
Port "8088"
Verbose true
Instance "192.168.xxx.xxx"
</Module>
</Plugin>
>**Notification**:
>Instance , port and host(role) fileds must be set.
>3)Two way to cite hadooproleconfig.json , either is ok.<br/>
>>a)Place hadooproleconfig.json path in one <Module "hadoop"> </Module> pair in collectd.conf.<br/>
>>b)Place hadooproleconfig.json file in you BaseDir which defined in collectd.conf.
>4)
If you update your collectd.conf or hadooproleconfig.json, you should restart your collectd application.
###Dependency
collectd: I test in version 5.6.0.
Hadoop: I test in version 2.6.0-cdh5.4.3.
Hbase: I test in version 1.0.0-chd5.4.3.
###Testing
>You can use hadoop.py as a single independent python file for debugging jmx, and you can also use it as plugin for collectd, a variable MyDebug is used as a swtich when used in the two different environments.