blob: 5700f71861da050ad1189ddfeb924ca819cad464 [file] [log] [blame]
<!DOCTYPE html>
<!--
| Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-interface/metron-rest/index.md at 2019-05-14
| Rendered using Apache Maven Fluido Skin 1.7
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="Date-Revision-yyyymmdd" content="20190514" />
<meta http-equiv="Content-Language" content="en" />
<title>Metron &#x2013; Metron REST</title>
<link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" />
<link rel="stylesheet" href="../../css/site.css" />
<link rel="stylesheet" href="../../css/print.css" media="print" />
<script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script>
<script type="text/javascript">
$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );
</script>
</head>
<body class="topBarDisabled">
<div class="container-fluid">
<div id="banner">
<div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../images/metron-logo.png" alt="Apache Metron" width="148px" height="48px"/></a></div>
<div class="pull-right"></div>
<div class="clear"><hr/></div>
</div>
<div id="breadcrumbs">
<ul class="breadcrumb">
<li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li>
<li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li>
<li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li>
<li class="active ">Metron REST</li>
<li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2019-05-14</li>
<li id="projectVersion" class="pull-right">Version: 0.7.1</li>
</ul>
</div>
<div class="row-fluid">
<div id="leftColumn" class="span2">
<div class="well sidebar-nav">
<ul class="nav nav-list">
<li class="nav-header">User Documentation</li>
<li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a>
<ul class="nav nav-list">
<li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li>
<li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li>
<li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li>
<li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li>
<li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li>
<li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li>
<li><a href="../../metron-interface/index.html" title="Interface"><span class="icon-chevron-down"></span>Interface</a>
<ul class="nav nav-list">
<li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li>
<li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li>
<li class="active"><a href="#"><span class="none"></span>Rest</a></li>
</ul>
</li>
<li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li>
<li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li>
<li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li>
<li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li>
<li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li>
<li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li>
</ul>
</li>
</ul>
<hr />
<div id="poweredBy">
<div class="clear"></div>
<div class="clear"></div>
<div class="clear"></div>
<div class="clear"></div>
<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a>
</div>
</div>
</div>
<div id="bodyColumn" class="span10" >
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<h1>Metron REST</h1>
<p><a name="Metron_REST"></a></p>
<p>This module provides a RESTful API for interacting with Metron.</p>
<ul>
<li><a href="#Prerequisites">Prerequisites</a></li>
<li><a href="#Installation">Installation</a></li>
<li><a href="#Configuration">Configuration</a></li>
<li><a href="#Usage">Usage</a></li>
<li><a href="#Security">Security</a></li>
<li><a href="#API">API</a></li>
<li><a href="#Testing">Testing</a></li>
</ul>
<div class="section">
<h2><a name="Prerequisites"></a>Prerequisites</h2>
<ul>
<li>A running Metron cluster</li>
<li>A running real-time store, either Elasticsearch or Solr depending on which one is enabled</li>
<li>Java 8 installed</li>
<li>Storm CLI and Metron topology scripts (start_parser_topology.sh, start_enrichment_topology.sh, start_elasticsearch_topology.sh) installed</li>
<li>A relational database</li>
</ul></div>
<div class="section">
<h2><a name="Installation"></a>Installation</h2>
<div class="section">
<h3><a name="From_Source"></a>From Source</h3>
<ol style="list-style-type: decimal">
<li>
<p>Package the application with Maven:</p>
<div>
<div>
<pre class="source">mvn clean package
</pre></div></div>
</li>
<li>
<p>Untar the archive in the $METRON_HOME directory. The directory structure will look like:</p>
<div>
<div>
<pre class="source">config
rest_application.yml
bin
metron-rest
lib
metron-rest-$METRON_VERSION.jar
</pre></div></div>
</li>
<li>
<p>Copy the <tt>$METRON_HOME/bin/metron-rest</tt> script to <tt>/etc/init.d/metron-rest</tt></p>
</li>
</ol></div>
<div class="section">
<h3><a name="From_Package_Manager"></a>From Package Manager</h3>
<ol style="list-style-type: decimal">
<li>
<p>Deploy the RPM at <tt>/metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-rest-$METRON_VERSION-*.noarch.rpm</tt></p>
</li>
<li>
<p>Install the RPM with:</p>
<div>
<div>
<pre class="source">rpm -ih metron-rest-$METRON_VERSION-*.noarch.rpm
</pre></div></div>
</li>
</ol></div></div>
<div class="section">
<h2><a name="Configuration"></a>Configuration</h2>
<p>The REST application depends on several configuration parameters:</p>
<div class="section">
<h3><a name="REQUIRED"></a>REQUIRED</h3>
<p>No optional parameter has a default.</p>
<table border="0" class="table table-striped">
<thead>
<tr class="a">
<th> Environment Variable </th>
<th> Description</th></tr>
</thead><tbody>
<tr class="b">
<td> ZOOKEEPER </td>
<td> Zookeeper quorum (ex. node1:2181,node2:2181)</td></tr>
<tr class="a">
<td> BROKERLIST </td>
<td> Kafka Broker list (ex. node1:6667,node2:6667)</td></tr>
<tr class="b">
<td> HDFS_URL </td>
<td> HDFS url or <tt>fs.defaultFS</tt> Hadoop setting (ex. <a class="externalLink" href="hdfs://node1:8020">hdfs://node1:8020</a>)</td></tr>
</tbody>
</table></div>
<div class="section">
<h3><a name="Optional_-_With_Defaults"></a>Optional - With Defaults</h3>
<table border="0" class="table table-striped">
<thead>
<tr class="a">
<th> Environment Variable </th>
<th> Description </th>
<th> Required </th>
<th> Default</th></tr>
</thead><tbody>
<tr class="b">
<td> METRON_LOG_DIR </td>
<td> Directory where the log file is written </td>
<td> Optional </td>
<td> /var/log/metron/</td></tr>
<tr class="a">
<td> METRON_PID_FILE </td>
<td> File where the pid is written </td>
<td> Optional </td>
<td> /var/run/metron/</td></tr>
<tr class="b">
<td> METRON_REST_PORT </td>
<td> REST application port </td>
<td> Optional </td>
<td> 8082</td></tr>
<tr class="a">
<td> METRON_JDBC_CLIENT_PATH </td>
<td> Path to JDBC client jar </td>
<td> Optional </td>
<td> H2 is bundled</td></tr>
<tr class="b">
<td> METRON_TEMP_GROK_PATH </td>
<td> Temporary directory used to test grok statements </td>
<td> Optional </td>
<td> ./patterns/temp</td></tr>
<tr class="a">
<td> METRON_DEFAULT_GROK_PATH </td>
<td> Defaults HDFS directory used to store grok statements </td>
<td> Optional </td>
<td> /apps/metron/patterns</td></tr>
<tr class="b">
<td> SECURITY_ENABLED </td>
<td> Enables Kerberos support </td>
<td> Optional </td>
<td> false</td></tr>
<tr class="a">
<td> METRON_USER_ROLE </td>
<td> Name of the role at the authentication provider that provides user access to Metron. </td>
<td> Optional </td>
<td> USER</td></tr>
<tr class="b">
<td> METRON_ADMIN_ROLE </td>
<td> Name of the role at the authentication provider that provides administrative access to Metron.</td>
<td> Optional </td>
<td> ADMIN</td></tr>
</tbody>
</table></div>
<div class="section">
<h3><a name="Optional_-_Blank_Defaults"></a>Optional - Blank Defaults</h3>
<table border="0" class="table table-striped">
<thead>
<tr class="a">
<th> Environment Variable </th>
<th> Description </th>
<th> Required</th></tr>
</thead><tbody>
<tr class="b">
<td> METRON_JDBC_DRIVER </td>
<td> JDBC driver class </td>
<td> Optional</td></tr>
<tr class="a">
<td> METRON_JDBC_URL </td>
<td> JDBC url </td>
<td> Optional</td></tr>
<tr class="b">
<td> METRON_JDBC_USERNAME </td>
<td> JDBC username </td>
<td> Optional</td></tr>
<tr class="a">
<td> METRON_JDBC_PLATFORM </td>
<td> JDBC platform (one of h2, mysql, postgres, oracle) </td>
<td> Optional</td></tr>
<tr class="b">
<td> METRON_JVMFLAGS </td>
<td> JVM flags added to the start command </td>
<td> Optional</td></tr>
<tr class="a">
<td> METRON_SPRING_PROFILES_ACTIVE </td>
<td> Active Spring profiles (see <a href="#spring-profiles">below</a>) </td>
<td> Optional</td></tr>
<tr class="b">
<td> METRON_SPRING_OPTIONS </td>
<td> Additional Spring input parameters </td>
<td> Optional</td></tr>
<tr class="a">
<td> METRON_PRINCIPAL_NAME </td>
<td> Kerberos principal for the metron user </td>
<td> Optional</td></tr>
<tr class="b">
<td> METRON_SERVICE_KEYTAB </td>
<td> Path to the Kerberos keytab for the metron user </td>
<td> Optional</td></tr>
</tbody>
</table>
<p>These are set in the <tt>/etc/default/metron</tt> file.</p></div></div>
<div class="section">
<h2><a name="Usage"></a>Usage</h2>
<p>The REST application can be accessed with the Swagger UI at <a class="externalLink" href="http://host:port/swagger-ui.html#/">http://host:port/swagger-ui.html#/</a>. The default port is 8082.</p>
<div class="section">
<h3><a name="Logging"></a>Logging</h3>
<p>Logging for the REST application can be configured in Ambari. Log levels can be changed at the root, package and class level:</p>
<ol style="list-style-type: decimal">
<li>
<p>Navigate to Services &gt; Metron &gt; Configs &gt; REST and locate the <tt>Metron Spring options</tt> setting.</p>
</li>
<li>
<p>Logging configuration is exposed through Spring properties as explained <a class="externalLink" href="https://docs.spring.io/spring-boot/docs/current/reference/html/howto-logging.html#howto-logging">here</a>.</p>
</li>
<li>
<p>The root logging level defaults to ERROR but can be changed to INFO by adding <tt>--logging.level.root=INFO</tt> to the <tt>Metron Spring options</tt> setting.</p>
</li>
<li>
<p>The Metron REST logging level can be changed to INFO by adding <tt>--logging.level.org.apache.metron.rest=INFO</tt>.</p>
</li>
<li>
<p>HTTP request and response logging can be enabled by adding <tt>--logging.level.org.springframework.web.filter.CommonsRequestLoggingFilter=DEBUG --logging.level.org.apache.metron.rest.web.filter.ResponseLoggingFilter=DEBUG</tt>.</p>
</li>
</ol></div>
<div class="section">
<h3><a name="Spring_Profiles"></a>Spring Profiles</h3>
<p>The REST application comes with a few <a class="externalLink" href="http://docs.spring.io/autorepo/docs/spring-boot/current/reference/html/boot-features-profiles.html">Spring Profiles</a> to aid in testing and development.</p>
<table border="0" class="table table-striped">
<thead>
<tr class="a">
<th> Profile </th>
<th> Description </th></tr>
</thead><tbody>
<tr class="b">
<td> test </td>
<td> adds test users <tt>[user, user1, user2, admin]</tt> to the database with password &#x201c;<tt>password</tt>&#x201d;. sets variables to in-memory services, only used for integration testing </td></tr>
<tr class="a">
<td> dev </td>
<td> adds test users <tt>[user, user1, user2, admin]</tt> to the database with password &#x201c;<tt>password</tt>&#x201d; </td></tr>
<tr class="b">
<td> vagrant </td>
<td> sets configuration variables to match the Metron vagrant environment </td></tr>
<tr class="a">
<td> docker </td>
<td> sets configuration variables to match the Metron docker environment </td></tr>
</tbody>
</table>
<p>Setting active profiles is done with the METRON_SPRING_PROFILES_ACTIVE variable. For example, set this variable in <tt>/etc/default/metron</tt> to configure the REST application for the Vagrant environment and add a test user:</p>
<div>
<div>
<pre class="source">METRON_SPRING_PROFILES_ACTIVE=&quot;vagrant,dev&quot;
</pre></div></div>
</div></div>
<div class="section">
<h2><a name="Security"></a>Security</h2>
<ul>
<li><a href="#Kerberos">Kerberos</a></li>
<li><a href="#LDAP_Authentication">LDAP Authentication</a></li>
<li><a href="#JDBC_Authentication">JDBC Authentication</a></li>
</ul>
<div class="section">
<h3><a name="Kerberos"></a>Kerberos</h3>
<p>Metron REST can be configured for a cluster with Kerberos enabled. A client JAAS file is required for Kafka and Zookeeper and a Kerberos keytab for the metron user principal is required for all other services. Configure these settings in the <tt>/etc/default/metron</tt> file:</p>
<div>
<div>
<pre class="source">SECURITY_ENABLED=true
METRON_JVMFLAGS=&quot;-Djava.security.auth.login.config=$METRON_HOME/client_jaas.conf&quot;
METRON_PRINCIPAL_NAME=&quot;metron@EXAMPLE.COM&quot;
METRON_SERVICE_KEYTAB=&quot;/etc/security/keytabs/metron.keytab&quot;
</pre></div></div>
</div>
<div class="section">
<h3><a name="LDAP_Authentication"></a>LDAP Authentication</h3>
<p>Metron REST can be configured to use LDAP for authentication and roles. Use the following steps to enable LDAP.</p>
<ol style="list-style-type: decimal">
<li>
<p>In Ambari, go to Metron &gt; Config &gt; Security &gt; Roles</p>
<ul>
<li>
<p>Set &#x201c;User Role Name&#x201d; to the name of the role at the authentication provider that provides user level access to Metron.</p>
</li>
<li>
<p>Set &#x201c;Admin Role Name&#x201d; to the name of the role at the authentication provider that provides administrative access to Metron.</p>
</li>
</ul>
</li>
<li>
<p>In Ambari, go to Metron &gt; Config &gt; Security &gt; LDAP</p>
<ul>
<li>
<p>Turn on LDAP using the toggle.</p>
</li>
<li>
<p>Set &#x201c;LDAP URL&#x201d; to your LDAP instance. For example, <tt>ldap://&lt;host&gt;:&lt;port&gt;</tt>.</p>
</li>
<li>
<p>Set &#x201c;Bind User&#x201d; to the name of the bind user. For example, <tt>cn=admin,dc=apache,dc=org</tt>.</p>
</li>
<li>
<p>Set the &#x201c;Bind User Password&#x201d;</p>
</li>
<li>
<p>Other fields may be required depending on your LDAP configuration.</p>
</li>
</ul>
</li>
<li>
<p>Save the changes and restart the required services.</p>
</li>
</ol>
<p>By default, configuration will default to matching Knox&#x2019;s Demo LDAP for convenience. This should only be used for development purposes. Manual instructions for setting up demo LDAP and finalizing configuration (e.g. setting up the user LDIF file) can be found in the <a href="../../metron-deployment/development/index.html#knox-demo-ldap">Development README</a>.</p>
<div class="section">
<h4><a name="LDAPS"></a>LDAPS</h4>
<p>There is configuration to provide a path to a truststore with SSL certificates and provide a password. Users should import certificates as needed to appropriate truststores. An example of doing this is:</p>
<div>
<div>
<pre class="source">keytool -import -alias &lt;alias&gt; -file &lt;certificate&gt; -keystore &lt;keystore_file&gt; -storepass &lt;password&gt;
</pre></div></div>
</div>
<div class="section">
<h4><a name="Roles"></a>Roles</h4>
<p>Roles used by Metron are <tt>ROLE_ADMIN</tt> and <tt>ROLE_USER</tt>. Metron will use a property in a group containing the appropriate role to construct this.</p>
<p>Metron can be configured to map the roles defined in your authorization provider to the authorities used internally for access control. This can be configured under Security &gt; Roles in Ambari.</p>
<p>For example, our ldif file could create this group:</p>
<div>
<div>
<pre class="source">dn: cn=admin,ou=groups,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass: groupofnames
cn: admin
description:admin group
member: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org
</pre></div></div>
<p>If we are using &#x201c;cn&#x201d; as our role attribute, Metron will give the &#x201c;admin&#x201d; user the role &#x201c;ROLE_ADMIN&#x201d;.</p>
<p>Similarly, we could give a user &#x201c;sam&#x201d; ROLE_USER with the following group:</p>
<div>
<div>
<pre class="source">dn: cn=user,ou=groups,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass: groupofnames
cn: user
description: user group
member: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org
</pre></div></div>
</div></div>
<div class="section">
<h3><a name="JDBC_Authentication"></a>JDBC Authentication</h3>
<p>The REST application persists data in a relational database and requires a dedicated database user and database (see <a class="externalLink" href="https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html">https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html</a> for more detail).<br />
Spring uses Hibernate as the default ORM framework but another framework is needed becaused Hibernate is not compatible with the Apache 2 license. For this reason Metron uses <a class="externalLink" href="https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html#boot-features-embedded-database-support">EclipseLink</a>. See the <a class="externalLink" href="https://github.com/spring-projects/spring-data-examples/tree/master/jpa/eclipselink">Spring Data JPA - EclipseLink</a> project for an example on how to configure EclipseLink in Spring.</p>
<p>The metron-rest module uses <a class="externalLink" href="http://projects.spring.io/spring-security/">Spring Security</a> for authentication and stores user credentials in the relational database configured above. The required tables are created automatically the first time the application is started so that should be done first. For example (continuing the MySQL example above), users can be added by connecting to MySQL and running:</p>
<div>
<div>
<pre class="source">use metronrest;
insert into users (username, password, enabled) values ('your_username','your_password',1);
insert into authorities (username, authority) values ('your_username', 'ROLE_USER');
</pre></div></div>
</div>
<div class="section">
<h3><a name="Development"></a>Development</h3>
<p>The REST application comes with <a class="externalLink" href="https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html#boot-features-embedded-database-support">embedded database support</a> for development purposes.</p>
<p>For example, edit these variables in <tt>/etc/default/metron</tt> before starting the application to configure H2:</p>
<div>
<div>
<pre class="source">METRON_JDBC_DRIVER=&quot;org.h2.Driver&quot;
METRON_JDBC_URL=&quot;jdbc:h2:file:~/metrondb&quot;
METRON_JDBC_USERNAME=&quot;root&quot;
METRON_JDBC_PLATFORM=&quot;h2&quot;
</pre></div></div>
</div>
<div class="section">
<h3><a name="Production"></a>Production</h3>
<p>The REST application should be configured with a production-grade database outside of development.</p>
<div class="section">
<h4><a name="Ambari_Install"></a>Ambari Install</h4>
<p>Installing with Ambari is recommended for production deployments. Ambari handles setup, configuration, and management of the REST component. This includes managing the PID file, directing logging, etc.</p></div>
<div class="section">
<h4><a name="Manual_Install"></a>Manual Install</h4>
<p>The following configures the application for MySQL:</p>
<ol style="list-style-type: decimal">
<li>
<p>Install MySQL if not already available (this example uses version 5.7, installation instructions can be found <a class="externalLink" href="https://dev.mysql.com/doc/refman/5.7/en/linux-installation-yum-repo.html">here</a>)</p>
</li>
<li>
<p>Create a metron user and REST database and permission the user for that database:</p>
<div>
<div>
<pre class="source">CREATE USER 'metron'@'node1' IDENTIFIED BY 'Myp@ssw0rd';
CREATE DATABASE IF NOT EXISTS metronrest;
GRANT ALL PRIVILEGES ON metronrest.* TO 'metron'@'node1';
</pre></div></div>
</li>
<li>
<p>Create the security tables as described in the <a class="externalLink" href="https://docs.spring.io/spring-security/site/docs/5.0.4.RELEASE/reference/htmlsingle/#user-schema">Spring Security Guide</a>.</p>
</li>
<li>
<p>Install the MySQL JDBC client onto the REST application host and configurate the METRON_JDBC_CLIENT_PATH variable:</p>
<div>
<div>
<pre class="source">cd $METRON_HOME/lib
wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.41.tar.gz
tar xf mysql-connector-java-5.1.41.tar.gz
</pre></div></div>
</li>
<li>
<p>Edit these variables in <tt>/etc/default/metron</tt> to configure the REST application for MySQL:</p>
<div>
<div>
<pre class="source">METRON_JDBC_DRIVER=&quot;com.mysql.jdbc.Driver&quot;
METRON_JDBC_URL=&quot;jdbc:mysql://mysql_host:3306/metronrest&quot;
METRON_JDBC_USERNAME=&quot;metron&quot;
METRON_JDBC_PLATFORM=&quot;mysql&quot;
METRON_JDBC_CLIENT_PATH=$METRON_HOME/lib/mysql-connector-java-5.1.41/mysql-connector-java-5.1.41-bin.jar
</pre></div></div>
</li>
<li>
<p>Switch to the metron user</p>
<div>
<div>
<pre class="source">sudo su - metron
</pre></div></div>
</li>
<li>
<p>Start the REST API. Adjust the password as necessary.</p>
<div>
<div>
<pre class="source">set -o allexport;
source /etc/default/metron;
set +o allexport;
export METRON_JDBC_PASSWORD='Myp@ssw0rd';
$METRON_HOME/bin/metron-rest.sh
unset METRON_JDBC_PASSWORD;
</pre></div></div>
</li>
</ol></div></div></div>
<div class="section">
<h2><a name="Pcap_Query"></a>Pcap Query</h2>
<p>The REST application exposes endpoints for querying Pcap data. For more information about filtering options see <a href="../../metron-platform/metron-pcap-backend/index.html#Query_Filter_Utility">Query Filter Utility</a>.</p>
<p>There is an endpoint available that will return Pcap data in <a class="externalLink" href="https://wiki.wireshark.org/PDML">PDML</a> format. <a class="externalLink" href="https://www.wireshark.org/">Wireshark</a> must be installed for this feature to work. Installing wireshark in CentOS can be done with <tt>yum -y install wireshark</tt>.</p>
<p>The REST application uses a Java Process object to call out to the <tt>pcap_to_pdml.sh</tt> script. This script is installed at <tt>$METRON_HOME/bin/pcap_to_pdml.sh</tt> by default. Out of the box it is a simple wrapper around the tshark command to transform raw pcap data to PDML. However it can be extended to do additional processing as long as the expected input/output is maintained. REST will supply the script with raw pcap data through standard in and expects PDML data serialized as XML.</p>
<p>Pcap query jobs can be configured for submission to a YARN queue. This setting is exposed as the Spring property <tt>pcap.yarn.queue</tt> and can be set in the PCAP tab under Metron service -&gt; Configs in Ambari. If configured, the REST application will set the <tt>mapreduce.job.queuename</tt> Hadoop property to that value. It is highly recommended that a dedicated YARN queue be created and configured for Pcap queries to prevent a job from consuming too many cluster resources. More information about setting up YARN queues can be found <a class="externalLink" href="https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html#Setting_up_queues">here</a>.</p>
<p>Pcap query results are stored in HDFS. The location of query results when run through the REST app is determined by a couple factors. The root of Pcap query results defaults to <tt>/apps/metron/pcap/output</tt> but can be changed with the Spring property <tt>pcap.final.output.path</tt>. Assuming the default Pcap query output directory, the path to a result page will follow this pattern:</p>
<div>
<div>
<pre class="source">/apps/metron/pcap/output/{username}/MAP_REDUCE/{job id}/page-{page number}.pcap
</pre></div></div>
<p>Over time Pcap query results will accumulate in HDFS. Currently these results are not cleaned up automatically so cluster administrators should be aware of this and monitor them. It is highly recommended that a process be put in place to periodically delete files and directories under the Pcap query results root.</p>
<p>Users should also be mindful of date ranges used in queries so they don&#x2019;t produce result sets that are too large. Currently there are no limits enforced on date ranges.</p>
<p>Queries can also be configured on a global level for setting the number of results per page via a Spring property <tt>pcap.page.size</tt>. This property can be set in the PCAP tab under Metron service -&gt; Configs, in Ambari. By default, this value is set to 10 pcaps per page, but you may choose to set this value higher based on observing frequenetly-run query result sizes. This setting works in conjunction with the property for setting finalizer threadpool size when optimizing query performance.</p>
<p>Pcap query jobs have a finalization routine that writes their results out to HDFS in pages. Depending on the size of your pcaps, the number or results typically returned, page sizing (described above), and available CPU cores for running your REST application, your performance can be improved by adjusting the number of files that can be written to HDFS in parallel. To this end, there is a threadpool used for this finalization step that can be configured to use a specified number of threads. This setting is exposed as the Spring property <tt>pcap.finalizer.threadpool.size</tt>. A default value of &#x201c;1&#x201d; is used if not specified by the user. Generally speaking, you should see a performance gain when this value is set to anything higher than 1. A sizeable increase in performance can be achieved, especially for larger numbers of files of smaller size, by increasing the number of threads. It should be noted that this property is parsed as a String to allow for more complex parallelism values. In addition to normal integer values, you can specify a multiple of the number of cores. If it&#x2019;s a string and ends with &#x201c;C&#x201d;, then strip the C and treat it as an integral multiple of the number of cores. If it&#x2019;s a string and does not end with a C, then treat it as a number in string form.</p></div>
<div class="section">
<h2><a name="API"></a>API</h2>
<p>Request and Response objects are JSON formatted. The JSON schemas are available in the Swagger UI.</p>
<table border="0" class="table table-striped">
<thead>
<tr class="a">
<th> </th></tr>
</thead><tbody>
<tr class="b">
<td> <a href="#POST_apiv1alertsuiescalate"> <tt>POST /api/v1/alerts/ui/escalate</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1alertsuisettings"> <tt>GET /api/v1/alerts/ui/settings</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1alertsuisettingsall"> <tt>GET /api/v1/alerts/ui/settings/all</tt></a></td></tr>
<tr class="a">
<td> <a href="#DELETE_apiv1alertsuisettings"> <tt>DELETE /api/v1/alerts/ui/settings</tt></a></td></tr>
<tr class="b">
<td> <a href="#POST_apiv1alertsuisettings"> <tt>POST /api/v1/alerts/ui/settings</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1globalconfig"> <tt>GET /api/v1/global/config</tt></a></td></tr>
<tr class="b">
<td> <a href="#DELETE_apiv1globalconfig"> <tt>DELETE /api/v1/global/config</tt></a></td></tr>
<tr class="a">
<td> <a href="#POST_apiv1globalconfig"> <tt>POST /api/v1/global/config</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1grokgetstatement"> <tt>GET /api/v1/grok/get/statement</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1groklist"> <tt>GET /api/v1/grok/list</tt></a></td></tr>
<tr class="b">
<td> <a href="#POST_apiv1grokvalidate"> <tt>POST /api/v1/grok/validate</tt></a></td></tr>
<tr class="a">
<td> <a href="#POST_apiv1hdfs"> <tt>POST /api/v1/hdfs</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1hdfs"> <tt>GET /api/v1/hdfs</tt></a></td></tr>
<tr class="a">
<td> <a href="#DELETE_apiv1hdfs"> <tt>DELETE /api/v1/hdfs</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1hdfslist"> <tt>GET /api/v1/hdfs/list</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1kafkatopic"> <tt>GET /api/v1/kafka/topic</tt></a></td></tr>
<tr class="b">
<td> <a href="#POST_apiv1kafkatopic"> <tt>POST /api/v1/kafka/topic</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1kafkatopicname"> <tt>GET /api/v1/kafka/topic/{name}</tt></a></td></tr>
<tr class="b">
<td> <a href="#DELETE_apiv1kafkatopicname"> <tt>DELETE /api/v1/kafka/topic/{name}</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1kafkatopicnamesample"> <tt>GET /api/v1/kafka/topic/{name}/sample</tt></a></td></tr>
<tr class="b">
<td> <a href="#POST_apiv1kafkatopicnameproduce"> <tt>POST /api/v1/kafka/topic/{name}/produce</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1metaalertsearchByAlert"> <tt>GET /api/v1/metaalert/searchByAlert</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1metaalertcreate"> <tt>GET /api/v1/metaalert/create</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1metaalertaddalert"> <tt>GET /api/v1/metaalert/add/alert</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1metaalertremovealert"> <tt>GET /api/v1/metaalert/remove/alert</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1metaalertupdatestatusguidstatus"> <tt>GET /api/v1/metaalert/update/status/{guid}/{status}</tt></a></td></tr>
<tr class="b">
<td> <a href="#POST_apiv1pcapfixed"> <tt>POST /api/v1/pcap/fixed</tt></a></td></tr>
<tr class="a">
<td> <a href="#POST_apiv1pcapquery"> <tt>POST /api/v1/pcap/query</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1pcap"> <tt>GET /api/v1/pcap</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1pcapjobId"> <tt>GET /api/v1/pcap/{jobId}</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1pcapjobIdpdml"> <tt>GET /api/v1/pcap/{jobId}/pdml</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1pcapjobIdraw"> <tt>GET /api/v1/pcap/{jobId}/raw</tt></a></td></tr>
<tr class="b">
<td> <a href="#DELETE_apiv1pcapkilljobId"> <tt>DELETE /api/v1/pcap/kill/{jobId}</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1pcapjobIdconfig"> <tt>GET /api/v1/pcap/{jobId}/config</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1searchsearch"> <tt>GET /api/v1/search/search</tt></a></td></tr>
<tr class="a">
<td> <a href="#POST_apiv1searchsearch"> <tt>POST /api/v1/search/search</tt></a></td></tr>
<tr class="b">
<td> <a href="#POST_apiv1searchgroup"> <tt>POST /api/v1/search/group</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1searchfindOne"> <tt>GET /api/v1/search/findOne</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1searchcolumnmetadata"> <tt>GET /api/v1/search/column/metadata</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1sensorenrichmentconfig"> <tt>GET /api/v1/sensor/enrichment/config</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1sensorenrichmentconfiglistavailableenrichments"> <tt>GET /api/v1/sensor/enrichment/config/list/available/enrichments</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1sensorenrichmentconfiglistavailablethreattriageaggregators"> <tt>GET /api/v1/sensor/enrichment/config/list/available/threat/triage/aggregators</tt></a></td></tr>
<tr class="b">
<td> <a href="#DELETE_apiv1sensorenrichmentconfigname"> <tt>DELETE /api/v1/sensor/enrichment/config/{name}</tt></a></td></tr>
<tr class="a">
<td> <a href="#POST_apiv1sensorenrichmentconfigname"> <tt>POST /api/v1/sensor/enrichment/config/{name}</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1sensorenrichmentconfigname"> <tt>GET /api/v1/sensor/enrichment/config/{name}</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1sensorindexingconfig"> <tt>GET /api/v1/sensor/indexing/config</tt></a></td></tr>
<tr class="b">
<td> <a href="#DELETE_apiv1sensorindexingconfigname"> <tt>DELETE /api/v1/sensor/indexing/config/{name}</tt></a></td></tr>
<tr class="a">
<td> <a href="#POST_apiv1sensorindexingconfigname"> <tt>POST /api/v1/sensor/indexing/config/{name}</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1sensorindexingconfigname"> <tt>GET /api/v1/sensor/indexing/config/{name}</tt></a></td></tr>
<tr class="a">
<td> <a href="#POST_apiv1sensorparserconfig"> <tt>POST /api/v1/sensor/parser/config</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1sensorparserconfig"> <tt>GET /api/v1/sensor/parser/config</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1sensorparserconfiglistavailable"> <tt>GET /api/v1/sensor/parser/config/list/available</tt></a></td></tr>
<tr class="b">
<td> <a href="#POST_apiv1sensorparserconfigparseMessage"> <tt>POST /api/v1/sensor/parser/config/parseMessage</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1sensorparserconfigreloadavailable"> <tt>GET /api/v1/sensor/parser/config/reload/available</tt></a></td></tr>
<tr class="b">
<td> <a href="#DELETE_apiv1sensorparserconfigname"> <tt>DELETE /api/v1/sensor/parser/config/{name}</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1sensorparserconfigname"> <tt>GET /api/v1/sensor/parser/config/{name}</tt></a></td></tr>
<tr class="b">
<td> <a href="#POST_apiv1sensorparsergroup"> <tt>POST /api/v1/sensor/parser/group</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1sensorparsergroupname"> <tt>GET /api/v1/sensor/parser/group/{name}</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1sensorparsergroup"> <tt>GET /api/v1/sensor/parser/group</tt></a></td></tr>
<tr class="a">
<td> <a href="#DELETE_apiv1sensorparsergroupname"> <tt>DELETE /api/v1/sensor/parser/group/{name}</tt></a></td></tr>
<tr class="b">
<td> <a href="#POST_apiv1stellarapplytransformations"> <tt>POST /api/v1/stellar/apply/transformations</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1stellarlist"> <tt>GET /api/v1/stellar/list</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1stellarlistfunctions"> <tt>GET /api/v1/stellar/list/functions</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1stellarlistsimplefunctions"> <tt>GET /api/v1/stellar/list/simple/functions</tt></a></td></tr>
<tr class="b">
<td> <a href="#POST_apiv1stellarvalidaterules"> <tt>POST /api/v1/stellar/validate/rules</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1storm"> <tt>GET /api/v1/storm</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1stormclientstatus"> <tt>GET /api/v1/storm/client/status</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1stormenrichment"> <tt>GET /api/v1/storm/enrichment</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1stormenrichmentactivate"> <tt>GET /api/v1/storm/enrichment/activate</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1stormenrichmentdeactivate"> <tt>GET /api/v1/storm/enrichment/deactivate</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1stormenrichmentstart"> <tt>GET /api/v1/storm/enrichment/start</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1stormenrichmentstop"> <tt>GET /api/v1/storm/enrichment/stop</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1stormindexingbatch"> <tt>GET /api/v1/storm/indexing/batch</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1stormindexingbatchactivate"> <tt>GET /api/v1/storm/indexing/batch/activate</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1stormindexingbatchdeactivate"> <tt>GET /api/v1/storm/indexing/batch/deactivate</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1stormindexingbatchstart"> <tt>GET /api/v1/storm/indexing/batch/start</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1stormindexingbatchstop"> <tt>GET /api/v1/storm/indexing/batch/stop</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1stormindexingrandomaccess"> <tt>GET /api/v1/storm/indexing/randomaccess</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1stormindexingrandomaccessactivate"> <tt>GET /api/v1/storm/indexing/randomaccess/activate</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1stormindexingrandomaccessdeactivate"> <tt>GET /api/v1/storm/indexing/randomaccess/deactivate</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1stormindexingrandomaccessstart"> <tt>GET /api/v1/storm/indexing/randomaccess/start</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1stormindexingrandomaccessstop"> <tt>GET /api/v1/storm/indexing/randomaccess/stop</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1stormparseractivatename"> <tt>GET /api/v1/storm/parser/activate/{name}</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1stormparserdeactivatename"> <tt>GET /api/v1/storm/parser/deactivate/{name}</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1stormparserstartname"> <tt>GET /api/v1/storm/parser/start/{name}</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1stormparserstopname"> <tt>GET /api/v1/storm/parser/stop/{name}</tt></a></td></tr>
<tr class="b">
<td> <a href="#GET_apiv1stormname"> <tt>GET /api/v1/storm/{name}</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1stormsupervisors"> <tt>GET /api/v1/storm/supervisors</tt></a></td></tr>
<tr class="b">
<td> <a href="#PATCH_apiv1updatepatch"> <tt>PATCH /api/v1/update/patch</tt></a></td></tr>
<tr class="a">
<td> <a href="#put-apiv1updateaddcomment"> <tt>POST /api/v1/update/add/comment</tt></a></td></tr>
<tr class="b">
<td> <a href="#put-apiv1updateremovecomment"> <tt>POST /api/v1/update/remove/comment</tt></a></td></tr>
<tr class="a">
<td> <a href="#GET_apiv1user"> <tt>GET /api/v1/user</tt></a></td></tr>
</tbody>
</table>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Falerts.2Fui.2Fescalate"></a><tt>POST /api/v1/alerts/ui/escalate</tt></h3>
<ul>
<li>Description: Escalates a list of alerts by producing it to the Kafka escalate topic</li>
<li>Input:
<ul>
<li>alerts - The alerts to be escalated</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Alerts were escalated</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Falerts.2Fui.2Fsettings"></a><tt>GET /api/v1/alerts/ui/settings</tt></h3>
<ul>
<li>Description: Retrieves the current user&#x2019;s settings</li>
<li>Returns:
<ul>
<li>200 - User settings</li>
<li>404 - he current user does not have settings</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Falerts.2Fui.2Fsettings.2Fall"></a><tt>GET /api/v1/alerts/ui/settings/all</tt></h3>
<ul>
<li>Description: Retrieves all users&#x2019; settings. Only users that are part of the &#x201c;ROLE_ADMIN&#x201d; role are allowed to get all user settings.</li>
<li>Returns:
<ul>
<li>200 - List of all user settings</li>
<li>403 - The current user does not have permission to get all user settings</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="DELETE_.2Fapi.2Fv1.2Falerts.2Fui.2Fsettings"></a><tt>DELETE /api/v1/alerts/ui/settings</tt></h3>
<ul>
<li>Description: Deletes a user&#x2019;s settings. Only users that are part of the &#x201c;ROLE_ADMIN&#x201d; role are allowed to delete user settings.</li>
<li>Input:
<ul>
<li>user - The user whose settings will be deleted</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - User settings were deleted</li>
<li>403 - The current user does not have permission to delete user settings</li>
<li>404 - User settings could not be found</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Falerts.2Fui.2Fsettings"></a><tt>POST /api/v1/alerts/ui/settings</tt></h3>
<ul>
<li>Description: Creates or updates the current user&#x2019;s settings</li>
<li>Input:
<ul>
<li>alertsUIUserSettings - The user settings to be saved</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - User settings updated. Returns saved settings.</li>
<li>201 - User settings created. Returns saved settings.</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fglobal.2Fconfig"></a><tt>GET /api/v1/global/config</tt></h3>
<ul>
<li>Description: Retrieves the current Global Config from Zookeeper</li>
<li>Returns:
<ul>
<li>200 - Returns current Global Config JSON in Zookeeper</li>
<li>404 - Global Config JSON was not found in Zookeeper</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="DELETE_.2Fapi.2Fv1.2Fglobal.2Fconfig"></a><tt>DELETE /api/v1/global/config</tt></h3>
<ul>
<li>Description: Deletes the current Global Config from Zookeeper</li>
<li>Returns:
<ul>
<li>200 - Global Config JSON was deleted</li>
<li>404 - Global Config JSON was not found in Zookeeper</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fglobal.2Fconfig"></a><tt>POST /api/v1/global/config</tt></h3>
<ul>
<li>Description: Creates or updates the Global Config in Zookeeper</li>
<li>Input:
<ul>
<li>globalConfig - The Global Config JSON to be saved</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Global Config updated. Returns saved Global Config JSON</li>
<li>201 - Global Config created. Returns saved Global Config JSON</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fgrok.2Fget.2Fstatement"></a><tt>GET /api/v1/grok/get/statement</tt></h3>
<ul>
<li>Description: Retrieves a Grok statement from the classpath</li>
<li>Input:
<ul>
<li>path - Path to classpath resource</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Grok statement</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fgrok.2Flist"></a><tt>GET /api/v1/grok/list</tt></h3>
<ul>
<li>Description: Lists the common Grok statements available in Metron</li>
<li>Returns:
<ul>
<li>200 - JSON object containing pattern label/Grok statements key value pairs</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fgrok.2Fvalidate"></a><tt>POST /api/v1/grok/validate</tt></h3>
<ul>
<li>Description: Applies a Grok statement to a sample message</li>
<li>Input:
<ul>
<li>grokValidation - Object containing Grok statement and sample message</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - JSON results</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fhdfs"></a><tt>POST /api/v1/hdfs</tt></h3>
<ul>
<li>Description: Writes contents to an HDFS file. Warning: this will overwrite the contents of a file if it already exists. Permissions must be set for all three groups if they are to be set. If any are missing, the default permissions will be used, and if any are invalid an exception will be thrown.</li>
<li>Input:
<ul>
<li>path - Path to HDFS file</li>
<li>contents - File contents</li>
<li>userMode - [optional] symbolic permission string for user portion of the permissions to be set on the file written. For example &#x2018;rwx&#x2019; or read, write, execute. The symbol &#x2018;-&#x2019; is used to exclude that permission such as &#x2018;rw-&#x2019; for read, write, no execute</li>
<li>groupMode - [optional] symbolic permission string for group portion of the permissions to be set on the file written. For example &#x2018;rwx&#x2019; or read, write, execute. The symbol &#x2018;-&#x2019; is used to exclude that permission such as &#x2018;rw-&#x2019; for read, write, no execute</li>
<li>otherMode - [optional] symbolic permission string for other portion of the permissions to be set on the file written. For example &#x2018;rwx&#x2019; or read, write, execute. The symbol &#x2018;-&#x2019; is used to exclude that permission such as &#x2018;rw-&#x2019; for read, write, no execute</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Contents were written</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fhdfs"></a><tt>GET /api/v1/hdfs</tt></h3>
<ul>
<li>Description: Reads a file from HDFS and returns the contents</li>
<li>Input:
<ul>
<li>path - Path to HDFS file</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns file contents</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="DELETE_.2Fapi.2Fv1.2Fhdfs"></a><tt>DELETE /api/v1/hdfs</tt></h3>
<ul>
<li>Description: Deletes a file from HDFS</li>
<li>Input:
<ul>
<li>path - Path to HDFS file</li>
<li>recursive - Delete files recursively</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - File was deleted</li>
<li>404 - File was not found in HDFS</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fhdfs.2Flist"></a><tt>GET /api/v1/hdfs/list</tt></h3>
<ul>
<li>Description: Lists an HDFS directory</li>
<li>Input:
<ul>
<li>path - Path to HDFS directory</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - HDFS directory list</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fkafka.2Ftopic"></a><tt>GET /api/v1/kafka/topic</tt></h3>
<ul>
<li>Description: Retrieves all Kafka topics</li>
<li>Returns:
<ul>
<li>200 - Returns a list of all Kafka topics</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fkafka.2Ftopic"></a><tt>POST /api/v1/kafka/topic</tt></h3>
<ul>
<li>Description: Creates a new Kafka topic</li>
<li>Input:
<ul>
<li>topic - Kafka topic</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns saved Kafka topic</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fkafka.2Ftopic.2F.7Bname.7D"></a><tt>GET /api/v1/kafka/topic/{name}</tt></h3>
<ul>
<li>Description: Retrieves a Kafka topic</li>
<li>Input:
<ul>
<li>name - Kafka topic name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns Kafka topic</li>
<li>404 - Kafka topic is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="DELETE_.2Fapi.2Fv1.2Fkafka.2Ftopic.2F.7Bname.7D"></a><tt>DELETE /api/v1/kafka/topic/{name}</tt></h3>
<ul>
<li>Description: Deletes a Kafka topic</li>
<li>Input:
<ul>
<li>name - Kafka topic name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Kafka topic was deleted</li>
<li>404 - Kafka topic is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fkafka.2Ftopic.2F.7Bname.7D.2Fsample"></a><tt>GET /api/v1/kafka/topic/{name}/sample</tt></h3>
<ul>
<li>Description: Retrieves a sample message from a Kafka topic using the most recent offset</li>
<li>Input:
<ul>
<li>name - Kafka topic name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns sample message</li>
<li>404 - Either Kafka topic is missing or contains no messages</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fkafka.2Ftopic.2F.7Bname.7D.2Fproduce"></a><tt>POST /api/v1/kafka/topic/{name}/produce</tt></h3>
<ul>
<li>Description: Produces a message to a Kafka topic</li>
<li>Input:
<ul>
<li>name - Kafka topic name</li>
<li>message - message to be published</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Published</li>
<li>404 - Kafka topic is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fmetaalert.2FsearchByAlert"></a><tt>POST /api/v1/metaalert/searchByAlert</tt></h3>
<ul>
<li>Description: Get all meta alerts that contain an alert.</li>
<li>Input:
<ul>
<li>guid - GUID of the alert</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Search results</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fmetaalert.2Fcreate"></a><tt>POST /api/v1/metaalert/create</tt></h3>
<ul>
<li>Description: Creates a new meta alert from a list of existing alerts. The meta alert status will initially be set to &#x2018;ACTIVE&#x2019; and summary statistics will be computed from the list of alerts. A list of groups included in the request are also added to the meta alert.</li>
<li>Input:
<ul>
<li>request - Meta alert create request which includes a list of alert get requests and a list of custom groups used to annotate a meta alert.</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - The GUID of the new meta alert</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fmetaalert.2Fadd.2Falert"></a><tt>POST /api/v1/metaalert/add/alert</tt></h3>
<ul>
<li>Description: Adds an alert to an existing meta alert. An alert will not be added if it is already contained in a meta alert.</li>
<li>Input:
<ul>
<li>request - Meta alert add request which includes a meta alert GUID and list of alert get requests</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns &#x2018;true&#x2019; if the alert was added and &#x2018;false&#x2019; if the meta alert did not change.</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fmetaalert.2Fremove.2Falert"></a><tt>POST /api/v1/metaalert/remove/alert</tt></h3>
<ul>
<li>Description: Removes an alert from an existing meta alert. If the alert to be removed is not in a meta alert, &#x2018;false&#x2019; will be returned.</li>
<li>Input:
<ul>
<li>request - Meta alert remove request which includes a meta alert GUID and list of alert get requests</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns &#x2018;true&#x2019; if the alert was removed and &#x2018;false&#x2019; if the meta alert did not change.</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fmetaalert.2Fupdate.2Fstatus.2F.7Bguid.7D.2F.7Bstatus.7D"></a><tt>POST /api/v1/metaalert/update/status/{guid}/{status}</tt></h3>
<ul>
<li>Description: Updates the status of a meta alert to either &#x2018;ACTIVE&#x2019; or &#x2018;INACTIVE&#x2019;.</li>
<li>Input:
<ul>
<li>guid - Meta alert GUID</li>
<li>status - Meta alert status with a value of either &#x2018;ACTIVE&#x2019; or &#x2018;INACTIVE&#x2019;</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns &#x2018;true&#x2019; if the status changed and &#x2018;false&#x2019; if it did not.</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fpcap.2Ffixed"></a><tt>POST /api/v1/pcap/fixed</tt></h3>
<ul>
<li>Description: Executes a Fixed Filter Pcap Query.</li>
<li>Input:
<ul>
<li>fixedPcapRequest - A Fixed Pcap Request which includes fixed filter fields like ip source address and protocol</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns a job status with job ID.</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fpcap.2Fquery"></a><tt>POST /api/v1/pcap/query</tt></h3>
<ul>
<li>Description: Executes a Query Filter Pcap Query.</li>
<li>Input:
<ul>
<li>queryPcapRequest - A Query Pcap Request which includes Stellar query field</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns a job status with job ID.</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fpcap"></a><tt>GET /api/v1/pcap</tt></h3>
<ul>
<li>Description: Gets a list of job statuses for Pcap query jobs that match the requested state.</li>
<li>Input:
<ul>
<li>state - Job state</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns a list of job statuses for jobs that match the requested state.</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fpcap.2F.7BjobId.7D"></a><tt>GET /api/v1/pcap/{jobId}</tt></h3>
<ul>
<li>Description: Gets job status for Pcap query job.</li>
<li>Input:
<ul>
<li>jobId - Job ID of submitted job</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns a job status for the Job ID.</li>
<li>404 - Job is missing.</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fpcap.2F.7BjobId.7D.2Fpdml"></a><tt>GET /api/v1/pcap/{jobId}/pdml</tt></h3>
<ul>
<li>Description: Gets Pcap Results for a page in PDML format.</li>
<li>Input:
<ul>
<li>jobId - Job ID of submitted job</li>
<li>page - Page number</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns PDML in json format.</li>
<li>404 - Job or page is missing.</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fpcap.2F.7BjobId.7D.2Fraw"></a><tt>GET /api/v1/pcap/{jobId}/raw</tt></h3>
<ul>
<li>Description: Download Pcap Results for a page.</li>
<li>Input:
<ul>
<li>jobId - Job ID of submitted job</li>
<li>page - Page number</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns Pcap as a file download.</li>
<li>404 - Job or page is missing.</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="DELETE_.2Fapi.2Fv1.2Fpcap.2Fkill.2F.7BjobId.7D"></a><tt>DELETE /api/v1/pcap/kill/{jobId}</tt></h3>
<ul>
<li>Description: Kills running job.</li>
<li>Input:
<ul>
<li>jobId - Job ID of submitted job</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Kills passed job.</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fpcap.2F.7BjobId.7D.2Fconfig"></a><tt>GET /api/v1/pcap/{jobId}/config</tt></h3>
<ul>
<li>Description: Gets job configuration for Pcap query job.</li>
<li>Input:
<ul>
<li>jobId - Job ID of submitted job</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns a map of job properties for the Job ID.</li>
<li>404 - Job is missing.</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fsearch.2Fsearch"></a><tt>POST /api/v1/search/search</tt></h3>
<ul>
<li>Description: Searches the indexing store. GUIDs must be quoted to ensure correct results.</li>
<li>Input:
<ul>
<li>searchRequest - Search request</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Search response</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fsearch.2Fgroup"></a><tt>POST /api/v1/search/group</tt></h3>
<ul>
<li>Description: Searches the indexing store and returns field groups. GUIDs must be quoted to ensure correct results. Groups are hierarchical and nested in the order the fields appear in the &#x2018;groups&#x2019; request parameter. The default sorting within groups is by count descending. A groupOrder type of count will sort based on then number of documents in a group while a groupType of term will sort by the groupBy term.</li>
<li>Input:
<ul>
<li>groupRequest - Group request
<ul>
<li>indices - list of indices to search</li>
<li>query - lucene query</li>
<li>scoreField - field used to compute a total score for each group</li>
<li>groups - List of groups (field name and sort order)</li>
</ul>
</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Group response</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fsearch.2FfindOne"></a><tt>GET /api/v1/search/findOne</tt></h3>
<ul>
<li>Description: Returns latest document for a guid and sensor</li>
<li>Input:
<ul>
<li>getRequest - Get request
<ul>
<li>guid - message UUID</li>
<li>sensorType - Sensor Type</li>
</ul>
</li>
<li>Example: Return <tt>bro</tt> document with UUID of <tt>000-000-0000</tt></li>
</ul>
</li>
</ul>
<div>
<div>
<pre class="source">{
&quot;guid&quot; : &quot;000-000-0000&quot;,
&quot;sensorType&quot; : &quot;bro&quot;
}
</pre></div></div>
<ul>
<li>Returns:
<ul>
<li>200 - Document representing the output</li>
<li>404 - Document with UUID and sensor type not found</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fsearch.2Fcolumn.2Fmetadata"></a><tt>GET /api/v1/search/column/metadata</tt></h3>
<ul>
<li>Description: Get index column metadata for a list of sensor types with duplicates removed. Column names and types for each sensor are retrieved from the most recent index. Columns that exist in multiple indices with different types will default to type &#x2018;other&#x2019;.</li>
<li>Input:
<ul>
<li>sensorTypes - Sensor Types</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Column Metadata</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fenrichment.2Fconfig"></a><tt>GET /api/v1/sensor/enrichment/config</tt></h3>
<ul>
<li>Description: Retrieves all SensorEnrichmentConfigs from Zookeeper</li>
<li>Returns:
<ul>
<li>200 - Returns all SensorEnrichmentConfigs</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fenrichment.2Fconfig.2Flist.2Favailable.2Fenrichments"></a><tt>GET /api/v1/sensor/enrichment/config/list/available/enrichments</tt></h3>
<ul>
<li>Description: Lists the available enrichments</li>
<li>Returns:
<ul>
<li>200 - Returns a list of available enrichments in lexicographical order</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fenrichment.2Fconfig.2Flist.2Favailable.2Fthreat.2Ftriage.2Faggregators"></a><tt>GET /api/v1/sensor/enrichment/config/list/available/threat/triage/aggregators</tt></h3>
<ul>
<li>Description: Lists the available threat triage aggregators</li>
<li>Returns:
<ul>
<li>200 - Returns a list of available threat triage aggregators</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="DELETE_.2Fapi.2Fv1.2Fsensor.2Fenrichment.2Fconfig.2F.7Bname.7D"></a><tt>DELETE /api/v1/sensor/enrichment/config/{name}</tt></h3>
<ul>
<li>Description: Deletes a SensorEnrichmentConfig from Zookeeper</li>
<li>Input:
<ul>
<li>name - SensorEnrichmentConfig name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - SensorEnrichmentConfig was deleted</li>
<li>404 - SensorEnrichmentConfig is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fsensor.2Fenrichment.2Fconfig.2F.7Bname.7D"></a><tt>POST /api/v1/sensor/enrichment/config/{name}</tt></h3>
<ul>
<li>Description: Updates or creates a SensorEnrichmentConfig in Zookeeper</li>
<li>Input:
<ul>
<li>sensorEnrichmentConfig - SensorEnrichmentConfig</li>
<li>name - SensorEnrichmentConfig name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - SensorEnrichmentConfig updated. Returns saved SensorEnrichmentConfig</li>
<li>201 - SensorEnrichmentConfig created. Returns saved SensorEnrichmentConfig</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fenrichment.2Fconfig.2F.7Bname.7D"></a><tt>GET /api/v1/sensor/enrichment/config/{name}</tt></h3>
<ul>
<li>Description: Retrieves a SensorEnrichmentConfig from Zookeeper</li>
<li>Input:
<ul>
<li>name - SensorEnrichmentConfig name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns SensorEnrichmentConfig</li>
<li>404 - SensorEnrichmentConfig is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Findexing.2Fconfig"></a><tt>GET /api/v1/sensor/indexing/config</tt></h3>
<ul>
<li>Description: Retrieves all SensorIndexingConfigs from Zookeeper</li>
<li>Returns:
<ul>
<li>200 - Returns all SensorIndexingConfigs</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="DELETE_.2Fapi.2Fv1.2Fsensor.2Findexing.2Fconfig.2F.7Bname.7D"></a><tt>DELETE /api/v1/sensor/indexing/config/{name}</tt></h3>
<ul>
<li>Description: Deletes a SensorIndexingConfig from Zookeeper</li>
<li>Input:
<ul>
<li>name - SensorIndexingConfig name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - SensorIndexingConfig was deleted</li>
<li>404 - SensorIndexingConfig is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fsensor.2Findexing.2Fconfig.2F.7Bname.7D"></a><tt>POST /api/v1/sensor/indexing/config/{name}</tt></h3>
<ul>
<li>Description: Updates or creates a SensorIndexingConfig in Zookeeper</li>
<li>Input:
<ul>
<li>sensorIndexingConfig - SensorIndexingConfig</li>
<li>name - SensorIndexingConfig name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - SensorIndexingConfig updated. Returns saved SensorIndexingConfig</li>
<li>201 - SensorIndexingConfig created. Returns saved SensorIndexingConfig</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Findexing.2Fconfig.2F.7Bname.7D"></a><tt>GET /api/v1/sensor/indexing/config/{name}</tt></h3>
<ul>
<li>Description: Retrieves a SensorIndexingConfig from Zookeeper</li>
<li>Input:
<ul>
<li>name - SensorIndexingConfig name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns SensorIndexingConfig</li>
<li>404 - SensorIndexingConfig is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fconfig.2F.7Bname.7D"></a><tt>POST /api/v1/sensor/parser/config/{name}</tt></h3>
<ul>
<li>Description: Updates or creates a SensorParserConfig in Zookeeper</li>
<li>Input:
<ul>
<li>sensorParserConfig - SensorParserConfig</li>
<li>name - SensorEnrichmentConfig name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - SensorParserConfig updated. Returns saved SensorParserConfig</li>
<li>201 - SensorParserConfig created. Returns saved SensorParserConfig</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fconfig"></a><tt>GET /api/v1/sensor/parser/config</tt></h3>
<ul>
<li>Description: Retrieves all SensorParserConfigs from Zookeeper</li>
<li>Returns:
<ul>
<li>200 - Returns all SensorParserConfigs</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fconfig.2Flist.2Favailable"></a><tt>GET /api/v1/sensor/parser/config/list/available</tt></h3>
<ul>
<li>Description: Lists the available parser classes that can be found on the classpath</li>
<li>Returns:
<ul>
<li>200 - Returns a list of available parser classes</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fconfig.2FparseMessage"></a><tt>POST /api/v1/sensor/parser/config/parseMessage</tt></h3>
<ul>
<li>Description: Parses a sample message given a SensorParserConfig</li>
<li>Input:
<ul>
<li>parseMessageRequest - Object containing a sample message and SensorParserConfig</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns parsed message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fconfig.2Freload.2Favailable"></a><tt>GET /api/v1/sensor/parser/config/reload/available</tt></h3>
<ul>
<li>Description: Scans the classpath for available parser classes and reloads the cached parser class list</li>
<li>Returns:
<ul>
<li>200 - Returns a list of available parser classes</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="DELETE_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fconfig.2F.7Bname.7D"></a><tt>DELETE /api/v1/sensor/parser/config/{name}</tt></h3>
<ul>
<li>Description: Deletes a SensorParserConfig from Zookeeper</li>
<li>Input:
<ul>
<li>name - SensorParserConfig name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - SensorParserConfig was deleted</li>
<li>404 - SensorParserConfig is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fconfig.2F.7Bname.7D"></a><tt>GET /api/v1/sensor/parser/config/{name}</tt></h3>
<ul>
<li>Description: Retrieves a SensorParserConfig from Zookeeper</li>
<li>Input:
<ul>
<li>name - SensorParserConfig name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns SensorParserConfig</li>
<li>404 - SensorParserConfig is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fgroup"></a><tt>POST /api/v1/sensor/parser/group</tt></h3>
<ul>
<li>Description: Updates or creates a SensorParserGroup in Zookeeper</li>
<li>Input:
<ul>
<li>sensorParserGroup - SensorParserGroup</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - SensorParserGroup updated. Returns saved SensorParserGroup</li>
<li>201 - SensorParserGroup created. Returns saved SensorParserGroup</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fgroup.2F.7Bname.7D"></a><tt>GET /api/v1/sensor/parser/group/{name}</tt></h3>
<ul>
<li>Description: Retrieves a SensorParserGroup from Zookeeper</li>
<li>Input:
<ul>
<li>name - SensorParserGroup name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns SensorParserGroup</li>
<li>404 - SensorParserGroup is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fgroup"></a><tt>GET /api/v1/sensor/parser/group</tt></h3>
<ul>
<li>Description: Retrieves all SensorParserGroups from Zookeeper</li>
<li>Returns:
<ul>
<li>200 - Returns all SensorParserGroups</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="DELETE_.2Fapi.2Fv1.2Fsensor.2Fparser.2Fgroup.2F.7Bname.7D"></a><tt>DELETE /api/v1/sensor/parser/group/{name}</tt></h3>
<ul>
<li>Description: Deletes a SensorParserGroup from Zookeeper</li>
<li>Input:
<ul>
<li>name - SensorParserGroup name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - SensorParserGroup was deleted</li>
<li>404 - SensorParserGroup is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fstellar.2Fapply.2Ftransformations"></a><tt>POST /api/v1/stellar/apply/transformations</tt></h3>
<ul>
<li>Description: Executes transformations against a sample message</li>
<li>Input:
<ul>
<li>transformationValidation - Object containing SensorParserConfig and sample message</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns transformation results</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstellar.2Flist"></a><tt>GET /api/v1/stellar/list</tt></h3>
<ul>
<li>Description: Retrieves field transformations</li>
<li>Returns:
<ul>
<li>200 - Returns a list field transformations</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstellar.2Flist.2Ffunctions"></a><tt>GET /api/v1/stellar/list/functions</tt></h3>
<ul>
<li>Description: Lists the Stellar functions that can be found on the classpath</li>
<li>Returns:
<ul>
<li>200 - Returns a list of Stellar functions</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstellar.2Flist.2Fsimple.2Ffunctions"></a><tt>GET /api/v1/stellar/list/simple/functions</tt></h3>
<ul>
<li>Description: Lists the simple Stellar functions (functions with only 1 input) that can be found on the classpath</li>
<li>Returns:
<ul>
<li>200 - Returns a list of simple Stellar functions</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fstellar.2Fvalidate.2Frules"></a><tt>POST /api/v1/stellar/validate/rules</tt></h3>
<ul>
<li>Description: Tests Stellar statements to ensure they are well-formed</li>
<li>Input:
<ul>
<li>statements - List of statements to validate</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns validation results</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm"></a><tt>GET /api/v1/storm</tt></h3>
<ul>
<li>Description: Retrieves the status of all Storm topologies</li>
<li>Returns:
<ul>
<li>200 - Returns a list of topologies with status information</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Fclient.2Fstatus"></a><tt>GET /api/v1/storm/client/status</tt></h3>
<ul>
<li>Description: Retrieves information about the Storm command line client</li>
<li>Returns:
<ul>
<li>200 - Returns storm command line client information</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Fenrichment"></a><tt>GET /api/v1/storm/enrichment</tt></h3>
<ul>
<li>Description: Retrieves the status of the Storm enrichment topology</li>
<li>Returns:
<ul>
<li>200 - Returns topology status information</li>
<li>404 - Topology is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Fenrichment.2Factivate"></a><tt>GET /api/v1/storm/enrichment/activate</tt></h3>
<ul>
<li>Description: Activates a Storm enrichment topology</li>
<li>Returns:
<ul>
<li>200 - Returns activate response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Fenrichment.2Fdeactivate"></a><tt>GET /api/v1/storm/enrichment/deactivate</tt></h3>
<ul>
<li>Description: Deactivates a Storm enrichment topology</li>
<li>Returns:
<ul>
<li>200 - Returns deactivate response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Fenrichment.2Fstart"></a><tt>GET /api/v1/storm/enrichment/start</tt></h3>
<ul>
<li>Description: Starts a Storm enrichment topology</li>
<li>Returns:
<ul>
<li>200 - Returns start response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Fenrichment.2Fstop"></a><tt>GET /api/v1/storm/enrichment/stop</tt></h3>
<ul>
<li>Description: Stops a Storm enrichment topology</li>
<li>Input:
<ul>
<li>stopNow - Stop the topology immediately</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns stop response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Findexing.2Fbatch"></a><tt>GET /api/v1/storm/indexing/batch</tt></h3>
<ul>
<li>Description: Retrieves the status of the Storm batch indexing topology</li>
<li>Returns:
<ul>
<li>200 - Returns topology status information</li>
<li>404 - Topology is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Findexing.2Fbatch.2Factivate"></a><tt>GET /api/v1/storm/indexing/batch/activate</tt></h3>
<ul>
<li>Description: Activates a Storm batch indexing topology</li>
<li>Returns:
<ul>
<li>200 - Returns activate response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Findexing.2Fbatch.2Fdeactivate"></a><tt>GET /api/v1/storm/indexing/batch/deactivate</tt></h3>
<ul>
<li>Description: Deactivates a Storm batch indexing topology</li>
<li>Returns:
<ul>
<li>200 - Returns deactivate response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Findexing.2Fbatch.2Fstart"></a><tt>GET /api/v1/storm/indexing/batch/start</tt></h3>
<ul>
<li>Description: Starts a Storm batch indexing topology</li>
<li>Returns:
<ul>
<li>200 - Returns start response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Findexing.2Fbatch.2Fstop"></a><tt>GET /api/v1/storm/indexing/batch/stop</tt></h3>
<ul>
<li>Description: Stops a Storm batch indexing topology</li>
<li>Input:
<ul>
<li>stopNow - Stop the topology immediately</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns stop response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Findexing.2Frandomaccess"></a><tt>GET /api/v1/storm/indexing/randomaccess</tt></h3>
<ul>
<li>Description: Retrieves the status of the Storm randomaccess indexing topology</li>
<li>Returns:
<ul>
<li>200 - Returns topology status information</li>
<li>404 - Topology is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Findexing.2Frandomaccess.2Factivate"></a><tt>GET /api/v1/storm/indexing/randomaccess/activate</tt></h3>
<ul>
<li>Description: Activates a Storm randomaccess indexing topology</li>
<li>Returns:
<ul>
<li>200 - Returns activate response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Findexing.2Frandomaccess.2Fdeactivate"></a><tt>GET /api/v1/storm/indexing/randomaccess/deactivate</tt></h3>
<ul>
<li>Description: Deactivates a Storm randomaccess indexing topology</li>
<li>Returns:
<ul>
<li>200 - Returns deactivate response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Findexing.2Frandomaccess.2Fstart"></a><tt>GET /api/v1/storm/indexing/randomaccess/start</tt></h3>
<ul>
<li>Description: Starts a Storm randomaccess indexing topology</li>
<li>Returns:
<ul>
<li>200 - Returns start response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Findexing.2Frandomaccess.2Fstop"></a><tt>GET /api/v1/storm/indexing/randomaccess/stop</tt></h3>
<ul>
<li>Description: Stops a Storm randomaccess indexing topology</li>
<li>Input:
<ul>
<li>stopNow - Stop the topology immediately</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns stop response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Fparser.2Factivate.2F.7Bname.7D"></a><tt>GET /api/v1/storm/parser/activate/{name}</tt></h3>
<ul>
<li>Description: Activates a Storm parser topology</li>
<li>Input:
<ul>
<li>name - Parser name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns activate response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Fparser.2Fdeactivate.2F.7Bname.7D"></a><tt>GET /api/v1/storm/parser/deactivate/{name}</tt></h3>
<ul>
<li>Description: Deactivates a Storm parser topology</li>
<li>Input:
<ul>
<li>name - Parser name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns deactivate response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Fparser.2Fstart.2F.7Bname.7D"></a><tt>GET /api/v1/storm/parser/start/{name}</tt></h3>
<ul>
<li>Description: Starts a Storm parser topology</li>
<li>Input:
<ul>
<li>name - Parser name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns start response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Fparser.2Fstop.2F.7Bname.7D"></a><tt>GET /api/v1/storm/parser/stop/{name}</tt></h3>
<ul>
<li>Description: Stops a Storm parser topology</li>
<li>Input:
<ul>
<li>name - Parser name</li>
<li>stopNow - Stop the topology immediately</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns stop response message</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2F.7Bname.7D"></a><tt>GET /api/v1/storm/{name}</tt></h3>
<ul>
<li>Description: Retrieves the status of a Storm topology</li>
<li>Input:
<ul>
<li>name - Topology name</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns topology status information</li>
<li>404 - Topology is missing</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fstorm.2Fsupervisors"></a><tt>GET /api/v1/storm/supervisors</tt></h3>
<ul>
<li>Description: Retrieves the status of all Storm Supervisors</li>
<li>Returns:
<ul>
<li>200 - Returns a list of the status of all Storm Supervisors</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="PATCH_.2Fapi.2Fv1.2Fupdate.2Fpatch"></a><tt>PATCH /api/v1/update/patch</tt></h3>
<ul>
<li>Description: Update a document with a patch</li>
<li>Input:
<ul>
<li>request - Patch Request
<ul>
<li>guid - The Patch UUID</li>
<li>sensorType - The sensor type</li>
<li>patch - An array of <a class="externalLink" href="https://tools.ietf.org/html/rfc6902">RFC 6902</a> patches.</li>
</ul>
</li>
<li>Example adding a field called <tt>project</tt> with value <tt>metron</tt> to the <tt>bro</tt> message with UUID of <tt>000-000-0000</tt> :
<div>
<div>
<pre class="source">{
&quot;guid&quot; : &quot;000-000-0000&quot;,
&quot;sensorType&quot; : &quot;bro&quot;,
&quot;patch&quot; : [
{
&quot;op&quot;: &quot;add&quot;
, &quot;path&quot;: &quot;/project&quot;
, &quot;value&quot;: &quot;metron&quot;
}
]
}
</pre></div></div>
</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Nothing</li>
<li>404 - Document not found</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fupdate.2Fadd.2Fcomment"></a><tt>POST /api/v1/update/add/comment</tt></h3>
<ul>
<li>Description: Add a comment to an alert</li>
<li>Input:
<ul>
<li>request - Comment add request</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns the complete alert document with comments added.</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="POST_.2Fapi.2Fv1.2Fupdate.2Fremove.2Fcomment"></a><tt>POST /api/v1/update/remove/comment</tt></h3>
<ul>
<li>Description: Remove a comment from an alert</li>
<li>Input:
<ul>
<li>request - Comment remove request</li>
</ul>
</li>
<li>Returns:
<ul>
<li>200 - Returns the complete alert document with comments removed.</li>
</ul>
</li>
</ul></div>
<div class="section">
<h3><a name="GET_.2Fapi.2Fv1.2Fuser"></a><tt>GET /api/v1/user</tt></h3>
<ul>
<li>Description: Retrieves the current user</li>
<li>Returns:
<ul>
<li>200 - Current user</li>
</ul>
</li>
</ul></div></div>
<div class="section">
<h2><a name="Testing"></a>Testing</h2>
<p>Profiles are includes for both the metron-docker and Full Dev environments.</p>
<div class="section">
<h3><a name="metron-docker"></a>metron-docker</h3>
<p>Start the <a href="../../metron-docker/index.html">metron-docker</a> environment. Build the metron-rest module and start it with the Spring Boot Maven plugin:</p>
<div>
<div>
<pre class="source">mvn clean package
mvn spring-boot:run -Drun.profiles=docker,dev
</pre></div></div>
<p>The metron-rest application will be available at <a class="externalLink" href="http://localhost:8080/swagger-ui.html#/">http://localhost:8080/swagger-ui.html#/</a>.</p></div>
<div class="section">
<h3><a name="Full_Dev"></a>Full Dev</h3>
<p>Start the <a href="../../metron-deployment/development/centos6/index.html">development environment</a>. Build the metron-rest module and start it with the Spring Boot Maven plugin:</p>
<div>
<div>
<pre class="source">mvn clean package
mvn spring-boot:run -Drun.profiles=vagrant,dev
</pre></div></div>
<p>The metron-rest application will be available at <a class="externalLink" href="http://localhost:8080/swagger-ui.html#/">http://localhost:8080/swagger-ui.html#/</a>.</p>
<p>To run the application locally on the Full Dev host (node1), follow the <a href="#Installation">Installation</a> instructions above. Then set the METRON_SPRING_PROFILES_ACTIVE variable in <tt>/etc/default/metron</tt>:</p>
<div>
<div>
<pre class="source">METRON_SPRING_PROFILES_ACTIVE=&quot;vagrant,dev&quot;
</pre></div></div>
<p>and start the application:</p>
<div>
<div>
<pre class="source">service metron-rest start
</pre></div></div>
<p>In a cluster with Kerberos enabled, update the security settings in <tt>/etc/default/metron</tt>. Security is disabled by default in the <tt>vagrant</tt> Spring profile so that setting must be overriden with the METRON_SPRING_OPTIONS variable:</p>
<div>
<div>
<pre class="source">METRON_SPRING_PROFILES_ACTIVE=&quot;vagrant,dev&quot;
METRON_JVMFLAGS=&quot;-Djava.security.auth.login.config=$METRON_HOME/client_jaas.conf&quot;
METRON_SPRING_OPTIONS=&quot;--kerberos.enabled=true&quot;
</pre></div></div>
<p>The metron-rest application will be available at <a class="externalLink" href="http://node1:8082/swagger-ui.html#/">http://node1:8082/swagger-ui.html#/</a>.</p></div></div>
<div class="section">
<h2><a name="License"></a>License</h2>
<p>This project depends on the Java Transaction API. See <a class="externalLink" href="https://java.net/projects/jta-spec/">https://java.net/projects/jta-spec/</a> for more details.</p></div>
</div>
</div>
</div>
<hr/>
<footer>
<div class="container-fluid">
<div class="row-fluid">
© 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo,
and the Apache Metron project logo are trademarks of The Apache Software Foundation.
</div>
</div>
</footer>
</body>
</html>