| <!DOCTYPE html> |
| <!-- |
| | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/ansible/roles/opentaxii/index.md at 2019-05-14 |
| | Rendered using Apache Maven Fluido Skin 1.7 |
| --> |
| <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
| <head> |
| <meta charset="UTF-8" /> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0" /> |
| <meta name="Date-Revision-yyyymmdd" content="20190514" /> |
| <meta http-equiv="Content-Language" content="en" /> |
| <title>Metron – OpenTAXII</title> |
| <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" /> |
| <link rel="stylesheet" href="../../../../css/site.css" /> |
| <link rel="stylesheet" href="../../../../css/print.css" media="print" /> |
| <script type="text/javascript" src="../../../../js/apache-maven-fluido-1.7.min.js"></script> |
| <script type="text/javascript"> |
| $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } ); |
| </script> |
| </head> |
| <body class="topBarDisabled"> |
| <div class="container-fluid"> |
| <div id="banner"> |
| <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../../images/metron-logo.png" alt="Apache Metron" width="148px" height="48px"/></a></div> |
| <div class="pull-right"></div> |
| <div class="clear"><hr/></div> |
| </div> |
| |
| <div id="breadcrumbs"> |
| <ul class="breadcrumb"> |
| <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li> |
| <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> |
| <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> |
| <li class="active ">OpenTAXII</li> |
| <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2019-05-14</li> |
| <li id="projectVersion" class="pull-right">Version: 0.7.1</li> |
| </ul> |
| </div> |
| <div class="row-fluid"> |
| <div id="leftColumn" class="span2"> |
| <div class="well sidebar-nav"> |
| <ul class="nav nav-list"> |
| <li class="nav-header">User Documentation</li> |
| <li><a href="../../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a> |
| <ul class="nav nav-list"> |
| <li><a href="../../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li> |
| <li><a href="../../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li> |
| <li><a href="../../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li> |
| <li><a href="../../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li> |
| <li><a href="../../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li> |
| <li><a href="../../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-down"></span>Deployment</a> |
| <ul class="nav nav-list"> |
| <li><a href="../../../../metron-deployment/Kerberos-ambari-setup.html" title="Kerberos-ambari-setup"><span class="none"></span>Kerberos-ambari-setup</a></li> |
| <li><a href="../../../../metron-deployment/Kerberos-manual-setup.html" title="Kerberos-manual-setup"><span class="none"></span>Kerberos-manual-setup</a></li> |
| <li><a href="../../../../metron-deployment/amazon-ec2/index.html" title="Amazon-ec2"><span class="none"></span>Amazon-ec2</a></li> |
| <li><a href="../../../../metron-deployment/ansible/index.html" title="Ansible"><span class="icon-chevron-down"></span>Ansible</a> |
| <ul class="nav nav-list"> |
| <li><a href="../../../../metron-deployment/ansible/roles/index.html" title="Roles"><span class="icon-chevron-down"></span>Roles</a> |
| <ul class="nav nav-list"> |
| <li class="active"><a href="#"><span class="none"></span>Opentaxii</a></li> |
| <li><a href="../../../../metron-deployment/ansible/roles/pcap_replay/index.html" title="Pcap_replay"><span class="none"></span>Pcap_replay</a></li> |
| <li><a href="../../../../metron-deployment/ansible/roles/sensor-stubs/index.html" title="Sensor-stubs"><span class="none"></span>Sensor-stubs</a></li> |
| <li><a href="../../../../metron-deployment/ansible/roles/sensor-test-mode/index.html" title="Sensor-test-mode"><span class="none"></span>Sensor-test-mode</a></li> |
| </ul> |
| </li> |
| </ul> |
| </li> |
| <li><a href="../../../../metron-deployment/development/index.html" title="Development"><span class="icon-chevron-right"></span>Development</a></li> |
| <li><a href="../../../../metron-deployment/other-examples/index.html" title="Other-examples"><span class="icon-chevron-right"></span>Other-examples</a></li> |
| <li><a href="../../../../metron-deployment/packaging/ambari/index.html" title="Ambari"><span class="icon-chevron-right"></span>Ambari</a></li> |
| <li><a href="../../../../metron-deployment/packaging/docker/ansible-docker/index.html" title="Ansible-docker"><span class="none"></span>Ansible-docker</a></li> |
| <li><a href="../../../../metron-deployment/packaging/docker/deb-docker/index.html" title="Deb-docker"><span class="none"></span>Deb-docker</a></li> |
| <li><a href="../../../../metron-deployment/packaging/docker/rpm-docker/index.html" title="Rpm-docker"><span class="none"></span>Rpm-docker</a></li> |
| <li><a href="../../../../metron-deployment/packaging/packer-build/index.html" title="Packer-build"><span class="none"></span>Packer-build</a></li> |
| </ul> |
| </li> |
| <li><a href="../../../../metron-interface/index.html" title="Interface"><span class="icon-chevron-right"></span>Interface</a></li> |
| <li><a href="../../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-right"></span>Platform</a></li> |
| <li><a href="../../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li> |
| <li><a href="../../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li> |
| <li><a href="../../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li> |
| <li><a href="../../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li> |
| <li><a href="../../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li> |
| </ul> |
| </li> |
| </ul> |
| <hr /> |
| <div id="poweredBy"> |
| <div class="clear"></div> |
| <div class="clear"></div> |
| <div class="clear"></div> |
| <div class="clear"></div> |
| <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../../images/logos/maven-feather.png" /></a> |
| </div> |
| </div> |
| </div> |
| <div id="bodyColumn" class="span10" > |
| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| <h1>OpenTAXII</h1> |
| <p><a name="OpenTAXII"></a></p> |
| <p>Installs <a class="externalLink" href="https://github.com/EclecticIQ/OpenTAXII">OpenTAXII</a> as a deamon that can be launched via a SysV service script. The complementary client implementation, <a class="externalLink" href="https://github.com/EclecticIQ/cabby">Cabby</a> is also installed.</p> |
| <p>OpenTAXII is a robust Python implementation of TAXII Services that delivers a rich feature set and friendly pythonic API. <a class="externalLink" href="https://stixproject.github.io/">TAXII</a> (Trusted Automated eXchange of Indicator Information) is a collection of specifications defining a set of services and message exchanges used for sharing cyber threat intelligence information between parties.</p> |
| <div class="section"> |
| <h2><a name="Getting_Started"></a>Getting Started</h2> |
| <p>After deployment completes the OpenTAXII service is installed and running. A set of <a class="externalLink" href="http://hailataxii.com/">Hail a TAXII</a> threat intel collections have been defined and configured. Use the <tt>status</tt> option to view the collections that have been defined.</p> |
| |
| <div> |
| <div> |
| <pre class="source">$ service opentaxii status |
| Checking opentaxii... Running |
| guest.phishtank_com 0 |
| guest.Abuse_ch 0 |
| guest.CyberCrime_Tracker 0 |
| guest.EmergingThreats_rules 0 |
| guest.Lehigh_edu 0 |
| guest.MalwareDomainList_Hostlist 0 |
| guest.blutmagie_de_torExits 0 |
| guest.dataForLast_7daysOnly 0 |
| guest.dshield_BlockList 0 |
| </pre></div></div> |
| |
| <p>Notice that each collections contain zero records. None of the data is automatically synced during deployment. To sync the data manually use the <tt>sync</tt> option as defined below. The following example does not provide a begin and end time so the data will be fetched for the current day only.</p> |
| |
| <div> |
| <div> |
| <pre class="source"># service opentaxii sync guest.blutmagie_de_torExits |
| 2016-04-21 20:34:42,511 INFO: Starting new HTTP connection (1): localhost |
| 2016-04-21 20:34:42,540 INFO: Response received for Inbox_Message from http://localhost:9000/services/inbox |
| 2016-04-21 20:34:42,542 INFO: Sending Inbox_Message to http://localhost:9000/services/inbox |
| ... |
| 2016-04-21 20:34:42,719 INFO: Response received for Poll_Request from http://localhost:9000/services/poll |
| 2016-04-21 20:34:42,719 INFO: Content blocks count: 1618, is partial: False |
| </pre></div></div> |
| |
| <p>The OpenTAXII service now contains 1,618 threat intel records indicating Tor Exit nodes.</p> |
| |
| <div> |
| <div> |
| <pre class="source">[root@source ~]# service opentaxii status |
| Checking opentaxii... Running |
| guest.phishtank_com 0 |
| guest.Abuse_ch 0 |
| guest.CyberCrime_Tracker 0 |
| guest.EmergingThreats_rules 0 |
| guest.Lehigh_edu 0 |
| guest.MalwareDomainList_Hostlist 0 |
| guest.blutmagie_de_torExits 1618 |
| guest.dataForLast_7daysOnly 0 |
| guest.dshield_BlockList 0 |
| </pre></div></div> |
| </div> |
| <div class="section"> |
| <h2><a name="Usage"></a>Usage</h2> |
| <p>A standard SysV script has been installed to manage OpenTAXII. The following functions are available.</p> |
| <p><tt>start</tt> <tt>stop</tt> <tt>restart</tt> the OpenTAXII service</p> |
| <p><tt>status</tt> of the OpenTAXII service. The command displays the collections that have been defined and the number of records in each.</p> |
| |
| <div> |
| <div> |
| <pre class="source">$ service opentaxii status |
| Checking opentaxii... Running |
| guest.phishtank_com 984 |
| guest.Abuse_ch 45 |
| guest.CyberCrime_Tracker 482 |
| guest.EmergingThreats_rules 0 |
| guest.Lehigh_edu 1030 |
| guest.MalwareDomainList_Hostlist 84 |
| guest.blutmagie_de_torExits 3236 |
| guest.dataForLast_7daysOnly 3377 |
| guest.dshield_BlockList 0 |
| </pre></div></div> |
| |
| <p><tt>setup</tt> Initializes the services and collections required to operate the OpenTAXII service. This will destroy all existing data. The user is prompted to continue before any data is destroyed.</p> |
| |
| <div> |
| <div> |
| <pre class="source"># service opentaxii setup |
| WARNING: force reset and destroy all opentaxii data? [Ny]: y |
| Stopping opentaxii ..Ok |
| 2016-04-21T19:56:01.886157Z [opentaxii.server] info: api.persistence.loaded {timestamp=2016-04-21T19:56:01.886157Z, logger=opentaxii.server, api_class=SQLDatabaseAPI, event=api.persistence.loaded, level=info} |
| 2016-04-21T19:56:01.896503Z [opentaxii.server] info: api.auth.loaded {timestamp=2016-04-21T19:56:01.896503Z, logger=opentaxii.server, api_class=SQLDatabaseAPI, event=api.auth.loaded, level=info} |
| 2016-04-21T19:56:01.896655Z [opentaxii.server] info: taxiiserver.configured {timestamp=2016-04-21T19:56:01.896655Z, logger=opentaxii.server, event=taxiiserver.configured, level=info} |
| ... |
| Ok |
| </pre></div></div> |
| |
| <p><tt>sync [collection] [begin-at] [end-at]</tt> Syncs the threat intel data available at <a class="externalLink" href="http://hailataxii.com/">Hail a TAXII</a>. If no begin and end date is provided then data is synced over the current day only.</p> |
| <ul> |
| |
| <li><tt>collection</tt> Name of the collection to sync.</li> |
| <li><tt>begin-at</tt> Exclusive begin of time window; ISO8601</li> |
| <li><tt>end-at</tt> Inclusive end of time window; ISO8601</li> |
| </ul> |
| |
| <div> |
| <div> |
| <pre class="source">$ service opentaxii sync guest.phishtank_com |
| + /usr/local/opentaxii/opentaxii-venv/bin/taxii-proxy --poll-path http://hailataxii.com/taxii-data --poll-collection guest.phishtank_com --inbox-path http://localhost:9000/services/guest.phishtank_com-inbox --inbox-collection guest.phishtank_com --binding urn:stix.mitre.org:xml:1.1.1 --begin 2016-04-21 --end 2016-04-22 |
| 2016-04-21 17:36:23,778 INFO: Sending Poll_Request to http://hailataxii.com/taxii-data |
| 2016-04-21 17:36:23,784 INFO: Starting new HTTP connection (1): hailataxii.com |
| 2016-04-21 17:36:24,175 INFO: Response received for Poll_Request from http://hailataxii.com/taxii-data |
| 2016-04-21 17:36:24,274 INFO: Sending Inbox_Message to http://localhost:9000/services/guest.phishtank_com-inbox |
| ... |
| 2016-04-21 17:36:34,867 INFO: Response received for Poll_Request from http://localhost:9000/services/guest.phishtank_com-poll |
| 2016-04-21 17:36:34,868 INFO: Content blocks count: 6993, is partial: False |
| </pre></div></div> |
| |
| <div class="section"> |
| <h3><a name="Troubleshooting"></a>Troubleshooting</h3> |
| <p>Should you need to explore the installation, here are instructions on doing so.</p> |
| <p>OpenTAXII is installed in a virtual environment. Before exploring the environment run the following commands to perform the necessary setup. The specific paths may change depending on your Ansible settings.</p> |
| |
| <div> |
| <div> |
| <pre class="source">export LD_LIBRARY_PATH=/opt/rh/python27/root/usr/lib64 |
| export OPENTAXII_CONFIG=/usr/local/opentaxii/etc/opentaxii-conf.yml |
| cd /usr/local/opentaxii |
| . opentaxii-venv/bin/activate |
| </pre></div></div> |
| |
| <p>Discover available services.</p> |
| |
| <div> |
| <div> |
| <pre class="source">taxii-discovery --discovery http://localhost:9000/services/discovery |
| taxii-discovery --discovery http://hailataxii.com/taxii-data |
| </pre></div></div> |
| |
| <p>Explore available collections.</p> |
| |
| <div> |
| <div> |
| <pre class="source">taxii-collections --discovery http://localhost:9000/services/discovery |
| taxii-collections --discovery http://hailataxii.com/taxii-data |
| </pre></div></div> |
| |
| <p>Read data from a collection.</p> |
| |
| <div> |
| <div> |
| <pre class="source">taxii-poll --discovery http://localhost:9000/services/discovery -c guest.phishtank_com |
| taxii-poll --discovery http://hailataxii.com/taxii-data -c guest.phishtank_com --begin 2016-04-20 |
| </pre></div></div> |
| |
| <p>Manually load data into a collection.</p> |
| |
| <div> |
| <div> |
| <pre class="source">taxii-push \ |
| --discovery http://localhost:9000/services/discovery \ |
| --dest phishtank \ |
| --content-file data.xml \ |
| --username guest \ |
| --password guest |
| </pre></div></div> |
| |
| <p>Fetch data from a remote service and mirror it locally.</p> |
| |
| <div> |
| <div> |
| <pre class="source">taxii-proxy --poll-path http://hailataxii.com/taxii-data \ |
| --poll-collection guest.phishtank_com \ |
| --inbox-path http://localhost:9000/services/guest.phishtank_com-inbox \ |
| --inbox-collection guest.phishtank_com \ |
| --binding urn:stix.mitre.org:xml:1.1.1 \ |
| --inbox-username guest \ |
| --inbox-password guest \ |
| --begin 2016-04-20 |
| </pre></div></div></div></div> |
| </div> |
| </div> |
| </div> |
| <hr/> |
| <footer> |
| <div class="container-fluid"> |
| <div class="row-fluid"> |
| © 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo, |
| and the Apache Metron project logo are trademarks of The Apache Software Foundation. |
| </div> |
| </div> |
| </footer> |
| </body> |
| </html> |
