| <!doctype html> | |
| <!--[if lt IE 7]><html lang="en-US" class="no-js lt-ie9 lt-ie8 lt-ie7"><![endif]--> | |
| <!--[if (IE 7)&!(IEMobile)]><html lang="en-US" class="no-js lt-ie9 lt-ie8"><![endif]--> | |
| <!--[if (IE 8)&!(IEMobile)]><html lang="en-US" class="no-js lt-ie9"><![endif]--> | |
| <!--[if gt IE 8]><!--> | |
| <html lang="en-US" class="no-js"> | |
| <!--<![endif]--> | |
| <head> | |
| <meta charset="utf-8"> | |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> | |
| <title>Apache Spot</title> | |
| <meta name="HandheldFriendly" content="True"> | |
| <meta name="MobileOptimized" content="320"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1"/> | |
| <link rel="apple-touch-icon" href="library/images/apple-touch-icon.png"> | |
| <link rel="icon" href="favicon.png"> | |
| <!--[if IE]> | |
| <link rel="shortcut icon" href="http://spot.incubator.apache.org/favicon.ico"> | |
| <![endif]--> | |
| <meta name="msapplication-TileColor" content="#f01d4f"> | |
| <meta name="msapplication-TileImage" content="library/images/win8-tile-icon.png"> | |
| <meta name="theme-color" content="#121212"> | |
| <link rel='dns-prefetch' href='//fonts.googleapis.com' /> | |
| <link rel='dns-prefetch' href='//s.w.org' /> | |
| <link rel="alternate" type="application/rss+xml" title="Apache Spot » Feed" href="feed/" /> | |
| <link rel='stylesheet' id='googleFonts-css' href='http://fonts.googleapis.com/css?family=Lato%3A400%2C700%2C400italic%2C700italic' type='text/css' media='all' /> | |
| <link rel='stylesheet' id='bones-stylesheet-css' href='library/css/style.css' type='text/css' media='all' /> | |
| <!--[if lt IE 9]> | |
| <link rel='stylesheet' id='bones-ie-only-css' href='http://spot.incubator.apache.org/library/css/ie.css' type='text/css' media='all' /> | |
| <![endif]--> | |
| <link rel='stylesheet' id='mm-css-css' href='library/css/meanmenu.css' type='text/css' media='all' /> | |
| <script type='text/javascript' src='library/js/libs/modernizr.custom.min.js'></script> | |
| <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js"></script> | |
| <script type='text/javascript' src='library/js/jquery-migrate.min.js'></script> | |
| <script type='text/javascript' src='library/js/jquery.meanmenu.js'></script> | |
| <script> | |
| (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ | |
| (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), | |
| m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) | |
| })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); | |
| ga('create', 'UA-87470508-1', 'auto'); | |
| ga('send', 'pageview'); | |
| </script> | |
| </head> | |
| <body class="home"> | |
| <div id="container"> | |
| <header class="header"> | |
| <div id="inner-header" class="wrap cf"> | |
| <p id="logo" class="h1" itemscope itemtype="http://schema.org/Organization"> | |
| <a href="http://spot.incubator.apache.org/" rel="nofollow"><img src="library/images/logo.png" alt="Apache Spot" /></a> | |
| </p> | |
| <nav> | |
| <ul id="menu-main-menu" class="nav top-nav cf"> | |
| <li id="menu-item-129" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-129"> | |
| <a href="get-started">Get Started</a> | |
| <ul class="sub-menu"> | |
| <li><a href="get-started">Get Started</a></li> | |
| <li><a href="get-started/supporting-apache">Supporting Apache</a></li> | |
| <li><a href="get-started/environment">Environment</a></li> | |
| <li><a href="get-started/architecture">Architecture</a></li> | |
| <li><a href="get-started/demo">Demo</a></li> | |
| </ul> | |
| </li> | |
| <li id="menu-item-5" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-5"> | |
| <a href="download">Download</a> | |
| </li> | |
| <li id="menu-item-130" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-130"> | |
| <a href="community">Community</a> | |
| <ul class="sub-menu com-sm"> | |
| <li class="dropmenu-head">Get in Touch</li> | |
| <li><a href="community" class="mail">Mailing Lists</a></li> | |
| <li class="divider"></li> | |
| <li><a href="community/committers">Project Committers</a></li> | |
| <li><a href="community/contribute">How to Contribute</a></li> | |
| <li class="divider"></li> | |
| <li class="dropmenu-head">Developer Resources</li> | |
| <li><a href="https://github.com/apache/incubator-spot" target="_blank" class="github">Github</a></li> | |
| <li><a href="https://issues.apache.org/jira/browse/SPOT/" target="_blank" class="jira">JIRA Issue Tracker</a></li> | |
| <li><a href="https://cwiki.apache.org/confluence/pages/viewpage.action?spaceKey=SPOT&title=Apache+Spot+%28Incubating%29+Home" target="_blank" class="">Confluence Site</a></li> | |
| <li class="divider"></li> | |
| <li class="dropmenu-head">Social Media</li> | |
| <li><a href="https://twitter.com/ApacheSpot" target="_blank" class="twitter-icon">Twitter</a></li> | |
| </ul> | |
| </li> | |
| <li id="menu-item-106" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-106"> | |
| <a href="doc">Documentation</a> | |
| </li> | |
| <li class="menu-item menu-item-has-children"> | |
| <a href="#">Project Components</a> | |
| <ul class="sub-menu"> | |
| <li><a href="project-components/ingestion">Ingestion</a></li> | |
| <li><a href="project-components/machine-learning">Machine Learning</a></li> | |
| <li><a href="project-components/suspicious-connects-analysis">Suspicous Connects Analysis</a></li> | |
| <li><a href="project-components/visualization">Visualization</a></li> | |
| <li class="under-dev">Under Development</li> | |
| <li><a href="project-components/open-data-models">Open Data Models</a></li> | |
| </ul> | |
| </li> | |
| <li id="menu-item-13" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-13"> | |
| <a href="blog">Blog</a> | |
| </li> | |
| </ul> | |
| </nav> | |
| </div> | |
| </header> | |
| <div id="mobile-nav"></div> | |
| <div id="masthead"> | |
| <div class="wrap cf"> | |
| <div class="m-all t-1of2 d-1of2"> | |
| <h1>Apache Spot (Incubating)</h1> | |
| <h1 class="thin">A Community Approach to Fighting Cyber Threats</h1> | |
| </div> | |
| <div class="m-all t-1of2 d-1of2 contribute"> | |
| <h3>Help Spot expand its hunting ground to see deeper into the darkness of cyber threats.</h3> | |
| <p class="btn-margin"><a href="community/contribute" class="y-btn">Become a Contributor</a></p> | |
| </div> | |
| </div> | |
| </div> | |
| <div id="at-a-glance"> | |
| <div class="wrap cf"> | |
| <h1>Apache Spot at a Glance</h1> | |
| <p>Apache Spot is a community-driven cybersecurity project, built from the ground up, to bring advanced analytics to all IT <a href="doc/index.html#telemetry">Telemetry</a> data on an open, scalable platform. It is an open source software for leveraging insights from flow and packet analysis. Spot expedites threat detection, investigation, and remediation via <a href="doc/index.html#machine-learning">machine learning</a> and consolidates all enterprise security data into a comprehensive IT telemetry hub based on open data models. Spot’s scalability and machine learning capabilities support an ecosystem of ML-based applications that can run simultaneously on a single, shared, enriched data set to provide organizations with maximum analytic flexibility. </p> | |
| <p>Apache Spot helps enterprises and service providers gain insight on their computed environments through transparency of service delivery and identification of potential security threats or attacks happening among resources operating at cloud scale. While current threat intelligence tools help, identifying unknown threats and attacks remains a challenge. Spot harnesses a diverse community of expertise from Centrify, Cloudera, Cybraics, Endgame, Intel, Jask, Streamsets, and Webroot. Apache Spot provides tools to accelerate companies’ ability to expose suspicious connections and previously unseen attacks using flow and packet analysis technologies.</p> | |
| </div> | |
| </div> | |
| <div id="content"> | |
| <div id="inner-content" class="wrap cf"> | |
| <main id="main" class="m-all t-all d-all cf" role="main" itemscope itemprop="mainContentOfPage" itemtype="http://schema.org/Blog"> | |
| <h1 class="center">Apache Spot Advantages</h1> | |
| <p>Apache Spot uses the best tools just to provide the best functionality preventing attacks, identifying unknown threats, and providing cybersecurity. Perimeter Flows, DNS, Proxy and Internal Flows provide remarkable advantages creating fast and efficient data analysis.</p> | |
| <p style="text-align: center;"><img src="library/images/advantages-1.png" alt="advantages" width="958" height="433" class="aligncenter size-full" /> | |
| </p> | |
| </main> | |
| </div> | |
| </div> | |
| <div id="and"> | |
| <div class="wrap cf"> | |
| <div class="and-row"> | |
| <div class="and-cell"> | |
| <img src="library/images/and.png" alt="" /> | |
| </div> | |
| <div class="and-cell"> | |
| <p> | |
| Apache Spot is functional after just one day and just keeps improving through feedback and machine learning. | |
| </p> | |
| </div> | |
| </div> | |
| </div> | |
| </div> | |
| <div id="how-it-works"> | |
| <div class="wrap cf"> | |
| <div class="m-all t-1of3 d-1of3"> | |
| <h1>How It Works</h1> | |
| <p> | |
| Apache Spot uses machine learning as a filter for separating bad traffic from benign and to characterize the unique behavior of network traffic. A proven process, of context enrichment, noise filtering, whitelisting and heuristics, is also applied to network data to produce a shortlist of most likely security threats. | |
| </p> | |
| <p class="btn-margin"> | |
| <a href="https://github.com/apache/incubator-spot" class="y-btn">More Info</a> | |
| </p> | |
| </div> | |
| <div class="m-all t-2of3 d-2of3"> | |
| <img src="library/images/how-it-works.png" alt="" /> | |
| </div> | |
| </div> | |
| <div class="wrap cf"> | |
| <h3>Parallel Ingest Framework.</h3> | |
| <p>The systems uses opens source’s optimized decoders for binary flow and pack data, to load into HDFS and data structures inside Hadoop. The decoded data is stored in multiple formats so it is available for searching, used by machine learning, transfer to law enforcement, or inputs to other systems.</p> | |
| <h3>Machine Learning.</h3> | |
| <p>The system uses a combination of Apache Spark to run scalable machine learning algorithms. The machine learning component works not only as a filter for separating bad traffic from benign, but also as a way to characterize the unique behavior of network traffic in an organization.</p> | |
| <h3>Operational Analytics.</h3> | |
| <p>In addition to machine learning, a proven process of context enrichment, noise filtering, whitelisting, and heuristics are applied to network data to produce a short list of the most likely patterns, which may be security threats.</p> | |
| </div> | |
| </div> | |
| <div id="key-features"> | |
| <div class="wrap cf"> | |
| <h1>Key Features</h1> | |
| <div class="m-all t-1of2 d-1of2"> | |
| <div class="table"> | |
| <div class="table-row"> | |
| <div class="table-cell"> | |
| <img src="library/images/magnify-icon.png" alt="" /> | |
| </div> | |
| <div class="table-cell"> | |
| <h3>Suspicious DNS packets</h3> | |
| </div> | |
| </div> | |
| </div> | |
| <p> | |
| Apache Spot is capable of performing deep-packet inspection of DNS traffic to build a profile of probable and improbable DNS payloads. After visualizing, normalizing, and conducting pattern searches, the analyst has a shortlist of the most likely threats present in DNS traffic. | |
| </p> | |
| </div> | |
| <div class="m-all t-1of2 d-1of2"> | |
| <div class="table"> | |
| <div class="table-row"> | |
| <div class="table-cell"> | |
| <img src="library/images/threat-icon.png" alt="" /> | |
| </div> | |
| <div class="table-cell"> | |
| <h3>Threat Incident and Response</h3> | |
| </div> | |
| </div> | |
| </div> | |
| <p> | |
| Given an IP address, Apache Spot gathers all the characteristics about the communication associated with it – the “social network” of that IP address. Then Apache Spot builds a timeline of the conversations that originated with that IP. | |
| </p> | |
| </div> | |
| <div class="m-all t-1of2 d-1of2"> | |
| <div class="table"> | |
| <div class="table-row"> | |
| <div class="table-cell"> | |
| <img src="library/images/connects.png" alt="" /> | |
| </div> | |
| <div class="table-cell"> | |
| <h3>Suspicious Connects</h3> | |
| </div> | |
| </div> | |
| </div> | |
| <p> | |
| Apache Spot uses advanced machine learning to build a model of the machines on the network and their communication patterns. The connections between the machines that are the lowest probability are then visualized, filtered for noise, and searched for known patterns. The result is the most likely threat patterns in the data, a few hundred flows picked from billions. | |
| </p> | |
| </div> | |
| <div class="m-all t-1of2 d-1of2"> | |
| <div class="table"> | |
| <div class="table-row"> | |
| <div class="table-cell"> | |
| <img src="library/images/storyboard-icon.png" alt="" /> | |
| </div> | |
| <div class="table-cell"> | |
| <h3>Storyboard</h3> | |
| </div> | |
| </div> | |
| </div> | |
| <p> | |
| After an analyst has investigated a threat, the need still exists to communicate the event up and across the organization. A “dashboard” gives quick answers to the questions you already know to ask. What the analyst requires is a “storyboard,” something that tells who, what, where, and how of the story in words and interactive visualizations. | |
| </p> | |
| </div> | |
| <div class="m-all t-1of2 d-1of2"> | |
| <div class="table"> | |
| <div class="table-row"> | |
| <div class="table-cell"> | |
| <img src="library/images/SPOT_OpenDataModel-Icon_v1.png" alt="" /> | |
| </div> | |
| <div class="table-cell"> | |
| <h3>Open Data Models</h3> | |
| </div> | |
| </div> | |
| </div> | |
| <p> | |
| Apache Spot delivers common Open Data Models for network, endpoint and user, providing a standard format of enriched event data that makes it easier to integrate cross application data to gain complete enterprise visibility and develop net new analytic functionality. Spot’s Open Data Models helps organizations quickly share new analytics with one another as new threats are discovered. | |
| </p> | |
| </div> | |
| <div class="m-all t-1of2 d-1of2"> | |
| <div class="table"> | |
| <div class="table-row"> | |
| <div class="table-cell"> | |
| <img src="library/images/SPOT_Collabration-Icon_v2.png" alt="" /> | |
| </div> | |
| <div class="table-cell"> | |
| <h3>Collaboration</h3> | |
| </div> | |
| </div> | |
| </div> | |
| <p> | |
| Apache Spot's Open Data Models help organizations to quickly share new analytics among them as new threats are discovered. And, with Hadoop, organizations able to run these analytics against comprehensive historic data sets, helping organizations identify past threats that have slipped through the cracks. With this capability, Spot aims to give security professionals the ability to collaborate like cybercriminals do. | |
| </p> | |
| </div> | |
| <div class="cf"></div> | |
| <p class="btn-margin"> | |
| <a href="https://github.com/apache/incubator-spot" class="y-btn">More Info</a> | |
| </p> | |
| </div> | |
| </div> | |
| <div id="open-data-models"> | |
| <div class="wrap cf"> | |
| <h1 class="center">Apache Spot Open Data Models (ODM)</h1> | |
| <div class="m-all t-1of2 d-1of2 center"> | |
| <img src="library/images/odm.png" alt="Apache Spot Open Data Models"> | |
| </div> | |
| <div class="m-all t-1of2 d-1of2"> | |
| <p>The primary use case initially supported by Spot includes Network Traffic Analysis for network flows (Netflow, sflow, etc.), DNS and Proxy. The Spot open data model strategy aims to extend Spot capabilities to support a broader set of cybersecurity use cases.</p> | |
| <h3>ODM at a Glance</h3> | |
| <ul> | |
| <li>Includes a growing catalog of packaged ingestion pipelines for common data sources</li> | |
| <li>Enriched events provide full context leading to better analytics and faster incident response</li> | |
| <li>Organizations maintain and control a single copy of their security data</li> | |
| </ul> | |
| <p class="btn-margin"><a href="project-components/open-data-models/" class="y-btn">Read More</a></p> | |
| </div> | |
| </div> | |
| </div> | |
| <div id="user-stories"> | |
| <div class="wrap cf"> | |
| <h1>Spot Fosters a Rich Application Ecosystem</h1> | |
| <p class="btn-margin center"> | |
| Spot accelerates the development of cybersecurity applications by providing a cybersecurity analytics framework. This means more solutions can be created faster. This is because Spot allows organizations to focus developing the analytics and visualizations for applications that discover cybercrime rather than spending time building systems to ingest, integrate, store, and process myriad volumes or varieties of security data. | |
| </p> | |
| <p class="center">Join the Apache Spot community and collaborate with us using a common framework.</p> | |
| </div> | |
| </div> | |
| <div id="use-case"> | |
| <div class="wrap cf"> | |
| <h1>Use Case</h1> | |
| <div class="quotes"> | |
| <div class="table"> | |
| <div class="table-row"> | |
| <div class="table-cell quote-man"> | |
| <img src="library/images/Apache-Spot-Icon_Network-Traffic.png" alt="" /> | |
| </div> | |
| <div class="table-cell"> | |
| <h3>Network Traffic Analytics</h3> | |
| <p> | |
| Spot allows organizations to detect potentially malicious activity by identifying suspicious network connections, by analyzing large amounts of <a href="doc/index.html#netflow">netflow</a>, <a href="doc/index.html#dns">DNS</a>, <a href="doc/index.html#proxy">proxy</a> data with algorithms that are available out of the box. | |
| </p> | |
| </div> | |
| </div> | |
| </div> | |
| <div class="table"> | |
| <div class="table-row"> | |
| <div class="table-cell quote-man"> | |
| <img src="library/images/Apache-Spot-Icon_Reduction-Mean-Time.png" alt="" /> | |
| </div> | |
| <div class="table-cell"> | |
| <h3>Reduction of mean time to incident detection & resolution (MTTR)</h3> | |
| <p> | |
| Spot provides the capability for a central data store that houses ALL the data needed to facilitate an investigation, returning investigative query results in seconds and minutes (vs. hours and days), which effectively reduces incident <a href="doc/index.html#mttr">MTTR</a> and minimizes the adverse impacts of a breach. | |
| </p> | |
| </div> | |
| </div> | |
| </div> | |
| <div class="table"> | |
| <div class="table-row"> | |
| <div class="table-cell quote-man"> | |
| <img src="library/images/Apache-Spot-Icon_Threat-Hunting.png" alt="" /> | |
| </div> | |
| <div class="table-cell"> | |
| <h3>Threat Hunting</h3> | |
| <p> | |
| Spot improves the efficacy of threat hunting by providing the analytic flexibility to perform <a href="doc/index.html#ad-hoc">ad-hoc</a> searches and queries over vast amounts of data, as well as applying ad-hoc algorithms to detect the needle in the haystack. | |
| </p> | |
| </div> | |
| </div> | |
| </div> | |
| <div class="table"> | |
| <div class="table-row"> | |
| <div class="table-cell quote-man"> | |
| <img src="library/images/Apache-Spot-Icon_Cybersecurity-Data-Management.png" alt="" /> | |
| </div> | |
| <div class="table-cell"> | |
| <h3>Cybersecurity Data Management</h3> | |
| <p> | |
| Offloading data from legacy cybersecurity systems (e.g., <a href="doc/index.html#siem">SIEMs</a>) to Spot delivers immediate economic value, because of the cost of data storage and processing with Hadoop. It also opens up future value as organizations deploy one of the many analytics use cases on their newly formed security data hub. | |
| </p> | |
| </div> | |
| </div> | |
| </div> | |
| </div> | |
| </div> | |
| <div class="arrow"> | |
| <div class="wrap cf"> | |
| <p> | |
| Identify the needle in the haystack with <strong>patterns</strong> that provide insight into potential threats. | |
| </p> | |
| </div> | |
| </div> | |
| </div> | |
| <div id="more-info"> | |
| <div class="wrap cf"> | |
| <p> | |
| <a href="https://github.com/apache/incubator-spot" class="y-btn" target="_blank">More Info</a> | |
| </p> | |
| <p style="margin-top:50px;"><img src="library/images/apache-incubator.png" alt="Apache Incubator" /> | |
| </p> | |
| <p class="disclaimer"> | |
| Apache Spot is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. | |
| </p> | |
| <p class="disclaimer"> | |
| The contents of this website are © 2020 Apache Software Foundation under the terms of the Apache License v2. Apache Spot and its logo are trademarks of the Apache Software Foundation. | |
| </p> | |
| </div> | |
| </div> | |
| <footer class="footer" role="contentinfo" itemscope itemtype="http://schema.org/WPFooter"> | |
| <div id="inner-footer" class="wrap cf"> | |
| <p class="source-org copyright" style="text-align:center;"> | |
| © 2020 Apache Spot. | |
| </p> | |
| </div> | |
| </footer> | |
| </div> | |
| <a href="#0" class="cd-top">Top</a> | |
| <script type='text/javascript' src='library/js/scripts.js'></script> | |
| </body> | |
| </html> |