| <!DOCTYPE html> |
| <!-- |
| | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-parsing/metron-parsers-common/message-parser-implementation-notes.md at 2019-05-14 |
| | Rendered using Apache Maven Fluido Skin 1.7 |
| --> |
| <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
| <head> |
| <meta charset="UTF-8" /> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0" /> |
| <meta name="Date-Revision-yyyymmdd" content="20190514" /> |
| <meta http-equiv="Content-Language" content="en" /> |
| <title>Metron – implementation notes</title> |
| <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.7.min.css" /> |
| <link rel="stylesheet" href="../../../css/site.css" /> |
| <link rel="stylesheet" href="../../../css/print.css" media="print" /> |
| <script type="text/javascript" src="../../../js/apache-maven-fluido-1.7.min.js"></script> |
| <script type="text/javascript"> |
| $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } ); |
| </script> |
| </head> |
| <body class="topBarDisabled"> |
| <div class="container-fluid"> |
| <div id="banner"> |
| <div class="pull-left"><a href="http://metron.apache.org/" id="bannerLeft"><img src="../../../images/metron-logo.png" alt="Apache Metron" width="148px" height="48px"/></a></div> |
| <div class="pull-right"></div> |
| <div class="clear"><hr/></div> |
| </div> |
| |
| <div id="breadcrumbs"> |
| <ul class="breadcrumb"> |
| <li class=""><a href="http://www.apache.org" class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li> |
| <li class=""><a href="http://metron.apache.org/" class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> |
| <li class=""><a href="../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> |
| <li class="active "> implementation notes</li> |
| <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2019-05-14</li> |
| <li id="projectVersion" class="pull-right">Version: 0.7.1</li> |
| </ul> |
| </div> |
| <div class="row-fluid"> |
| <div id="leftColumn" class="span2"> |
| <div class="well sidebar-nav"> |
| <ul class="nav nav-list"> |
| <li class="nav-header">User Documentation</li> |
| <li><a href="../../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a> |
| <ul class="nav nav-list"> |
| <li><a href="../../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li> |
| <li><a href="../../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li> |
| <li><a href="../../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li> |
| <li><a href="../../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li> |
| <li><a href="../../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li> |
| <li><a href="../../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li> |
| <li><a href="../../../metron-interface/index.html" title="Interface"><span class="icon-chevron-right"></span>Interface</a></li> |
| <li><a href="../../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a> |
| <ul class="nav nav-list"> |
| <li><a href="../../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li> |
| <li><a href="../../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li> |
| <li><a href="../../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li> |
| <li><a href="../../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li> |
| <li><a href="../../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li> |
| <li><a href="../../../metron-platform/metron-hbase-server/index.html" title="Hbase-server"><span class="none"></span>Hbase-server</a></li> |
| <li><a href="../../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li> |
| <li><a href="../../../metron-platform/metron-job/index.html" title="Job"><span class="none"></span>Job</a></li> |
| <li><a href="../../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li> |
| <li><a href="../../../metron-platform/metron-parsing/index.html" title="Parsing"><span class="icon-chevron-down"></span>Parsing</a> |
| <ul class="nav nav-list"> |
| <li><a href="../../../metron-platform/metron-parsing/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li> |
| <li><a href="../../../metron-platform/metron-parsing/metron-parsers-common/index.html" title="Parsers-common"><span class="icon-chevron-down"></span>Parsers-common</a> |
| <ul class="nav nav-list"> |
| <li><a href="../../../metron-platform/metron-parsing/metron-parsers-common/3rdPartyParser.html" title="3rdPartyParser"><span class="none"></span>3rdPartyParser</a></li> |
| <li><a href="../../../metron-platform/metron-parsing/metron-parsers-common/ParserChaining.html" title="ParserChaining"><span class="none"></span>ParserChaining</a></li> |
| <li class="active"><a href="#"><span class="none"></span>message-parser-implementation-notes</a></li> |
| <li><a href="../../../metron-platform/metron-parsing/metron-parsers-common/parser-testing.html" title="parser-testing"><span class="none"></span>parser-testing</a></li> |
| <li><a href="../../../metron-platform/metron-parsing/metron-parsers-common/src/test/java/org/apache/metron/parsers/paloalto/index.html" title="Paloalto"><span class="none"></span>Paloalto</a></li> |
| </ul> |
| </li> |
| <li><a href="../../../metron-platform/metron-parsing/metron-parsing-storm/index.html" title="Parsing-storm"><span class="none"></span>Parsing-storm</a></li> |
| </ul> |
| </li> |
| <li><a href="../../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li> |
| <li><a href="../../../metron-platform/metron-solr/index.html" title="Solr"><span class="none"></span>Solr</a></li> |
| <li><a href="../../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li> |
| </ul> |
| </li> |
| <li><a href="../../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li> |
| <li><a href="../../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li> |
| <li><a href="../../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li> |
| <li><a href="../../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li> |
| <li><a href="../../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li> |
| </ul> |
| </li> |
| </ul> |
| <hr /> |
| <div id="poweredBy"> |
| <div class="clear"></div> |
| <div class="clear"></div> |
| <div class="clear"></div> |
| <div class="clear"></div> |
| <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../../images/logos/maven-feather.png" /></a> |
| </div> |
| </div> |
| </div> |
| <div id="bodyColumn" class="span10" > |
| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| <h1><tt>MessageParser</tt> implementation notes</h1> |
| <p><a name="MessageParser_implementation_notes"></a></p> |
| <ol style="list-style-type: decimal"> |
| |
| <li>Supporting multiple JSONObject returns from a single byte[] The original <tt>MessageParser</tt> interface supported parsing a message and returning a <tt>List<JSONObject></tt>. Therefore explicitly supporting multiple messages from one input. While this is fine, it only allows for the complete failure of a message for any reason. There can only be one exception thrown. This means that if there <i>are</i> multiple messages in the buffer, any one failure will necessarily fail all of them. To improve on this situation, a new method was added to the <tt>MessageParser</tt> interface (with a default implementation), that introduces a return type to provide not only the JSONObjects produced, but also a <tt>Map</tt> of messages -> throwable.</li> |
| </ol> |
| <p>To support this in your parser, you should:</p> |
| <ul> |
| |
| <li>Implement the new method</li> |
| </ul> |
| |
| <div> |
| <div> |
| <pre class="source"> @Override |
| public Optional<MessageParserResult<JSONObject>> parseOptionalResult(byte[] rawMessage) |
| </pre></div></div> |
| |
| <ul> |
| |
| <li>Implement the original <tt>List<JSONObject> parse(byte[] message)</tt> to delegate to that method such as below:</li> |
| </ul> |
| |
| <div> |
| <div> |
| <pre class="source"> @Override |
| public List<JSONObject> parse(byte[] rawMessage) { |
| Optional<MessageParserResult<JSONObject>> resultOptional = parseOptionalResult(rawMessage); |
| if (!resultOptional.isPresent()) { |
| return Collections.EMPTY_LIST; |
| } |
| Map<Object,Throwable> errors = resultOptional.get().getMessageThrowables(); |
| if (!errors.isEmpty()) { |
| throw new RuntimeException(errors.entrySet().iterator().next().getValue()); |
| } |
| |
| return resultOptional.get().getMessages(); |
| } |
| </pre></div></div> |
| |
| <ul> |
| |
| <li> |
| |
| <p>You <i>may</i> want to govern treating the incoming buffer as multiline or not by adding a configuration option for your parser, such as <tt>"multiline":"true"|"false"</tt></p> |
| </li> |
| <li> |
| |
| <p>See the org.apache.metron.parsers.GrokParser for an example of this implementation.</p> |
| </li> |
| </ul> |
| <p>The Metron system itself will call the new <tt>parseOptionalResult</tt> method during processing. The default implementation in the interface handles backwards compatability with previous implementations.</p> |
| </div> |
| </div> |
| </div> |
| <hr/> |
| <footer> |
| <div class="container-fluid"> |
| <div class="row-fluid"> |
| © 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo, |
| and the Apache Metron project logo are trademarks of The Apache Software Foundation. |
| </div> |
| </div> |
| </footer> |
| </body> |
| </html> |
