<!DOCTYPE html><html><head><title>Building Evaluation Metrics</title><meta charset="utf-8"/><meta content="IE=edge,chrome=1" http-equiv="X-UA-Compatible"/><meta name="viewport" content="width=device-width, initial-scale=1.0"/><meta class="swiftype" name="title" data-type="string" content="Building Evaluation Metrics"/><link rel="canonical" href="https://docs.prediction.io/evaluation/metricbuild/"/><link href="/images/favicon/normal-b330020a.png" rel="shortcut icon"/><link href="/images/favicon/apple-c0febcf2.png" rel="apple-touch-icon"/><link href="//fonts.googleapis.com/css?family=Open+Sans:300italic,400italic,600italic,700italic,800italic,400,300,600,700,800" rel="stylesheet"/><link href="//maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" rel="stylesheet"/><link href="/stylesheets/application-3598c7d7.css" rel="stylesheet" type="text/css"/><script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.2/html5shiv.min.js"></script><script src="//cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script><script src="//use.typekit.net/pqo0itb.js"></script><script>try{Typekit.load({ async: true });}catch(e){}</script></head><body><div id="global"><header><div class="container" id="header-wrapper"><div class="row"><div class="col-sm-12"><div id="logo-wrapper"><span id="drawer-toggle"></span><a href="#"></a><a href="http://predictionio.incubator.apache.org/"><img alt="PredictionIO" id="logo" src="/images/logos/logo-ee2b9bb3.png"/></a></div><div id="menu-wrapper"><div id="header-nav-options-wrapper"><ul><li><a href="/">Install & Doc</a></li> <li><a href="/support">Support</a></li> </ul></div><div id="pill-wrapper"><a class="pill left" href="//templates.prediction.io/">TEMPLATES</a> <a class="pill right" href="//github.com/apache/incubator-predictionio/">OPEN SOURCE</a></div></div><img class="mobile-search-bar-toggler hidden-md hidden-lg" src="/images/icons/search-glass-704bd4ff.png"/></div></div></div></header><div id="search-bar-row-wrapper"><div class="container-fluid" id="search-bar-row"><div class="row"><div class="col-md-9 col-sm-11 col-xs-11"><div class="hidden-md hidden-lg" id="mobile-page-heading-wrapper"><p>PredictionIO Docs</p><h4>Building Evaluation Metrics</h4></div><h4 class="hidden-sm hidden-xs">PredictionIO Docs</h4></div><div class="col-md-3 col-sm-1 col-xs-1 hidden-md hidden-lg"><img id="left-menu-indicator" src="/images/icons/down-arrow-dfe9f7fe.png"/></div><div class="col-md-3 col-sm-12 col-xs-12 swiftype-wrapper"><div class="swiftype"><form class="search-form"><img class="search-box-toggler hidden-xs hidden-sm" src="/images/icons/search-glass-704bd4ff.png"/><div class="search-box"><img src="/images/icons/search-glass-704bd4ff.png"/><input type="text" id="st-search-input" class="st-search-input" placeholder="Search Doc..."/></div><img class="swiftype-row-hider hidden-md hidden-lg" src="/images/icons/drawer-toggle-active-fcbef12a.png"/></form></div></div><div class="mobile-left-menu-toggler hidden-md hidden-lg"></div></div></div></div><div id="page" class="container-fluid"><div class="row"><div id="left-menu-wrapper" class="col-md-3"><nav id="nav-main"><ul><li class="level-1"><a class="expandible" href="/"><span>Apache PredictionIO (incubating) Documentation</span></a><ul><li class="level-2"><a class="final" href="/"><span>Welcome to Apache PredictionIO (incubating)</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Getting Started</span></a><ul><li class="level-2"><a class="final" href="/start/"><span>A Quick Intro</span></a></li><li class="level-2"><a class="final" href="/install/"><span>Installing Apache PredictionIO (incubating)</span></a></li><li class="level-2"><a class="final" href="/start/download/"><span>Downloading an Engine Template</span></a></li><li class="level-2"><a class="final" href="/start/deploy/"><span>Deploying Your First Engine</span></a></li><li class="level-2"><a class="final" href="/start/customize/"><span>Customizing the Engine</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Integrating with Your App</span></a><ul><li class="level-2"><a class="final" href="/appintegration/"><span>App Integration Overview</span></a></li><li class="level-2"><a class="expandible" href="/sdk/"><span>List of SDKs</span></a><ul><li class="level-3"><a class="final" href="/sdk/java/"><span>Java & Android SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/php/"><span>PHP SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/python/"><span>Python SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/ruby/"><span>Ruby SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/community/"><span>Community Powered SDKs</span></a></li></ul></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Deploying an Engine</span></a><ul><li class="level-2"><a class="final" href="/deploy/"><span>Deploying as a Web Service</span></a></li><li class="level-2"><a class="final" href="/cli/#engine-commands"><span>Engine Command-line Interface</span></a></li><li class="level-2"><a class="final" href="/deploy/engineparams/"><span>Setting Engine Parameters</span></a></li><li class="level-2"><a class="final" href="/deploy/enginevariants/"><span>Deploying Multiple Engine Variants</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Customizing an Engine</span></a><ul><li class="level-2"><a class="final" href="/customize/"><span>Learning DASE</span></a></li><li class="level-2"><a class="final" href="/customize/dase/"><span>Implement DASE</span></a></li><li class="level-2"><a class="final" href="/customize/troubleshooting/"><span>Troubleshooting Engine Development</span></a></li><li class="level-2"><a class="final" href="/api/current/#package"><span>Engine Scala APIs</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Collecting and Analyzing Data</span></a><ul><li class="level-2"><a class="final" href="/datacollection/"><span>Event Server Overview</span></a></li><li class="level-2"><a class="final" href="/cli/#event-server-commands"><span>Event Server Command-line Interface</span></a></li><li class="level-2"><a class="final" href="/datacollection/eventapi/"><span>Collecting Data with REST/SDKs</span></a></li><li class="level-2"><a class="final" href="/datacollection/eventmodel/"><span>Events Modeling</span></a></li><li class="level-2"><a class="final" href="/datacollection/webhooks/"><span>Unifying Multichannel Data with Webhooks</span></a></li><li class="level-2"><a class="final" href="/datacollection/channel/"><span>Channel</span></a></li><li class="level-2"><a class="final" href="/datacollection/batchimport/"><span>Importing Data in Batch</span></a></li><li class="level-2"><a class="final" href="/datacollection/analytics/"><span>Using Analytics Tools</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Choosing an Algorithm(s)</span></a><ul><li class="level-2"><a class="final" href="/algorithm/"><span>Built-in Algorithm Libraries</span></a></li><li class="level-2"><a class="final" href="/algorithm/switch/"><span>Switching to Another Algorithm</span></a></li><li class="level-2"><a class="final" href="/algorithm/multiple/"><span>Combining Multiple Algorithms</span></a></li><li class="level-2"><a class="final" href="/algorithm/custom/"><span>Adding Your Own Algorithms</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>ML Tuning and Evaluation</span></a><ul><li class="level-2"><a class="final" href="/evaluation/"><span>Overview</span></a></li><li class="level-2"><a class="final" href="/evaluation/paramtuning/"><span>Hyperparameter Tuning</span></a></li><li class="level-2"><a class="final" href="/evaluation/evaluationdashboard/"><span>Evaluation Dashboard</span></a></li><li class="level-2"><a class="final" href="/evaluation/metricchoose/"><span>Choosing Evaluation Metrics</span></a></li><li class="level-2"><a class="final active" href="/evaluation/metricbuild/"><span>Building Evaluation Metrics</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>System Architecture</span></a><ul><li class="level-2"><a class="final" href="/system/"><span>Architecture Overview</span></a></li><li class="level-2"><a class="final" href="/system/anotherdatastore/"><span>Using Another Data Store</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Engine Template Gallery</span></a><ul><li class="level-2"><a class="final" href="http://templates.prediction.io"><span>Browse</span></a></li><li class="level-2"><a class="final" href="/community/submit-template/"><span>Submit your Engine as a Template</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Demo Tutorials</span></a><ul><li class="level-2"><a class="final" href="/demo/tapster/"><span>Comics Recommendation Demo</span></a></li><li class="level-2"><a class="final" href="/demo/community/"><span>Community Contributed Demo</span></a></li><li class="level-2"><a class="final" href="/demo/textclassification/"><span>Text Classification Engine Tutorial</span></a></li></ul></li><li class="level-1"><a class="expandible" href="/community/"><span>Getting Involved</span></a><ul><li class="level-2"><a class="final" href="/community/contribute-code/"><span>Contribute Code</span></a></li><li class="level-2"><a class="final" href="/community/contribute-documentation/"><span>Contribute Documentation</span></a></li><li class="level-2"><a class="final" href="/community/contribute-sdk/"><span>Contribute a SDK</span></a></li><li class="level-2"><a class="final" href="/community/contribute-webhook/"><span>Contribute a Webhook</span></a></li><li class="level-2"><a class="final" href="/community/projects/"><span>Community Projects</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Getting Help</span></a><ul><li class="level-2"><a class="final" href="/resources/faq/"><span>FAQs</span></a></li><li class="level-2"><a class="final" href="/support/"><span>Community Support</span></a></li><li class="level-2"><a class="final" href="/support/#enterprise-support"><span>Enterprise Support</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Resources</span></a><ul><li class="level-2"><a class="final" href="/resources/intellij/"><span>Developing Engines with IntelliJ IDEA</span></a></li><li class="level-2"><a class="final" href="/resources/upgrade/"><span>Upgrade Instructions</span></a></li><li class="level-2"><a class="final" href="/resources/glossary/"><span>Glossary</span></a></li></ul></li></ul></nav></div><div class="col-md-9 col-sm-12"><div class="content-header hidden-md hidden-lg"><div id="breadcrumbs" class="hidden-sm hidden xs"><ul><li><a href="#">ML Tuning and Evaluation</a><span class="spacer">&gt;</span></li><li><span class="last">Building Evaluation Metrics</span></li></ul></div><div id="page-title"><h1>Building Evaluation Metrics</h1></div></div><div id="table-of-content-wrapper"><h5>On this page</h5><aside id="table-of-contents"><ul> <li> <a href="#overview">Overview</a> </li> <li> <a href="#example-1-accuracy-metric">Example 1: Accuracy Metric</a> </li> <li> <a href="#example-2-precision-metric">Example 2: Precision Metric</a> </li> </ul> </aside><hr/><a id="edit-page-link" href="https://github.com/apache/incubator-predictionio/tree/livedoc/docs/manual/source/evaluation/metricbuild.html.md"><img src="/images/icons/edit-pencil-d6c1bb3d.png"/>Edit this page</a></div><div class="content-header hidden-sm hidden-xs"><div id="breadcrumbs" class="hidden-sm hidden xs"><ul><li><a href="#">ML Tuning and Evaluation</a><span class="spacer">&gt;</span></li><li><span class="last">Building Evaluation Metrics</span></li></ul></div><div id="page-title"><h1>Building Evaluation Metrics</h1></div></div><div class="content"><p>PredictionIO enables developer to implement evaluation custom evaluation metric with just a few lines of code. We illustrate it with <a href="/templates/classification/quickstart/">the classification template</a>.</p><h2 id='overview' class='header-anchors'>Overview</h2><p>A simplistic form of metric is a function which takes a <code>(Query, PredictedResult, ActualResult)</code>-tuple (<em>QPA-tuple</em>) as input and return a score. Exploiting this properties allows us to implement custom metric with a single line of code (plus some boilerplates). We demonstate this with two metrics: accuracy and precision.</p> <h2 id='example-1:-accuracy-metric' class='header-anchors'>Example 1: Accuracy Metric</h2><p>Accuracy is a metric capturing the portion of correct prediction among all test data points. A way to model this is for each correct QPA-tuple, we give a score of 1.0 and otherwise 0.0, then we take an average of all tuple scores.</p><p>PredictionIO has a [[AverageMetric]] helper class which provides this feature. This class takes 4 type parameters, [[EvalInfo]], [[Query]], [[PredictedResult]], and [[ActualResult]], these types can be found from the engine&#39;s signature. Line 5 below is the custom calculation.</p><div class="highlight scala"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1
2
3
4
5
6</pre></td><td class="code"><pre><span class="k">case</span> <span class="k">class</span> <span class="nc">Accuracy</span>
  <span class="k">extends</span> <span class="nc">AverageMetric</span><span class="o">[</span><span class="kt">EmptyEvaluationInfo</span>, <span class="kt">Query</span>, <span class="kt">PredictedResult</span>, <span class="kt">ActualResult</span><span class="o">]</span> <span class="o">{</span>
  <span class="k">def</span> <span class="n">calculate</span><span class="o">(</span><span class="n">query</span><span class="k">:</span> <span class="kt">Query</span><span class="o">,</span> <span class="n">predicted</span><span class="k">:</span> <span class="kt">PredictedResult</span><span class="o">,</span> <span class="n">actual</span><span class="k">:</span> <span class="kt">ActualResult</span><span class="o">)</span>
  <span class="k">:</span> <span class="kt">Double</span> <span class="o">=</span> 
    <span class="o">(</span><span class="k">if</span> <span class="o">(</span><span class="n">predicted</span><span class="o">.</span><span class="n">label</span> <span class="o">==</span> <span class="n">actual</span><span class="o">.</span><span class="n">label</span><span class="o">)</span> <span class="mf">1.0</span> <span class="k">else</span> <span class="mf">0.0</span><span class="o">)</span>
<span class="o">}</span>
</pre></td></tr></tbody></table> </div> <p>Once we define a metric, we tell PredictionIO we are using it in the <code>Evaluation</code> object. We can run the following command to kick start the evaluation.</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1
2
3
4
5</pre></td><td class="code"><pre><span class="gp">$ </span>pio build
...
<span class="gp">$ </span>pio <span class="nb">eval </span>org.template.classification.AccuracyEvaluation <span class="se">\</span>
    org.template.classification.EngineParamsList 
...
</pre></td></tr></tbody></table> </div> <p>(See MyClassification/src/main/scala/<strong><em>Evaluation.scala</em></strong> for full usage.)</p><h2 id='example-2:-precision-metric' class='header-anchors'>Example 2: Precision Metric</h2><p>Precision is a metric for binary classifier capturing the portion of correction prediction among all <em>positive</em> predictions. We don&#39;t care about the cases where the QPA-tuple gives a negative prediction. (Recall that a binary classifier only provide two output values: <em>positive</em> and <em>negative</em>.) The following table illustrates all four cases:</p> <table><thead> <tr> <th style="text-align: center">PredictedResult</th> <th style="text-align: center">ActualResult</th> <th style="text-align: center">Value</th> </tr> </thead><tbody> <tr> <td style="text-align: center">Positive</td> <td style="text-align: center">Positive</td> <td style="text-align: center">1.0</td> </tr> <tr> <td style="text-align: center">Positive</td> <td style="text-align: center">Negative</td> <td style="text-align: center">0.0</td> </tr> <tr> <td style="text-align: center">Negative</td> <td style="text-align: center">Positive</td> <td style="text-align: center">Don&#39;t care</td> </tr> <tr> <td style="text-align: center">Negative</td> <td style="text-align: center">Negative</td> <td style="text-align: center">Don&#39;t care</td> </tr> </tbody></table> <p>Calculating the precision metric is a slightly more involved procedure than calculating the accuracy metric as we have to specially handle the <em>don&#39;t care</em> negative cases.</p><p>PredictionIO provides a helper class <code>OptionAverageMetric</code> allows user to specify <em>don&#39;t care</em> values as <code>None</code>. It only aggregates the non-None values. Lines 3 to 4 is the method signature of <code>calcuate</code> method. The key difference is that the return value is a <code>Option[Double]</code>, in contrast to <code>Double</code> for <code>AverageMetric</code>. This class only computes the average of <code>Some(.)</code> results. Lines 5 to 13 are the actual logic. The first <code>if</code> factors out the positively predicted case, and the computation is simliar to the accuracy metric. The negatively predicted case are the <em>don&#39;t cares</em>, which we return <code>None</code>.</p><div class="highlight scala"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15</pre></td><td class="code"><pre><span class="k">case</span> <span class="k">class</span> <span class="nc">Precision</span><span class="o">(</span><span class="n">label</span><span class="k">:</span> <span class="kt">Double</span><span class="o">)</span>
  <span class="k">extends</span> <span class="nc">OptionAverageMetric</span><span class="o">[</span><span class="kt">EmptyEvaluationInfo</span>, <span class="kt">Query</span>, <span class="kt">PredictedResult</span>, <span class="kt">ActualResult</span><span class="o">]</span> <span class="o">{</span>
  <span class="k">def</span> <span class="n">calculate</span><span class="o">(</span><span class="n">query</span><span class="k">:</span> <span class="kt">Query</span><span class="o">,</span> <span class="n">predicted</span><span class="k">:</span> <span class="kt">PredictedResult</span><span class="o">,</span> <span class="n">actual</span><span class="k">:</span> <span class="kt">ActualResult</span><span class="o">)</span>
  <span class="k">:</span> <span class="kt">Option</span><span class="o">[</span><span class="kt">Double</span><span class="o">]</span> <span class="k">=</span> <span class="o">{</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">predicted</span><span class="o">.</span><span class="n">label</span> <span class="o">==</span> <span class="n">label</span><span class="o">)</span> <span class="o">{</span>
      <span class="k">if</span> <span class="o">(</span><span class="n">predicted</span><span class="o">.</span><span class="n">label</span> <span class="o">==</span> <span class="n">actual</span><span class="o">.</span><span class="n">label</span><span class="o">)</span> <span class="o">{</span>
        <span class="nc">Some</span><span class="o">(</span><span class="mf">1.0</span><span class="o">)</span>  <span class="c1">// True positive
</span>      <span class="o">}</span> <span class="k">else</span> <span class="o">{</span>
        <span class="nc">Some</span><span class="o">(</span><span class="mf">0.0</span><span class="o">)</span>  <span class="c1">// False positive
</span>      <span class="o">}</span>
    <span class="o">}</span> <span class="k">else</span> <span class="o">{</span>
      <span class="nc">None</span>  <span class="c1">// Unrelated case for calcuating precision
</span>    <span class="o">}</span>
  <span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table> </div> <p>We define a new <code>Evaluation</code> object to tell PredictionIO how to use this new precision metric.</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1
2
3</pre></td><td class="code"><pre>object PrecisionEvaluation extends Evaluation <span class="o">{</span>
  engineMetric <span class="o">=</span> <span class="o">(</span>ClassificationEngine<span class="o">()</span>, new Precision<span class="o">(</span>label <span class="o">=</span> 1.0<span class="o">))</span>
<span class="o">}</span>
</pre></td></tr></tbody></table> </div> <p>We can kickstarts the evaluation with the following command, notice that we are reusing the same engine params list as before. This address the separation of concern when we conduct hyperparameter tuning.</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46</pre></td><td class="code"><pre><span class="gp">$ </span>pio build
...
<span class="gp">$ </span>pio <span class="nb">eval </span>org.template.classification.PrecisionEvaluation <span class="se">\</span>
    org.template.classification.EngineParamsList 
...
<span class="o">[</span>INFO] <span class="o">[</span>CoreWorkflow<span class="nv">$]</span> Starting evaluation instance ID: SMhzYbJ9QgKkD0fQzTA7MA
...
<span class="o">[</span>INFO] <span class="o">[</span>MetricEvaluator] Iteration 0
<span class="o">[</span>INFO] <span class="o">[</span>MetricEvaluator] EngineParams: <span class="o">{</span><span class="s2">"dataSourceParams"</span>:<span class="o">{</span><span class="s2">""</span>:<span class="o">{</span><span class="s2">"appId"</span>:19,<span class="s2">"evalK"</span>:5<span class="o">}}</span>,<span class="s2">"preparatorParams"</span>:<span class="o">{</span><span class="s2">""</span>:<span class="o">{}}</span>,<span class="s2">"algorithmParamsList"</span>:[<span class="o">{</span><span class="s2">"naive"</span>:<span class="o">{</span><span class="s2">"lambda"</span>:10.0<span class="o">}}]</span>,<span class="s2">"servingParams"</span>:<span class="o">{</span><span class="s2">""</span>:<span class="o">{}}}</span>
<span class="o">[</span>INFO] <span class="o">[</span>MetricEvaluator] Result: MetricScores<span class="o">(</span>0.8846153846153846,List<span class="o">())</span>
<span class="o">[</span>INFO] <span class="o">[</span>MetricEvaluator] Iteration 1
<span class="o">[</span>INFO] <span class="o">[</span>MetricEvaluator] EngineParams: <span class="o">{</span><span class="s2">"dataSourceParams"</span>:<span class="o">{</span><span class="s2">""</span>:<span class="o">{</span><span class="s2">"appId"</span>:19,<span class="s2">"evalK"</span>:5<span class="o">}}</span>,<span class="s2">"preparatorParams"</span>:<span class="o">{</span><span class="s2">""</span>:<span class="o">{}}</span>,<span class="s2">"algorithmParamsList"</span>:[<span class="o">{</span><span class="s2">"naive"</span>:<span class="o">{</span><span class="s2">"lambda"</span>:100.0<span class="o">}}]</span>,<span class="s2">"servingParams"</span>:<span class="o">{</span><span class="s2">""</span>:<span class="o">{}}}</span>
<span class="o">[</span>INFO] <span class="o">[</span>MetricEvaluator] Result: MetricScores<span class="o">(</span>0.7936507936507936,List<span class="o">())</span>
<span class="o">[</span>INFO] <span class="o">[</span>MetricEvaluator] Iteration 2
<span class="o">[</span>INFO] <span class="o">[</span>MetricEvaluator] EngineParams: <span class="o">{</span><span class="s2">"dataSourceParams"</span>:<span class="o">{</span><span class="s2">""</span>:<span class="o">{</span><span class="s2">"appId"</span>:19,<span class="s2">"evalK"</span>:5<span class="o">}}</span>,<span class="s2">"preparatorParams"</span>:<span class="o">{</span><span class="s2">""</span>:<span class="o">{}}</span>,<span class="s2">"algorithmParamsList"</span>:[<span class="o">{</span><span class="s2">"naive"</span>:<span class="o">{</span><span class="s2">"lambda"</span>:1000.0<span class="o">}}]</span>,<span class="s2">"servingParams"</span>:<span class="o">{</span><span class="s2">""</span>:<span class="o">{}}}</span>
<span class="o">[</span>INFO] <span class="o">[</span>MetricEvaluator] Result: MetricScores<span class="o">(</span>0.37593984962406013,List<span class="o">())</span>
<span class="o">[</span>INFO] <span class="o">[</span>CoreWorkflow<span class="nv">$]</span> Updating evaluation instance with result: MetricEvaluatorResult:
  <span class="c"># engine params evaluated: 3</span>
Optimal Engine Params:
  <span class="o">{</span>
  <span class="s2">"dataSourceParams"</span>:<span class="o">{</span>
    <span class="s2">""</span>:<span class="o">{</span>
      <span class="s2">"appId"</span>:19,
      <span class="s2">"evalK"</span>:5
    <span class="o">}</span>
  <span class="o">}</span>,
  <span class="s2">"preparatorParams"</span>:<span class="o">{</span>
    <span class="s2">""</span>:<span class="o">{</span>

    <span class="o">}</span>
  <span class="o">}</span>,
  <span class="s2">"algorithmParamsList"</span>:[
    <span class="o">{</span>
      <span class="s2">"naive"</span>:<span class="o">{</span>
        <span class="s2">"lambda"</span>:10.0
      <span class="o">}</span>
    <span class="o">}</span>
  <span class="o">]</span>,
  <span class="s2">"servingParams"</span>:<span class="o">{</span>
    <span class="s2">""</span>:<span class="o">{</span>

    <span class="o">}</span>
  <span class="o">}</span>
<span class="o">}</span>
Metrics:
  org.template.classification.Precision: 0.8846153846153846
</pre></td></tr></tbody></table> </div> <p>(See MyClassification/src/main/scala/<strong><em>PrecisionEvaluation.scala</em></strong> for the full usage.)</p></div></div></div></div><footer><div class="container"><div class="seperator"></div><div class="row"><div class="col-md-6 col-xs-6 footer-link-column"><div class="footer-link-column-row"><h4>Community</h4><ul><li><a href="//docs.prediction.io/install/" target="blank">Download</a></li><li><a href="//docs.prediction.io/" target="blank">Docs</a></li><li><a href="//github.com/apache/incubator-predictionio" target="blank">GitHub</a></li><li><a href="mailto:user-subscribe@predictionio.incubator.apache.org" target="blank">Subscribe to User Mailing List</a></li><li><a href="//stackoverflow.com/questions/tagged/predictionio" target="blank">Stackoverflow</a></li></ul></div></div><div class="col-md-6 col-xs-6 footer-link-column"><div class="footer-link-column-row"><h4>Contribute</h4><ul><li><a href="//predictionio.incubator.apache.org/community/contribute-code/" target="blank">Contribute</a></li><li><a href="//github.com/apache/incubator-predictionio" target="blank">Source Code</a></li><li><a href="//issues.apache.org/jira/browse/PIO" target="blank">Bug Tracker</a></li><li><a href="mailto:dev-subscribe@predictionio.incubator.apache.org" target="blank">Subscribe to Development Mailing List</a></li></ul></div></div></div></div><div id="footer-bottom"><div class="container"><div class="row"><div class="col-md-12"><div id="footer-logo-wrapper"><img alt="PredictionIO" src="/images/logos/logo-white-d1e9c6e6.png"/></div><div id="social-icons-wrapper"><a class="github-button" href="https://github.com/apache/incubator-predictionio" data-style="mega" data-count-href="/apache/incubator-predictionio/stargazers" data-count-api="/repos/apache/incubator-predictionio#stargazers_count" data-count-aria-label="# stargazers on GitHub" aria-label="Star apache/incubator-predictionio on GitHub">Star</a> <a class="github-button" href="https://github.com/apache/incubator-predictionio/fork" data-icon="octicon-git-branch" data-style="mega" data-count-href="/apache/incubator-predictionio/network" data-count-api="/repos/apache/incubator-predictionio#forks_count" data-count-aria-label="# forks on GitHub" aria-label="Fork apache/incubator-predictionio on GitHub">Fork</a> <script id="github-bjs" async="" defer="" src="https://buttons.github.io/buttons.js"></script><a href="//www.facebook.com/predictionio" target="blank"><img alt="PredictionIO on Twitter" src="/images/icons/twitter-ea9dc152.png"/></a> <a href="//twitter.com/predictionio" target="blank"><img alt="PredictionIO on Facebook" src="/images/icons/facebook-5c57939c.png"/></a> </div></div></div></div></div></footer></div><script>(function(w,d,t,u,n,s,e){w['SwiftypeObject']=n;w[n]=w[n]||function(){
(w[n].q=w[n].q||[]).push(arguments);};s=d.createElement(t);
e=d.getElementsByTagName(t)[0];s.async=1;s.src=u;e.parentNode.insertBefore(s,e);
})(window,document,'script','//s.swiftypecdn.com/install/v1/st.js','_st');

_st('install','HaUfpXXV87xoB_zzCQ45');</script><script src="/javascripts/application-5a24945b.js"></script></body></html>