| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| --> |
| <div class="container-fluid" id="main"> |
| <div class="row"> |
| <h5 class="over-title margin-bottom-15">Create DQ Model</h5> |
| </div> |
| <div class="row"> |
| <div class="row"> |
| <div class="col-lg-6 col-md-6 col-sm-6 ruletypes"> |
| <section id="panel-1" class="panel panel-red" style="cursor:pointer" (click)="click('ac')"> |
| <div class="panel-heading"> |
| <span style="font-size:20px">Accuracy</span> |
| <span class="pull-right" style="font-size:20px"> |
| <span class="fa fa-arrow-circle-right" (click)="click('ac')"></span> |
| </span> |
| </div> |
| <div class="swMain panel-body"> |
| <label class="label-definition">Definition: Measured by how the values agree with an identified source of |
| truth</label> |
| <ul style="border-radius:0; background: none"> |
| <li> |
| <a class="selected"> |
| <div class="stepNumber"> |
| 1 |
| </div> |
| <span class="stepDesc text-small"> Choose Source </span> |
| </a> |
| </li> |
| <li> |
| <a> |
| <div class="stepNumber"> |
| 2 |
| </div> |
| <span class="stepDesc text-small"> Choose Target </span> |
| </a> |
| </li> |
| <li> |
| <a> |
| <div class="stepNumber"> |
| 3 |
| </div> |
| <span class="stepDesc text-small"> Mapping Source and Target </span> |
| </a> |
| </li> |
| <li> |
| <a> |
| <div class="stepNumber"> |
| 4 |
| </div> |
| <span class="stepDesc text-small"> Partition Configuration </span> |
| </a> |
| </li> |
| <li> |
| <a> |
| <div class="stepNumber"> |
| 5 |
| </div> |
| <span class="stepDesc text-small"> Configuration </span> |
| </a> |
| </li> |
| </ul> |
| <div> |
| <ol> |
| <li>Select the source dataset and fields which will be used for comparision</li> |
| <li>Select the target dataset and fields which will be used for comparision</li> |
| <li>Mapping the target fields with source</li> |
| <li>Set partition configuration for source dataset and target dataset</li> |
| <li>Set basic configuration for your model (name, system, threshold, etc.)</li> |
| </ol> |
| </div> |
| </div> |
| <div class="panel-footer stepDesc"> |
| <label>Example:</label> suppose source table A has 1000 records and target table B only has 999 records |
| pefectly matched with A for selected fields, then Accuracy Rate(%) = 999/1000 * 100% = 99.9% |
| </div> |
| </section> |
| </div> |
| <div class="col-lg-6 col-md-6 col-sm-6 ruletypes"> |
| <section id="panel-2" class="panel panel-primary" (click)="click('pr')"> |
| <div class="panel-heading"> |
| <span style="font-size:20px">Data Profiling</span> |
| <span class="pull-right" style="font-size:20px"> |
| <span class="fa fa-arrow-circle-right"></span> |
| </span> |
| </div> |
| <div class="swMain panel-body"> |
| <label class="label-definition">Definition: Data profiling is the process of examining the data available in |
| an existing data set and collecting statistics and information about that data</label> |
| <ul style="border-radius:0; background: none"> |
| <li> |
| <a class="selected" style="cursor:default"> |
| <div class="stepNumber"> |
| 1 |
| </div> |
| <span class="stepDesc text-small"> Choose Target </span> |
| </a> |
| </li> |
| <li> |
| <a style="cursor:default"> |
| <div class="stepNumber"> |
| 2 |
| </div> |
| <span class="stepDesc text-small">Define/Select Models </span> |
| </a> |
| </li> |
| <li> |
| <a style="cursor:default"> |
| <div class="stepNumber"> |
| 3 |
| </div> |
| <span class="stepDesc text-small"> Partition Configuration </span> |
| </a> |
| </li> |
| <li> |
| <a style="cursor:default"> |
| <div class="stepNumber"> |
| 4 |
| </div> |
| <span class="stepDesc text-small"> Configuration </span> |
| </a> |
| </li> |
| </ul> |
| <div> |
| <ol> |
| <li>Select the target dataset and fields which want to be checked</li> |
| <li>Define your syntax check logic which will be applied on the selected fields</li> |
| <li>Set partition configuration for target dataset</li> |
| <li>Set basic configuration for your model(name, system, threshold, etc.)</li> |
| </ol> |
| </div> |
| </div> |
| <div class="panel-footer stepDesc"> |
| <label>Example:</label> Check the data range(minimum, maximum) within a set of allowable values |
| </div> |
| </section> |
| </div> |
| </div> |
| <div class="row"> |
| <div class="col-lg-6 col-md-6 col-sm-6 ruletypes"> |
| <section id="panel-3" class="panel panel-green" (click)="click('pub')"> |
| <div class="panel-heading"> |
| <span style="font-size:20px">Publish</span> |
| <span class="pull-right" style="font-size:20px"> |
| <span class="fa fa-arrow-circle-right"></span> |
| </span> |
| </div> |
| <div class="swMain panel-body"> |
| <label class="label-definition">Definition: Publish is the process of storing user's own quality data and |
| visualizing it</label> |
| <ul style="border-radius:0; background: none"> |
| <li> |
| <div class="selected1child" style="cursor:default"> |
| <div class="stepNumber"> |
| 1 |
| </div> |
| <span class="stepDesc text-small"> Configuration </span> |
| </div> |
| </li> |
| </ul> |
| <div> |
| <ol> |
| <li>Set basic configuration for your model(name, system, threshold, etc.)</li> |
| </ol> |
| </div> |
| </div> |
| <div class="panel-footer stepDesc"> |
| <label>Example:</label> any data ... |
| </div> |
| </section> |
| </div> |
| <div class="col-lg-6 col-md-6 col-sm-6 ruletypes"> |
| <section id="panel-4" class="panel panel-yellow" (click)="click('raw')"> |
| <div class="panel-heading"> |
| <span style="font-size:20px">JSON/YAML measure</span> |
| <span class="pull-right" style="font-size:20px"> |
| <span class="fa fa-arrow-circle-right"></span> |
| </span> |
| </div> |
| <div class="swMain panel-body"> |
| <label class="label-definition">Definition: Submit measure with JSON/YAML measure definition utilizing |
| advanced configuration capabilities that are not available through other flows</label> |
| <ul style="border-radius:0; background: none"> |
| <li> |
| <div class="selected1child" style="cursor:default"> |
| <div class="stepNumber"> |
| 1 |
| </div> |
| <span class="stepDesc text-small"> Configuration </span> |
| </div> |
| </li> |
| </ul> |
| <div> |
| <ol> |
| <li>Create measure directly from JSON/YAML</li> |
| </ol> |
| </div> |
| </div> |
| <div class="panel-footer stepDesc"> |
| <label>Example:</label> measure with custom configuration like custom spark sql |
| </div> |
| </section> |
| </div> |
| </div> |
| </div> |
| </div> |