| <!DOCTYPE html> |
| <!-- Start _layouts/doc_page.html--> |
| <html lang="en"> |
| |
| <head> |
| <!-- Start _include/site_head.html --> |
| <meta charset="UTF-8" /> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| <meta name="description" content=""> |
| <meta name="author" content="datasketches"> |
| |
| <title>DataSketches | </title> |
| |
| <link rel="shortcut icon" href="/img/favicon.png"> |
| |
| <!-- original source: https://maxcdn.bootstrapcdn.com/font-awesome/4.1.0/css/font-awesome.min.css --> |
| <link rel="stylesheet" href="/css/font-awesome.min.css"> |
| <!-- original source: https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css --> |
| <link rel="stylesheet" href="/css/bootstrap.min.css"> |
| |
| <link rel="stylesheet" href="/css/fonts.css" type="text/css"> |
| |
| <link rel="stylesheet" href="/css/main.css"> |
| <link rel="stylesheet" href="/css/header.css"> |
| <link rel="stylesheet" href="/css/footer.css"> |
| <link rel="stylesheet" href="/css/syntax.css"> |
| <link rel="stylesheet" href="/css/docs.css"> |
| |
| <script type="text/x-mathjax-config"> |
| MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]},showMathMenu:false,showMathMenuMSIE:false,showProcessingMessages:false}); |
| </script> |
| |
| <!-- original source: https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMX_HTML-full --> |
| <script type="text/javascript" src="/js/MathJax.js?config=TeX-AMS_HTML"></script> |
| |
| <!-- original source: https://code.jquery.com/jquery.min.js --> |
| <script src="/js/jquery.min.js"></script> |
| |
| <!-- original source: https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/js/bootstrap.min.js --> |
| <script src="/js/bootstrap.min.js"></script> <!-- 3.2.0--> |
| <!-- End _include/site_head.html --> |
| |
| </head> |
| |
| <body> |
| <!-- Start _include/nav_bar.html --> |
| <div class="navbar navbar-inverse navbar-static-top ds-nav"> |
| <div class="container"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a href="/" style="padding-top: 0px; padding-bottom: 0px;"> |
| <span class="ds-small-h-logo"></span></a> |
| </div> |
| <div class="navbar-collapse collapse"> |
| <ul class="nav navbar-nav navbar-right"> |
| <li> |
| <a href="/docs/Background/TheChallenge.html"> |
| <span class="fa fa-info-circle"></span> DOCUMENTATION</a> |
| </li> |
| <li> |
| <a href="/docs/Community/Downloads.html"> |
| <span class="fa fa-download"></span> DOWNLOAD</a> |
| </li> |
| <!-- |
| <li> |
| <a href="/docs/Architecture/Components.html"> |
| <span class="fa fa-github"></span> GITHUB</a> |
| </li> |
| --> |
| <li> |
| <a href="/docs/Community/Research.html"> |
| <span class="fa fa-paper-plane"></span> RESEARCH</a> |
| </li> |
| <li> |
| <a href="/docs/Community/index.html" style="padding-top: 0; padding-bottom: 0;"> |
| <img class="ds-small-man" src="/img/datasketches-ManWhite.svg"/>COMMUNITY</a> |
| </li> |
| <li> |
| <ul class="nav navbar-nav navbar-right ds-nav"> |
| <li class="dropdown ds-nav" > |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false" style="padding-top: 0; padding-bottom: 0;"><img class="apache-logo" src="/img/feather.svg"/>Apache <span class="caret"></span></a> |
| <ul class="dropdown-menu ds-nav"> |
| <li><a href="https://www.apache.org/" target="_blank">Foundation</a></li> |
| <li><a href="https://www.apache.org/events/current-event" target="_blank">Events</a></li> |
| <li><a href="https://www.apache.org/licenses/" target="_blank">License</a></li> |
| <li><a href="https://privacy.apache.org/policies/privacy-policy-public.html" target="_blank">Privacy Policy</a></li> |
| <li><a href="https://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li> |
| <li><a href="https://www.apache.org/security/" target="_blank">Security</a></li> |
| <li><a href="https://www.apache.org/foundation/sponsorship.html" target="_blank">Sponsorship</a></li> |
| </ul> |
| </li> |
| </ul> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| <!-- End _include/nav_bar.html --> |
| |
| |
| <!-- Start _include/javadocs.html --> |
| <div class="ds-header"> |
| <div class="container"> |
| <h4>API Snapshots: |
| <a href="https://apache.github.io/datasketches-java/4.2.0/">Java Core</a>, |
| <a href="https://apache.github.io/datasketches-cpp/5.0.0/">C++ Core</a>, |
| <a href="https://apache.github.io/datasketches-python/main/">Python</a>, |
| <a href="https://apache.github.io/datasketches-memory/master/">Memory</a>, |
| <a href="/api/pig/snapshot/apidocs/index.html">Pig</a>, |
| <a href="/api/hive/snapshot/apidocs/index.html">Hive</a>, |
| </h4> |
| </div> |
| </div> |
| <!-- End _include/javadocs.html --> |
| |
| |
| <div class="container"> |
| <div class="row"> |
| <!-- Start ToC Block --> |
| <div class="col-md-3"> |
| <div class="searchbox" style="position:relative"> |
| <gcse:searchbox-only></gcse:searchbox-only> |
| </div> |
| <!-- Start _includes/toc.html --> |
| <!-- Computer Generated File, Do Not Edit! --> |
| <link rel="stylesheet" href="/css/toc.css"> |
| <div id="toc" class="nav toc hidden-print"> |
| |
| <p id="background"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_background">Background</a> |
| </p> |
| <div class="collapse" id="collapse_background"> |
| <li><a href="/docs/Background/TheChallenge.html">•The Challenge</a></li> |
| <li><a href="/docs/Background/SketchOrigins.html">•Sketch Origins</a></li> |
| <li><a href="/docs/Background/SketchElements.html">•Sketch Elements</a></li> |
| <li><a href="/docs/Background/Presentations.html">•Presentations</a></li> |
| <li><a href="https://github.com/apache/datasketches-website/tree/master/docs/pdf/DataSketches_deck.pdf">•Overview Slide Deck</a></li> |
| </div> |
| |
| <p id="architecture-and-design"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_architecture_and_design">Architecture And Design</a> |
| </p> |
| <div class="collapse" id="collapse_architecture_and_design"> |
| <li><a href="/docs/Architecture/MajorSketchFamilies.html">•The Major Sketch Families</a></li> |
| <li><a href="/docs/Architecture/LargeScale.html">•Large Scale Computing</a></li> |
| <li><a href="/docs/Architecture/KeyFeatures.html">•Key Features</a></li> |
| <li><a href="/docs/Architecture/SketchFeaturesMatrix.html">•Sketch Features Matrix</a></li> |
| <li><a href="/docs/Architecture/Components.html">•Components</a></li> |
| <li><a href="/docs/Architecture/SketchesByComponent.html">•Sketches by Component</a></li> |
| <li><a href="/docs/Architecture/SketchCriteria.html">•Sketch Criteria</a></li> |
| |
| <p id="memory-component"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_memory_component">Memory Component</a> |
| </p> |
| <div class="collapse" id="collapse_memory_component"> |
| <li><a href="/docs/Memory/MemoryComponent.html">•Memory Component</a></li> |
| <li><a href="/docs/Memory/MemoryPerformance.html">•Memory Component Performance</a></li> |
| </div> |
| <li><a href="/docs/Architecture/OrderSensitivity.html">•Notes on Order Sensitivity</a></li> |
| <li><a href="/docs/Architecture/Concurrency.html">•Notes on Concurrency</a></li> |
| </div> |
| |
| <p id="sketch-families"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_sketch_families">Sketch Families</a> |
| </p> |
| <div class="collapse" id="collapse_sketch_families"> |
| |
| <p id="distinct-counting"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_distinct_counting">Distinct Counting</a> |
| </p> |
| <div class="collapse" id="collapse_distinct_counting"> |
| <li><a href="/docs/DistinctCountFeaturesMatrix.html">•Features Matrix</a></li> |
| <li><a href="/docs/DistinctCountMeritComparisons.html">•Figures-of-Merit Comparison</a></li> |
| |
| <p id="cpc-sketches"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_cpc_sketches">CPC Sketches</a> |
| </p> |
| <div class="collapse" id="collapse_cpc_sketches"> |
| <li><a href="/docs/CPC/CPC.html">•CPC Sketch</a></li> |
| <li><a href="/docs/CPC/CpcPerformance.html">•CPC Sketch Performance</a></li> |
| |
| <p id="cpc-examples"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_cpc_examples">CPC Examples</a> |
| </p> |
| <div class="collapse" id="collapse_cpc_examples"> |
| <li><a href="/docs/CPC/CpcJavaExample.html">•CPC Sketch Java Example</a></li> |
| <li><a href="/docs/CPC/CpcCppExample.html">•CPC Sketch C++ Example</a></li> |
| <li><a href="/docs/CPC/CpcPigExample.html">•CPC Sketch Pig UDFs</a></li> |
| <li><a href="/docs/CPC/CpcHiveExample.html">•CPC Sketch Hive UDFs</a></li> |
| </div> |
| </div> |
| |
| <p id="hyperloglog-sketches"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_hyperloglog_sketches">HyperLogLog Sketches</a> |
| </p> |
| <div class="collapse" id="collapse_hyperloglog_sketches"> |
| <li><a href="/docs/HLL/HLL.html">•HLL Sketch</a></li> |
| <li><a href="/docs/HLL/HllMap.html">•HLL Map Sketch</a></li> |
| |
| <p id="hll-examples"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_hll_examples">HLL Examples</a> |
| </p> |
| <div class="collapse" id="collapse_hll_examples"> |
| <li><a href="/docs/HLL/HllJavaExample.html">•HLL Sketch Java Example</a></li> |
| <li><a href="/docs/HLL/HllCppExample.html">•HLL Sketch C++ Example</a></li> |
| <li><a href="/docs/HLL/HllPigUDFs.html">•HLL Sketch Pig UDFs</a></li> |
| <li><a href="/docs/HLL/HllHiveUDFs.html">•HLL Sketch Hive UDFs</a></li> |
| </div> |
| |
| <p id="hll-studies"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_hll_studies">HLL Studies</a> |
| </p> |
| <div class="collapse" id="collapse_hll_studies"> |
| <li><a href="/docs/HLL/HllPerformance.html">•HLL Sketch Performance</a></li> |
| <li><a href="/docs/HLL/Hll_vs_CS_Hllpp.html">•HLL vs Clearspring HLL++</a></li> |
| <li><a href="/docs/HLL/HllSketchVsDruidHyperLogLogCollector.html">•HLL Sketch vs Druid HyperLogLogCollector</a></li> |
| </div> |
| </div> |
| |
| <p id="theta-sketches"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_sketches">Theta Sketches</a> |
| </p> |
| <div class="collapse" id="collapse_theta_sketches"> |
| <li><a href="/docs/Theta/ThetaSketchFramework.html">•Theta Sketch Framework</a></li> |
| |
| <p id="theta-examples"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_examples">Theta Examples</a> |
| </p> |
| <div class="collapse" id="collapse_theta_examples"> |
| <li><a href="/docs/Theta/ConcurrentThetaSketch.html">•Concurrent Theta Sketch</a></li> |
| <li><a href="/docs/Theta/ThetaJavaExample.html">•Theta Sketch Java Example</a></li> |
| <li><a href="/docs/Theta/ThetaSparkExample.html">•Theta Sketch Spark Example</a></li> |
| <li><a href="/docs/Theta/ThetaPigUDFs.html">•Theta Sketch Pig UDFs</a></li> |
| <li><a href="/docs/Theta/ThetaHiveUDFs.html">•Theta Sketch Hive UDFs</a></li> |
| </div> |
| |
| <p id="kmv-tutorial"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_kmv_tutorial">KMV Tutorial</a> |
| </p> |
| <div class="collapse" id="collapse_kmv_tutorial"> |
| <li><a href="/docs/Theta/InverseEstimate.html">•The Inverse Estimate</a></li> |
| <li><a href="/docs/Theta/KMVempty.html">•Empty Sketch</a></li> |
| <li><a href="/docs/Theta/KMVfirstEst.html">•First Estimator</a></li> |
| <li><a href="/docs/Theta/KMVbetterEst.html">•Better Estimator</a></li> |
| <li><a href="/docs/Theta/KMVrejection.html">•Rejection Rules</a></li> |
| <li><a href="/docs/Theta/KMVupdateVkth.html">•Update V(kth) Rule</a></li> |
| </div> |
| |
| <p id="set-operations-and-p-sampling"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_set_operations_and_p-sampling">Set Operations and P-sampling</a> |
| </p> |
| <div class="collapse" id="collapse_set_operations_and_p-sampling"> |
| <li><a href="/docs/Theta/ThetaSketchSetOps.html">•Set Operations</a></li> |
| <li><a href="/docs/Theta/ThetaSetOpsCornerCases.html">•Model & Test Set Operations</a></li> |
| <li><a href="/docs/Theta/ThetaPSampling.html">•<i>p</i>-Sampling</a></li> |
| </div> |
| |
| <p id="accuracy"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_accuracy">Accuracy</a> |
| </p> |
| <div class="collapse" id="collapse_accuracy"> |
| <li><a href="/docs/Theta/ThetaAccuracy.html">•Basic Accuracy</a></li> |
| <li><a href="/docs/Theta/ThetaAccuracyPlots.html">•Accuracy Plots</a></li> |
| <li><a href="/docs/Theta/ThetaErrorTable.html">•Relative Error Table</a></li> |
| <li><a href="/docs/Theta/ThetaSketchSetOpsAccuracy.html">•SetOp Accuracy</a></li> |
| <li><a href="/docs/Theta/AccuracyOfDifferentKUnions.html">•Unions With Different k</a></li> |
| </div> |
| |
| <p id="size"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_size">Size</a> |
| </p> |
| <div class="collapse" id="collapse_size"> |
| <li><a href="/docs/Theta/ThetaSize.html">•Theta Sketch Size</a></li> |
| </div> |
| |
| <p id="speed"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_speed">Speed</a> |
| </p> |
| <div class="collapse" id="collapse_speed"> |
| <li><a href="/docs/Theta/ThetaUpdateSpeed.html">•Update Speed</a></li> |
| <li><a href="/docs/Theta/ThetaMergeSpeed.html">•Merge Speed</a></li> |
| </div> |
| |
| <p id="theta-sketch-theory"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_sketch_theory">Theta Sketch Theory</a> |
| </p> |
| <div class="collapse" id="collapse_theta_sketch_theory"> |
| <li><a href="https://github.com/apache/datasketches-website/tree/master/docs/pdf/ThetaSketchFramework.pdf">•Theta Sketch Framework (PDF)</a></li> |
| <li><a href="https://github.com/apache/datasketches-website/tree/master/docs/pdf/ThetaSketchEquations.pdf">•Theta Sketch Equations (PDF)</a></li> |
| <li><a href="https://github.com/apache/datasketches-website/tree/master/docs/pdf/DataSketches.pdf">•DataSketches (PDF)</a></li> |
| <li><a href="/docs/Theta/ThetaConfidenceIntervals.html">•Confidence Intervals Notes</a></li> |
| <li><a href="/docs/Theta/ThetaMergingAlgorithm.html">•Merging Algorithm Notes</a></li> |
| <li><a href="/docs/Theta/ThetaReferences.html">•Theta References</a></li> |
| </div> |
| </div> |
| |
| <p id="tuple-sketches"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_tuple_sketches">Tuple Sketches</a> |
| </p> |
| <div class="collapse" id="collapse_tuple_sketches"> |
| <li><a href="/docs/Tuple/TupleOverview.html">•Tuple Overview</a></li> |
| |
| <p id="tuple-examples"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_tuple_examples">Tuple Examples</a> |
| </p> |
| <div class="collapse" id="collapse_tuple_examples"> |
| <li><a href="/docs/Tuple/TupleJavaExample.html">•Tuple Java Example</a></li> |
| <li><a href="/docs/Tuple/TupleEngagementExample.html">•Tuple Engagement Example</a></li> |
| <li><a href="/docs/Tuple/TuplePigUDFs.html">•Tuple Pig UDFs</a></li> |
| <li><a href="/docs/Tuple/TupleHiveUDFs.html">•Tuple Hive UDFs</a></li> |
| </div> |
| </div> |
| </div> |
| |
| <p id="most-frequent"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_most_frequent">Most Frequent</a> |
| </p> |
| <div class="collapse" id="collapse_most_frequent"> |
| <li><a href="/docs/Frequency/FrequencySketchesOverview.html">•Frequency Sketches Overview</a></li> |
| |
| <p id="frequent-item-sketches"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_frequent_item_sketches">Frequent Item Sketches</a> |
| </p> |
| <div class="collapse" id="collapse_frequent_item_sketches"> |
| <li><a href="/docs/Frequency/FrequentItemsOverview.html">•Frequent Items Overview</a></li> |
| <li><a href="/docs/Frequency/FrequentItemsErrorTable.html">•Frequent Items Error Table</a></li> |
| <li><a href="/docs/Frequency/FrequentItemsReferences.html">•Frequent Items References</a></li> |
| <li><a href="/docs/Frequency/FrequentItemsPerformance.html">•Frequent Items Performance</a></li> |
| |
| <p id="most-frequent-examples"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_most_frequent_examples">Most Frequent Examples</a> |
| </p> |
| <div class="collapse" id="collapse_most_frequent_examples"> |
| <li><a href="/docs/Frequency/FrequentItemsJavaExample.html">•Frequent Items Java Example</a></li> |
| <li><a href="/docs/Frequency/FrequentItemsCppExample.html">•Frequent Items C++ Example</a></li> |
| <li><a href="/docs/Frequency/FrequentItemsPigUDFs.html">•Frequent Items Pig UDFs</a></li> |
| <li><a href="/docs/Frequency/FrequentItemsHiveUDFs.html">•Frequent Items Hive UDFs</a></li> |
| </div> |
| </div> |
| |
| <p id="frequent-distinct-sketches"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_frequent_distinct_sketches">Frequent Distinct Sketches</a> |
| </p> |
| <div class="collapse" id="collapse_frequent_distinct_sketches"> |
| <li><a href="/docs/Frequency/FrequentDistinctTuplesSketch.html">•Frequent Distinct Tuples Sketch</a></li> |
| </div> |
| </div> |
| |
| <p id="quantiles-and-histograms"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_and_histograms">Quantiles And Histograms</a> |
| </p> |
| <div class="collapse" id="collapse_quantiles_and_histograms"> |
| <li><a href="/docs/Quantiles/SketchingQuantilesAndRanksTutorial.html">•Quantiles and Ranks Tutorial</a></li> |
| <li><a href="/docs/Quantiles/QuantilesOverview.html">•Quantiles Overview</a></li> |
| <li><a href="/docs/KLL/KLLSketch.html">•KLL Floats sketch</a></li> |
| <li><a href="/docs/KLL/KLLAccuracyAndSize.html">•KLL Sketch Accuracy and Size</a></li> |
| <li><a href="/docs/REQ/ReqSketch.html">•REQ Floats sketch</a></li> |
| <li><a href="/docs/Quantiles/OrigQuantilesSketch.html">•Original QuantilesSketch</a></li> |
| |
| <p id="quantiles-examples"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_examples">Quantiles Examples</a> |
| </p> |
| <div class="collapse" id="collapse_quantiles_examples"> |
| <li><a href="/docs/Quantiles/QuantilesJavaExample.html">•Quantiles Sketch Java Example</a></li> |
| <li><a href="/docs/KLL/KLLCppExample.html">•KLL Quantiles Sketch C++ Example</a></li> |
| <li><a href="/docs/Quantiles/QuantilesPigUDFs.html">•Quantiles Sketch Pig UDFs</a></li> |
| <li><a href="/docs/Quantiles/QuantilesHiveUDFs.html">•Quantiles Sketch Hive UDFs</a></li> |
| </div> |
| |
| <p id="quantiles-studies"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_studies">Quantiles Studies</a> |
| </p> |
| <div class="collapse" id="collapse_quantiles_studies"> |
| <li><a href="/docs/QuantilesStudies/DruidApproxHistogramStudy.html">•Druid Approximate Histogram</a></li> |
| <li><a href="/docs/QuantilesStudies/MomentsSketchStudy.html">•Moments Sketch Study</a></li> |
| <li><a href="/docs/QuantilesStudies/QuantilesStreamAStudy.html">•Quantiles StreamA Study</a></li> |
| <li><a href="/docs/QuantilesStudies/ExactQuantiles.html">•Exact Quantiles for Studies</a></li> |
| </div> |
| |
| <p id="quantiles-sketch-theory"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_sketch_theory">Quantiles Sketch Theory</a> |
| </p> |
| <div class="collapse" id="collapse_quantiles_sketch_theory"> |
| <li><a href="https://github.com/apache/datasketches-website/tree/master/docs/pdf/Quantiles_KLL.pdf">•Optimal Quantile Approximation in Streams</a></li> |
| <li><a href="/docs/Quantiles/QuantilesReferences.html">•Quantiles References</a></li> |
| </div> |
| </div> |
| |
| <p id="sampling"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_sampling">Sampling</a> |
| </p> |
| <div class="collapse" id="collapse_sampling"> |
| <li><a href="/docs/Sampling/ReservoirSampling.html">•Reservoir Sampling</a></li> |
| <li><a href="/docs/Sampling/ReservoirSamplingPerformance.html">•Reservoir Sampling Performance</a></li> |
| <li><a href="/docs/Sampling/VarOptSampling.html">•VarOpt Sampling</a></li> |
| |
| <p id="sampling-examples"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_sampling_examples">Sampling Examples</a> |
| </p> |
| <div class="collapse" id="collapse_sampling_examples"> |
| <li><a href="/docs/Sampling/ReservoirSamplingJava.html">•Reservoir Sampling Java Example</a></li> |
| <li><a href="/docs/Sampling/ReservoirSamplingPigUDFs.html">•Reservoir Sampling Pig UDFs</a></li> |
| <li><a href="/docs/Sampling/VarOptSamplingJava.html">•VarOpt Sampling Java Example</a></li> |
| <li><a href="/docs/Sampling/VarOptPigUDFs.html">•VarOpt Sampling Pig UDFs</a></li> |
| </div> |
| </div> |
| </div> |
| |
| <p id="system-integrations"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_system_integrations">System Integrations</a> |
| </p> |
| <div class="collapse" id="collapse_system_integrations"> |
| <li><a href="/docs/SystemIntegrations/ApacheDruidIntegration.html">•Using Sketches in ApacheDruid</a></li> |
| <li><a href="/docs/SystemIntegrations/ApacheHiveIntegration.html">•Using Sketches in Apache Hive</a></li> |
| <li><a href="/docs/SystemIntegrations/ApachePigIntegration.html">•Using Sketches in Apache Pig</a></li> |
| <li><a href="/docs/SystemIntegrations/PostgreSQLIntegration.html">•Using Sketches in PostgreSQL</a></li> |
| </div> |
| |
| <p id="community"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_community">Community</a> |
| </p> |
| <div class="collapse" id="collapse_community"> |
| <li><a href="/docs/Community/index.html">•Community</a></li> |
| <li><a href="/docs/Community/Downloads.html">•Downloads</a></li> |
| <li><a href="/docs/Community/NewCommitterProcess.html">•Committer Process</a></li> |
| <li><a href="/docs/Community/ReleaseProcessForCppComponents.html">•Release Process For CPP Components</a></li> |
| <li><a href="/docs/Community/ReleaseProcessForJavaComponents.html">•Release Process For Java Components</a></li> |
| <li><a href="/docs/Community/Transitioning.html">•Transitioning from prior GitHub Site</a></li> |
| </div> |
| |
| <p id="research"> |
| <a data-toggle="collapse" class="menu collapsed" href="#collapse_research">Research</a> |
| </p> |
| <div class="collapse" id="collapse_research"> |
| <li><a href="/docs/Community/Research.html">•Research</a></li> |
| </div> |
| </div> |
| <!-- End _includes/toc.html --> |
| |
| |
| <!-- Start _includes/tocScript.html --> |
| <script> |
| (function () { |
| |
| var findLineItem = function (path) { |
| return document.querySelector(`#toc [href="${path}"]`); |
| }; |
| |
| function findNavItem(path) { |
| return document.querySelector(`.nav [href="${path}"]`); |
| } |
| |
| var highlighLineItem = function (element) { |
| element.classList.add('highlight'); |
| }; |
| |
| var checkHasClass = function (element, className) { |
| return element.className.split(' ').find(function (item) { return item === className || '' }) |
| } |
| |
| var findAllCollapseParents = function (element) { |
| var collapseMenus = []; |
| var elementPointer = element; |
| while (elementPointer !== document.body) { |
| if (checkHasClass(elementPointer, 'collapse')) { |
| collapseMenus.push(elementPointer); |
| } |
| elementPointer = elementPointer.parentElement |
| } |
| return collapseMenus |
| }; |
| |
| var openMenuItem = function (element) { |
| // $(element).collapse('show') would start a transition, adding `in` class instead. |
| element.classList.add('in'); |
| }; |
| |
| var openAllFromList = function (elementList) { |
| elementList.forEach(openMenuItem); |
| }; |
| |
| var highlightAndOpenMenu = function () { |
| // Highlight & expand nav item in the TOC |
| var currentLineItem = findLineItem(document.location.pathname); |
| highlighLineItem(currentLineItem); |
| openAllFromList(findAllCollapseParents(currentLineItem)); |
| |
| // Highlight nav item in top navigation |
| highlighLineItem(findNavItem(document.location.pathname)); |
| }; |
| |
| $(highlightAndOpenMenu); |
| |
| }()); |
| </script> |
| <!-- End _includes/tocScript.html --> |
| |
| </div> |
| <!-- End ToC Block --> |
| <div class="col-md-9 doc-content"> |
| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| --> |
| <h1 id="new-committer-process">New Committer Process</h1> |
| |
| <ul> |
| <li>Reference: <a href="https://community.apache.org/newcommitter.html#new-committer-process">New Committer Process</a></li> |
| </ul> |
| |
| <h2 id="create-proposal-for-new-candidate">Create Proposal for New Candidate</h2> |
| <ul> |
| <li><strong><em>NOTE</em></strong> On private datasketches mail list ONLY!</li> |
| </ul> |
| |
| <p>Here are some topics to consider when writing your proposal about the candidate. Any one of which may be sufficient to invite someone to join us as a new committer.</p> |
| |
| <h3 id="interpersonal-skills">Interpersonal Skills</h3> |
| |
| <ul> |
| <li>Ability to work cooperatively with peers |
| <ul> |
| <li>Is this person willing to help peers to balance workload?</li> |
| <li>Does this person respond positively to criticism?</li> |
| <li>Does this person participate in the group decision-making processes?</li> |
| </ul> |
| </li> |
| <li>Ability and willingness to guide and mentor others |
| <ul> |
| <li>How clearly does this person communicate?</li> |
| <li>Does this person take pains to explain elementary concepts?</li> |
| <li>Does this person demonstrate patience and respect when answering user queries?</li> |
| </ul> |
| </li> |
| </ul> |
| |
| <h3 id="product-contributions">Product Contributions</h3> |
| <p>This can be in a number of areas including:</p> |
| |
| <ul> |
| <li>PRs or code grants</li> |
| <li>Documentation</li> |
| <li>CI/CD automation</li> |
| <li>Build systems</li> |
| <li>Website structure and content</li> |
| </ul> |
| |
| <h3 id="marketing-contributions">Marketing Contributions</h3> |
| <ul> |
| <li>Evangelizing the use of sketches to solve problems</li> |
| <li>Organizing workshops, meetups</li> |
| <li>Doing or promoting tech talks with our team and with external scientists and engineers.</li> |
| <li>Writing blogposts and “what’s new” posts for community growth</li> |
| </ul> |
| |
| <h3 id="scientific-contributions">Scientific Contributions</h3> |
| |
| <ul> |
| <li>Has this person authored research papers on sketching algorithms and contributed code that was developed as part of that research to our project?</li> |
| <li>Has this person provided substantial assistance to us in understanding the nuances |
| of a theoretical research paper on sketching and how it could be implemented in our library?</li> |
| <li>Has this person demonstrated a high level of interest in having their theoretical work implemented in our library even though this person may not be a fluent programmer?</li> |
| </ul> |
| |
| <h3 id="community-contributions">Community Contributions</h3> |
| |
| <ul> |
| <li>Has this person participated in our mailing lists, Slack discussions, and GitHub issues?</li> |
| <li>Is this person known and respected by our active committers and our PMC?</li> |
| <li>Has exhibited a high level of interest and commitment to learn, |
| not only about our code base, but about the underlying |
| theory of sketching as well?</li> |
| </ul> |
| |
| <h2 id="write-proposal-in-a-discuss-thread">Write Proposal in a DISCUSS thread</h2> |
| <p><strong><em>NOTE</em></strong> On private datasketches mail list ONLY!</p> |
| |
| <hr /> |
| <p>From: (PMC Member)<br /> |
| To: private datasketches mail list <strong>ONLY! DO NOT CC!</strong><br /> |
| Subject: [DISCUSS] Proposal to invite <candidate> as new committer.</candidate></p> |
| |
| <p>Dear PMC,</p> |
| |
| <p>This is a proposal to offer to [Candidate] to join us as a committer on our project.</p> |
| |
| <p>(justification)</p> |
| |
| <p>This discussion will end on (month/ day at HH:MM [TimeZone]).</p> |
| |
| <p>If this discussion produces a positive consensus or no objections, I will use <a href="https://www.apache.org/foundation/voting.html">lazy consensus</a> and proceed to a vote.</p> |
| |
| <p>Regards, |
| (PMC Member)</p> |
| |
| <hr /> |
| |
| <h2 id="vote">Vote</h2> |
| <h3 id="write-a-vote-email-similar-to-the-following">Write a vote email similar to the following:</h3> |
| <p><strong><em>NOTE</em></strong> On private datasketches mail list ONLY!</p> |
| |
| <hr /> |
| <p>From: (PMC Member)<br /> |
| To: private datasketches mail list<br /> |
| Subject: [VOTE] [Candidate] as new Committer<br /></p> |
| |
| <p>(Summarize the results of the DISCUSS thread )</p> |
| |
| <p>The vote will be performed as follows:</p> |
| |
| <ul> |
| <li>Voting ends X days from today, i.e, midnight UTC on YYYY-MM-DD, <TimeZone>. (we recommend at least 3 days)</TimeZone></li> |
| <li>At least 3 (+1) PMC votes and no (-1) votes.</li> |
| </ul> |
| |
| <p>Please vote accordingly:</p> |
| |
| <p>[ ] +1 approve<br /> |
| [ ] +0 no opinion<br /> |
| [ ] -1 disapprove with the reason</p> |
| |
| <p>Regards,<br /> |
| (PMC Member)</p> |
| |
| <hr /> |
| |
| <h2 id="close-the-vote">Close the Vote</h2> |
| <p><strong><em>NOTE</em></strong> On private datasketches mail list ONLY!</p> |
| |
| <p>After sufficient votes have been obtained and the time as elapsed, issue an email similar to the following:</p> |
| |
| <hr /> |
| <p>Subject: [RESULT][VOTE] [Candidate] as new Committer</p> |
| |
| <p>The vote is now closed.</p> |
| |
| <p>Passed/Did not pass with X (+1) binding votes (list names), and Y 0 votes and Z -1 votes.</p> |
| |
| <p>Regards, |
| (PMC MEMBER)</p> |
| |
| <hr /> |
| |
| <h2 id="committer-invite-example">Committer Invite Example</h2> |
| <p><strong><em>NOTE</em></strong> CC’d to private datasketches mail list ONLY!</p> |
| |
| <p>Write a letter to the candidate similar to the following:</p> |
| |
| <hr /> |
| <p>To: [Candidate]<br /> |
| CC: private datasketches mail list<br /> |
| From: (PMC Member)<br /> |
| Subject: Invitation to become DataSketches Committer<br /></p> |
| |
| <p>Hello [Candidate]</p> |
| |
| <p>The datasketches Project Management Committee (PMC) hereby offers |
| you committer privileges to the project. These privileges are offered on |
| the understanding that you’ll use them reasonably and with common sense. We |
| like to work on trust rather than unnecessary constraints.</p> |
| |
| <p>Being a committer grants you write access to all of our repositories. In our normal code committing process we all use PRs and like to have at least one other committer or PMC member review the code before it is merged into <em>master</em>. However, for trivial edits (e.g. typos, documentation corrections) you may make commits on your own.</p> |
| |
| <p>Being a committer does not require you to participate any more than |
| you already do. It does tend to make one even more involved as you will |
| probably find that you spend more time with our project. Of course, you can decline and |
| instead remain as a contributor, participating as you do now.</p> |
| |
| <p>A. This personal invitation is a chance for you to accept or decline |
| in private. Either way, please let us know in reply to the |
| private datasketches mail list address only.</p> |
| |
| <p>B. If you accept, you will receive a follow-up message with |
| the next steps to establish you as a committer.</p> |
| |
| <p>Regards |
| (PMC MEMBER)</p> |
| |
| <hr /> |
| |
| <h2 id="if-candidate-accepts-followup-instructions">If Candidate Accepts, Followup Instructions</h2> |
| <p><strong><em>NOTE</em></strong> CC’d to private datasketches mail list ONLY!</p> |
| |
| <hr /> |
| <p>To: [Candidate]<br /> |
| CC: private datasketches mail list<br /> |
| From: (PMC Member)<br /> |
| Subject: Welcome to the Apache DataSketches Project as a Committer!</p> |
| |
| <p>Dear (Candidate),</p> |
| |
| <p>Welcome to our project! We are excited to have you join us.</p> |
| |
| <p>To complete your admission as a committer you need to register an <em>Individual Contributor License Agreement</em> (ICLA) with ASF.</p> |
| |
| <ul> |
| <li>Explanation of the ICLA: https://www.apache.org/licenses/contributor-agreements.html</li> |
| <li>ICLA form: https://www.apache.org/licenses/icla.pdf. |
| <ul> |
| <li>On the form choose a preferred Apache ID. This will become your Apache email address, e.g. <ApacheID>@apache.org. When your account gets set up you will be able to specify a forwarding email address for messages sent to your apache.org email. You can look to see if your preferred ID is already taken at <https://people.apache.org/committer-index.html>.</ApacheID></li> |
| <li>Under notify project write <em>Apache DataSketches</em></li> |
| </ul> |
| </li> |
| <li>Note that the ICLA contains your confidential information and should only be sent directly to <a href="mailto:secretary@apache.org">secretary@apache.org</a> and not CC’d to this private@ email address nor to me</li> |
| <li>In your letter to the secretary please request the secretary to notify private datasketches mail list once the ICLA has been recorded.</li> |
| <li>Once you have sent your ICLA to the secretary please notify us on this private@ email address so that we can coordinate with the secretary in getting your account set up.</li> |
| </ul> |
| |
| <p>After the ICLA has been recorded and your account created we will announce your joining the project on <a href="mailto:dev@datasketches.apache.org">dev@datasketches.apache.org</a>.</p> |
| |
| <p>The developer section of the website describes the roles and provides other resources:</p> |
| <ul> |
| <li><a href="https://www.apache.org/foundation/how-it-works.html">https://www.apache.org/foundation/how-it-works.html</a></li> |
| <li><a href="https://www.apache.org/dev/">https://www.apache.org/dev/</a></li> |
| </ul> |
| |
| <p>The Apache Incubator project also has some useful information for new committers:</p> |
| <ul> |
| <li><a href="https://incubator.apache.org/guides/committer.html">https://incubator.apache.org/guides/committer.html</a></li> |
| <li><a href="https://incubator.apache.org/guides/ppmc.html">https://incubator.apache.org/guides/ppmc.html</a></li> |
| </ul> |
| |
| <p>Just as before you became a committer, participation in any ASF community |
| requires adherence to the ASF Code of Conduct:</p> |
| <ul> |
| <li><a href="https://www.apache.org/foundation/policies/conduct.html">https://www.apache.org/foundation/policies/conduct.html</a></li> |
| </ul> |
| |
| <p>Again, welcome to the DataSketches project!</p> |
| |
| <p>Best Regards,<br /> |
| (PMC Member)</p> |
| |
| <hr /> |
| |
| <h2 id="sample-letter-from-secretary-confirming-filing-of-the-icla">Sample Letter from Secretary confirming filing of the ICLA</h2> |
| <p><strong><em>NOTE</em></strong> CC’d to private datasketches mail list ONLY!</p> |
| |
| <hr /> |
| <p>From: Matt Sicker (secretary)<br /> |
| To: private datasketches mail list<br /> |
| Subject: [FORM] Account Request - candidate@gxyz.com: [Candidate]<br /> |
| Date:<br /> |
| List: private datasketches mail list</p> |
| |
| <p>Prospective userid: 123456<br /> |
| Full Name: [Candidate]<br /> |
| Forwarding emal address: blah<br /></p> |
| |
| <hr /> |
| |
| <h2 id="account-creation">Account Creation</h2> |
| <p><strong><em>NOTE</em></strong> CC’d to private datasketches mail list ONLY!</p> |
| |
| <p>If the ICLA identifies the project and a valid Apache id, and the [RESULT][VOTE] message has been posted to the PMC private list, then the account creation request is made by the secretary or assistant who files the ICLA.</p> |
| |
| <p>Otherwise, new account requests will only be accepted from PMC chairs and ASF members.</p> |
| |
| <ul> |
| <li>See <a href="https://www.apache.org/dev/pmc.html#newcommitter">https://www.apache.org/dev/pmc.html#newcommitter</a></li> |
| </ul> |
| |
| <h2 id="completing-the-new-committer-account-setup">Completing the New Committer Account Setup</h2> |
| <p>After the committer account is established.</p> |
| |
| <hr /> |
| |
| <p>To: private datasketches mail list, (new committer)<br /> |
| Subject: Completing the setup of your Account</p> |
| |
| <p>Hello (new committer),<br /> |
| The ASF Infrastructure team has set up your committer account with |
| the username ‘blah’ with an Apache email of <blah>@apache.org. |
| The next portion of the setup you must do.</blah></p> |
| |
| <p>Please follow the instructions at the <a href="https://infra.apache.org/new-committers-guide.html">Guide For New Project Committers</a> to set up your SSH, |
| svn password, svn configuration, mail forwarding, GitHub ID, etc.</p> |
| |
| <ul> |
| <li>For new PMC members only: |
| <ul> |
| <li>Please subscribe to the DataSketches Project Management |
| Committee mailing list private datasketches mail list.</li> |
| </ul> |
| </li> |
| </ul> |
| |
| <p>As a committer on the DataSketches Project you will have commit access to the following DataSketches repositories as follows:</p> |
| |
| <ul> |
| <li>https://github.com/apache/datasketches-java</li> |
| <li>https://github.com/apache/datasketches-cpp</li> |
| <li>https://github.com/apache/datasketches-hive</li> |
| <li>https://github.com/apache/datasketches-pig</li> |
| <li>https://github.com/apache/datasketches-postgresql</li> |
| <li>https://github.com/apache/datasketches-characterization</li> |
| <li>https://github.com/apache/datasketches-website</li> |
| <li>https://github.com/apache/datasketches-memory</li> |
| <li>https://github.com/apache/datasketches-vector</li> |
| <li>https://github.com/apache/datasketches-server</li> |
| <li>https://github.com/apache/datasketches-experimentation</li> |
| </ul> |
| |
| <p>Please become familiar with the following ASF resources:</p> |
| |
| <ul> |
| <li><a href="https://www.apache.org/dev/">Apache developer’s pages</a></li> |
| <li><a href="https://gitbox.apache.org/setup/">Account Setup</a> (This is the part only you can do.)</li> |
| </ul> |
| |
| <p>Once this is completed, I will send you an invite to our ASF slack channel (with your ASF account) and will announce your joining our project on our dev@ list.</p> |
| |
| <p>ASF documentation is maintained by volunteers and hence can be out-of-date and incomplete, so we beg your understanding, and, of course, perhaps you can help us fix that!</p> |
| |
| <p>Naturally, if you have any questions be sure to ask us on the <a href="mailto:dev@datasketches.apache.org">dev@datasketches.apache.org</a> mailing list or on our slack channel.</p> |
| |
| <p>Regards, |
| (PMC Member) |
| —</p> |
| |
| <h2 id="adding-the-new-committer-to-the-datasketches-roster">Adding the New Committer to the DataSketches Roster</h2> |
| |
| <h3 id="pmc">PMC</h3> |
| <p>Any PMC Member can add the new committer to the roster on the <a href="https://whimsy.apache.org/roster/committee/datasketches">DataSketches Roster</a> by clicking the <strong>ADD</strong> button at the top of the page.</p> |
| |
| <h2 id="committer-announce-template">Committer Announce Template</h2> |
| <p>After the committer account is established.</p> |
| |
| <hr /> |
| <p>From: PMC Member |
| To: dev@datasketches.apache.org |
| Subject: Please Welcome (new committer) as new committer</p> |
| |
| <p>I am happy to announce that (new committer) has joined us as a new committer!</p> |
| |
| <p>(add specific details here about new committer)</p> |
| |
| <p>Regards, |
| (PMC Member)</p> |
| |
| <hr /> |
| |
| |
| </div> <!-- End content --> |
| </div> <!-- End row --> |
| </div> <!-- End Container --> |
| |
| <!-- Start _include/page_footer.html --> |
| <footer class="ds-footer"> |
| <div class="container"> |
| <div class="text-center"> |
| <p> |
| <div>Copyright © 2024 <a href="https://www.apache.org">Apache Software Foundation</a>, |
| Licensed under the Apache License, Version 2.0. All Rights Reserved. |
| | <a href="https://privacy.apache.org/policies/privacy-policy-public.html">Privacy Policy</a><br/> |
| Apache DataSketches, Apache, the Apache feather logo, and the Apache DataSketches project logos are trademarks of The Apache Software Foundation.<br/> |
| All other marks mentioned may be trademarks or registered trademarks of their respective owners. |
| </div> |
| </p> |
| </div> |
| </div> |
| </footer> |
| <!-- End _include/page_footer.html --> |
| |
| </body> |
| |
| </html> |
| <!-- End _layouts/doc_page.html--> |
