published: true title: DataSketches layout: html_page id: home
<span class="fa fa-database fa-4x"></span>
<p class="text-justify">This library has been specifically designed for big data systems.
Included are adaptors for Hadoop Pig and Hive, which also can be used as examples for other systems,
and many other capabilities typically required in big data analysis systems.
For example, a Memory package for managing large off-heap memory data structures.</p>
<span class="fa fa-bar-chart-o fa-4x"></span><br>
<p class="text-justify">Built-in Theta Sketch set operators (Union, Intersection, Difference)
produce sketches as a result (and not just a number) enabling full set expressions of cardinality,
such as ((A ∪ B) ∩ (C ∪ D)) \ (E ∪ F).
This capability along with predictable and superior accuracy
(compared with <i>Include/Exclude</i> approaches) enable unprecedented analysis capabilities
for fast queries. </p>