Google Git
Sign in
apache / tez

Mirror of Apache Tez

Mirrored from http://gitbox.apache.org/repos/asf/tez.git
Clone this repo:

Branches

More...

Tags

More...
  1. 3ce961b TEZ-4208 : Pipelinesorter uses single SortSpan after spill (Rajesh Balamohan via Ashutosh Chauhan) by Rajesh Balamohan · 3 hours ago master
  2. 5f3cac5 TEZ-4212. Fix build checkstyle configuration and suppressions dtd URLs (Jonathan Eagles reviewed by László Bodor) by Jonathan Eagles · 2 days ago
  3. 3c405c9 TEZ-4172: Let tasks be killed after too many overall attempts (László Bodor reviewed by Jonathan Turner Eagles) by László Bodor · 2 days ago
  4. 69a2bde TEZ-4203. Findbugs: MergeThread.shuffleSchedulerThread; locked 80% of time by Rajesh Balamohan · 5 days ago
  5. 9d2b61b TEZ-4204: Data race in RootInputInitializerManager (Mustafa Iman via Ashutosh Chauhan) by Mustafa Iman · 7 days ago

Apache Tez

Apache Tez is a generic data-processing pipeline engine envisioned as a low-level engine for higher abstractions such as Apache Hadoop Map-Reduce, Apache Pig, Apache Hive etc.

At its heart, tez is very simple and has just two components:

  • The data-processing pipeline engine where-in one can plug-in input, processing and output implementations to perform arbitrary data-processing. Every ‘task’ in tez has the following:
  • Input to consume key/value pairs from.
  • Processor to process them.
  • Output to collect the processed key/value pairs.
  • A master for the data-processing application, where-by one can put together arbitrary data-processing ‘tasks’ described above into a task-DAG to process data as desired. The generic master is implemented as a Apache Hadoop YARN ApplicationMaster.