Google Git
Sign in
apache / tez / 199ab0198d86344560675d2f46970de0f4053123
commit199ab0198d86344560675d2f46970de0f4053123[log] [tgz]
authorSiddharth Seth <sseth@apache.org>Wed Dec 03 17:12:20 2014 -0800
committerSiddharth Seth <sseth@apache.org>Thu Dec 04 14:00:30 2014 -0800
treec972faab4a91f0783b5e07843116e878b8395138
parent52318e26fd7cffbaa4ab048cc31e83b2bb1caa4f [diff]
TEZ-1808. Job can fail since name of intermediate files can be too long
in specific situation. Contributed by Tsuyoshi OZAWA.
(cherry picked from commit 87b4c6b40cbe607b9d16b9d7e4a8f965757b5196)

Conflicts:
	CHANGES.txt
(cherry picked from commit 20c04e5534cddc20702fd025c407a2ccadc39fe9)
2 files changed
tree: c972faab4a91f0783b5e07843116e878b8395138
  1. docs/
  2. tez-api/
  3. tez-common/
  4. tez-dag/
  5. tez-dist/
  6. tez-examples/
  7. tez-mapreduce/
  8. tez-plugins/
  9. tez-runtime-internals/
  10. tez-runtime-library/
  11. tez-tests/
  12. tez-tools/
  13. INSTALL.md ⇨ docs/src/site/markdown/install.md
  14. .gitignore
  15. BUILDING.txt
  16. CHANGES.txt
  17. KEYS
  18. LICENSE.txt
  19. NOTICE.txt
  20. pom.xml
  21. README.md
README.md

Apache Tez

Apache Tez is a generic data-processing pipeline engine envisioned as a low-level engine for higher abstractions such as Apache Hadoop Map-Reduce, Apache Pig, Apache Hive etc.

At it's heart, tez is very simple and has just two components:

  • The data-processing pipeline engine where-in one can plug-in input, processing and output implementations to perform arbitrary data-processing. Every ‘task’ in tez has the following:
  • Input to consume key/value pairs from.
  • Processor to process them.
  • Output to collect the processed key/value pairs.
  • A master for the data-processing application, where-by one can put together arbitrary data-processing ‘tasks’ described above into a task-DAG to process data as desired. The generic master is implemented as a Apache Hadoop YARN ApplicationMaster.