This is a post-hoc analysis tool for Apache Tez which splits an aggregated yarn log file to separate files into a hierarchical folder structure.
.
├── vc0525.your.domain.com_8041
│ └── container_e10_1575565459633_0004_01_000001
│ ├── container-localizer-syslog
│ ├── dag_1575565459633_0004_1-tez-dag.pb.txt
│ ├── dag_1575565459633_0004_1.dot
│ ├── prelaunch.err
│ ├── prelaunch.out
│ ├── stderr
│ ├── stdout
│ ├── syslog
│ ├── syslog_dag_1575565459633_0004_1
│ └── syslog_dag_1575565459633_0004_1_post
├── vc0526.your.domain.com_8041
│ └── container_e10_1575565459633_0004_01_000004
│ ├── container-localizer-syslog
│ ├── prelaunch.err
│ ├── prelaunch.out
│ ├── stderr
│ ├── stdout
│ ├── syslog
│ └── syslog_attempt_1575565459633_0004_1_00_000000_2
├── vc0528.your.domain.com_8041
│ └── container_e10_1575565459633_0004_01_000002
│ ├── container-localizer-syslog
│ ├── prelaunch.err
│ ├── prelaunch.out
│ ├── stderr
│ ├── stdout
│ ├── syslog
│ └── syslog_attempt_1575565459633_0004_1_00_000000_0
├── vc0529.your.domain.com_8041
│ └── container_e10_1575565459633_0004_01_000005
│ ├── container-localizer-syslog
│ ├── prelaunch.err
│ ├── prelaunch.out
│ ├── stderr
│ ├── stdout
│ ├── syslog
│ └── syslog_attempt_1575565459633_0004_1_00_000000_3
└── vc0536.your.domain.com_8041
└── container_e10_1575565459633_0004_01_000003
├── container-localizer-syslog
├── prelaunch.err
├── prelaunch.out
├── stderr
├── stdout
├── syslog
└── syslog_attempt_1575565459633_0004_1_00_000000_1
To use the tool, run e.g.
tez-log-splitter.sh application_1576254620247_0010 (app log is fetched from yarn)tez-log-splitter.sh ~/path/to/application_1576254620247_0010.log (...when app log is already on your computer)tez-log-splitter.sh ~/path/to/application_1576254620247_0010.log.gz (...when app log is already on your computer in gz)