blob: 70e4f085a2070acce46c41579aa92bf272b15138 [file] [log] [blame]
PREHOOK: query: EXPLAIN
SELECT tmp.key
FROM (SELECT x1.key AS key FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
UNION ALL
SELECT x2.key AS key FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)) tmp
ORDER BY tmp.key
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: EXPLAIN
SELECT tmp.key
FROM (SELECT x1.key AS key FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
UNION ALL
SELECT x2.key AS key FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)) tmp
ORDER BY tmp.key
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Map 1 <- Map 5 (BROADCAST_EDGE), Union 2 (CONTAINS)
Map 4 <- Reducer 6 (BROADCAST_EDGE), Union 2 (CONTAINS)
Reducer 3 <- Union 2 (SIMPLE_EDGE)
Reducer 6 <- Map 5 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: x1
filterExpr: key is not null (type: boolean)
probeDecodeDetails: cacheKey:HASH_MAP_MAPJOIN_53_container, bigKeyColName:key, smallTablePos:1, keyRatio:0.004
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Map 5
Statistics: Num rows: 39 Data size: 3393 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Statistics: Num rows: 78 Data size: 6786 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: x2
filterExpr: key is not null (type: boolean)
probeDecodeDetails: cacheKey:HASH_MAP_MAPJOIN_54_container, bigKeyColName:key, smallTablePos:1, keyRatio:0.004
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Reducer 6
Statistics: Num rows: 39 Data size: 3393 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Statistics: Num rows: 78 Data size: 6786 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 5
Map Operator Tree:
TableScan
alias: y2
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 3
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
Statistics: Num rows: 78 Data size: 6786 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 78 Data size: 6786 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Union 2
Vertex: Union 2
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: SELECT tmp.key
FROM (SELECT x1.key AS key FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
UNION ALL
SELECT x2.key AS key FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)) tmp
ORDER BY tmp.key
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: SELECT tmp.key
FROM (SELECT x1.key AS key FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
UNION ALL
SELECT x2.key AS key FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)) tmp
ORDER BY tmp.key
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
RUN: Stage-1:MAPRED
128
128
128
128
128
128
146
146
146
146
150
150
213
213
213
213
224
224
224
224
238
238
238
238
255
255
255
255
273
273
273
273
273
273
278
278
278
278
311
311
311
311
311
311
369
369
369
369
369
369
401
401
401
401
401
401
401
401
401
401
406
406
406
406
406
406
406
406
66
66
98
98
98
98
PREHOOK: query: EXPLAIN
SELECT tmp.key
FROM (SELECT x1.key AS key FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
UNION ALL
SELECT x2.key AS key FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)) tmp
ORDER BY tmp.key
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: EXPLAIN
SELECT tmp.key
FROM (SELECT x1.key AS key FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
UNION ALL
SELECT x2.key AS key FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)) tmp
ORDER BY tmp.key
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE), Union 3 (CONTAINS)
Reducer 4 <- Union 3 (SIMPLE_EDGE)
Reducer 7 <- Map 6 (SIMPLE_EDGE), Map 8 (SIMPLE_EDGE), Union 3 (CONTAINS)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: x1
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 5
Map Operator Tree:
TableScan
alias: y1
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 6
Map Operator Tree:
TableScan
alias: x2
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 8
Map Operator Tree:
TableScan
alias: y2
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
Statistics: Num rows: 39 Data size: 3393 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Statistics: Num rows: 78 Data size: 6786 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
Statistics: Num rows: 78 Data size: 6786 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 78 Data size: 6786 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 7
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
Statistics: Num rows: 39 Data size: 3393 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Statistics: Num rows: 78 Data size: 6786 Basic stats: COMPLETE Column stats: COMPLETE
Union 3
Vertex: Union 3
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: SELECT tmp.key
FROM (SELECT x1.key AS key FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
UNION ALL
SELECT x2.key AS key FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)) tmp
ORDER BY tmp.key
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: SELECT tmp.key
FROM (SELECT x1.key AS key FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
UNION ALL
SELECT x2.key AS key FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)) tmp
ORDER BY tmp.key
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
RUN: Stage-1:MAPRED
128
128
128
128
128
128
146
146
146
146
150
150
213
213
213
213
224
224
224
224
238
238
238
238
255
255
255
255
273
273
273
273
273
273
278
278
278
278
311
311
311
311
311
311
369
369
369
369
369
369
401
401
401
401
401
401
401
401
401
401
406
406
406
406
406
406
406
406
66
66
98
98
98
98
PREHOOK: query: EXPLAIN
SELECT tmp.key
FROM (SELECT x1.key AS key FROM src1 x1 GROUP BY x1.key
UNION ALL
SELECT x2.key AS key FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)) tmp
ORDER BY tmp.key
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: EXPLAIN
SELECT tmp.key
FROM (SELECT x1.key AS key FROM src1 x1 GROUP BY x1.key
UNION ALL
SELECT x2.key AS key FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)) tmp
ORDER BY tmp.key
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Map 6 <- Reducer 5 (BROADCAST_EDGE), Union 3 (CONTAINS)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 3 (CONTAINS)
Reducer 4 <- Union 3 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: x1
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: key
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: key (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 16 Data size: 1376 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 16 Data size: 1376 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 6
Map Operator Tree:
TableScan
alias: x2
filterExpr: key is not null (type: boolean)
probeDecodeDetails: cacheKey:HASH_MAP_MAPJOIN_35_container, bigKeyColName:key, smallTablePos:1, keyRatio:0.004
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Reducer 5
Statistics: Num rows: 39 Data size: 3393 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Statistics: Num rows: 55 Data size: 4769 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 16 Data size: 1376 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Statistics: Num rows: 55 Data size: 4769 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
Statistics: Num rows: 55 Data size: 4785 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 55 Data size: 4785 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Union 3
Vertex: Union 3
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: SELECT tmp.key
FROM (SELECT x1.key AS key FROM src1 x1 GROUP BY x1.key
UNION ALL
SELECT x2.key AS key FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)) tmp
ORDER BY tmp.key
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: SELECT tmp.key
FROM (SELECT x1.key AS key FROM src1 x1 GROUP BY x1.key
UNION ALL
SELECT x2.key AS key FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)) tmp
ORDER BY tmp.key
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
RUN: Stage-1:MAPRED
128
128
128
128
146
146
146
150
150
213
213
213
224
224
224
238
238
238
255
255
255
273
273
273
273
278
278
278
311
311
311
311
369
369
369
369
401
401
401
401
401
401
406
406
406
406
406
66
66
98
98
98
PREHOOK: query: EXPLAIN
SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: EXPLAIN
SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Map 1 <- Map 6 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (BROADCAST_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: x2
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Map 6
Statistics: Num rows: 39 Data size: 3393 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
minReductionHashAggr: 0.94871795
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 6
Map Operator Tree:
TableScan
alias: y2
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Left Semi Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Reducer 5
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
keys: _col0 (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: bigint)
null sort order: zz
sort order: ++
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: bigint)
outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
RUN: Stage-1:MAPRED
128 1
146 1
150 1
213 1
224 1
238 1
255 1
273 1
278 1
311 1
369 1
401 1
406 1
66 1
98 1
PREHOOK: query: EXPLAIN
SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: EXPLAIN
SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Map 1 <- Map 6 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (BROADCAST_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: x2
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Map 6
Statistics: Num rows: 39 Data size: 3393 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
minReductionHashAggr: 0.94871795
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 6
Map Operator Tree:
TableScan
alias: y2
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Left Semi Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Reducer 5
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
keys: _col0 (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: bigint)
null sort order: zz
sort order: ++
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: bigint)
outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src x1 JOIN src1 y1 ON (x1.key = y1.key)
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
RUN: Stage-1:MAPRED
128 1
146 1
150 1
213 1
224 1
238 1
255 1
273 1
278 1
311 1
369 1
401 1
406 1
66 1
98 1
PREHOOK: query: EXPLAIN
SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src1 x1
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: EXPLAIN
SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src1 x1
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Map 5 <- Map 1 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (BROADCAST_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: x1
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: key (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 16 Data size: 1376 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 16 Data size: 1376 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 5
Map Operator Tree:
TableScan
alias: x2
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Map 1
Statistics: Num rows: 39 Data size: 3393 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
minReductionHashAggr: 0.94871795
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 16 Data size: 1376 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Left Semi Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Map 5
Statistics: Num rows: 2 Data size: 172 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
keys: _col0 (type: string)
minReductionHashAggr: 0.5
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: bigint)
null sort order: zz
sort order: ++
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: bigint)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src1 x1
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src1 x1
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
RUN: Stage-1:MAPRED
128 1
146 1
150 1
213 1
224 1
238 1
255 1
273 1
278 1
311 1
369 1
401 1
406 1
66 1
98 1
PREHOOK: query: EXPLAIN
SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src1 x1
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: EXPLAIN
SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src1 x1
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Map 5 <- Map 1 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (BROADCAST_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: x1
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: key (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 16 Data size: 1376 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 16 Data size: 1376 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 5
Map Operator Tree:
TableScan
alias: x2
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Map 1
Statistics: Num rows: 39 Data size: 3393 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
minReductionHashAggr: 0.94871795
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 16 Data size: 1376 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Left Semi Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Map 5
Statistics: Num rows: 2 Data size: 172 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
keys: _col0 (type: string)
minReductionHashAggr: 0.5
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: bigint)
null sort order: zz
sort order: ++
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: bigint)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src1 x1
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: SELECT tmp1.key as key, count(*) as cnt
FROM (SELECT x1.key AS key
FROM src1 x1
GROUP BY x1.key) tmp1
JOIN (SELECT x2.key AS key
FROM src x2 JOIN src1 y2 ON (x2.key = y2.key)
GROUP BY x2.key) tmp2
ON (tmp1.key = tmp2.key)
GROUP BY tmp1.key
ORDER BY key, cnt
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
RUN: Stage-1:MAPRED
128 1
146 1
150 1
213 1
224 1
238 1
255 1
273 1
278 1
311 1
369 1
401 1
406 1
66 1
98 1
PREHOOK: query: CREATE TABLE part_table_n0(key string, value string) PARTITIONED BY (partitionId int)
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@part_table_n0
POSTHOOK: query: CREATE TABLE part_table_n0(key string, value string) PARTITIONED BY (partitionId int)
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@part_table_n0
RUN: Stage-0:DDL
PREHOOK: query: INSERT OVERWRITE TABLE part_table_n0 PARTITION (partitionId=1)
SELECT key, value FROM src ORDER BY key, value LIMIT 100
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Output: default@part_table_n0@partitionid=1
POSTHOOK: query: INSERT OVERWRITE TABLE part_table_n0 PARTITION (partitionId=1)
SELECT key, value FROM src ORDER BY key, value LIMIT 100
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Output: default@part_table_n0@partitionid=1
POSTHOOK: Lineage: part_table_n0 PARTITION(partitionid=1).key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
POSTHOOK: Lineage: part_table_n0 PARTITION(partitionid=1).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
RUN: Stage-1:MAPRED
RUN: Stage-2:DEPENDENCY_COLLECTION
RUN: Stage-0:MOVE
RUN: Stage-3:STATS
PREHOOK: query: INSERT OVERWRITE TABLE part_table_n0 PARTITION (partitionId=2)
SELECT key, value FROM src1 ORDER BY key, value
PREHOOK: type: QUERY
PREHOOK: Input: default@src1
PREHOOK: Output: default@part_table_n0@partitionid=2
POSTHOOK: query: INSERT OVERWRITE TABLE part_table_n0 PARTITION (partitionId=2)
SELECT key, value FROM src1 ORDER BY key, value
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src1
POSTHOOK: Output: default@part_table_n0@partitionid=2
POSTHOOK: Lineage: part_table_n0 PARTITION(partitionid=2).key SIMPLE [(src1)src1.FieldSchema(name:key, type:string, comment:default), ]
POSTHOOK: Lineage: part_table_n0 PARTITION(partitionid=2).value SIMPLE [(src1)src1.FieldSchema(name:value, type:string, comment:default), ]
RUN: Stage-1:MAPRED
RUN: Stage-2:DEPENDENCY_COLLECTION
RUN: Stage-0:MOVE
RUN: Stage-3:STATS
PREHOOK: query: EXPLAIN
SELECT count(*)
FROM part_table_n0 x JOIN src1 y ON (x.key = y.key)
PREHOOK: type: QUERY
PREHOOK: Input: default@part_table_n0
PREHOOK: Input: default@part_table_n0@partitionid=1
PREHOOK: Input: default@part_table_n0@partitionid=2
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: EXPLAIN
SELECT count(*)
FROM part_table_n0 x JOIN src1 y ON (x.key = y.key)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part_table_n0
POSTHOOK: Input: default@part_table_n0@partitionid=1
POSTHOOK: Input: default@part_table_n0@partitionid=2
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: x
filterExpr: key is not null (type: boolean)
probeDecodeDetails: cacheKey:HASH_MAP_MAPJOIN_29_container, bigKeyColName:key, smallTablePos:1, keyRatio:0.336
Statistics: Num rows: 125 Data size: 10875 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 125 Data size: 10875 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 125 Data size: 10875 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
input vertices:
1 Map 3
Statistics: Num rows: 42 Data size: 336 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
minReductionHashAggr: 0.97619045
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 3
Map Operator Tree:
TableScan
alias: y
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: SELECT count(*)
FROM part_table_n0 x JOIN src1 y ON (x.key = y.key)
PREHOOK: type: QUERY
PREHOOK: Input: default@part_table_n0
PREHOOK: Input: default@part_table_n0@partitionid=1
PREHOOK: Input: default@part_table_n0@partitionid=2
PREHOOK: Input: default@src1
#### A masked pattern was here ####
POSTHOOK: query: SELECT count(*)
FROM part_table_n0 x JOIN src1 y ON (x.key = y.key)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part_table_n0
POSTHOOK: Input: default@part_table_n0@partitionid=1
POSTHOOK: Input: default@part_table_n0@partitionid=2
POSTHOOK: Input: default@src1
#### A masked pattern was here ####
RUN: Stage-1:MAPRED
121
PREHOOK: query: EXPLAIN
SELECT x.key FROM (
SELECT c.key FROM
(SELECT a.key FROM src a JOIN src b ON a.key=b.key GROUP BY a.key) tmp
JOIN src c ON tmp.key=c.key
UNION ALL
SELECT c.key FROM
(SELECT a.key FROM src a JOIN src b ON a.key=b.key GROUP BY a.key) tmp
JOIN src c ON tmp.key=c.key
) x order by x.key
PREHOOK: type: QUERY
PREHOOK: Input: default@src
#### A masked pattern was here ####
POSTHOOK: query: EXPLAIN
SELECT x.key FROM (
SELECT c.key FROM
(SELECT a.key FROM src a JOIN src b ON a.key=b.key GROUP BY a.key) tmp
JOIN src c ON tmp.key=c.key
UNION ALL
SELECT c.key FROM
(SELECT a.key FROM src a JOIN src b ON a.key=b.key GROUP BY a.key) tmp
JOIN src c ON tmp.key=c.key
) x order by x.key
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Map 1 <- Map 7 (BROADCAST_EDGE), Union 2 (CONTAINS)
Map 5 <- Map 6 (BROADCAST_EDGE), Union 2 (CONTAINS)
Map 6 <- Map 7 (BROADCAST_EDGE)
Map 7 <- Map 4 (BROADCAST_EDGE)
Reducer 3 <- Union 2 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: c
filterExpr: key is not null (type: boolean)
probeDecodeDetails: cacheKey:HASH_MAP_MAPJOIN_106_container, bigKeyColName:key, smallTablePos:1, keyRatio:0.632
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Left Semi Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Map 7
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Statistics: Num rows: 1000 Data size: 87000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: b
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 5
Map Operator Tree:
TableScan
alias: c
filterExpr: key is not null (type: boolean)
probeDecodeDetails: cacheKey:HASH_MAP_MAPJOIN_108_container, bigKeyColName:key, smallTablePos:1, keyRatio:0.632
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Left Semi Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Map 6
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Statistics: Num rows: 1000 Data size: 87000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 6
Map Operator Tree:
TableScan
alias: a
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Map 7
Statistics: Num rows: 791 Data size: 68817 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
minReductionHashAggr: 0.6005057
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 316 Data size: 27492 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 316 Data size: 27492 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 7
Map Operator Tree:
TableScan
alias: b
filterExpr: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
input vertices:
1 Map 4
Statistics: Num rows: 791 Data size: 68817 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
minReductionHashAggr: 0.6005057
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 316 Data size: 27492 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 316 Data size: 27492 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 3
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
Statistics: Num rows: 1000 Data size: 87000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1000 Data size: 87000 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Union 2
Vertex: Union 2
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: SELECT x.key FROM (
SELECT c.key FROM
(SELECT a.key FROM src a JOIN src b ON a.key=b.key GROUP BY a.key) tmp
JOIN src c ON tmp.key=c.key
UNION ALL
SELECT c.key FROM
(SELECT a.key FROM src a JOIN src b ON a.key=b.key GROUP BY a.key) tmp
JOIN src c ON tmp.key=c.key
) x order by x.key
PREHOOK: type: QUERY
PREHOOK: Input: default@src
#### A masked pattern was here ####
POSTHOOK: query: SELECT x.key FROM (
SELECT c.key FROM
(SELECT a.key FROM src a JOIN src b ON a.key=b.key GROUP BY a.key) tmp
JOIN src c ON tmp.key=c.key
UNION ALL
SELECT c.key FROM
(SELECT a.key FROM src a JOIN src b ON a.key=b.key GROUP BY a.key) tmp
JOIN src c ON tmp.key=c.key
) x order by x.key
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
#### A masked pattern was here ####
RUN: Stage-1:MAPRED
0
0
0
0
0
0
10
10
100
100
100
100
103
103
103
103
104
104
104
104
105
105
11
11
111
111
113
113
113
113
114
114
116
116
118
118
118
118
119
119
119
119
119
119
12
12
12
12
120
120
120
120
125
125
125
125
126
126
128
128
128
128
128
128
129
129
129
129
131
131
133
133
134
134
134
134
136
136
137
137
137
137
138
138
138
138
138
138
138
138
143
143
145
145
146
146
146
146
149
149
149
149
15
15
15
15
150
150
152
152
152
152
153
153
155
155
156
156
157
157
158
158
160
160
162
162
163
163
164
164
164
164
165
165
165
165
166
166
167
167
167
167
167
167
168
168
169
169
169
169
169
169
169
169
17
17
170
170
172
172
172
172
174
174
174
174
175
175
175
175
176
176
176
176
177
177
178
178
179
179
179
179
18
18
18
18
180
180
181
181
183
183
186
186
187
187
187
187
187
187
189
189
19
19
190
190
191
191
191
191
192
192
193
193
193
193
193
193
194
194
195
195
195
195
196
196
197
197
197
197
199
199
199
199
199
199
2
2
20
20
200
200
200
200
201
201
202
202
203
203
203
203
205
205
205
205
207
207
207
207
208
208
208
208
208
208
209
209
209
209
213
213
213
213
214
214
216
216
216
216
217
217
217
217
218
218
219
219
219
219
221
221
221
221
222
222
223
223
223
223
224
224
224
224
226
226
228
228
229
229
229
229
230
230
230
230
230
230
230
230
230
230
233
233
233
233
235
235
237
237
237
237
238
238
238
238
239
239
239
239
24
24
24
24
241
241
242
242
242
242
244
244
247
247
248
248
249
249
252
252
255
255
255
255
256
256
256
256
257
257
258
258
26
26
26
26
260
260
262
262
263
263
265
265
265
265
266
266
27
27
272
272
272
272
273
273
273
273
273
273
274
274
275
275
277
277
277
277
277
277
277
277
278
278
278
278
28
28
280
280
280
280
281
281
281
281
282
282
282
282
283
283
284
284
285
285
286
286
287
287
288
288
288
288
289
289
291
291
292
292
296
296
298
298
298
298
298
298
30
30
302
302
305
305
306
306
307
307
307
307
308
308
309
309
309
309
310
310
311
311
311
311
311
311
315
315
316
316
316
316
316
316
317
317
317
317
318
318
318
318
318
318
321
321
321
321
322
322
322
322
323
323
325
325
325
325
327
327
327
327
327
327
33
33
331
331
331
331
332
332
333
333
333
333
335
335
336
336
338
338
339
339
34
34
341
341
342
342
342
342
344
344
344
344
345
345
348
348
348
348
348
348
348
348
348
348
35
35
35
35
35
35
351
351
353
353
353
353
356
356
360
360
362
362
364
364
365
365
366
366
367
367
367
367
368
368
369
369
369
369
369
369
37
37
37
37
373
373
374
374
375
375
377
377
378
378
379
379
382
382
382
382
384
384
384
384
384
384
386
386
389
389
392
392
393
393
394
394
395
395
395
395
396
396
396
396
396
396
397
397
397
397
399
399
399
399
4
4
400
400
401
401
401
401
401
401
401
401
401
401
402
402
403
403
403
403
403
403
404
404
404
404
406
406
406
406
406
406
406
406
407
407
409
409
409
409
409
409
41
41
411
411
413
413
413
413
414
414
414
414
417
417
417
417
417
417
418
418
419
419
42
42
42
42
421
421
424
424
424
424
427
427
429
429
429
429
43
43
430
430
430
430
430
430
431
431
431
431
431
431
432
432
435
435
436
436
437
437
438
438
438
438
438
438
439
439
439
439
44
44
443
443
444
444
446
446
448
448
449
449
452
452
453
453
454
454
454
454
454
454
455
455
457
457
458
458
458
458
459
459
459
459
460
460
462
462
462
462
463
463
463
463
466
466
466
466
466
466
467
467
468
468
468
468
468
468
468
468
469
469
469
469
469
469
469
469
469
469
47
47
470
470
472
472
475
475
477
477
478
478
478
478
479
479
480
480
480
480
480
480
481
481
482
482
483
483
484
484
485
485
487
487
489
489
489
489
489
489
489
489
490
490
491
491
492
492
492
492
493
493
494
494
495
495
496
496
497
497
498
498
498
498
498
498
5
5
5
5
5
5
51
51
51
51
53
53
54
54
57
57
58
58
58
58
64
64
65
65
66
66
67
67
67
67
69
69
70
70
70
70
70
70
72
72
72
72
74
74
76
76
76
76
77
77
78
78
8
8
80
80
82
82
83
83
83
83
84
84
84
84
85
85
86
86
87
87
9
9
90
90
90
90
90
90
92
92
95
95
95
95
96
96
97
97
97
97
98
98
98
98