blob: 8cd983d213bda59729bebae53603de955af7d710 [file] [log] [blame]
PREHOOK: query: CREATE TABLE agg1 (col0 INT, col1 STRING, col2 DOUBLE)
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@agg1
POSTHOOK: query: CREATE TABLE agg1 (col0 INT, col1 STRING, col2 DOUBLE)
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@agg1
PREHOOK: query: INSERT INTO TABLE agg1 select key,value,key from src tablesample (1 rows)
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Output: default@agg1
POSTHOOK: query: INSERT INTO TABLE agg1 select key,value,key from src tablesample (1 rows)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Output: default@agg1
POSTHOOK: Lineage: agg1.col0 EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
POSTHOOK: Lineage: agg1.col1 SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
POSTHOOK: Lineage: agg1.col2 EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
Warning: Shuffle Join MERGEJOIN[26][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: EXPLAIN
SELECT single_use_subq11.a1 AS a1,
single_use_subq11.a2 AS a2
FROM (SELECT Sum(agg1.col2) AS a1
FROM agg1
GROUP BY agg1.col0) single_use_subq12
JOIN (SELECT alias.a2 AS a0,
alias.a1 AS a1,
alias.a1 AS a2
FROM (SELECT agg1.col1 AS a0,
'42' AS a1,
agg1.col0 AS a2
FROM agg1
UNION ALL
SELECT agg1.col1 AS a0,
'41' AS a1,
agg1.col0 AS a2
FROM agg1) alias
GROUP BY alias.a2,
alias.a1) single_use_subq11
ON ( single_use_subq11.a0 = single_use_subq11.a0 )
PREHOOK: type: QUERY
PREHOOK: Input: default@agg1
#### A masked pattern was here ####
POSTHOOK: query: EXPLAIN
SELECT single_use_subq11.a1 AS a1,
single_use_subq11.a2 AS a2
FROM (SELECT Sum(agg1.col2) AS a1
FROM agg1
GROUP BY agg1.col0) single_use_subq12
JOIN (SELECT alias.a2 AS a0,
alias.a1 AS a1,
alias.a1 AS a2
FROM (SELECT agg1.col1 AS a0,
'42' AS a1,
agg1.col0 AS a2
FROM agg1
UNION ALL
SELECT agg1.col1 AS a0,
'41' AS a1,
agg1.col0 AS a2
FROM agg1) alias
GROUP BY alias.a2,
alias.a1) single_use_subq11
ON ( single_use_subq11.a0 = single_use_subq11.a0 )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@agg1
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Map 4 <- Union 5 (CONTAINS)
Map 7 <- Union 5 (CONTAINS)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (XPROD_EDGE), Reducer 6 (XPROD_EDGE)
Reducer 6 <- Union 5 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: agg1
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: col0 (type: int)
outputColumnNames: col0
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: col0 (type: int)
minReductionHashAggr: 0.0
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: agg1
filterExpr: col0 is not null (type: boolean)
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: col0 is not null (type: boolean)
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: '42' (type: string), col0 (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 90 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: int)
minReductionHashAggr: 0.5
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 90 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: int)
Statistics: Num rows: 1 Data size: 90 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 7
Map Operator Tree:
TableScan
alias: agg1
filterExpr: col0 is not null (type: boolean)
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: col0 is not null (type: boolean)
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: '41' (type: string), col0 (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 90 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: int)
minReductionHashAggr: 0.5
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 90 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: int)
Statistics: Num rows: 1 Data size: 90 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col1, _col2
Statistics: Num rows: 1 Data size: 172 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: string), _col2 (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 172 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 172 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 90 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col0 (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 172 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 172 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string)
Union 5
Vertex: Union 5
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[26][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: SELECT single_use_subq11.a1 AS a1,
single_use_subq11.a2 AS a2
FROM (SELECT Sum(agg1.col2) AS a1
FROM agg1
GROUP BY agg1.col0) single_use_subq12
JOIN (SELECT alias.a2 AS a0,
alias.a1 AS a1,
alias.a1 AS a2
FROM (SELECT agg1.col1 AS a0,
'42' AS a1,
agg1.col0 AS a2
FROM agg1
UNION ALL
SELECT agg1.col1 AS a0,
'41' AS a1,
agg1.col0 AS a2
FROM agg1) alias
GROUP BY alias.a2,
alias.a1) single_use_subq11
ON ( single_use_subq11.a0 = single_use_subq11.a0 )
PREHOOK: type: QUERY
PREHOOK: Input: default@agg1
#### A masked pattern was here ####
POSTHOOK: query: SELECT single_use_subq11.a1 AS a1,
single_use_subq11.a2 AS a2
FROM (SELECT Sum(agg1.col2) AS a1
FROM agg1
GROUP BY agg1.col0) single_use_subq12
JOIN (SELECT alias.a2 AS a0,
alias.a1 AS a1,
alias.a1 AS a2
FROM (SELECT agg1.col1 AS a0,
'42' AS a1,
agg1.col0 AS a2
FROM agg1
UNION ALL
SELECT agg1.col1 AS a0,
'41' AS a1,
agg1.col0 AS a2
FROM agg1) alias
GROUP BY alias.a2,
alias.a1) single_use_subq11
ON ( single_use_subq11.a0 = single_use_subq11.a0 )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@agg1
#### A masked pattern was here ####
42 42
41 41