Google Git
Sign in
apache / hive / 6b095edc31a9baa0ae805e2b5aea88101284d84b / . / ql / src / test / results / clientpositive / llap / subquery_notin.q.out
blob: e26868dbe62b179aab349771895f0b7cc09674f3 [file] [log] [blame]
Warning: Shuffle Join MERGEJOIN[35][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain
select *
from src
where src.key not in
( select key from src s1
where s1.key > '2'
)
PREHOOK: type: QUERY
PREHOOK: Input: default@src
#### A masked pattern was here ####
POSTHOOK: query: explain
select *
from src
where src.key not in
( select key from src s1
where s1.key > '2'
)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (XPROD_EDGE), Reducer 5 (XPROD_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: src
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Filter Operator
predicate: (key > '2') (type: boolean)
Statistics: Num rows: 166 Data size: 14442 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: key (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 105 Data size: 9135 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 105 Data size: 9135 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(key)
minReductionHashAggr: 0.99
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col3
Statistics: Num rows: 666 Data size: 119216 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 666 Data size: 119216 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string), _col3 (type: boolean)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col0, _col1, _col3, _col4, _col5
Statistics: Num rows: 666 Data size: 129872 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col4 (type: bigint), _col5 (type: bigint), _col3 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col5
Statistics: Num rows: 666 Data size: 129872 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((_col2 = 0L) or (_col5 is null and (_col3 >= _col2) and _col0 is not null)) (type: boolean)
Statistics: Num rows: 666 Data size: 129872 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 666 Data size: 118548 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 666 Data size: 118548 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 105 Data size: 9135 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), true (type: boolean)
outputColumnNames: _col0, _col1
Statistics: Num rows: 105 Data size: 9555 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 105 Data size: 9555 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[37][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select *
from src
where src.key not in ( select key from src s1 where s1.key > '2')
order by key
PREHOOK: type: QUERY
PREHOOK: Input: default@src
#### A masked pattern was here ####
POSTHOOK: query: select *
from src
where src.key not in ( select key from src s1 where s1.key > '2')
order by key
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
#### A masked pattern was here ####
0 val_0
0 val_0
0 val_0
10 val_10
100 val_100
100 val_100
103 val_103
103 val_103
104 val_104
104 val_104
105 val_105
11 val_11
111 val_111
113 val_113
113 val_113
114 val_114
116 val_116
118 val_118
118 val_118
119 val_119
119 val_119
119 val_119
12 val_12
12 val_12
120 val_120
120 val_120
125 val_125
125 val_125
126 val_126
128 val_128
128 val_128
128 val_128
129 val_129
129 val_129
131 val_131
133 val_133
134 val_134
134 val_134
136 val_136
137 val_137
137 val_137
138 val_138
138 val_138
138 val_138
138 val_138
143 val_143
145 val_145
146 val_146
146 val_146
149 val_149
149 val_149
15 val_15
15 val_15
150 val_150
152 val_152
152 val_152
153 val_153
155 val_155
156 val_156
157 val_157
158 val_158
160 val_160
162 val_162
163 val_163
164 val_164
164 val_164
165 val_165
165 val_165
166 val_166
167 val_167
167 val_167
167 val_167
168 val_168
169 val_169
169 val_169
169 val_169
169 val_169
17 val_17
170 val_170
172 val_172
172 val_172
174 val_174
174 val_174
175 val_175
175 val_175
176 val_176
176 val_176
177 val_177
178 val_178
179 val_179
179 val_179
18 val_18
18 val_18
180 val_180
181 val_181
183 val_183
186 val_186
187 val_187
187 val_187
187 val_187
189 val_189
19 val_19
190 val_190
191 val_191
191 val_191
192 val_192
193 val_193
193 val_193
193 val_193
194 val_194
195 val_195
195 val_195
196 val_196
197 val_197
197 val_197
199 val_199
199 val_199
199 val_199
2 val_2
PREHOOK: query: explain
select p_mfgr, b.p_name, p_size
from part b
where b.p_name not in
(select p_name
from (select p_mfgr, p_name, p_size, rank() over(partition by p_mfgr order by p_size) as r from part) a
where r <= 2 and b.p_mfgr = a.p_mfgr
)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain
select p_mfgr, b.p_name, p_size
from part b
where b.p_name not in
(select p_name
from (select p_mfgr, p_name, p_size, rank() over(partition by p_mfgr order by p_size) as r from part) a
where r <= 2 and b.p_mfgr = a.p_mfgr
)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Reducer 4 (SIMPLE_EDGE)
Reducer 6 <- Reducer 4 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: b
Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_name (type: string), p_mfgr (type: string), p_size (type: int)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: string)
Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col2 (type: int)
Filter Operator
predicate: p_mfgr is not null (type: boolean)
Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Top N Key Operator
sort order: ++
keys: p_mfgr (type: string), p_size (type: int)
null sort order: az
Map-reduce partition columns: p_mfgr (type: string)
Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
top n: 3
Reduce Output Operator
key expressions: p_mfgr (type: string), p_size (type: int)
null sort order: az
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_name (type: string)
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col4, _col5
Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col2 (type: int), _col4 (type: bigint), _col5 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: string), _col1 (type: string)
1 _col0 (type: string), _col2 (type: string)
outputColumnNames: _col0, _col1, _col2, _col4, _col5, _col7
Statistics: Num rows: 34 Data size: 8162 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col4 is null or (_col4 = 0L) or (_col7 is not null or _col0 is null or (_col5 < _col4)) is not true) (type: boolean)
Statistics: Num rows: 24 Data size: 5764 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: string), _col0 (type: string), _col2 (type: int)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 24 Data size: 5352 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 24 Data size: 5352 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col1 (type: string), KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: int)
outputColumnNames: _col1, _col2, _col5
Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
input alias: ptf_0
output shape: _col1: string, _col2: string, _col5: int
type: WINDOWING
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
order by: _col5 ASC NULLS LAST
partition by: _col2
raw input shape:
window functions:
window function definition
alias: rank_window_0
arguments: _col5
name: rank
window function: GenericUDAFRankEvaluator
window frame: ROWS PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (rank_window_0 <= 2) (type: boolean)
Statistics: Num rows: 8 Data size: 1784 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: string), _col2 (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 8 Data size: 1784 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col0)
keys: _col1 (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 5 Data size: 570 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 5 Data size: 570 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
PTF Operator
Function definitions:
Input definition
input alias: ptf_0
output shape: _col1: string, _col2: string, _col5: int
type: WINDOWING
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
order by: _col5 ASC NULLS LAST
partition by: _col2
raw input shape:
window functions:
window function definition
alias: rank_window_0
arguments: _col5
name: rank
window function: GenericUDAFRankEvaluator
window frame: ROWS PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((rank_window_0 <= 2) and _col1 is not null) (type: boolean)
Statistics: Num rows: 8 Data size: 1784 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: string), _col2 (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 8 Data size: 1784 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 8 Data size: 1752 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
Statistics: Num rows: 8 Data size: 1752 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 5 Data size: 570 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 5 Data size: 570 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 8 Data size: 1752 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), true (type: boolean), _col1 (type: string)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 8 Data size: 1784 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col2 (type: string)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col2 (type: string)
Statistics: Num rows: 8 Data size: 1784 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select p_mfgr, b.p_name, p_size
from part b
where b.p_name not in
(select p_name
from (select p_mfgr, p_name, p_size, rank() over(partition by p_mfgr order by p_size) as r from part) a
where r <= 2 and b.p_mfgr = a.p_mfgr
)
order by p_mfgr, b.p_name
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select p_mfgr, b.p_name, p_size
from part b
where b.p_name not in
(select p_name
from (select p_mfgr, p_name, p_size, rank() over(partition by p_mfgr order by p_size) as r from part) a
where r <= 2 and b.p_mfgr = a.p_mfgr
)
order by p_mfgr, b.p_name
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
Manufacturer#1 almond antique chartreuse lavender yellow 34
Manufacturer#1 almond antique salmon chartreuse burlywood 6
Manufacturer#1 almond aquamarine burnished black steel 28
Manufacturer#1 almond aquamarine pink moccasin thistle 42
Manufacturer#2 almond antique violet turquoise frosted 40
Manufacturer#2 almond aquamarine rose maroon antique 25
Manufacturer#2 almond aquamarine sandy cyan gainsboro 18
Manufacturer#3 almond antique chartreuse khaki white 17
Manufacturer#3 almond antique metallic orange dim 19
Manufacturer#3 almond antique olive coral navajo 45
Manufacturer#4 almond antique violet mint lemon 39
Manufacturer#4 almond aquamarine floral ivory bisque 27
Manufacturer#4 almond azure aquamarine papaya violet 12
Manufacturer#5 almond antique blue firebrick mint 31
Manufacturer#5 almond aquamarine dodger light gainsboro 46
Manufacturer#5 almond azure blanched chiffon midnight 23
Warning: Shuffle Join MERGEJOIN[50][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain
select p_name, p_size
from
part where part.p_size not in
(select avg(p_size)
from (select p_size, rank() over(partition by p_mfgr order by p_size) as r from part) a
where r <= 2
)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain
select p_name, p_size
from
part where part.p_size not in
(select avg(p_size)
from (select p_size, rank() over(partition by p_mfgr order by p_size) as r from part) a
where r <= 2
)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (XPROD_EDGE), Reducer 6 (XPROD_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Reducer 4 (CUSTOM_SIMPLE_EDGE)
Reducer 6 <- Reducer 4 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: part
Statistics: Num rows: 26 Data size: 3250 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_name (type: string), p_size (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 26 Data size: 3250 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToDouble(_col1) (type: double)
null sort order: z
sort order: +
Map-reduce partition columns: UDFToDouble(_col1) (type: double)
Statistics: Num rows: 26 Data size: 3250 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: int)
Top N Key Operator
sort order: ++
keys: p_mfgr (type: string), p_size (type: int)
null sort order: az
Map-reduce partition columns: p_mfgr (type: string)
Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
top n: 3
Reduce Output Operator
key expressions: p_mfgr (type: string), p_size (type: int)
null sort order: az
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 UDFToDouble(_col1) (type: double)
1 _col0 (type: double)
outputColumnNames: _col0, _col1, _col3
Statistics: Num rows: 27 Data size: 3383 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 27 Data size: 3383 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: int), _col3 (type: boolean)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col0, _col1, _col3, _col4, _col5
Statistics: Num rows: 27 Data size: 3815 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: int), _col4 (type: bigint), _col5 (type: bigint), _col3 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col5
Statistics: Num rows: 27 Data size: 3815 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((_col2 = 0L) or (_col5 is null and (_col3 >= _col2) and _col1 is not null)) (type: boolean)
Statistics: Num rows: 27 Data size: 3815 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 27 Data size: 3375 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 27 Data size: 3375 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: int)
outputColumnNames: _col2, _col5
Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
input alias: ptf_0
output shape: _col2: string, _col5: int
type: WINDOWING
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
order by: _col5 ASC NULLS LAST
partition by: _col2
raw input shape:
window functions:
window function definition
alias: rank_window_0
arguments: _col5
name: rank
window function: GenericUDAFRankEvaluator
window frame: ROWS PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (rank_window_0 <= 2) (type: boolean)
Statistics: Num rows: 8 Data size: 816 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col5 (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 816 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col0), count(_col0)
minReductionHashAggr: 0.875
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
PTF Operator
Function definitions:
Input definition
input alias: ptf_0
output shape: _col2: string, _col5: int
type: WINDOWING
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
order by: _col5 ASC NULLS LAST
partition by: _col2
raw input shape:
window functions:
window function definition
alias: rank_window_0
arguments: _col5
name: rank
window function: GenericUDAFRankEvaluator
window frame: ROWS PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (rank_window_0 <= 2) (type: boolean)
Statistics: Num rows: 8 Data size: 816 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col5 (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 816 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col0), count(_col0)
minReductionHashAggr: 0.875
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), count(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col1 is not null and UDFToDouble(_col0) is not null) (type: boolean)
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: (UDFToDouble(_col0) / _col1) (type: double), true (type: boolean)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: double)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: double)
Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), count(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: (UDFToDouble(_col0) / _col1) (type: double)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col0)
mode: complete
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[52][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select p_name, p_size
from
part where part.p_size not in
(select avg(p_size)
from (select p_size, rank() over(partition by p_mfgr order by p_size) as r from part) a
where r <= 2
)
order by p_name, p_size
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select p_name, p_size
from
part where part.p_size not in
(select avg(p_size)
from (select p_size, rank() over(partition by p_mfgr order by p_size) as r from part) a
where r <= 2
)
order by p_name, p_size
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
almond antique blue firebrick mint 31
almond antique burnished rose metallic 2
almond antique burnished rose metallic 2
almond antique chartreuse khaki white 17
almond antique chartreuse lavender yellow 34
almond antique forest lavender goldenrod 14
almond antique gainsboro frosted violet 10
almond antique metallic orange dim 19
almond antique misty red olive 1
almond antique olive coral navajo 45
almond antique sky peru orange 2
almond antique violet chocolate turquoise 14
almond antique violet mint lemon 39
almond antique violet turquoise frosted 40
almond aquamarine burnished black steel 28
almond aquamarine dodger light gainsboro 46
almond aquamarine floral ivory bisque 27
almond aquamarine midnight light salmon 2
almond aquamarine pink moccasin thistle 42
almond aquamarine rose maroon antique 25
almond aquamarine sandy cyan gainsboro 18
almond aquamarine yellow dodger mint 7
almond azure aquamarine papaya violet 12
almond azure blanched chiffon midnight 23
PREHOOK: query: explain
select p_mfgr, p_name, p_size
from part b where b.p_size not in
(select min(p_size)
from (select p_mfgr, p_size, rank() over(partition by p_mfgr order by p_size) as r from part) a
where r <= 2 and b.p_mfgr = a.p_mfgr
)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain
select p_mfgr, p_name, p_size
from part b where b.p_size not in
(select min(p_size)
from (select p_mfgr, p_size, rank() over(partition by p_mfgr order by p_size) as r from part) a
where r <= 2 and b.p_mfgr = a.p_mfgr
)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 7 (SIMPLE_EDGE)
Reducer 4 <- Reducer 3 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
Reducer 7 <- Reducer 5 (SIMPLE_EDGE)
Reducer 8 <- Reducer 5 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: b
Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_name (type: string), p_mfgr (type: string), p_size (type: int)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: string)
Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col2 (type: int)
Filter Operator
predicate: p_mfgr is not null (type: boolean)
Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
Top N Key Operator
sort order: ++
keys: p_mfgr (type: string), p_size (type: int)
null sort order: az
Map-reduce partition columns: p_mfgr (type: string)
Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
top n: 3
Reduce Output Operator
key expressions: p_mfgr (type: string), p_size (type: int)
null sort order: az
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col4
Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: if(_col4 is null, sq_count_check(0L, true), sq_count_check(_col4, true)) (type: boolean)
Statistics: Num rows: 13 Data size: 3003 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col2 (type: int)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 13 Data size: 2899 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: string)
Statistics: Num rows: 13 Data size: 2899 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col2 (type: int)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col4, _col5
Statistics: Num rows: 13 Data size: 3107 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string), _col2 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col1 (type: string), _col2 (type: int)
Statistics: Num rows: 13 Data size: 3107 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col4 (type: bigint), _col5 (type: bigint)
Reducer 4
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: string), _col2 (type: int)
1 _col2 (type: string), _col0 (type: int)
outputColumnNames: _col0, _col1, _col2, _col4, _col5, _col7
Statistics: Num rows: 18 Data size: 4326 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col4 is null or (_col4 = 0L) or (_col7 is not null or _col2 is null or (_col5 < _col4)) is not true) (type: boolean)
Statistics: Num rows: 13 Data size: 3127 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: string), _col0 (type: string), _col2 (type: int)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 13 Data size: 2899 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 13 Data size: 2899 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: int)
outputColumnNames: _col2, _col5
Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
input alias: ptf_0
output shape: _col2: string, _col5: int
type: WINDOWING
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
order by: _col5 ASC NULLS LAST
partition by: _col2
raw input shape:
window functions:
window function definition
alias: rank_window_0
arguments: _col5
name: rank
window function: GenericUDAFRankEvaluator
window frame: ROWS PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (rank_window_0 <= 2) (type: boolean)
Statistics: Num rows: 8 Data size: 816 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 816 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 5 Data size: 490 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 5 Data size: 490 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
input alias: ptf_0
output shape: _col2: string, _col5: int
type: WINDOWING
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
order by: _col5 ASC NULLS LAST
partition by: _col2
raw input shape:
window functions:
window function definition
alias: rank_window_0
arguments: _col5
name: rank
window function: GenericUDAFRankEvaluator
window frame: ROWS PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (rank_window_0 <= 2) (type: boolean)
Statistics: Num rows: 8 Data size: 816 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col5 (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 8 Data size: 816 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(_col1)
keys: _col0 (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 5 Data size: 510 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 5 Data size: 510 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: int)
PTF Operator
Function definitions:
Input definition
input alias: ptf_0
output shape: _col2: string, _col5: int
type: WINDOWING
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
order by: _col5 ASC NULLS LAST
partition by: _col2
raw input shape:
window functions:
window function definition
alias: rank_window_0
arguments: _col5
name: rank
window function: GenericUDAFRankEvaluator
window frame: ROWS PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (rank_window_0 <= 2) (type: boolean)
Statistics: Num rows: 8 Data size: 816 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col5 (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 8 Data size: 816 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(_col1)
keys: _col0 (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 5 Data size: 510 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 5 Data size: 510 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: int)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 5 Data size: 490 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
keys: _col0 (type: string)
mode: complete
outputColumnNames: _col0, _col1
Statistics: Num rows: 5 Data size: 530 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 5 Data size: 530 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 7
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: min(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 5 Data size: 510 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col1)
keys: _col0 (type: string)
mode: complete
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 5 Data size: 570 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 5 Data size: 570 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 8
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: min(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 5 Data size: 510 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col1 is not null (type: boolean)
Statistics: Num rows: 5 Data size: 510 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: int), true (type: boolean), _col0 (type: string)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 5 Data size: 530 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col0 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col2 (type: string), _col0 (type: int)
Statistics: Num rows: 5 Data size: 530 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select p_mfgr, p_name, p_size
from part b where b.p_size not in
(select min(p_size)
from (select p_mfgr, p_size, rank() over(partition by p_mfgr order by p_size) as r from part) a
where r <= 2 and b.p_mfgr = a.p_mfgr
)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select p_mfgr, p_name, p_size
from part b where b.p_size not in
(select min(p_size)
from (select p_mfgr, p_size, rank() over(partition by p_mfgr order by p_size) as r from part) a
where r <= 2 and b.p_mfgr = a.p_mfgr
)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
Manufacturer#1 almond antique chartreuse lavender yellow 34
Manufacturer#1 almond antique salmon chartreuse burlywood 6
Manufacturer#1 almond aquamarine burnished black steel 28
Manufacturer#1 almond aquamarine pink moccasin thistle 42
Manufacturer#2 almond antique violet chocolate turquoise 14
Manufacturer#2 almond antique violet turquoise frosted 40
Manufacturer#2 almond aquamarine rose maroon antique 25
Manufacturer#2 almond aquamarine sandy cyan gainsboro 18
Manufacturer#3 almond antique chartreuse khaki white 17
Manufacturer#3 almond antique forest lavender goldenrod 14
Manufacturer#3 almond antique metallic orange dim 19
Manufacturer#3 almond antique olive coral navajo 45
Manufacturer#4 almond antique gainsboro frosted violet 10
Manufacturer#4 almond antique violet mint lemon 39
Manufacturer#4 almond aquamarine floral ivory bisque 27
Manufacturer#4 almond azure aquamarine papaya violet 12
Manufacturer#5 almond antique blue firebrick mint 31
Manufacturer#5 almond antique medium spring khaki 6
Manufacturer#5 almond aquamarine dodger light gainsboro 46
Manufacturer#5 almond azure blanched chiffon midnight 23
Warning: Shuffle Join MERGEJOIN[41][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 4' is a cross product
PREHOOK: query: select li.l_partkey, count(*)
from lineitem li
where li.l_linenumber = 1 and
li.l_orderkey not in (select l_orderkey from lineitem where l_shipmode = 'AIR')
group by li.l_partkey
PREHOOK: type: QUERY
PREHOOK: Input: default@lineitem
#### A masked pattern was here ####
POSTHOOK: query: select li.l_partkey, count(*)
from lineitem li
where li.l_linenumber = 1 and
li.l_orderkey not in (select l_orderkey from lineitem where l_shipmode = 'AIR')
group by li.l_partkey
POSTHOOK: type: QUERY
POSTHOOK: Input: default@lineitem
#### A masked pattern was here ####
106170 1
119477 1
119767 1
123076 1
139636 1
175839 1
182052 1
21636 1
22630 1
450 1
59694 1
61931 1
7068 1
85951 1
88035 1
88362 1
Warning: Shuffle Join MERGEJOIN[37][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select *
from src
where not src.key in ( select key from src s1 where s1.key > '2')
order by key
PREHOOK: type: QUERY
PREHOOK: Input: default@src
#### A masked pattern was here ####
POSTHOOK: query: select *
from src
where not src.key in ( select key from src s1 where s1.key > '2')
order by key
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
#### A masked pattern was here ####
0 val_0
0 val_0
0 val_0
10 val_10
100 val_100
100 val_100
103 val_103
103 val_103
104 val_104
104 val_104
105 val_105
11 val_11
111 val_111
113 val_113
113 val_113
114 val_114
116 val_116
118 val_118
118 val_118
119 val_119
119 val_119
119 val_119
12 val_12
12 val_12
120 val_120
120 val_120
125 val_125
125 val_125
126 val_126
128 val_128
128 val_128
128 val_128
129 val_129
129 val_129
131 val_131
133 val_133
134 val_134
134 val_134
136 val_136
137 val_137
137 val_137
138 val_138
138 val_138
138 val_138
138 val_138
143 val_143
145 val_145
146 val_146
146 val_146
149 val_149
149 val_149
15 val_15
15 val_15
150 val_150
152 val_152
152 val_152
153 val_153
155 val_155
156 val_156
157 val_157
158 val_158
160 val_160
162 val_162
163 val_163
164 val_164
164 val_164
165 val_165
165 val_165
166 val_166
167 val_167
167 val_167
167 val_167
168 val_168
169 val_169
169 val_169
169 val_169
169 val_169
17 val_17
170 val_170
172 val_172
172 val_172
174 val_174
174 val_174
175 val_175
175 val_175
176 val_176
176 val_176
177 val_177
178 val_178
179 val_179
179 val_179
18 val_18
18 val_18
180 val_180
181 val_181
183 val_183
186 val_186
187 val_187
187 val_187
187 val_187
189 val_189
19 val_19
190 val_190
191 val_191
191 val_191
192 val_192
193 val_193
193 val_193
193 val_193
194 val_194
195 val_195
195 val_195
196 val_196
197 val_197
197 val_197
199 val_199
199 val_199
199 val_199
2 val_2
PREHOOK: query: create view T1_v as
select key from src where key <'11'
PREHOOK: type: CREATEVIEW
PREHOOK: Input: default@src
PREHOOK: Output: database:default
PREHOOK: Output: default@T1_v
POSTHOOK: query: create view T1_v as
select key from src where key <'11'
POSTHOOK: type: CREATEVIEW
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@T1_v
POSTHOOK: Lineage: t1_v.key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
PREHOOK: query: create view T2_v as
select case when key > '104' then null else key end as key from T1_v
PREHOOK: type: CREATEVIEW
PREHOOK: Input: default@src
PREHOOK: Input: default@t1_v
PREHOOK: Output: database:default
PREHOOK: Output: default@T2_v
POSTHOOK: query: create view T2_v as
select case when key > '104' then null else key end as key from T1_v
POSTHOOK: type: CREATEVIEW
POSTHOOK: Input: default@src
POSTHOOK: Input: default@t1_v
POSTHOOK: Output: database:default
POSTHOOK: Output: default@T2_v
POSTHOOK: Lineage: t2_v.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
Warning: Shuffle Join MERGEJOIN[34][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain
select *
from T1_v where T1_v.key not in (select T2_v.key from T2_v)
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@t1_v
PREHOOK: Input: default@t2_v
#### A masked pattern was here ####
POSTHOOK: query: explain
select *
from T1_v where T1_v.key not in (select T2_v.key from T2_v)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@t1_v
POSTHOOK: Input: default@t2_v
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (XPROD_EDGE), Reducer 4 (XPROD_EDGE)
Reducer 4 <- Map 1 (CUSTOM_SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: src
filterExpr: ((key < '11') or ((key > '104') is not true and (key < '11'))) (type: boolean)
properties:
insideView TRUE
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < '11') (type: boolean)
Statistics: Num rows: 166 Data size: 14442 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
Statistics: Num rows: 166 Data size: 14442 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 166 Data size: 14442 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: if((key > '104'), null, key) (type: string)
outputColumnNames: _col0
Statistics: Num rows: 166 Data size: 14442 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col0)
minReductionHashAggr: 0.99
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Filter Operator
predicate: ((key > '104') is not true and (key < '11')) (type: boolean)
Statistics: Num rows: 83 Data size: 7221 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: if((key > '104'), null, key) (type: string)
outputColumnNames: _col0
Statistics: Num rows: 83 Data size: 7221 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 53 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 53 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col2
Statistics: Num rows: 249 Data size: 21999 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 249 Data size: 21999 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col2 (type: boolean)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col0, _col2, _col3, _col4
Statistics: Num rows: 249 Data size: 25983 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col3 (type: bigint), _col4 (type: bigint), _col2 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col4
Statistics: Num rows: 249 Data size: 25983 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((_col1 = 0L) or (_col4 is null and (_col2 >= _col1) and _col0 is not null)) (type: boolean)
Statistics: Num rows: 249 Data size: 25983 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
Statistics: Num rows: 249 Data size: 21663 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 249 Data size: 21663 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 53 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), true (type: boolean)
outputColumnNames: _col0, _col1
Statistics: Num rows: 53 Data size: 299 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 53 Data size: 299 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[34][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select *
from T1_v where T1_v.key not in (select T2_v.key from T2_v)
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Input: default@t1_v
PREHOOK: Input: default@t2_v
#### A masked pattern was here ####
POSTHOOK: query: select *
from T1_v where T1_v.key not in (select T2_v.key from T2_v)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Input: default@t1_v
POSTHOOK: Input: default@t2_v
#### A masked pattern was here ####
PREHOOK: query: explain select * from part where p_brand <> 'Brand#14' AND p_size NOT IN (select (p_size*p_size) from part p where p.p_type = part.p_type ) AND p_size <> 340
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain select * from part where p_brand <> 'Brand#14' AND p_size NOT IN (select (p_size*p_size) from part p where p.p_type = part.p_type ) AND p_size <> 340
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: part
filterExpr: (((p_size <> 340) and (p_brand <> 'Brand#14')) or p_type is not null or (((p_size * p_size) <> 340) and p_type is not null and p_size is not null)) (type: boolean)
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((p_size <> 340) and (p_brand <> 'Brand#14')) (type: boolean)
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_partkey (type: int), p_name (type: string), p_mfgr (type: string), p_brand (type: string), p_type (type: string), p_size (type: int), p_container (type: string), p_retailprice (type: double), p_comment (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col4 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col4 (type: string)
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Filter Operator
predicate: p_type is not null (type: boolean)
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_type (type: string), (p_size * p_size) (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col1)
keys: _col0 (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 24 Data size: 2880 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 24 Data size: 2880 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Filter Operator
predicate: (((p_size * p_size) <> 340) and p_type is not null and p_size is not null) (type: boolean)
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: (p_size * p_size) (type: int), p_type (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col1 (type: string), _col0 (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: int)
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col4 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10, _col11
Statistics: Num rows: 26 Data size: 16510 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col4 (type: string), _col5 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col4 (type: string), _col5 (type: int)
Statistics: Num rows: 26 Data size: 16510 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col10 (type: bigint), _col11 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col4 (type: string), _col5 (type: int)
1 _col2 (type: string), _col0 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10, _col11, _col13
Statistics: Num rows: 26 Data size: 16614 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col10 is null or (_col10 = 0L) or (_col13 is not null or _col5 is null or (_col11 < _col10)) is not true) (type: boolean)
Statistics: Num rows: 14 Data size: 8946 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 14 Data size: 8666 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 14 Data size: 8666 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 24 Data size: 2880 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 24 Data size: 2880 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: int), true (type: boolean), _col0 (type: string)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 24 Data size: 2688 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col0 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col2 (type: string), _col0 (type: int)
Statistics: Num rows: 24 Data size: 2688 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select * from part where p_brand <> 'Brand#14' AND p_size NOT IN (select (p_size*p_size) from part p where p.p_type = part.p_type ) AND p_size <> 340
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select * from part where p_brand <> 'Brand#14' AND p_size NOT IN (select (p_size*p_size) from part p where p.p_type = part.p_type ) AND p_size <> 340
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
105685 almond antique violet chocolate turquoise Manufacturer#2 Brand#22 MEDIUM ANODIZED COPPER 14 MED CAN 1690.68 ly pending requ
110592 almond antique salmon chartreuse burlywood Manufacturer#1 Brand#15 PROMO BURNISHED NICKEL 6 JUMBO PKG 1602.59 to the furiously
112398 almond antique metallic orange dim Manufacturer#3 Brand#32 MEDIUM BURNISHED BRASS 19 JUMBO JAR 1410.39 ole car
132666 almond aquamarine rose maroon antique Manufacturer#2 Brand#24 SMALL POLISHED NICKEL 25 MED BOX 1698.66 even
144293 almond antique olive coral navajo Manufacturer#3 Brand#34 STANDARD POLISHED STEEL 45 JUMBO CAN 1337.29 ag furiously about
146985 almond aquamarine midnight light salmon Manufacturer#2 Brand#23 MEDIUM BURNISHED COPPER 2 SM CASE 2031.98 s cajole caref
15103 almond aquamarine dodger light gainsboro Manufacturer#5 Brand#53 ECONOMY BURNISHED STEEL 46 LG PACK 1018.1 packages hinder carefu
155733 almond antique sky peru orange Manufacturer#5 Brand#53 SMALL PLATED BRASS 2 WRAP DRUM 1788.73 furiously. bra
17273 almond antique forest lavender goldenrod Manufacturer#3 Brand#35 PROMO ANODIZED TIN 14 JUMBO CASE 1190.27 along the
17927 almond aquamarine yellow dodger mint Manufacturer#4 Brand#41 ECONOMY BRUSHED COPPER 7 SM PKG 1844.92 ites. eve
191709 almond antique violet turquoise frosted Manufacturer#2 Brand#22 ECONOMY POLISHED STEEL 40 MED BOX 1800.7 haggle
192697 almond antique blue firebrick mint Manufacturer#5 Brand#52 MEDIUM BURNISHED TIN 31 LG DRUM 1789.69 ickly ir
195606 almond aquamarine sandy cyan gainsboro Manufacturer#2 Brand#25 STANDARD PLATED TIN 18 SM PKG 1701.6 ic de
33357 almond azure aquamarine papaya violet Manufacturer#4 Brand#41 STANDARD ANODIZED TIN 12 WRAP CASE 1290.35 reful
42669 almond antique medium spring khaki Manufacturer#5 Brand#51 STANDARD BURNISHED TIN 6 MED CAN 1611.66 sits haggl
45261 almond aquamarine floral ivory bisque Manufacturer#4 Brand#42 SMALL PLATED STEEL 27 WRAP CASE 1206.26 careful
48427 almond antique violet mint lemon Manufacturer#4 Brand#42 PROMO POLISHED STEEL 39 SM CASE 1375.42 hely ironic i
49671 almond antique gainsboro frosted violet Manufacturer#4 Brand#41 SMALL BRUSHED BRASS 10 SM BOX 1620.67 ccounts run quick
65667 almond aquamarine pink moccasin thistle Manufacturer#1 Brand#12 LARGE BURNISHED STEEL 42 JUMBO CASE 1632.66 e across the expr
78486 almond azure blanched chiffon midnight Manufacturer#5 Brand#52 LARGE BRUSHED BRASS 23 MED BAG 1464.48 hely blith
85768 almond antique chartreuse lavender yellow Manufacturer#1 Brand#12 LARGE BRUSHED STEEL 34 SM BAG 1753.76 refull
86428 almond aquamarine burnished black steel Manufacturer#1 Brand#12 STANDARD ANODIZED STEEL 28 WRAP BAG 1414.42 arefully
90681 almond antique chartreuse khaki white Manufacturer#3 Brand#31 MEDIUM BURNISHED TIN 17 SM CASE 1671.68 are slyly after the sl
Warning: Shuffle Join MERGEJOIN[41][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select * from part where (p_size-1) NOT IN (select min(p_size) from part group by p_type) order by p_partkey
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain select * from part where (p_size-1) NOT IN (select min(p_size) from part group by p_type) order by p_partkey
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (XPROD_EDGE), Reducer 7 (XPROD_EDGE)
Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
Reducer 7 <- Reducer 5 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: part
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_partkey (type: int), p_name (type: string), p_mfgr (type: string), p_brand (type: string), p_type (type: string), p_size (type: int), p_container (type: string), p_retailprice (type: double), p_comment (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: (_col5 - 1) (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: (_col5 - 1) (type: int)
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Select Operator
expressions: p_type (type: string), p_size (type: int)
outputColumnNames: p_type, p_size
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(p_size)
keys: p_type (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: int)
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 (_col5 - 1) (type: int)
1 _col0 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10
Statistics: Num rows: 29 Data size: 18059 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 29 Data size: 18059 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col10 (type: boolean)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10, _col11, _col12
Statistics: Num rows: 29 Data size: 18523 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col11 (type: bigint), _col12 (type: bigint), _col10 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col12
Statistics: Num rows: 29 Data size: 18523 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((_col9 = 0L) or (_col12 is null and (_col10 >= _col9) and _col5 is not null)) (type: boolean)
Statistics: Num rows: 29 Data size: 18523 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: int), VALUE._col0 (type: string), VALUE._col1 (type: string), VALUE._col2 (type: string), VALUE._col3 (type: string), VALUE._col4 (type: int), VALUE._col5 (type: string), VALUE._col6 (type: double), VALUE._col7 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: min(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: int)
outputColumnNames: _col1
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col1 is not null (type: boolean)
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col1 (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col1)
minReductionHashAggr: 0.9583333
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), true (type: boolean)
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 24 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Reducer 7
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[41][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from part where (p_size-1) NOT IN (select min(p_size) from part group by p_type) order by p_partkey
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select * from part where (p_size-1) NOT IN (select min(p_size) from part group by p_type) order by p_partkey
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
105685 almond antique violet chocolate turquoise Manufacturer#2 Brand#22 MEDIUM ANODIZED COPPER 14 MED CAN 1690.68 ly pending requ
110592 almond antique salmon chartreuse burlywood Manufacturer#1 Brand#15 PROMO BURNISHED NICKEL 6 JUMBO PKG 1602.59 to the furiously
132666 almond aquamarine rose maroon antique Manufacturer#2 Brand#24 SMALL POLISHED NICKEL 25 MED BOX 1698.66 even
144293 almond antique olive coral navajo Manufacturer#3 Brand#34 STANDARD POLISHED STEEL 45 JUMBO CAN 1337.29 ag furiously about
17273 almond antique forest lavender goldenrod Manufacturer#3 Brand#35 PROMO ANODIZED TIN 14 JUMBO CASE 1190.27 along the
192697 almond antique blue firebrick mint Manufacturer#5 Brand#52 MEDIUM BURNISHED TIN 31 LG DRUM 1789.69 ickly ir
33357 almond azure aquamarine papaya violet Manufacturer#4 Brand#41 STANDARD ANODIZED TIN 12 WRAP CASE 1290.35 reful
40982 almond antique misty red olive Manufacturer#3 Brand#32 ECONOMY PLATED COPPER 1 LG PKG 1922.98 c foxes can s
42669 almond antique medium spring khaki Manufacturer#5 Brand#51 STANDARD BURNISHED TIN 6 MED CAN 1611.66 sits haggl
45261 almond aquamarine floral ivory bisque Manufacturer#4 Brand#42 SMALL PLATED STEEL 27 WRAP CASE 1206.26 careful
48427 almond antique violet mint lemon Manufacturer#4 Brand#42 PROMO POLISHED STEEL 39 SM CASE 1375.42 hely ironic i
49671 almond antique gainsboro frosted violet Manufacturer#4 Brand#41 SMALL BRUSHED BRASS 10 SM BOX 1620.67 ccounts run quick
65667 almond aquamarine pink moccasin thistle Manufacturer#1 Brand#12 LARGE BURNISHED STEEL 42 JUMBO CASE 1632.66 e across the expr
78486 almond azure blanched chiffon midnight Manufacturer#5 Brand#52 LARGE BRUSHED BRASS 23 MED BAG 1464.48 hely blith
85768 almond antique chartreuse lavender yellow Manufacturer#1 Brand#12 LARGE BRUSHED STEEL 34 SM BAG 1753.76 refull
90681 almond antique chartreuse khaki white Manufacturer#3 Brand#31 MEDIUM BURNISHED TIN 17 SM CASE 1671.68 are slyly after the sl
Warning: Shuffle Join MERGEJOIN[39][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select * from part where (p_partkey*p_size) NOT IN (select min(p_partkey) from part group by p_type)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain select * from part where (p_partkey*p_size) NOT IN (select min(p_partkey) from part group by p_type)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (XPROD_EDGE), Reducer 6 (XPROD_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Reducer 4 (SIMPLE_EDGE)
Reducer 6 <- Reducer 4 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: part
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_partkey (type: int), p_name (type: string), p_mfgr (type: string), p_brand (type: string), p_type (type: string), p_size (type: int), p_container (type: string), p_retailprice (type: double), p_comment (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: (_col0 * _col5) (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: (_col0 * _col5) (type: int)
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Select Operator
expressions: p_partkey (type: int), p_type (type: string)
outputColumnNames: p_partkey, p_type
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(p_partkey)
keys: p_type (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: int)
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 (_col0 * _col5) (type: int)
1 _col0 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10
Statistics: Num rows: 26 Data size: 16194 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 26 Data size: 16194 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col10 (type: boolean)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10, _col11, _col12
Statistics: Num rows: 26 Data size: 16610 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col11 (type: bigint), _col12 (type: bigint), _col10 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col12
Statistics: Num rows: 26 Data size: 16610 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((_col9 = 0L) or (_col12 is null and (_col10 >= _col9) and _col0 is not null and _col5 is not null)) (type: boolean)
Statistics: Num rows: 26 Data size: 16610 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: min(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: int)
outputColumnNames: _col1
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col1 is not null (type: boolean)
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col1 (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col1)
minReductionHashAggr: 0.9583333
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), true (type: boolean)
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 24 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[39][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from part where (p_partkey*p_size) NOT IN (select min(p_partkey) from part group by p_type)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select * from part where (p_partkey*p_size) NOT IN (select min(p_partkey) from part group by p_type)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
105685 almond antique violet chocolate turquoise Manufacturer#2 Brand#22 MEDIUM ANODIZED COPPER 14 MED CAN 1690.68 ly pending requ
110592 almond antique salmon chartreuse burlywood Manufacturer#1 Brand#15 PROMO BURNISHED NICKEL 6 JUMBO PKG 1602.59 to the furiously
112398 almond antique metallic orange dim Manufacturer#3 Brand#32 MEDIUM BURNISHED BRASS 19 JUMBO JAR 1410.39 ole car
121152 almond antique burnished rose metallic Manufacturer#1 Brand#14 PROMO PLATED TIN 2 JUMBO BOX 1173.15 e pinto beans h
121152 almond antique burnished rose metallic Manufacturer#1 Brand#14 PROMO PLATED TIN 2 JUMBO BOX 1173.15 e pinto beans h
132666 almond aquamarine rose maroon antique Manufacturer#2 Brand#24 SMALL POLISHED NICKEL 25 MED BOX 1698.66 even
144293 almond antique olive coral navajo Manufacturer#3 Brand#34 STANDARD POLISHED STEEL 45 JUMBO CAN 1337.29 ag furiously about
146985 almond aquamarine midnight light salmon Manufacturer#2 Brand#23 MEDIUM BURNISHED COPPER 2 SM CASE 2031.98 s cajole caref
15103 almond aquamarine dodger light gainsboro Manufacturer#5 Brand#53 ECONOMY BURNISHED STEEL 46 LG PACK 1018.1 packages hinder carefu
155733 almond antique sky peru orange Manufacturer#5 Brand#53 SMALL PLATED BRASS 2 WRAP DRUM 1788.73 furiously. bra
17273 almond antique forest lavender goldenrod Manufacturer#3 Brand#35 PROMO ANODIZED TIN 14 JUMBO CASE 1190.27 along the
17927 almond aquamarine yellow dodger mint Manufacturer#4 Brand#41 ECONOMY BRUSHED COPPER 7 SM PKG 1844.92 ites. eve
191709 almond antique violet turquoise frosted Manufacturer#2 Brand#22 ECONOMY POLISHED STEEL 40 MED BOX 1800.7 haggle
192697 almond antique blue firebrick mint Manufacturer#5 Brand#52 MEDIUM BURNISHED TIN 31 LG DRUM 1789.69 ickly ir
195606 almond aquamarine sandy cyan gainsboro Manufacturer#2 Brand#25 STANDARD PLATED TIN 18 SM PKG 1701.6 ic de
33357 almond azure aquamarine papaya violet Manufacturer#4 Brand#41 STANDARD ANODIZED TIN 12 WRAP CASE 1290.35 reful
42669 almond antique medium spring khaki Manufacturer#5 Brand#51 STANDARD BURNISHED TIN 6 MED CAN 1611.66 sits haggl
45261 almond aquamarine floral ivory bisque Manufacturer#4 Brand#42 SMALL PLATED STEEL 27 WRAP CASE 1206.26 careful
48427 almond antique violet mint lemon Manufacturer#4 Brand#42 PROMO POLISHED STEEL 39 SM CASE 1375.42 hely ironic i
49671 almond antique gainsboro frosted violet Manufacturer#4 Brand#41 SMALL BRUSHED BRASS 10 SM BOX 1620.67 ccounts run quick
65667 almond aquamarine pink moccasin thistle Manufacturer#1 Brand#12 LARGE BURNISHED STEEL 42 JUMBO CASE 1632.66 e across the expr
78486 almond azure blanched chiffon midnight Manufacturer#5 Brand#52 LARGE BRUSHED BRASS 23 MED BAG 1464.48 hely blith
85768 almond antique chartreuse lavender yellow Manufacturer#1 Brand#12 LARGE BRUSHED STEEL 34 SM BAG 1753.76 refull
86428 almond aquamarine burnished black steel Manufacturer#1 Brand#12 STANDARD ANODIZED STEEL 28 WRAP BAG 1414.42 arefully
90681 almond antique chartreuse khaki white Manufacturer#3 Brand#31 MEDIUM BURNISHED TIN 17 SM CASE 1671.68 are slyly after the sl
PREHOOK: query: explain cbo select count(*) as c from part as e where p_size + 100 NOT IN (select p_partkey from part where p_name = e.p_name)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain cbo select count(*) as c from part as e where p_size + 100 NOT IN (select p_partkey from part where p_name = e.p_name)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
CBO PLAN:
HiveAggregate(group=[{}], agg#0=[count()])
HiveFilter(condition=[OR(IS NULL($2), =($2, 0), IS NOT TRUE(OR(IS NOT NULL($6), IS NULL($1), <($3, $2))))])
HiveProject(p_name=[$0], p_size=[$1], c=[$4], ck=[$5], $f15=[$2], p_partkey=[$6], literalTrue=[$7], p_name0=[$8])
HiveJoin(condition=[AND(=($8, $0), =($2, $6))], joinType=[left], algorithm=[none], cost=[not available])
HiveJoin(condition=[=($3, $0)], joinType=[left], algorithm=[none], cost=[not available])
HiveProject(p_name=[$1], p_size=[$5], +=[+($5, 100)])
HiveTableScan(table=[[default, part]], table:alias=[e])
HiveProject(p_name=[$0], c=[$1], ck=[$2])
HiveAggregate(group=[{1}], c=[COUNT()], ck=[COUNT($0)])
HiveFilter(condition=[IS NOT NULL($1)])
HiveTableScan(table=[[default, part]], table:alias=[part])
HiveProject(p_partkey=[$0], literalTrue=[true], p_name=[$1])
HiveAggregate(group=[{0, 1}])
HiveFilter(condition=[AND(IS NOT NULL($1), IS NOT NULL($0))])
HiveTableScan(table=[[default, part]], table:alias=[part])
PREHOOK: query: explain select count(*) as c from part as e where p_size + 100 NOT IN (select p_partkey from part where p_name = e.p_name)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain select count(*) as c from part as e where p_size + 100 NOT IN (select p_partkey from part where p_name = e.p_name)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 4 <- Reducer 3 (CUSTOM_SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
Reducer 6 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: e
Statistics: Num rows: 26 Data size: 3250 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_name (type: string), p_size (type: int), (p_size + 100) (type: int)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 26 Data size: 3354 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 26 Data size: 3354 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: int), _col2 (type: int)
Filter Operator
predicate: p_name is not null (type: boolean)
Statistics: Num rows: 26 Data size: 3250 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(p_partkey)
keys: p_name (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 25 Data size: 3425 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 3425 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Filter Operator
predicate: (p_name is not null and p_partkey is not null) (type: boolean)
Statistics: Num rows: 26 Data size: 3250 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: p_name (type: string), p_partkey (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 25 Data size: 3125 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: int)
Statistics: Num rows: 25 Data size: 3125 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col4, _col5
Statistics: Num rows: 26 Data size: 3770 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col2 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col2 (type: int)
Statistics: Num rows: 26 Data size: 3770 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: int), _col4 (type: bigint), _col5 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: string), _col2 (type: int)
1 _col2 (type: string), _col0 (type: int)
outputColumnNames: _col1, _col4, _col5, _col7
Statistics: Num rows: 26 Data size: 624 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: int), _col4 (type: bigint), _col5 (type: bigint), _col7 (type: boolean)
outputColumnNames: _col1, _col2, _col3, _col6
Statistics: Num rows: 26 Data size: 624 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col2 is null or (_col2 = 0L) or (_col6 is not null or _col1 is null or (_col3 < _col2)) is not true) (type: boolean)
Statistics: Num rows: 14 Data size: 336 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
Statistics: Num rows: 14 Data size: 336 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
minReductionHashAggr: 0.9285714
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 25 Data size: 3425 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 3425 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 25 Data size: 3125 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: int), true (type: boolean), _col0 (type: string)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 25 Data size: 3225 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col0 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col2 (type: string), _col0 (type: int)
Statistics: Num rows: 25 Data size: 3225 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select count(*) as c from part as e where p_size + 100 NOT IN (select p_partkey from part where p_name = e.p_name)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select count(*) as c from part as e where p_size + 100 NOT IN (select p_partkey from part where p_name = e.p_name)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
26
Warning: Shuffle Join MERGEJOIN[39][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select * from part where floor(p_retailprice) NOT IN (select floor(min(p_retailprice)) from part group by p_type)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain select * from part where floor(p_retailprice) NOT IN (select floor(min(p_retailprice)) from part group by p_type)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (XPROD_EDGE), Reducer 6 (XPROD_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Reducer 4 (SIMPLE_EDGE)
Reducer 6 <- Reducer 4 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: part
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_partkey (type: int), p_name (type: string), p_mfgr (type: string), p_brand (type: string), p_type (type: string), p_size (type: int), p_container (type: string), p_retailprice (type: double), p_comment (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: floor(_col7) (type: bigint)
null sort order: z
sort order: +
Map-reduce partition columns: floor(_col7) (type: bigint)
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Select Operator
expressions: p_type (type: string), p_retailprice (type: double)
outputColumnNames: p_type, p_retailprice
Statistics: Num rows: 26 Data size: 2912 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(p_retailprice)
keys: p_type (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2688 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 24 Data size: 2688 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: double)
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 floor(_col7) (type: bigint)
1 _col0 (type: bigint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10
Statistics: Num rows: 26 Data size: 16198 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 26 Data size: 16198 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col10 (type: boolean)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10, _col11, _col12
Statistics: Num rows: 26 Data size: 16614 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col11 (type: bigint), _col12 (type: bigint), _col10 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col12
Statistics: Num rows: 26 Data size: 16614 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((_col9 = 0L) or (_col12 is null and (_col10 >= _col9) and _col7 is not null)) (type: boolean)
Statistics: Num rows: 26 Data size: 16614 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: min(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2688 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: double)
outputColumnNames: _col1
Statistics: Num rows: 24 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col1 is not null (type: boolean)
Statistics: Num rows: 24 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: floor(_col1) (type: bigint)
outputColumnNames: _col0
Statistics: Num rows: 24 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: bigint)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 24 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: bigint)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: bigint)
Statistics: Num rows: 24 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: floor(_col1) (type: bigint)
outputColumnNames: _col0
Statistics: Num rows: 24 Data size: 2688 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col0)
minReductionHashAggr: 0.9583333
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: bigint)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 24 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: bigint), true (type: boolean)
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 288 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: bigint)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: bigint)
Statistics: Num rows: 24 Data size: 288 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[39][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from part where floor(p_retailprice) NOT IN (select floor(min(p_retailprice)) from part group by p_type)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select * from part where floor(p_retailprice) NOT IN (select floor(min(p_retailprice)) from part group by p_type)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
192697 almond antique blue firebrick mint Manufacturer#5 Brand#52 MEDIUM BURNISHED TIN 31 LG DRUM 1789.69 ickly ir
PREHOOK: query: explain cbo select * from part where p_name NOT IN (select p_name from part p where p.p_size = part.p_size AND part.p_size + 121150 = p.p_partkey )
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain cbo select * from part where p_name NOT IN (select p_name from part p where p.p_size = part.p_size AND part.p_size + 121150 = p.p_partkey )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
CBO PLAN:
HiveProject(p_partkey=[$0], p_name=[$1], p_mfgr=[$2], p_brand=[$3], p_type=[$4], p_size=[$5], p_container=[$6], p_retailprice=[$7], p_comment=[$8])
HiveFilter(condition=[OR(IS NULL($9), =($9, 0), IS NOT TRUE(OR(IS NOT NULL($13), IS NULL($1), <($10, $9))))])
HiveProject(p_partkey=[$0], p_name=[$1], p_mfgr=[$2], p_brand=[$3], p_type=[$4], p_size=[$5], p_container=[$6], p_retailprice=[$7], p_comment=[$8], c=[$13], ck=[$14], $f17=[$10], p_name0=[$15], literalTrue=[$16], p_size0=[$17], p_partkey0=[$18])
HiveJoin(condition=[AND(=($1, $15), =($17, $5), =($10, $18))], joinType=[left], algorithm=[none], cost=[not available])
HiveJoin(condition=[AND(=($11, $5), =($9, $12))], joinType=[left], algorithm=[none], cost=[not available])
HiveProject(p_partkey=[$0], p_name=[$1], p_mfgr=[$2], p_brand=[$3], p_type=[$4], p_size=[$5], p_container=[$6], p_retailprice=[$7], p_comment=[$8], $f14=[+($5, 121150)], +=[+($5, 121150)])
HiveTableScan(table=[[default, part]], table:alias=[part])
HiveProject(p_size=[$1], p_partkey=[$0], c=[$2], ck=[$3])
HiveAggregate(group=[{0, 5}], c=[COUNT()], ck=[COUNT($1)])
HiveFilter(condition=[AND(IS NOT NULL($5), IS NOT NULL($0))])
HiveTableScan(table=[[default, part]], table:alias=[p])
HiveProject(p_name=[$1], literalTrue=[true], p_size=[$2], p_partkey=[$0])
HiveAggregate(group=[{0, 1, 5}])
HiveFilter(condition=[AND(IS NOT NULL($5), IS NOT NULL($0), IS NOT NULL($1))])
HiveTableScan(table=[[default, part]], table:alias=[p])
PREHOOK: query: explain select * from part where p_name NOT IN (select p_name from part p where p.p_size = part.p_size AND part.p_size + 121150 = p.p_partkey )
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain select * from part where p_name NOT IN (select p_name from part p where p.p_size = part.p_size AND part.p_size + 121150 = p.p_partkey )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: part
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_partkey (type: int), p_name (type: string), p_mfgr (type: string), p_brand (type: string), p_type (type: string), p_size (type: int), p_container (type: string), p_retailprice (type: double), p_comment (type: string), (p_size + 121150) (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
Statistics: Num rows: 26 Data size: 16198 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col5 (type: int), _col9 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col5 (type: int), _col9 (type: int)
Statistics: Num rows: 26 Data size: 16198 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Filter Operator
predicate: (p_size is not null and p_partkey is not null) (type: boolean)
Statistics: Num rows: 26 Data size: 3354 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(p_name)
keys: p_size (type: int), p_partkey (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 25 Data size: 600 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
Statistics: Num rows: 25 Data size: 600 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col2 (type: bigint), _col3 (type: bigint)
Filter Operator
predicate: (p_size is not null and p_partkey is not null and p_name is not null) (type: boolean)
Statistics: Num rows: 26 Data size: 3354 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: p_name (type: string), p_size (type: int), p_partkey (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 25 Data size: 3225 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: int), _col2 (type: int)
null sort order: zzz
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col1 (type: int), _col2 (type: int)
Statistics: Num rows: 25 Data size: 3225 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col5 (type: int), _col9 (type: int)
1 _col0 (type: int), _col1 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10, _col13, _col14
Statistics: Num rows: 52 Data size: 32828 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string), _col5 (type: int), _col10 (type: int)
null sort order: zzz
sort order: +++
Map-reduce partition columns: _col1 (type: string), _col5 (type: int), _col10 (type: int)
Statistics: Num rows: 52 Data size: 32828 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col13 (type: bigint), _col14 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: string), _col5 (type: int), _col10 (type: int)
1 _col0 (type: string), _col2 (type: int), _col3 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col13, _col14, _col16
Statistics: Num rows: 52 Data size: 32828 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col13 (type: bigint), _col14 (type: bigint), _col16 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col13
Statistics: Num rows: 52 Data size: 32828 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col9 is null or (_col9 = 0L) or (_col13 is not null or _col1 is null or (_col10 < _col9)) is not true) (type: boolean)
Statistics: Num rows: 52 Data size: 32828 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 52 Data size: 32188 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 52 Data size: 32188 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: int), KEY._col1 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 25 Data size: 600 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
Statistics: Num rows: 25 Data size: 600 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col2 (type: bigint), _col3 (type: bigint)
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: int), KEY._col2 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 25 Data size: 3225 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), true (type: boolean), _col1 (type: int), _col2 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 25 Data size: 3325 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col2 (type: int), _col3 (type: int)
null sort order: zzz
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col2 (type: int), _col3 (type: int)
Statistics: Num rows: 25 Data size: 3325 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select * from part where p_name NOT IN (select p_name from part p where p.p_size = part.p_size AND part.p_size + 121150 = p.p_partkey )
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select * from part where p_name NOT IN (select p_name from part p where p.p_size = part.p_size AND part.p_size + 121150 = p.p_partkey )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
105685 almond antique violet chocolate turquoise Manufacturer#2 Brand#22 MEDIUM ANODIZED COPPER 14 MED CAN 1690.68 ly pending requ
110592 almond antique salmon chartreuse burlywood Manufacturer#1 Brand#15 PROMO BURNISHED NICKEL 6 JUMBO PKG 1602.59 to the furiously
112398 almond antique metallic orange dim Manufacturer#3 Brand#32 MEDIUM BURNISHED BRASS 19 JUMBO JAR 1410.39 ole car
132666 almond aquamarine rose maroon antique Manufacturer#2 Brand#24 SMALL POLISHED NICKEL 25 MED BOX 1698.66 even
144293 almond antique olive coral navajo Manufacturer#3 Brand#34 STANDARD POLISHED STEEL 45 JUMBO CAN 1337.29 ag furiously about
146985 almond aquamarine midnight light salmon Manufacturer#2 Brand#23 MEDIUM BURNISHED COPPER 2 SM CASE 2031.98 s cajole caref
15103 almond aquamarine dodger light gainsboro Manufacturer#5 Brand#53 ECONOMY BURNISHED STEEL 46 LG PACK 1018.1 packages hinder carefu
155733 almond antique sky peru orange Manufacturer#5 Brand#53 SMALL PLATED BRASS 2 WRAP DRUM 1788.73 furiously. bra
17273 almond antique forest lavender goldenrod Manufacturer#3 Brand#35 PROMO ANODIZED TIN 14 JUMBO CASE 1190.27 along the
17927 almond aquamarine yellow dodger mint Manufacturer#4 Brand#41 ECONOMY BRUSHED COPPER 7 SM PKG 1844.92 ites. eve
191709 almond antique violet turquoise frosted Manufacturer#2 Brand#22 ECONOMY POLISHED STEEL 40 MED BOX 1800.7 haggle
192697 almond antique blue firebrick mint Manufacturer#5 Brand#52 MEDIUM BURNISHED TIN 31 LG DRUM 1789.69 ickly ir
195606 almond aquamarine sandy cyan gainsboro Manufacturer#2 Brand#25 STANDARD PLATED TIN 18 SM PKG 1701.6 ic de
33357 almond azure aquamarine papaya violet Manufacturer#4 Brand#41 STANDARD ANODIZED TIN 12 WRAP CASE 1290.35 reful
40982 almond antique misty red olive Manufacturer#3 Brand#32 ECONOMY PLATED COPPER 1 LG PKG 1922.98 c foxes can s
42669 almond antique medium spring khaki Manufacturer#5 Brand#51 STANDARD BURNISHED TIN 6 MED CAN 1611.66 sits haggl
45261 almond aquamarine floral ivory bisque Manufacturer#4 Brand#42 SMALL PLATED STEEL 27 WRAP CASE 1206.26 careful
48427 almond antique violet mint lemon Manufacturer#4 Brand#42 PROMO POLISHED STEEL 39 SM CASE 1375.42 hely ironic i
49671 almond antique gainsboro frosted violet Manufacturer#4 Brand#41 SMALL BRUSHED BRASS 10 SM BOX 1620.67 ccounts run quick
65667 almond aquamarine pink moccasin thistle Manufacturer#1 Brand#12 LARGE BURNISHED STEEL 42 JUMBO CASE 1632.66 e across the expr
78486 almond azure blanched chiffon midnight Manufacturer#5 Brand#52 LARGE BRUSHED BRASS 23 MED BAG 1464.48 hely blith
85768 almond antique chartreuse lavender yellow Manufacturer#1 Brand#12 LARGE BRUSHED STEEL 34 SM BAG 1753.76 refull
86428 almond aquamarine burnished black steel Manufacturer#1 Brand#12 STANDARD ANODIZED STEEL 28 WRAP BAG 1414.42 arefully
90681 almond antique chartreuse khaki white Manufacturer#3 Brand#31 MEDIUM BURNISHED TIN 17 SM CASE 1671.68 are slyly after the sl
PREHOOK: query: explain select * from part where p_name NOT IN (select p_name from part p where p.p_size = part.p_size AND part.p_partkey= p.p_partkey )
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain select * from part where p_name NOT IN (select p_name from part p where p.p_size = part.p_size AND part.p_partkey= p.p_partkey )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: part
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_partkey (type: int), p_name (type: string), p_mfgr (type: string), p_brand (type: string), p_type (type: string), p_size (type: int), p_container (type: string), p_retailprice (type: double), p_comment (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col5 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col5 (type: int)
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Filter Operator
predicate: (p_size is not null and p_partkey is not null) (type: boolean)
Statistics: Num rows: 26 Data size: 3354 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(p_name)
keys: p_partkey (type: int), p_size (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 25 Data size: 600 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
Statistics: Num rows: 25 Data size: 600 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col2 (type: bigint), _col3 (type: bigint)
Filter Operator
predicate: (p_size is not null and p_partkey is not null and p_name is not null) (type: boolean)
Statistics: Num rows: 26 Data size: 3354 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: p_partkey (type: int), p_size (type: int), p_name (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 25 Data size: 3225 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: string)
null sort order: zzz
sort order: +++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int), _col2 (type: string)
Statistics: Num rows: 25 Data size: 3225 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: int), _col5 (type: int)
1 _col0 (type: int), _col1 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col11, _col12
Statistics: Num rows: 26 Data size: 16510 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col5 (type: int), _col1 (type: string)
null sort order: zzz
sort order: +++
Map-reduce partition columns: _col0 (type: int), _col5 (type: int), _col1 (type: string)
Statistics: Num rows: 26 Data size: 16510 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col2 (type: string), _col3 (type: string), _col4 (type: string), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col11 (type: bigint), _col12 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: int), _col5 (type: int), _col1 (type: string)
1 _col2 (type: int), _col3 (type: int), _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col11, _col12, _col14
Statistics: Num rows: 26 Data size: 16614 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col11 is null or (_col11 = 0L) or (_col14 is not null or _col1 is null or (_col12 < _col11)) is not true) (type: boolean)
Statistics: Num rows: 14 Data size: 8946 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 14 Data size: 8666 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 14 Data size: 8666 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: int), KEY._col1 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 25 Data size: 600 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
Statistics: Num rows: 25 Data size: 600 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col2 (type: bigint), _col3 (type: bigint)
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int), KEY._col1 (type: int), KEY._col2 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 25 Data size: 3225 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), true (type: boolean), _col0 (type: int), _col1 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 25 Data size: 3325 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: int), _col3 (type: int), _col0 (type: string)
null sort order: zzz
sort order: +++
Map-reduce partition columns: _col2 (type: int), _col3 (type: int), _col0 (type: string)
Statistics: Num rows: 25 Data size: 3325 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select * from part where p_name NOT IN (select p_name from part p where p.p_size = part.p_size AND part.p_partkey= p.p_partkey )
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select * from part where p_name NOT IN (select p_name from part p where p.p_size = part.p_size AND part.p_partkey= p.p_partkey )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
PREHOOK: query: explain select p_name from (select p_name, p_type, p_brand as brand from part) fpart where fpart.p_type NOT IN (select p_type+2 from part where part.p_brand = fpart.brand)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain select p_name from (select p_name, p_type, p_brand as brand from part) fpart where fpart.p_type NOT IN (select p_type+2 from part where part.p_brand = fpart.brand)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: part
Statistics: Num rows: 26 Data size: 8242 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_name (type: string), p_type (type: string), p_brand (type: string)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 26 Data size: 8242 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col2 (type: string)
Statistics: Num rows: 26 Data size: 8242 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string)
Filter Operator
predicate: p_brand is not null (type: boolean)
Statistics: Num rows: 26 Data size: 5096 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_brand (type: string), (UDFToDouble(p_type) + 2.0D) (type: double)
outputColumnNames: _col0, _col1
Statistics: Num rows: 26 Data size: 5096 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col1)
keys: _col0 (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 16 Data size: 1728 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 16 Data size: 1728 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Filter Operator
predicate: (p_brand is not null and UDFToDouble(p_type) is not null) (type: boolean)
Statistics: Num rows: 26 Data size: 5096 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: (UDFToDouble(p_type) + 2.0D) (type: double), p_brand (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 26 Data size: 5096 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: double), _col1 (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2400 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: double), _col1 (type: string)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: double), _col1 (type: string)
Statistics: Num rows: 24 Data size: 2400 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col2 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col4, _col5
Statistics: Num rows: 26 Data size: 8658 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToDouble(_col1) (type: double), _col2 (type: string)
null sort order: zz
sort order: ++
Map-reduce partition columns: UDFToDouble(_col1) (type: double), _col2 (type: string)
Statistics: Num rows: 26 Data size: 8658 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string), _col4 (type: bigint), _col5 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 UDFToDouble(_col1) (type: double), _col2 (type: string)
1 _col0 (type: double), _col2 (type: string)
outputColumnNames: _col0, _col1, _col4, _col5, _col7
Statistics: Num rows: 26 Data size: 6370 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col4 is null or (_col4 = 0L) or (_col7 is not null or _col1 is null or (_col5 < _col4)) is not true) (type: boolean)
Statistics: Num rows: 15 Data size: 3675 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
Statistics: Num rows: 15 Data size: 1815 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 15 Data size: 1815 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 16 Data size: 1728 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 16 Data size: 1728 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: double), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2400 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: double), true (type: boolean), _col1 (type: string)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 24 Data size: 2496 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: double), _col2 (type: string)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: double), _col2 (type: string)
Statistics: Num rows: 24 Data size: 2496 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select p_name from (select p_name, p_type, p_brand as brand from part) fpart where fpart.p_type NOT IN (select p_type+2 from part where part.p_brand = fpart.brand)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select p_name from (select p_name, p_type, p_brand as brand from part) fpart where fpart.p_type NOT IN (select p_type+2 from part where part.p_brand = fpart.brand)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
PREHOOK: query: explain select p_name from (select p_name, p_type, p_size+1 as size from part) fpart where fpart.p_type NOT IN (select p_type from part where (part.p_size+1) = fpart.size)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain select p_name from (select p_name, p_type, p_size+1 as size from part) fpart where fpart.p_type NOT IN (select p_type from part where (part.p_size+1) = fpart.size)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: part
Statistics: Num rows: 26 Data size: 5954 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_name (type: string), p_type (type: string), (p_size + 1) (type: int)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 26 Data size: 5954 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col2 (type: int)
Statistics: Num rows: 26 Data size: 5954 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string)
Filter Operator
predicate: p_size is not null (type: boolean)
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: (p_size + 1) (type: int), p_type (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col1)
keys: _col0 (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 21 Data size: 420 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 21 Data size: 420 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Filter Operator
predicate: (p_size is not null and p_type is not null) (type: boolean)
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_type (type: string), (p_size + 1) (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: int)
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col2 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0, _col1, _col2, _col4, _col5
Statistics: Num rows: 26 Data size: 6370 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string), _col2 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col1 (type: string), _col2 (type: int)
Statistics: Num rows: 26 Data size: 6370 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col4 (type: bigint), _col5 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: string), _col2 (type: int)
1 _col0 (type: string), _col2 (type: int)
outputColumnNames: _col0, _col1, _col4, _col5, _col7
Statistics: Num rows: 26 Data size: 6370 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col4 is null or (_col4 = 0L) or (_col7 is not null or _col1 is null or (_col5 < _col4)) is not true) (type: boolean)
Statistics: Num rows: 14 Data size: 3430 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
Statistics: Num rows: 14 Data size: 1694 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 14 Data size: 1694 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 21 Data size: 420 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 21 Data size: 420 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), true (type: boolean), _col1 (type: int)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 24 Data size: 2688 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col2 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col2 (type: int)
Statistics: Num rows: 24 Data size: 2688 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select p_name from (select p_name, p_type, p_size+1 as size from part) fpart where fpart.p_type NOT IN (select p_type from part where (part.p_size+1) = fpart.size+1)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select p_name from (select p_name, p_type, p_size+1 as size from part) fpart where fpart.p_type NOT IN (select p_type from part where (part.p_size+1) = fpart.size+1)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
almond antique blue firebrick mint
almond antique burnished rose metallic
almond antique burnished rose metallic
almond antique chartreuse khaki white
almond antique chartreuse lavender yellow
almond antique forest lavender goldenrod
almond antique gainsboro frosted violet
almond antique medium spring khaki
almond antique metallic orange dim
almond antique misty red olive
almond antique olive coral navajo
almond antique salmon chartreuse burlywood
almond antique sky peru orange
almond antique violet chocolate turquoise
almond antique violet mint lemon
almond antique violet turquoise frosted
almond aquamarine burnished black steel
almond aquamarine dodger light gainsboro
almond aquamarine floral ivory bisque
almond aquamarine midnight light salmon
almond aquamarine pink moccasin thistle
almond aquamarine rose maroon antique
almond aquamarine sandy cyan gainsboro
almond aquamarine yellow dodger mint
almond azure aquamarine papaya violet
almond azure blanched chiffon midnight
Warning: Shuffle Join MERGEJOIN[61][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select key, count(*) from src where value NOT IN (select key from src) group by key having count(*) in (select count(*) from src s1 where s1.key = '90' group by s1.key )
PREHOOK: type: QUERY
PREHOOK: Input: default@src
#### A masked pattern was here ####
POSTHOOK: query: explain select key, count(*) from src where value NOT IN (select key from src) group by key having count(*) in (select count(*) from src s1 where s1.key = '90' group by s1.key )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (XPROD_EDGE), Reducer 7 (XPROD_EDGE)
Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
Reducer 5 <- Reducer 4 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 6 <- Map 1 (SIMPLE_EDGE)
Reducer 7 <- Map 1 (CUSTOM_SIMPLE_EDGE)
Reducer 8 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: src
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: string)
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Filter Operator
predicate: (key = '90') (type: boolean)
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
keys: true (type: boolean)
minReductionHashAggr: 0.5
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: boolean)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: boolean)
Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Select Operator
expressions: key (type: string)
outputColumnNames: key
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(key)
minReductionHashAggr: 0.99
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: key (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 316 Data size: 27492 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 316 Data size: 27492 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col3
Statistics: Num rows: 1000 Data size: 180004 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1000 Data size: 180004 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string), _col3 (type: boolean)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col0, _col1, _col3, _col4, _col5
Statistics: Num rows: 1000 Data size: 196004 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col4 (type: bigint), _col5 (type: bigint), _col3 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col5
Statistics: Num rows: 1000 Data size: 196004 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((_col2 = 0L) or (_col5 is null and (_col3 >= _col2) and _col1 is not null)) (type: boolean)
Statistics: Num rows: 1000 Data size: 196004 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
Statistics: Num rows: 1000 Data size: 196004 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
keys: _col0 (type: string)
minReductionHashAggr: 0.684
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 316 Data size: 30020 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 316 Data size: 30020 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 316 Data size: 30020 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col1 is not null (type: boolean)
Statistics: Num rows: 316 Data size: 30020 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: bigint)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: bigint)
Statistics: Num rows: 316 Data size: 30020 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Reducer 5
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Semi Join 0 to 1
keys:
0 _col1 (type: bigint)
1 _col0 (type: bigint)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 95 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 95 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: boolean)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: bigint)
outputColumnNames: _col1
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col1 is not null (type: boolean)
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: bigint)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: bigint)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: bigint)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: bigint)
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 7
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 8
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 316 Data size: 27492 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), true (type: boolean)
outputColumnNames: _col0, _col1
Statistics: Num rows: 316 Data size: 28756 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 316 Data size: 28756 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[61][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select key, count(*) from src where value NOT IN (select key from src) group by key having count(*) in (select count(*) from src s1 where s1.key = '90' group by s1.key )
PREHOOK: type: QUERY
PREHOOK: Input: default@src
#### A masked pattern was here ####
POSTHOOK: query: select key, count(*) from src where value NOT IN (select key from src) group by key having count(*) in (select count(*) from src s1 where s1.key = '90' group by s1.key )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
#### A masked pattern was here ####
0 3
119 3
128 3
167 3
187 3
193 3
199 3
208 3
273 3
298 3
311 3
316 3
318 3
327 3
35 3
369 3
384 3
396 3
403 3
409 3
417 3
430 3
431 3
438 3
454 3
466 3
480 3
498 3
5 3
70 3
90 3
PREHOOK: query: explain select key, count(*) from src where value NOT IN (select concat('v', value) from src sc where sc.key = src.key ) group by key having count(*) in (select count(*) from src s1 where s1.key = '90' group by s1.key )
PREHOOK: type: QUERY
PREHOOK: Input: default@src
#### A masked pattern was here ####
POSTHOOK: query: explain select key, count(*) from src where value NOT IN (select concat('v', value) from src sc where sc.key = src.key ) group by key having count(*) in (select count(*) from src s1 where s1.key = '90' group by s1.key )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 7 (SIMPLE_EDGE)
Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
Reducer 5 <- Reducer 4 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
Reducer 6 <- Map 1 (SIMPLE_EDGE)
Reducer 7 <- Map 1 (SIMPLE_EDGE)
Reducer 8 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: src
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), concat('v', value) (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col1)
keys: _col0 (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 316 Data size: 32548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 316 Data size: 32548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Filter Operator
predicate: (key is not null and concat('v', value) is not null) (type: boolean)
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: concat('v', value) (type: string), key (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col1 (type: string), _col0 (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 316 Data size: 85636 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
Statistics: Num rows: 316 Data size: 85636 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key = '90') (type: boolean)
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
keys: true (type: boolean)
minReductionHashAggr: 0.5
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: boolean)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: boolean)
Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col3, _col4
Statistics: Num rows: 500 Data size: 97000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
Statistics: Num rows: 500 Data size: 97000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col3 (type: bigint), _col4 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: string), _col1 (type: string)
1 _col2 (type: string), _col0 (type: string)
outputColumnNames: _col0, _col1, _col3, _col4, _col6
Statistics: Num rows: 500 Data size: 99000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col3 is null or (_col3 = 0L) or (_col6 is not null or _col1 is null or (_col4 < _col3)) is not true) (type: boolean)
Statistics: Num rows: 252 Data size: 49896 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
Statistics: Num rows: 252 Data size: 49896 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
keys: _col0 (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 252 Data size: 23940 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 252 Data size: 23940 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 252 Data size: 23940 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col1 is not null (type: boolean)
Statistics: Num rows: 252 Data size: 23940 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: bigint)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: bigint)
Statistics: Num rows: 252 Data size: 23940 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Reducer 5
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Semi Join 0 to 1
keys:
0 _col1 (type: bigint)
1 _col0 (type: bigint)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 95 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 95 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 316 Data size: 32548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 316 Data size: 32548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 7
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 316 Data size: 85636 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: string), true (type: boolean), _col0 (type: string)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 316 Data size: 86900 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col0 (type: string)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col2 (type: string), _col0 (type: string)
Statistics: Num rows: 316 Data size: 86900 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Reducer 8
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: boolean)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: bigint)
outputColumnNames: _col1
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col1 is not null (type: boolean)
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: bigint)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: bigint)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: bigint)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: bigint)
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select key, count(*) from src where value NOT IN (select concat('v', value) from src sc where sc.key = src.key ) group by key having count(*) in (select count(*) from src s1 where s1.key = '90' group by s1.key )
PREHOOK: type: QUERY
PREHOOK: Input: default@src
#### A masked pattern was here ####
POSTHOOK: query: select key, count(*) from src where value NOT IN (select concat('v', value) from src sc where sc.key = src.key ) group by key having count(*) in (select count(*) from src s1 where s1.key = '90' group by s1.key )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
#### A masked pattern was here ####
0 3
119 3
128 3
167 3
187 3
193 3
199 3
208 3
273 3
298 3
311 3
316 3
318 3
327 3
35 3
369 3
384 3
396 3
403 3
409 3
417 3
430 3
431 3
438 3
454 3
466 3
480 3
498 3
5 3
70 3
90 3
Warning: Shuffle Join MERGEJOIN[41][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select * from part where (p_size-1) NOT IN (select min(p_size) from part group by p_type) order by p_brand
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain select * from part where (p_size-1) NOT IN (select min(p_size) from part group by p_type) order by p_brand
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (XPROD_EDGE), Reducer 7 (XPROD_EDGE)
Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
Reducer 7 <- Reducer 5 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: part
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_partkey (type: int), p_name (type: string), p_mfgr (type: string), p_brand (type: string), p_type (type: string), p_size (type: int), p_container (type: string), p_retailprice (type: double), p_comment (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: (_col5 - 1) (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: (_col5 - 1) (type: int)
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Select Operator
expressions: p_type (type: string), p_size (type: int)
outputColumnNames: p_type, p_size
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(p_size)
keys: p_type (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: int)
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 (_col5 - 1) (type: int)
1 _col0 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10
Statistics: Num rows: 29 Data size: 18059 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 29 Data size: 18059 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col10 (type: boolean)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10, _col11, _col12
Statistics: Num rows: 29 Data size: 18523 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col11 (type: bigint), _col12 (type: bigint), _col10 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col12
Statistics: Num rows: 29 Data size: 18523 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((_col9 = 0L) or (_col12 is null and (_col10 >= _col9) and _col5 is not null)) (type: boolean)
Statistics: Num rows: 29 Data size: 18523 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col3 (type: string)
null sort order: z
sort order: +
Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: int), VALUE._col1 (type: string), VALUE._col2 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: string), VALUE._col4 (type: int), VALUE._col5 (type: string), VALUE._col6 (type: double), VALUE._col7 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: min(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: int)
outputColumnNames: _col1
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col1 is not null (type: boolean)
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col1 (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col1)
minReductionHashAggr: 0.9583333
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), true (type: boolean)
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 24 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Reducer 7
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[41][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from part where (p_size-1) NOT IN (select min(p_size) from part group by p_type) order by p_brand
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select * from part where (p_size-1) NOT IN (select min(p_size) from part group by p_type) order by p_brand
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
105685 almond antique violet chocolate turquoise Manufacturer#2 Brand#22 MEDIUM ANODIZED COPPER 14 MED CAN 1690.68 ly pending requ
110592 almond antique salmon chartreuse burlywood Manufacturer#1 Brand#15 PROMO BURNISHED NICKEL 6 JUMBO PKG 1602.59 to the furiously
132666 almond aquamarine rose maroon antique Manufacturer#2 Brand#24 SMALL POLISHED NICKEL 25 MED BOX 1698.66 even
144293 almond antique olive coral navajo Manufacturer#3 Brand#34 STANDARD POLISHED STEEL 45 JUMBO CAN 1337.29 ag furiously about
17273 almond antique forest lavender goldenrod Manufacturer#3 Brand#35 PROMO ANODIZED TIN 14 JUMBO CASE 1190.27 along the
192697 almond antique blue firebrick mint Manufacturer#5 Brand#52 MEDIUM BURNISHED TIN 31 LG DRUM 1789.69 ickly ir
33357 almond azure aquamarine papaya violet Manufacturer#4 Brand#41 STANDARD ANODIZED TIN 12 WRAP CASE 1290.35 reful
40982 almond antique misty red olive Manufacturer#3 Brand#32 ECONOMY PLATED COPPER 1 LG PKG 1922.98 c foxes can s
42669 almond antique medium spring khaki Manufacturer#5 Brand#51 STANDARD BURNISHED TIN 6 MED CAN 1611.66 sits haggl
45261 almond aquamarine floral ivory bisque Manufacturer#4 Brand#42 SMALL PLATED STEEL 27 WRAP CASE 1206.26 careful
48427 almond antique violet mint lemon Manufacturer#4 Brand#42 PROMO POLISHED STEEL 39 SM CASE 1375.42 hely ironic i
49671 almond antique gainsboro frosted violet Manufacturer#4 Brand#41 SMALL BRUSHED BRASS 10 SM BOX 1620.67 ccounts run quick
65667 almond aquamarine pink moccasin thistle Manufacturer#1 Brand#12 LARGE BURNISHED STEEL 42 JUMBO CASE 1632.66 e across the expr
78486 almond azure blanched chiffon midnight Manufacturer#5 Brand#52 LARGE BRUSHED BRASS 23 MED BAG 1464.48 hely blith
85768 almond antique chartreuse lavender yellow Manufacturer#1 Brand#12 LARGE BRUSHED STEEL 34 SM BAG 1753.76 refull
90681 almond antique chartreuse khaki white Manufacturer#3 Brand#31 MEDIUM BURNISHED TIN 17 SM CASE 1671.68 are slyly after the sl
Warning: Shuffle Join MERGEJOIN[43][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select * from part where (p_size-1) NOT IN (select min(p_size) from part group by p_type) order by p_brand, p_partkey limit 4
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain select * from part where (p_size-1) NOT IN (select min(p_size) from part group by p_type) order by p_brand, p_partkey limit 4
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (XPROD_EDGE), Reducer 7 (XPROD_EDGE)
Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
Reducer 7 <- Reducer 5 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: part
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_partkey (type: int), p_name (type: string), p_mfgr (type: string), p_brand (type: string), p_type (type: string), p_size (type: int), p_container (type: string), p_retailprice (type: double), p_comment (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: (_col5 - 1) (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: (_col5 - 1) (type: int)
Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Select Operator
expressions: p_type (type: string), p_size (type: int)
outputColumnNames: p_type, p_size
Statistics: Num rows: 26 Data size: 2808 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(p_size)
keys: p_type (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: int)
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 (_col5 - 1) (type: int)
1 _col0 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10
Statistics: Num rows: 29 Data size: 18059 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 29 Data size: 18059 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col10 (type: boolean)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10, _col11, _col12
Statistics: Num rows: 29 Data size: 18523 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col11 (type: bigint), _col12 (type: bigint), _col10 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col12
Statistics: Num rows: 29 Data size: 18523 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((_col9 = 0L) or (_col12 is null and (_col10 >= _col9) and _col5 is not null)) (type: boolean)
Statistics: Num rows: 29 Data size: 18523 Basic stats: COMPLETE Column stats: COMPLETE
Top N Key Operator
sort order: ++
keys: _col3 (type: string), _col0 (type: int)
null sort order: zz
Statistics: Num rows: 29 Data size: 18523 Basic stats: COMPLETE Column stats: COMPLETE
top n: 4
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col3 (type: string), _col0 (type: int)
null sort order: zz
sort order: ++
Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string), _col2 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: int), VALUE._col0 (type: string), VALUE._col1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col2 (type: string), VALUE._col3 (type: int), VALUE._col4 (type: string), VALUE._col5 (type: double), VALUE._col6 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 4
Statistics: Num rows: 4 Data size: 2476 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 4 Data size: 2476 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: min(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 2592 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: int)
outputColumnNames: _col1
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col1 is not null (type: boolean)
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col1 (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col1)
minReductionHashAggr: 0.9583333
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 24 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), true (type: boolean)
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 24 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Reducer 7
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Stage: Stage-0
Fetch Operator
limit: 4
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[43][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from part where (p_size-1) NOT IN (select min(p_size) from part group by p_type) order by p_brand, p_partkey limit 4
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select * from part where (p_size-1) NOT IN (select min(p_size) from part group by p_type) order by p_brand, p_partkey limit 4
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
105685 almond antique violet chocolate turquoise Manufacturer#2 Brand#22 MEDIUM ANODIZED COPPER 14 MED CAN 1690.68 ly pending requ
110592 almond antique salmon chartreuse burlywood Manufacturer#1 Brand#15 PROMO BURNISHED NICKEL 6 JUMBO PKG 1602.59 to the furiously
65667 almond aquamarine pink moccasin thistle Manufacturer#1 Brand#12 LARGE BURNISHED STEEL 42 JUMBO CASE 1632.66 e across the expr
85768 almond antique chartreuse lavender yellow Manufacturer#1 Brand#12 LARGE BRUSHED STEEL 34 SM BAG 1753.76 refull
Warning: Shuffle Join MERGEJOIN[47][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select * from src where key NOT IN (select p_name from part UNION ALL select p_brand from part)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
PREHOOK: Input: default@src
#### A masked pattern was here ####
POSTHOOK: query: explain select * from src where key NOT IN (select p_name from part UNION ALL select p_brand from part)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
POSTHOOK: Input: default@src
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Map 11 <- Union 9 (CONTAINS)
Map 4 <- Union 5 (CONTAINS)
Map 7 <- Union 5 (CONTAINS)
Map 8 <- Union 9 (CONTAINS)
Reducer 10 <- Union 9 (CUSTOM_SIMPLE_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 3 <- Reducer 10 (XPROD_EDGE), Reducer 2 (XPROD_EDGE)
Reducer 6 <- Union 5 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: src
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 11
Map Operator Tree:
TableScan
alias: part
Statistics: Num rows: 26 Data size: 2392 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_brand (type: string)
outputColumnNames: _col0
Statistics: Num rows: 26 Data size: 2392 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col0)
minReductionHashAggr: 0.9807692
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: part
filterExpr: p_name is not null (type: boolean)
Statistics: Num rows: 26 Data size: 3146 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_name is not null (type: boolean)
Statistics: Num rows: 26 Data size: 3146 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_name (type: string)
outputColumnNames: _col0
Statistics: Num rows: 26 Data size: 3146 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
minReductionHashAggr: 0.5192308
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 25 Data size: 3025 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 3025 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 7
Map Operator Tree:
TableScan
alias: part
filterExpr: p_brand is not null (type: boolean)
Statistics: Num rows: 26 Data size: 2392 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_brand is not null (type: boolean)
Statistics: Num rows: 26 Data size: 2392 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_brand (type: string)
outputColumnNames: _col0
Statistics: Num rows: 26 Data size: 2392 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
minReductionHashAggr: 0.5192308
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 25 Data size: 3025 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 3025 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 8
Map Operator Tree:
TableScan
alias: part
Statistics: Num rows: 26 Data size: 3146 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_name (type: string)
outputColumnNames: _col0
Statistics: Num rows: 26 Data size: 3146 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col0)
minReductionHashAggr: 0.9807692
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 10
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col3
Statistics: Num rows: 539 Data size: 96102 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 539 Data size: 96102 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string), _col3 (type: boolean)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col0, _col1, _col3, _col4, _col5
Statistics: Num rows: 539 Data size: 104726 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col4 (type: bigint), _col5 (type: bigint), _col3 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col5
Statistics: Num rows: 539 Data size: 104726 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((_col2 = 0L) or (_col5 is null and (_col3 >= _col2) and _col0 is not null)) (type: boolean)
Statistics: Num rows: 539 Data size: 104726 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 539 Data size: 95942 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 539 Data size: 95942 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 25 Data size: 3025 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), true (type: boolean)
outputColumnNames: _col0, _col1
Statistics: Num rows: 25 Data size: 3125 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 3125 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Union 5
Vertex: Union 5
Union 9
Vertex: Union 9
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[47][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from src where key NOT IN (select p_name from part UNION ALL select p_brand from part)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
PREHOOK: Input: default@src
#### A masked pattern was here ####
POSTHOOK: query: select * from src where key NOT IN (select p_name from part UNION ALL select p_brand from part)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
POSTHOOK: Input: default@src
#### A masked pattern was here ####
0 val_0
0 val_0
0 val_0
10 val_10
100 val_100
100 val_100
103 val_103
103 val_103
104 val_104
104 val_104
105 val_105
11 val_11
111 val_111
113 val_113
113 val_113
114 val_114
116 val_116
118 val_118
118 val_118
119 val_119
119 val_119
119 val_119
12 val_12
12 val_12
120 val_120
120 val_120
125 val_125
125 val_125
126 val_126
128 val_128
128 val_128
128 val_128
129 val_129
129 val_129
131 val_131
133 val_133
134 val_134
134 val_134
136 val_136
137 val_137
137 val_137
138 val_138
138 val_138
138 val_138
138 val_138
143 val_143
145 val_145
146 val_146
146 val_146
149 val_149
149 val_149
15 val_15
15 val_15
150 val_150
152 val_152
152 val_152
153 val_153
155 val_155
156 val_156
157 val_157
158 val_158
160 val_160
162 val_162
163 val_163
164 val_164
164 val_164
165 val_165
165 val_165
166 val_166
167 val_167
167 val_167
167 val_167
168 val_168
169 val_169
169 val_169
169 val_169
169 val_169
17 val_17
170 val_170
172 val_172
172 val_172
174 val_174
174 val_174
175 val_175
175 val_175
176 val_176
176 val_176
177 val_177
178 val_178
179 val_179
179 val_179
18 val_18
18 val_18
180 val_180
181 val_181
183 val_183
186 val_186
187 val_187
187 val_187
187 val_187
189 val_189
19 val_19
190 val_190
191 val_191
191 val_191
192 val_192
193 val_193
193 val_193
193 val_193
194 val_194
195 val_195
195 val_195
196 val_196
197 val_197
197 val_197
199 val_199
199 val_199
199 val_199
2 val_2
20 val_20
200 val_200
200 val_200
201 val_201
202 val_202
203 val_203
203 val_203
205 val_205
205 val_205
207 val_207
207 val_207
208 val_208
208 val_208
208 val_208
209 val_209
209 val_209
213 val_213
213 val_213
214 val_214
216 val_216
216 val_216
217 val_217
217 val_217
218 val_218
219 val_219
219 val_219
221 val_221
221 val_221
222 val_222
223 val_223
223 val_223
224 val_224
224 val_224
226 val_226
228 val_228
229 val_229
229 val_229
230 val_230
230 val_230
230 val_230
230 val_230
230 val_230
233 val_233
233 val_233
235 val_235
237 val_237
237 val_237
238 val_238
238 val_238
239 val_239
239 val_239
24 val_24
24 val_24
241 val_241
242 val_242
242 val_242
244 val_244
247 val_247
248 val_248
249 val_249
252 val_252
255 val_255
255 val_255
256 val_256
256 val_256
257 val_257
258 val_258
26 val_26
26 val_26
260 val_260
262 val_262
263 val_263
265 val_265
265 val_265
266 val_266
27 val_27
272 val_272
272 val_272
273 val_273
273 val_273
273 val_273
274 val_274
275 val_275
277 val_277
277 val_277
277 val_277
277 val_277
278 val_278
278 val_278
28 val_28
280 val_280
280 val_280
281 val_281
281 val_281
282 val_282
282 val_282
283 val_283
284 val_284
285 val_285
286 val_286
287 val_287
288 val_288
288 val_288
289 val_289
291 val_291
292 val_292
296 val_296
298 val_298
298 val_298
298 val_298
30 val_30
302 val_302
305 val_305
306 val_306
307 val_307
307 val_307
308 val_308
309 val_309
309 val_309
310 val_310
311 val_311
311 val_311
311 val_311
315 val_315
316 val_316
316 val_316
316 val_316
317 val_317
317 val_317
318 val_318
318 val_318
318 val_318
321 val_321
321 val_321
322 val_322
322 val_322
323 val_323
325 val_325
325 val_325
327 val_327
327 val_327
327 val_327
33 val_33
331 val_331
331 val_331
332 val_332
333 val_333
333 val_333
335 val_335
336 val_336
338 val_338
339 val_339
34 val_34
341 val_341
342 val_342
342 val_342
344 val_344
344 val_344
345 val_345
348 val_348
348 val_348
348 val_348
348 val_348
348 val_348
35 val_35
35 val_35
35 val_35
351 val_351
353 val_353
353 val_353
356 val_356
360 val_360
362 val_362
364 val_364
365 val_365
366 val_366
367 val_367
367 val_367
368 val_368
369 val_369
369 val_369
369 val_369
37 val_37
37 val_37
373 val_373
374 val_374
375 val_375
377 val_377
378 val_378
379 val_379
382 val_382
382 val_382
384 val_384
384 val_384
384 val_384
386 val_386
389 val_389
392 val_392
393 val_393
394 val_394
395 val_395
395 val_395
396 val_396
396 val_396
396 val_396
397 val_397
397 val_397
399 val_399
399 val_399
4 val_4
400 val_400
401 val_401
401 val_401
401 val_401
401 val_401
401 val_401
402 val_402
403 val_403
403 val_403
403 val_403
404 val_404
404 val_404
406 val_406
406 val_406
406 val_406
406 val_406
407 val_407
409 val_409
409 val_409
409 val_409
41 val_41
411 val_411
413 val_413
413 val_413
414 val_414
414 val_414
417 val_417
417 val_417
417 val_417
418 val_418
419 val_419
42 val_42
42 val_42
421 val_421
424 val_424
424 val_424
427 val_427
429 val_429
429 val_429
43 val_43
430 val_430
430 val_430
430 val_430
431 val_431
431 val_431
431 val_431
432 val_432
435 val_435
436 val_436
437 val_437
438 val_438
438 val_438
438 val_438
439 val_439
439 val_439
44 val_44
443 val_443
444 val_444
446 val_446
448 val_448
449 val_449
452 val_452
453 val_453
454 val_454
454 val_454
454 val_454
455 val_455
457 val_457
458 val_458
458 val_458
459 val_459
459 val_459
460 val_460
462 val_462
462 val_462
463 val_463
463 val_463
466 val_466
466 val_466
466 val_466
467 val_467
468 val_468
468 val_468
468 val_468
468 val_468
469 val_469
469 val_469
469 val_469
469 val_469
469 val_469
47 val_47
470 val_470
472 val_472
475 val_475
477 val_477
478 val_478
478 val_478
479 val_479
480 val_480
480 val_480
480 val_480
481 val_481
482 val_482
483 val_483
484 val_484
485 val_485
487 val_487
489 val_489
489 val_489
489 val_489
489 val_489
490 val_490
491 val_491
492 val_492
492 val_492
493 val_493
494 val_494
495 val_495
496 val_496
497 val_497
498 val_498
498 val_498
498 val_498
5 val_5
5 val_5
5 val_5
51 val_51
51 val_51
53 val_53
54 val_54
57 val_57
58 val_58
58 val_58
64 val_64
65 val_65
66 val_66
67 val_67
67 val_67
69 val_69
70 val_70
70 val_70
70 val_70
72 val_72
72 val_72
74 val_74
76 val_76
76 val_76
77 val_77
78 val_78
8 val_8
80 val_80
82 val_82
83 val_83
83 val_83
84 val_84
84 val_84
85 val_85
86 val_86
87 val_87
9 val_9
90 val_90
90 val_90
90 val_90
92 val_92
95 val_95
95 val_95
96 val_96
97 val_97
97 val_97
98 val_98
98 val_98
PREHOOK: query: explain select count(*) as c from part as e where p_size + 100 not in ( select p_type from part where p_brand = e.p_brand)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: explain select count(*) as c from part as e where p_size + 100 not in ( select p_type from part where p_brand = e.p_brand)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 4 <- Reducer 3 (CUSTOM_SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
Reducer 6 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: e
Statistics: Num rows: 26 Data size: 2496 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_brand (type: string), p_size (type: int), (p_size + 100) (type: int)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 26 Data size: 2600 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 26 Data size: 2600 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: int), _col2 (type: int)
Filter Operator
predicate: p_brand is not null (type: boolean)
Statistics: Num rows: 26 Data size: 5096 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(p_type)
keys: p_brand (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 16 Data size: 1728 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 16 Data size: 1728 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Filter Operator
predicate: (p_brand is not null and p_type is not null and UDFToDouble(p_type) is not null) (type: boolean)
Statistics: Num rows: 26 Data size: 5096 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: p_brand (type: string), p_type (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 4704 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
Statistics: Num rows: 24 Data size: 4704 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col4, _col5
Statistics: Num rows: 26 Data size: 3016 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), UDFToDouble(_col2) (type: double)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), UDFToDouble(_col2) (type: double)
Statistics: Num rows: 26 Data size: 3016 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: int), _col4 (type: bigint), _col5 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: string), UDFToDouble(_col2) (type: double)
1 _col2 (type: string), UDFToDouble(_col0) (type: double)
outputColumnNames: _col1, _col4, _col5, _col7
Statistics: Num rows: 29 Data size: 688 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: int), _col4 (type: bigint), _col5 (type: bigint), _col7 (type: boolean)
outputColumnNames: _col1, _col2, _col3, _col6
Statistics: Num rows: 29 Data size: 688 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col2 is null or (_col2 = 0L) or (_col6 is not null or _col1 is null or (_col3 < _col2)) is not true) (type: boolean)
Statistics: Num rows: 16 Data size: 380 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
Statistics: Num rows: 16 Data size: 380 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
minReductionHashAggr: 0.9375
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 16 Data size: 1728 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 16 Data size: 1728 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 24 Data size: 4704 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: string), true (type: boolean), _col0 (type: string)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 24 Data size: 4800 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), UDFToDouble(_col0) (type: double)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col2 (type: string), UDFToDouble(_col0) (type: double)
Statistics: Num rows: 24 Data size: 4800 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select count(*) as c from part as e where p_size + 100 not in ( select p_type from part where p_brand = e.p_brand)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
#### A masked pattern was here ####
POSTHOOK: query: select count(*) as c from part as e where p_size + 100 not in ( select p_type from part where p_brand = e.p_brand)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
26
PREHOOK: query: CREATE TABLE t1_n0 (c1 INT, c2 CHAR(100))
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@t1_n0
POSTHOOK: query: CREATE TABLE t1_n0 (c1 INT, c2 CHAR(100))
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@t1_n0
PREHOOK: query: INSERT INTO t1_n0 VALUES (null,null), (1,''), (2,'abcde'), (100,'abcdefghij')
PREHOOK: type: QUERY
PREHOOK: Input: _dummy_database@_dummy_table
PREHOOK: Output: default@t1_n0
POSTHOOK: query: INSERT INTO t1_n0 VALUES (null,null), (1,''), (2,'abcde'), (100,'abcdefghij')
POSTHOOK: type: QUERY
POSTHOOK: Input: _dummy_database@_dummy_table
POSTHOOK: Output: default@t1_n0
POSTHOOK: Lineage: t1_n0.c1 SCRIPT []
POSTHOOK: Lineage: t1_n0.c2 SCRIPT []
PREHOOK: query: CREATE TABLE t2_n0 (c1 INT)
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@t2_n0
POSTHOOK: query: CREATE TABLE t2_n0 (c1 INT)
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@t2_n0
PREHOOK: query: INSERT INTO t2_n0 VALUES (null), (2), (100)
PREHOOK: type: QUERY
PREHOOK: Input: _dummy_database@_dummy_table
PREHOOK: Output: default@t2_n0
POSTHOOK: query: INSERT INTO t2_n0 VALUES (null), (2), (100)
POSTHOOK: type: QUERY
POSTHOOK: Input: _dummy_database@_dummy_table
POSTHOOK: Output: default@t2_n0
POSTHOOK: Lineage: t2_n0.c1 SCRIPT []
Warning: Shuffle Join MERGEJOIN[33][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain SELECT c1 FROM t1_n0 WHERE c1 NOT IN (SELECT c1 FROM t2_n0)
PREHOOK: type: QUERY
PREHOOK: Input: default@t1_n0
PREHOOK: Input: default@t2_n0
#### A masked pattern was here ####
POSTHOOK: query: explain SELECT c1 FROM t1_n0 WHERE c1 NOT IN (SELECT c1 FROM t2_n0)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t1_n0
POSTHOOK: Input: default@t2_n0
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (XPROD_EDGE), Reducer 6 (XPROD_EDGE)
Reducer 5 <- Map 4 (SIMPLE_EDGE)
Reducer 6 <- Map 4 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: t1_n0
Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: c1 (type: int)
outputColumnNames: _col0
Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: t2_n0
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: c1 is not null (type: boolean)
Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: c1 (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: c1 (type: int)
outputColumnNames: c1
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(c1)
minReductionHashAggr: 0.6666666
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0, _col2
Statistics: Num rows: 4 Data size: 28 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 4 Data size: 28 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col2 (type: boolean)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col0, _col2, _col3, _col4
Statistics: Num rows: 4 Data size: 92 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col3 (type: bigint), _col4 (type: bigint), _col2 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col4
Statistics: Num rows: 4 Data size: 92 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((_col1 = 0L) or (_col4 is null and (_col2 >= _col1) and _col0 is not null)) (type: boolean)
Statistics: Num rows: 4 Data size: 92 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int)
outputColumnNames: _col0
Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), true (type: boolean)
outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[33][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: SELECT c1 FROM t1_n0 WHERE c1 NOT IN (SELECT c1 FROM t2_n0)
PREHOOK: type: QUERY
PREHOOK: Input: default@t1_n0
PREHOOK: Input: default@t2_n0
#### A masked pattern was here ####
POSTHOOK: query: SELECT c1 FROM t1_n0 WHERE c1 NOT IN (SELECT c1 FROM t2_n0)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t1_n0
POSTHOOK: Input: default@t2_n0
#### A masked pattern was here ####
PREHOOK: query: explain cbo SELECT c1 FROM t1_n0 WHERE c1 NOT IN (SELECT c1 FROM t2_n0 where t1_n0.c2=t2_n0.c1)
PREHOOK: type: QUERY
PREHOOK: Input: default@t1_n0
PREHOOK: Input: default@t2_n0
#### A masked pattern was here ####
POSTHOOK: query: explain cbo SELECT c1 FROM t1_n0 WHERE c1 NOT IN (SELECT c1 FROM t2_n0 where t1_n0.c2=t2_n0.c1)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t1_n0
POSTHOOK: Input: default@t2_n0
#### A masked pattern was here ####
CBO PLAN:
HiveProject(c1=[$0])
HiveFilter(condition=[OR(IS NULL($1), =($1, 0), IS NOT TRUE(OR(IS NOT NULL($5), IS NULL($0), <($2, $1))))])
HiveProject(c1=[$0], c=[$4], ck=[$5], c20=[$2], c10=[$6], literalTrue=[$7], c100=[$8])
HiveJoin(condition=[AND(=($0, $6), =($2, $8))], joinType=[left], algorithm=[none], cost=[not available])
HiveJoin(condition=[=($1, $3)], joinType=[left], algorithm=[none], cost=[not available])
HiveProject(c1=[$0], c20=[CAST($1):DOUBLE], CAST=[CAST($1):DOUBLE])
HiveTableScan(table=[[default, t1_n0]], table:alias=[t1_n0])
HiveProject(c10=[$0], c=[$1], ck=[$2])
HiveAggregate(group=[{0}], c=[COUNT()], ck=[COUNT($1)])
HiveProject(c10=[CAST($0):DOUBLE], c1=[$0])
HiveFilter(condition=[IS NOT NULL(CAST($0):DOUBLE)])
HiveTableScan(table=[[default, t2_n0]], table:alias=[t2_n0])
HiveProject(c1=[$0], literalTrue=[true], c10=[$1])
HiveAggregate(group=[{0, 1}])
HiveProject(c1=[$0], c10=[CAST($0):DOUBLE])
HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL(CAST($0):DOUBLE))])
HiveTableScan(table=[[default, t2_n0]], table:alias=[t2_n0])
PREHOOK: query: explain SELECT c1 FROM t1_n0 WHERE c1 NOT IN (SELECT c1 FROM t2_n0 where t1_n0.c2=t2_n0.c1)
PREHOOK: type: QUERY
PREHOOK: Input: default@t1_n0
PREHOOK: Input: default@t2_n0
#### A masked pattern was here ####
POSTHOOK: query: explain SELECT c1 FROM t1_n0 WHERE c1 NOT IN (SELECT c1 FROM t2_n0 where t1_n0.c2=t2_n0.c1)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t1_n0
POSTHOOK: Input: default@t2_n0
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 5 <- Map 4 (SIMPLE_EDGE)
Reducer 6 <- Map 4 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: t1_n0
Statistics: Num rows: 4 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: c1 (type: int), UDFToDouble(c2) (type: double)
outputColumnNames: _col0, _col1
Statistics: Num rows: 4 Data size: 48 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: double)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: double)
Statistics: Num rows: 4 Data size: 48 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int)
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: t2_n0
filterExpr: (UDFToDouble(c1) is not null or (c1 is not null and UDFToDouble(c1) is not null)) (type: boolean)
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: UDFToDouble(c1) is not null (type: boolean)
Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToDouble(c1) (type: double), c1 (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col1)
keys: _col0 (type: double)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 2 Data size: 48 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: double)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: double)
Statistics: Num rows: 2 Data size: 48 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Filter Operator
predicate: (c1 is not null and UDFToDouble(c1) is not null) (type: boolean)
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: c1 (type: int), UDFToDouble(c1) (type: double)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: int), _col1 (type: double)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: double)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col1 (type: double)
Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: double)
1 _col0 (type: double)
outputColumnNames: _col0, _col2, _col4, _col5
Statistics: Num rows: 4 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col2 (type: double)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col2 (type: double)
Statistics: Num rows: 4 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col4 (type: bigint), _col5 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: int), _col2 (type: double)
1 _col0 (type: int), _col2 (type: double)
outputColumnNames: _col0, _col4, _col5, _col7
Statistics: Num rows: 5 Data size: 92 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col4 (type: bigint), _col5 (type: bigint), _col7 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col5
Statistics: Num rows: 5 Data size: 92 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col1 is null or (_col1 = 0L) or (_col5 is not null or _col0 is null or (_col2 < _col1)) is not true) (type: boolean)
Statistics: Num rows: 5 Data size: 92 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int)
outputColumnNames: _col0
Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: double)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 2 Data size: 48 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: double)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: double)
Statistics: Num rows: 2 Data size: 48 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int), KEY._col1 (type: double)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), true (type: boolean), _col1 (type: double)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col2 (type: double)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col2 (type: double)
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: SELECT c1 FROM t1_n0 WHERE c1 NOT IN (SELECT c1 FROM t2_n0 where t1_n0.c2=t2_n0.c1)
PREHOOK: type: QUERY
PREHOOK: Input: default@t1_n0
PREHOOK: Input: default@t2_n0
#### A masked pattern was here ####
POSTHOOK: query: SELECT c1 FROM t1_n0 WHERE c1 NOT IN (SELECT c1 FROM t2_n0 where t1_n0.c2=t2_n0.c1)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t1_n0
POSTHOOK: Input: default@t2_n0
#### A masked pattern was here ####
1
100
2
NULL
PREHOOK: query: DROP TABLE t1_n0
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@t1_n0
PREHOOK: Output: default@t1_n0
POSTHOOK: query: DROP TABLE t1_n0
POSTHOOK: type: DROPTABLE
POSTHOOK: Input: default@t1_n0
POSTHOOK: Output: default@t1_n0
PREHOOK: query: DROP TABLE t2_n0
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@t2_n0
PREHOOK: Output: default@t2_n0
POSTHOOK: query: DROP TABLE t2_n0
POSTHOOK: type: DROPTABLE
POSTHOOK: Input: default@t2_n0
POSTHOOK: Output: default@t2_n0
PREHOOK: query: create table t1_n0(a int, b int)
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@t1_n0
POSTHOOK: query: create table t1_n0(a int, b int)
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@t1_n0
PREHOOK: query: insert into t1_n0 values(1,0), (1,0),(1,0)
PREHOOK: type: QUERY
PREHOOK: Input: _dummy_database@_dummy_table
PREHOOK: Output: default@t1_n0
POSTHOOK: query: insert into t1_n0 values(1,0), (1,0),(1,0)
POSTHOOK: type: QUERY
POSTHOOK: Input: _dummy_database@_dummy_table
POSTHOOK: Output: default@t1_n0
POSTHOOK: Lineage: t1_n0.a SCRIPT []
POSTHOOK: Lineage: t1_n0.b SCRIPT []
PREHOOK: query: create table t2_n0(a int, b int)
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@t2_n0
POSTHOOK: query: create table t2_n0(a int, b int)
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@t2_n0
PREHOOK: query: insert into t2_n0 values(2,1), (3,1), (NULL,1)
PREHOOK: type: QUERY
PREHOOK: Input: _dummy_database@_dummy_table
PREHOOK: Output: default@t2_n0
POSTHOOK: query: insert into t2_n0 values(2,1), (3,1), (NULL,1)
POSTHOOK: type: QUERY
POSTHOOK: Input: _dummy_database@_dummy_table
POSTHOOK: Output: default@t2_n0
POSTHOOK: Lineage: t2_n0.a SCRIPT []
POSTHOOK: Lineage: t2_n0.b SCRIPT []
PREHOOK: query: explain select t1_n0.a from t1_n0 where t1_n0.b NOT IN (select t2_n0.a from t2_n0 where t2_n0.b=t1_n0.a)
PREHOOK: type: QUERY
PREHOOK: Input: default@t1_n0
PREHOOK: Input: default@t2_n0
#### A masked pattern was here ####
POSTHOOK: query: explain select t1_n0.a from t1_n0 where t1_n0.b NOT IN (select t2_n0.a from t2_n0 where t2_n0.b=t1_n0.a)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t1_n0
POSTHOOK: Input: default@t2_n0
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 5 <- Map 4 (SIMPLE_EDGE)
Reducer 6 <- Map 4 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: t1_n0
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: a (type: int), b (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: int)
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: t2_n0
filterExpr: (b is not null or (a is not null and b is not null)) (type: boolean)
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: b is not null (type: boolean)
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(a)
keys: b (type: int)
minReductionHashAggr: 0.6666666
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Filter Operator
predicate: (a is not null and b is not null) (type: boolean)
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: b (type: int), a (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0, _col1, _col3, _col4
Statistics: Num rows: 3 Data size: 72 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
Statistics: Num rows: 3 Data size: 72 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col3 (type: bigint), _col4 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: int), _col1 (type: int)
1 _col2 (type: int), _col0 (type: int)
outputColumnNames: _col0, _col1, _col3, _col4, _col6
Statistics: Num rows: 4 Data size: 112 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col3 is null or (_col3 = 0L) or (_col6 is not null or _col1 is null or (_col4 < _col3)) is not true) (type: boolean)
Statistics: Num rows: 4 Data size: 112 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int)
outputColumnNames: _col0
Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int), KEY._col1 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: int), true (type: boolean), _col0 (type: int)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 2 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: int), _col0 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col2 (type: int), _col0 (type: int)
Statistics: Num rows: 2 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select t1_n0.a from t1_n0 where t1_n0.b NOT IN (select t2_n0.a from t2_n0 where t2_n0.b=t1_n0.a)
PREHOOK: type: QUERY
PREHOOK: Input: default@t1_n0
PREHOOK: Input: default@t2_n0
#### A masked pattern was here ####
POSTHOOK: query: select t1_n0.a from t1_n0 where t1_n0.b NOT IN (select t2_n0.a from t2_n0 where t2_n0.b=t1_n0.a)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t1_n0
POSTHOOK: Input: default@t2_n0
#### A masked pattern was here ####
PREHOOK: query: drop table t1_n0
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@t1_n0
PREHOOK: Output: default@t1_n0
POSTHOOK: query: drop table t1_n0
POSTHOOK: type: DROPTABLE
POSTHOOK: Input: default@t1_n0
POSTHOOK: Output: default@t1_n0
PREHOOK: query: drop table t2_n0
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@t2_n0
PREHOOK: Output: default@t2_n0
POSTHOOK: query: drop table t2_n0
POSTHOOK: type: DROPTABLE
POSTHOOK: Input: default@t2_n0
POSTHOOK: Output: default@t2_n0
PREHOOK: query: create table t7(i int, j int)
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@t7
POSTHOOK: query: create table t7(i int, j int)
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@t7
PREHOOK: query: insert into t7 values(null, 5), (4, 15)
PREHOOK: type: QUERY
PREHOOK: Input: _dummy_database@_dummy_table
PREHOOK: Output: default@t7
POSTHOOK: query: insert into t7 values(null, 5), (4, 15)
POSTHOOK: type: QUERY
POSTHOOK: Input: _dummy_database@_dummy_table
POSTHOOK: Output: default@t7
POSTHOOK: Lineage: t7.i SCRIPT []
POSTHOOK: Lineage: t7.j SCRIPT []
PREHOOK: query: create table fixOb(i int, j int)
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@fixOb
POSTHOOK: query: create table fixOb(i int, j int)
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@fixOb
PREHOOK: query: insert into fixOb values(-1, 5), (-1, 15)
PREHOOK: type: QUERY
PREHOOK: Input: _dummy_database@_dummy_table
PREHOOK: Output: default@fixob
POSTHOOK: query: insert into fixOb values(-1, 5), (-1, 15)
POSTHOOK: type: QUERY
POSTHOOK: Input: _dummy_database@_dummy_table
POSTHOOK: Output: default@fixob
POSTHOOK: Lineage: fixob.i SCRIPT []
POSTHOOK: Lineage: fixob.j SCRIPT []
PREHOOK: query: explain select * from fixOb where j NOT IN (select i from t7 where t7.j=fixOb.j)
PREHOOK: type: QUERY
PREHOOK: Input: default@fixob
PREHOOK: Input: default@t7
#### A masked pattern was here ####
POSTHOOK: query: explain select * from fixOb where j NOT IN (select i from t7 where t7.j=fixOb.j)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@fixob
POSTHOOK: Input: default@t7
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 5 <- Map 4 (SIMPLE_EDGE)
Reducer 6 <- Map 4 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: fixob
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: i (type: int), j (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: int)
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int)
Execution mode: vectorized, llap
LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: t7
filterExpr: (j is not null or (j = i)) (type: boolean)
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: j is not null (type: boolean)
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(i)
keys: j (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 2 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 2 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Filter Operator
predicate: (j = i) (type: boolean)
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: j (type: int), i (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0, _col1, _col3, _col4
Statistics: Num rows: 2 Data size: 48 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: int)
Statistics: Num rows: 2 Data size: 48 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col3 (type: bigint), _col4 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: int)
1 _col1 (type: int)
outputColumnNames: _col0, _col1, _col3, _col4, _col5
Statistics: Num rows: 2 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col3 is null or (_col3 = 0L) or (_col5 is not null or _col1 is null or (_col4 < _col3)) is not true) (type: boolean)
Statistics: Num rows: 2 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 2 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 2 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int), KEY._col1 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: true (type: boolean), _col0 (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: int)
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select * from fixOb where j NOT IN (select i from t7 where t7.j=fixOb.j)
PREHOOK: type: QUERY
PREHOOK: Input: default@fixob
PREHOOK: Input: default@t7
#### A masked pattern was here ####
POSTHOOK: query: select * from fixOb where j NOT IN (select i from t7 where t7.j=fixOb.j)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@fixob
POSTHOOK: Input: default@t7
#### A masked pattern was here ####
-1 15
PREHOOK: query: drop table t7
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@t7
PREHOOK: Output: default@t7
POSTHOOK: query: drop table t7
POSTHOOK: type: DROPTABLE
POSTHOOK: Input: default@t7
POSTHOOK: Output: default@t7
PREHOOK: query: drop table fixOb
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@fixob
PREHOOK: Output: default@fixob
POSTHOOK: query: drop table fixOb
POSTHOOK: type: DROPTABLE
POSTHOOK: Input: default@fixob
POSTHOOK: Output: default@fixob
PREHOOK: query: create table t_n0(i int, j int)
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@t_n0
POSTHOOK: query: create table t_n0(i int, j int)
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@t_n0
PREHOOK: query: insert into t_n0 values(1,2), (4,5), (7, NULL)
PREHOOK: type: QUERY
PREHOOK: Input: _dummy_database@_dummy_table
PREHOOK: Output: default@t_n0
POSTHOOK: query: insert into t_n0 values(1,2), (4,5), (7, NULL)
POSTHOOK: type: QUERY
POSTHOOK: Input: _dummy_database@_dummy_table
POSTHOOK: Output: default@t_n0
POSTHOOK: Lineage: t_n0.i SCRIPT []
POSTHOOK: Lineage: t_n0.j SCRIPT []
PREHOOK: query: explain select t_n0.i from t_n0 where t_n0.j NOT IN (select t1_n0.i from t_n0 t1_n0 where t1_n0.j=t_n0.j)
PREHOOK: type: QUERY
PREHOOK: Input: default@t_n0
#### A masked pattern was here ####
POSTHOOK: query: explain select t_n0.i from t_n0 where t_n0.j NOT IN (select t1_n0.i from t_n0 t1_n0 where t1_n0.j=t_n0.j)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t_n0
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: t_n0
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: i (type: int), j (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: int)
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int)
Filter Operator
predicate: j is not null (type: boolean)
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(i)
keys: j (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 2 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 2 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Filter Operator
predicate: (j = i) (type: boolean)
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: j (type: int), i (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0, _col1, _col3, _col4
Statistics: Num rows: 3 Data size: 72 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: int)
Statistics: Num rows: 3 Data size: 72 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col3 (type: bigint), _col4 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: int)
1 _col1 (type: int)
outputColumnNames: _col0, _col1, _col3, _col4, _col5
Statistics: Num rows: 3 Data size: 84 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col3 is null or (_col3 = 0L) or (_col5 is not null or _col1 is null or (_col4 < _col3)) is not true) (type: boolean)
Statistics: Num rows: 3 Data size: 84 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int)
outputColumnNames: _col0
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 2 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 2 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int), KEY._col1 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: true (type: boolean), _col0 (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: int)
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select t_n0.i from t_n0 where t_n0.j NOT IN (select t1_n0.i from t_n0 t1_n0 where t1_n0.j=t_n0.j)
PREHOOK: type: QUERY
PREHOOK: Input: default@t_n0
#### A masked pattern was here ####
POSTHOOK: query: select t_n0.i from t_n0 where t_n0.j NOT IN (select t1_n0.i from t_n0 t1_n0 where t1_n0.j=t_n0.j)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t_n0
#### A masked pattern was here ####
1
4
7
PREHOOK: query: explain select t_n0.i from t_n0 where t_n0.i NOT IN (select t1_n0.i from t_n0 t1_n0 where t1_n0.j=t_n0.j)
PREHOOK: type: QUERY
PREHOOK: Input: default@t_n0
#### A masked pattern was here ####
POSTHOOK: query: explain select t_n0.i from t_n0 where t_n0.i NOT IN (select t1_n0.i from t_n0 t1_n0 where t1_n0.j=t_n0.j)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t_n0
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: t_n0
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: i (type: int), j (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: int)
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int)
Filter Operator
predicate: j is not null (type: boolean)
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(i)
keys: j (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 2 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 2 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Filter Operator
predicate: (j is not null and i is not null) (type: boolean)
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: i (type: int), j (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0, _col1, _col3, _col4
Statistics: Num rows: 3 Data size: 72 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
Statistics: Num rows: 3 Data size: 72 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col3 (type: bigint), _col4 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: int), _col1 (type: int)
1 _col0 (type: int), _col2 (type: int)
outputColumnNames: _col0, _col3, _col4, _col6
Statistics: Num rows: 3 Data size: 72 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col3 is null or (_col3 = 0L) or (_col6 is not null or _col0 is null or (_col4 < _col3)) is not true) (type: boolean)
Statistics: Num rows: 3 Data size: 72 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int)
outputColumnNames: _col0
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 2 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 2 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int), KEY._col1 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), true (type: boolean), _col1 (type: int)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 2 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int), _col2 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col2 (type: int)
Statistics: Num rows: 2 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select t_n0.i from t_n0 where t_n0.i NOT IN (select t1_n0.i from t_n0 t1_n0 where t1_n0.j=t_n0.j)
PREHOOK: type: QUERY
PREHOOK: Input: default@t_n0
#### A masked pattern was here ####
POSTHOOK: query: select t_n0.i from t_n0 where t_n0.i NOT IN (select t1_n0.i from t_n0 t1_n0 where t1_n0.j=t_n0.j)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t_n0
#### A masked pattern was here ####
7
Warning: Shuffle Join MERGEJOIN[33][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select t_n0.i from t_n0 where t_n0.j NOT IN (select t1_n0.i from t_n0 t1_n0 )
PREHOOK: type: QUERY
PREHOOK: Input: default@t_n0
#### A masked pattern was here ####
POSTHOOK: query: explain select t_n0.i from t_n0 where t_n0.j NOT IN (select t1_n0.i from t_n0 t1_n0 )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t_n0
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (XPROD_EDGE), Reducer 5 (XPROD_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: t_n0
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: i (type: int), j (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: int)
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int)
Filter Operator
predicate: i is not null (type: boolean)
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: i (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: i (type: int)
outputColumnNames: i
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(i)
minReductionHashAggr: 0.6666666
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0, _col1, _col3
Statistics: Num rows: 3 Data size: 36 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 3 Data size: 36 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int), _col1 (type: int), _col3 (type: boolean)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col0, _col1, _col3, _col4, _col5
Statistics: Num rows: 3 Data size: 84 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), _col1 (type: int), _col4 (type: bigint), _col5 (type: bigint), _col3 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col5
Statistics: Num rows: 3 Data size: 84 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((_col2 = 0L) or (_col5 is null and (_col3 >= _col2) and _col1 is not null)) (type: boolean)
Statistics: Num rows: 3 Data size: 84 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int)
outputColumnNames: _col0
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), true (type: boolean)
outputColumnNames: _col0, _col1
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[33][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select t_n0.i from t_n0 where t_n0.j NOT IN (select t1_n0.i from t_n0 t1_n0 )
PREHOOK: type: QUERY
PREHOOK: Input: default@t_n0
#### A masked pattern was here ####
POSTHOOK: query: select t_n0.i from t_n0 where t_n0.j NOT IN (select t1_n0.i from t_n0 t1_n0 )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t_n0
#### A masked pattern was here ####
1
4
Warning: Shuffle Join MERGEJOIN[33][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select t_n0.i from t_n0 where t_n0.i NOT IN (select t1_n0.i from t_n0 t1_n0 )
PREHOOK: type: QUERY
PREHOOK: Input: default@t_n0
#### A masked pattern was here ####
POSTHOOK: query: explain select t_n0.i from t_n0 where t_n0.i NOT IN (select t1_n0.i from t_n0 t1_n0 )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t_n0
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (XPROD_EDGE), Reducer 5 (XPROD_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: t_n0
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: i (type: int)
outputColumnNames: _col0
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: i is not null (type: boolean)
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: i (type: int)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: i (type: int)
outputColumnNames: i
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(i)
minReductionHashAggr: 0.6666666
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Right Outer Join 0 to 1
keys:
0 _col0 (type: int)
1 _col0 (type: int)
outputColumnNames: _col1, _col2
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean), _col2 (type: int)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col1, _col2, _col3, _col4
Statistics: Num rows: 3 Data size: 72 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: int), _col3 (type: bigint), _col4 (type: bigint), _col1 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col4
Statistics: Num rows: 3 Data size: 72 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((_col1 = 0L) or (_col4 is null and (_col2 >= _col1) and _col0 is not null)) (type: boolean)
Statistics: Num rows: 3 Data size: 72 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int)
outputColumnNames: _col0
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 3 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: int), true (type: boolean)
outputColumnNames: _col0, _col1
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean)
Reducer 5
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[33][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select t_n0.i from t_n0 where t_n0.i NOT IN (select t1_n0.i from t_n0 t1_n0 )
PREHOOK: type: QUERY
PREHOOK: Input: default@t_n0
#### A masked pattern was here ####
POSTHOOK: query: select t_n0.i from t_n0 where t_n0.i NOT IN (select t1_n0.i from t_n0 t1_n0 )
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t_n0
#### A masked pattern was here ####
PREHOOK: query: drop table t1_n0
PREHOOK: type: DROPTABLE
POSTHOOK: query: drop table t1_n0
POSTHOOK: type: DROPTABLE
Warning: Shuffle Join MERGEJOIN[54][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 5' is a cross product
PREHOOK: query: explain select *
from src b
where b.key not in
(select a.key
from src a
where b.value > a.value and a.key > '9'
)
PREHOOK: type: QUERY
PREHOOK: Input: default@src
#### A masked pattern was here ####
POSTHOOK: query: explain select *
from src b
where b.key not in
(select a.key
from src a
where b.value > a.value and a.key > '9'
)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
#### A masked pattern was here ####
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 4 <- Map 1 (SIMPLE_EDGE)
Reducer 5 <- Map 1 (XPROD_EDGE), Reducer 7 (XPROD_EDGE)
Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
Reducer 7 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: b
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col1 (type: string)
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Filter Operator
predicate: ((key > '9') and value is not null) (type: boolean)
Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: key (type: string), value (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string)
Filter Operator
predicate: value is not null (type: boolean)
Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: value (type: string)
minReductionHashAggr: 0.4
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 307 Data size: 27937 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 307 Data size: 27937 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col1 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col3, _col4
Statistics: Num rows: 500 Data size: 97000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 97000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string), _col3 (type: bigint), _col4 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Left Outer Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col3, _col4, _col6, _col7
residual filter predicates: {(_col1 > _col7)}
Statistics: Num rows: 762 Data size: 172813 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col3 is null or (_col3 = 0L) or (_col6 is not null or _col0 is null or (_col4 < _col3)) is not true) (type: boolean)
Statistics: Num rows: 383 Data size: 86937 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 383 Data size: 68174 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
Statistics: Num rows: 383 Data size: 68174 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), true (type: boolean), _col1 (type: string)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 166 Data size: 30212 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 166 Data size: 30212 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: boolean), _col2 (type: string)
Reducer 5
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col0, _col1, _col2
residual filter predicates: {(_col0 > _col2)}
Statistics: Num rows: 16987 Data size: 4569503 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 16987 Data size: 4569503 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(), count(_col1)
keys: _col0 (type: string)
minReductionHashAggr: 0.98192734
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 307 Data size: 32849 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 307 Data size: 32849 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 6
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 307 Data size: 32849 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
null sort order: z
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 307 Data size: 32849 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reducer 7
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 307 Data size: 27937 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
Statistics: Num rows: 307 Data size: 27937 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Warning: Shuffle Join MERGEJOIN[54][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 5' is a cross product
PREHOOK: query: select *
from src b
where b.key not in
(select a.key
from src a
where b.value > a.value and a.key > '9'
)
PREHOOK: type: QUERY
PREHOOK: Input: default@src
#### A masked pattern was here ####
POSTHOOK: query: select *
from src b
where b.key not in
(select a.key
from src a
where b.value > a.value and a.key > '9'
)
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
#### A masked pattern was here ####
0 val_0
0 val_0
0 val_0
10 val_10
100 val_100
100 val_100
103 val_103
103 val_103
104 val_104
104 val_104
105 val_105
11 val_11
111 val_111
113 val_113
113 val_113
114 val_114
116 val_116
118 val_118
118 val_118
119 val_119
119 val_119
119 val_119
12 val_12
12 val_12
120 val_120
120 val_120
125 val_125
125 val_125
126 val_126
128 val_128
128 val_128
128 val_128
129 val_129
129 val_129
131 val_131
133 val_133
134 val_134
134 val_134
136 val_136
137 val_137
137 val_137
138 val_138
138 val_138
138 val_138
138 val_138
143 val_143
145 val_145
146 val_146
146 val_146
149 val_149
149 val_149
15 val_15
15 val_15
150 val_150
152 val_152
152 val_152
153 val_153
155 val_155
156 val_156
157 val_157
158 val_158
160 val_160
162 val_162
163 val_163
164 val_164
164 val_164
165 val_165
165 val_165
166 val_166
167 val_167
167 val_167
167 val_167
168 val_168
169 val_169
169 val_169
169 val_169
169 val_169
17 val_17
170 val_170
172 val_172
172 val_172
174 val_174
174 val_174
175 val_175
175 val_175
176 val_176
176 val_176
177 val_177
178 val_178
179 val_179
179 val_179
18 val_18
18 val_18
180 val_180
181 val_181
183 val_183
186 val_186
187 val_187
187 val_187
187 val_187
189 val_189
19 val_19
190 val_190
191 val_191
191 val_191
192 val_192
193 val_193
193 val_193
193 val_193
194 val_194
195 val_195
195 val_195
196 val_196
197 val_197
197 val_197
199 val_199
199 val_199
199 val_199
2 val_2
20 val_20
200 val_200
200 val_200
201 val_201
202 val_202
203 val_203
203 val_203
205 val_205
205 val_205
207 val_207
207 val_207
208 val_208
208 val_208
208 val_208
209 val_209
209 val_209
213 val_213
213 val_213
214 val_214
216 val_216
216 val_216
217 val_217
217 val_217
218 val_218
219 val_219
219 val_219
221 val_221
221 val_221
222 val_222
223 val_223
223 val_223
224 val_224
224 val_224
226 val_226
228 val_228
229 val_229
229 val_229
230 val_230
230 val_230
230 val_230
230 val_230
230 val_230
233 val_233
233 val_233
235 val_235
237 val_237
237 val_237
238 val_238
238 val_238
239 val_239
239 val_239
24 val_24
24 val_24
241 val_241
242 val_242
242 val_242
244 val_244
247 val_247
248 val_248
249 val_249
252 val_252
255 val_255
255 val_255
256 val_256
256 val_256
257 val_257
258 val_258
26 val_26
26 val_26
260 val_260
262 val_262
263 val_263
265 val_265
265 val_265
266 val_266
27 val_27
272 val_272
272 val_272
273 val_273
273 val_273
273 val_273
274 val_274
275 val_275
277 val_277
277 val_277
277 val_277
277 val_277
278 val_278
278 val_278
28 val_28
280 val_280
280 val_280
281 val_281
281 val_281
282 val_282
282 val_282
283 val_283
284 val_284
285 val_285
286 val_286
287 val_287
288 val_288
288 val_288
289 val_289
291 val_291
292 val_292
296 val_296
298 val_298
298 val_298
298 val_298
30 val_30
302 val_302
305 val_305
306 val_306
307 val_307
307 val_307
308 val_308
309 val_309
309 val_309
310 val_310
311 val_311
311 val_311
311 val_311
315 val_315
316 val_316
316 val_316
316 val_316
317 val_317
317 val_317
318 val_318
318 val_318
318 val_318
321 val_321
321 val_321
322 val_322
322 val_322
323 val_323
325 val_325
325 val_325
327 val_327
327 val_327
327 val_327
33 val_33
331 val_331
331 val_331
332 val_332
333 val_333
333 val_333
335 val_335
336 val_336
338 val_338
339 val_339
34 val_34
341 val_341
342 val_342
342 val_342
344 val_344
344 val_344
345 val_345
348 val_348
348 val_348
348 val_348
348 val_348
348 val_348
35 val_35
35 val_35
35 val_35
351 val_351
353 val_353
353 val_353
356 val_356
360 val_360
362 val_362
364 val_364
365 val_365
366 val_366
367 val_367
367 val_367
368 val_368
369 val_369
369 val_369
369 val_369
37 val_37
37 val_37
373 val_373
374 val_374
375 val_375
377 val_377
378 val_378
379 val_379
382 val_382
382 val_382
384 val_384
384 val_384
384 val_384
386 val_386
389 val_389
392 val_392
393 val_393
394 val_394
395 val_395
395 val_395
396 val_396
396 val_396
396 val_396
397 val_397
397 val_397
399 val_399
399 val_399
4 val_4
400 val_400
401 val_401
401 val_401
401 val_401
401 val_401
401 val_401
402 val_402
403 val_403
403 val_403
403 val_403
404 val_404
404 val_404
406 val_406
406 val_406
406 val_406
406 val_406
407 val_407
409 val_409
409 val_409
409 val_409
41 val_41
411 val_411
413 val_413
413 val_413
414 val_414
414 val_414
417 val_417
417 val_417
417 val_417
418 val_418
419 val_419
42 val_42
42 val_42
421 val_421
424 val_424
424 val_424
427 val_427
429 val_429
429 val_429
43 val_43
430 val_430
430 val_430
430 val_430
431 val_431
431 val_431
431 val_431
432 val_432
435 val_435
436 val_436
437 val_437
438 val_438
438 val_438
438 val_438
439 val_439
439 val_439
44 val_44
443 val_443
444 val_444
446 val_446
448 val_448
449 val_449
452 val_452
453 val_453
454 val_454
454 val_454
454 val_454
455 val_455
457 val_457
458 val_458
458 val_458
459 val_459
459 val_459
460 val_460
462 val_462
462 val_462
463 val_463
463 val_463
466 val_466
466 val_466
466 val_466
467 val_467
468 val_468
468 val_468
468 val_468
468 val_468
469 val_469
469 val_469
469 val_469
469 val_469
469 val_469
47 val_47
470 val_470
472 val_472
475 val_475
477 val_477
478 val_478
478 val_478
479 val_479
480 val_480
480 val_480
480 val_480
481 val_481
482 val_482
483 val_483
484 val_484
485 val_485
487 val_487
489 val_489
489 val_489
489 val_489
489 val_489
490 val_490
491 val_491
492 val_492
492 val_492
493 val_493
494 val_494
495 val_495
496 val_496
497 val_497
498 val_498
498 val_498
498 val_498
5 val_5
5 val_5
5 val_5
51 val_51
51 val_51
53 val_53
54 val_54
57 val_57
58 val_58
58 val_58
64 val_64
65 val_65
66 val_66
67 val_67
67 val_67
69 val_69
70 val_70
70 val_70
70 val_70
72 val_72
72 val_72
74 val_74
76 val_76
76 val_76
77 val_77
78 val_78
8 val_8
80 val_80
82 val_82
83 val_83
83 val_83
84 val_84
84 val_84
85 val_85
86 val_86
87 val_87
9 val_9
90 val_90
90 val_90
90 val_90
92 val_92
95 val_95
95 val_95
96 val_96
97 val_97
97 val_97
98 val_98
98 val_98