blob: 0e1d3c68d6d95f577784214b7b2f06547b063925 [file] [log] [blame]
PREHOOK: query: create table studenttab10k (name string, age int, gpa double)
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@studenttab10k
POSTHOOK: query: create table studenttab10k (name string, age int, gpa double)
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@studenttab10k
PREHOOK: query: create table votertab10k (name string, age int, registration string, contributions float)
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@votertab10k
POSTHOOK: query: create table votertab10k (name string, age int, registration string, contributions float)
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@votertab10k
PREHOOK: query: explain select s.name, count(distinct registration)
from studenttab10k s join votertab10k v
on (s.name = v.name)
group by s.name
PREHOOK: type: QUERY
POSTHOOK: query: explain select s.name, count(distinct registration)
from studenttab10k s join votertab10k v
on (s.name = v.name)
group by s.name
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-5 is a root stage
Stage-2 depends on stages: Stage-5
Stage-0 depends on stages: Stage-2
STAGE PLANS:
Stage: Stage-5
Map Reduce Local Work
Alias -> Map Local Tables:
$hdt$_0:s
Fetch Operator
limit: -1
Alias -> Map Local Operator Tree:
$hdt$_0:s
TableScan
alias: s
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
predicate: name is not null (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: name (type: string)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
HashTable Sink Operator
keys:
0 _col0 (type: string)
1 _col0 (type: string)
Stage: Stage-2
Map Reduce
Map Operator Tree:
TableScan
alias: v
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
predicate: name is not null (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: name (type: string), registration (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col2
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Group By Operator
aggregations: count(DISTINCT _col2)
keys: _col0 (type: string), _col2 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Local Work:
Map Reduce Local Work
Reduce Operator Tree:
Group By Operator
aggregations: count(DISTINCT KEY._col1:0._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select s.name, count(distinct registration)
from studenttab10k s join votertab10k v
on (s.name = v.name)
group by s.name
PREHOOK: type: QUERY
PREHOOK: Input: default@studenttab10k
PREHOOK: Input: default@votertab10k
#### A masked pattern was here ####
POSTHOOK: query: select s.name, count(distinct registration)
from studenttab10k s join votertab10k v
on (s.name = v.name)
group by s.name
POSTHOOK: type: QUERY
POSTHOOK: Input: default@studenttab10k
POSTHOOK: Input: default@votertab10k
#### A masked pattern was here ####
PREHOOK: query: create table studenttab10k_smb (name string, age int, gpa double) clustered by (name) sorted by (name) into 2 buckets
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@studenttab10k_smb
POSTHOOK: query: create table studenttab10k_smb (name string, age int, gpa double) clustered by (name) sorted by (name) into 2 buckets
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@studenttab10k_smb
PREHOOK: query: create table votertab10k_smb (name string, age int, registration string, contributions float) clustered by (name) sorted by (name) into 2 buckets
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@votertab10k_smb
POSTHOOK: query: create table votertab10k_smb (name string, age int, registration string, contributions float) clustered by (name) sorted by (name) into 2 buckets
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@votertab10k_smb
PREHOOK: query: explain select s.name, count(distinct registration)
from studenttab10k_smb s join votertab10k_smb v
on (s.name = v.name)
group by s.name
PREHOOK: type: QUERY
POSTHOOK: query: explain select s.name, count(distinct registration)
from studenttab10k_smb s join votertab10k_smb v
on (s.name = v.name)
group by s.name
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
TableScan
alias: s
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
predicate: name is not null (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: name (type: string)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Sorted Merge Bucket Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col2
Group By Operator
aggregations: count(DISTINCT _col2)
keys: _col0 (type: string), _col2 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
Reduce Operator Tree:
Group By Operator
aggregations: count(DISTINCT KEY._col1:0._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
File Output Operator
compressed: false
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select s.name, count(distinct registration)
from studenttab10k_smb s join votertab10k_smb v
on (s.name = v.name)
group by s.name
PREHOOK: type: QUERY
PREHOOK: Input: default@studenttab10k_smb
PREHOOK: Input: default@votertab10k_smb
#### A masked pattern was here ####
POSTHOOK: query: select s.name, count(distinct registration)
from studenttab10k_smb s join votertab10k_smb v
on (s.name = v.name)
group by s.name
POSTHOOK: type: QUERY
POSTHOOK: Input: default@studenttab10k_smb
POSTHOOK: Input: default@votertab10k_smb
#### A masked pattern was here ####
PREHOOK: query: load data local inpath '../../data/files/empty/000000_0' into table studenttab10k_smb
PREHOOK: type: LOAD
#### A masked pattern was here ####
PREHOOK: Output: default@studenttab10k_smb
POSTHOOK: query: load data local inpath '../../data/files/empty/000000_0' into table studenttab10k_smb
POSTHOOK: type: LOAD
#### A masked pattern was here ####
POSTHOOK: Output: default@studenttab10k_smb
PREHOOK: query: load data local inpath '../../data/files/empty/000001_0' into table studenttab10k_smb
PREHOOK: type: LOAD
#### A masked pattern was here ####
PREHOOK: Output: default@studenttab10k_smb
POSTHOOK: query: load data local inpath '../../data/files/empty/000001_0' into table studenttab10k_smb
POSTHOOK: type: LOAD
#### A masked pattern was here ####
POSTHOOK: Output: default@studenttab10k_smb
PREHOOK: query: load data local inpath '../../data/files/empty/000000_0' into table votertab10k_smb
PREHOOK: type: LOAD
#### A masked pattern was here ####
PREHOOK: Output: default@votertab10k_smb
POSTHOOK: query: load data local inpath '../../data/files/empty/000000_0' into table votertab10k_smb
POSTHOOK: type: LOAD
#### A masked pattern was here ####
POSTHOOK: Output: default@votertab10k_smb
PREHOOK: query: load data local inpath '../../data/files/empty/000001_0' into table votertab10k_smb
PREHOOK: type: LOAD
#### A masked pattern was here ####
PREHOOK: Output: default@votertab10k_smb
POSTHOOK: query: load data local inpath '../../data/files/empty/000001_0' into table votertab10k_smb
POSTHOOK: type: LOAD
#### A masked pattern was here ####
POSTHOOK: Output: default@votertab10k_smb
PREHOOK: query: explain select s.name, count(distinct registration)
from studenttab10k_smb s join votertab10k_smb v
on (s.name = v.name)
group by s.name
PREHOOK: type: QUERY
POSTHOOK: query: explain select s.name, count(distinct registration)
from studenttab10k_smb s join votertab10k_smb v
on (s.name = v.name)
group by s.name
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
TableScan
alias: s
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
predicate: name is not null (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: name (type: string)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Sorted Merge Bucket Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col2
Group By Operator
aggregations: count(DISTINCT _col2)
keys: _col0 (type: string), _col2 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
Reduce Operator Tree:
Group By Operator
aggregations: count(DISTINCT KEY._col1:0._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
File Output Operator
compressed: false
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select s.name, count(distinct registration)
from studenttab10k_smb s join votertab10k_smb v
on (s.name = v.name)
group by s.name
PREHOOK: type: QUERY
PREHOOK: Input: default@studenttab10k_smb
PREHOOK: Input: default@votertab10k_smb
#### A masked pattern was here ####
POSTHOOK: query: select s.name, count(distinct registration)
from studenttab10k_smb s join votertab10k_smb v
on (s.name = v.name)
group by s.name
POSTHOOK: type: QUERY
POSTHOOK: Input: default@studenttab10k_smb
POSTHOOK: Input: default@votertab10k_smb
#### A masked pattern was here ####
PREHOOK: query: create table studenttab10k_part (name string, age int, gpa double) partitioned by (p string) clustered by (name) sorted by (name) into 2 buckets
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@studenttab10k_part
POSTHOOK: query: create table studenttab10k_part (name string, age int, gpa double) partitioned by (p string) clustered by (name) sorted by (name) into 2 buckets
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@studenttab10k_part
PREHOOK: query: create table votertab10k_part (name string, age int, registration string, contributions float) partitioned by (p string) clustered by (name) sorted by (name) into 2 buckets
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@votertab10k_part
POSTHOOK: query: create table votertab10k_part (name string, age int, registration string, contributions float) partitioned by (p string) clustered by (name) sorted by (name) into 2 buckets
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@votertab10k_part
PREHOOK: query: load data local inpath '../../data/files/empty/000000_0' into table studenttab10k_part partition (p='foo')
PREHOOK: type: LOAD
#### A masked pattern was here ####
PREHOOK: Output: default@studenttab10k_part
POSTHOOK: query: load data local inpath '../../data/files/empty/000000_0' into table studenttab10k_part partition (p='foo')
POSTHOOK: type: LOAD
#### A masked pattern was here ####
POSTHOOK: Output: default@studenttab10k_part
POSTHOOK: Output: default@studenttab10k_part@p=foo
PREHOOK: query: load data local inpath '../../data/files/empty/000001_0' into table studenttab10k_part partition (p='foo')
PREHOOK: type: LOAD
#### A masked pattern was here ####
PREHOOK: Output: default@studenttab10k_part@p=foo
POSTHOOK: query: load data local inpath '../../data/files/empty/000001_0' into table studenttab10k_part partition (p='foo')
POSTHOOK: type: LOAD
#### A masked pattern was here ####
POSTHOOK: Output: default@studenttab10k_part@p=foo
PREHOOK: query: load data local inpath '../../data/files/empty/000000_0' into table votertab10k_part partition (p='foo')
PREHOOK: type: LOAD
#### A masked pattern was here ####
PREHOOK: Output: default@votertab10k_part
POSTHOOK: query: load data local inpath '../../data/files/empty/000000_0' into table votertab10k_part partition (p='foo')
POSTHOOK: type: LOAD
#### A masked pattern was here ####
POSTHOOK: Output: default@votertab10k_part
POSTHOOK: Output: default@votertab10k_part@p=foo
PREHOOK: query: load data local inpath '../../data/files/empty/000001_0' into table votertab10k_part partition (p='foo')
PREHOOK: type: LOAD
#### A masked pattern was here ####
PREHOOK: Output: default@votertab10k_part@p=foo
POSTHOOK: query: load data local inpath '../../data/files/empty/000001_0' into table votertab10k_part partition (p='foo')
POSTHOOK: type: LOAD
#### A masked pattern was here ####
POSTHOOK: Output: default@votertab10k_part@p=foo
PREHOOK: query: explain select s.name, count(distinct registration)
from studenttab10k_part s join votertab10k_part v
on (s.name = v.name)
where s.p = 'bar'
and v.p = 'bar'
group by s.name
PREHOOK: type: QUERY
POSTHOOK: query: explain select s.name, count(distinct registration)
from studenttab10k_part s join votertab10k_part v
on (s.name = v.name)
where s.p = 'bar'
and v.p = 'bar'
group by s.name
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
TableScan
alias: s
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
predicate: ((p = 'bar') and name is not null) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: name (type: string)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Sorted Merge Bucket Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col3
Group By Operator
aggregations: count(DISTINCT _col3)
keys: _col0 (type: string), _col3 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
Reduce Operator Tree:
Group By Operator
aggregations: count(DISTINCT KEY._col1:0._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
File Output Operator
compressed: false
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
PREHOOK: query: select s.name, count(distinct registration)
from studenttab10k_part s join votertab10k_part v
on (s.name = v.name)
where s.p = 'bar'
and v.p = 'bar'
group by s.name
PREHOOK: type: QUERY
PREHOOK: Input: default@studenttab10k_part
PREHOOK: Input: default@votertab10k_part
#### A masked pattern was here ####
POSTHOOK: query: select s.name, count(distinct registration)
from studenttab10k_part s join votertab10k_part v
on (s.name = v.name)
where s.p = 'bar'
and v.p = 'bar'
group by s.name
POSTHOOK: type: QUERY
POSTHOOK: Input: default@studenttab10k_part
POSTHOOK: Input: default@votertab10k_part
#### A masked pattern was here ####
PREHOOK: query: drop table studenttab10k
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@studenttab10k
PREHOOK: Output: default@studenttab10k
POSTHOOK: query: drop table studenttab10k
POSTHOOK: type: DROPTABLE
POSTHOOK: Input: default@studenttab10k
POSTHOOK: Output: default@studenttab10k
PREHOOK: query: drop table votertab10k
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@votertab10k
PREHOOK: Output: default@votertab10k
POSTHOOK: query: drop table votertab10k
POSTHOOK: type: DROPTABLE
POSTHOOK: Input: default@votertab10k
POSTHOOK: Output: default@votertab10k
PREHOOK: query: drop table studenttab10k_smb
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@studenttab10k_smb
PREHOOK: Output: default@studenttab10k_smb
POSTHOOK: query: drop table studenttab10k_smb
POSTHOOK: type: DROPTABLE
POSTHOOK: Input: default@studenttab10k_smb
POSTHOOK: Output: default@studenttab10k_smb
PREHOOK: query: drop table votertab10k_smb
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@votertab10k_smb
PREHOOK: Output: default@votertab10k_smb
POSTHOOK: query: drop table votertab10k_smb
POSTHOOK: type: DROPTABLE
POSTHOOK: Input: default@votertab10k_smb
POSTHOOK: Output: default@votertab10k_smb
PREHOOK: query: drop table studenttab10k_part
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@studenttab10k_part
PREHOOK: Output: default@studenttab10k_part
POSTHOOK: query: drop table studenttab10k_part
POSTHOOK: type: DROPTABLE
POSTHOOK: Input: default@studenttab10k_part
POSTHOOK: Output: default@studenttab10k_part
PREHOOK: query: drop table votertab10k_part
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@votertab10k_part
PREHOOK: Output: default@votertab10k_part
POSTHOOK: query: drop table votertab10k_part
POSTHOOK: type: DROPTABLE
POSTHOOK: Input: default@votertab10k_part
POSTHOOK: Output: default@votertab10k_part