| PREHOOK: query: DESCRIBE FUNCTION explode |
| PREHOOK: type: DESCFUNCTION |
| POSTHOOK: query: DESCRIBE FUNCTION explode |
| POSTHOOK: type: DESCFUNCTION |
| explode(a) - separates the elements of array a into multiple rows, or the elements of a map into multiple rows and columns |
| PREHOOK: query: DESCRIBE FUNCTION EXTENDED explode |
| PREHOOK: type: DESCFUNCTION |
| POSTHOOK: query: DESCRIBE FUNCTION EXTENDED explode |
| POSTHOOK: type: DESCFUNCTION |
| explode(a) - separates the elements of array a into multiple rows, or the elements of a map into multiple rows and columns |
| Function class:org.apache.hadoop.hive.ql.udf.generic.GenericUDTFExplode |
| Function type:BUILTIN |
| PREHOOK: query: EXPLAIN EXTENDED SELECT explode(array(1,2,3)) AS myCol FROM src tablesample (1 rows) |
| PREHOOK: type: QUERY |
| POSTHOOK: query: EXPLAIN EXTENDED SELECT explode(array(1,2,3)) AS myCol FROM src tablesample (1 rows) |
| POSTHOOK: type: QUERY |
| STAGE DEPENDENCIES: |
| Stage-0 is a root stage |
| |
| STAGE PLANS: |
| Stage: Stage-0 |
| Fetch Operator |
| limit: -1 |
| Processor Tree: |
| TableScan |
| alias: src |
| Row Limit Per Split: 1 |
| Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE |
| GatherStats: false |
| Select Operator |
| expressions: array(1,2,3) (type: array<int>) |
| outputColumnNames: _col0 |
| Statistics: Num rows: 500 Data size: 28000 Basic stats: COMPLETE Column stats: COMPLETE |
| UDTF Operator |
| Statistics: Num rows: 500 Data size: 28000 Basic stats: COMPLETE Column stats: COMPLETE |
| function name: explode |
| ListSink |
| |
| PREHOOK: query: EXPLAIN EXTENDED SELECT a.myCol, count(1) FROM (SELECT explode(array(1,2,3)) AS myCol FROM src tablesample (1 rows)) a GROUP BY a.myCol |
| PREHOOK: type: QUERY |
| POSTHOOK: query: EXPLAIN EXTENDED SELECT a.myCol, count(1) FROM (SELECT explode(array(1,2,3)) AS myCol FROM src tablesample (1 rows)) a GROUP BY a.myCol |
| POSTHOOK: type: QUERY |
| STAGE DEPENDENCIES: |
| Stage-1 is a root stage |
| Stage-0 depends on stages: Stage-1 |
| |
| STAGE PLANS: |
| Stage: Stage-1 |
| Map Reduce |
| Map Operator Tree: |
| TableScan |
| alias: src |
| Row Limit Per Split: 1 |
| Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE |
| GatherStats: false |
| Select Operator |
| expressions: array(1,2,3) (type: array<int>) |
| outputColumnNames: _col0 |
| Statistics: Num rows: 500 Data size: 28000 Basic stats: COMPLETE Column stats: COMPLETE |
| UDTF Operator |
| Statistics: Num rows: 500 Data size: 28000 Basic stats: COMPLETE Column stats: COMPLETE |
| function name: explode |
| Group By Operator |
| aggregations: count(1) |
| keys: col (type: int) |
| mode: hash |
| outputColumnNames: _col0, _col1 |
| Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE |
| Reduce Output Operator |
| key expressions: _col0 (type: int) |
| null sort order: a |
| sort order: + |
| Map-reduce partition columns: _col0 (type: int) |
| Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE |
| tag: -1 |
| value expressions: _col1 (type: bigint) |
| auto parallelism: false |
| Path -> Alias: |
| #### A masked pattern was here #### |
| Path -> Partition: |
| #### A masked pattern was here #### |
| Partition |
| base file name: src |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}} |
| bucket_count -1 |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.comments 'default','default' |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.src |
| numFiles 1 |
| numRows 500 |
| rawDataSize 5312 |
| serialization.ddl struct src { string key, string value} |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| totalSize 5812 |
| #### A masked pattern was here #### |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}} |
| bucket_count -1 |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.comments 'default','default' |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.src |
| numFiles 1 |
| numRows 500 |
| rawDataSize 5312 |
| serialization.ddl struct src { string key, string value} |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| totalSize 5812 |
| #### A masked pattern was here #### |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| name: default.src |
| name: default.src |
| Truncated Path -> Alias: |
| /src [a:src] |
| Needs Tagging: false |
| Reduce Operator Tree: |
| Group By Operator |
| aggregations: count(VALUE._col0) |
| keys: KEY._col0 (type: int) |
| mode: mergepartial |
| outputColumnNames: _col0, _col1 |
| Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE |
| File Output Operator |
| compressed: false |
| GlobalTableId: 0 |
| #### A masked pattern was here #### |
| NumFilesPerFileSink: 1 |
| Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE |
| #### A masked pattern was here #### |
| table: |
| input format: org.apache.hadoop.mapred.SequenceFileInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat |
| properties: |
| columns _col0,_col1 |
| columns.types int:bigint |
| escape.delim \ |
| hive.serialization.extend.additional.nesting.levels true |
| serialization.escape.crlf true |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| TotalFiles: 1 |
| GatherStats: false |
| MultiFileSpray: false |
| |
| Stage: Stage-0 |
| Fetch Operator |
| limit: -1 |
| Processor Tree: |
| ListSink |
| |
| PREHOOK: query: SELECT explode(array(1,2,3)) AS myCol FROM src tablesample (1 rows) |
| PREHOOK: type: QUERY |
| PREHOOK: Input: default@src |
| #### A masked pattern was here #### |
| POSTHOOK: query: SELECT explode(array(1,2,3)) AS myCol FROM src tablesample (1 rows) |
| POSTHOOK: type: QUERY |
| POSTHOOK: Input: default@src |
| #### A masked pattern was here #### |
| 1 |
| 2 |
| 3 |
| PREHOOK: query: SELECT explode(array(1,2,3)) AS (myCol) FROM src tablesample (1 rows) |
| PREHOOK: type: QUERY |
| PREHOOK: Input: default@src |
| #### A masked pattern was here #### |
| POSTHOOK: query: SELECT explode(array(1,2,3)) AS (myCol) FROM src tablesample (1 rows) |
| POSTHOOK: type: QUERY |
| POSTHOOK: Input: default@src |
| #### A masked pattern was here #### |
| 1 |
| 2 |
| 3 |
| PREHOOK: query: SELECT a.myCol, count(1) FROM (SELECT explode(array(1,2,3)) AS myCol FROM src tablesample (1 rows)) a GROUP BY a.myCol |
| PREHOOK: type: QUERY |
| PREHOOK: Input: default@src |
| #### A masked pattern was here #### |
| POSTHOOK: query: SELECT a.myCol, count(1) FROM (SELECT explode(array(1,2,3)) AS myCol FROM src tablesample (1 rows)) a GROUP BY a.myCol |
| POSTHOOK: type: QUERY |
| POSTHOOK: Input: default@src |
| #### A masked pattern was here #### |
| 1 1 |
| 2 1 |
| 3 1 |
| PREHOOK: query: EXPLAIN EXTENDED SELECT explode(map(1,'one',2,'two',3,'three')) AS (key,val) FROM src tablesample (1 rows) |
| PREHOOK: type: QUERY |
| POSTHOOK: query: EXPLAIN EXTENDED SELECT explode(map(1,'one',2,'two',3,'three')) AS (key,val) FROM src tablesample (1 rows) |
| POSTHOOK: type: QUERY |
| STAGE DEPENDENCIES: |
| Stage-0 is a root stage |
| |
| STAGE PLANS: |
| Stage: Stage-0 |
| Fetch Operator |
| limit: -1 |
| Processor Tree: |
| TableScan |
| alias: src |
| Row Limit Per Split: 1 |
| Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE |
| GatherStats: false |
| Select Operator |
| expressions: map(1:'one',2:'two',3:'three') (type: map<int,string>) |
| outputColumnNames: _col0 |
| Statistics: Num rows: 500 Data size: 259500 Basic stats: COMPLETE Column stats: COMPLETE |
| UDTF Operator |
| Statistics: Num rows: 500 Data size: 259500 Basic stats: COMPLETE Column stats: COMPLETE |
| function name: explode |
| ListSink |
| |
| PREHOOK: query: EXPLAIN EXTENDED SELECT a.key, a.val, count(1) FROM (SELECT explode(map(1,'one',2,'two',3,'three')) AS (key,val) FROM src tablesample (1 rows)) a GROUP BY a.key, a.val |
| PREHOOK: type: QUERY |
| POSTHOOK: query: EXPLAIN EXTENDED SELECT a.key, a.val, count(1) FROM (SELECT explode(map(1,'one',2,'two',3,'three')) AS (key,val) FROM src tablesample (1 rows)) a GROUP BY a.key, a.val |
| POSTHOOK: type: QUERY |
| STAGE DEPENDENCIES: |
| Stage-1 is a root stage |
| Stage-0 depends on stages: Stage-1 |
| |
| STAGE PLANS: |
| Stage: Stage-1 |
| Map Reduce |
| Map Operator Tree: |
| TableScan |
| alias: src |
| Row Limit Per Split: 1 |
| Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE |
| GatherStats: false |
| Select Operator |
| expressions: map(1:'one',2:'two',3:'three') (type: map<int,string>) |
| outputColumnNames: _col0 |
| Statistics: Num rows: 500 Data size: 259500 Basic stats: COMPLETE Column stats: COMPLETE |
| UDTF Operator |
| Statistics: Num rows: 500 Data size: 259500 Basic stats: COMPLETE Column stats: COMPLETE |
| function name: explode |
| Group By Operator |
| aggregations: count(1) |
| keys: key (type: int), value (type: string) |
| mode: hash |
| outputColumnNames: _col0, _col1, _col2 |
| Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE |
| Reduce Output Operator |
| key expressions: _col0 (type: int), _col1 (type: string) |
| null sort order: aa |
| sort order: ++ |
| Map-reduce partition columns: _col0 (type: int), _col1 (type: string) |
| Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE |
| tag: -1 |
| value expressions: _col2 (type: bigint) |
| auto parallelism: false |
| Path -> Alias: |
| #### A masked pattern was here #### |
| Path -> Partition: |
| #### A masked pattern was here #### |
| Partition |
| base file name: src |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}} |
| bucket_count -1 |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.comments 'default','default' |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.src |
| numFiles 1 |
| numRows 500 |
| rawDataSize 5312 |
| serialization.ddl struct src { string key, string value} |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| totalSize 5812 |
| #### A masked pattern was here #### |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}} |
| bucket_count -1 |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.comments 'default','default' |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.src |
| numFiles 1 |
| numRows 500 |
| rawDataSize 5312 |
| serialization.ddl struct src { string key, string value} |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| totalSize 5812 |
| #### A masked pattern was here #### |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| name: default.src |
| name: default.src |
| Truncated Path -> Alias: |
| /src [a:src] |
| Needs Tagging: false |
| Reduce Operator Tree: |
| Group By Operator |
| aggregations: count(VALUE._col0) |
| keys: KEY._col0 (type: int), KEY._col1 (type: string) |
| mode: mergepartial |
| outputColumnNames: _col0, _col1, _col2 |
| Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE |
| File Output Operator |
| compressed: false |
| GlobalTableId: 0 |
| #### A masked pattern was here #### |
| NumFilesPerFileSink: 1 |
| Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE |
| #### A masked pattern was here #### |
| table: |
| input format: org.apache.hadoop.mapred.SequenceFileInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat |
| properties: |
| columns _col0,_col1,_col2 |
| columns.types int:string:bigint |
| escape.delim \ |
| hive.serialization.extend.additional.nesting.levels true |
| serialization.escape.crlf true |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| TotalFiles: 1 |
| GatherStats: false |
| MultiFileSpray: false |
| |
| Stage: Stage-0 |
| Fetch Operator |
| limit: -1 |
| Processor Tree: |
| ListSink |
| |
| PREHOOK: query: SELECT explode(map(1,'one',2,'two',3,'three')) AS (key,val) FROM src tablesample (1 rows) |
| PREHOOK: type: QUERY |
| PREHOOK: Input: default@src |
| #### A masked pattern was here #### |
| POSTHOOK: query: SELECT explode(map(1,'one',2,'two',3,'three')) AS (key,val) FROM src tablesample (1 rows) |
| POSTHOOK: type: QUERY |
| POSTHOOK: Input: default@src |
| #### A masked pattern was here #### |
| 1 one |
| 2 two |
| 3 three |
| PREHOOK: query: SELECT a.key, a.val, count(1) FROM (SELECT explode(map(1,'one',2,'two',3,'three')) AS (key,val) FROM src tablesample (1 rows)) a GROUP BY a.key, a.val |
| PREHOOK: type: QUERY |
| PREHOOK: Input: default@src |
| #### A masked pattern was here #### |
| POSTHOOK: query: SELECT a.key, a.val, count(1) FROM (SELECT explode(map(1,'one',2,'two',3,'three')) AS (key,val) FROM src tablesample (1 rows)) a GROUP BY a.key, a.val |
| POSTHOOK: type: QUERY |
| POSTHOOK: Input: default@src |
| #### A masked pattern was here #### |
| 1 one 1 |
| 2 two 1 |
| 3 three 1 |
| PREHOOK: query: drop table lazy_array_map |
| PREHOOK: type: DROPTABLE |
| POSTHOOK: query: drop table lazy_array_map |
| POSTHOOK: type: DROPTABLE |
| PREHOOK: query: create table lazy_array_map (map_col map<int,string>, array_col array<string>) |
| PREHOOK: type: CREATETABLE |
| PREHOOK: Output: database:default |
| PREHOOK: Output: default@lazy_array_map |
| POSTHOOK: query: create table lazy_array_map (map_col map<int,string>, array_col array<string>) |
| POSTHOOK: type: CREATETABLE |
| POSTHOOK: Output: database:default |
| POSTHOOK: Output: default@lazy_array_map |
| PREHOOK: query: INSERT OVERWRITE TABLE lazy_array_map select map(1,'one',2,'two',3,'three'), array('100','200','300') FROM src tablesample (1 rows) |
| PREHOOK: type: QUERY |
| PREHOOK: Input: default@src |
| PREHOOK: Output: default@lazy_array_map |
| POSTHOOK: query: INSERT OVERWRITE TABLE lazy_array_map select map(1,'one',2,'two',3,'three'), array('100','200','300') FROM src tablesample (1 rows) |
| POSTHOOK: type: QUERY |
| POSTHOOK: Input: default@src |
| POSTHOOK: Output: default@lazy_array_map |
| POSTHOOK: Lineage: lazy_array_map.array_col EXPRESSION [] |
| POSTHOOK: Lineage: lazy_array_map.map_col EXPRESSION [] |
| PREHOOK: query: SELECT array_col, myCol from lazy_array_map lateral view explode(array_col) X AS myCol |
| PREHOOK: type: QUERY |
| PREHOOK: Input: default@lazy_array_map |
| #### A masked pattern was here #### |
| POSTHOOK: query: SELECT array_col, myCol from lazy_array_map lateral view explode(array_col) X AS myCol |
| POSTHOOK: type: QUERY |
| POSTHOOK: Input: default@lazy_array_map |
| #### A masked pattern was here #### |
| ["100","200","300"] 100 |
| ["100","200","300"] 200 |
| ["100","200","300"] 300 |
| PREHOOK: query: SELECT map_col, myKey, myValue from lazy_array_map lateral view explode(map_col) X AS myKey, myValue |
| PREHOOK: type: QUERY |
| PREHOOK: Input: default@lazy_array_map |
| #### A masked pattern was here #### |
| POSTHOOK: query: SELECT map_col, myKey, myValue from lazy_array_map lateral view explode(map_col) X AS myKey, myValue |
| POSTHOOK: type: QUERY |
| POSTHOOK: Input: default@lazy_array_map |
| #### A masked pattern was here #### |
| {1:"one",2:"two",3:"three"} 1 one |
| {1:"one",2:"two",3:"three"} 2 two |
| {1:"one",2:"two",3:"three"} 3 three |
