| PREHOOK: query: DROP TABLE IF EXISTS s |
| PREHOOK: type: DROPTABLE |
| POSTHOOK: query: DROP TABLE IF EXISTS s |
| POSTHOOK: type: DROPTABLE |
| PREHOOK: query: CREATE TABLE s (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE |
| PREHOOK: type: CREATETABLE |
| PREHOOK: Output: database:default |
| PREHOOK: Output: default@s |
| POSTHOOK: query: CREATE TABLE s (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE |
| POSTHOOK: type: CREATETABLE |
| POSTHOOK: Output: database:default |
| POSTHOOK: Output: default@s |
| PREHOOK: query: LOAD DATA LOCAL INPATH '../../data/files/kv1.txt' INTO TABLE s |
| PREHOOK: type: LOAD |
| #### A masked pattern was here #### |
| PREHOOK: Output: default@s |
| POSTHOOK: query: LOAD DATA LOCAL INPATH '../../data/files/kv1.txt' INTO TABLE s |
| POSTHOOK: type: LOAD |
| #### A masked pattern was here #### |
| POSTHOOK: Output: default@s |
| PREHOOK: query: desc formatted s |
| PREHOOK: type: DESCTABLE |
| PREHOOK: Input: default@s |
| POSTHOOK: query: desc formatted s |
| POSTHOOK: type: DESCTABLE |
| POSTHOOK: Input: default@s |
| # col_name data_type comment |
| key string default |
| value string default |
| |
| # Detailed Table Information |
| Database: default |
| #### A masked pattern was here #### |
| Retention: 0 |
| #### A masked pattern was here #### |
| Table Type: MANAGED_TABLE |
| Table Parameters: |
| bucketing_version 2 |
| numFiles 1 |
| numRows 0 |
| rawDataSize 0 |
| totalSize 5812 |
| #### A masked pattern was here #### |
| |
| # Storage Information |
| SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| InputFormat: org.apache.hadoop.mapred.TextInputFormat |
| OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| Compressed: No |
| Num Buckets: -1 |
| Bucket Columns: [] |
| Sort Columns: [] |
| Storage Desc Params: |
| serialization.format 1 |
| PREHOOK: query: explain extended analyze table s compute statistics for columns |
| PREHOOK: type: ANALYZE_TABLE |
| PREHOOK: Input: default@s |
| PREHOOK: Output: default@s |
| #### A masked pattern was here #### |
| POSTHOOK: query: explain extended analyze table s compute statistics for columns |
| POSTHOOK: type: ANALYZE_TABLE |
| POSTHOOK: Input: default@s |
| POSTHOOK: Output: default@s |
| #### A masked pattern was here #### |
| STAGE DEPENDENCIES: |
| Stage-0 is a root stage |
| Stage-2 depends on stages: Stage-0 |
| |
| STAGE PLANS: |
| Stage: Stage-0 |
| Tez |
| #### A masked pattern was here #### |
| Edges: |
| Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE) |
| #### A masked pattern was here #### |
| Vertices: |
| Map 1 |
| Map Operator Tree: |
| TableScan |
| alias: s |
| Statistics: Num rows: 1 Data size: 368 Basic stats: COMPLETE Column stats: NONE |
| Statistics Aggregation Key Prefix: default.s/ |
| GatherStats: true |
| Select Operator |
| expressions: key (type: string), value (type: string) |
| outputColumnNames: key, value |
| Statistics: Num rows: 1 Data size: 368 Basic stats: COMPLETE Column stats: NONE |
| Group By Operator |
| aggregations: max(length(key)), avg(COALESCE(length(key),0)), count(1), count(key), compute_bit_vector(key, 'hll'), max(length(value)), avg(COALESCE(length(value),0)), count(value), compute_bit_vector(value, 'hll') |
| minReductionHashAggr: 0.99 |
| mode: hash |
| outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8 |
| Statistics: Num rows: 1 Data size: 840 Basic stats: COMPLETE Column stats: NONE |
| Reduce Output Operator |
| bucketingVersion: 2 |
| null sort order: |
| numBuckets: -1 |
| sort order: |
| Statistics: Num rows: 1 Data size: 840 Basic stats: COMPLETE Column stats: NONE |
| tag: -1 |
| value expressions: _col0 (type: int), _col1 (type: struct<count:bigint,sum:double,input:int>), _col2 (type: bigint), _col3 (type: bigint), _col4 (type: binary), _col5 (type: int), _col6 (type: struct<count:bigint,sum:double,input:int>), _col7 (type: bigint), _col8 (type: binary) |
| auto parallelism: false |
| Execution mode: llap |
| LLAP IO: all inputs |
| Path -> Alias: |
| #### A masked pattern was here #### |
| Path -> Partition: |
| #### A masked pattern was here #### |
| Partition |
| base file name: s |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| bucket_count -1 |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.s |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.comments 'default','default' |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.s |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| name: default.s |
| name: default.s |
| Truncated Path -> Alias: |
| /s [s] |
| Reducer 2 |
| Execution mode: llap |
| Needs Tagging: false |
| Reduce Operator Tree: |
| Group By Operator |
| aggregations: max(VALUE._col0), avg(VALUE._col1), count(VALUE._col2), count(VALUE._col3), compute_bit_vector(VALUE._col4), max(VALUE._col5), avg(VALUE._col6), count(VALUE._col7), compute_bit_vector(VALUE._col8) |
| mode: mergepartial |
| outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8 |
| Statistics: Num rows: 1 Data size: 840 Basic stats: COMPLETE Column stats: NONE |
| Select Operator |
| expressions: 'STRING' (type: string), UDFToLong(COALESCE(_col0,0)) (type: bigint), COALESCE(_col1,0) (type: double), (_col2 - _col3) (type: bigint), COALESCE(ndv_compute_bit_vector(_col4),0) (type: bigint), _col4 (type: binary), 'STRING' (type: string), UDFToLong(COALESCE(_col5,0)) (type: bigint), COALESCE(_col6,0) (type: double), (_col2 - _col7) (type: bigint), COALESCE(ndv_compute_bit_vector(_col8),0) (type: bigint), _col8 (type: binary) |
| outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11 |
| Statistics: Num rows: 1 Data size: 840 Basic stats: COMPLETE Column stats: NONE |
| File Output Operator |
| bucketingVersion: 2 |
| compressed: false |
| GlobalTableId: 0 |
| #### A masked pattern was here #### |
| NumFilesPerFileSink: 1 |
| Statistics: Num rows: 1 Data size: 840 Basic stats: COMPLETE Column stats: NONE |
| #### A masked pattern was here #### |
| table: |
| input format: org.apache.hadoop.mapred.SequenceFileInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat |
| properties: |
| bucketing_version -1 |
| columns _col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8,_col9,_col10,_col11 |
| columns.types string:bigint:double:bigint:bigint:binary:string:bigint:double:bigint:bigint:binary |
| escape.delim \ |
| hive.serialization.extend.additional.nesting.levels true |
| serialization.escape.crlf true |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| TotalFiles: 1 |
| GatherStats: false |
| MultiFileSpray: false |
| |
| Stage: Stage-2 |
| Stats Work |
| Basic Stats Work: |
| Stats Aggregation Key Prefix: default.s/ |
| Column Stats Desc: |
| Columns: key, value |
| Column Types: string, string |
| Table: default.s |
| Is Table Level Stats: true |
| |
| PREHOOK: query: analyze table s compute statistics for columns |
| PREHOOK: type: ANALYZE_TABLE |
| PREHOOK: Input: default@s |
| PREHOOK: Output: default@s |
| #### A masked pattern was here #### |
| POSTHOOK: query: analyze table s compute statistics for columns |
| POSTHOOK: type: ANALYZE_TABLE |
| POSTHOOK: Input: default@s |
| POSTHOOK: Output: default@s |
| #### A masked pattern was here #### |
| PREHOOK: query: desc formatted s |
| PREHOOK: type: DESCTABLE |
| PREHOOK: Input: default@s |
| POSTHOOK: query: desc formatted s |
| POSTHOOK: type: DESCTABLE |
| POSTHOOK: Input: default@s |
| # col_name data_type comment |
| key string default |
| value string default |
| |
| # Detailed Table Information |
| Database: default |
| #### A masked pattern was here #### |
| Retention: 0 |
| #### A masked pattern was here #### |
| Table Type: MANAGED_TABLE |
| Table Parameters: |
| COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"key\":\"true\",\"value\":\"true\"}} |
| bucketing_version 2 |
| numFiles 1 |
| numRows 500 |
| rawDataSize 5312 |
| totalSize 5812 |
| #### A masked pattern was here #### |
| |
| # Storage Information |
| SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| InputFormat: org.apache.hadoop.mapred.TextInputFormat |
| OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| Compressed: No |
| Num Buckets: -1 |
| Bucket Columns: [] |
| Sort Columns: [] |
| Storage Desc Params: |
| serialization.format 1 |
| PREHOOK: query: DROP TABLE IF EXISTS spart |
| PREHOOK: type: DROPTABLE |
| POSTHOOK: query: DROP TABLE IF EXISTS spart |
| POSTHOOK: type: DROPTABLE |
| PREHOOK: query: CREATE TABLE spart (key STRING COMMENT 'default', value STRING COMMENT 'default') |
| PARTITIONED BY (ds STRING, hr STRING) |
| STORED AS TEXTFILE |
| PREHOOK: type: CREATETABLE |
| PREHOOK: Output: database:default |
| PREHOOK: Output: default@spart |
| POSTHOOK: query: CREATE TABLE spart (key STRING COMMENT 'default', value STRING COMMENT 'default') |
| PARTITIONED BY (ds STRING, hr STRING) |
| STORED AS TEXTFILE |
| POSTHOOK: type: CREATETABLE |
| POSTHOOK: Output: database:default |
| POSTHOOK: Output: default@spart |
| PREHOOK: query: LOAD DATA LOCAL INPATH "../../data/files/kv1.txt" |
| OVERWRITE INTO TABLE spart PARTITION (ds="2008-04-08", hr="11") |
| PREHOOK: type: LOAD |
| #### A masked pattern was here #### |
| PREHOOK: Output: default@spart |
| POSTHOOK: query: LOAD DATA LOCAL INPATH "../../data/files/kv1.txt" |
| OVERWRITE INTO TABLE spart PARTITION (ds="2008-04-08", hr="11") |
| POSTHOOK: type: LOAD |
| #### A masked pattern was here #### |
| POSTHOOK: Output: default@spart |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| PREHOOK: query: LOAD DATA LOCAL INPATH "../../data/files/kv1.txt" |
| OVERWRITE INTO TABLE spart PARTITION (ds="2008-04-08", hr="12") |
| PREHOOK: type: LOAD |
| #### A masked pattern was here #### |
| PREHOOK: Output: default@spart |
| POSTHOOK: query: LOAD DATA LOCAL INPATH "../../data/files/kv1.txt" |
| OVERWRITE INTO TABLE spart PARTITION (ds="2008-04-08", hr="12") |
| POSTHOOK: type: LOAD |
| #### A masked pattern was here #### |
| POSTHOOK: Output: default@spart |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=12 |
| PREHOOK: query: desc formatted spart |
| PREHOOK: type: DESCTABLE |
| PREHOOK: Input: default@spart |
| POSTHOOK: query: desc formatted spart |
| POSTHOOK: type: DESCTABLE |
| POSTHOOK: Input: default@spart |
| # col_name data_type comment |
| key string default |
| value string default |
| |
| # Partition Information |
| # col_name data_type comment |
| ds string |
| hr string |
| |
| # Detailed Table Information |
| Database: default |
| #### A masked pattern was here #### |
| Retention: 0 |
| #### A masked pattern was here #### |
| Table Type: MANAGED_TABLE |
| Table Parameters: |
| bucketing_version 2 |
| numFiles 2 |
| numPartitions 2 |
| numRows 0 |
| rawDataSize 0 |
| totalSize 11624 |
| #### A masked pattern was here #### |
| |
| # Storage Information |
| SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| InputFormat: org.apache.hadoop.mapred.TextInputFormat |
| OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| Compressed: No |
| Num Buckets: -1 |
| Bucket Columns: [] |
| Sort Columns: [] |
| Storage Desc Params: |
| serialization.format 1 |
| PREHOOK: query: explain extended analyze table spart compute statistics for columns |
| PREHOOK: type: ANALYZE_TABLE |
| PREHOOK: Input: default@spart |
| PREHOOK: Input: default@spart@ds=2008-04-08/hr=11 |
| PREHOOK: Input: default@spart@ds=2008-04-08/hr=12 |
| PREHOOK: Output: default@spart |
| PREHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| PREHOOK: Output: default@spart@ds=2008-04-08/hr=12 |
| #### A masked pattern was here #### |
| POSTHOOK: query: explain extended analyze table spart compute statistics for columns |
| POSTHOOK: type: ANALYZE_TABLE |
| POSTHOOK: Input: default@spart |
| POSTHOOK: Input: default@spart@ds=2008-04-08/hr=11 |
| POSTHOOK: Input: default@spart@ds=2008-04-08/hr=12 |
| POSTHOOK: Output: default@spart |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=12 |
| #### A masked pattern was here #### |
| STAGE DEPENDENCIES: |
| Stage-0 is a root stage |
| Stage-2 depends on stages: Stage-0 |
| |
| STAGE PLANS: |
| Stage: Stage-0 |
| Tez |
| #### A masked pattern was here #### |
| Edges: |
| Reducer 2 <- Map 1 (SIMPLE_EDGE) |
| #### A masked pattern was here #### |
| Vertices: |
| Map 1 |
| Map Operator Tree: |
| TableScan |
| alias: spart |
| Statistics: Num rows: 196 Data size: 257552 Basic stats: PARTIAL Column stats: PARTIAL |
| Statistics Aggregation Key Prefix: default.spart/ |
| GatherStats: true |
| Select Operator |
| expressions: key (type: string), value (type: string), ds (type: string), hr (type: string) |
| outputColumnNames: key, value, ds, hr |
| Statistics: Num rows: 196 Data size: 257552 Basic stats: PARTIAL Column stats: PARTIAL |
| Group By Operator |
| aggregations: max(length(key)), avg(COALESCE(length(key),0)), count(1), count(key), compute_bit_vector(key, 'hll'), max(length(value)), avg(COALESCE(length(value),0)), count(value), compute_bit_vector(value, 'hll') |
| keys: ds (type: string), hr (type: string) |
| minReductionHashAggr: 0.99 |
| mode: hash |
| outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10 |
| Statistics: Num rows: 196 Data size: 233824 Basic stats: PARTIAL Column stats: PARTIAL |
| Reduce Output Operator |
| bucketingVersion: 2 |
| key expressions: _col0 (type: string), _col1 (type: string) |
| null sort order: zz |
| numBuckets: -1 |
| sort order: ++ |
| Map-reduce partition columns: _col0 (type: string), _col1 (type: string) |
| Statistics: Num rows: 196 Data size: 233824 Basic stats: PARTIAL Column stats: PARTIAL |
| tag: -1 |
| value expressions: _col2 (type: int), _col3 (type: struct<count:bigint,sum:double,input:int>), _col4 (type: bigint), _col5 (type: bigint), _col6 (type: binary), _col7 (type: int), _col8 (type: struct<count:bigint,sum:double,input:int>), _col9 (type: bigint), _col10 (type: binary) |
| auto parallelism: true |
| Execution mode: llap |
| LLAP IO: all inputs |
| Path -> Alias: |
| #### A masked pattern was here #### |
| Path -> Partition: |
| #### A masked pattern was here #### |
| Partition |
| base file name: hr=11 |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| partition values: |
| ds 2008-04-08 |
| hr 11 |
| properties: |
| column.name.delimiter , |
| columns key,value |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.spart |
| partition_columns ds/hr |
| partition_columns.types string:string |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.comments 'default','default' |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.spart |
| partition_columns ds/hr |
| partition_columns.types string:string |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| name: default.spart |
| name: default.spart |
| #### A masked pattern was here #### |
| Partition |
| base file name: hr=12 |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| partition values: |
| ds 2008-04-08 |
| hr 12 |
| properties: |
| column.name.delimiter , |
| columns key,value |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.spart |
| partition_columns ds/hr |
| partition_columns.types string:string |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.comments 'default','default' |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.spart |
| partition_columns ds/hr |
| partition_columns.types string:string |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| name: default.spart |
| name: default.spart |
| Truncated Path -> Alias: |
| /spart/ds=2008-04-08/hr=11 [spart] |
| /spart/ds=2008-04-08/hr=12 [spart] |
| Reducer 2 |
| Execution mode: llap |
| Needs Tagging: false |
| Reduce Operator Tree: |
| Group By Operator |
| aggregations: max(VALUE._col0), avg(VALUE._col1), count(VALUE._col2), count(VALUE._col3), compute_bit_vector(VALUE._col4), max(VALUE._col5), avg(VALUE._col6), count(VALUE._col7), compute_bit_vector(VALUE._col8) |
| keys: KEY._col0 (type: string), KEY._col1 (type: string) |
| mode: mergepartial |
| outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10 |
| Statistics: Num rows: 98 Data size: 115440 Basic stats: PARTIAL Column stats: PARTIAL |
| Select Operator |
| expressions: 'STRING' (type: string), UDFToLong(COALESCE(_col2,0)) (type: bigint), COALESCE(_col3,0) (type: double), (_col4 - _col5) (type: bigint), COALESCE(ndv_compute_bit_vector(_col6),0) (type: bigint), _col6 (type: binary), 'STRING' (type: string), UDFToLong(COALESCE(_col7,0)) (type: bigint), COALESCE(_col8,0) (type: double), (_col4 - _col9) (type: bigint), COALESCE(ndv_compute_bit_vector(_col10),0) (type: bigint), _col10 (type: binary), _col0 (type: string), _col1 (type: string) |
| outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13 |
| Statistics: Num rows: 98 Data size: 115440 Basic stats: PARTIAL Column stats: PARTIAL |
| File Output Operator |
| bucketingVersion: 2 |
| compressed: false |
| GlobalTableId: 0 |
| #### A masked pattern was here #### |
| NumFilesPerFileSink: 1 |
| Statistics: Num rows: 98 Data size: 115440 Basic stats: PARTIAL Column stats: PARTIAL |
| #### A masked pattern was here #### |
| table: |
| input format: org.apache.hadoop.mapred.SequenceFileInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat |
| properties: |
| bucketing_version -1 |
| columns _col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8,_col9,_col10,_col11,_col12,_col13 |
| columns.types string:bigint:double:bigint:bigint:binary:string:bigint:double:bigint:bigint:binary:string:string |
| escape.delim \ |
| hive.serialization.extend.additional.nesting.levels true |
| serialization.escape.crlf true |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| TotalFiles: 1 |
| GatherStats: false |
| MultiFileSpray: false |
| |
| Stage: Stage-2 |
| Stats Work |
| Basic Stats Work: |
| Stats Aggregation Key Prefix: default.spart/ |
| Column Stats Desc: |
| Columns: key, value |
| Column Types: string, string |
| Table: default.spart |
| Is Table Level Stats: false |
| |
| PREHOOK: query: analyze table spart compute statistics for columns |
| PREHOOK: type: ANALYZE_TABLE |
| PREHOOK: Input: default@spart |
| PREHOOK: Input: default@spart@ds=2008-04-08/hr=11 |
| PREHOOK: Input: default@spart@ds=2008-04-08/hr=12 |
| PREHOOK: Output: default@spart |
| PREHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| PREHOOK: Output: default@spart@ds=2008-04-08/hr=12 |
| #### A masked pattern was here #### |
| POSTHOOK: query: analyze table spart compute statistics for columns |
| POSTHOOK: type: ANALYZE_TABLE |
| POSTHOOK: Input: default@spart |
| POSTHOOK: Input: default@spart@ds=2008-04-08/hr=11 |
| POSTHOOK: Input: default@spart@ds=2008-04-08/hr=12 |
| POSTHOOK: Output: default@spart |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=12 |
| #### A masked pattern was here #### |
| PREHOOK: query: desc formatted spart |
| PREHOOK: type: DESCTABLE |
| PREHOOK: Input: default@spart |
| POSTHOOK: query: desc formatted spart |
| POSTHOOK: type: DESCTABLE |
| POSTHOOK: Input: default@spart |
| # col_name data_type comment |
| key string default |
| value string default |
| |
| # Partition Information |
| # col_name data_type comment |
| ds string |
| hr string |
| |
| # Detailed Table Information |
| Database: default |
| #### A masked pattern was here #### |
| Retention: 0 |
| #### A masked pattern was here #### |
| Table Type: MANAGED_TABLE |
| Table Parameters: |
| COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} |
| bucketing_version 2 |
| numFiles 2 |
| numPartitions 2 |
| numRows 1000 |
| rawDataSize 10624 |
| totalSize 11624 |
| #### A masked pattern was here #### |
| |
| # Storage Information |
| SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| InputFormat: org.apache.hadoop.mapred.TextInputFormat |
| OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| Compressed: No |
| Num Buckets: -1 |
| Bucket Columns: [] |
| Sort Columns: [] |
| Storage Desc Params: |
| serialization.format 1 |
| PREHOOK: query: desc formatted spart PARTITION(ds='2008-04-08', hr=11) |
| PREHOOK: type: DESCTABLE |
| PREHOOK: Input: default@spart |
| POSTHOOK: query: desc formatted spart PARTITION(ds='2008-04-08', hr=11) |
| POSTHOOK: type: DESCTABLE |
| POSTHOOK: Input: default@spart |
| # col_name data_type comment |
| key string default |
| value string default |
| |
| # Partition Information |
| # col_name data_type comment |
| ds string |
| hr string |
| |
| # Detailed Partition Information |
| Partition Value: [2008-04-08, 11] |
| Database: default |
| Table: spart |
| #### A masked pattern was here #### |
| Partition Parameters: |
| COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"key\":\"true\",\"value\":\"true\"}} |
| numFiles 1 |
| numRows 500 |
| rawDataSize 5312 |
| totalSize 5812 |
| #### A masked pattern was here #### |
| |
| # Storage Information |
| SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| InputFormat: org.apache.hadoop.mapred.TextInputFormat |
| OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| Compressed: No |
| Num Buckets: -1 |
| Bucket Columns: [] |
| Sort Columns: [] |
| Storage Desc Params: |
| serialization.format 1 |
| PREHOOK: query: desc formatted spart PARTITION(ds='2008-04-08', hr=12) |
| PREHOOK: type: DESCTABLE |
| PREHOOK: Input: default@spart |
| POSTHOOK: query: desc formatted spart PARTITION(ds='2008-04-08', hr=12) |
| POSTHOOK: type: DESCTABLE |
| POSTHOOK: Input: default@spart |
| # col_name data_type comment |
| key string default |
| value string default |
| |
| # Partition Information |
| # col_name data_type comment |
| ds string |
| hr string |
| |
| # Detailed Partition Information |
| Partition Value: [2008-04-08, 12] |
| Database: default |
| Table: spart |
| #### A masked pattern was here #### |
| Partition Parameters: |
| COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"key\":\"true\",\"value\":\"true\"}} |
| numFiles 1 |
| numRows 500 |
| rawDataSize 5312 |
| totalSize 5812 |
| #### A masked pattern was here #### |
| |
| # Storage Information |
| SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| InputFormat: org.apache.hadoop.mapred.TextInputFormat |
| OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| Compressed: No |
| Num Buckets: -1 |
| Bucket Columns: [] |
| Sort Columns: [] |
| Storage Desc Params: |
| serialization.format 1 |
| PREHOOK: query: DROP TABLE IF EXISTS spart |
| PREHOOK: type: DROPTABLE |
| PREHOOK: Input: default@spart |
| PREHOOK: Output: default@spart |
| POSTHOOK: query: DROP TABLE IF EXISTS spart |
| POSTHOOK: type: DROPTABLE |
| POSTHOOK: Input: default@spart |
| POSTHOOK: Output: default@spart |
| PREHOOK: query: CREATE TABLE spart (key STRING COMMENT 'default', value STRING COMMENT 'default') |
| PARTITIONED BY (ds STRING, hr STRING) |
| STORED AS TEXTFILE |
| PREHOOK: type: CREATETABLE |
| PREHOOK: Output: database:default |
| PREHOOK: Output: default@spart |
| POSTHOOK: query: CREATE TABLE spart (key STRING COMMENT 'default', value STRING COMMENT 'default') |
| PARTITIONED BY (ds STRING, hr STRING) |
| STORED AS TEXTFILE |
| POSTHOOK: type: CREATETABLE |
| POSTHOOK: Output: database:default |
| POSTHOOK: Output: default@spart |
| PREHOOK: query: LOAD DATA LOCAL INPATH "../../data/files/kv1.txt" |
| OVERWRITE INTO TABLE spart PARTITION (ds="2008-04-08", hr="11") |
| PREHOOK: type: LOAD |
| #### A masked pattern was here #### |
| PREHOOK: Output: default@spart |
| POSTHOOK: query: LOAD DATA LOCAL INPATH "../../data/files/kv1.txt" |
| OVERWRITE INTO TABLE spart PARTITION (ds="2008-04-08", hr="11") |
| POSTHOOK: type: LOAD |
| #### A masked pattern was here #### |
| POSTHOOK: Output: default@spart |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| PREHOOK: query: LOAD DATA LOCAL INPATH "../../data/files/kv1.txt" |
| OVERWRITE INTO TABLE spart PARTITION (ds="2008-04-08", hr="12") |
| PREHOOK: type: LOAD |
| #### A masked pattern was here #### |
| PREHOOK: Output: default@spart |
| POSTHOOK: query: LOAD DATA LOCAL INPATH "../../data/files/kv1.txt" |
| OVERWRITE INTO TABLE spart PARTITION (ds="2008-04-08", hr="12") |
| POSTHOOK: type: LOAD |
| #### A masked pattern was here #### |
| POSTHOOK: Output: default@spart |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=12 |
| PREHOOK: query: desc formatted spart |
| PREHOOK: type: DESCTABLE |
| PREHOOK: Input: default@spart |
| POSTHOOK: query: desc formatted spart |
| POSTHOOK: type: DESCTABLE |
| POSTHOOK: Input: default@spart |
| # col_name data_type comment |
| key string default |
| value string default |
| |
| # Partition Information |
| # col_name data_type comment |
| ds string |
| hr string |
| |
| # Detailed Table Information |
| Database: default |
| #### A masked pattern was here #### |
| Retention: 0 |
| #### A masked pattern was here #### |
| Table Type: MANAGED_TABLE |
| Table Parameters: |
| bucketing_version 2 |
| numFiles 2 |
| numPartitions 2 |
| numRows 0 |
| rawDataSize 0 |
| totalSize 11624 |
| #### A masked pattern was here #### |
| |
| # Storage Information |
| SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| InputFormat: org.apache.hadoop.mapred.TextInputFormat |
| OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| Compressed: No |
| Num Buckets: -1 |
| Bucket Columns: [] |
| Sort Columns: [] |
| Storage Desc Params: |
| serialization.format 1 |
| PREHOOK: query: explain extended analyze table spart partition(ds,hr) compute statistics for columns |
| PREHOOK: type: ANALYZE_TABLE |
| PREHOOK: Input: default@spart |
| PREHOOK: Input: default@spart@ds=2008-04-08/hr=11 |
| PREHOOK: Input: default@spart@ds=2008-04-08/hr=12 |
| PREHOOK: Output: default@spart |
| PREHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| PREHOOK: Output: default@spart@ds=2008-04-08/hr=12 |
| #### A masked pattern was here #### |
| POSTHOOK: query: explain extended analyze table spart partition(ds,hr) compute statistics for columns |
| POSTHOOK: type: ANALYZE_TABLE |
| POSTHOOK: Input: default@spart |
| POSTHOOK: Input: default@spart@ds=2008-04-08/hr=11 |
| POSTHOOK: Input: default@spart@ds=2008-04-08/hr=12 |
| POSTHOOK: Output: default@spart |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=12 |
| #### A masked pattern was here #### |
| STAGE DEPENDENCIES: |
| Stage-0 is a root stage |
| Stage-2 depends on stages: Stage-0 |
| |
| STAGE PLANS: |
| Stage: Stage-0 |
| Tez |
| #### A masked pattern was here #### |
| Edges: |
| Reducer 2 <- Map 1 (SIMPLE_EDGE) |
| #### A masked pattern was here #### |
| Vertices: |
| Map 1 |
| Map Operator Tree: |
| TableScan |
| alias: spart |
| Statistics: Num rows: 196 Data size: 257552 Basic stats: PARTIAL Column stats: PARTIAL |
| Statistics Aggregation Key Prefix: default.spart/ |
| GatherStats: true |
| Select Operator |
| expressions: key (type: string), value (type: string), ds (type: string), hr (type: string) |
| outputColumnNames: key, value, ds, hr |
| Statistics: Num rows: 196 Data size: 257552 Basic stats: PARTIAL Column stats: PARTIAL |
| Group By Operator |
| aggregations: max(length(key)), avg(COALESCE(length(key),0)), count(1), count(key), compute_bit_vector(key, 'hll'), max(length(value)), avg(COALESCE(length(value),0)), count(value), compute_bit_vector(value, 'hll') |
| keys: ds (type: string), hr (type: string) |
| minReductionHashAggr: 0.99 |
| mode: hash |
| outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10 |
| Statistics: Num rows: 196 Data size: 233824 Basic stats: PARTIAL Column stats: PARTIAL |
| Reduce Output Operator |
| bucketingVersion: 2 |
| key expressions: _col0 (type: string), _col1 (type: string) |
| null sort order: zz |
| numBuckets: -1 |
| sort order: ++ |
| Map-reduce partition columns: _col0 (type: string), _col1 (type: string) |
| Statistics: Num rows: 196 Data size: 233824 Basic stats: PARTIAL Column stats: PARTIAL |
| tag: -1 |
| value expressions: _col2 (type: int), _col3 (type: struct<count:bigint,sum:double,input:int>), _col4 (type: bigint), _col5 (type: bigint), _col6 (type: binary), _col7 (type: int), _col8 (type: struct<count:bigint,sum:double,input:int>), _col9 (type: bigint), _col10 (type: binary) |
| auto parallelism: true |
| Execution mode: llap |
| LLAP IO: all inputs |
| Path -> Alias: |
| #### A masked pattern was here #### |
| Path -> Partition: |
| #### A masked pattern was here #### |
| Partition |
| base file name: hr=11 |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| partition values: |
| ds 2008-04-08 |
| hr 11 |
| properties: |
| column.name.delimiter , |
| columns key,value |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.spart |
| partition_columns ds/hr |
| partition_columns.types string:string |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.comments 'default','default' |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.spart |
| partition_columns ds/hr |
| partition_columns.types string:string |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| name: default.spart |
| name: default.spart |
| #### A masked pattern was here #### |
| Partition |
| base file name: hr=12 |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| partition values: |
| ds 2008-04-08 |
| hr 12 |
| properties: |
| column.name.delimiter , |
| columns key,value |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.spart |
| partition_columns ds/hr |
| partition_columns.types string:string |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.comments 'default','default' |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.spart |
| partition_columns ds/hr |
| partition_columns.types string:string |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| name: default.spart |
| name: default.spart |
| Truncated Path -> Alias: |
| /spart/ds=2008-04-08/hr=11 [spart] |
| /spart/ds=2008-04-08/hr=12 [spart] |
| Reducer 2 |
| Execution mode: llap |
| Needs Tagging: false |
| Reduce Operator Tree: |
| Group By Operator |
| aggregations: max(VALUE._col0), avg(VALUE._col1), count(VALUE._col2), count(VALUE._col3), compute_bit_vector(VALUE._col4), max(VALUE._col5), avg(VALUE._col6), count(VALUE._col7), compute_bit_vector(VALUE._col8) |
| keys: KEY._col0 (type: string), KEY._col1 (type: string) |
| mode: mergepartial |
| outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10 |
| Statistics: Num rows: 98 Data size: 115440 Basic stats: PARTIAL Column stats: PARTIAL |
| Select Operator |
| expressions: 'STRING' (type: string), UDFToLong(COALESCE(_col2,0)) (type: bigint), COALESCE(_col3,0) (type: double), (_col4 - _col5) (type: bigint), COALESCE(ndv_compute_bit_vector(_col6),0) (type: bigint), _col6 (type: binary), 'STRING' (type: string), UDFToLong(COALESCE(_col7,0)) (type: bigint), COALESCE(_col8,0) (type: double), (_col4 - _col9) (type: bigint), COALESCE(ndv_compute_bit_vector(_col10),0) (type: bigint), _col10 (type: binary), _col0 (type: string), _col1 (type: string) |
| outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13 |
| Statistics: Num rows: 98 Data size: 115440 Basic stats: PARTIAL Column stats: PARTIAL |
| File Output Operator |
| bucketingVersion: 2 |
| compressed: false |
| GlobalTableId: 0 |
| #### A masked pattern was here #### |
| NumFilesPerFileSink: 1 |
| Statistics: Num rows: 98 Data size: 115440 Basic stats: PARTIAL Column stats: PARTIAL |
| #### A masked pattern was here #### |
| table: |
| input format: org.apache.hadoop.mapred.SequenceFileInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat |
| properties: |
| bucketing_version -1 |
| columns _col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8,_col9,_col10,_col11,_col12,_col13 |
| columns.types string:bigint:double:bigint:bigint:binary:string:bigint:double:bigint:bigint:binary:string:string |
| escape.delim \ |
| hive.serialization.extend.additional.nesting.levels true |
| serialization.escape.crlf true |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| TotalFiles: 1 |
| GatherStats: false |
| MultiFileSpray: false |
| |
| Stage: Stage-2 |
| Stats Work |
| Basic Stats Work: |
| Stats Aggregation Key Prefix: default.spart/ |
| Column Stats Desc: |
| Columns: key, value |
| Column Types: string, string |
| Table: default.spart |
| Is Table Level Stats: false |
| |
| PREHOOK: query: analyze table spart partition(ds,hr) compute statistics for columns |
| PREHOOK: type: ANALYZE_TABLE |
| PREHOOK: Input: default@spart |
| PREHOOK: Input: default@spart@ds=2008-04-08/hr=11 |
| PREHOOK: Input: default@spart@ds=2008-04-08/hr=12 |
| PREHOOK: Output: default@spart |
| PREHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| PREHOOK: Output: default@spart@ds=2008-04-08/hr=12 |
| #### A masked pattern was here #### |
| POSTHOOK: query: analyze table spart partition(ds,hr) compute statistics for columns |
| POSTHOOK: type: ANALYZE_TABLE |
| POSTHOOK: Input: default@spart |
| POSTHOOK: Input: default@spart@ds=2008-04-08/hr=11 |
| POSTHOOK: Input: default@spart@ds=2008-04-08/hr=12 |
| POSTHOOK: Output: default@spart |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=12 |
| #### A masked pattern was here #### |
| PREHOOK: query: desc formatted spart |
| PREHOOK: type: DESCTABLE |
| PREHOOK: Input: default@spart |
| POSTHOOK: query: desc formatted spart |
| POSTHOOK: type: DESCTABLE |
| POSTHOOK: Input: default@spart |
| # col_name data_type comment |
| key string default |
| value string default |
| |
| # Partition Information |
| # col_name data_type comment |
| ds string |
| hr string |
| |
| # Detailed Table Information |
| Database: default |
| #### A masked pattern was here #### |
| Retention: 0 |
| #### A masked pattern was here #### |
| Table Type: MANAGED_TABLE |
| Table Parameters: |
| COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} |
| bucketing_version 2 |
| numFiles 2 |
| numPartitions 2 |
| numRows 1000 |
| rawDataSize 10624 |
| totalSize 11624 |
| #### A masked pattern was here #### |
| |
| # Storage Information |
| SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| InputFormat: org.apache.hadoop.mapred.TextInputFormat |
| OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| Compressed: No |
| Num Buckets: -1 |
| Bucket Columns: [] |
| Sort Columns: [] |
| Storage Desc Params: |
| serialization.format 1 |
| PREHOOK: query: desc formatted spart PARTITION(ds='2008-04-08', hr=11) |
| PREHOOK: type: DESCTABLE |
| PREHOOK: Input: default@spart |
| POSTHOOK: query: desc formatted spart PARTITION(ds='2008-04-08', hr=11) |
| POSTHOOK: type: DESCTABLE |
| POSTHOOK: Input: default@spart |
| # col_name data_type comment |
| key string default |
| value string default |
| |
| # Partition Information |
| # col_name data_type comment |
| ds string |
| hr string |
| |
| # Detailed Partition Information |
| Partition Value: [2008-04-08, 11] |
| Database: default |
| Table: spart |
| #### A masked pattern was here #### |
| Partition Parameters: |
| COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"key\":\"true\",\"value\":\"true\"}} |
| numFiles 1 |
| numRows 500 |
| rawDataSize 5312 |
| totalSize 5812 |
| #### A masked pattern was here #### |
| |
| # Storage Information |
| SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| InputFormat: org.apache.hadoop.mapred.TextInputFormat |
| OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| Compressed: No |
| Num Buckets: -1 |
| Bucket Columns: [] |
| Sort Columns: [] |
| Storage Desc Params: |
| serialization.format 1 |
| PREHOOK: query: desc formatted spart PARTITION(ds='2008-04-08', hr=12) |
| PREHOOK: type: DESCTABLE |
| PREHOOK: Input: default@spart |
| POSTHOOK: query: desc formatted spart PARTITION(ds='2008-04-08', hr=12) |
| POSTHOOK: type: DESCTABLE |
| POSTHOOK: Input: default@spart |
| # col_name data_type comment |
| key string default |
| value string default |
| |
| # Partition Information |
| # col_name data_type comment |
| ds string |
| hr string |
| |
| # Detailed Partition Information |
| Partition Value: [2008-04-08, 12] |
| Database: default |
| Table: spart |
| #### A masked pattern was here #### |
| Partition Parameters: |
| COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"key\":\"true\",\"value\":\"true\"}} |
| numFiles 1 |
| numRows 500 |
| rawDataSize 5312 |
| totalSize 5812 |
| #### A masked pattern was here #### |
| |
| # Storage Information |
| SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| InputFormat: org.apache.hadoop.mapred.TextInputFormat |
| OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| Compressed: No |
| Num Buckets: -1 |
| Bucket Columns: [] |
| Sort Columns: [] |
| Storage Desc Params: |
| serialization.format 1 |
| PREHOOK: query: DROP TABLE IF EXISTS spart |
| PREHOOK: type: DROPTABLE |
| PREHOOK: Input: default@spart |
| PREHOOK: Output: default@spart |
| POSTHOOK: query: DROP TABLE IF EXISTS spart |
| POSTHOOK: type: DROPTABLE |
| POSTHOOK: Input: default@spart |
| POSTHOOK: Output: default@spart |
| PREHOOK: query: CREATE TABLE spart (key STRING COMMENT 'default', value STRING COMMENT 'default') |
| PARTITIONED BY (ds STRING, hr STRING) |
| STORED AS TEXTFILE |
| PREHOOK: type: CREATETABLE |
| PREHOOK: Output: database:default |
| PREHOOK: Output: default@spart |
| POSTHOOK: query: CREATE TABLE spart (key STRING COMMENT 'default', value STRING COMMENT 'default') |
| PARTITIONED BY (ds STRING, hr STRING) |
| STORED AS TEXTFILE |
| POSTHOOK: type: CREATETABLE |
| POSTHOOK: Output: database:default |
| POSTHOOK: Output: default@spart |
| PREHOOK: query: LOAD DATA LOCAL INPATH "../../data/files/kv1.txt" |
| OVERWRITE INTO TABLE spart PARTITION (ds="2008-04-08", hr="11") |
| PREHOOK: type: LOAD |
| #### A masked pattern was here #### |
| PREHOOK: Output: default@spart |
| POSTHOOK: query: LOAD DATA LOCAL INPATH "../../data/files/kv1.txt" |
| OVERWRITE INTO TABLE spart PARTITION (ds="2008-04-08", hr="11") |
| POSTHOOK: type: LOAD |
| #### A masked pattern was here #### |
| POSTHOOK: Output: default@spart |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| PREHOOK: query: LOAD DATA LOCAL INPATH "../../data/files/kv1.txt" |
| OVERWRITE INTO TABLE spart PARTITION (ds="2008-04-08", hr="12") |
| PREHOOK: type: LOAD |
| #### A masked pattern was here #### |
| PREHOOK: Output: default@spart |
| POSTHOOK: query: LOAD DATA LOCAL INPATH "../../data/files/kv1.txt" |
| OVERWRITE INTO TABLE spart PARTITION (ds="2008-04-08", hr="12") |
| POSTHOOK: type: LOAD |
| #### A masked pattern was here #### |
| POSTHOOK: Output: default@spart |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=12 |
| PREHOOK: query: desc formatted spart |
| PREHOOK: type: DESCTABLE |
| PREHOOK: Input: default@spart |
| POSTHOOK: query: desc formatted spart |
| POSTHOOK: type: DESCTABLE |
| POSTHOOK: Input: default@spart |
| # col_name data_type comment |
| key string default |
| value string default |
| |
| # Partition Information |
| # col_name data_type comment |
| ds string |
| hr string |
| |
| # Detailed Table Information |
| Database: default |
| #### A masked pattern was here #### |
| Retention: 0 |
| #### A masked pattern was here #### |
| Table Type: MANAGED_TABLE |
| Table Parameters: |
| bucketing_version 2 |
| numFiles 2 |
| numPartitions 2 |
| numRows 0 |
| rawDataSize 0 |
| totalSize 11624 |
| #### A masked pattern was here #### |
| |
| # Storage Information |
| SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| InputFormat: org.apache.hadoop.mapred.TextInputFormat |
| OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| Compressed: No |
| Num Buckets: -1 |
| Bucket Columns: [] |
| Sort Columns: [] |
| Storage Desc Params: |
| serialization.format 1 |
| PREHOOK: query: explain extended analyze table spart partition(hr="11") compute statistics for columns |
| PREHOOK: type: ANALYZE_TABLE |
| PREHOOK: Input: default@spart |
| PREHOOK: Input: default@spart@ds=2008-04-08/hr=11 |
| PREHOOK: Output: default@spart |
| PREHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| #### A masked pattern was here #### |
| POSTHOOK: query: explain extended analyze table spart partition(hr="11") compute statistics for columns |
| POSTHOOK: type: ANALYZE_TABLE |
| POSTHOOK: Input: default@spart |
| POSTHOOK: Input: default@spart@ds=2008-04-08/hr=11 |
| POSTHOOK: Output: default@spart |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| #### A masked pattern was here #### |
| STAGE DEPENDENCIES: |
| Stage-0 is a root stage |
| Stage-2 depends on stages: Stage-0 |
| |
| STAGE PLANS: |
| Stage: Stage-0 |
| Tez |
| #### A masked pattern was here #### |
| Edges: |
| Reducer 2 <- Map 1 (SIMPLE_EDGE) |
| #### A masked pattern was here #### |
| Vertices: |
| Map 1 |
| Map Operator Tree: |
| TableScan |
| alias: spart |
| filterExpr: (hr = '11') (type: boolean) |
| Statistics: Num rows: 117 Data size: 121232 Basic stats: PARTIAL Column stats: PARTIAL |
| Statistics Aggregation Key Prefix: default.spart/ |
| GatherStats: true |
| Select Operator |
| expressions: key (type: string), value (type: string), ds (type: string) |
| outputColumnNames: key, value, ds |
| Statistics: Num rows: 117 Data size: 121232 Basic stats: PARTIAL Column stats: PARTIAL |
| Group By Operator |
| aggregations: max(length(key)), avg(COALESCE(length(key),0)), count(1), count(key), compute_bit_vector(key, 'hll'), max(length(value)), avg(COALESCE(length(value),0)), count(value), compute_bit_vector(value, 'hll') |
| keys: ds (type: string), '11' (type: string) |
| minReductionHashAggr: 0.99 |
| mode: hash |
| outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10 |
| Statistics: Num rows: 117 Data size: 118336 Basic stats: PARTIAL Column stats: PARTIAL |
| Reduce Output Operator |
| bucketingVersion: 2 |
| key expressions: _col0 (type: string), '11' (type: string) |
| null sort order: zz |
| numBuckets: -1 |
| sort order: ++ |
| Map-reduce partition columns: _col0 (type: string), '11' (type: string) |
| Statistics: Num rows: 117 Data size: 118336 Basic stats: PARTIAL Column stats: PARTIAL |
| tag: -1 |
| value expressions: _col2 (type: int), _col3 (type: struct<count:bigint,sum:double,input:int>), _col4 (type: bigint), _col5 (type: bigint), _col6 (type: binary), _col7 (type: int), _col8 (type: struct<count:bigint,sum:double,input:int>), _col9 (type: bigint), _col10 (type: binary) |
| auto parallelism: true |
| Execution mode: llap |
| LLAP IO: all inputs |
| Path -> Alias: |
| #### A masked pattern was here #### |
| Path -> Partition: |
| #### A masked pattern was here #### |
| Partition |
| base file name: hr=11 |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| partition values: |
| ds 2008-04-08 |
| hr 11 |
| properties: |
| column.name.delimiter , |
| columns key,value |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.spart |
| partition_columns ds/hr |
| partition_columns.types string:string |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.comments 'default','default' |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.spart |
| partition_columns ds/hr |
| partition_columns.types string:string |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| name: default.spart |
| name: default.spart |
| Truncated Path -> Alias: |
| /spart/ds=2008-04-08/hr=11 [spart] |
| Reducer 2 |
| Execution mode: llap |
| Needs Tagging: false |
| Reduce Operator Tree: |
| Group By Operator |
| aggregations: max(VALUE._col0), avg(VALUE._col1), count(VALUE._col2), count(VALUE._col3), compute_bit_vector(VALUE._col4), max(VALUE._col5), avg(VALUE._col6), count(VALUE._col7), compute_bit_vector(VALUE._col8) |
| keys: KEY._col0 (type: string), '11' (type: string) |
| mode: mergepartial |
| outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10 |
| Statistics: Num rows: 58 Data size: 57920 Basic stats: PARTIAL Column stats: PARTIAL |
| Select Operator |
| expressions: 'STRING' (type: string), UDFToLong(COALESCE(_col2,0)) (type: bigint), COALESCE(_col3,0) (type: double), (_col4 - _col5) (type: bigint), COALESCE(ndv_compute_bit_vector(_col6),0) (type: bigint), _col6 (type: binary), 'STRING' (type: string), UDFToLong(COALESCE(_col7,0)) (type: bigint), COALESCE(_col8,0) (type: double), (_col4 - _col9) (type: bigint), COALESCE(ndv_compute_bit_vector(_col10),0) (type: bigint), _col10 (type: binary), _col0 (type: string), '11' (type: string) |
| outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13 |
| Statistics: Num rows: 58 Data size: 57920 Basic stats: PARTIAL Column stats: PARTIAL |
| File Output Operator |
| bucketingVersion: 2 |
| compressed: false |
| GlobalTableId: 0 |
| #### A masked pattern was here #### |
| NumFilesPerFileSink: 1 |
| Statistics: Num rows: 58 Data size: 57920 Basic stats: PARTIAL Column stats: PARTIAL |
| #### A masked pattern was here #### |
| table: |
| input format: org.apache.hadoop.mapred.SequenceFileInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat |
| properties: |
| bucketing_version -1 |
| columns _col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8,_col9,_col10,_col11,_col12,_col13 |
| columns.types string:bigint:double:bigint:bigint:binary:string:bigint:double:bigint:bigint:binary:string:string |
| escape.delim \ |
| hive.serialization.extend.additional.nesting.levels true |
| serialization.escape.crlf true |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| TotalFiles: 1 |
| GatherStats: false |
| MultiFileSpray: false |
| |
| Stage: Stage-2 |
| Stats Work |
| Basic Stats Work: |
| Stats Aggregation Key Prefix: default.spart/ |
| Column Stats Desc: |
| Columns: key, value |
| Column Types: string, string |
| Table: default.spart |
| Is Table Level Stats: false |
| |
| PREHOOK: query: analyze table spart partition(hr="11") compute statistics for columns |
| PREHOOK: type: ANALYZE_TABLE |
| PREHOOK: Input: default@spart |
| PREHOOK: Input: default@spart@ds=2008-04-08/hr=11 |
| PREHOOK: Output: default@spart |
| PREHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| #### A masked pattern was here #### |
| POSTHOOK: query: analyze table spart partition(hr="11") compute statistics for columns |
| POSTHOOK: type: ANALYZE_TABLE |
| POSTHOOK: Input: default@spart |
| POSTHOOK: Input: default@spart@ds=2008-04-08/hr=11 |
| POSTHOOK: Output: default@spart |
| POSTHOOK: Output: default@spart@ds=2008-04-08/hr=11 |
| #### A masked pattern was here #### |
| PREHOOK: query: desc formatted spart |
| PREHOOK: type: DESCTABLE |
| PREHOOK: Input: default@spart |
| POSTHOOK: query: desc formatted spart |
| POSTHOOK: type: DESCTABLE |
| POSTHOOK: Input: default@spart |
| # col_name data_type comment |
| key string default |
| value string default |
| |
| # Partition Information |
| # col_name data_type comment |
| ds string |
| hr string |
| |
| # Detailed Table Information |
| Database: default |
| #### A masked pattern was here #### |
| Retention: 0 |
| #### A masked pattern was here #### |
| Table Type: MANAGED_TABLE |
| Table Parameters: |
| bucketing_version 2 |
| numFiles 2 |
| numPartitions 2 |
| numRows 500 |
| rawDataSize 5312 |
| totalSize 11624 |
| #### A masked pattern was here #### |
| |
| # Storage Information |
| SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| InputFormat: org.apache.hadoop.mapred.TextInputFormat |
| OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| Compressed: No |
| Num Buckets: -1 |
| Bucket Columns: [] |
| Sort Columns: [] |
| Storage Desc Params: |
| serialization.format 1 |
| PREHOOK: query: desc formatted spart PARTITION(ds='2008-04-08', hr=11) |
| PREHOOK: type: DESCTABLE |
| PREHOOK: Input: default@spart |
| POSTHOOK: query: desc formatted spart PARTITION(ds='2008-04-08', hr=11) |
| POSTHOOK: type: DESCTABLE |
| POSTHOOK: Input: default@spart |
| # col_name data_type comment |
| key string default |
| value string default |
| |
| # Partition Information |
| # col_name data_type comment |
| ds string |
| hr string |
| |
| # Detailed Partition Information |
| Partition Value: [2008-04-08, 11] |
| Database: default |
| Table: spart |
| #### A masked pattern was here #### |
| Partition Parameters: |
| COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"key\":\"true\",\"value\":\"true\"}} |
| numFiles 1 |
| numRows 500 |
| rawDataSize 5312 |
| totalSize 5812 |
| #### A masked pattern was here #### |
| |
| # Storage Information |
| SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| InputFormat: org.apache.hadoop.mapred.TextInputFormat |
| OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| Compressed: No |
| Num Buckets: -1 |
| Bucket Columns: [] |
| Sort Columns: [] |
| Storage Desc Params: |
| serialization.format 1 |
| PREHOOK: query: desc formatted spart PARTITION(ds='2008-04-08', hr=12) |
| PREHOOK: type: DESCTABLE |
| PREHOOK: Input: default@spart |
| POSTHOOK: query: desc formatted spart PARTITION(ds='2008-04-08', hr=12) |
| POSTHOOK: type: DESCTABLE |
| POSTHOOK: Input: default@spart |
| # col_name data_type comment |
| key string default |
| value string default |
| |
| # Partition Information |
| # col_name data_type comment |
| ds string |
| hr string |
| |
| # Detailed Partition Information |
| Partition Value: [2008-04-08, 12] |
| Database: default |
| Table: spart |
| #### A masked pattern was here #### |
| Partition Parameters: |
| numFiles 1 |
| numRows 0 |
| rawDataSize 0 |
| totalSize 5812 |
| #### A masked pattern was here #### |
| |
| # Storage Information |
| SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| InputFormat: org.apache.hadoop.mapred.TextInputFormat |
| OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| Compressed: No |
| Num Buckets: -1 |
| Bucket Columns: [] |
| Sort Columns: [] |
| Storage Desc Params: |
| serialization.format 1 |
