| PREHOOK: query: CREATE TABLE bucket4_1(key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS |
| PREHOOK: type: CREATETABLE |
| PREHOOK: Output: database:default |
| PREHOOK: Output: default@bucket4_1 |
| POSTHOOK: query: CREATE TABLE bucket4_1(key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS |
| POSTHOOK: type: CREATETABLE |
| POSTHOOK: Output: database:default |
| POSTHOOK: Output: default@bucket4_1 |
| PREHOOK: query: explain extended |
| insert overwrite table bucket4_1 |
| select * from src |
| PREHOOK: type: QUERY |
| PREHOOK: Input: default@src |
| PREHOOK: Output: default@bucket4_1 |
| POSTHOOK: query: explain extended |
| insert overwrite table bucket4_1 |
| select * from src |
| POSTHOOK: type: QUERY |
| POSTHOOK: Input: default@src |
| POSTHOOK: Output: default@bucket4_1 |
| OPTIMIZED SQL: SELECT `key`, `value` |
| FROM `default`.`src` |
| STAGE DEPENDENCIES: |
| Stage-1 is a root stage |
| Stage-2 depends on stages: Stage-1 |
| Stage-0 depends on stages: Stage-2 |
| Stage-3 depends on stages: Stage-0 |
| |
| STAGE PLANS: |
| Stage: Stage-1 |
| Tez |
| #### A masked pattern was here #### |
| Edges: |
| Reducer 2 <- Map 1 (SIMPLE_EDGE) |
| #### A masked pattern was here #### |
| Vertices: |
| Map 1 |
| Map Operator Tree: |
| TableScan |
| alias: src |
| Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE |
| GatherStats: false |
| Select Operator |
| expressions: UDFToInteger(key) (type: int), value (type: string) |
| outputColumnNames: _col0, _col1 |
| Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE |
| Reduce Output Operator |
| bucketingVersion: 2 |
| key expressions: _col0 (type: int) |
| null sort order: a |
| numBuckets: -1 |
| sort order: + |
| Map-reduce partition columns: _col0 (type: int) |
| Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE |
| tag: -1 |
| value expressions: _col1 (type: string) |
| auto parallelism: false |
| Execution mode: vectorized, llap |
| LLAP IO: all inputs |
| Path -> Alias: |
| #### A masked pattern was here #### |
| Path -> Partition: |
| #### A masked pattern was here #### |
| Partition |
| base file name: src |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| bucket_count -1 |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.src |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.comments 'default','default' |
| columns.types string:string |
| #### A masked pattern was here #### |
| name default.src |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| name: default.src |
| name: default.src |
| Truncated Path -> Alias: |
| /src [src] |
| Reducer 2 |
| Execution mode: llap |
| Needs Tagging: false |
| Reduce Operator Tree: |
| Select Operator |
| expressions: KEY.reducesinkkey0 (type: int), VALUE._col0 (type: string) |
| outputColumnNames: _col0, _col1 |
| Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE |
| File Output Operator |
| bucketingVersion: 2 |
| compressed: false |
| GlobalTableId: 1 |
| #### A masked pattern was here #### |
| NumFilesPerFileSink: 2 |
| Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE |
| #### A masked pattern was here #### |
| table: |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| SORTBUCKETCOLSPREFIX TRUE |
| bucket_count 2 |
| bucket_field_name key |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.comments |
| columns.types int:string |
| #### A masked pattern was here #### |
| name default.bucket4_1 |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| name: default.bucket4_1 |
| TotalFiles: 2 |
| GatherStats: true |
| MultiFileSpray: true |
| Select Operator |
| expressions: _col0 (type: int), _col1 (type: string) |
| outputColumnNames: key, value |
| Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE |
| Group By Operator |
| aggregations: min(key), max(key), count(1), count(key), compute_bit_vector(key, 'hll'), max(length(value)), avg(COALESCE(length(value),0)), count(value), compute_bit_vector(value, 'hll') |
| mode: complete |
| outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8 |
| Statistics: Num rows: 1 Data size: 332 Basic stats: COMPLETE Column stats: COMPLETE |
| Select Operator |
| expressions: 'LONG' (type: string), UDFToLong(_col0) (type: bigint), UDFToLong(_col1) (type: bigint), (_col2 - _col3) (type: bigint), COALESCE(ndv_compute_bit_vector(_col4),0) (type: bigint), _col4 (type: binary), 'STRING' (type: string), UDFToLong(COALESCE(_col5,0)) (type: bigint), COALESCE(_col6,0) (type: double), (_col2 - _col7) (type: bigint), COALESCE(ndv_compute_bit_vector(_col8),0) (type: bigint), _col8 (type: binary) |
| outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11 |
| Statistics: Num rows: 1 Data size: 530 Basic stats: COMPLETE Column stats: COMPLETE |
| File Output Operator |
| bucketingVersion: 2 |
| compressed: false |
| GlobalTableId: 0 |
| #### A masked pattern was here #### |
| NumFilesPerFileSink: 1 |
| Statistics: Num rows: 1 Data size: 530 Basic stats: COMPLETE Column stats: COMPLETE |
| #### A masked pattern was here #### |
| table: |
| input format: org.apache.hadoop.mapred.SequenceFileInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat |
| properties: |
| bucketing_version -1 |
| columns _col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8,_col9,_col10,_col11 |
| columns.types string:bigint:bigint:bigint:bigint:binary:string:bigint:double:bigint:bigint:binary |
| escape.delim \ |
| hive.serialization.extend.additional.nesting.levels true |
| serialization.escape.crlf true |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| TotalFiles: 1 |
| GatherStats: false |
| MultiFileSpray: false |
| |
| Stage: Stage-2 |
| Dependency Collection |
| |
| Stage: Stage-0 |
| Move Operator |
| tables: |
| replace: true |
| #### A masked pattern was here #### |
| table: |
| input format: org.apache.hadoop.mapred.TextInputFormat |
| output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat |
| properties: |
| SORTBUCKETCOLSPREFIX TRUE |
| bucket_count 2 |
| bucket_field_name key |
| bucketing_version 2 |
| column.name.delimiter , |
| columns key,value |
| columns.comments |
| columns.types int:string |
| #### A masked pattern was here #### |
| name default.bucket4_1 |
| serialization.format 1 |
| serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
| name: default.bucket4_1 |
| |
| Stage: Stage-3 |
| Stats Work |
| Basic Stats Work: |
| #### A masked pattern was here #### |
| Column Stats Desc: |
| Columns: key, value |
| Column Types: int, string |
| Table: default.bucket4_1 |
| Is Table Level Stats: true |
| |
| PREHOOK: query: insert overwrite table bucket4_1 |
| select * from src |
| PREHOOK: type: QUERY |
| PREHOOK: Input: default@src |
| PREHOOK: Output: default@bucket4_1 |
| POSTHOOK: query: insert overwrite table bucket4_1 |
| select * from src |
| POSTHOOK: type: QUERY |
| POSTHOOK: Input: default@src |
| POSTHOOK: Output: default@bucket4_1 |
| POSTHOOK: Lineage: bucket4_1.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ] |
| POSTHOOK: Lineage: bucket4_1.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ] |
| PREHOOK: query: explain |
| select * from bucket4_1 tablesample (bucket 1 out of 2) s |
| PREHOOK: type: QUERY |
| PREHOOK: Input: default@bucket4_1 |
| #### A masked pattern was here #### |
| POSTHOOK: query: explain |
| select * from bucket4_1 tablesample (bucket 1 out of 2) s |
| POSTHOOK: type: QUERY |
| POSTHOOK: Input: default@bucket4_1 |
| #### A masked pattern was here #### |
| STAGE DEPENDENCIES: |
| Stage-0 is a root stage |
| |
| STAGE PLANS: |
| Stage: Stage-0 |
| Fetch Operator |
| limit: -1 |
| Processor Tree: |
| TableScan |
| alias: s |
| Filter Operator |
| predicate: (((hash(key) & 2147483647) % 2) = 0) (type: boolean) |
| Select Operator |
| expressions: key (type: int), value (type: string) |
| outputColumnNames: _col0, _col1 |
| ListSink |
| |
| PREHOOK: query: select * from bucket4_1 tablesample (bucket 1 out of 2) s |
| PREHOOK: type: QUERY |
| PREHOOK: Input: default@bucket4_1 |
| #### A masked pattern was here #### |
| POSTHOOK: query: select * from bucket4_1 tablesample (bucket 1 out of 2) s |
| POSTHOOK: type: QUERY |
| POSTHOOK: Input: default@bucket4_1 |
| #### A masked pattern was here #### |
| 2 val_2 |
| 10 val_10 |
| 17 val_17 |
| 19 val_19 |
| 20 val_20 |
| 27 val_27 |
| 30 val_30 |
| 41 val_41 |
| 43 val_43 |
| 51 val_51 |
| 51 val_51 |
| 54 val_54 |
| 57 val_57 |
| 58 val_58 |
| 58 val_58 |
| 64 val_64 |
| 65 val_65 |
| 67 val_67 |
| 67 val_67 |
| 69 val_69 |
| 70 val_70 |
| 70 val_70 |
| 70 val_70 |
| 77 val_77 |
| 80 val_80 |
| 83 val_83 |
| 83 val_83 |
| 84 val_84 |
| 84 val_84 |
| 85 val_85 |
| 86 val_86 |
| 90 val_90 |
| 90 val_90 |
| 90 val_90 |
| 96 val_96 |
| 97 val_97 |
| 97 val_97 |
| 98 val_98 |
| 98 val_98 |
| 105 val_105 |
| 113 val_113 |
| 113 val_113 |
| 116 val_116 |
| 119 val_119 |
| 119 val_119 |
| 119 val_119 |
| 126 val_126 |
| 133 val_133 |
| 134 val_134 |
| 134 val_134 |
| 137 val_137 |
| 137 val_137 |
| 146 val_146 |
| 146 val_146 |
| 149 val_149 |
| 149 val_149 |
| 153 val_153 |
| 156 val_156 |
| 157 val_157 |
| 158 val_158 |
| 162 val_162 |
| 163 val_163 |
| 164 val_164 |
| 164 val_164 |
| 165 val_165 |
| 165 val_165 |
| 169 val_169 |
| 169 val_169 |
| 169 val_169 |
| 169 val_169 |
| 177 val_177 |
| 178 val_178 |
| 181 val_181 |
| 187 val_187 |
| 187 val_187 |
| 187 val_187 |
| 190 val_190 |
| 191 val_191 |
| 191 val_191 |
| 195 val_195 |
| 195 val_195 |
| 197 val_197 |
| 197 val_197 |
| 199 val_199 |
| 199 val_199 |
| 199 val_199 |
| 201 val_201 |
| 202 val_202 |
| 208 val_208 |
| 208 val_208 |
| 208 val_208 |
| 214 val_214 |
| 216 val_216 |
| 216 val_216 |
| 221 val_221 |
| 221 val_221 |
| 226 val_226 |
| 229 val_229 |
| 229 val_229 |
| 233 val_233 |
| 233 val_233 |
| 237 val_237 |
| 237 val_237 |
| 238 val_238 |
| 238 val_238 |
| 239 val_239 |
| 239 val_239 |
| 244 val_244 |
| 248 val_248 |
| 249 val_249 |
| 252 val_252 |
| 255 val_255 |
| 255 val_255 |
| 256 val_256 |
| 256 val_256 |
| 260 val_260 |
| 266 val_266 |
| 272 val_272 |
| 272 val_272 |
| 284 val_284 |
| 286 val_286 |
| 287 val_287 |
| 289 val_289 |
| 291 val_291 |
| 292 val_292 |
| 307 val_307 |
| 307 val_307 |
| 308 val_308 |
| 310 val_310 |
| 311 val_311 |
| 311 val_311 |
| 311 val_311 |
| 315 val_315 |
| 316 val_316 |
| 316 val_316 |
| 316 val_316 |
| 317 val_317 |
| 317 val_317 |
| 325 val_325 |
| 325 val_325 |
| 327 val_327 |
| 327 val_327 |
| 327 val_327 |
| 332 val_332 |
| 336 val_336 |
| 338 val_338 |
| 339 val_339 |
| 341 val_341 |
| 342 val_342 |
| 342 val_342 |
| 344 val_344 |
| 344 val_344 |
| 348 val_348 |
| 348 val_348 |
| 348 val_348 |
| 348 val_348 |
| 348 val_348 |
| 351 val_351 |
| 353 val_353 |
| 353 val_353 |
| 360 val_360 |
| 362 val_362 |
| 364 val_364 |
| 365 val_365 |
| 368 val_368 |
| 369 val_369 |
| 369 val_369 |
| 369 val_369 |
| 377 val_377 |
| 378 val_378 |
| 384 val_384 |
| 384 val_384 |
| 384 val_384 |
| 386 val_386 |
| 392 val_392 |
| 393 val_393 |
| 394 val_394 |
| 396 val_396 |
| 396 val_396 |
| 396 val_396 |
| 397 val_397 |
| 397 val_397 |
| 399 val_399 |
| 399 val_399 |
| 402 val_402 |
| 404 val_404 |
| 404 val_404 |
| 407 val_407 |
| 409 val_409 |
| 409 val_409 |
| 409 val_409 |
| 411 val_411 |
| 414 val_414 |
| 414 val_414 |
| 417 val_417 |
| 417 val_417 |
| 417 val_417 |
| 419 val_419 |
| 421 val_421 |
| 427 val_427 |
| 430 val_430 |
| 430 val_430 |
| 430 val_430 |
| 432 val_432 |
| 435 val_435 |
| 437 val_437 |
| 443 val_443 |
| 444 val_444 |
| 446 val_446 |
| 449 val_449 |
| 452 val_452 |
| 453 val_453 |
| 454 val_454 |
| 454 val_454 |
| 454 val_454 |
| 458 val_458 |
| 458 val_458 |
| 459 val_459 |
| 459 val_459 |
| 463 val_463 |
| 463 val_463 |
| 466 val_466 |
| 466 val_466 |
| 466 val_466 |
| 467 val_467 |
| 472 val_472 |
| 478 val_478 |
| 478 val_478 |
| 479 val_479 |
| 480 val_480 |
| 480 val_480 |
| 480 val_480 |
| 482 val_482 |
| 483 val_483 |
| 484 val_484 |
| 485 val_485 |
| 490 val_490 |
| 491 val_491 |
| 495 val_495 |
| 497 val_497 |
| 498 val_498 |
| 498 val_498 |
| 498 val_498 |
