| These Parquet files were created by modifying Impala's HdfsParquetTableWriter. |
| |
| String Data |
| ----------- |
| These files have a single nullable string column 's'. |
| |
| dict-encoded-negative-len.parq: a single dictionary-encoded value with a negative length. |
| dict-encoded-out-of-bounds.parq: a single dictionary-encoded value with a length past end of page. |
| plain-encoded-negative-len.parq: a single plain-encoded value with a negative length. |
| plain-encoded-out-of-bounds.parq: a single plain-encoded value with a length past end of page. |
| |
| Decimal Data |
| ----------- |
| illegal_decimals.parq has 8 decimal columns(d1, d2, ..., d8). All of them have illegal schema in the footer. |
| Generated by modifying HdfsParquetTableWriter::WriteFileFooter by these: |
| |
| Status HdfsParquetTableWriter::WriteFileFooter() { |
| + file_metadata_.schema[1].__set_precision(0); |
| + file_metadata_.schema[1].__isset.precision = false; |
| + file_metadata_.schema[1].logicalType.DECIMAL.precision = 0; |
| + file_metadata_.schema[1].logicalType.__isset.DECIMAL = false; |
| + |
| + file_metadata_.schema[2].__set_precision(20); |
| + file_metadata_.schema[2].logicalType.DECIMAL.precision = 20; |
| + file_metadata_.schema[2].logicalType.__isset.DECIMAL = true; |
| + file_metadata_.schema[2].__isset.logicalType = true; |
| + |
| + file_metadata_.schema[3].__set_precision(-1); |
| + file_metadata_.schema[3].logicalType.DECIMAL.precision = -1; |
| + file_metadata_.schema[3].logicalType.__isset.DECIMAL = true; |
| + file_metadata_.schema[3].__isset.logicalType = true; |
| + |
| + file_metadata_.schema[4].__set_type(parquet::Type::FIXED_LEN_BYTE_ARRAY); |
| + file_metadata_.schema[4].__isset.type_length = false; |
| + |
| + file_metadata_.schema[5].__set_type(parquet::Type::FIXED_LEN_BYTE_ARRAY); |
| + file_metadata_.schema[5].__set_type_length(0); |
| + |
| + file_metadata_.schema[6].__set_scale(-1); |
| + file_metadata_.schema[6].logicalType.DECIMAL.scale = -1; |
| + file_metadata_.schema[6].logicalType.__isset.DECIMAL = true; |
| + file_metadata_.schema[6].__isset.logicalType = true; |
| + |
| + file_metadata_.schema[7].__set_scale(4); |
| + file_metadata_.schema[7].__set_precision(2); |
| + file_metadata_.schema[7].logicalType.DECIMAL.scale = 4; |
| + file_metadata_.schema[7].logicalType.DECIMAL.precision = 2; |
| + file_metadata_.schema[7].logicalType.__isset.DECIMAL = true; |
| + file_metadata_.schema[7].__isset.logicalType = true; |
| |
| Then create the table and insert one row into it: |
| |
| create table my_decimal_tbl (d1 decimal(4,2), d2 decimal(4,2), ..., d7 decimal(4,2)) stored as parquet; |
| insert into my_decimal_tbl values (cast(0 as decimal(4,2)), cast(0 as decimal(4,2)), ...); |