TsFile metadata falls into two categories:
Footer metadata starts after SEPARATOR (0x02). Ranges of TimeseriesMetadata and measurement index nodes may be interleaved. The offset and end_offset fields in the indexes delimit each range. Every offset is an absolute byte offset measured from the beginning of the file.
A lookup follows the hierarchy below. Each level narrows the search, from a table to the Pages that may contain matching values:
| Level | Identifies | Primary use |
|---|---|---|
| File metadata | Tables and device-index roots | Enter the correct index tree |
| Device index | Devices | Locate one device's measurement index |
| Measurement index | Measurement-name ranges | Locate Timeseries Metadata |
| Timeseries Metadata | One logical series | Select Chunk offsets and series Statistics |
| Chunk Metadata | One physical Chunk | Seek to data and prune by Chunk Statistics |
| Page Header | One Page | Prune by Page Statistics and read its payload |
Page-, Chunk-, and timeseries-level Statistics use the same type-specific structure:
Statistics := UVarInt count int64 start_time int64 end_time ... type-specific fields
count is the number of values covered by the structure. It equals the number of points in a non-aligned series, the number of non-null values in an aligned value column, or the number of timestamps in an aligned time column.
Type-specific fields immediately follow the common prefix in the order shown below:
| Data type | Type-specific fields |
|---|---|
BOOLEAN | bool first, bool last, int64 true_count |
INT32, DATE | int32 min, int32 max, int32 first, int32 last, int64 sum |
INT64, TIMESTAMP | int64 min, int64 max, int64 first, int64 last, float64 sum |
FLOAT | float32 min, float32 max, float32 first, float32 last, float64 sum |
DOUBLE | float64 min, float64 max, float64 first, float64 last, float64 sum |
TEXT | Binary32 first, Binary32 last |
STRING | Binary32 first, Binary32 last, Binary32 min, Binary32 max |
BLOB, OBJECT, VECTOR | No type-specific fields |
Statistics do not carry a data type identifier. The type used to parse Statistics comes from the Chunk Header or Timeseries Metadata.
first and last describe value order, not the minimum and maximum. For an ordered timeseries, they correspond to the values at the first and last covered timestamps. The start_time and end_time fields are inclusive. Numeric sum fields are intended for aggregation; notably, an INT64 sum is stored as float64 and can lose integer precision for large magnitudes.
Statistics help rule out data that cannot match a query. For example, a Page whose time range falls entirely before the requested range can be skipped. Overlapping ranges only mean the Page might match; the reader still needs to check its values. Types without minimum/maximum fields, such as BLOB, OBJECT, and TEXT, cannot support range pruning through Statistics.
Chunk Statistics summarize all Pages in that Chunk; series Statistics summarize all of its Chunks. Parent counts add the child counts, and parent time bounds cover the child ranges.
Each timeseries has one TimeseriesMetadata structure:
TimeseriesMetadata := byte timeseries_type VarString measurement_id byte data_type UVarInt chunk_metadata_list_size Statistics series_statistics byte chunk_metadata_list[chunk_metadata_list_size]
The high bits of timeseries_type identify the column role, and the low six bits identify the Chunk Metadata layout:
bit 7 = 1: aligned time column bit 6 = 1: aligned value column low 6 bits = 0: exactly one Chunk; ChunkMetadata omits Statistics low 6 bits = 1: more than one Chunk; every ChunkMetadata contains Statistics
When (timeseries_type & 0x3f) == 0, the timeseries has exactly one Chunk:
ChunkMetadata := int64 offset_of_chunk_header
The Chunk uses series_statistics as its Statistics.
When (timeseries_type & 0x3f) != 0:
ChunkMetadata := int64 offset_of_chunk_header Statistics chunk_statistics
offset_of_chunk_header points to the Chunk marker, which is the first byte of the Chunk Header.
chunk_metadata_list_size is the serialized byte length of the list. Add it to the list start to find the end, then read Chunk Metadata records until that end is reached exactly. The field does not give the number of Chunks.
MetadataIndexNode := UVarInt child_count MetadataEntry child[child_count] int64 end_offset byte node_type
node_type determines the node level, Entry encoding, and referenced data:
| Value | Name | Entry encoding | Entry refers to |
|---|---|---|---|
| 0 | INTERNAL_DEVICE | DeviceIndexEntry | A child device-index node |
| 1 | LEAF_DEVICE | DeviceIndexEntry | The measurement-index root of a device |
| 2 | INTERNAL_MEASUREMENT | MeasurementIndexEntry | A child measurement-index node |
| 3 | LEAF_MEASUREMENT | MeasurementIndexEntry | A range of Timeseries Metadata |
The Entry structures are:
DeviceIndexEntry := DeviceID first_device_id int64 offset MeasurementIndexEntry := VarString first_measurement_id int64 offset
An Entry describes a half-open byte range [offset, next_offset). The next Entry supplies next_offset; for the last Entry, use the node's end_offset. The end byte belongs to the next range, not the current one.
entry[0].offset <= entry[1].offset <= ... <= end_offset
An Entry key is the first key in the range it covers. Device indexes compare DeviceID values segment by segment, with null before non-null and a shorter tuple before a longer tuple when their common prefixes are equal. Measurement indexes use lexicographic measurement-name order.
To find key K in an index node, select the last Entry whose first key is less than or equal to K. The selected byte range begins at that Entry‘s offset and ends at the next Entry’s offset, or at the node's end_offset for the last Entry. An internal node range contains another index node; a leaf range contains the object named in the table above.
If no Entry key is less than or equal to K, the key is absent from this node. A measurement leaf can cover several Timeseries Metadata records; read that range to find the exact measurement.
Before following a node, check that:
child_count is consistent with the number of complete Entries in the node.end_offset.The first-key representation allows an implementation to binary-search Entries within a node. The file format does not require a particular in-memory search algorithm.
TsFileMetadata immediately precedes the file's trailing length field:
TsFileMetadata := UVarInt table_index_count TableIndexRoot table_index[table_index_count] UVarInt table_schema_count NamedTableSchema table_schema[table_schema_count] int64 meta_offset BloomFilter bloom_filter VarInt property_count Property property[property_count] TableIndexRoot := VarString table_name MetadataIndexNode device_index_root NamedTableSchema := VarString table_name TableSchema schema Property := VarString key VarBinary value
Property keys are UTF-8 strings; values are nullable raw byte arrays, not UTF-8 strings. For example, value bytes ff fe 00 80 serialize as 08 ff fe 00 80: 08 is VarInt(4). A null value is 01 (VarInt(-1)), while an empty value is 00. Preserve unknown values byte-for-byte, and decode them as text only when the property contract requires it.
meta_offset points to the 0x02 separator at the end of the data section. There is no marker before TsFileMetadata; its starting position is determined by the 4-byte length stored at the end of the file.
The physical order of map entries has no file-format meaning. table_index contains device-index roots for both tree-model and table-model data. table_schema contains schemas for the explicit table model.
The trailing four-byte metadata length is outside TsFileMetadata. For a file length F and metadata length L, a reader obtains the exact byte interval using:
file_metadata_start = F - 6 - 4 - L file_metadata_end = F - 6 - 4 # exclusive
The reader must consume exactly L bytes. This check detects a schema mismatch or malformed length-delimited field before the trailing magic is accepted as the file terminator.
TableSchema := UVarInt column_count TableColumn column[column_count] TableColumn := MeasurementSchema schema int32 column_category MeasurementSchema := String32 column_name byte data_type byte encoding_type byte compression_type int32 property_count SchemaProperty property[property_count] SchemaProperty := String32 key String32 value
MeasurementSchema uses String32, rather than the VarString used by most other footer fields.
Column category identifiers are:
| Value | Category |
|---|---|
| 0 | TAG |
| 1 | FIELD |
| 2 | ATTRIBUTE |
| 3 | TIME |
The schema preserves column order. In particular, TAG column order determines the order of DeviceID segments. FIELD Chunks are identified by measurement_id; their physical order does not have to match the FIELD column order in the schema.
TAG columns identify a device, FIELD columns hold measured values, ATTRIBUTE columns describe an entity, and TIME identifies the time column. A column needs both its category and its data_type: the category describes its role, while the type describes its values.