<!-- Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License. -->

# FLOAT/DOUBLE TS_2DIFF Wire Format (Java Canonical Layout)

This document specifies the canonical on-disk layout of FLOAT/DOUBLE TS_2DIFF
pages, derived from the Java reference implementation
(`FloatEncoder`, `FloatDecoder`, `DeltaBinaryEncoder`, `BitMap`).
The Java layout is the cross-language compatibility boundary. Other layouts
produced by earlier C++ writers (raw bit-cast, per-block wrapper metadata) are
implementation artifacts outside the compatibility scope; the C++ decoder
treats them as a format error.

## Encoding Pipeline

TS_2DIFF encodes integers. Floating-point values go through a wrapper that
converts each value to an integer, encodes the integers with
`IntDeltaEncoder` (FLOAT) or `LongDeltaEncoder` (DOUBLE), and emits page-wide
conversion metadata.

Given `maxPointNumber = mpn` and `maxPointValue = 10^mpn` (`mpn <= 0` implies
`maxPointValue = 1`), each value maps to one of three stored forms:

| Condition                              | Stored bits                 | Decoder action            |
| -------------------------------------- | --------------------------- | ------------------------- |
| `round(v * 10^mpn)` fits the int type  | `round(v * 10^mpn)`         | divide by `10^mpn`        |
| scaled overflows but `v` itself fits   | `round(v)`                  | divide by `1`             |
| `v` out of int range, or NaN           | `floatToIntBits(v)` / `doubleToLongBits(v)` | restore raw bits |

The three forms are tracked per page as a tri-state flag list
(`underflowFlags` in Java):

- `true`  -> scaled form
- `false` -> rounded form (scale overflow)
- `null`  -> raw IEEE 754 bits (value overflow or NaN)

`round` is Java `Math.round` semantics, `floor(x + 0.5)` — ties go towards
`+infinity` (`-2.5 -> -2`), unlike C `lround`'s ties-away-from-zero.

## Page Layout

```text
# Form 1: every value stored in scaled form (no bitmap at all)
[maxPointNumber varint]
[TS_2DIFF block 1][TS_2DIFF block 2]...[final block]

# Form 2: at least one value is 'false' (scale overflow), none is 'null'
[Integer.MAX_VALUE varint]                    # 0xFF 0xFF 0xFF 0xFF 0x07
[pageValueCount varint]
[scaled-bitmap, pageValueCount/8+1 bytes]     # marks 'true' entries
[maxPointNumber varint]
[TS_2DIFF block 1]...[final block]

# Form 3: at least one value is 'null' (raw bits)
[Integer.MAX_VALUE-1 varint]                  # 0xFF 0xFF 0xFF 0xFF 0x06
[pageValueCount varint]
[scaled-bitmap, pageValueCount/8+1 bytes]     # marks 'true' entries
[raw-bitmap, pageValueCount/8+1 bytes]        # marks 'null' entries
[maxPointNumber varint]
[TS_2DIFF block 1]...[final block]
```

Key invariants:

- `maxPointNumber` appears exactly once per page, before the first integer
  block (for Forms 2/3 it appears after the bitmaps).
- The bitmaps cover the entire page, not individual TS_2DIFF blocks. The
  decoder keeps one page-wide `position` that never resets between blocks;
  only a page-level `reset()` clears it.
- Bitmap byte length is always `size/8 + 1`, even when `size % 8 == 0`
  (`BitMap.getSizeOfBytes`).
- Bitmap bit order is LSB-first within each byte: position `p` maps to
  `bits[p / 8] & (1 << (p % 8))`.
- `pageValueCount` counts all values of the page (across blocks).
- A first-page byte of `0x00` is the normal encoding of `maxPointNumber = 0`
  (an explicit `max_point_number=0` property on the Java side; reachable
  but not the default — see Encoder Construction below). It is not a
  legacy marker.
- NaN handling is writer-dependent: Java `floatToIntBits` /
  `doubleToLongBits` canonicalize any NaN to `0x7fc00000` /
  `0x7ff8000000000000`, while the C++ encoder preserves the payload bits.
  Both are valid raw-bit entries; readers restore the bits as stored.

## Integer Block Layout

Identical to the integer TS_2DIFF format (`DeltaBinaryEncoder`):

```text
[writeIndex int32 BE]   # number of deltas in this block
[bitWidth  int32 BE]
[block-specific header] # first value; min delta
[packed data]           # writeIndex * bitWidth bits
```

A block stores `writeIndex + 1` values (first value + writeIndex deltas).
`BLOCK_DEFAULT_SIZE = 128` is only `DeltaBinaryEncoder`'s default buffer
size, not a wire-format limit: Java exposes block-size constructors
(`IntDeltaEncoder(int)`), so `writeIndex` is bounded only by the declared
page value count and the packed bytes available. A 300-value page from the
default encoder produces blocks of 129, 129, 42 values.

## Encoder Construction

The Java `TSEncodingBuilder.Ts2Diff` field initializes `maxPointNumber = 0`,
but the standard schema write path (`MeasurementSchema.getValueEncoder`)
always calls `initFromProps()`, which replaces it with the schema's
`max_point_number` property or, when the property is absent, with
`TSFileConfig.floatPrecision` (current default `2`). A writer that
explicitly sets `max_point_number = 0` produces Form 1 pages starting with
`0x00`. The C++ `FloatTS2DIFFEncoder` / `DoubleTS2DIFFEncoder` default to
`2`, matching the standard Java schema path. The value stored in the
stream is self-describing, so files written with other `maxPointNumber`
values remain readable.

## Decoder State Machine

Per page, exactly once, the decoder reads the leading marker:

1. Read varint `tag`.
2. `tag == Integer.MAX_VALUE`   -> read `count` varint, `count/8+1` bytes
   scaled-bitmap, then varint `maxPointNumber` (Form 2).
3. `tag == Integer.MAX_VALUE-1` -> additionally read a second
   `count/8+1` bytes raw-bitmap (Form 3).
4. Otherwise `tag` itself is `maxPointNumber` (Form 1); `mpn <= 0` means
   `maxPointValue = 1`.

Then values are decoded from the integer blocks. For value at page position
`p`:

- raw-bitmap (if present) marks `p` -> `intBitsToFloat` / `longBitsToDouble`
- else scaled-bitmap (if present) marks `p` -> `value / 10^mpn`
- else -> `value / 1`

Any input that does not conform to this grammar (for example, an integer
TS_2DIFF block header where the page metadata is expected) is a format error.