blob: 99e8335680fd6ba16e1db2ccbf416dff8a71dd40 [file] [view]
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# SQLite + TsFile extension
`tsfile_sqlite` is an experimental SQLite extension for querying TsFile table-model
files and maintaining writable tables with mutable SQLite hot rows and immutable
TsFile history.
Documentation:
- User manual: [English](USER_GUIDE_EN.md) |
[Chinese](USER_GUIDE.md).
- Technical report: [English](TECHNICAL_GUIDE_EN.md) |
[Chinese](TECHNICAL_GUIDE.md).
Build from the repository root:
```bash
cmake -S cpp -B cpp/build/sqlite \
-DBUILD_SQLITE_EXTENSION=ON -DTSFILE_BUILD_SHARED=ON -DBUILD_TEST=ON
cmake --build cpp/build/sqlite --target tsfile_sqlite TsFile_Sqlite_Test -j
ctest --test-dir cpp/build/sqlite/test -R '^TsFileSqliteTest$' --output-on-failure
```
Use SQLite 3.31 or newer with extension loading enabled. On macOS, select
extension-capable headers and libraries, for example by adding
`-DSQLite3_INCLUDE_DIR=/opt/homebrew/opt/sqlite/include` and
`-DSQLite3_LIBRARY=/opt/homebrew/opt/sqlite/lib/libsqlite3.dylib` for Homebrew on
Apple Silicon. Deploy the extension next to `libtsfile` from the same build.
All three creation modes use `tsfile_hybrid`:
```sql
.load /absolute/path/to/tsfile_sqlite
-- Empty writable table: declare its schema and time unit.
CREATE VIRTUAL TABLE sensor USING tsfile_hybrid(
time TIMESTAMP TIME,
device STRING TAG,
temperature DOUBLE FIELD,
directory='/absolute/path/to/sensor-segments',
timestamp_precision='ms'
);
-- Existing history plus a writable hot area: infer the source schema.
CREATE VIRTUAL TABLE continued USING tsfile_hybrid(
file='/archive/history.tsfile',
source_table='sensor',
directory='/absolute/path/to/continued-segments'
);
-- Existing history without a directory: query only.
CREATE VIRTUAL TABLE temp.history USING tsfile_hybrid(
file='/archive/history.tsfile',
source_table='sensor'
);
INSERT INTO sensor VALUES (1700000000000, 'device-1', 21.5);
SELECT * FROM sensor ORDER BY time;
SELECT tsfile_seal('main.sensor', 1700000000001);
SELECT tsfile_export('main.sensor', '/export/sensor-001');
SELECT * FROM tsfile_table_info('main.sensor');
SELECT * FROM tsfile_verify('main.sensor');
```
The writable external-file mode requires a known time unit; supply
`timestamp_precision='ms'`, `'us'`, or `'ns'` when the source has no precision
property. New rows must be strictly later than the selected source table's maximum
time. External files stay unchanged. Actual updates or deletes of cold rows fail
with `SQLITE_READONLY` and roll back the whole statement.
Export automatically seals all current hot rows, commits that seal, then publishes
an independent snapshot as one standard TsFile (zero files for an empty table).
It must run outside an explicit transaction. An output failure after sealing keeps
the committed seal; the error reports the stage.
This version replaces the prototype's `column=` syntax, hidden management columns,
and shadow-table layout. It does not automatically migrate prototype databases.