The Go package github.com/apache/tsfile/go/tsfile is a cgo binding over the C++ implementation, so a C/C++ build of TsFile must be available before Go code can be built. There are two ways to get it:
The Go module itself is fetched from GitHub in both cases; it is not published to the module proxy yet.
CGO_ENABLED=1Install the runtime and development files as described on the download and installation page, for example the SDK archive, the DEB/RPM packages, the Homebrew formula, or the Windows ZIP. Headers are installed under an include/tsfile subdirectory, so point cgo at that directory and at the library directory of the install prefix:
export TSFILE_HOME=/path/to/tsfile-sdk # or: $(brew --prefix local/tsfile250/tsfile) export CGO_CFLAGS="-I$TSFILE_HOME/include/tsfile" export CGO_LDFLAGS="-L$TSFILE_HOME/lib -Wl,-rpath,$TSFILE_HOME/lib"
The -rpath entry lets the compiled program find libtsfile at run time; drop it when the shared library is installed in a standard system directory. For a system installation under /usr, the compiler finds the library without -L, but the headers still need -I/usr/include/tsfile. The environment variables are added to the package's own flags, whose paths point at a source checkout under cpp/target/build; when you use the installed packages those paths do not exist, and the linker prints a search-path warning that can be ignored.
On Windows, extract the ZIP and set the same variables in PowerShell. The ZIP ships bin\tsfile.dll and an import library in lib\tsfile.lib that the MinGW linker accepts, so -ltsfile resolves through -L as usual:
$env:TSFILE_HOME = "C:\path\to\tsfile-2.5.0-windows-x86_64" $env:CGO_CFLAGS = "-I$env:TSFILE_HOME\include\tsfile" $env:CGO_LDFLAGS = "-L$env:TSFILE_HOME\lib -ltsfile" $env:Path = "$env:TSFILE_HOME\bin;$env:Path"
Windows has no run-time library search path, so the compiled program finds tsfile.dll through PATH; keep bin on PATH, or copy tsfile.dll next to the executable. The DLL links the C runtime statically and depends only on the system libraries, so no Visual C++ redistributable is required.
cgo reads these variables when it compiles the package, so set them in every shell where you build a program that links against the installed library, or record them with go env -w CGO_CFLAGS=... CGO_LDFLAGS=.... They are not needed when running an already built program.
Clone the source code and build the C++ library in the repository root:
git clone https://github.com/apache/tsfile.git cd tsfile ./mvnw -P with-cpp -DskipTests package
This produces the headers under cpp/target/build/include and the library under cpp/target/build/lib that the Go package links against by default.
On Windows, build with MinGW. MinGW-w64 is this project‘s default C++ generator on Windows, and it is the compiler that cgo uses for the Go package, so -Dcpp.toolchain=mingw makes the build produce a DLL and an import library that match the Go toolchain directly. The build writes tsfile.dll to cpp\target\build\lib instead of a package’s bin directory, so point PATH there (or copy the DLL next to the executable) in addition to the MinGW-w64 bin directory that provides the compiler:
.\mvnw.cmd -P with-cpp -Dcpp.toolchain=mingw -DskipTests package $env:Path = "$PWD\cpp\target\build\lib;C:\mingw64\bin;$env:Path"
Run that PATH assignment in every new terminal before building or running Go programs.
With the installed packages (option 1), fetch the module directly in your Go project:
go get github.com/apache/tsfile/go@develop go mod tidy
The module has no tagged release yet, so @develop resolves to a pseudo-version of the branch head. Record that pseudo-version in go.mod, or pin a specific commit, for reproducible builds.
With a source build (option 2), replace the module with the go directory of your checkout; the package then finds the headers and the library built next to it:
go mod edit -replace github.com/apache/tsfile/go=/path/to/tsfile/go go get github.com/apache/tsfile/go
To run the bundled example instead, from the go directory of the checkout:
cd go go run ./examples/table_read_write
schema := tsfile.TableSchema{
Table: "table1",
Columns: []tsfile.ColumnSchema{
{Name: "id1", DataType: tsfile.DataTypeString, Category: tsfile.ColumnCategoryTag},
{Name: "id2", DataType: tsfile.DataTypeString, Category: tsfile.ColumnCategoryTag},
{Name: "s1", DataType: tsfile.DataTypeInt32, Category: tsfile.ColumnCategoryField},
{Name: "s2", DataType: tsfile.DataTypeBoolean, Category: tsfile.ColumnCategoryField},
},
}
writer, err := tsfile.NewWriter("test.tsfile", schema, tsfile.WithMemoryThreshold(10*1024*1024))
if err != nil {
return err
}
defer writer.Close()
NewWriter creates or truncates the file. WithMemoryThreshold is optional and defaults to 128 MB.
tablet, err := tsfile.NewTablet([]tsfile.TabletColumn{
{Name: "id1", DataType: tsfile.DataTypeString},
{Name: "id2", DataType: tsfile.DataTypeString},
{Name: "s1", DataType: tsfile.DataTypeInt32},
{Name: "s2", DataType: tsfile.DataTypeBoolean},
}, 5)
if err != nil {
return err
}
defer tablet.Close()
for row := 0; row < 5; row++ {
if err := tablet.AddTimestamp(row, int64(row)); err != nil {
return err
}
tablet.SetString(row, 0, "id1_filed_1")
tablet.SetString(row, 1, "id2_filed_1")
tablet.SetInt32(row, 2, int32(row))
tablet.SetBool(row, 3, true)
}
if err := writer.WriteTableTablet(tablet); err != nil {
return err
}
Tablet rows and columns are zero-based. Every setter validates the index and the column type, and returns an error when the row or column is out of range or the value does not match the declared type; check those errors in the same way as AddTimestamp.
Data can also be written from an Arrow record batch or table. The schema must contain a time column of type int64 or timestamp[ns] without a time zone, followed by exactly the columns of the table schema:
arrowSchema := arrow.NewSchema([]arrow.Field{
{Name: "time", Type: arrow.PrimitiveTypes.Int64},
{Name: "id1", Type: arrow.BinaryTypes.String},
{Name: "id2", Type: arrow.BinaryTypes.String},
{Name: "s1", Type: arrow.PrimitiveTypes.Int32},
{Name: "s2", Type: arrow.FixedWidthTypes.Boolean},
}, nil)
builder := array.NewRecordBuilder(memory.DefaultAllocator, arrowSchema)
defer builder.Release()
timeBuilder := builder.Field(0).(*array.Int64Builder)
id1Builder := builder.Field(1).(*array.StringBuilder)
id2Builder := builder.Field(2).(*array.StringBuilder)
s1Builder := builder.Field(3).(*array.Int32Builder)
s2Builder := builder.Field(4).(*array.BooleanBuilder)
for row := 5; row < 10; row++ {
timeBuilder.Append(int64(row))
id1Builder.Append("id1_filed_1")
id2Builder.Append("id2_filed_1")
s1Builder.Append(int32(row))
s2Builder.Append(true)
}
record := builder.NewRecord()
defer record.Release()
if err := writer.WriteArrowBatch(record); err != nil {
return err
}
writer.Close()
The sample code of using these interfaces is in https://github.com/apache/tsfile/blob/develop/go/examples/table_read_write/main.go
reader, err := tsfile.NewReader("test.tsfile")
if err != nil {
return err
}
defer reader.Close()
result, err := reader.Query(
"table1",
[]string{"id1", "id2", "s1", "s2"},
tsfile.WithTimeRange(2, 8),
tsfile.WithTagFilter(tsfile.And(tsfile.Eq("id1", "id1_filed_1"), tsfile.Eq("id2", "id2_filed_1"))),
tsfile.WithLimit(100),
)
if err != nil {
return err
}
defer result.Close()
Time range, tag filter, offset, limit, and batch size are all optional and are pushed down to the native reader.
metadata := result.Metadata()
for _, column := range metadata {
fmt.Printf("%s(%d) ", column.Name, column.DataType)
}
fmt.Println()
for {
hasNext, err := result.Next()
if err != nil {
return err
}
if !hasNext {
break
}
for column := 1; column <= len(metadata); column++ {
isNull, err := result.IsNull(column)
if err != nil {
return err
}
if isNull {
fmt.Print("null ")
continue
}
switch metadata[column-1].DataType {
case tsfile.DataTypeBoolean:
value, err := result.Bool(column)
if err != nil {
return err
}
fmt.Print(value, " ")
case tsfile.DataTypeInt32, tsfile.DataTypeDate:
value, err := result.Int32(column)
if err != nil {
return err
}
fmt.Print(value, " ")
case tsfile.DataTypeInt64, tsfile.DataTypeTimestamp:
value, err := result.Int64(column)
if err != nil {
return err
}
fmt.Print(value, " ")
case tsfile.DataTypeFloat:
value, err := result.Float32(column)
if err != nil {
return err
}
fmt.Print(value, " ")
case tsfile.DataTypeDouble:
value, err := result.Float64(column)
if err != nil {
return err
}
fmt.Print(value, " ")
case tsfile.DataTypeString, tsfile.DataTypeText:
value, err := result.String(column)
if err != nil {
return err
}
fmt.Print(value, " ")
}
}
fmt.Println()
}
Result-set columns are one-based; column 1 is the timestamp and the selected columns follow in query order. The typed getters (Bool, Int32, Int64, Float32, Float64, String, Bytes) reject a column whose data type does not match, so dispatch on the metadata as shown above.
When WithBatchSize is positive the query runs in batch mode and rows are returned as Arrow record batches:
result, err := reader.Query("table1", []string{"id1", "id2", "s1", "s2"}, tsfile.WithBatchSize(1024))
if err != nil {
return err
}
defer result.Close()
for {
record, err := result.ReadArrowRecordBatch()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
return err
}
fmt.Println(record.NumRows())
record.Release()
}
result.Close()
reader.Close()
Closing the reader also closes every result set created from it.
The sample code of using these interfaces is in https://github.com/apache/tsfile/blob/develop/go/examples/table_read_write/main.go
Note: The above read/write examples are all based on the table model interface. For details about the interface definition, please refer to Go Interface Definition. If you need information regarding the tree model, please contact us.