How to use:

1. pip install adbc_driver_manager
   pip install adbc_driver_flightsql
2. Modify my_uri, my_db_kwargs, sql in test.py
3. python test.py

What can this demo do:

This is a python demo for doris arrow flight sql, you can use this to test various connection
methods for sending queries to the doris arrow flight server, help you understand how to use arrow flight sql
and test performance.

Performance test

Section 6.1 of https://github.com/apache/doris/issues/25514 is the performance test
results of the doris arrow flight sql using python.

Logical type metadata

Some Doris types share an Arrow storage type with ordinary strings or integers. The doris_type field metadata identifies LARGEINT, IPV4, IPV6, UUID, JSON, and VARIANT, including fields nested in arrays, maps, and structs. Consumers should inspect each nested Arrow field instead of inferring a type from its value.

LARGEINT retains its Arrow string encoding, including the full signed 128-bit range. PyArrow does not automatically convert custom metadata into Python types; a client can use doris_type=LARGEINT to safely convert that field's non-NULL values with int(value), while leaving ordinary STRING fields unchanged.

For types whose physical Arrow layout is unchanged, queries planned by older FEs retain the legacy metadata layout during rolling upgrades. Upgraded FEs advertise complete logical type metadata in the FlightInfo schema even when some BEs are older. An older BE's DoGet stream may still omit these markers; use the FlightInfo schema as the logical type reference until the upgrade finishes. No session setting is required.

Run python test_nested_type_metadata.py with DORIS_FLIGHT_SQL_URI and optional DORIS_USER / DORIS_PASSWORD to verify the metadata and values against a cluster. The tests execute only read-only queries.

Notes

 For more details, refer to [Python Usage] in the document https://doris.apache.org/zh-CN/docs/dev/db-connect/arrow-flight-sql-connect

Native Variant V2 results

Arrow Flight SQL / ADBC returns Variant V2 as native binary values automatically:

SELECT parse_to_variant('{"key":42}') AS v;
-- Request text explicitly when the client needs JSON strings.
SELECT CAST(variant_column AS STRING) FROM example_table;

Variant V2 fields, including nested fields, use the arrow.parquet.variant extension with struct<metadata: binary not null, value: binary not null> storage. SQL NULL is a null struct. Variant null is a non-null struct containing the encoded null value. V2 values retain their physical scalar types and decimal scales. Each Arrow row carries only the dictionary keys it uses, rather than copying keys from unrelated rows.

Master uses Variant V2 throughout. Native Variant queries require backends supporting this binary layout; a backend returning UTF8 is rejected. Use an explicit SQL cast to STRING for text output. Native encoding accepts up to 128 nested levels.

ADBC can transport this schema and its binary values. A client without a registered Variant extension exposes the struct with ARROW:extension:name field metadata. Receiving native VARIANT does not automatically decode it to Python dictionaries or pandas objects; use a Parquet Variant decoder, or explicitly cast the result to STRING.

To check ADBC query and partition reads against a running Variant V2 cluster:

pip install adbc_driver_flightsql pyarrow
export DORIS_FLIGHT_URI='grpc://localhost:8815'
export DORIS_USER='root'
# Set DORIS_PASSWORD in the environment if authentication requires it.
python test_variant.py