<!--
Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
-->
This library contains code for ADBC drivers built on top of the Thrift protocol with Arrow support:
Each driver is at a different state of implementation.
Typically, Thrift code is generated from the Thrift compiler. And that is mostly true here as well. However, some files were further edited to include Arrow support. These contain the phrase BUT THIS FILE HAS BEEN HAND EDITED TO SUPPORT ARROW SO REGENERATE AT YOUR OWN RISK at the top. Some of these files include:
arrow-adbc/csharp/src/Drivers/Apache/Thrift/Service/Rpc/Thrift/TBinaryColumn.cs arrow-adbc/csharp/src/Drivers/Apache/Thrift/Service/Rpc/Thrift/TBoolColumn.cs arrow-adbc/csharp/src/Drivers/Apache/Thrift/Service/Rpc/Thrift/TByteColumn.cs arrow-adbc/csharp/src/Drivers/Apache/Thrift/Service/Rpc/Thrift/TDoubleColumn.cs arrow-adbc/csharp/src/Drivers/Apache/Thrift/Service/Rpc/Thrift/TI16Column.cs arrow-adbc/csharp/src/Drivers/Apache/Thrift/Service/Rpc/Thrift/TI32Column.cs arrow-adbc/csharp/src/Drivers/Apache/Thrift/Service/Rpc/Thrift/TI64Column.cs arrow-adbc/csharp/src/Drivers/Apache/Thrift/Service/Rpc/Thrift/TStringColumn.cs
The Hive classes serve as the base class for Spark and Impala, since both of those platform implement Hive capabilities.
Core functionality of the Hive classes beyond the base library implementation is under development, has limited functionality, and may produce errors.
The Imapala classes are under development, have limited functionality, and may produce errors.
The Spark classes are intended for use against native Spark and Spark on Databricks.
The following table depicts how the Spark ADBC driver converts a Spark type to an Arrow type and a .NET type:
| Spark Type | Arrow Type | C# Type |
|---|---|---|
| ARRAY* | String | string |
| BIGINT | Int64 | long |
| BINARY | Binary | byte[] |
| BOOLEAN | Boolean | bool |
| CHAR | String | string |
| DATE | Date32 | DateTime |
| DECIMAL | Decimal128 | SqlDecimal |
| DOUBLE | Double | double |
| FLOAT | Float | float |
| INT | Int32 | int |
| INTERVAL_DAY_TIME+ | String | string |
| INTERVAL_YEAR_MONTH+ | String | string |
| MAP* | String | string |
| NULL | Null | null |
| SMALLINT | Int16 | short |
| STRING | String | string |
| STRUCT* | String | string |
| TIMESTAMP | Timestamp | DateTimeOffset |
| TINYINT | Int8 | sbyte |
| UNION | String | string |
| USER_DEFINED | String | string |
| VARCHAR | String | string |
* Complex types are returned as strings
+ Interval types are returned as strings
SparkConnection.GetObjects is not fully tested at this timeConnection.GetTableSchema does not return correct precision and scale for NUMERIC/DECIMAL types.NULL value is returned for a BINARY type it is instead being returned as an empty array instead of the expected null.nullable by default, which may not be accurate.