blob: 4f02f09dd557866d911f7f25ee661cb303cea5f8 [file] [log] [blame] [view]
import ChangeLog from '../changelog/connector-milvus.md';
# Milvus
> Milvus source connector
## Description
This Milvus source connector reads data from Milvus or Zilliz Cloud, it has the following features:
- support read and write data by partition
- support read dynamic schema data into Metadata Column
- json data will be converted to json string and sink as json as well
- retry automatically to bypass ratelimit and grpc limit
## Key Features
- [x] [batch](../../concept/connector-v2-features.md)
- [x] [exactly-once](../../concept/connector-v2-features.md)
- [ ] [column projection](../../concept/connector-v2-features.md)
## Data Type Mapping
| Milvus Data Type | SeaTunnel Data Type |
|---------------------|---------------------|
| INT8 | TINYINT |
| INT16 | SMALLINT |
| INT32 | INT |
| INT64 | BIGINT |
| FLOAT | FLOAT |
| DOUBLE | DOUBLE |
| BOOL | BOOLEAN |
| JSON | STRING |
| ARRAY | ARRAY |
| VARCHAR | STRING |
| FLOAT_VECTOR | FLOAT_VECTOR |
| BINARY_VECTOR | BINARY_VECTOR |
| FLOAT16_VECTOR | FLOAT16_VECTOR |
| BFLOAT16_VECTOR | BFLOAT16_VECTOR |
| SPARSE_FLOAT_VECTOR | SPARSE_FLOAT_VECTOR |
## Source Options
| Name | Type | Required | Default | Description |
|------------|--------|----------|---------|--------------------------------------------------------------------------------------------|
| url | String | Yes | - | The URL to connect to Milvus or Zilliz Cloud. |
| token | String | Yes | - | User:password |
| database | String | Yes | default | Read data from which database. |
| collection | String | No | - | If set, will only read one collection, otherwise will read all collections under database. |
## Task Example
```bash
source {
Milvus {
url = "http://127.0.0.1:19530"
token = "username:password"
database = "default"
}
}
```
## Changelog
<ChangeLog />