blob: 73a7669a46d319f743415d99216d766d585c31e6 [file] [log] [blame]
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Elasticsearch
> Elasticsearch sink connector
## Description
Output data to `Elasticsearch`.
:::tip
Engine Supported and plugin name
* [x] Spark: Elasticsearch(supported `ElasticSearch version is >= 2.x and <7.0.0`)
* [x] Flink: Elasticsearch(supported `ElasticSearch version = 7.x`, if you want use Elasticsearch version is 6.x,
please use the source code to repackage by execute `mvn clean package -Delasticsearch=6`)
:::
## Options
<Tabs
groupId="engine-type"
defaultValue="spark"
values={[
{label: 'Spark', value: 'spark'},
{label: 'Flink', value: 'flink'},
]}>
<TabItem value="spark">
| name | type | required | default value |
| ----------------- | ------ | -------- | ------------- |
| hosts | array | yes | - |
| index_type | string | no | - |
| index_time_format | string | no | yyyy.MM.dd |
| index | string | no | seatunnel |
| es.* | string | no | |
| common-options | string | no | - |
</TabItem>
<TabItem value="flink">
| name | type | required | default value |
| ----------------- | ------ | -------- | ------------- |
| hosts | array | yes | - |
| index_type | string | no | log |
| index_time_format | string | no | yyyy.MM.dd |
| index | string | no | seatunnel |
| common-options | string | no | - |
| parallelism | int | no | - |
</TabItem>
</Tabs>
### hosts [array]
`Elasticsearch` cluster address, the format is `host:port` , allowing multiple hosts to be specified. Such as `["host1:9200", "host2:9200"]` .
### index_type [string]
`Elasticsearch` index type, it is recommended not to specify in elasticsearch 7 and above
#### index_time_format [string]
When the format in the `index` parameter is `xxxx-${now}` , `index_time_format` can specify the time format of the `index` name, and the default value is `yyyy.MM.dd` . The commonly used time formats are listed as follows:
| Symbol | Description |
| ------ | ------------------ |
| y | Year |
| M | Month |
| d | Day of month |
| H | Hour in day (0-23) |
| m | Minute in hour |
| s | Second in minute |
See [Java SimpleDateFormat](https://docs.oracle.com/javase/tutorial/i18n/format/simpleDateFormat.html) for detailed time format syntax.
### index [string]
Elasticsearch `index` name. If you need to generate an `index` based on time, you can specify a time variable, such as `seatunnel-${now}` . `now` represents the current data processing time.
<Tabs
groupId="engine-type"
defaultValue="spark"
values={[
{label: 'Spark', value: 'spark'},
{label: 'Flink', value: 'flink'},
]}>
<TabItem value="spark">
### es.* [string]
Users can also specify multiple optional parameters. For a detailed list of parameters, see [Parameters Supported by Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html#cfg-mapping).
For example, the way to specify `es.batch.size.entries` is: `es.batch.size.entries = 100000` . If these non-essential parameters are not specified, they will use the default values given in the official documentation.
</TabItem>
<TabItem value="flink">
### parallelism [`Int`]
The parallelism of an individual operator, data source, or data sink
</TabItem>
</Tabs>
### common options [string]
Sink plugin common parameters, please refer to [Sink Plugin](common-options.md) for details
## Examples
```bash
elasticsearch {
hosts = ["localhost:9200"]
index = "seatunnel"
}
```