Drill ElasticSearch storage plugin allows you to perform SQL queries against ElasticSearch indices. This storage plugin implementation is based on Apache Calcite adapter for ElasticSearch.
For more details about supported versions please refer to Supported versions page.
This storage plugin supports the following optimizations:
Besides these optimizations, ElasticSearch storage plugin supports the schema provisioning feature. For more details please refer to Specifying the Schema as Table Function Parameter.
The plugin can be registered in Apache Drill using the drill web interface by navigating to the storage
page. Following is the default registration configuration.
{ "type": "elastic", "hosts": [ "http://localhost:9200" ], "username": null, "password": null, "disableSSLVerification": false, "enabled": false }
The ElasticSearch plugin supports user translation, which allows each individual user to access ElasticSearch with their own credentials.
Simply add the following to the storage plugin configuration:
"authType": "USER_TRANSLATION", "credentialsProvider": { "credentialsProviderType": "PlainCredentialsProvider" }
Most of the common classes required for creating storage plugins based on Calcite adapters are placed in the java-exec
module, so they can be reused in future plugin implementations.
Here is the list of the classes that may be useful:
VertexDrelConverterRule
with VertexDrel
- used to hold plugin-specific part of the plan at the end of the LOGICAL
planning phase.EnumerableIntermediatePrelConverterRule
with EnumerableIntermediatePrel
- the same as above, but for the PHYSICAL
planning phase.EnumerablePrel
- responsible for generating java code that will be executed to query the storage plugin data source.EnumerableRecordReader
- executes java code generated in EnumerablePrel
and transforms obtained results to Drill internal representation.