tree 4c75edbbfc9eaa537fa7addf4f638450a7ca9353
parent 8e6cdbeae0788594ed64334a1019a7cef424abff
author Arina Ielchiieva <arina.yelchiyeva@gmail.com> 1573826512 +0200
committer Arina Ielchiieva <arina.yelchiyeva@gmail.com> 1574243680 +0200

DRILL-7388: Kafka improvements

1. Upgraded Kafka libraries to 2.3.1 (DRILL-6739).
2. Added new options to support the same features as native JSON reader:
  a. store.kafka.reader.skip_invalid_records, default: false (DRILL-6723);
  b. store.kafka.reader.allow_nan_inf, default: true;
  c. store.kafka.reader.allow_escape_any_char, default: false.
3. Fixed issue when Kafka topic contains only one message (DRILL-7388).
4. Replaced Gson parser with Jackson to parse JSON in the same manner as Drill native Json reader.
5. Performance improvements: Kafka consumers will be closed async, fixed issue with resource leak (DRILL-7290), moved to debug unnecessary info logging.
6. Updated bootstrap-storage-plugins.json to reflect actual Kafka connection properties.
7. Added unit tests.
8. Refactoring and code clean up.

closes #1901