[BEAM-12993] Update to Debezium 1.7.0.Final (#15636)
* [BEAM-12993] Update to Debezium 1.7.0.Final
This upgrade is needed to allow Debezium to connect to many cloud
provider's managed Postgres instances. This includes Google CloudSQL
and Azure.
* [BEAM-12993] Add `CHANGES.md` entry
diff --git a/CHANGES.md b/CHANGES.md
index c282f07..647b574 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -108,6 +108,7 @@
## I/Os
+* `DebeziumIO` now uses Debezium 1.7.0.Final ([BEAM-12993](https://issues.apache.org/jira/browse/BEAM-12993)).
* Support for X source added (Java/Python) ([BEAM-X](https://issues.apache.org/jira/browse/BEAM-X)).
## New Features / Improvements
diff --git a/sdks/java/io/debezium/build.gradle b/sdks/java/io/debezium/build.gradle
index 722528f..692736b 100644
--- a/sdks/java/io/debezium/build.gradle
+++ b/sdks/java/io/debezium/build.gradle
@@ -54,8 +54,8 @@
permitUnusedDeclared "org.apache.kafka:connect-json:2.5.0" // BEAM-11761
// Debezium dependencies
- compile group: 'io.debezium', name: 'debezium-core', version: '1.3.1.Final'
- testCompile group: 'io.debezium', name: 'debezium-connector-mysql', version: '1.3.1.Final'
+ compile group: 'io.debezium', name: 'debezium-core', version: '1.7.0.Final'
+ testCompile group: 'io.debezium', name: 'debezium-connector-mysql', version: '1.7.0.Final'
}
test {
diff --git a/sdks/java/io/debezium/expansion-service/build.gradle b/sdks/java/io/debezium/expansion-service/build.gradle
index a183c91..f3a0a30 100644
--- a/sdks/java/io/debezium/expansion-service/build.gradle
+++ b/sdks/java/io/debezium/expansion-service/build.gradle
@@ -38,7 +38,7 @@
runtime library.java.slf4j_jdk14
// Debezium runtime dependencies
- def debezium_version = '1.3.1.Final'
+ def debezium_version = '1.7.0.Final'
runtimeOnly group: 'io.debezium', name: 'debezium-connector-mysql', version: debezium_version
runtimeOnly group: 'io.debezium', name: 'debezium-connector-postgres', version: debezium_version
runtimeOnly group: 'io.debezium', name: 'debezium-connector-sqlserver', version: debezium_version
diff --git a/sdks/java/io/debezium/src/README.md b/sdks/java/io/debezium/src/README.md
index 4cf9be8..14356d6 100644
--- a/sdks/java/io/debezium/src/README.md
+++ b/sdks/java/io/debezium/src/README.md
@@ -25,7 +25,7 @@
### Getting Started
-DebeziumIO uses [Debezium Connectors v1.3](https://debezium.io/documentation/reference/1.3/connectors/) to connect to Apache Beam. All you need to do is choose the Debezium Connector that suits your Debezium setup and pick a [Serializable Function](https://beam.apache.org/releases/javadoc/2.23.0/org/apache/beam/sdk/transforms/SerializableFunction.html), then you will be able to connect to Apache Beam and start building your own Pipelines.
+DebeziumIO uses [Debezium Connectors v1.7](https://debezium.io/documentation/reference/1.7/connectors/) to connect to Apache Beam. All you need to do is choose the Debezium Connector that suits your Debezium setup and pick a [Serializable Function](https://beam.apache.org/releases/javadoc/2.23.0/org/apache/beam/sdk/transforms/SerializableFunction.html), then you will be able to connect to Apache Beam and start building your own Pipelines.
These connectors have been successfully tested and are known to work fine:
* MySQL Connector
@@ -65,7 +65,7 @@
|Method|Params|Description|
|-|-|-|
|`.withConnectionProperty(propName, propValue)`|_String_, _String_|Adds a custom property to the connector.|
-> **Note:** For more information on custom properties, see your [Debezium Connector](https://debezium.io/documentation/reference/1.3/connectors/) specific documentation.
+> **Note:** For more information on custom properties, see your [Debezium Connector](https://debezium.io/documentation/reference/1.7/connectors/) specific documentation.
Example of a MySQL Debezium Connector setup:
```
@@ -165,7 +165,7 @@
### Requirements and Supported versions
- JDK v8
-- Debezium Connectors v1.3
+- Debezium Connectors v1.7
- Apache Beam 2.25
## Running Unit Tests
diff --git a/sdks/java/io/debezium/src/test/java/org/apache/beam/io/debezium/DebeziumIOMySqlConnectorIT.java b/sdks/java/io/debezium/src/test/java/org/apache/beam/io/debezium/DebeziumIOMySqlConnectorIT.java
index 6056ca0..c9b1baf 100644
--- a/sdks/java/io/debezium/src/test/java/org/apache/beam/io/debezium/DebeziumIOMySqlConnectorIT.java
+++ b/sdks/java/io/debezium/src/test/java/org/apache/beam/io/debezium/DebeziumIOMySqlConnectorIT.java
@@ -89,7 +89,7 @@
.withMaxNumberOfRecords(30)
.withCoder(StringUtf8Coder.of()));
String expected =
- "{\"metadata\":{\"connector\":\"mysql\",\"version\":\"1.3.1.Final\",\"name\":\"dbserver1\","
+ "{\"metadata\":{\"connector\":\"mysql\",\"version\":\"1.7.0.Final\",\"name\":\"dbserver1\","
+ "\"database\":\"inventory\",\"schema\":\"mysql-bin.000003\",\"table\":\"addresses\"},\"before\":null,"
+ "\"after\":{\"fields\":{\"zip\":\"76036\",\"city\":\"Euless\","
+ "\"street\":\"3183 Moore Avenue\",\"id\":10,\"state\":\"Texas\",\"customer_id\":1001,"
