[flink] optimize flink Data-Evolution-Merge-Into (#7324) This PR optimize Data-Evolution-Merge-Into in several aspects: 1. Introduce a MergeIntoUpdateChecker to check if some global-indexed columns are updated (This is same as Spark's implementation) 2. Use calcite to rename target table (current implementation is based on regex, which is very unstable) 3. Use calcite to find _row_id field (if exists) in source table. We can eliminate join process.
Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations. Paimon innovatively combines lake format and LSM structure, bringing realtime streaming updates into the lake architecture.
Background and documentation are available at https://paimon.apache.org
Paimon's former name was Flink Table Store, developed from the Flink community. The architecture refers to some design concepts of Iceberg. Thanks to Apache Flink and Apache Iceberg.
Paimon tracks issues in GitHub and prefers to receive contributions as pull requests.
Please make sure you are subscribed to the mailing list you are posting to! If you are not subscribed to the mailing list, your message will either be rejected (dev@ list) or you won't receive the response (user@ list).
You can join the Paimon community on Slack. Paimon channel is in ASF Slack workspace.
user@paimon.apache.org to apply for an ASF Slack invitation. Then join Paimon channel.JDK 8/11 is required for building the project. Maven version >=3.6.3.
mvn clean install -DskipTests command to build the project.mvn spotless:apply to format the project (both Java and Scala).paimon-common/target/generated-sources/antlr4 as Sources Root.The code in this repository is licensed under the Apache Software License 2.