This repo contains the code that integrate Hudi with Spark. The repo is split into the following modules
hudi-spark hudi-spark3.3.x hudi-spark3.4.x hudi-spark3.5.x hudi-spark-common
HoodieSpark3_2ExtendedSqlAstBuilder have comments in the spark3.2's code fork from org.apache.spark.sql.catalyst.parser.AstBuilder, and additional withTimeTravel method.SqlBase.g4 have comments in the code forked from spark3.2's parser, and add SparkSQL Syntax TIMESTAMP AS OF and VERSION AS OF.| version | support |
|---|---|
| 2.4.x | No |
| 3.0.x | No |
| 3.1.2 | No |
| 3.2.0 | Yes |
Spark3.3 support time travel syntax link SPARK-37219. Once Spark 3.3 released. The files in the following list will be removed:
HoodieSpark3_3ExtendedSqlAstBuilder.scala, HoodieSpark3_3ExtendedSqlParser.scala, TimeTravelRelation.scala, SqlBase.g4, HoodieSqlBase.g4 Tracking Jira: HUDI-4468Some other improvements undergoing: