Google Git
Sign in
apache / hudi / refs/heads/master / . / rfc
tree: b0bd24c7d667bd03c0f7967d89ef0e7d992ce8b2 [path history] [tgz]
  1. rfc-100/
  2. rfc-101/
  3. rfc-27/
  4. rfc-34/
  5. rfc-37/
  6. rfc-38/
  7. rfc-39/
  8. rfc-40/
  9. rfc-41/
  10. rfc-42/
  11. rfc-44/
  12. rfc-45/
  13. rfc-46/
  14. rfc-47/
  15. rfc-48/
  16. rfc-49/
  17. rfc-50/
  18. rfc-51/
  19. rfc-53/
  20. rfc-55/
  21. rfc-56/
  22. rfc-57/
  23. rfc-60/
  24. rfc-63/
  25. rfc-65/
  26. rfc-66/
  27. rfc-68/
  28. rfc-69/
  29. rfc-73/
  30. rfc-76/
  31. rfc-77/
  32. rfc-78/
  33. rfc-8/
  34. rfc-80/
  35. rfc-82/
  36. rfc-83/
  37. rfc-84/
  38. rfc-85/
  39. rfc-87/
  40. rfc-89/
  41. rfc-91/
  42. rfc-92/
  43. rfc-93/
  44. rfc-98/
  45. README.md
  46. template.md
rfc/README.md

RFCs

The RFC process is documented on our site. Please familiarize yourself with it, before working a new RFC.

Status can be one of these values.

StatusMeaning
:eyes: UNDER REVIEWRFC has been proposed and community is actively debating the design/proposal.
:hammer_and_wrench: IN PROGRESSThe initial phase of implementation is underway.
:arrows_counterclockwise: ONGOINGSome or most work has landed; community continues to improve or build follow on phases.
:x: ABANDONEDThe proposal was not implemented, due to various reasons.
:white_check_mark: COMPLETEDAll work is deemed complete.

The list of all RFCs can be found here.

Older RFC content is still here.

RFC NumberTitleStatus
1CSV Source Support for Delta Streamer:white_check_mark: COMPLETED
2ORC Storage in Hudi:white_check_mark: COMPLETED
3Timeline Service with Incremental File System View Syncing:white_check_mark: COMPLETED
4Faster Hive incremental pull queries:white_check_mark: COMPLETED
5HUI (Hudi WebUI):x: ABANDONED
6Add indexing support to the log file:x: ABANDONED
7Point in time Time-Travel queries on Hudi table:white_check_mark: COMPLETED
8Metadata based Record Index:white_check_mark: COMPLETED
9Hudi Dataset Snapshot Exporter:white_check_mark: COMPLETED
10Restructuring and auto-generation of docs:white_check_mark: COMPLETED
11Refactor of the configuration framework of hudi project:x: ABANDONED
12Efficient Migration of Large Parquet Tables to Apache Hudi:white_check_mark: COMPLETED
13Integrate Hudi with Flink:white_check_mark: COMPLETED
14JDBC incremental puller:white_check_mark: COMPLETED
15HUDI File Listing Improvements:white_check_mark: COMPLETED
16Abstraction for HoodieInputFormat and RecordReader:white_check_mark: COMPLETED
17Abstract common meta sync module support multiple meta service:white_check_mark: COMPLETED
18Insert Overwrite API:white_check_mark: COMPLETED
19Clustering data for freshness and query performance:white_check_mark: COMPLETED
20handle failed records:arrows_counterclockwise: ONGOING
21Allow HoodieRecordKey to be Virtual:white_check_mark: COMPLETED
22Snapshot Isolation using Optimistic Concurrency Control for multi-writers:white_check_mark: COMPLETED
23Hudi Observability metrics collection:x: ABANDONED
24Hoodie Flink Writer Proposal:white_check_mark: COMPLETED
25Spark SQL Extension For Hudi:white_check_mark: COMPLETED
26Optimization For Hudi Table Query:white_check_mark: COMPLETED
27Data skipping index to improve query performance:white_check_mark: COMPLETED
28Support Z-order curve:white_check_mark: COMPLETED
29Hash Index:white_check_mark: COMPLETED
30Batch operation:x: ABANDONED
31Hive integration Improvement:x: ABANDONED
32Kafka Connect Sink for Hudi:arrows_counterclockwise: ONGOING
33Hudi supports more comprehensive Schema Evolution:white_check_mark: COMPLETED
34Hudi BigQuery Integration:white_check_mark: COMPLETED
35Make Flink MOR table writing streaming friendly:white_check_mark: COMPLETED
36HUDI Metastore Server:arrows_counterclockwise: ONGOING
37Hudi Metadata based Bloom Index:white_check_mark: COMPLETED
38Spark Datasource V2 Integration:white_check_mark: COMPLETED
39Incremental source for Debezium:white_check_mark: COMPLETED
40Connector for Trino:white_check_mark: COMPLETED
41Snowflake Integration, supported via Apache XTable (Incubating):x: ABANDONED
42Consistent Hashing Index:arrows_counterclockwise: ONGOING
43Table Management Service:x: ABANDONED
44Hudi Connector for Presto:white_check_mark: COMPLETED
45Asynchronous Metadata Indexing:white_check_mark: COMPLETED
46Optimizing Record Payload Handling:white_check_mark: COMPLETED
47Add Call Produce Command for Spark SQL:white_check_mark: COMPLETED
48LogCompaction for MOR tables:white_check_mark: COMPLETED
49Support sync with DataHub:white_check_mark: COMPLETED
50Improve Timeline Server:x: ABANDONED
51Change Data Capture:arrows_counterclockwise: ONGOING
52Introduce Secondary Index to Improve HUDI Query Performance:x: ABANDONED
53Use Lock-Free Message Queue Improving Hoodie Writing Efficiency:white_check_mark: COMPLETED
54New Table APIs and Streamline Hudi Configs:x: ABANDONED
55Improve Hive/Meta sync class design and hierarchies:white_check_mark: COMPLETED
56Early Conflict Detection For Multi-Writer:white_check_mark: COMPLETED
57DeltaStreamer Protobuf Support:white_check_mark: COMPLETED
58Integrate column stats index with all query engines:white_check_mark: COMPLETED
59Multiple event_time Fields Latest Verification in a Single Table:eyes: UNDER REVIEW
60Federated Storage Layer:eyes: UNDER REVIEW
61Snapshot view management:eyes: UNDER REVIEW
62Diagnostic Reporter:eyes: UNDER REVIEW
63Expression Indexes:arrows_counterclockwise: ONGOING
64New Hudi Table Spec API for Query Integrations:eyes: UNDER REVIEW
65Partition TTL Management:white_check_mark: COMPLETED
66Non Blocking Concurrency Control:white_check_mark: COMPLETED
67Hudi Bundle Standards:white_check_mark: COMPLETED
68A More Effective HoodieMergeHandler for COW Table with Parquet:x: ABANDONED
69Hudi 1.x:white_check_mark: COMPLETED
70Hudi Reverse Streamer:eyes: UNDER REVIEW
71Enhance OCC conflict detection:eyes: UNDER REVIEW
72Redesign Hudi-Spark Integration:arrows_counterclockwise: ONGOING
73Multi-Table Transactions:eyes: UNDER REVIEW
74HoodieStorage: Hudi Storage Abstraction and APIs:arrows_counterclockwise: ONGOING
75Hudi-Native HFile Reader and Writer:white_check_mark: COMPLETED
76Auto Record key generation:white_check_mark: COMPLETED
77Secondary Index:white_check_mark: COMPLETED
781.0 Migration:hammer_and_wrench: IN PROGRESS
79Robust handling of spark task retries and failures:x: ABANDONED
80Column Groups:hammer_and_wrench: IN PROGRESS
81Log Compaction with Merge Sort:eyes: UNDER REVIEW
82Concurrent schema evolution detection:white_check_mark: COMPLETED
83Incremental Table Service:white_check_mark: COMPLETED
84Optimized SerDe of DataStream in Flink operators:white_check_mark: COMPLETED
85Hudi Issue and Sprint Management in Jira:white_check_mark: COMPLETED
86DataFrame Implementation of HUDI write path:eyes: UNDER REVIEW
87Avro elimination for Flink writer:hammer_and_wrench: IN PROGRESS
88New Schema/DataType/Expression Abstractions:eyes: UNDER REVIEW
89Dynamic Partition Level Bucket Index:eyes: UNDER REVIEW
90Add support for cancellable clustering table service plans:eyes: UNDER REVIEW
91Storage-based lock provider using conditional writes:hammer_and_wrench: IN PROGRESS
92Support Bitmap Index:hammer_and_wrench: IN PROGRESS
93Pluggable Table Formats in Hudi:hammer_and_wrench: IN PROGRESS
94Hudi Timeline User Interface (UI):eyes: UNDER REVIEW
95Hudi Flink Source Based on FLIP-27:eyes: UNDER REVIEW
96Introduce Unified Bucket Index:eyes: UNDER REVIEW
97Deprecate Hudi Payload Class Usage:eyes: UNDER REVIEW
98Spark Datasource V2 Read:eyes: UNDER REVIEW
99Hudi Type System Redesign:eyes: UNDER REVIEW
100Unstructured Data Storage in Hudi:eyes: UNDER REVIEW
101Updates to the HoodieRecordMerger API:hammer_and_wrench: IN PROGRESS