Google Git
Sign in
apache/celeborn/HEAD
  1. 391ef4b [CELEBORN-2273] Fix cache mutation in TagsManager.getTaggedWorkers() by ShlomiTubul · 2 days ago main
  2. dca3749 [CELEBORN-2218] Bump lz4-java version from 1.8.0 to 1.10.4 to resolve CVE‐2025‐12183 and CVE-2025-66566 by SteNicholas · 4 days ago
  3. 13ea40c [CELEBORN-2272] Add LZ4TPCDSDataBenchmark by Cheng Pan · 5 days ago
  4. ffdb416 [CELEBORN-2270] Fix problem with eviction to tiered storage during partition split by Enrico Olivelli · 5 days ago
  5. 0a67e2b [CELEBORN-2049] Bump Ratis version from 3.1.3 to 3.2.1 by SteNicholas · 5 days ago
  6. fddb817 [CELEBORN-2226][CIP-14] Support RetryFetchChunk functionality for Cel… by afterincomparableyum · 6 days ago
  7. deb7538 [CELEBORN-2248] Implement lazy loading for columnar shuffle classes and skew shuffle method using static holder pattern by zhenghuan · 7 days ago
  8. bc4dc12 [CELEBORN-2239] Support Spark 4.1 by SteNicholas · 8 days ago
  9. e119902 [CELEBORN-2268] Improve test coverage for MEMORY and S3 storage by Enrico Olivelli · 8 days ago
  10. 9d48e6c [CELEBORN-2258] Bump Netty version from 4.1.118.Final to 4.2.10.Final by SteNicholas · 8 days ago
  11. ad5a881 [CELEBORN-2267][FOLLOWUP] Add Cpp-Write Java-Read integration tests for LZ4 and ZSTD by afterincomparableyum · 9 days ago
  12. 2460efd [CELEBORN-2222][CIP-14] Support Retrying when createReader failed for CelebornInputStream in CppClient by afterincomparableyum · 9 days ago
  13. 4d97a85 [CELEBORN-2271] StorageManager#saveCommittedFileInfosExecutor should call shutdown before awaitTermination by Prateek Srivastava · 9 days ago
  14. feb3ed9 [CELEBORN-2262] Prepare S3 directory only once and cache s3 client for MultiPartUploader by Enrico Olivelli · 10 days ago
  15. ac6d1cf [CELEBORN-2269] Update Cpp TransportClient to resolve hostnames via DNS by jay.narale · 11 days ago
  16. e4836b7 [CELEBORN-2266] Modernize Protobuf CMake usage and add install rules by jay.narale · 2 weeks ago
  17. 13cd4a9 [CELEBORN-2265] Do not waste resources on hotpath for debug logging in HashBasedShuffleWriter and SortBasedShuffleWriter by Enrico Olivelli · 3 weeks ago
  18. eb7a720 [CELEBORN-2259] The S3MultipartUploadHandler uses fs.s3a.aws.credentials.provider by Dzeri96 · 3 weeks ago
  19. 8e6f4d5 [CELEBORN-2063] Parallelize the create partition writer in handleReserveSlots to speed up the reserveSlots RPC process time by SteNicholas · 3 weeks ago
  20. b1cbdab [CELEBORN-2221][CIP-14] Support writing with compression in C++ client by afterincomparableyum · 3 weeks ago
  21. 4ed9cbd [CELEBORN-2263] Fix IndexOutOfBoundsException while reading from S3 by Enrico Olivelli · 3 weeks ago
  22. 81d89f3 [CELEBORN-2256] Helm chart: add support for setting annotations on the service account (to support eks.amazonaws.com/role-arn) by Enrico Olivelli · 4 weeks ago
  23. b659439e [CELEBORN-2254] Fix support for S3 and add a simple integration test by Enrico Olivelli · 4 weeks ago
  24. 2097ad0 [CELEBORN-2205] Introduce metrics to fetch chunk for memory and local disk by xxx · 4 weeks ago
  25. 4752936 [CELEBORN-1577][FOLLOWUP] Fix master resource consumption metrics by Sanskar Modi · 4 weeks ago
  26. 46a49a8 [CELEBORN-2250] Fix lock contention in ReducePartitionCommitHandler.finishMapperAttempt via fine-grained locks by yew1eb · 6 weeks ago
  27. f04dddd [CELEBORN-2251] Introducing a shim layer and a common-tiered submodule for Flink clients by luogen.lg · 6 weeks ago
  28. e81cea0 [CELEBORN-2245] Bump maven 3.9.12 by sychen · 6 weeks ago
  29. 9083461 [CELEBORN-2249] Bump Spark from 3.5.7 to 3.5.8 by yew1eb · 7 weeks ago
  30. 4749382 [CELEBORN-2211] Avoid allocating additional buffers When HdfsFlushTask writes data by xxx · 7 weeks ago
  31. 7e4bea0 [CELEBORN-2243][FOLLOWUP][SPARK-2] During the close phase of hashWriter, pushData and mergeData are sent in parallel by jiucheng · 8 weeks ago
  32. 42ff69f [CELEBORN-2229][CIP-14] Add support for celeborn.client.push.maxBytesSizeInFlight in CppClient by afterincomparableyum · 8 weeks ago
  33. 9fca94b [CELEBORN-2243] During the close phase of hashWriter, pushData and mergeData are sent in parallel by jiucheng · 8 weeks ago
  34. d30c02e [CELEBORN-2235][CIP-14] Adapt Java end's serialization to CppWriterClient by HolyLow · 9 weeks ago
  35. 2dd1b7a [CELEBORN-2210] When a flushBuffer consolidation OOM exception occurs… by xxx · 9 weeks ago
  36. 783a7cc [CELEBORN-2236] Avoiding regular expressions for DiskFileInfo storage type determination by xxx · 9 weeks ago
  37. 38532d7 [CELEBORN-2166][FOLLOWUP] Update config celeborn.client.shuffleDataLostOnUnknownWorker.enabled version to 0.6.3 by Wang, Fei · 10 weeks ago
  38. 526f7bb [CELEBORN-2240] Adapt to SPARK-51756 which add a new parameter `checksumValue` in `MapStatus.apply` by Cheng Pan · 3 months ago
  39. ac8cffb [CELEBORN-2231] Upgrade jersey version to 2.47 to fix CVE-2025-12383 by Wang, Fei · 3 months ago
  40. 5789b4e [CELEBORN-2217] Use a separate thread to handle RpcEndpointVerifier messages by TheodoreLx · 3 months ago
  41. 0f663d0 [CELEBORN-1577][FOLLOWUP] Fix addGauge error in registerUserResourceCons… by xxx · 3 months ago
  42. 77a8921 [CELEBORN-2212] Optimize the sorting efficiency of memoryWriters when evicting the largest memory file by xxx · 3 months ago
  43. 36d6b0c [CELEBORN-2237] Support Flink 2.2 by SteNicholas · 3 months ago
  44. 2233699 [CELEBORN-2214] Add extraInitContainers for worker in helm by Ping Zhang · 3 months ago
  45. d546398 [CELEBORN-2234] Bump jetty version to 9.4.58.v20250814 to fix GHSA-qh8g-58pp-2wxh by Wang, Fei · 3 months ago
  46. f35b6b8 [CELEBORN-2206][CIP-14] Support PushData and Revive in Cpp's ShuffleClient by HolyLow · 3 months ago
  47. ef2586a [MINOR] add makefile to do docs gen by Ping Zhang · 3 months ago
  48. e5cef16 [CELEBORN-2209] Introduce ReadBufferUsageRatio metric to monitor credit stream read buffer usage by SteNicholas · 3 months ago
  49. 36cdc29 [CELEBORN-2206][CIP-14] Support PushDataCallback in CppClient by HolyLow · 3 months ago
  50. 8966c9b [CELEBORN-2208] Log the partition reader wait time if exceeds the threshold by Wang, Fei · 4 months ago
  51. 5d6ce21 [CELEBORN-1983][FOLLOWUP] Fix fetch fail not throw due to reach spark maxTaskFailures by Xianming Lei · 4 months ago
  52. cc0d1ba [CELEBORN-2152] Support merge buffers on the worker side to improve memory utilization by TheodoreLx · 4 months ago
  53. 3d61ced [CELEBORN-2207] Master StatefulSet should add ratis-log4j.properties for ratis shell by SteNicholas · 4 months ago
  54. ffff5bb [CELEBORN-2196][CIP-14] Support ReviveManager in CppClient by HolyLow · 4 months ago
  55. b678991 [CELEBORN-2203] Set celeborn.master.internal.endpoints in the configmap by Ping Zhang · 4 months ago
  56. 2874227 [MINOR] Add calculation steps for tuning section by Ping Zhang · 4 months ago
  57. 79f0d31 [CELEBORN-2200] Throw IOException when compressed data header corrupted by jiang13021 · 4 months ago
  58. 835ee97 [CELEBORN-2202] Add configmap checksum to the statefulset annotation by pingzh · 4 months ago
  59. 5e4d80b [CELEBORN-2189] Allow config worker pod terminationGracePeriodSeconds in chart by pingzh · 4 months ago
  60. d12aafd [CELEBORN-2105] RpcMetricsTracker should clean up metrics for stopping Inbox by dz · 4 months ago
  61. 00bcf8b [CELEBORN-2198] Fix NPE in tryWithTimeoutAndCallback test due to lazy deviceCheckThreadPool not initialized by zhouhai · 4 months ago
  62. 7efff4b [CELEBORN-1258][FOLLOWUP] Introduce --show-cluster-apps-info master command to show cluster application's info by SteNicholas · 4 months ago
  63. da323ef [CELEBORN-2047][FOLLOWUP] MapPartitionData should close dataChannel, indexChannel, dataInputStream and indexInputStream by SteNicholas · 4 months ago
  64. 7619945 [CELEBORN-2192] ReadBufferDispatcher should add timeout constraints to fast fail in case of timeout by SteNicholas · 4 months ago
  65. 00dc8cb [CELEBORN-2193] Bump Flink from 2.0.0, 2.1.0 to 2.0.1, 2.1.1 by SteNicholas · 4 months ago
  66. e81aa57 [CELEBORN-2195] Align log4j2.xml and metrics.properties of charts with templates by SteNicholas · 4 months ago
  67. 7244f18 [CELEBORN-2194] Change default value of celeborn.worker.directMemoryRatioForReadBuffer by SteNicholas · 4 months ago
  68. 04231e0 [CELEBORN-2188] Abort multipart upload for S3 and OSS in DfsTierWriter#handleException by xxx · 4 months ago
  69. 8ccd4da [CELEBORN-2182][CIP-14] Support PushState and PushStrategy in CppClient by HolyLow · 4 months ago
  70. 005d76d [CELEBORN-2186] Add correctness related section to bug report and PR templates by James Xu · 4 months ago
  71. 67f7c4b [CELEBORN-2185] Support native kqueue transport on BSD/MacOS by SteNicholas · 4 months ago
  72. 1157d6a [CELEBORN-2166] Fast fail reduce stage if shuffle data is lost because of worker lost by Sanskar Modi · 4 months ago
  73. e168fd5 [CELEBORN-2111] Introduce metrics to flush time for different storage types by xxx · 4 months ago
  74. d3c5766 [CELEBORN-2184] Introduce AvailableReadBuffer metric to monitor available memory for credit stream read buffer by SteNicholas · 4 months ago
  75. a518f22 [CELEBORN-2178] Close hadoopFs FileSystem for stopping master by xxx · 4 months ago
  76. 62db42e [CELEBORN-2032] Create reader should change to peer by taskAttemptId by Xianming Lei · 5 months ago
  77. 5867901 [CELEBORN-2181] Modify the shuffleAllocations order in the disk info log by sychen · 5 months ago
  78. 07a3df9 [CELEBORN-2180] Fix Invalid RequestId during RegisterApplicationInfo by JuniverseCoder · 5 months ago
  79. 770c074 [CELEBORN-2169][CIP-14] Support ConcurrentHashMap and refactor reducerFileGroupInfos by HolyLow · 5 months ago
  80. 29070da [CELEBORN-2176] Fix uncaught exception in DataPusher by lijianfu03 · 5 months ago
  81. fb87030 [CELEBORN-2163] PushDataHandler should increment WriteDataFailCount for file writer exception of MapPartition PushData by SteNicholas · 5 months ago
  82. 9482808 [CELEBORN-2159] Fix dfs storage type check in StorageManager#cleanupExpiredShuffleKey by xxx · 5 months ago
  83. 7c4f7b1 [CELEBORN-2170] Refactor ByteBuffer's readToReadOnlyuffer interface by HolyLow · 5 months ago
  84. 79c6bf6 [CELEBORN-2173] jersey-test-framework-core dependency should exclude junit5 dependencies to execute java test cases for CI by Nicholas Jiang · 5 months ago
  85. c1f73a3 [CELEBORN-2171] Fix array index error in submitRetryPushMergedData by xxx · 5 months ago
  86. a7735d9 [CELEBORN-2157][CIP-14] Support sending RegisterShuffle/Revive/MapperEnd messages for NettyRpcEndpointRef in cppClient by HolyLow · 5 months ago
  87. 2fbac7b [CELEBORN-2164] Fix incorrect filtering conditions in updateDiskInfos by xxx · 5 months ago
  88. 534a84b [CELEBORN-2168] Bump Flink from 1.20.2 to 1.20.3 by SteNicholas · 5 months ago
  89. 7a29201 [CELEBORN-2167] Bump Spark from 3.5.6 to 3.5.7 by SteNicholas · 5 months ago
  90. ba6450c [CELEBORN-2165] Fix endless swagger openapi.json security items by Wang, Fei · 5 months ago
  91. a1caa61 [CELEBORN-2162] Supress client side connection error logging by Aravind Patnam · 5 months ago
  92. 11d1c08 [CELEBORN-2160] Speedup CommitHandler.finishMapperAttempt by Mridul Muralidharan · 5 months ago
  93. 944efa8 [CELEBORN-2142] DfsTierWriter should create for unavailable disks by xxx · 5 months ago
  94. 1bf9d30 [CELEBORN-2161] Support DB delete failure policy by sychen · 5 months ago
  95. 61211ed [CELEBORN-2058] Add retry to avoid committing failed with HDFS storage by taowenjun · 5 months ago
  96. 62776b3 [CELEBORN-2131] Add sorting duration logs in FileSorter by sychen · 5 months ago
  97. f20e9fa [CELEBORN-2130] Add support for redirection application registration to the leader by Aravind Patnam · 6 months ago
  98. 834fab2 [MINOR] Fix config doc about DB description by sychen · 6 months ago
  99. 7fc4f24 [CELEBORN-2146] Setting the DFS replication factor for balanced fault tolerance and storage efficiency by xxx · 6 months ago
  100. f0ce486 [CELEBORN-2150] Fix the match condition in checkIfWorkingDirCleaned by xxx · 6 months ago