Google Git
Sign in
apache/celeborn/HEAD
  1. 163bcb3 [CELEBORN-2297] Update workflow to manually install Helm and chart-testing by sychen · 13 hours ago main
  2. a30166d [CELEBORN-2305] Bump Ratis version from 3.2.1 to 3.2.2 by SteNicholas · 13 hours ago
  3. c688c76 [CELEBORN-2063][FOLLOWUP] Fix timeout unit for parallel creation of partition writer by SteNicholas · 31 hours ago
  4. c246031 [CELEBORN-2294][FOLLOWUP] Fix flaky test SparkUtilsSuite by Xianming Lei · 2 days ago
  5. 84830d9 [CELEBORN-2304] Fix timeout unit mismatch in disk monitor check by sychen · 2 days ago
  6. 7f1bac3 [CELEBORN-2302] Fix NPE in MemoryManager.close() when readBufferDispatcher is not initialized by sychen · 3 days ago
  7. 234ff2d [CELEBORN-2219][CIP-14] Support PushMergedData in CppClient by afterincomparableyum · 3 days ago
  8. ca8533c [CELEBORN-2300] Change the default value of celeborn.port.maxRetries from 1 to 16 by sychen · 3 days ago
  9. 37aed7e [CELEBORN-2295] CommitHandler should support retry interval by Xianming Lei · 7 days ago
  10. 37a27bc [CELEBORN-2296] Fix race condition in MemoryManager singleton initialization by sychen · 9 days ago
  11. 097f1df [CELEBORN-2294] The shuffle fetch failed report from the zombie stage should be ignored by Xianming Lei · 9 days ago
  12. c3adbca [CELEBORN-2293] Fix ConcurrentModificationException in WorkerStatusTracker.shuttingWorkers by Kartikay Bhutani · 9 days ago
  13. d3b7513 [CELEBORN-2292] Fix ArithmeticException when PUSH_DATA_HAND_SHAKE fails before any data written by luogen.lg · 10 days ago
  14. 42f1a08 [CELEBORN-2284] Fix TLS Memory Leak by Aravind Patnam · 12 days ago
  15. 235f07d [CELEBORN-2277] Replace synchronized in Flusher.getWorkerIndex with AtomicInteger by sychen · 12 days ago
  16. 6fc5565 [CELEBORN-2291] Support fsync on commit to ensure shuffle data durability by Kartikay Bhutani · 2 weeks ago
  17. 3773c65 [CELEBORN-2285] Bump maven 3.9.14 by sychen · 3 weeks ago
  18. 28a0733 [CELEBORN-2276] Fix race condition in MemoryManager.releaseSortMemory by sychen · 3 weeks ago
  19. 5fc0f19 [CELEBORN-2282] Eliminate redundant HashMap lookups in CelebornInputStream#fillBuffer by yew1eb · 3 weeks ago
  20. b4cb5a0 [CELEBORN-2283][BUG] Fix missing return in Master.handleRequestSlots when all workers are excluded by yew1eb · 3 weeks ago
  21. 7b25797 [CELEBORN-2274] Fix replicate channels not resumed when transitioning from PUSH_AND_REPLICATE_PAUSED to PUSH_PAUSED by Shuai Lu · 3 weeks ago
  22. 15cba46 [MINOR] Update repository references from incubator-gluten to gluten for TLP graduation of gluten by SteNicholas · 3 weeks ago
  23. af0ba1a [CELEBORN-2281] Improve error logging and null checks in CreditStreamManager by sychen · 3 weeks ago
  24. 400e951 [CELEBORN-2280] Support celeborn.network.memory.allocator.type to specify netty memory allocator by SteNicholas · 4 weeks ago
  25. 8c2c952 [CELEBORN-2279] Update log level from `INFO` to `ERROR` for console output in spark-it tests by sychen · 4 weeks ago
  26. 4b157c6 [CELEBORN-2278] Make HTTP auth bypass API paths configurable by Aravind Patnam · 5 weeks ago
  27. b78177f [CELEBORN-2238] Fix RuntimeException during stream cleanup preventing peer failover by Shuai Lu · 5 weeks ago
  28. 391ef4b [CELEBORN-2273] Fix cache mutation in TagsManager.getTaggedWorkers() by ShlomiTubul · 5 weeks ago
  29. dca3749 [CELEBORN-2218] Bump lz4-java version from 1.8.0 to 1.10.4 to resolve CVE‐2025‐12183 and CVE-2025-66566 by SteNicholas · 6 weeks ago
  30. 13ea40c [CELEBORN-2272] Add LZ4TPCDSDataBenchmark by Cheng Pan · 6 weeks ago
  31. ffdb416 [CELEBORN-2270] Fix problem with eviction to tiered storage during partition split by Enrico Olivelli · 6 weeks ago
  32. 0a67e2b [CELEBORN-2049] Bump Ratis version from 3.1.3 to 3.2.1 by SteNicholas · 6 weeks ago
  33. fddb817 [CELEBORN-2226][CIP-14] Support RetryFetchChunk functionality for Cel… by afterincomparableyum · 6 weeks ago
  34. deb7538 [CELEBORN-2248] Implement lazy loading for columnar shuffle classes and skew shuffle method using static holder pattern by zhenghuan · 6 weeks ago
  35. bc4dc12 [CELEBORN-2239] Support Spark 4.1 by SteNicholas · 6 weeks ago
  36. e119902 [CELEBORN-2268] Improve test coverage for MEMORY and S3 storage by Enrico Olivelli · 6 weeks ago
  37. 9d48e6c [CELEBORN-2258] Bump Netty version from 4.1.118.Final to 4.2.10.Final by SteNicholas · 6 weeks ago
  38. ad5a881 [CELEBORN-2267][FOLLOWUP] Add Cpp-Write Java-Read integration tests for LZ4 and ZSTD by afterincomparableyum · 6 weeks ago
  39. 2460efd [CELEBORN-2222][CIP-14] Support Retrying when createReader failed for CelebornInputStream in CppClient by afterincomparableyum · 6 weeks ago
  40. 4d97a85 [CELEBORN-2271] StorageManager#saveCommittedFileInfosExecutor should call shutdown before awaitTermination by Prateek Srivastava · 6 weeks ago
  41. feb3ed9 [CELEBORN-2262] Prepare S3 directory only once and cache s3 client for MultiPartUploader by Enrico Olivelli · 6 weeks ago
  42. ac6d1cf [CELEBORN-2269] Update Cpp TransportClient to resolve hostnames via DNS by jay.narale · 7 weeks ago
  43. e4836b7 [CELEBORN-2266] Modernize Protobuf CMake usage and add install rules by jay.narale · 7 weeks ago
  44. 13cd4a9 [CELEBORN-2265] Do not waste resources on hotpath for debug logging in HashBasedShuffleWriter and SortBasedShuffleWriter by Enrico Olivelli · 7 weeks ago
  45. eb7a720 [CELEBORN-2259] The S3MultipartUploadHandler uses fs.s3a.aws.credentials.provider by Dzeri96 · 7 weeks ago
  46. 8e6f4d5 [CELEBORN-2063] Parallelize the create partition writer in handleReserveSlots to speed up the reserveSlots RPC process time by SteNicholas · 8 weeks ago
  47. b1cbdab [CELEBORN-2221][CIP-14] Support writing with compression in C++ client by afterincomparableyum · 8 weeks ago
  48. 4ed9cbd [CELEBORN-2263] Fix IndexOutOfBoundsException while reading from S3 by Enrico Olivelli · 8 weeks ago
  49. 81d89f3 [CELEBORN-2256] Helm chart: add support for setting annotations on the service account (to support eks.amazonaws.com/role-arn) by Enrico Olivelli · 9 weeks ago
  50. b659439e [CELEBORN-2254] Fix support for S3 and add a simple integration test by Enrico Olivelli · 9 weeks ago
  51. 2097ad0 [CELEBORN-2205] Introduce metrics to fetch chunk for memory and local disk by xxx · 9 weeks ago
  52. 4752936 [CELEBORN-1577][FOLLOWUP] Fix master resource consumption metrics by Sanskar Modi · 9 weeks ago
  53. 46a49a8 [CELEBORN-2250] Fix lock contention in ReducePartitionCommitHandler.finishMapperAttempt via fine-grained locks by yew1eb · 2 months ago
  54. f04dddd [CELEBORN-2251] Introducing a shim layer and a common-tiered submodule for Flink clients by luogen.lg · 3 months ago
  55. e81cea0 [CELEBORN-2245] Bump maven 3.9.12 by sychen · 3 months ago
  56. 9083461 [CELEBORN-2249] Bump Spark from 3.5.7 to 3.5.8 by yew1eb · 3 months ago
  57. 4749382 [CELEBORN-2211] Avoid allocating additional buffers When HdfsFlushTask writes data by xxx · 3 months ago
  58. 7e4bea0 [CELEBORN-2243][FOLLOWUP][SPARK-2] During the close phase of hashWriter, pushData and mergeData are sent in parallel by jiucheng · 3 months ago
  59. 42ff69f [CELEBORN-2229][CIP-14] Add support for celeborn.client.push.maxBytesSizeInFlight in CppClient by afterincomparableyum · 3 months ago
  60. 9fca94b [CELEBORN-2243] During the close phase of hashWriter, pushData and mergeData are sent in parallel by jiucheng · 3 months ago
  61. d30c02e [CELEBORN-2235][CIP-14] Adapt Java end's serialization to CppWriterClient by HolyLow · 3 months ago
  62. 2dd1b7a [CELEBORN-2210] When a flushBuffer consolidation OOM exception occurs… by xxx · 3 months ago
  63. 783a7cc [CELEBORN-2236] Avoiding regular expressions for DiskFileInfo storage type determination by xxx · 3 months ago
  64. 38532d7 [CELEBORN-2166][FOLLOWUP] Update config celeborn.client.shuffleDataLostOnUnknownWorker.enabled version to 0.6.3 by Wang, Fei · 3 months ago
  65. 526f7bb [CELEBORN-2240] Adapt to SPARK-51756 which add a new parameter `checksumValue` in `MapStatus.apply` by Cheng Pan · 4 months ago
  66. ac8cffb [CELEBORN-2231] Upgrade jersey version to 2.47 to fix CVE-2025-12383 by Wang, Fei · 4 months ago
  67. 5789b4e [CELEBORN-2217] Use a separate thread to handle RpcEndpointVerifier messages by TheodoreLx · 4 months ago
  68. 0f663d0 [CELEBORN-1577][FOLLOWUP] Fix addGauge error in registerUserResourceCons… by xxx · 4 months ago
  69. 77a8921 [CELEBORN-2212] Optimize the sorting efficiency of memoryWriters when evicting the largest memory file by xxx · 4 months ago
  70. 36d6b0c [CELEBORN-2237] Support Flink 2.2 by SteNicholas · 4 months ago
  71. 2233699 [CELEBORN-2214] Add extraInitContainers for worker in helm by Ping Zhang · 4 months ago
  72. d546398 [CELEBORN-2234] Bump jetty version to 9.4.58.v20250814 to fix GHSA-qh8g-58pp-2wxh by Wang, Fei · 4 months ago
  73. f35b6b8 [CELEBORN-2206][CIP-14] Support PushData and Revive in Cpp's ShuffleClient by HolyLow · 4 months ago
  74. ef2586a [MINOR] add makefile to do docs gen by Ping Zhang · 4 months ago
  75. e5cef16 [CELEBORN-2209] Introduce ReadBufferUsageRatio metric to monitor credit stream read buffer usage by SteNicholas · 4 months ago
  76. 36cdc29 [CELEBORN-2206][CIP-14] Support PushDataCallback in CppClient by HolyLow · 4 months ago
  77. 8966c9b [CELEBORN-2208] Log the partition reader wait time if exceeds the threshold by Wang, Fei · 5 months ago
  78. 5d6ce21 [CELEBORN-1983][FOLLOWUP] Fix fetch fail not throw due to reach spark maxTaskFailures by Xianming Lei · 5 months ago
  79. cc0d1ba [CELEBORN-2152] Support merge buffers on the worker side to improve memory utilization by TheodoreLx · 5 months ago
  80. 3d61ced [CELEBORN-2207] Master StatefulSet should add ratis-log4j.properties for ratis shell by SteNicholas · 5 months ago
  81. ffff5bb [CELEBORN-2196][CIP-14] Support ReviveManager in CppClient by HolyLow · 5 months ago
  82. b678991 [CELEBORN-2203] Set celeborn.master.internal.endpoints in the configmap by Ping Zhang · 5 months ago
  83. 2874227 [MINOR] Add calculation steps for tuning section by Ping Zhang · 5 months ago
  84. 79f0d31 [CELEBORN-2200] Throw IOException when compressed data header corrupted by jiang13021 · 5 months ago
  85. 835ee97 [CELEBORN-2202] Add configmap checksum to the statefulset annotation by pingzh · 5 months ago
  86. 5e4d80b [CELEBORN-2189] Allow config worker pod terminationGracePeriodSeconds in chart by pingzh · 5 months ago
  87. d12aafd [CELEBORN-2105] RpcMetricsTracker should clean up metrics for stopping Inbox by dz · 5 months ago
  88. 00bcf8b [CELEBORN-2198] Fix NPE in tryWithTimeoutAndCallback test due to lazy deviceCheckThreadPool not initialized by zhouhai · 5 months ago
  89. 7efff4b [CELEBORN-1258][FOLLOWUP] Introduce --show-cluster-apps-info master command to show cluster application's info by SteNicholas · 5 months ago
  90. da323ef [CELEBORN-2047][FOLLOWUP] MapPartitionData should close dataChannel, indexChannel, dataInputStream and indexInputStream by SteNicholas · 5 months ago
  91. 7619945 [CELEBORN-2192] ReadBufferDispatcher should add timeout constraints to fast fail in case of timeout by SteNicholas · 5 months ago
  92. 00dc8cb [CELEBORN-2193] Bump Flink from 2.0.0, 2.1.0 to 2.0.1, 2.1.1 by SteNicholas · 5 months ago
  93. e81aa57 [CELEBORN-2195] Align log4j2.xml and metrics.properties of charts with templates by SteNicholas · 5 months ago
  94. 7244f18 [CELEBORN-2194] Change default value of celeborn.worker.directMemoryRatioForReadBuffer by SteNicholas · 5 months ago
  95. 04231e0 [CELEBORN-2188] Abort multipart upload for S3 and OSS in DfsTierWriter#handleException by xxx · 5 months ago
  96. 8ccd4da [CELEBORN-2182][CIP-14] Support PushState and PushStrategy in CppClient by HolyLow · 5 months ago
  97. 005d76d [CELEBORN-2186] Add correctness related section to bug report and PR templates by James Xu · 5 months ago
  98. 67f7c4b [CELEBORN-2185] Support native kqueue transport on BSD/MacOS by SteNicholas · 5 months ago
  99. 1157d6a [CELEBORN-2166] Fast fail reduce stage if shuffle data is lost because of worker lost by Sanskar Modi · 5 months ago
  100. e168fd5 [CELEBORN-2111] Introduce metrics to flush time for different storage types by xxx · 5 months ago