Google Git
Sign in
apache / celeborn / HEAD
  1. 81d89f3 [CELEBORN-2256] Helm chart: add support for setting annotations on the service account (to support eks.amazonaws.com/role-arn) by Enrico Olivelli · 23 hours ago main
  2. b659439e [CELEBORN-2254] Fix support for S3 and add a simple integration test by Enrico Olivelli · 24 hours ago
  3. 2097ad0 [CELEBORN-2205] Introduce metrics to fetch chunk for memory and local disk by xxx · 2 days ago
  4. 4752936 [CELEBORN-1577][FOLLOWUP] Fix master resource consumption metrics by Sanskar Modi · 2 days ago
  5. 46a49a8 [CELEBORN-2250] Fix lock contention in ReducePartitionCommitHandler.finishMapperAttempt via fine-grained locks by yew1eb · 10 days ago
  6. f04dddd [CELEBORN-2251] Introducing a shim layer and a common-tiered submodule for Flink clients by luogen.lg · 2 weeks ago
  7. e81cea0 [CELEBORN-2245] Bump maven 3.9.12 by sychen · 2 weeks ago
  8. 9083461 [CELEBORN-2249] Bump Spark from 3.5.7 to 3.5.8 by yew1eb · 2 weeks ago
  9. 4749382 [CELEBORN-2211] Avoid allocating additional buffers When HdfsFlushTask writes data by xxx · 3 weeks ago
  10. 7e4bea0 [CELEBORN-2243][FOLLOWUP][SPARK-2] During the close phase of hashWriter, pushData and mergeData are sent in parallel by jiucheng · 3 weeks ago
  11. 42ff69f [CELEBORN-2229][CIP-14] Add support for celeborn.client.push.maxBytesSizeInFlight in CppClient by afterincomparableyum · 4 weeks ago
  12. 9fca94b [CELEBORN-2243] During the close phase of hashWriter, pushData and mergeData are sent in parallel by jiucheng · 4 weeks ago
  13. d30c02e [CELEBORN-2235][CIP-14] Adapt Java end's serialization to CppWriterClient by HolyLow · 5 weeks ago
  14. 2dd1b7a [CELEBORN-2210] When a flushBuffer consolidation OOM exception occurs… by xxx · 5 weeks ago
  15. 783a7cc [CELEBORN-2236] Avoiding regular expressions for DiskFileInfo storage type determination by xxx · 5 weeks ago
  16. 38532d7 [CELEBORN-2166][FOLLOWUP] Update config celeborn.client.shuffleDataLostOnUnknownWorker.enabled version to 0.6.3 by Wang, Fei · 5 weeks ago
  17. 526f7bb [CELEBORN-2240] Adapt to SPARK-51756 which add a new parameter `checksumValue` in `MapStatus.apply` by Cheng Pan · 7 weeks ago
  18. ac8cffb [CELEBORN-2231] Upgrade jersey version to 2.47 to fix CVE-2025-12383 by Wang, Fei · 7 weeks ago
  19. 5789b4e [CELEBORN-2217] Use a separate thread to handle RpcEndpointVerifier messages by TheodoreLx · 8 weeks ago
  20. 0f663d0 [CELEBORN-1577][FOLLOWUP] Fix addGauge error in registerUserResourceCons… by xxx · 8 weeks ago
  21. 77a8921 [CELEBORN-2212] Optimize the sorting efficiency of memoryWriters when evicting the largest memory file by xxx · 8 weeks ago
  22. 36d6b0c [CELEBORN-2237] Support Flink 2.2 by SteNicholas · 8 weeks ago
  23. 2233699 [CELEBORN-2214] Add extraInitContainers for worker in helm by Ping Zhang · 8 weeks ago
  24. d546398 [CELEBORN-2234] Bump jetty version to 9.4.58.v20250814 to fix GHSA-qh8g-58pp-2wxh by Wang, Fei · 8 weeks ago
  25. f35b6b8 [CELEBORN-2206][CIP-14] Support PushData and Revive in Cpp's ShuffleClient by HolyLow · 9 weeks ago
  26. ef2586a [MINOR] add makefile to do docs gen by Ping Zhang · 9 weeks ago
  27. e5cef16 [CELEBORN-2209] Introduce ReadBufferUsageRatio metric to monitor credit stream read buffer usage by SteNicholas · 10 weeks ago
  28. 36cdc29 [CELEBORN-2206][CIP-14] Support PushDataCallback in CppClient by HolyLow · 10 weeks ago
  29. 8966c9b [CELEBORN-2208] Log the partition reader wait time if exceeds the threshold by Wang, Fei · 3 months ago
  30. 5d6ce21 [CELEBORN-1983][FOLLOWUP] Fix fetch fail not throw due to reach spark maxTaskFailures by Xianming Lei · 3 months ago
  31. cc0d1ba [CELEBORN-2152] Support merge buffers on the worker side to improve memory utilization by TheodoreLx · 3 months ago
  32. 3d61ced [CELEBORN-2207] Master StatefulSet should add ratis-log4j.properties for ratis shell by SteNicholas · 3 months ago
  33. ffff5bb [CELEBORN-2196][CIP-14] Support ReviveManager in CppClient by HolyLow · 3 months ago
  34. b678991 [CELEBORN-2203] Set celeborn.master.internal.endpoints in the configmap by Ping Zhang · 3 months ago
  35. 2874227 [MINOR] Add calculation steps for tuning section by Ping Zhang · 3 months ago
  36. 79f0d31 [CELEBORN-2200] Throw IOException when compressed data header corrupted by jiang13021 · 3 months ago
  37. 835ee97 [CELEBORN-2202] Add configmap checksum to the statefulset annotation by pingzh · 3 months ago
  38. 5e4d80b [CELEBORN-2189] Allow config worker pod terminationGracePeriodSeconds in chart by pingzh · 3 months ago
  39. d12aafd [CELEBORN-2105] RpcMetricsTracker should clean up metrics for stopping Inbox by dz · 3 months ago
  40. 00bcf8b [CELEBORN-2198] Fix NPE in tryWithTimeoutAndCallback test due to lazy deviceCheckThreadPool not initialized by zhouhai · 3 months ago
  41. 7efff4b [CELEBORN-1258][FOLLOWUP] Introduce --show-cluster-apps-info master command to show cluster application's info by SteNicholas · 3 months ago
  42. da323ef [CELEBORN-2047][FOLLOWUP] MapPartitionData should close dataChannel, indexChannel, dataInputStream and indexInputStream by SteNicholas · 3 months ago
  43. 7619945 [CELEBORN-2192] ReadBufferDispatcher should add timeout constraints to fast fail in case of timeout by SteNicholas · 3 months ago
  44. 00dc8cb [CELEBORN-2193] Bump Flink from 2.0.0, 2.1.0 to 2.0.1, 2.1.1 by SteNicholas · 3 months ago
  45. e81aa57 [CELEBORN-2195] Align log4j2.xml and metrics.properties of charts with templates by SteNicholas · 3 months ago
  46. 7244f18 [CELEBORN-2194] Change default value of celeborn.worker.directMemoryRatioForReadBuffer by SteNicholas · 3 months ago
  47. 04231e0 [CELEBORN-2188] Abort multipart upload for S3 and OSS in DfsTierWriter#handleException by xxx · 3 months ago
  48. 8ccd4da [CELEBORN-2182][CIP-14] Support PushState and PushStrategy in CppClient by HolyLow · 3 months ago
  49. 005d76d [CELEBORN-2186] Add correctness related section to bug report and PR templates by James Xu · 3 months ago
  50. 67f7c4b [CELEBORN-2185] Support native kqueue transport on BSD/MacOS by SteNicholas · 3 months ago
  51. 1157d6a [CELEBORN-2166] Fast fail reduce stage if shuffle data is lost because of worker lost by Sanskar Modi · 3 months ago
  52. e168fd5 [CELEBORN-2111] Introduce metrics to flush time for different storage types by xxx · 3 months ago
  53. d3c5766 [CELEBORN-2184] Introduce AvailableReadBuffer metric to monitor available memory for credit stream read buffer by SteNicholas · 3 months ago
  54. a518f22 [CELEBORN-2178] Close hadoopFs FileSystem for stopping master by xxx · 3 months ago
  55. 62db42e [CELEBORN-2032] Create reader should change to peer by taskAttemptId by Xianming Lei · 4 months ago
  56. 5867901 [CELEBORN-2181] Modify the shuffleAllocations order in the disk info log by sychen · 4 months ago
  57. 07a3df9 [CELEBORN-2180] Fix Invalid RequestId during RegisterApplicationInfo by JuniverseCoder · 4 months ago
  58. 770c074 [CELEBORN-2169][CIP-14] Support ConcurrentHashMap and refactor reducerFileGroupInfos by HolyLow · 4 months ago
  59. 29070da [CELEBORN-2176] Fix uncaught exception in DataPusher by lijianfu03 · 4 months ago
  60. fb87030 [CELEBORN-2163] PushDataHandler should increment WriteDataFailCount for file writer exception of MapPartition PushData by SteNicholas · 4 months ago
  61. 9482808 [CELEBORN-2159] Fix dfs storage type check in StorageManager#cleanupExpiredShuffleKey by xxx · 4 months ago
  62. 7c4f7b1 [CELEBORN-2170] Refactor ByteBuffer's readToReadOnlyuffer interface by HolyLow · 4 months ago
  63. 79c6bf6 [CELEBORN-2173] jersey-test-framework-core dependency should exclude junit5 dependencies to execute java test cases for CI by Nicholas Jiang · 4 months ago
  64. c1f73a3 [CELEBORN-2171] Fix array index error in submitRetryPushMergedData by xxx · 4 months ago
  65. a7735d9 [CELEBORN-2157][CIP-14] Support sending RegisterShuffle/Revive/MapperEnd messages for NettyRpcEndpointRef in cppClient by HolyLow · 4 months ago
  66. 2fbac7b [CELEBORN-2164] Fix incorrect filtering conditions in updateDiskInfos by xxx · 4 months ago
  67. 534a84b [CELEBORN-2168] Bump Flink from 1.20.2 to 1.20.3 by SteNicholas · 4 months ago
  68. 7a29201 [CELEBORN-2167] Bump Spark from 3.5.6 to 3.5.7 by SteNicholas · 4 months ago
  69. ba6450c [CELEBORN-2165] Fix endless swagger openapi.json security items by Wang, Fei · 4 months ago
  70. a1caa61 [CELEBORN-2162] Supress client side connection error logging by Aravind Patnam · 4 months ago
  71. 11d1c08 [CELEBORN-2160] Speedup CommitHandler.finishMapperAttempt by Mridul Muralidharan · 4 months ago
  72. 944efa8 [CELEBORN-2142] DfsTierWriter should create for unavailable disks by xxx · 4 months ago
  73. 1bf9d30 [CELEBORN-2161] Support DB delete failure policy by sychen · 4 months ago
  74. 61211ed [CELEBORN-2058] Add retry to avoid committing failed with HDFS storage by taowenjun · 4 months ago
  75. 62776b3 [CELEBORN-2131] Add sorting duration logs in FileSorter by sychen · 4 months ago
  76. f20e9fa [CELEBORN-2130] Add support for redirection application registration to the leader by Aravind Patnam · 5 months ago
  77. 834fab2 [MINOR] Fix config doc about DB description by sychen · 5 months ago
  78. 7fc4f24 [CELEBORN-2146] Setting the DFS replication factor for balanced fault tolerance and storage efficiency by xxx · 5 months ago
  79. f0ce486 [CELEBORN-2150] Fix the match condition in checkIfWorkingDirCleaned by xxx · 5 months ago
  80. 922e65a [CELEBORN-2153] Fix NPE problem that occurs during concurrent merge by TheodoreLx · 5 months ago
  81. 52d3057 [CELEBORN-2154] Optimize the exception handling of DFS read to avoid tasks from getting stuck by xxx · 5 months ago
  82. 578a1c9 [CELEBORN-2155] Avoid using duplicate diskFileInfoMap functions by xxx · 5 months ago
  83. f7962a9 [CELEBORN-2145] QuotaManager should respect celeborn.quota.interruptShuffle.enabled of client config by Nicholas Jiang · 5 months ago
  84. ab0c5af [CELEBORN-2124][CIP-14] Support PushData network interfaces in cppClient by HolyLow · 5 months ago
  85. 9ccae98 [CELEBORN-2149] Remove the deprecated ReleaseSlots by xxx · 5 months ago
  86. e7f7f76 [CELEBORN-2109] Add EvictedLocalFileCount and EvictedDfsFileCount met… by xxx · 5 months ago
  87. 2011c44 [CELEBORN-935][FOLLOWUP] Remove WorkerRemove in MetaHandler by xxx · 5 months ago
  88. 3ebb258 [MINOR] Fix ReducePartitionMetaHandler logger by xxx · 5 months ago
  89. 9f7758f [CELEBORN-2144] Bump spark 4.0.1 by Wang, Fei · 5 months ago
  90. 3efc60c [CELEBORN-2047] Reuse FileChannel/FSDataInputStream in PartitionDataReader by daowu.hzy · 5 months ago
  91. 449cb5d [CELEBORN-2127] When fileWriter is closed, it should return HARD_SPLIT StatusCode by xxx · 5 months ago
  92. 750aeef [CELEBORN-2139] Fix the condition for using OSS storage by xxx · 5 months ago
  93. 6c10244 [CELEBORN-2138] Avoiding multiple accesses to HDFS When writting index file by taowenjun · 5 months ago
  94. 7f08eb8 [CELEBORN-2137] Remove unused MAPGROUP PartitionType by SteNicholas · 5 months ago
  95. d038dd2 [CELEBORN-1258] Support to register application info with user identifier and extra info by Wang, Fei · 5 months ago
  96. 2817f7f [CELEBORN-2104] Clean up sources of NettyRpcEnv, Master and Worker to avoid thread leaks by dz · 5 months ago
  97. ffdaef9 [CELEBORN-2097] Support Zstd Compression in CppClient by Jray · 5 months ago
  98. 1858903 [CELEBORN-2135] Rename Blaze to Auron by sychen · 5 months ago
  99. db70473 [CELEBORN-2134] When creating a DiskFile, retrieve the storage type b… by xxx · 5 months ago
  100. 1be3094 [CELEBORN-2132] Enhance ratis peer add operation to support clientAddress & adminAddress by gaoyajun02 · 5 months ago