Google Git
Sign in
apache/uniffle/HEAD
  1. 9073f3b [#2786] improvement(client): Avoid repeated cleanup scans in DecompressionWorker (#2787) by Zhen Wang · 11 hours ago master
  2. dfeeb16 [#2769] improvement(ci): clean build outputs between sequential Maven profiles (#2770) by Zhen Wang · 8 days ago
  3. 069ead8 chore: use Java 17 for style checks (#2782) by Zhen Wang · 9 days ago
  4. 3bdffc5 [#2783] fix(spark): correct compression ratio and failure counters in Spark UI (#2784) by Zhen Wang · 9 days ago
  5. b1ac32f [#2780] fix(client): Fix prefetch executor thread leaks (#2781) by Zhen Wang · 3 weeks ago
  6. a080644 fix(spark): Fix shuffle cleanup failure with RssShuffleManager on Spark 4 (#2778) by Zhen Wang · 4 weeks ago
  7. a5372e9 [#2775] fix(ci): download envtest binaries from controller-tools releases (#2776) by YangJie · 5 weeks ago
  8. d0d7749 [#2771] feat(spark): Support Spark 4.2 (#2772) by YangJie · 6 weeks ago
  9. ec77694 Replace fixed sleeps with Awaitility in buffer manager tests (#2773) by Zhen Wang · 7 weeks ago
  10. 20167ff chore(hadoop): switch to hadoop shaded client (#2767) by Zhen Wang · 7 weeks ago
  11. 1bbfd79 fix(spark): remove shuffle write status on unregister (#2766) by Zhen Wang · 8 weeks ago
  12. 392573e [#2758] fix(rpc): followup to fix the ShuffleServerOnRandomPortTest CI failure (#2765) by Junfan Zhang · 8 weeks ago
  13. 755e05f [#2758] fix(rpc): simplify server port fallback to improve success rate (#2759) by Junfan Zhang · 8 weeks ago
  14. d31ff12 [#2763] fix(spark)(gluten): use a 1.0 spill ratio for final clear to prevent potential data loss (#2764) by Zhen Wang · 8 weeks ago
  15. 018ac28 [#2761] fix(spark): remove unnecessary synchronized (#2762) by Zhen Wang · 9 weeks ago
  16. 9df64d2 [#2755] fix(ci): upgrade dorny/paths-filter to match apache allow list (#2756) by Junfan Zhang · 2 months ago
  17. 8cd7747 [#2753] fix(spark): ignore removed failed blocks (#2754) by Junfan Zhang · 2 months ago
  18. 7f08785 [#2750] feat(spark): Support Spark 4.1 (#2751) by YangJie · 4 months ago
  19. c81ef85 [#1805] feat(spark): Support Spark 4 in client-spark/extension (#2749) by YangJie · 4 months ago
  20. c18774d [#1805] feat(spark): Support Spark 4.0.2 (#2748) by YangJie · 4 months ago
  21. 6acfd53 fix(spark): Increment segmentIndex when skipping segment due to crc check failure (#2746) by Zhen Wang · 5 months ago
  22. eb69200 [#2739] feat(server): Trigger flush when there are too many blocks in shuffle buffer (#2744) by xianjingfeng · 5 months ago
  23. ae417e6 fix(spark): potential hang with skipped segments on overlapping decompression (#2745) by Junfan Zhang · 6 months ago
  24. b80940d [#2740] improvement(server) Just check block count while checking commit result (#2742) by xianjingfeng · 6 months ago
  25. b98b488 [#2738] feat(server): add metrics to track shuffle data block count and avg block size (#2741) by xianjingfeng · 6 months ago
  26. 2f0b954 [#2601][FOLLOWUP] fix(spark): Release segmentPermits before buffer getting to avoid deadlock in decompression worker (#2737) by Zhen Wang · 6 months ago
  27. 2963220 fix(spark): Correct shuffle read time metrics in spark UI tab (#2736) by Zhen Wang · 6 months ago
  28. b324cc3 [#2716] feat(spark): Introduce option of max segments decompression to control memory usage (#2735) by Junfan Zhang · 6 months ago
  29. 4637321 [#2730] improvement(common): Use built-in `CompositeFileRegion` and remove unnecessary `deallocate` method (#2731) by Zhen Wang · 6 months ago
  30. 5986591 [#2733] fix(spark): Calculate total value for `ShuffleReadTimesSummary` (#2734) by Zhen Wang · 6 months ago
  31. 9c0c27d [#2725] fix(spark)(partition-split): Add fallback under load-balance mode and fix stale assignment missing callback that caused timeout (#2729) by Junfan Zhang · 7 months ago
  32. 2731cf2 [#2724] refactor(spark): Introduce `ReassignExecutor` to simplify shuffle writer logic (#2727) by Junfan Zhang · 7 months ago
  33. bdea9e6 chore(spark): Remove logs of successful heartbeat (#2728) by Junfan Zhang · 7 months ago
  34. eb53a1a [#2725] feat(spark): Introduce optional fast-switch and ignore retry-count checking for stale assignment (#2726) by Junfan Zhang · 7 months ago
  35. 1f809ed [#2718] feat(spark): Eliminate copy in WriterBuffer when compression off for Gluten (#2720) by Junfan Zhang · 7 months ago
  36. 69b1b45 [#2714] feat(spark): Respect compression type when activating overlapping compression mechanism (#2715) by Junfan Zhang · 7 months ago
  37. 82ed9f8 [#2716] feat(client): More overlapping decompression stats to log (#2717) by Junfan Zhang · 7 months ago
  38. 4290cfe feat(doc): Update spark related performance guide in doc (#2713) by Junfan Zhang · 8 months ago
  39. 3525bab [#2711] fix(spark): Race condition on deferred compressed block initialization (#2712) by Junfan Zhang · 8 months ago
  40. f43f66f [#2707] fix(server): Catch up on any failures in `calcTopNShuffleDataSize` (#2708) by xianjingfeng · 8 months ago
  41. 2d5fc0a [#2709] fix(spark): Fix serialization error in Spark History UI (#2710) by Zhen Wang · 8 months ago
  42. 1365115 [#2705] fix(spark): Use read-write lock for `MutableShuffleHandleInfo` to avoid global locking (#2706) by Zhen Wang · 8 months ago
  43. cf29d36 [#2674] improvement(client): use ack val to check the block send result (#2703) by xianjingfeng · 8 months ago
  44. 7867d59 [#2701] fix(server): release the memory of duplicate blocks (#2702) by xianjingfeng · 9 months ago
  45. a2c2d05 [#2697] refactor(spark): Involve related writer stats info into ShuffleWriteTaskStats (#2698) by Junfan Zhang · 9 months ago
  46. 7420351 [#2686] fix(client): Prefetch should be finished once shuffle result is empty or null (#2696) by Junfan Zhang · 9 months ago
  47. c7e23b6 chore: Fix grammar in RssException message (#2695) by Mark Wadham · 9 months ago
  48. 741ecba [#2691] feat(client): Introduce the `HARD_SPLIT_FROM_SERVER` response status code (#2694) by Junfan Zhang · 9 months ago
  49. 4c2fd4e chore: Update lz4 to address CVE-2025-12183 (#2693) by advancedxy · 9 months ago
  50. 61e47b3 [#2672] fix(server): NPE in PartitionedShuffleBlockIdManager (#2690) by KCH · 9 months ago
  51. 5bbe25e [#2675] test(spark)(followup): Add tests for Roaring64NavigableMap optimization in checkSentBlockCount (#2692) by zhan7236 · 9 months ago
  52. 8cb6625 [#1603] feat(spark): Disable dataPusher initialization for Spark Driver (#2688) by zhan7236 · 9 months ago
  53. b53c535 [#2675] improvement(spark): Optimize `checkSentBlockCount` by using Roaring64NavigableMap (#2687) by zhan7236 · 9 months ago
  54. fa80c34 [#2684] fix: Infinite memory data reading due to duplicate blockId (#2685) by Junfan Zhang · 9 months ago
  55. 43bfd20 [#2679] fix(spark): Potential data mismatch on overlapping decompression (#2680) by Junfan Zhang · 10 months ago
  56. b6848f8 [#2682] feat(spark): Make shuffleWriteTaskStats visible about integrity validation for Gluten (#2683) by Junfan Zhang · 10 months ago
  57. d6df94c [#2673] feat(spark)(part-2): Merge partition stats for partition split on integrity validation (#2681) by Junfan Zhang · 10 months ago
  58. afe1b9a improvement(spark): Move the reassign info logs to DEBUG to cut down on noise (#2677) by Junfan Zhang · 10 months ago
  59. de55bd9 [#2673] feat(spark)(part-1): Add client-side support for storing partition stats on shuffle servers (#2669) by Junfan Zhang · 10 months ago
  60. b40c509 [#2665] feat(spark): Reconstruct the shuffle handle from initial spark handle it haven't bee updated (#2667) by Junfan Zhang · 10 months ago
  61. d6c5988 feat(spark): Show shuffle failures into spark UI (#2668) by Junfan Zhang · 10 months ago
  62. f736c73 [#2648] fix(spark): Incorrect fetched bytes metric when overlapping decompression is enabled (#2650) by Junfan Zhang · 10 months ago
  63. a37936f [#2652] feat(spark): Add compression for task write stats (#2666) by Junfan Zhang · 10 months ago
  64. 1f371e8 feat(spark): Make integrity validation disabled by default (#2664) by Junfan Zhang · 10 months ago
  65. 17d2b25 [#2649] feat(spark): Introduce timeout mechanism when getting the decompressing data (#2651) by Junfan Zhang · 10 months ago
  66. 6e24451 refactor: Enhance spark client logs (#2662) by Junfan Zhang · 10 months ago
  67. d9815c0 [#2652] feat(spark): Add detailed integrity validation failure analysis (#2657) by Junfan Zhang · 10 months ago
  68. bef547d [#2654] fix(spark): NPE on adding data into overlapping decompression worker (#2661) by Junfan Zhang · 10 months ago
  69. 8124152 improvement(spark): Simplify client output logs for writer/reader (#2660) by Junfan Zhang · 10 months ago
  70. 8bfe1d3 chore: fix typo in `applicationpage.js` (#2656) by Ruilei Ma · 11 months ago
  71. 6aef846 [#2652] feat(spark): Introduce partition records number check to ensure data consistency (#2653) by Junfan Zhang · 11 months ago
  72. 5671a05 fix(spark): decompression time is always 0 when overlapping decompression is enabled (#2647) by Junfan Zhang · 11 months ago
  73. 5edf952 [#2517] fix(client): IllegalReferenceCountException about ShuffleBlockInfo (#2638) by Neo Chien · 11 months ago
  74. 42c5d9f [#2644] feat(spark): Involve shuffle failure into the event logs (#2645) by Junfan Zhang · 11 months ago
  75. 11881ab [#2640] feat(spark): Involve background prefetch time in spark UI (#2641) by Junfan Zhang · 11 months ago
  76. 8952913 chore: Add the space for ComposedClientReadHandler log (#2643) by Junfan Zhang · 11 months ago
  77. 1642c4d [#2494] feat(spark): Involve background overlapping decompress time in spark UI (#2639) by Junfan Zhang · 11 months ago
  78. 4805d13 [#2636] feat(spark): Cache shuffle handle info for reader to reduce RPC cost when partition reassign is enabled (#2637) by Junfan Zhang · 11 months ago
  79. 8729261 improvement(spark): Always reset decompression buffer with explicit position and limit (#2634) by Junfan Zhang · 11 months ago
  80. 1d162dc [#2631] fix(server): Potential data loss due to the shuffle result report retry (#2632) by yl09099 · 12 months ago
  81. 770eab1 [#2494] feat(spark): Add more statistics about overlapping decompression (#2633) by Junfan Zhang · 12 months ago
  82. 1bd7468 [#2592] fix(spark): Skip failure when reporting shuffle write metrics to driver (#2629) by Junfan Zhang · 12 months ago
  83. 96e96f8 [#2626] feat(spark): Respect rss.client.rpc.maxAttempts in ShuffleManagerClient (#2627) by Junfan Zhang · 12 months ago
  84. 3ccd91f [#2614] improvement(client): Add test case for Incorrect header length for getLocalShuffleDataV3 (#2617) by Neo Chien · 12 months ago
  85. 6ad3aa0 [#2618] fix(spark): Invalid reassign status show in spark UI tab (#2620) by Junfan Zhang · 12 months ago
  86. abca581 [#2622] fix(spark): Make shuffleServerInfo comparable on updatePartitionSplitAssignment (#2623) by Junfan Zhang · 12 months ago
  87. 9338529 [#2619] fix(spark): NPE in ShuffleReadTimes.merge (#2621) by Junfan Zhang · 12 months ago
  88. ad66fe9 Revert "Remove protected branch (#2615)" (#2624) by Junfan Zhang · 12 months ago
  89. 67bd7af fix: Remove incubator to correct uniffle svn url (#2625) by Junfan Zhang · 12 months ago
  90. e0a49b9 Just a minor by Junfan Zhang · 12 months ago
  91. 10aa39d Remove protected branch (#2615) by roryqi · 12 months ago
  92. 1a46e2d [#2599] fix(spark): Fix bug the incorrect shuffle read metric for spark (#2600) by Neo Chien · 12 months ago
  93. 04964f3 [#2606] feat(mr): Add safety switch for map-stage combiner (#2607) by l.zonghai · 12 months ago
  94. 7015613 [#2609] feat(spark): Expose `checkDataIfAnyFailure` method so that Gluten can invoke it to trigger reassign ASSP (#2610) by Junfan Zhang · 12 months ago
  95. 9fdde02 [#2591] fix(client): Missing task_id propagation in getLocalShuffleDataV3 (#2605) by Junfan Zhang · 12 months ago
  96. 14a5098 [#2591] fix(client): Incorrect header length for getLocalShuffleDataV3 (#2604) by Junfan Zhang · 12 months ago
  97. 96bf76c [#2591] feat(client): Introduce the mechanism to report localfile read plan (#2603) by Junfan Zhang · 1 year ago
  98. 1e48bc6 [#2601] feat(spark): Introduce overlapping decompression for shuffle read (#2602) by Junfan Zhang · 1 year ago
  99. 2a32171 [#2569] feat(spark): Add statistic of shuffle read times (#2598) by Junfan Zhang · 1 year ago
  100. d5e689c [#2592] fix(spark): Ignore failure when reporting shuffle read metrics to driver (#2593) by Junfan Zhang · 1 year ago