Google Git
Sign in
apache/uniffle/HEAD
  1. b98b488 [#2738] feat(server): add metrics to track shuffle data block count and avg block size (#2741) by xianjingfeng · 2 days ago master
  2. 2f0b954 [#2601][FOLLOWUP] fix(spark): Release segmentPermits before buffer getting to avoid deadlock in decompression worker (#2737) by Zhen Wang · 2 weeks ago
  3. 2963220 fix(spark): Correct shuffle read time metrics in spark UI tab (#2736) by Zhen Wang · 2 weeks ago
  4. b324cc3 [#2716] feat(spark): Introduce option of max segments decompression to control memory usage (#2735) by Junfan Zhang · 3 weeks ago
  5. 4637321 [#2730] improvement(common): Use built-in `CompositeFileRegion` and remove unnecessary `deallocate` method (#2731) by Zhen Wang · 3 weeks ago
  6. 5986591 [#2733] fix(spark): Calculate total value for `ShuffleReadTimesSummary` (#2734) by Zhen Wang · 3 weeks ago
  7. 9c0c27d [#2725] fix(spark)(partition-split): Add fallback under load-balance mode and fix stale assignment missing callback that caused timeout (#2729) by Junfan Zhang · 5 weeks ago
  8. 2731cf2 [#2724] refactor(spark): Introduce `ReassignExecutor` to simplify shuffle writer logic (#2727) by Junfan Zhang · 5 weeks ago
  9. bdea9e6 chore(spark): Remove logs of successful heartbeat (#2728) by Junfan Zhang · 5 weeks ago
  10. eb53a1a [#2725] feat(spark): Introduce optional fast-switch and ignore retry-count checking for stale assignment (#2726) by Junfan Zhang · 6 weeks ago
  11. 1f809ed [#2718] feat(spark): Eliminate copy in WriterBuffer when compression off for Gluten (#2720) by Junfan Zhang · 7 weeks ago
  12. 69b1b45 [#2714] feat(spark): Respect compression type when activating overlapping compression mechanism (#2715) by Junfan Zhang · 7 weeks ago
  13. 82ed9f8 [#2716] feat(client): More overlapping decompression stats to log (#2717) by Junfan Zhang · 7 weeks ago
  14. 4290cfe feat(doc): Update spark related performance guide in doc (#2713) by Junfan Zhang · 8 weeks ago
  15. 3525bab [#2711] fix(spark): Race condition on deferred compressed block initialization (#2712) by Junfan Zhang · 9 weeks ago
  16. f43f66f [#2707] fix(server): Catch up on any failures in `calcTopNShuffleDataSize` (#2708) by xianjingfeng · 10 weeks ago
  17. 2d5fc0a [#2709] fix(spark): Fix serialization error in Spark History UI (#2710) by Zhen Wang · 10 weeks ago
  18. 1365115 [#2705] fix(spark): Use read-write lock for `MutableShuffleHandleInfo` to avoid global locking (#2706) by Zhen Wang · 2 months ago
  19. cf29d36 [#2674] improvement(client): use ack val to check the block send result (#2703) by xianjingfeng · 3 months ago
  20. 7867d59 [#2701] fix(server): release the memory of duplicate blocks (#2702) by xianjingfeng · 3 months ago
  21. a2c2d05 [#2697] refactor(spark): Involve related writer stats info into ShuffleWriteTaskStats (#2698) by Junfan Zhang · 3 months ago
  22. 7420351 [#2686] fix(client): Prefetch should be finished once shuffle result is empty or null (#2696) by Junfan Zhang · 3 months ago
  23. c7e23b6 chore: Fix grammar in RssException message (#2695) by Mark Wadham · 3 months ago
  24. 741ecba [#2691] feat(client): Introduce the `HARD_SPLIT_FROM_SERVER` response status code (#2694) by Junfan Zhang · 3 months ago
  25. 4c2fd4e chore: Update lz4 to address CVE-2025-12183 (#2693) by advancedxy · 4 months ago
  26. 61e47b3 [#2672] fix(server): NPE in PartitionedShuffleBlockIdManager (#2690) by KCH · 4 months ago
  27. 5bbe25e [#2675] test(spark)(followup): Add tests for Roaring64NavigableMap optimization in checkSentBlockCount (#2692) by zhan7236 · 4 months ago
  28. 8cb6625 [#1603] feat(spark): Disable dataPusher initialization for Spark Driver (#2688) by zhan7236 · 4 months ago
  29. b53c535 [#2675] improvement(spark): Optimize `checkSentBlockCount` by using Roaring64NavigableMap (#2687) by zhan7236 · 4 months ago
  30. fa80c34 [#2684] fix: Infinite memory data reading due to duplicate blockId (#2685) by Junfan Zhang · 4 months ago
  31. 43bfd20 [#2679] fix(spark): Potential data mismatch on overlapping decompression (#2680) by Junfan Zhang · 4 months ago
  32. b6848f8 [#2682] feat(spark): Make shuffleWriteTaskStats visible about integrity validation for Gluten (#2683) by Junfan Zhang · 4 months ago
  33. d6df94c [#2673] feat(spark)(part-2): Merge partition stats for partition split on integrity validation (#2681) by Junfan Zhang · 4 months ago
  34. afe1b9a improvement(spark): Move the reassign info logs to DEBUG to cut down on noise (#2677) by Junfan Zhang · 4 months ago
  35. de55bd9 [#2673] feat(spark)(part-1): Add client-side support for storing partition stats on shuffle servers (#2669) by Junfan Zhang · 4 months ago
  36. b40c509 [#2665] feat(spark): Reconstruct the shuffle handle from initial spark handle it haven't bee updated (#2667) by Junfan Zhang · 4 months ago
  37. d6c5988 feat(spark): Show shuffle failures into spark UI (#2668) by Junfan Zhang · 4 months ago
  38. f736c73 [#2648] fix(spark): Incorrect fetched bytes metric when overlapping decompression is enabled (#2650) by Junfan Zhang · 4 months ago
  39. a37936f [#2652] feat(spark): Add compression for task write stats (#2666) by Junfan Zhang · 4 months ago
  40. 1f371e8 feat(spark): Make integrity validation disabled by default (#2664) by Junfan Zhang · 5 months ago
  41. 17d2b25 [#2649] feat(spark): Introduce timeout mechanism when getting the decompressing data (#2651) by Junfan Zhang · 5 months ago
  42. 6e24451 refactor: Enhance spark client logs (#2662) by Junfan Zhang · 5 months ago
  43. d9815c0 [#2652] feat(spark): Add detailed integrity validation failure analysis (#2657) by Junfan Zhang · 5 months ago
  44. bef547d [#2654] fix(spark): NPE on adding data into overlapping decompression worker (#2661) by Junfan Zhang · 5 months ago
  45. 8124152 improvement(spark): Simplify client output logs for writer/reader (#2660) by Junfan Zhang · 5 months ago
  46. 8bfe1d3 chore: fix typo in `applicationpage.js` (#2656) by Ruilei Ma · 5 months ago
  47. 6aef846 [#2652] feat(spark): Introduce partition records number check to ensure data consistency (#2653) by Junfan Zhang · 5 months ago
  48. 5671a05 fix(spark): decompression time is always 0 when overlapping decompression is enabled (#2647) by Junfan Zhang · 5 months ago
  49. 5edf952 [#2517] fix(client): IllegalReferenceCountException about ShuffleBlockInfo (#2638) by Neo Chien · 5 months ago
  50. 42c5d9f [#2644] feat(spark): Involve shuffle failure into the event logs (#2645) by Junfan Zhang · 5 months ago
  51. 11881ab [#2640] feat(spark): Involve background prefetch time in spark UI (#2641) by Junfan Zhang · 5 months ago
  52. 8952913 chore: Add the space for ComposedClientReadHandler log (#2643) by Junfan Zhang · 5 months ago
  53. 1642c4d [#2494] feat(spark): Involve background overlapping decompress time in spark UI (#2639) by Junfan Zhang · 5 months ago
  54. 4805d13 [#2636] feat(spark): Cache shuffle handle info for reader to reduce RPC cost when partition reassign is enabled (#2637) by Junfan Zhang · 6 months ago
  55. 8729261 improvement(spark): Always reset decompression buffer with explicit position and limit (#2634) by Junfan Zhang · 6 months ago
  56. 1d162dc [#2631] fix(server): Potential data loss due to the shuffle result report retry (#2632) by yl09099 · 6 months ago
  57. 770eab1 [#2494] feat(spark): Add more statistics about overlapping decompression (#2633) by Junfan Zhang · 6 months ago
  58. 1bd7468 [#2592] fix(spark): Skip failure when reporting shuffle write metrics to driver (#2629) by Junfan Zhang · 6 months ago
  59. 96e96f8 [#2626] feat(spark): Respect rss.client.rpc.maxAttempts in ShuffleManagerClient (#2627) by Junfan Zhang · 6 months ago
  60. 3ccd91f [#2614] improvement(client): Add test case for Incorrect header length for getLocalShuffleDataV3 (#2617) by Neo Chien · 6 months ago
  61. 6ad3aa0 [#2618] fix(spark): Invalid reassign status show in spark UI tab (#2620) by Junfan Zhang · 6 months ago
  62. abca581 [#2622] fix(spark): Make shuffleServerInfo comparable on updatePartitionSplitAssignment (#2623) by Junfan Zhang · 6 months ago
  63. 9338529 [#2619] fix(spark): NPE in ShuffleReadTimes.merge (#2621) by Junfan Zhang · 6 months ago
  64. ad66fe9 Revert "Remove protected branch (#2615)" (#2624) by Junfan Zhang · 6 months ago
  65. 67bd7af fix: Remove incubator to correct uniffle svn url (#2625) by Junfan Zhang · 6 months ago
  66. e0a49b9 Just a minor by Junfan Zhang · 6 months ago
  67. 10aa39d Remove protected branch (#2615) by roryqi · 6 months ago
  68. 1a46e2d [#2599] fix(spark): Fix bug the incorrect shuffle read metric for spark (#2600) by Neo Chien · 6 months ago
  69. 04964f3 [#2606] feat(mr): Add safety switch for map-stage combiner (#2607) by l.zonghai · 6 months ago
  70. 7015613 [#2609] feat(spark): Expose `checkDataIfAnyFailure` method so that Gluten can invoke it to trigger reassign ASSP (#2610) by Junfan Zhang · 6 months ago
  71. 9fdde02 [#2591] fix(client): Missing task_id propagation in getLocalShuffleDataV3 (#2605) by Junfan Zhang · 6 months ago
  72. 14a5098 [#2591] fix(client): Incorrect header length for getLocalShuffleDataV3 (#2604) by Junfan Zhang · 6 months ago
  73. 96bf76c [#2591] feat(client): Introduce the mechanism to report localfile read plan (#2603) by Junfan Zhang · 6 months ago
  74. 1e48bc6 [#2601] feat(spark): Introduce overlapping decompression for shuffle read (#2602) by Junfan Zhang · 7 months ago
  75. 2a32171 [#2569] feat(spark): Add statistic of shuffle read times (#2598) by Junfan Zhang · 7 months ago
  76. d5e689c [#2592] fix(spark): Ignore failure when reporting shuffle read metrics to driver (#2593) by Junfan Zhang · 7 months ago
  77. 32f4ac6 [#2575] fix(spark): Fix java.lang.IndexOutOfBoundsException: len is negative (#2589) by Neo Chien · 7 months ago
  78. f3bc84f [#2494] feat(spark): Enable overlapping compression by default (#2588) by Junfan Zhang · 7 months ago
  79. fe0ff7e [#2586] fix(spark): Support writer switching servers on partition split with LOAD_BALANCE mode without reassign (#2587) by Junfan Zhang · 7 months ago
  80. e787d87 [#2583] fix(spark): Enable taskIds filter only on AQE and multi replicas for reader (#2584) by Junfan Zhang · 7 months ago
  81. 0facb7b [#2568] feat(spark): Use space-efficient protobuf for `MutableShuffleHandleInfo` to reduce RPC memory overhead (#2578) by Junfan Zhang · 7 months ago
  82. 9b611cf [#2527] docs: Add some docs for LAB (#2585) by xianjingfeng · 7 months ago
  83. 7f1586e [#2579] fix(spark): Correct partition length for overlapping compression (#2580) by Junfan Zhang · 7 months ago
  84. e6f0941 [#2581] fix(spark): Use `SparkContext.getActive` instead of `getOrCreate` to align with method semantics (#2582) by Zhen Wang · 7 months ago
  85. a1974f6 [#2576] fix: Warm up java version var to eliminate lock on creating concurrent hashmap (#2577) by Junfan Zhang · 7 months ago
  86. 4eb83ee [#2571] fix(client): Race condition when adding shuffle servers (#2574) by Junfan Zhang · 8 months ago
  87. 7414ed5 [MINOR] chore(CI): bump dorny/paths-filter from v3.0.2 to de90cc6fb38fc0963ad72b210f1f284cd68cea36 (#2570) by xianjingfeng · 8 months ago
  88. 41d0fc5 [#2558] improvement(server): Limit the max flush event count for a single buffer (#2562) by xianjingfeng · 8 months ago
  89. 99d5c3a [#2525][FOLLOWUP] fix(server): remove metric `buffer_block_size` (#2567) by xianjingfeng · 8 months ago
  90. 066e71e [#2563] improvement(spark): Add more logs of shuffle write on reassignment failure (#2564) by Junfan Zhang · 8 months ago
  91. 1713c1f [#2549] fix(spark): Invalid remote storage configuration was propagated during application registration (#2550) by Junfan Zhang · 8 months ago
  92. be3a1ff [#2560] improvement(client): Fast fail on hadoop reader initialization failure (#2551) by Junfan Zhang · 8 months ago
  93. 8fa51bb [MINOR] improvement(client): Shorten log for multi replica client reader (#2561) by Junfan Zhang · 8 months ago
  94. a5086b3 [#2555] feat(server): support dynamically modifying the tags of shuffle server (#2557) by xianjingfeng · 8 months ago
  95. 850db71 [MINOR] improvement(client): Simplify logging of heartbeat failures (#2559) by xianjingfeng · 8 months ago
  96. 4b256e9 [#2492][FOLLOWUP] improvement: change the default value of chunkPoolCapacityRatio (#2554) by xianjingfeng · 8 months ago
  97. ccc534b [#2539] fix(spark): NPE in DataPusher when sendShuffleData fails (#2556) by Yunchi Pang · 8 months ago
  98. 03f42c5 [#2525] improvement(server): add some metrics for LAB (#2553) by xianjingfeng · 8 months ago
  99. 4629fc6 [#2544] fix(client): NPE about StatisticsCodec (#2547) by SalvadorRomo · 8 months ago
  100. 10df07d [#2533] fix(client): NPE on rendering spark UI tab (#2552) by Zhen Wang · 8 months ago