Google Git
Sign in
apache / uniffle / HEAD
  1. 82ed9f8 [#2716] feat(client): More overlapping decompression stats to log (#2717) by Junfan Zhang · 3 hours ago master
  2. 4290cfe feat(doc): Update spark related performance guide in doc (#2713) by Junfan Zhang · 7 days ago
  3. 3525bab [#2711] fix(spark): Race condition on deferred compressed block initialization (#2712) by Junfan Zhang · 13 days ago
  4. f43f66f [#2707] fix(server): Catch up on any failures in `calcTopNShuffleDataSize` (#2708) by xianjingfeng · 2 weeks ago
  5. 2d5fc0a [#2709] fix(spark): Fix serialization error in Spark History UI (#2710) by Zhen Wang · 2 weeks ago
  6. 1365115 [#2705] fix(spark): Use read-write lock for `MutableShuffleHandleInfo` to avoid global locking (#2706) by Zhen Wang · 3 weeks ago
  7. cf29d36 [#2674] improvement(client): use ack val to check the block send result (#2703) by xianjingfeng · 4 weeks ago
  8. 7867d59 [#2701] fix(server): release the memory of duplicate blocks (#2702) by xianjingfeng · 5 weeks ago
  9. a2c2d05 [#2697] refactor(spark): Involve related writer stats info into ShuffleWriteTaskStats (#2698) by Junfan Zhang · 6 weeks ago
  10. 7420351 [#2686] fix(client): Prefetch should be finished once shuffle result is empty or null (#2696) by Junfan Zhang · 6 weeks ago
  11. c7e23b6 chore: Fix grammar in RssException message (#2695) by Mark Wadham · 7 weeks ago
  12. 741ecba [#2691] feat(client): Introduce the `HARD_SPLIT_FROM_SERVER` response status code (#2694) by Junfan Zhang · 7 weeks ago
  13. 4c2fd4e chore: Update lz4 to address CVE-2025-12183 (#2693) by advancedxy · 8 weeks ago
  14. 61e47b3 [#2672] fix(server): NPE in PartitionedShuffleBlockIdManager (#2690) by KCH · 8 weeks ago
  15. 5bbe25e [#2675] test(spark)(followup): Add tests for Roaring64NavigableMap optimization in checkSentBlockCount (#2692) by zhan7236 · 8 weeks ago
  16. 8cb6625 [#1603] feat(spark): Disable dataPusher initialization for Spark Driver (#2688) by zhan7236 · 8 weeks ago
  17. b53c535 [#2675] improvement(spark): Optimize `checkSentBlockCount` by using Roaring64NavigableMap (#2687) by zhan7236 · 8 weeks ago
  18. fa80c34 [#2684] fix: Infinite memory data reading due to duplicate blockId (#2685) by Junfan Zhang · 8 weeks ago
  19. 43bfd20 [#2679] fix(spark): Potential data mismatch on overlapping decompression (#2680) by Junfan Zhang · 9 weeks ago
  20. b6848f8 [#2682] feat(spark): Make shuffleWriteTaskStats visible about integrity validation for Gluten (#2683) by Junfan Zhang · 10 weeks ago
  21. d6df94c [#2673] feat(spark)(part-2): Merge partition stats for partition split on integrity validation (#2681) by Junfan Zhang · 10 weeks ago
  22. afe1b9a improvement(spark): Move the reassign info logs to DEBUG to cut down on noise (#2677) by Junfan Zhang · 2 months ago
  23. de55bd9 [#2673] feat(spark)(part-1): Add client-side support for storing partition stats on shuffle servers (#2669) by Junfan Zhang · 3 months ago
  24. b40c509 [#2665] feat(spark): Reconstruct the shuffle handle from initial spark handle it haven't bee updated (#2667) by Junfan Zhang · 3 months ago
  25. d6c5988 feat(spark): Show shuffle failures into spark UI (#2668) by Junfan Zhang · 3 months ago
  26. f736c73 [#2648] fix(spark): Incorrect fetched bytes metric when overlapping decompression is enabled (#2650) by Junfan Zhang · 3 months ago
  27. a37936f [#2652] feat(spark): Add compression for task write stats (#2666) by Junfan Zhang · 3 months ago
  28. 1f371e8 feat(spark): Make integrity validation disabled by default (#2664) by Junfan Zhang · 3 months ago
  29. 17d2b25 [#2649] feat(spark): Introduce timeout mechanism when getting the decompressing data (#2651) by Junfan Zhang · 3 months ago
  30. 6e24451 refactor: Enhance spark client logs (#2662) by Junfan Zhang · 3 months ago
  31. d9815c0 [#2652] feat(spark): Add detailed integrity validation failure analysis (#2657) by Junfan Zhang · 3 months ago
  32. bef547d [#2654] fix(spark): NPE on adding data into overlapping decompression worker (#2661) by Junfan Zhang · 3 months ago
  33. 8124152 improvement(spark): Simplify client output logs for writer/reader (#2660) by Junfan Zhang · 3 months ago
  34. 8bfe1d3 chore: fix typo in `applicationpage.js` (#2656) by Ruilei Ma · 3 months ago
  35. 6aef846 [#2652] feat(spark): Introduce partition records number check to ensure data consistency (#2653) by Junfan Zhang · 3 months ago
  36. 5671a05 fix(spark): decompression time is always 0 when overlapping decompression is enabled (#2647) by Junfan Zhang · 3 months ago
  37. 5edf952 [#2517] fix(client): IllegalReferenceCountException about ShuffleBlockInfo (#2638) by Neo Chien · 3 months ago
  38. 42c5d9f [#2644] feat(spark): Involve shuffle failure into the event logs (#2645) by Junfan Zhang · 3 months ago
  39. 11881ab [#2640] feat(spark): Involve background prefetch time in spark UI (#2641) by Junfan Zhang · 4 months ago
  40. 8952913 chore: Add the space for ComposedClientReadHandler log (#2643) by Junfan Zhang · 4 months ago
  41. 1642c4d [#2494] feat(spark): Involve background overlapping decompress time in spark UI (#2639) by Junfan Zhang · 4 months ago
  42. 4805d13 [#2636] feat(spark): Cache shuffle handle info for reader to reduce RPC cost when partition reassign is enabled (#2637) by Junfan Zhang · 4 months ago
  43. 8729261 improvement(spark): Always reset decompression buffer with explicit position and limit (#2634) by Junfan Zhang · 4 months ago
  44. 1d162dc [#2631] fix(server): Potential data loss due to the shuffle result report retry (#2632) by yl09099 · 4 months ago
  45. 770eab1 [#2494] feat(spark): Add more statistics about overlapping decompression (#2633) by Junfan Zhang · 4 months ago
  46. 1bd7468 [#2592] fix(spark): Skip failure when reporting shuffle write metrics to driver (#2629) by Junfan Zhang · 4 months ago
  47. 96e96f8 [#2626] feat(spark): Respect rss.client.rpc.maxAttempts in ShuffleManagerClient (#2627) by Junfan Zhang · 4 months ago
  48. 3ccd91f [#2614] improvement(client): Add test case for Incorrect header length for getLocalShuffleDataV3 (#2617) by Neo Chien · 4 months ago
  49. 6ad3aa0 [#2618] fix(spark): Invalid reassign status show in spark UI tab (#2620) by Junfan Zhang · 4 months ago
  50. abca581 [#2622] fix(spark): Make shuffleServerInfo comparable on updatePartitionSplitAssignment (#2623) by Junfan Zhang · 4 months ago
  51. 9338529 [#2619] fix(spark): NPE in ShuffleReadTimes.merge (#2621) by Junfan Zhang · 4 months ago
  52. ad66fe9 Revert "Remove protected branch (#2615)" (#2624) by Junfan Zhang · 4 months ago
  53. 67bd7af fix: Remove incubator to correct uniffle svn url (#2625) by Junfan Zhang · 4 months ago
  54. e0a49b9 Just a minor by Junfan Zhang · 4 months ago
  55. 10aa39d Remove protected branch (#2615) by roryqi · 4 months ago
  56. 1a46e2d [#2599] fix(spark): Fix bug the incorrect shuffle read metric for spark (#2600) by Neo Chien · 4 months ago
  57. 04964f3 [#2606] feat(mr): Add safety switch for map-stage combiner (#2607) by l.zonghai · 4 months ago
  58. 7015613 [#2609] feat(spark): Expose `checkDataIfAnyFailure` method so that Gluten can invoke it to trigger reassign ASSP (#2610) by Junfan Zhang · 4 months ago
  59. 9fdde02 [#2591] fix(client): Missing task_id propagation in getLocalShuffleDataV3 (#2605) by Junfan Zhang · 5 months ago
  60. 14a5098 [#2591] fix(client): Incorrect header length for getLocalShuffleDataV3 (#2604) by Junfan Zhang · 5 months ago
  61. 96bf76c [#2591] feat(client): Introduce the mechanism to report localfile read plan (#2603) by Junfan Zhang · 5 months ago
  62. 1e48bc6 [#2601] feat(spark): Introduce overlapping decompression for shuffle read (#2602) by Junfan Zhang · 5 months ago
  63. 2a32171 [#2569] feat(spark): Add statistic of shuffle read times (#2598) by Junfan Zhang · 5 months ago
  64. d5e689c [#2592] fix(spark): Ignore failure when reporting shuffle read metrics to driver (#2593) by Junfan Zhang · 5 months ago
  65. 32f4ac6 [#2575] fix(spark): Fix java.lang.IndexOutOfBoundsException: len is negative (#2589) by Neo Chien · 5 months ago
  66. f3bc84f [#2494] feat(spark): Enable overlapping compression by default (#2588) by Junfan Zhang · 5 months ago
  67. fe0ff7e [#2586] fix(spark): Support writer switching servers on partition split with LOAD_BALANCE mode without reassign (#2587) by Junfan Zhang · 5 months ago
  68. e787d87 [#2583] fix(spark): Enable taskIds filter only on AQE and multi replicas for reader (#2584) by Junfan Zhang · 5 months ago
  69. 0facb7b [#2568] feat(spark): Use space-efficient protobuf for `MutableShuffleHandleInfo` to reduce RPC memory overhead (#2578) by Junfan Zhang · 5 months ago
  70. 9b611cf [#2527] docs: Add some docs for LAB (#2585) by xianjingfeng · 5 months ago
  71. 7f1586e [#2579] fix(spark): Correct partition length for overlapping compression (#2580) by Junfan Zhang · 5 months ago
  72. e6f0941 [#2581] fix(spark): Use `SparkContext.getActive` instead of `getOrCreate` to align with method semantics (#2582) by Zhen Wang · 6 months ago
  73. a1974f6 [#2576] fix: Warm up java version var to eliminate lock on creating concurrent hashmap (#2577) by Junfan Zhang · 6 months ago
  74. 4eb83ee [#2571] fix(client): Race condition when adding shuffle servers (#2574) by Junfan Zhang · 6 months ago
  75. 7414ed5 [MINOR] chore(CI): bump dorny/paths-filter from v3.0.2 to de90cc6fb38fc0963ad72b210f1f284cd68cea36 (#2570) by xianjingfeng · 6 months ago
  76. 41d0fc5 [#2558] improvement(server): Limit the max flush event count for a single buffer (#2562) by xianjingfeng · 6 months ago
  77. 99d5c3a [#2525][FOLLOWUP] fix(server): remove metric `buffer_block_size` (#2567) by xianjingfeng · 6 months ago
  78. 066e71e [#2563] improvement(spark): Add more logs of shuffle write on reassignment failure (#2564) by Junfan Zhang · 6 months ago
  79. 1713c1f [#2549] fix(spark): Invalid remote storage configuration was propagated during application registration (#2550) by Junfan Zhang · 6 months ago
  80. be3a1ff [#2560] improvement(client): Fast fail on hadoop reader initialization failure (#2551) by Junfan Zhang · 6 months ago
  81. 8fa51bb [MINOR] improvement(client): Shorten log for multi replica client reader (#2561) by Junfan Zhang · 6 months ago
  82. a5086b3 [#2555] feat(server): support dynamically modifying the tags of shuffle server (#2557) by xianjingfeng · 6 months ago
  83. 850db71 [MINOR] improvement(client): Simplify logging of heartbeat failures (#2559) by xianjingfeng · 6 months ago
  84. 4b256e9 [#2492][FOLLOWUP] improvement: change the default value of chunkPoolCapacityRatio (#2554) by xianjingfeng · 6 months ago
  85. ccc534b [#2539] fix(spark): NPE in DataPusher when sendShuffleData fails (#2556) by Yunchi Pang · 6 months ago
  86. 03f42c5 [#2525] improvement(server): add some metrics for LAB (#2553) by xianjingfeng · 6 months ago
  87. 4629fc6 [#2544] fix(client): NPE about StatisticsCodec (#2547) by SalvadorRomo · 6 months ago
  88. 10df07d [#2533] fix(client): NPE on rendering spark UI tab (#2552) by Zhen Wang · 6 months ago
  89. f57eabe [#2545] feat: Align with spark executor cores on overlapping compression (#2548) by Junfan Zhang · 6 months ago
  90. f1323ab [#2536] feat(spark): Controllable concurrency for overlapping compression (#2541) by Junfan Zhang · 7 months ago
  91. 262519e [#2540] feat(spark): Show compression ratio into spark UI tab (#2542) by Junfan Zhang · 7 months ago
  92. ece59ee [#2537] feat(spark): Introduce option to activate small cache in grpc server (#2538) by Junfan Zhang · 7 months ago
  93. 77e6ab1 [#2494] feat(spark): Overlapping compression to avoid block shuffle writing (#2511) by Junfan Zhang · 7 months ago
  94. f2d7d86 [#2526] improvement: Reset the chunkID if it overflow (#2531) by OmarBustamante · 7 months ago
  95. ad7babb [#2496] improvement(server): Use tryLock instead of synchronized on buffer flushing (#2524) by xianjingfeng · 7 months ago
  96. f194cc5 [#2529] fix(spark3): Incorrect clientInfo without nettyPort if netty is enabled (#2530) by Junfan Zhang · 7 months ago
  97. 80aa170 [#2494] feat(client): Introduce codec statistics to measure de/compression performance (#2528) by Junfan Zhang · 7 months ago
  98. 887d042 [#2496] improvement(server): Improve the performance of flushing single buffer (#2523) by xianjingfeng · 7 months ago
  99. 7781bf6 [#2520] fix(spark3): Shuffle write total duration is incorrectly accumulated in event log (#2521) by Junfan Zhang · 7 months ago
  100. 69b2d51 [#2503] fix(spark): Skip reassignment number check on partition split (#2504) by Junfan Zhang · 7 months ago