tree 7d3a8f4e87d3752d47d57fd28c7e65c7dbd52486
parent 6d193278fa6bc44c4a61014ff8a91e8e053107af
author QI Jiale <qijiale76@foxmail.com> 1715126366 +0800
committer GitHub <noreply@github.com> 1715126366 +0800
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsFcBAABCAAQBQJmOsBeCRC1aQ7uu5UhlAAAazQQAKXP7ANJ09XPGK7EV7RcU+II
 LmnRtmgdaAS2/qoAr82fbrPJjiyCJlq5s+Mw1ulr1gpK3fYeqvO+SFwzcGRwYyg6
 5cGMUy8AnGDMHBNkwTOx5VjxdZ9dWpuulsqao4BlFTuYmev9V2zAZkIy1vVssOXw
 W1j/T9N34ihCIwZ9/19Cm2HUMiKD+P5um1XHb90OT2yqtd1whm6LHceuvdDjtvvn
 bVl1Gs9OO5PUsKvIuU7umDqgZ1ThrlNewzG4aUm4ubHxlJwiPXcSQQ33WxcSJL2G
 0358QyzocVpi0L8yeFXxlZ4uIOKxhCFo22UKAmk/snPBKLsW6MJzu+h+3D/u8LUF
 HKWix/CjX27g84eJdLTYV0uzjFD96yAtleX+J7agK+Oou90gKeWhPKYLKw2rYOHB
 bQzpPMlcaXMxg8LokxmwDm1tSay7hoUvY+ki+Gxt9uKZFdJav80Oyr0QWn2J/vFE
 /hGQT7KfJqMEI5UGoVEeOh/kQRjzOh5P/7TuZX76AhFCe0PZk9T7+vLcf2Sn3etI
 a19su2/MghaXU7lZ9drhDnIOLGqF6KOvlW8PlWVk73sNj9MGa78PB18zcD0ov7QP
 26dit+mQfKX+3opxFqtoaKf3Mf9DNSpYN3+hRCEFCYMMGIDFH1/78p0MenVduykJ
 /ai24vKYkWZ9S+gxeeYR
 =HnKz
 -----END PGP SIGNATURE-----
 

[#1543] improvement(spark): Speed up reading when both sort and combine are used. (#1640)

### What changes were proposed in this pull request?

Backport SPARK-46512 to Uniffle.

After the shuffle reader obtains the block, it will first perform a combine operation, and then perform a sort operation. It is known that both combine and sort may generate temporary files, so the performance may be poor when both sort and combine are used. In fact, combine operations can be performed during the sort process, and we can avoid the combine spill file.
See https://issues.apache.org/jira/browse/SPARK-46512 for details.

### Why are the changes needed?

Fix: #1543 

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Existing UT and Integration Test.