| ================================================================================================ |
| Arrow Cache vs Default Cache |
| ================================================================================================ |
| |
| ================================================================================================ |
| Cache primitive types |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure |
| AMD EPYC 9V74 80-Core Processor |
| Cache 5M rows with primitives: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| --------------------------------------------------------------------------------------------------------------------------- |
| Default cache - write + read 1986 2009 33 2.5 397.1 1.0X |
| Default cache - write + read (uncompressed) 1238 1260 30 4.0 247.7 1.6X |
| Arrow cache - write + read 1397 1402 8 3.6 279.3 1.4X |
| Arrow cache - write + read (zstd level -1) 1838 1855 23 2.7 367.7 1.1X |
| Arrow cache - write + read (zstd level 1) 1922 1946 33 2.6 384.5 1.0X |
| Arrow cache - write + read (zstd level 3) 2020 2024 5 2.5 404.0 1.0X |
| |
| |
| ================================================================================================ |
| Cache then filter |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure |
| AMD EPYC 9V74 80-Core Processor |
| Cache 5M rows, then filter: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| ------------------------------------------------------------------------------------------------------------------------ |
| Default cache - filter 1698 1708 14 2.9 339.6 1.0X |
| Default cache - filter (uncompressed) 1429 1446 24 3.5 285.9 1.2X |
| Arrow cache - filter 1548 1555 10 3.2 309.6 1.1X |
| Arrow cache - filter (zstd level -1) 1961 1969 11 2.5 392.3 0.9X |
| Arrow cache - filter (zstd level 1) 1981 2004 32 2.5 396.3 0.9X |
| Arrow cache - filter (zstd level 3) 2063 2067 5 2.4 412.6 0.8X |
| |
| |
| ================================================================================================ |
| Cache columnar input (Parquet) |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure |
| AMD EPYC 9V74 80-Core Processor |
| Cache 2M rows from Parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| ----------------------------------------------------------------------------------------------------------------------------- |
| Default cache - columnar input 1648 1652 6 1.2 824.1 1.0X |
| Default cache - columnar input (uncompressed) 1458 1479 30 1.4 728.9 1.1X |
| Arrow cache - columnar input 1482 1501 26 1.3 741.2 1.1X |
| Arrow cache - columnar input (zstd level -1) 1722 1724 2 1.2 861.0 1.0X |
| Arrow cache - columnar input (zstd level 1) 1768 1772 6 1.1 883.8 0.9X |
| Arrow cache - columnar input (zstd level 3) 1788 1811 33 1.1 893.9 0.9X |
| |
| |
| ================================================================================================ |
| Re-cache Arrow cached data (zero-copy test) |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure |
| AMD EPYC 9V74 80-Core Processor |
| Re-cache 2M rows (zero-copy): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| -------------------------------------------------------------------------------------------------------------------------------- |
| Default cache - cache a cached DF 411 425 11 4.9 205.4 1.0X |
| Default cache - cache a cached DF (uncompressed) 211 228 31 9.5 105.3 2.0X |
| Arrow cache - cache a cached DF (zero-copy) 135 147 16 14.8 67.5 3.0X |
| Arrow cache - cache a cached DF (zstd level -1) 332 339 6 6.0 166.2 1.2X |
| Arrow cache - cache a cached DF (zstd level 1) 346 351 4 5.8 172.8 1.2X |
| Arrow cache - cache a cached DF (zstd level 3) 355 372 24 5.6 177.3 1.2X |
| |
| |
| ================================================================================================ |
| Cache with column pruning (select 1 of 20 columns) |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure |
| AMD EPYC 9V74 80-Core Processor |
| Cache 5M rows, select 1 column: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| ----------------------------------------------------------------------------------------------------------------------------- |
| Default cache - select 1 of 20 columns 11451 11500 69 0.4 2290.2 1.0X |
| Default cache - select 1 of 20 (uncompressed) 4335 4510 248 1.2 867.0 2.6X |
| Arrow cache - select 1 of 20 5185 5241 80 1.0 1036.9 2.2X |
| Arrow cache - select 1 of 20 (zstd level -1) 8336 8383 67 0.6 1667.2 1.4X |
| Arrow cache - select 1 of 20 (zstd level 1) 8353 8376 33 0.6 1670.5 1.4X |
| Arrow cache - select 1 of 20 (zstd level 3) 8590 8671 114 0.6 1718.0 1.3X |
| |
| |
| |
