| ================================================================================================ |
| Arrow Cache vs Default Cache |
| ================================================================================================ |
| |
| ================================================================================================ |
| Cache primitive types |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 21.0.12+8-LTS on Linux 6.17.0-1020-azure |
| AMD EPYC 7763 64-Core Processor |
| Cache 5M rows with primitives: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| --------------------------------------------------------------------------------------------------------------------------- |
| Default cache - write + read 1778 1868 127 2.8 355.6 1.0X |
| Default cache - write + read (uncompressed) 1138 1151 18 4.4 227.6 1.6X |
| Arrow cache - write + read 1328 1339 16 3.8 265.5 1.3X |
| Arrow cache - write + read (zstd level -1) 1802 1875 104 2.8 360.3 1.0X |
| Arrow cache - write + read (zstd level 1) 1804 1831 38 2.8 360.8 1.0X |
| Arrow cache - write + read (zstd level 3) 1875 1918 61 2.7 375.0 0.9X |
| |
| |
| ================================================================================================ |
| Cache then filter |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 21.0.12+8-LTS on Linux 6.17.0-1020-azure |
| AMD EPYC 7763 64-Core Processor |
| Cache 5M rows, then filter: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| ------------------------------------------------------------------------------------------------------------------------ |
| Default cache - filter 1489 1524 50 3.4 297.8 1.0X |
| Default cache - filter (uncompressed) 1280 1321 58 3.9 256.0 1.2X |
| Arrow cache - filter 1387 1395 11 3.6 277.5 1.1X |
| Arrow cache - filter (zstd level -1) 1769 1773 6 2.8 353.8 0.8X |
| Arrow cache - filter (zstd level 1) 1867 1873 8 2.7 373.3 0.8X |
| Arrow cache - filter (zstd level 3) 1900 1906 8 2.6 380.0 0.8X |
| |
| |
| ================================================================================================ |
| Cache columnar input (Parquet) |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 21.0.12+8-LTS on Linux 6.17.0-1020-azure |
| AMD EPYC 7763 64-Core Processor |
| Cache 2M rows from Parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| ----------------------------------------------------------------------------------------------------------------------------- |
| Default cache - columnar input 1470 1532 87 1.4 735.2 1.0X |
| Default cache - columnar input (uncompressed) 1309 1411 145 1.5 654.5 1.1X |
| Arrow cache - columnar input 1313 1345 46 1.5 656.3 1.1X |
| Arrow cache - columnar input (zstd level -1) 1509 1530 28 1.3 754.7 1.0X |
| Arrow cache - columnar input (zstd level 1) 1522 1530 12 1.3 760.8 1.0X |
| Arrow cache - columnar input (zstd level 3) 1568 1589 30 1.3 784.1 0.9X |
| |
| |
| ================================================================================================ |
| Re-cache Arrow cached data (zero-copy test) |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 21.0.12+8-LTS on Linux 6.17.0-1020-azure |
| AMD EPYC 7763 64-Core Processor |
| Re-cache 2M rows (zero-copy): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| -------------------------------------------------------------------------------------------------------------------------------- |
| Default cache - cache a cached DF 382 418 39 5.2 191.1 1.0X |
| Default cache - cache a cached DF (uncompressed) 191 201 14 10.4 95.7 2.0X |
| Arrow cache - cache a cached DF (zero-copy) 123 135 13 16.3 61.3 3.1X |
| Arrow cache - cache a cached DF (zstd level -1) 311 313 2 6.4 155.5 1.2X |
| Arrow cache - cache a cached DF (zstd level 1) 321 326 4 6.2 160.4 1.2X |
| Arrow cache - cache a cached DF (zstd level 3) 336 338 1 6.0 167.8 1.1X |
| |
| |
| ================================================================================================ |
| Cache with column pruning (select 1 of 20 columns) |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 21.0.12+8-LTS on Linux 6.17.0-1020-azure |
| AMD EPYC 7763 64-Core Processor |
| Cache 5M rows, select 1 column: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| ----------------------------------------------------------------------------------------------------------------------------- |
| Default cache - select 1 of 20 columns 9740 9775 49 0.5 1948.1 1.0X |
| Default cache - select 1 of 20 (uncompressed) 4030 4085 78 1.2 805.9 2.4X |
| Arrow cache - select 1 of 20 5177 5227 72 1.0 1035.3 1.9X |
| Arrow cache - select 1 of 20 (zstd level -1) 8122 8158 52 0.6 1624.3 1.2X |
| Arrow cache - select 1 of 20 (zstd level 1) 8132 8152 29 0.6 1626.4 1.2X |
| Arrow cache - select 1 of 20 (zstd level 3) 8531 8575 62 0.6 1706.3 1.1X |
| |
| |
| |
