Google Git Google Git
Sign in
apache/datafusion-comet/HEAD
  1. c09d8e8 fix: fall back for brotli Iceberg writes when Hadoop's BrotliCodec cannot be loaded (#6996) by Andy Grove · 32 hours ago main
  2. ca1f444 perf: read shuffle blocks directly in native Iceberg writes (#6926) by Andy Grove · 32 hours ago
  3. f068814 fix: report input and SQL metrics while a native shuffle write runs (#7584) by Oleks V · 2 days ago
  4. 260dcad build(deps-dev): bump org.bouncycastle:bcprov-jdk18on in /spark (#7595) by dependabot[bot] · 2 days ago
  5. cffa281 build(deps): bump parquet in /native/native-executors (#7533) by dependabot[bot] · 2 days ago
  6. 1789d06 build(deps): bump the codeql-actions group with 2 updates (#7531) by dependabot[bot] · 2 days ago
  7. e59d937 fix: raise NUMERIC_OUT_OF_SUPPORTED_RANGE for a string with more than 38 integral digits cast to decimal (#7578) by Steve Vaughan · 3 days ago
  8. 2f9552d feat(iceberg): native Iceberg scans of HDFS-backed tables via iceberg-rust's hdfs-native backend (#5898) by Michael Taranov · 3 days ago
  9. 1e552bd chore: remove the native columnar-to-row implementation (#7505) by Andy Grove · 4 days ago
  10. 8c145b7 refactor: move the patch-release version checks into SparkFixes (#7577) by Steve Vaughan · 4 days ago
  11. 567dea8 fix: resolve SHORT_IDS and region zone suffixes in native string-to-timestamp cast (#7215) by NingLin Liu · 4 days ago
  12. 8f4ce5e chore: remove the Spark 3.4 shim source sets (#7560) by Andy Grove · 4 days ago
  13. acddec4 fix: keep URL passwords and SAS queries out of Azure and object store errors, keys and logs (#7127) by dustin · 4 days ago
  14. 8386a34 fix: respect SQL execution config when captured keys are unset (#7348) by Wu Shuwen · 4 days ago
  15. 4de6704 test: cover a native Iceberg read of a data file that omits an optional nested field (#7359) by Yunchi Pang · 6 days ago
  16. 9ee48d2 feat: record the commit time in comet-git-info.properties (#7564) by Steve Vaughan · 6 days ago
  17. f109b77 chore: remove Spark 3.4 Maven profiles and stop running Spark 3.4 CI (#7521) by Andy Grove · 6 days ago
  18. 23049f4 fix: include licenses for bundled lz4_flex and Zstandard (#7382) by Manu Zhang · 6 days ago
  19. 146c9bb chore: remove dead code from the release build (#7524) by Andy Grove · 6 days ago
  20. a872f69 ci: label pull requests from workflow_run instead of pull_request_target (#7482) by Andy Grove · 6 days ago
  21. 891008e fix: report SinglePartition from a native Iceberg scan that replaces one (#7424) by Oleks V · 7 days ago
  22. c7b2cfd ci: run Spark SQL 4.1 and Iceberg 1.11 on every pull request (#7516) by Andy Grove · 7 days ago
  23. 624dff5 build(deps): bump the all-other-cargo-deps group across 1 directory with 2 updates (#7532) by dependabot[bot] · 7 days ago
  24. 3b404c8 perf: evaluate the child of `size` once when it needs no NULL guard (#7473) by Oleks V · 7 days ago
  25. cd19831 fix: declare the input order of a native window over a sorted cache (#7256) by Oleks V · 7 days ago
  26. 7376d85 perf: evaluate CASE, AND, OR and IN over struct fields for the whole batch (#7538) by Oleks V · 7 days ago
  27. eaffbff feat: add cluster settings to the Comet Native Executor binaries (#7420) by Andy Grove · 7 days ago
  28. ea1f093 ci: move the macOS PR build to the macos-15 runner image (#7350) by Andy Grove · 7 days ago
  29. 5dcf745 docs: add a docs-review skill (#7153) by Abhishek Mohan · 8 days ago
  30. 780a860 docs: document `spark.broadcast.compress` for large Comet broadcasts (#7492) by Oleks V · 8 days ago
  31. b82f5ea feat: add support for comet cartesian join (#6938) by Bhargava Vadlamani · 9 days ago
  32. 5252c82 fix: give zstd the real destination capacity when compressing cached batches (#7307) by Andy Grove · 9 days ago
  33. 8fc24a7 perf: merge hll_sketch_agg partial states into one union per group (#7345) by Oleks V · 9 days ago
  34. 4e6b122 fix: preserve float collect_set semantics in distinct aggregate chains (#7220) by LinSimon-901101 · 9 days ago
  35. 638bb12 build(deps): bump arrow in /native/native-executors (#7415) by dependabot[bot] · 9 days ago
  36. 0f7f8e4 feat: adding `elapsed_compute` to the writer (#5143) by Oleks V · 10 days ago
  37. 1ad7938 feat: run min and max over struct inputs natively (#7191) by Parth Chandra · 10 days ago
  38. eab3ba6 build(deps): bump the all-other-cargo-deps group (#7411) by dependabot[bot] · 10 days ago
  39. 7bf572e build(deps): bump the codeql-actions group across 1 directory with 2 updates (#6215) by dependabot[bot] · 10 days ago
  40. 209d8e3 docs: make benchmark descriptions consistent with the 1.2.0 results (#7462) by Andy Grove · 10 days ago
  41. 97ad03b fix: write the Spark version into the footer of natively written Parquet files (#7053) by Andy Grove · 10 days ago
  42. b010bbb docs: update TPC-DS benchmark results for Comet 1.2.0 (#7444) by Andy Grove · 10 days ago
  43. f78c0b1 docs: add a documentation style guide and apply it to the remaining pages (#7151) by Abhishek Mohan · 10 days ago
  44. 44b91c4 chore: update documentation links for 1.2.0 release (#7454) by Andy Grove · 10 days ago
  45. 644a870 deps: bump arrow and parquet to 59.3.2 (#7419) by Anton Borisov · 11 days ago
  46. a4d716b deps: upgrade datasketches to 0.6.0 and drop the HLL decode workarounds (#7402) by Andy Grove · 11 days ago
  47. 3439d48 test: skip two Spark 3.4 tests that cannot pass there (#7423) by Andy Grove · 11 days ago
  48. 79b5ddd feat: add comet-scheduler and comet-executor for Comet Native Executors (#7299) by Andy Grove · 11 days ago
  49. 268ede5 fix: preserve TRY overflow semantics in nested integer arithmetic (#7379) by Chao Sun · 12 days ago
  50. 383ec56 fix: merge all Bloom filter states and respect sliced offsets (#7173) by Chao Sun · 13 days ago
  51. 044b3ea perf: specialize scattered Boolean shuffle gathers (#6430) by Chao Sun · 13 days ago
  52. 5e16b8b fix: pass the session batch size to the native FilterExec (#7280) by Parth Chandra · 13 days ago
  53. 3099445 feat: preserve columnar batches across observations (#7325) by Ping Zhang · 13 days ago
  54. 7deeba2 fix: evaluate the right operand of AND, OR and IN only for rows the left side has not decided (#7326) by Andy Grove · 14 days ago
  55. 1676a16 perf: coalesce native shuffle batches before broadcast serialization (#7258) by Chao Sun · 14 days ago
  56. 052a467 perf: optimize native parse_url extraction (#7329) by Chao Sun · 2 weeks ago
  57. 0f90a7d docs: correct the weekly community call time and video link (#7346) by Andy Grove · 2 weeks ago
  58. 51c64b9 docs: recommend -XX:TrimNativeHeapInterval for executors OOMKilled by retained glibc memory (#7342) by Andy Grove · 2 weeks ago
  59. 802280a ci: run the pull request tier for fork pull requests on the fork (#7311) by Andy Grove · 2 weeks ago
  60. 36b432c test: compare the V2 file insert's closed files as a set (#7335) by Andy Grove · 2 weeks ago
  61. 9acd434 perf: reduce per-batch broadcast serializer allocations (#7310) by Chao Sun · 2 weeks ago
  62. 4c473ea refactor: move literal conversion and query context lookup out of create_expr (#7236) by Andy Grove · 2 weeks ago
  63. 5652f82 build: bump pyo3 to 0.29 and add a cargo audit config (#7327) by Andy Grove · 2 weeks ago
  64. 15305af fix: apply the session's Comet configs in map stages of jobs outside a SQL execution (#7302) by Oleks V · 2 weeks ago
  65. 962db53 feat: bump iceberg-rust past apache/iceberg-rust#3255 and narrow the nested schema evolution fallback (#7028) by Andy Grove · 2 weeks ago
  66. 8aac721 fix: write a sliced batch to the in-memory cache without the rows before it (#7225) by Andy Grove · 2 weeks ago
  67. d8a272d perf: reduce repeated integer keys before left-semi sort-merge joins (#7190) by Chao Sun · 2 weeks ago
  68. aad0fa2 perf: remove per-partition tail copy and flush from the native shuffle writer (#7234) by Andy Grove · 2 weeks ago
  69. ce3e665 fix: declare required child distribution on aggregate, window and join operators (#7231) by Andy Grove · 2 weeks ago
  70. 7f50037 fix: preserve make_date evaluation across lazy operators (#7283) by Ping Zhang · 2 weeks ago
  71. 7f73d86 docs: explain spark.comet.shuffle.native.maxBufferBytes in the shuffle tuning guide (#7308) by Oleks V · 2 weeks ago
  72. d7f4fca fix: preserve columnar output when reverting a cached AQE result stage (#6852) by Andy Grove · 2 weeks ago
  73. 7323814 build: format the generated release docs with the latest prettier (#7295) by Andy Grove · 2 weeks ago
  74. f02e117 docs: add the 1.2.0 changelog and known differences to the upgrade guide (#7294) by Andy Grove · 2 weeks ago
  75. b137ca5 build(deps-dev): bump com.google.guava:guava (#7259) by dependabot[bot] · 2 weeks ago
  76. 621b9cc fix: decode invalid UTF-8 in nested Parquet BINARY values read as strings (#7106) by Manu Zhang · 2 weeks ago
  77. a828088 chore: add branch protection for branch-1.2 (#7232) by Andy Grove · 2 weeks ago
  78. d8ead26 fix: spill a native PartialMerge whose input is sorted on some of its keys (#7222) by Andy Grove · 2 weeks ago
  79. 61fbbe2 fix: fall back to Spark when the driver cannot use the ABFS authentication (#7221) by Andy Grove · 2 weeks ago
  80. 967d09b fix: preserve make_date argument and parent evaluation (#7261) by Ping Zhang · 2 weeks ago
  81. f1121ca feat: support the S3A magic committer in native Parquet writes (#6940) by Andy Grove · 2 weeks ago
  82. 84944e9 fix: copy rows retained by identity broadcasts (#7260) by Ping Zhang · 2 weeks ago
  83. 4e54f1c fix: split a coalesced broadcast before a string or binary vector passes 1 GiB (#7193) by Oleks V · 2 weeks ago
  84. 01fc37a fix: don't treat planner-inserted casts as Parquet schema conversions (#7159) by Steve Vaughan · 2 weeks ago
  85. 92be8e8 fix: align nested collection buffer nullability before spilling (#5903) by Erik Bogado · 2 weeks ago
  86. 283e604 Avoid fetching remaining shuffle blocks when closing early (#7229) by Wu Shuwen · 2 weeks ago
  87. 7d251b3 test: pin localTableScan off in Iceberg write detection helpers (#7141) by Yunchi Pang · 3 weeks ago
  88. 6fe73d9 fix: point the external sort OOM error at Comet's memory settings (#7104) by ump45nose · 3 weeks ago
  89. cb19d43 fix: prefer Unsupported when combining nested cast support (#6501) by liupoyi-1031 · 3 weeks ago
  90. ca9f52a fix: pick the signed zero Spark ranks first in native percentile (#6928) by Feng Zhang · 3 weeks ago
  91. 7b311ca perf: read cache statistics null counts from Comet's Arrow vectors (#6853) by Andy Grove · 3 weeks ago
  92. d1fc83a docs: run Comet SQL in Kubernetes example (#6949) by Isaac Emmanuel · 3 weeks ago
  93. 6451808 perf: build an AQE cache that ends in a Spark operator straight into Arrow (#6991) by Andy Grove · 3 weeks ago
  94. 1dec4c0 fix: match SPARK-58821 and SPARK-60118 in sequence and string to decimal cast (#7171) by Steve Vaughan · 3 weeks ago
  95. c4a0b1f fix: match Spark nullness for missing nested Parquet fields (#7085) by KUAN-HAO HUANG · 3 weeks ago
  96. f1778b9 fix: hash NullType and ANSI interval values natively (#7084) by ump45nose · 3 weeks ago
  97. d924a73 fix: reject TIMESTAMP(NANOS) Parquet columns read as timestamp (#7089) by ump45nose · 3 weeks ago
  98. 79c4172 fix: point the spill limit error at spark.comet.maxTempDirectorySize (#7098) by ump45nose · 3 weeks ago
  99. 72c4847 fix: preserve Spark errors for Parquet timestamp overflow (#5891) by Peter Lee · 3 weeks ago
  100. c39eb71 fix: compare every row when arrays_overlap gets a scalar beside a column (#6743) by Han-Wen Tsao · 3 weeks ago