Apache Spark-compatible expressions for Apache DataFusion

Clone this repo:

Branches

  1. 664a7be ci: add cargo fmt and clippy workflow (#53) by Andy Grove · 10 hours ago main
  2. 9a219ee test: run sqllogictest files in CI (#51) by Andy Grove · 10 hours ago
  3. b947917 ci: add Apache RAT license header check (#50) by Andy Grove · 10 hours ago
  4. 4061d37 docs: add audit-datafusion-spark-expression agent skill and AGENTS.md (#52) by Andy Grove · 11 hours ago
  5. 420c280 docs: add README, LICENSE.txt and NOTICE.txt (#49) by Andy Grove · 11 hours ago

DataFusion Spark

Apache DataFusion expressions that emulate Apache Spark's behavior.

This crate provides Spark-compatible scalar, aggregate, and table functions that can be registered with a DataFusion SessionContext.

Repository layout

  • crates/spark – the datafusion-spark crate
  • crates/sqllogictest – SQL logic tests for the Spark functions

Building and testing

cargo build
cargo test

License

Licensed under the Apache License, Version 2.0.