Google Git
Sign in
apache / datafusion / refs/heads/branch-36
commite53ccd0756644e6522e6f8c41c4497b47e4f4ceb[log] [tgz]
authorAndy Grove <andygrove73@gmail.com>Mon Feb 19 13:50:08 2024 -0700
committerAndy Grove <andygrove73@gmail.com>Mon Feb 19 13:50:47 2024 -0700
tree3ce43053c919e5dacc3787932bda27ede00129a9
parentbf6f83b3d228fb386f9b4b20c254fa58e2412660 [diff]
Remove circular dependency (#9279)
3 files changed
tree: 3ce43053c919e5dacc3787932bda27ede00129a9
  1. .github/
  2. benchmarks/
  3. ci/
  4. conbench/
  5. datafusion/
  6. datafusion-cli/
  7. datafusion-examples/
  8. dev/
  9. docs/
  10. python/
  11. test-utils/
  12. parquet-testing
  13. testing
  14. .asf.yaml
  15. .dockerignore
  16. .editorconfig
  17. .gitattributes
  18. .github_changelog_generator
  19. .gitignore
  20. .gitmodules
  21. .pre-commit-config.yaml
  22. Cargo.toml
  23. CHANGELOG.md
  24. CODE_OF_CONDUCT.md
  25. CONTRIBUTING.md
  26. header
  27. LICENSE.txt
  28. NOTICE.txt
  29. pre-commit.sh
  30. README.md
  31. rustfmt.toml
README.md

DataFusion

Crates.io Apache licensed Build Status Discord chat

Website | Guides | API Docs | Chat

DataFusion is a very fast, extensible query engine for building high-quality data-centric systems in Rust, using the Apache Arrow in-memory format. Python Bindings are also available. DataFusion offers SQL and Dataframe APIs, excellent performance, built-in support for CSV, Parquet, JSON, and Avro, extensive customization, and a great community.

Here are links to some important information

What can you do with this crate?

DataFusion is great for building projects such as domain specific query engines, new database platforms and data pipelines, query languages and more. It lets you start quickly from a fully working engine, and then customize those features specific to your use. Click Here to see a list known users.

Contributing to DataFusion

Please see the contributor guide and communication pages for more information.

Crate features

This crate has several features which can be specified in your Cargo.toml.

Default features:

  • array_expressions: functions for working with arrays such as array_to_string
  • compression: reading files compressed with xz2, bzip2, flate2, and zstd
  • crypto_expressions: cryptographic functions such as md5 and sha256
  • encoding_expressions: encode and decode functions
  • parquet: support for reading the Apache Parquet format
  • regex_expressions: regular expression functions, such as regexp_match
  • unicode_expressions: Include unicode aware functions such as character_length

Optional features:

  • avro: support for reading the Apache Avro format
  • backtrace: include backtrace information in error messages
  • pyarrow: conversions between PyArrow and DataFusion types
  • serde: enable arrow-schema's serde feature

Rust Version Compatibility

Datafusion crate is tested with the minimum required stable Rust version