Google Git
Sign in
apache / arrow-experimental-rs-arrow2 / a93ebbe8acfa5623385fe236effdf1496d7ba982
commita93ebbe8acfa5623385fe236effdf1496d7ba982[log] [tgz]
authorHeres, Daniel <danielheres@gmail.com>Thu Apr 15 07:03:49 2021 -0400
committerAndrew Lamb <andrew@nerdnetworks.org>Thu Apr 15 07:03:49 2021 -0400
tree5f43f176db0e54d7d4d48a08bfb93344d39b3d85
parent9e83b26ba18ab93f0afab492b199b323db0a9192 [diff]
ARROW-12390: [Rust] Inline from_trusted_len_iter, try_from_trusted_len_iter, extend_from_slice

This helps with further optimizing performance on quite some kernels:

```
length                  time:   [721.26 ns 725.59 ns 731.42 ns]
                        change: [-69.510% -69.125% -68.566%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  7 (7.00%) high mild
  4 (4.00%) high severe
take i32 512            time:   [376.15 ns 378.51 ns 381.09 ns]
                        change: [-18.681% -18.212% -17.542%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

take i32 1024           time:   [640.71 ns 641.87 ns 643.19 ns]
                        change: [-28.880% -28.695% -28.516%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

take i32 nulls 512      time:   [634.80 ns 635.63 ns 636.56 ns]
                        change: [-24.797% -23.403% -22.272%] (p = 0.00 < 0.05)
                        Performance has improved.

take i32 nulls 1024     time:   [1.0084 us 1.0093 us 1.0103 us]
                        change: [-44.482% -42.312% -40.209%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) low mild
  5 (5.00%) high mild
  2 (2.00%) high severe

take bool 512           time:   [1.4068 us 1.4188 us 1.4315 us]
                        change: [-49.139% -48.684% -48.208%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

take bool 1024          time:   [2.6662 us 2.6807 us 2.6973 us]
                        change: [-55.452% -55.133% -54.803%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

take bool nulls 512     time:   [1.3331 us 1.3452 us 1.3617 us]
                        change: [-31.198% -30.327% -29.414%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

take bool nulls 1024    time:   [2.4789 us 2.4968 us 2.5229 us]
                        change: [-40.358% -39.969% -39.558%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

```

FYI @jorgecarleitao

Closes #10039 from Dandandan/inline_from_trusted_len_iter

Authored-by: Heres, Daniel <danielheres@gmail.com>
Signed-off-by: Andrew Lamb <andrew@nerdnetworks.org>
2 files changed
tree: 5f43f176db0e54d7d4d48a08bfb93344d39b3d85
  1. .github/
  2. ci/
  3. dev/
  4. format/
  5. rust/
  6. testing
  7. .asf.yaml
  8. .clang-format
  9. .clang-tidy
  10. .clang-tidy-ignore
  11. .dir-locals.el
  12. .dockerignore
  13. .env
  14. .gitattributes
  15. .gitignore
  16. .gitmodules
  17. .hadolint.yaml
  18. .pre-commit-config.yaml
  19. .readthedocs.yml
  20. .travis.yml
  21. appveyor.yml
  22. CHANGELOG.md
  23. cmake-format.py
  24. CODE_OF_CONDUCT.md
  25. CONTRIBUTING.md
  26. docker-compose.yml
  27. header
  28. LICENSE.txt
  29. NOTICE.txt
  30. README.md
  31. run-cmake-format.py
README.md

Apache Arrow

Build Status Coverage Status Fuzzing Status License Twitter Follow

Powering In-Memory Analytics

Apache Arrow is a development platform for in-memory analytics. It contains a set of technologies that enable big data systems to process and move data fast.

Major components of the project include:

Arrow is an Apache Software Foundation project. Learn more at arrow.apache.org.

What's in the Arrow libraries?

The reference Arrow libraries contain many distinct software components:

  • Columnar vector and table-like containers (similar to data frames) supporting flat or nested types
  • Fast, language agnostic metadata messaging layer (using Google's Flatbuffers library)
  • Reference-counted off-heap buffer memory management, for zero-copy memory sharing and handling memory-mapped files
  • IO interfaces to local and remote filesystems
  • Self-describing binary wire formats (streaming and batch/file-like) for remote procedure calls (RPC) and interprocess communication (IPC)
  • Integration tests for verifying binary compatibility between the implementations (e.g. sending data from Java to C++)
  • Conversions to and from other in-memory data structures
  • Readers and writers for various widely-used file formats (such as Parquet, CSV)

Implementation status

The official Arrow libraries in this repository are in different stages of implementing the Arrow format and related features. See our current feature matrix on git master.

How to Contribute

Please read our latest project contribution guide.

Getting involved

Even if you do not plan to contribute to Apache Arrow itself or Arrow integrations in other projects, we'd be happy to have you involved: