Breaking changes:
Implemented enhancements:
ArrowNativeTypeOp
#2753 [arrow]Fixed bugs:
parquet::arrow::arrow_writer::ArrowWriter
ignores page size properties #2853 [parquet]the trait bound for<'de> arrow::datatypes::Schema: serde::de::Deserialize<'de> is not satisfied
#2825 [arrow]Closed issues:
Merged pull requests:
snap
option to README #2847 [parquet] (exyi)interleave
kernel (#1523) #2838 [arrow] (tustvold)ArrowNativeTypeOp
#2756 [arrow] (HaoYang670)Breaking changes:
ArrowNativeType
(#1918) #2793 [parquet] [arrow] (tustvold)ArrowNativeType::FromStr
#2775 [arrow] (tustvold)arrow-array
crate (#2594) #2769 [arrow] (tustvold)dyn_arith_dict
feature flag #2760 [arrow] (tustvold)arrow-data
into a separate crate #2746 [arrow] (tustvold)Implemented enhancements:
binary
function should not panic on unequaled array length. #2721 [arrow]Fixed bugs:
try_unary_dict
should check value type of dictionary array #2754 [arrow]Closed issues:
23.0.0
(next release after 22.0.0
) #2665 [parquet] [arrow] [arrow-flight]Merged pull requests:
verify_release_candidate.sh
for new arrow subcrates #2752 (alamb)Breaking changes:
Implemented enhancements:
RecordBatchOptions
#2728 [arrow]CHANGELOG-old.md
#2726DivideByZero
error for float modulus #2720 [arrow]try_binary
should not panic on unequaled array length. #2715 [arrow]DataType
function to detect nested types #2704 [arrow]RecordBatch::project
#2690 [arrow]min_max_helper
#2657 [arrow]downcast_primitive_array
in arithmetic kernels #2639 [arrow]Fixed bugs:
DivideByZero
in the Divide
kernel #2719 [arrow]arrow
crate does not build with features = ["ffi"]
and default_features = false
. #2670 [arrow]RowSelector
having row_count
of 0 #2669 [parquet]crate::array::layout
#2659 [arrow]CastOptions
#2648 [arrow]flight.proto
and fix schema to SchemaResult #2571 [arrow] [arrow-flight]SchemaResult
in IPC deviates from other implementations #2445 [arrow] [arrow-flight]Closed issues:
Merged pull requests:
CHANGELOG-old.md
#2732 (iajoiner)try_binary
and checked_ops
, and remove math_checked_op
#2717 [arrow] (HaoYang670)numeric_cast_with_error
#2661 [arrow] (liukun4515)test_fuzz_async_reader_selection
#2656 [parquet] (thinkharderdev)CastOption
for casting numeric #2649 [arrow] (liukun4515)downcast_primitive_array
in arithmetic kernels #2640 [arrow] (viirya)Breaking changes:
total_cmp
for floating value ordering and remove nan_ordering
feature flag #2614 [arrow] (viirya)dyn_cmp_dict
#2597 [arrow] (tustvold)Implemented enhancements:
total_cmp
for floating value ordering and remove nan_ordering
feature flag #2613 [arrow]dyn_cmp_dict
feature flag to gate dyn comparison of dictionary arrays #2596 [arrow]do_get()
#2581 [arrow-flight]eq_dyn
, neq_dyn
, lt_dyn
, lt_eq_dyn
, gt_dyn
, gt_eq_dyn
#2569 [arrow]sum
for dictionary array #2565 [arrow]lt_dyn
, lt_eq_dyn
, gt_dyn
, gt_eq_dyn
#2538 [arrow]eq_dyn
and neq_dyn
#2535 [arrow]like_utf8_scalar
for %pat%
#2519 [arrow]Fixed bugs:
ParquetFileArrowReader::get_record_reader[_by_colum]
batch_size
overallocates #2321 [parquet]Documentation updates:
Closed issues:
Interval(DayTime)
to Timestamp(Nanosecond, None)
#2606 [arrow]length
field for Buffer
#2524 [arrow]const_generic
. #2390 [arrow]const_generic
#2384 [arrow]Merged pull requests:
length
field for Buffer and use more Buffer
in IPC reader to avoid memory copy. #2557 [arrow] [arrow-flight] (HaoYang670)Decimal
) when creating array in IPC reader #2547 [arrow] (HaoYang670)eq_dyn
and neq_dyn
#2533 [arrow] (viirya)create_array
in IPC reader. #2525 [arrow] (HaoYang670)%pat%
(>3x speedup) #2521 [arrow] (Dandandan)Breaking changes:
ColumnCloseResult
(#2465) #2466 [parquet] (tustvold)ChunkReader
into SerializedPageReader
(#2463) #2464 [parquet] (tustvold)Implemented enhancements:
ChunkReader
into SerializedPageReader
#2463 [parquet]RowFilter
withinParquetRecordBatchReader
#2431 [parquet]StructBuilder::len
#2429 [arrow]ParquetRecordBatchReaderBuilder
) #2427 [parquet]OffsetIndex
to Prune IO in ParquetRecordBatchStream
#2426 [parquet]peek_next_page
and skip_next_page
in InMemoryPageReader
#2406 [parquet]Utf8
/LargeUtf8
to Binary
/LargeBinary
#2402 [arrow]Decimal128
and Decimal256
arrays #2375 [arrow]skip_records
#2358 [parquet]read_buffer
Arrow IPC API to return Result<_>
#2342 [arrow]skip_records
in GenericColumnReader
to skip across row groups #2331 [parquet]Decimal256
#2320 [arrow]DeltaBitPackDecoder
#2281 [parquet]ParquetRecordBatchStream
to support row filtering in DataFusion #2270 [parquet]ArrayReader::skip_records
API #2197 [parquet]Fixed bugs:
DoPutUpdateResult
#2403 [arrow-flight]AsyncFileReader
No Longer Object-Safe #2372 [parquet]Closed issues:
Merged pull requests:
MapArrayReader
(#2484) (#1699) (#1561) #2500 [parquet] (tustvold)split_second
function #2436 [arrow] (viirya)Decimal
and DecimalArray
using const_generic
#2383 [parquet] [arrow] (HaoYang670)Decimal128
and Decimal256
arrays #2376 [arrow] (viirya)Box<dyn AsyncFileReader>
#2368 [parquet] (tustvold)Decimal256
validation based on bytes comparison and add benchmark test #2360 [parquet] [arrow] (liukun4515)field.rs
and add more tests #2345 [arrow] (HaoYang670)Breaking changes:
GenericBinaryArray
and GenericListArray
: add PREFIX
and data type constructor #2327 [parquet] [arrow] (HaoYang670)safe
(#2302) #2303 [arrow] (tustvold)test_utils
from default features (#2298) #2299 [arrow] (tustvold)DataType::Decimal
to DataType::Decimal128
#2229 [parquet] [arrow] (viirya)Decimal128Iter
and Decimal256Iter
and do maximum precision/scale check #2140 [arrow] (viirya)Implemented enhancements:
ListArray
#2311 [arrow]FlightSqlService
trait to pass session info along #2308 [arrow-flight]take_bits
for non-null indices #2306 [arrow]ffi
#2302 [arrow]ffi::ArrowArray::try_new
is safe #2301 [arrow]JsonEqual
trait #2296 [arrow]with_precision_and_scale
to Decimal
array traits #2291 [arrow]ArrayReader::next_batch
with ArrayReader::read_records
and ArrayReader::consume_batch
#2236 [parquet]DataType::Decimal
to DataType::Decimal128
#2228 [arrow]append_option
support to Decimal128Builder
and Decimal256Builder
#2224 [arrow]FixedSizeBinaryArray
and FixedSizeListArray
from array_binary.rs
and array_list.rs
#2217 [arrow]Box
Values in PrimitiveDictionaryBuilder
#2215 [arrow]Hash
for Schema
#2182 [arrow]GenericStringBuilder
should use GenericBinaryBuilder
#2156 [arrow]Decimal128
and Decimal256
#2139 [arrow]ArrayAccessor
in Decimal128Iter
and Decimal256Iter
#2138 [arrow]ArrayAccessor
and FromIterator
in Cast Kernels #2137 [arrow]TypedDictionaryArray
for more ergonomic interaction with DictionaryArray
#2136 [arrow]ArrayAccessor
in Comparison Kernels #2135 [arrow]peek_next_page()
and skip_next_page
in InMemoryColumnChunkReader
#2129 [parquet]Decimal256
#2112 [arrow]skip_def_levels
for ColumnLevelDecoder
#2107 [parquet]Time32
/ Time64
#2053 [arrow]Fixed bugs:
Decimal128
or Decimal256
#2256 [arrow]IntervalMonthDayNanoType::make_value()
does not match C implementation #2234 [arrow]FlightSqlService
trait does not allow impl
s to do handshake #2210 [arrow-flight]EnabledStatistics::None
not working #2185 [parquet]Documentation updates:
Performance improvements:
Closed issues:
Merged pull requests:
Request<FlightDescriptor>
to FlightSqlService
impl
s #2309 [parquet] [arrow-flight] (avantgardnerio)with_precision_and_scale
to BasicDecimalArray
trait #2292 [parquet] [arrow] (viirya)fn get_data_type
by const DATA_TYPE
in BinaryArray and StringArray #2289 [arrow] (HaoYang670)from_list
function. #2277 [arrow] (HaoYang670)like_utf8
/ nlike_utf8
kernels #2258 [arrow] (daniel-martinez-maqueda-sap)FixedSizeList
to array_fixed_size_list.rs
#2250 [arrow] (HaoYang670)Schema::fields
and Schema::metadata
pub
(public) #2239 [arrow] (alamb)IntervalMonthDayNanoType::make_value()
to conform to specifications #2235 [arrow] (avantgardnerio)arrow
CI on changes to arrow #2227 (alamb)FixedSizeBinaryArray
to array_fixed_size_binary.rs
#2218 [arrow] (HaoYang670)FlightSqlService
trait to proxy handshake #2211 [arrow-flight] (avantgardnerio)serde_json
feature name #2209 [parquet] (flisky)Hash
for Schema
#2183 [arrow] (crepererum)StringBuilder
use BinaryBuilder
#2181 [arrow] (HaoYang670)BYTE_ARRAY
type #2160 [parquet] (liukun4515)peek\_next\_page
and skip\_next\_page
for `InMemoryColumnCh… #2155 [parquet] (thinkharderdev)arrow
changes #2152 (alamb)object_store
code from object_store_rs to arrow-rs #2081 (alamb)validate_utf8
performance #2048 [arrow] (tfeda)Breaking changes:
DecimalArray``/DecimalBuilder
to Decimal128Array
/Decimal128Builder
#2101 [arrow]append
methods to be infallible where possible #2103 [parquet] [arrow] (jhorstmann)UnionArray::child
(#2035) #2099 [arrow] (tustvold)preserve_order
feature from serde_json
dependency (#2095) #2098 [parquet] [arrow] (tustvold)weekday
and weekday0
kernels to to num_days_from_monday
and num_days_since_sunday
#2066 [arrow] (alamb)null_count
from write_batch_with_statistics
#2047 [parquet] (tustvold)Implemented enhancements:
total_cmp
from std #2130 [arrow]ParquetRecordBatchStream
#2110 [parquet]GenericBinaryBuilder
should use buffer builders directly. #2104 [arrow]generate_decimal256_case
arrow integration test #2093 [arrow]weekday
and weekday0
kernels to to num_days_from_monday
and days_since_sunday
#2065 [arrow]filter_dict
#2062 [arrow]set_bits
#2060 [arrow]BooleanBuilder
#2058 [arrow]BooleanArray::from_iter
should omit validity buffer if all values are valid #2055 [arrow]DICTIONARY_ORDERED
flag if a field's dictionary is ordered #2049 [arrow]peek_next_page()
and skip_next_page
in SerializedPageReader
#2043 [parquet]MapType
#2037 [arrow]DecimalArrayBuilder
should use FixedSizedBinaryBuilder
#2026 [arrow]serialized_reader
read specific Page by passing row ranges. #1976 [parquet]Fixed bugs:
type_id
and value_offset
are incorrect for sliced UnionArray
#2086 [arrow]take
kernel does not handle null indices correctly #2057 [arrow]write_batch_with_statistics
#2046 [parquet]WriterProperties
#2014 [parquet]Documentation updates:
Closed issues:
serde_json
specify the preserve_order
feature in arrow
package #2095 [arrow]skip_values
in DictionaryDecoder #2079 [parquet]skip_values
in ByteArrayColumnValueDecoder
#2072 [parquet]Builder::append
methods returning results even though they are infallible #2071UnionArray::child
#2035Merged pull requests:
total_cmp
from std #2131 [arrow] (Dandandan)match
arms have incompatible types #2121 (alamb)GenericBinaryBuilder
to use buffer builders directly. #2117 [arrow] (HaoYang670)DataType::Decimal256
#2094 [parquet] [arrow] (viirya)DecimalBuilder
should use FixedSizeBinaryBuilder
#2092 [arrow] (HaoYang670)FixedSizeBinaryArray
to DecimalArray
#2041 [arrow] (HaoYang670)Breaking changes:
EnabledStatistics
to control level of statistics generated #2022 [parquet] (tustvold)Decimal256Builder
and Decimal256Array
; Decimal arrays now implement BasicDecimalArray
trait #2000 [parquet] [arrow] (viirya)ColumnReader::read_batch
#1995 [parquet] [arrow] (tustvold)PrimitiveBuilder::finish_dict
(#1978) #1980 [arrow] (tustvold)NullType
#1942 [arrow] (liukun4515)Implemented enhancements:
DataType::Dictionary
support to subtract_scalar
, multiply_scalar
, divide_scalar
#2019 [arrow]add_scalar
kernel #2017 [arrow]FixedSizeBinaryBuilder
#2007 [arrow]Decimal256Builder
and Decimal256Array
#1999 [arrow]DictionaryArray
in unary
kernel #1989 [arrow]Array
s #1987 [arrow]DictionaryArray
in divide
kernel #1982 [arrow]Into<ArrayData>
for T: Array
#1979 [arrow]DictionaryArray
in multiply
kernel #1972 [arrow]DictionaryArray
in subtract
kernel #1970 [arrow]DecimalArray::length
as a constant #1967 [arrow]DictionaryArray
in add
kernel #1950 [arrow]Field
#1934 [arrow]StringDictionaryBuilder
faster #1851 [arrow]concat_elements_utf8
should accept arbitrary number of input arrays #1748 [arrow]Fixed bugs:
ColumnWriterImpl::write_batch_with_statistics
incorrect distinct count in statistics #2016 [parquet]ColumnWriterImpl::write_batch_with_statistics
can write incorrect page statistics #2015 [parquet]RowFormatter
is not part of the public api #2008 [parquet]ColumnReader::read_batch
For Corrupted Files #1997 [parquet]PrimitiveBuilder::finish_dict
does not validate dictionary offsets #1978 [arrow]n_buffers
in FFI_ArrowArray
#1959 [arrow]DecimalArray::from_fixed_size_list_array
fails when offset > 0
#1958 [arrow]Send
+ Sync
impl for Allocation
may not be sound unless Allocation
is Send
+ Sync
as well #1944 [arrow]NullType
#1923 [arrow]Documentation updates:
Closed issues:
min_bytes
and max_bytes
return wrong size #2021 [parquet]Decimal
s by using constant generic. #2001DecimalArray
to a new file #1985 [arrow]DictionaryArray
in multiply
kernel #1974null_count
of DictionaryArray #1962 [arrow]TIMESTAMP
in schema #1920 [parquet]Merged pull requests:
Field
: with_name
, with_data_type
and with_nullable
#2024 [arrow] (alamb)FixedSizeBinaryBuilder
#2013 [arrow] (HaoYang670)Into<ArrayData>
for T: Array
#1992 [parquet] [arrow] (heyrutvik)FixedSizeListBuilder
#1988 [arrow] (HaoYang670)DecimalArray
to array_decimal.rs #1986 [arrow] (HaoYang670)const
#1968 [arrow] (HaoYang670)from_fixed_size_list
when offset > 0 #1964 [arrow] (HaoYang670)FixedSizeListArray::value_length
#1957 [arrow] (HaoYang670)ColumnMetadata
after the column chunk data, not the ColumnChunk
#1947 [parquet] (liukun4515)Breaking changes:
RecordBatch
for non-nullable fields containing null values #1890 [arrow] (andygrove)ArrayData::validate_dict_offsets
to ArrayData::validate_values
#1889 [arrow] (frolovdev)Decimal128
API and use it in DecimalArray and DecimalBuilder #1871 [parquet] [arrow] (viirya)safe
(#996) (#1027) #1866 [parquet] [arrow] (tustvold)Implemented enhancements:
ArrowWriter
being used with a cursor #1927 [parquet]cast
to/from NULL
and DataType::Decimal
#1921 [arrow]Decimal256
API #1913 [arrow]DictionaryArray::key
function #1911 [arrow]ListArrays
in MutableArrayData
#1884 [arrow]PrimitiveArray::from_iter
should omit validity buffer if all values are valid #1856 [arrow]from(v: Vec<Option<&[u8]>>)
and from(v: Vec<&[u8]>)
for FixedSizedBInaryArray
#1852 [arrow]Vec
-inspired APIs to BufferBuilder
#1850 [arrow]nilike
support in comparison
#1845 [arrow]arrow::array::builder
module #1843 [arrow]quarter
support in temporal
kernels #1835 [arrow]ArrayData::validate_dictionary_offset
to ArrayData::validate_values
#1812 [arrow]substring
kernel #1801 [arrow]substring_by_char
kernel #1800 [arrow]Fixed bugs:
dynamic_types
example does not print the projection #1902 [arrow]log2(0)
panicked at 'attempt to subtract with overflow', parquet/src/util/bit_util.rs:148:5
#1901 [parquet]combine_option_bitmap
needs to use bit slices #1899 [arrow]RecordBatch
with null values in non-nullable fields does not cause an error #1888 [arrow]regex
dependency #1874 [arrow]Documentation updates:
DictionaryArray::lookup_key
#1910 [arrow] (alamb)Closed issues:
Merged pull requests:
arrow::array::builder
module (#1843) #1928 [arrow] (tustvold)checked_add/sub().unwrap()
with +/-
#1924 [arrow] (HaoYang670)NULL
to/from Decimal
#1922 [arrow] (liukun4515)Decimal256
API #1914 [arrow] (viirya)DictionaryArray::key
function #1912 [arrow] (alamb)bit_util
of Parquet. #1905 [parquet] (HaoYang670)weekday
#1894 [arrow] (alamb)MutableArrayData
#1885 [arrow] (jhorstmann)regex
dependency to 1.5.6 #1875 [arrow] (martin-g)BitSliceIterator
and BitIndexIterator
(#1864) #1865 [arrow] (tustvold)from
methods for FixedSizeBinaryArray
#1854 [arrow] (HaoYang670)substring
kernel. #1853 [arrow] (HaoYang670)nilike
support in comparison
#1846 [arrow] (MazterQyou)test_command_help
#1844 [parquet] (viirya)quarter
support in temporal
#1836 [arrow] (MazterQyou)substring_by_char
by about 2.5x #1832 [arrow] (HaoYang670)Breaking changes:
ArrowNativeType
and OffsetSizeTrait
for safety (#1028) #1819 [arrow] (tustvold)csv::infer_file_schema
by removing redundant ref #1776 [arrow] (tustvold)Implemented enhancements:
ListArray
#1817 [arrow]UnionArray
in V4 IPC message #1793 [arrow]combine_option_bitmap
should accept arbitrary number of input arrays. #1780 [arrow]substring_by_char
kernels for slicing on character boundaries #1768 [arrow]PageIndex
from column metadata #1761 [parquet]DataType::Utf8
to DataType::Boolean
#1740 [arrow]FileWriter
. #1691 [parquet]stdout
#1687 [parquet]Fixed bugs:
flight_data_to_arrow_batch
does not support RecordBatch
es with no columns #1783 [arrow-flight]features=["zstd"]
#1630 [parquet]Documentation updates:
CHANGELOG-old.md
#1759 (alamb)Closed issues:
DataType::Decimal
Non-Compliant? #1779 [arrow]Parquet
is a part of Arrow
? #1715 [parquet] [arrow]Merged pull requests:
PageIterator
Documentation #1831 [parquet] (Ted-Jiang)resolver v2
, test more feature flag combinations in CI, fix errors (#1630) #1822 [parquet] [arrow] (tustvold)arrow/benches/string_kernels.rs
to arrow/benches/substring_kernels.rs
#1805 [arrow] (HaoYang670)Substring_by_char
#1784 [arrow] (HaoYang670)ParquetFileArrowReader::try_new
#1782 [parquet] (tustvold)ChunkReader
for Bytes
, deprecate SliceableCursor
#1775 [parquet] (tustvold)validate_decimal_precision
check in DecimalBuilder.append_value
for interop test #1767 [arrow] (viirya)Utf8
to Boolean
#1738 [arrow] (MazterQyou)Breaking changes:
ArrayDataBuilder::null_bit_buffer
to accept Option<Buffer>
rather than Buffer
#1739 [arrow] (HaoYang670)null_count
from ArrayData::try_new()
#1721 [arrow] (HaoYang670)std:io::Write
rather custom ParquetWriter
trait (#1717) (#1163) #1719 [parquet] (tustvold)ProjectionMask
(#1701) #1716 [parquet] (tustvold)Implemented enhancements:
string
kernel to concatenate_elements
#1747 [arrow]ArrayDataBuilder::null_bit_buffer
should accept Option<Buffer>
as input type #1737 [arrow]BinaryArray
#1724 [arrow]generate_unions_case
integration test #1712 [arrow]leaf_columns
argument to ArrowReader::get_record_reader_by_columns
#1653 [parquet]string_concat
kernel #1540 [arrow]Fixed bugs:
ArrayData::try_new
cannot always return expected error. #1707 [arrow]substring
kernel): The null buffer is not aligned when offset != 0
#1639 [arrow]Documentation updates:
Merged pull requests:
string_concat
to concat_elements_utf8
#1754 [arrow] (alamb)string
kernel to concat_elements
. #1752 [arrow] (HaoYang670)cargo install
installs not globally #1732 [parquet] (kazuk)min_binary
and max_binary
aggregate kernels #1725 [arrow] (HaoYang670)null_bit_buffer
in ArrayData::try_new()
#1714 [arrow] (HaoYang670)generate_unions_case
integration test #1713 [arrow] (viirya)offset
in substring
kernel. #1704 [arrow] (HaoYang670)OffsetSizeTrait
to extend num::Integer
#1702 [arrow] (HaoYang670)Breaking changes:
bytes
in parquet rather than custom Buffer implementation (#1474) #1683 [parquet] (tustvold)OffsetSize::fn is_large
to const OffsetSize::IS_LARGE
#1664 [parquet] [arrow] (HaoYang670)StringOffsetTrait
and BinaryOffsetTrait
#1645 [arrow] (HaoYang670)generate_nested_dictionary_case
integration test failure #1636 [arrow] [arrow-flight] (viirya)Implemented enhancements:
DataType::Duration
in ffi interface #1688 [arrow]generate_unions_case
integration test #1676 [arrow]DictionaryArray
support for bit_length
kernel #1673 [arrow]DictionaryArray
support for length
kernel #1672 [arrow]Field
#1646 [arrow]StringOffsetTrait
and BinaryOffsetTrait
#1644 [arrow]UnionArray::from(data: ArrayData)
#1643 [arrow]pub fn offsets_buffer
, pub fn types_ids_buffer
and pub fn data_buffer
for ArrayDataBuilder
#1640 [arrow]generate_nested_dictionary_case
integration test failure for Rust cases #1635 [arrow]ArrowWriter
row group flush in public API #1626 [parquet]substring
support for FixedSizeBinaryArray
#1618 [arrow]UnionArray
s #1594 [arrow]length
kernel #1489 [arrow]DecimalType
support in new_null_array
#1659 [arrow] (yjshen)Fixed bugs:
UnionArray::is_null
incorrect #1625 [arrow]arrow::async_reader
#1617 [parquet]Documentation updates:
Closed issues:
OffsetSizeTrait::IS_LARGE
as a const value #1658OffsetSizeTrait
s? #1638UnionArray
Changes DataType #1595Merged pull requests:
GenericListArray::get_type
#1650 [arrow] (HaoYang670)UnionArray
s #1648 [arrow] (tfeda)dict_id
and dict_is_ordered
from equality comparison of Field
#1647 [arrow] (viirya)substring
support for FixedSizeBinaryArray
#1633 [arrow] (HaoYang670)async
into doc features #1349 [parquet] (HaoYang670)Breaking changes:
parquet::basic::LogicalType
to be more idomatic #1612 [parquet] (tfeda)ArrayData
, UnionArray
, and MapArray
#1589 [arrow] (tustvold)&Option<T>
with Option<&T>
in several arrow
and parquet
APIs #1571 [parquet] [arrow] (tfeda)Implemented enhancements:
BinaryArray
in substring
kernel #1593 [arrow]Clone
for JSON DecoderOptions
#1580 [arrow]substring
kernel #1575 [arrow]DataType::Null
in cast
kernel #1572 [arrow] (WinkerDu)Fixed bugs:
force_validate
is active) #1587 [parquet]ipc::reader::tests::projection_should_work
fails validation #1548 [arrow]Documentation updates:
Closed issues:
&Option<T>
with Option<&T>
in some APIs #1556 [parquet] [arrow]parquet::basic::LogicalType
#1554 [parquet]substring
function as unsafe
and rename it. #1541 [arrow]Merged pull requests:
div_ceil
#1629 [arrow] (HaoYang670)substring
support for binary #1608 [arrow] (HaoYang670)ParquetTypeConverter
#1592 [parquet] (viirya)Clone
and PartialEq
for json DecoderOptions
#1581 [arrow] (alamb)substring
#1577 [arrow] (HaoYang670)Option<T>
rather than Option<&T>
for copy types in substring kernel #1576 [arrow] (tustvold)projection_should_work
#1573 [arrow] (viirya)Breaking changes:
ArrowReaderOptions
to ParquetFileArrowReader
, add option to skip decoding arrow metadata from parquet (#1459) #1558 [parquet] (tustvold)RecordBatch
with zero columns but non zero row count, add field to RecordBatchOptions
(#1536) #1552 [arrow] (tustvold)DecoderOptions
#1539 [arrow] (alamb)prost
, prost-derive
and prost-types
to 0.10, tonic
, and tonic-build
to 0.7
#1510 [arrow-flight] (alamb)DecoderOptions
and support custom format_string
for each field #1451 [arrow] (sum12)Implemented enhancements:
FixedSizeBinary
in the Arrow C data interface #1553 [arrow]ParquetRecordBatchReader
#1537 [parquet]RecordBatch
with zero columns but non zero row count #1536 [arrow]Date32
/Date64
<--> String
/LargeString
in cast
kernel #1535 [arrow]Vec
or String
#1516 [arrow]substring
kernel #1511 [arrow]Fixed bugs:
filter
produces invalid sparse UnionArray
s #1547 [arrow]GenericListBuilder
is not exposed. #1518 [arrow]substring
kernel panics when chars > U+0x007F #1478 [arrow]Documentation updates:
substring
kernel #1529 [arrow] (HaoYang670)GenericListBuilder
#1525 [arrow] (comath)take
kernel documentation #1524 [arrow] (alamb)Closed issues:
min_max_helper
#1400Merged pull requests:
into_buffers
for UnionArray #1567 [arrow] (viirya)MapArray
into separate module (#1483) #1562 [parquet] (tustvold)ParquetRecordBatchReader
#1560 [parquet] (tustvold)StreamReader
#1550 [arrow] (dispanser)cast
kernel #1534 [arrow] (yjshen)remove-old-releases.sh
executable #1522 (alamb)sort
kernel #1519 [arrow] (HaoYang670)substring
kernel by about 2x #1512 [arrow] (HaoYang670)new_from_strings
to create MapArrays
#1507 [arrow] (viirya)Implemented enhancements:
size_hint
and ExactSizedIterator
for DecimalArray #1505 [arrow]StringArray
#1493 [arrow]length
kernel support for ListArray
#1470 [arrow]BinaryArray
s #1464 [arrow]DictionaryArray::try_new()
#1313 [arrow]Fixed bugs:
Documentation updates:
BinaryArray
and LargeBinaryArray
#1455 [arrow]GenericBinaryArray::take_iter_unchecked
#1454 [arrow]BinaryOffsetSizeTrait
#1453 [arrow]FixedSizeBinaryArray
is confusing. #1452 [arrow]BinaryArray
and LargeBinaryArray
#1471 [arrow] (HaoYang670)Closed issues:
packed_simd
v.s. portable_simd
, which should be used? #1492Merged pull requests:
size_hint
and ExactSizedIterator
for DecimalArray
#1506 [arrow] (alamb)StringArray::num_chars
for calculating number of characters #1503 [arrow] (HaoYang670)try_from_trusted_len_iter
#1497 [arrow] (jhorstmann)length
kernel support for List Array #1488 [arrow] (HaoYang670)Decimal
data type #1487 [arrow] (yjshen)length
kernel on Binary Array #1465 [arrow] (HaoYang670)Breaking changes:
filter_row_groups
with ReadOptions
in parquet SerializedFileReader #1389 [parquet] (yjshen)IPC Reader
file / streams #1339 [arrow] [arrow-flight] (Dandandan)Implemented enhancements:
ListArray
and LargeListArray
more readable #1433if
and abs
in shift()
#1427FixedSizeBinaryArray
#1419shift
#1408UnionArray
s #1394 [arrow]FixedSizeListArrays
s #1393 [arrow]rust-version
to 1.59 #1377Fixed bugs:
GenericListArray
is confusing #1424Documentation updates:
GenericListArray
#1450 [arrow] (HaoYang670)Merged pull requests:
ListArray
and LargeListArray
#1447 [arrow] (HaoYang670)filter
kernel should work with FixedSizeListArrays #1434 [arrow] (viirya)FixedSizeListArray
#1426 [arrow] (HaoYang670)default-features = false
for zstd
in the parquet crate to support wasm32-unknown-unknown
#1414 [parquet] (kylebarron)UnionArray
infilter
kernel #1412 [arrow] (viirya)shift
#1409 [arrow] (HaoYang670)FixedSizeBinaryArray
input all none #1405 [arrow] (jackwener)Breaking changes:
ColumnChunkMetaData
#1346 [parquet] (shanisolomon)has_
methods in ColumnChunkMetaData
#1345 [parquet] (shanisolomon)Implemented enhancements:
IpcWriteOptions
#1381 [arrow]MapArray
in filter
kernel #1378 [arrow]week
temporal kernel #1375 [arrow]compare_dict_op
#1371 [arrow]arrow::array::builder::MapBuilder
public #1354 [arrow]StructArray::from
#1351 [arrow]RecordBatch::validate_new_batch
#1350 [arrow]write
method to JsonWriter #1340 [arrow]Bitmap::new
#1337 [arrow]compare_dict_op
#1329 [arrow]as_decimal_array(arr: &dyn Array) -> &DecimalArray
#1312 [arrow]eq_dyn
, neq_dyn
, lt_dyn
, lt_eq_dyn
, gt_dyn
, gt_eq_dyn
#1201 [arrow]Fixed bugs:
cargo clippy
fails on the master
branch #1362 [arrow]ArrowArray::try_from_raw
should not assume the pointers are from Arc #1333 [arrow]with_datetime_format()
to CSV WriterBuilder #1272 [arrow]Performance improvements:
Closed issues:
ColumnChunkMetadata
#1332Merged pull requests:
day
and month
in temporal.rs #1388 [arrow] (Ted-Jiang)Clone
for IpcWriteOptions
#1382 [arrow] (matthewmturner)week
in temporal.rs #1376 [arrow] (Ted-Jiang)min_max_string
#1374 [arrow] (HaoYang670)take_iter_unchecked
#1372 [arrow] (viirya)Option
) #1367 [arrow] (viirya)RecordBatch::validate_new_batch
#1361 [arrow] (HaoYang670)StructArray::from
#1360 [arrow] (HaoYang670)as_decimal_array
function #1356 [arrow] (liukun4515)Bitmap::new
#1343 [arrow] (HaoYang670)ArrowArray::try_from_raw
should not assume the pointers are from Arc #1334 [arrow] (viirya)compare_dict_op
#1330 [arrow] (viirya)Implemented enhancements:
DeltaBitPackDecoder
#1281 [parquet]ColumnChunkMetadata
#1322 [parquet] (shanisolomon)ColumnChunkMetadata
#1318 [parquet] (shanisolomon)ColumnChunkMetadata
#1309 [parquet] (shanisolomon)DictionaryArray::try_new()
to create dictionaries from pre existing arrays #1300 [arrow] (alamb)DictionaryArray::keys_iter
, and take_iter
for other array types #1296 [arrow] (viirya)rle
decoder public under experimental
feature #1271 [parquet] (zeevm)DictionaryArray
support in eq_dyn
kernel #1263 [arrow] (viirya)Fixed bugs:
len
is not a parameter of MutableArrayData::extend
#1316data_type
is private in Rust Parquet 8.0.0 #1302 [parquet]Documentation updates:
Performance improvements:
compute
module (up to 10x faster) #1248 [parquet] [arrow] (tustvold)Closed issues:
DictionaryArrays
from Key
and Value
arrays #1299DictionaryArray
#1295 [arrow]BitReader
#1282Merged pull requests:
MutableArrayData::extend
#1336 [arrow] (HaoYang670)dead_code
#1324 [arrow] (gsserge)to_thrift
#1320 [parquet] (shanisolomon)clippy::type_complexity
#1310 [arrow] (gsserge)clippy::float_equality_without_abs
#1305 [arrow] (gsserge)clippy::vec_init_then_push
lint #1303 [arrow] (gsserge)LevelEncoder
variants to conform to Rust standards #1273 [parquet] (HaoYang670)DecimalArray
API ergonomics: add iter()
, FromIterator
, with_precision_and_scale
#1223 [arrow] (alamb)Breaking changes:
Send
+ Sync
to DataType
, RowGroupReader
, FileReader
, ChunkReader
. #1264Bitmap::len
to Bitmap::bit_len
to clarify its meaning #1242 [parquet] [arrow] (HaoYang670)memory-check
feature #1222 [arrow] (jhorstmann)RecordBatches
before writing a parquet row group in ArrowWriter
#1214 [parquet] [arrow] (tustvold)Implemented enhancements:
async
arrow parquet reader #1154 [parquet] [arrow] (tustvold)Bitmap::len
to Bitmap::bit_len
#1233eq_dyn
etc. #1202 [arrow]BinaryArray
#1108impl ArrowNativeType for i128
#1098Copy
trait bound from dyn scalar kernels #1243 [arrow] (matthewmturner)into_inner
for IPC FileWriter
#1236 [arrow] (yjshen)array::builder::make_builder
to make it available for downstream #1235 [arrow] (yjshen)Fixed bugs:
Unknown configuration option rust-version
when running the rust format command #1240Bitmap
Length Validation is Incorrect #1231 [arrow]ListArray
or MapArray
ignore offsets #1226 [parquet]memory-tracking
crate feature #1171parquet::data_type
and parquet::arrow::schema
experimental #1244 [parquet] (tustvold)Documentation updates:
Performance improvements:
simd
feature enabled (except for division/modulo) #1221 [arrow] (jhorstmann)Closed issues:
UnalignedBitChunkIterator
to that iterates through already aligned u64
blocks #1227ArrowArrayReader
in parquet #1197 [parquet]Merged pull requests:
Breaking changes:
ArrowSignedNumericType
to Simplify and reduce code duplication in arithmetic kernels #1161 [arrow] (jhorstmann)Implemented enhancements:
arbitrary_precision
feature flag #1174 [arrow]Err
from JSON writer rather than panic!
for unsupported types #1157 [arrow]scalar
mathematics kernels for Array
and scalar value #1153 [arrow]DecimalArray
in sort kernel #1137from_iter_values
for binary array #1188 [arrow] (Jimexist)MapArray
in json writer #1149 [arrow] (helgikrs)Fixed bugs:
RecordBatch
containing Float16
triggers a panic #1193 [arrow]GenericStringArray::from_iter_values
if reported iterator upper bound is incorrect #1144 [arrow]simd
feature asserts #1136 [arrow]Security fixes:
Documentation updates:
from_trusted_len_iter
#1148 [arrow] (alamb)Performance improvements:
Closed issues:
Merged pull requests:
Float16Array
#1194 [arrow] (helgikrs)sort
and take
kernels #1172 [arrow] (liukun4515)GenericStringArray::from_vec
and GenericStringArray::from_opt_vec
#1147 [arrow] (alamb)Breaking changes:
pretty_format_batches
now returns Result<impl Display>
rather than String
: #975MutableBuffer::typed_data_mut
is marked unsafe
: #1029UnionMode
, UnionArray::new()
marked unsafe
: #885New Features:
Float16Array
types #888UnionArray
#654eq_dyn_scalar
), including DictionaryArray
: #1113Enhancements:
Schema::with_metadata
and Field::with_metadata
#1092Array
for ArrayRef
for easier use #1129FixedSizeBinaryArray
#1097pyo3
, parquet-format
, prost
, tonic
lexicographical_partition_ranges
#998Fixed bugs:
take
kernel (#1046) (#1057)rc
feature of serde (#990) (#1016)Changes:
Features / Fixes:
Features / Fixes:
Other:
Breaking changes:
ArrayData::new()
with ArrayData::try_new()
and unsafe ArrayData::new_unchecked
#822 [parquet] [arrow] (alamb)Implemented enhancements:
ArrowWriter
#788Fixed bugs:
minute
and second
temporal kernels do not respect timezone #500Documentation updates:
Implemented enhancements:
Fixed bugs:
RowColumIter
#762Implemented enhancements:
append_nulls
and append_trusted_len_iter
for PrimitiveBuilder #725Fixed bugs:
unary
kernel doesn't respect offset #738new_null_array
creates invalid struct arrays #734Bitmap::len
returns the number of bytes, not bits. #730Closed issues:
Implemented enhancements:
Fixed bugs:
Closed issues:
Implemented enhancements:
value
method of boolean and primitive arrays #645Fixed bugs:
Closed issues:
Implemented enhancements:
limit
if sorted unstably #553Fixed bugs:
~/.cargo
is in a git checkout #589array::ffi::tests::test_struct
and other ffi tests #580Documentation updates:
Closed issues:
Breaking changes:
Implemented enhancements:
RecordBatch::concat
#537 [arrow] (silathdiir)BitReader
/DeltaBitPackDecoder
#325 [parquet] (kornholi)Fixed bugs:
ahash v0.7.4
depends on itself. Cycle #544default-features=false
#529Documentation updates:
Merged pull requests:
StringArray
creation from String Vec #522 [arrow] (silathdiir)ArrayData
equality and update unit tests #472 (Jimexist)SliceableCursor
to be constructed from an Arc
directly #369 (crepererum)take
#281 (Dandandan)Breaking changes:
Implemented enhancements:
Fixed bugs:
read_scalar
and read_scalar_at
allow transmuting values without unsafe
blocks #476Security fixes:
Closed issues:
Implemented enhancements:
Fixed bugs:
not
kernel does not take offset of null buffer into account #417WriterProperties.max_row_group_size
not wired up #257Breaking changes:
Implemented enhancements:
Arc
-based constructor for parquet::util::cursor::SliceableCursor
#368shrink_to
/ shrink_to_fit
to MutableBuffer
#297Fixed bugs:
Implemented enhancements:
Fixed bugs:
select_nth_unstable_by
found for mutable reference &mut [T]
#283metadata
when deserialize #241 [arrow]Security fixes:
Documentation updates:
Closed issues: