Breaking changes:
zip
compute function to operator on Scalar
values via Datum
#5086 [arrow] (Nathan-Fenner)Implemented enhancements:
quote
and escape
in Csv WriterBuilder
#5146 [arrow]bytes::Bytes
into arrow_buffer::Buffer
without copy #5104 [arrow]48.0.1
arrow patch release #5050 [parquet] [arrow]zip
compute kernel to take Scalar
/ Datum
#5011 [arrow]Fixed bugs:
Z
in timezone is ignored #5182 [arrow]Utf8
to decimal failure #5127 [arrow]Performance improvements:
Closed issues:
Merged pull requests:
try_new
when casting between structs to propagate error #5226 [arrow] (viirya)entries
to MapBuilder
to return both key and value array builders #5218 [arrow] (viirya)From<OffsetBuffer<T>>
for ScalarBuffer<T>
#5203 [arrow] (mbrobbel)From<BufferBuilder<T>>
for Buffer
#5202 [arrow] (mbrobbel)From<BufferBuilder<T>>
for ScalarBuffer<T>
#5201 [arrow] (mbrobbel)BooleanArray::into_parts
method #5191 [arrow] (mbrobbel)Buffer::from_raw_parts
#5190 [arrow] (mbrobbel)LargeListBuilder
to make_builder
#5171 [arrow] (viirya)as casts
in parquet #5168 [parquet] (psvri)is_null
datatype shortcut method #5157 [arrow] (comphead)MetadataLoader
public #5137 [parquet] (AdamGS)ArrayBuilder
for Box<dyn ArrayBuilder>
#5109 [arrow] (viirya)Breaking changes:
Implemented enhancements:
RecordBatch::remove_column()
and Schema::remove_field()
#4952 [arrow]arrow_json
: support binary
deserialization #4945 [arrow]ParquetRecordWriter
proc macro in parquet_derive
, but ParquetRecordReader
is missing #4772 [parquet]Fixed bugs:
Documentation updates:
Merged pull requests:
Field::remove()
, Schema::remove()
, and RecordBatch::remove_column()
APIs #4959 [arrow] (Folyd)RecordReader
trait and proc macro to implement it for a struct #4773 [parquet] (Joseph-Rance)Breaking changes:
Null
rather than ""
) #4942 [arrow] (haohuaijin)Implemented enhancements:
Null
datatype #4901 [arrow]csv::WriterBuilder
#4735 [arrow]Fixed bugs:
None
in csv? #4939 [arrow]Utf8
for empty columns #4903 [arrow]NullArray::is_null()
returns false
incorrectly #4835 [arrow]infer_json_schema_from_seekable
#4814 [arrow]Closed issues:
Merged pull requests:
FileWriter
schema getter #4940 [arrow] (haixuanTao)SubstraitPlan
structure in arrow_flight::sql (#4932) #4933 [arrow] [arrow-flight] (amartins23)FlightDataEncoder
API, add examples #4916 [arrow] [arrow-flight] (alamb)FlightDataEncoder
to always resend batch dictionaries #4896 [arrow] [arrow-flight] (alexwilcoxson-rel)Array::is_null
#4838 [arrow] (alamb)cast_string_to_decimal
#4830 [arrow] (jonahgao)Breaking changes:
RowConverter
(#4811) #4819 [arrow] [arrow-flight] (tustvold)std::io::Error
to IoError
and add IpcError
variant #4726 [arrow] [arrow-flight] (alexandreyc)Implemented enhancements:
DynScalar
abstraction (something that makes it easy to create scalar Datum
s) #4781 [arrow]Datum
is not exported as part of arrow
(it is only exported in arrow_array
) #4780 [arrow]Scalar
is not exported as part of arrow
(it is only exported in arrow_array
) #4779 [arrow]Fixed bugs:
concat_batches
errors with “schema mismatch” error when only metadata differs #4799 [arrow]cmp
kernels with DictionaryArrays: Option::unwrap()
on a None
value' #4788 [arrow]*ListArray
in 46.0.0 #4746 [arrow]Box<dyn RecordBatchReader + Send>
#4751 [arrow] (wjones127)Closed issues:
Merged pull requests:
Scalar
#4793 [arrow] (alamb)Breaking changes:
batches_to_flight_data
forces clone #4656 [arrow]Array
Logical Nullability #4691 [parquet] [arrow] (tustvold)Implemented enhancements:
concat_batches
for 0 columns #4661 [arrow]filter_record_batch
should support filtering record batch without columns #4647 [arrow]lexicographical_partition_ranges
#4614 [arrow]DataType::List
correctly #4544 [arrow]Fixed bugs:
RowInterner::size()
much too low for high cardinality dictionary columns #4645 [arrow]List(FixedSizeBinary)
array equality check may return wrong result #4637 [arrow]arrow::compute::nullif
panics if NullArray
is provided #4634 [arrow]MutableBuffer::set_null_bits
can be bypassed #4620 [arrow]compressed_size
, file offset
. #4610 [parquet]new_null_array
generates an invalid union array #4600 [arrow]arrow::compute::concat
panics for dense union arrays with non-trivial type IDs #4578 [arrow]Closed issues:
Merged pull requests:
ArrowReaderBuilder
construction (#4674) #4676 [parquet] (tustvold)concat_batches
for 0 columns #4662 [arrow] (Dandandan)From<Vec<T>>
for BufferBuilder
and MutableBuffer
#4650 [arrow] (mbrobbel)Bucket
size in OrderPreservingInterner #4646 [arrow] (alamb)Default
,Extend
and FromIterator
for BufferBuilder
#4638 [arrow] (mbrobbel)NullArray
in nullif
#4635 [arrow] (kawadakk)BufferBuilder
to arrow-buffer
#4630 [arrow] (mbrobbel)FromIterator
for MutableBuffer
#4624 [arrow] (mbrobbel)MutableBuffer::set_null_bits
#4621 [arrow] (kawadakk)Default
for arrow_buffer::buffer::MutableBuffer
#4619 [arrow] (mbrobbel)Breaking changes:
Implemented enhancements:
BooleanBuilder
: Add validity_slice
method for accessing validity bits #4535 [arrow]FixedSizedListArray
for length
kernel #4517 [arrow]RowCoverter::convert
that targets an existing Rows
#4479 [arrow]Fixed bugs:
assertion failed: idx < self.len
when casting DictionaryArrays with nulls #4576 [arrow]FixedSizeListBuilder::new
#4549 [arrow]Merged pull requests:
is_empty
impl for arrow::array::ArrayBuilder
#4543 [arrow] (mbrobbel)FixedSizedListArray
for length
kernel #4520 [arrow] (Weijun-H)Breaking changes:
Implemented enhancements:
Date
- Date
kernel #4383 [arrow]Fixed bugs:
AsyncArrowWriter
to a file corrupts the footer for large columns #4526 [parquet]Merged pull requests:
Breaking changes:
Implemented enhancements:
Row
to Rows
#4466 [arrow]FlightSqlService
to handle custom actions and commands #4439NullBuilder
#4429 [arrow]Fixed bugs:
42.0.0
: Bad parquet column indexes for All Null Columns, resulting in Parquet error: StructArrayReader out of sync
on read #4459 [parquet]Documentation updates:
Merged pull requests:
NullBuilder
#4430 [arrow] (izveigor).5
the same as 0.5
#4425 [arrow] (alamb)Breaking changes:
Implemented enhancements:
millisecond
and microsecond
functions #4374 [arrow]CommandGetXdbcTypeInfo
responses (XdbcInfoValue
and builders) #4257 [arrow] [arrow-flight]Fixed bugs:
Closed issues:
dict_id, dict_is_ordered
into DataType? #4325Merged pull requests:
take
kernel performance on primitive arrays, fix bad null index handling (#4404) #4405 [arrow] (tustvold)BooleanBuffer::new_unset
and BooleanBuffer::new_set
and BooleanArray::new_null
constructors #4402 [arrow] (tustvold)Hash
impls for CastOptions
and FormatOptions
#4395 [arrow] (alamb)WriterProperites
and ReaderProperties
docs #4392 [parquet] (alamb)microsecond
and millisecond
kernels #4375 [arrow] (izveigor)Breaking changes:
FlightData
, FlightInfo
, FlightEndpoint
, Locaation
and Ticket
#4294 [arrow] [arrow-flight] (alamb)Implemented enhancements:
CommandGetCatalogs
, CommandGetSchemas
, and CommandGetTables
requests #4295 [arrow] [arrow-flight]FlightInfo
to make it easier to create new requests #4281 [arrow] [arrow-flight]CommandGetSqlInfo
responses (SqlInfoValue
and builders) #4256 [arrow] [arrow-flight]Intervals
#4181 [arrow]Fixed bugs:
Merged pull requests:
page_size
consistently, deprecate pagesize
in parquet WriterProperties #4313 [parquet] (alamb)CommandGetCatalogs
, CommandGetSchemas
, and CommandGetTables
requests #4296 [arrow] [arrow-flight] (roeap)ArrowWriter
memory usage: Buffer Pages in ArrowWriter instead of RecordBatch (#3871) #4280 [parquet] (tustvold)Debug
impls for ArrowWriter
and SerializedFileWriter
#4278 [parquet] (alamb)RecordBatchWriter
to arrow
crate #4277 [arrow] (alexandreyc)Intervals
#4182 [arrow] (izveigor)Breaking changes:
Implemented enhancements:
Interval
+/- Interval
#4178 [arrow]print_column_chunk_metadata()
#4172 [parquet]Fixed bugs:
Merged pull requests:
RecordBatchReader
for CSV reader #4195 [arrow] (alexandreyc)Breaking changes:
StructArray
w/ FieldRef
#4116 [parquet] [arrow] (crepererum)Implemented enhancements:
Interval + Timestamp
and Interval + Date
in addition to Timestamp + Interval
and Interval + Date
#4094 [arrow]Fixed bugs:
Documentation updates:
Closed issues:
Merged pull requests:
bitwise
shift left/right #4148 [arrow] (Weijun-H)as_primitive
#4143 [arrow] (Weijun-H)Interval
+ Timestamp
and Interval
+ Date
#4117 [arrow] (Weijun-H)Breaking changes:
Implemented enhancements:
Timestamp
+
/-
Interval
types #3963 [arrow]Fixed bugs:
Documentation updates:
Closed issues:
Merged pull requests:
Hash
trait to SortOptions. #4089 [arrow] (mustafasrepo)Breaking changes:
Implemented enhancements:
Fixed bugs:
Closed issues:
Merged pull requests:
arrow-flight
crates #4012 [arrow] [arrow-flight] (alamb)RecordBatch
by adding Serde
support to RawDecoder
(#3949) #3979 [arrow] (tustvold)Binary
/LargeBinary
and FixedSizeBinary
#3961 [arrow] (Weijun-H)Breaking changes:
Implemented enhancements:
Fixed bugs:
Merged pull requests:
ListBuilder
and GenericListBuilder
#3891 [arrow] (alamb)mul_checked
to avoid silent overflow in interval arithmetic #3886 [arrow] (Weijun-H)Breaking changes:
FlightSqlServiceClient
improvements #3788 [arrow] [arrow-flight] (crepererum)Implemented enhancements:
Fixed bugs:
Closed issues:
Merged pull requests:
Utf8
and LargeUtf8
--> Interval
#3762 [arrow] (doki23)into_primitive_dict_builder
to DictionaryArray
#3715 [arrow] (viirya)Breaking changes:
FlightClient::do_put
#3464 [arrow] [arrow-flight] (alamb)Implemented enhancements:
take_run
improvements #3701 [arrow]as_mut_any
in Array trait #3655Array
--> Display
formatter that supports more options and is configurable #3638 [parquet] [arrow]Fixed bugs:
Merged pull requests:
Ord
/PartialOrd
for SortOptions
#3723 [arrow] (crepererum)take_run
improvements #3705 [arrow] (askoa)RunArray
#3695 [arrow] (askoa)Breaking changes:
Implemented enhancements:
array_value_to_string
in arrow-csv
#3483 [arrow]Fixed bugs:
Closed issues:
32.0.0
of arrow
/arrow-flight
/parquet
/parquet-derive
(next release after 31.0.0
) #3584 [parquet] [arrow] [arrow-flight]Merged pull requests:
Field
#3630 [arrow] (chunshao90)Breaking changes:
StringArray
construction with Vec<Option<String>>
#3602 [arrow] (sinistersnare)RunEndEncodedArray
#3553 [parquet] [arrow] (askoa)Implemented enhancements:
From<Vec<Option<String>>>
impl for GenericStringArray<OffsetSize>
#3599 [arrow]Error::Source
for ArrowError and FlightError #3566 [arrow] [arrow-flight]External
variant to ParquetError #3285 [parquet]Fixed bugs:
nullif
incorrectly calculates null_count
, sometimes panics with subtraction overflow error #3579 [arrow]Closed issues:
Merged pull requests:
std::error::Error::source
for ArrowError
and FlightError
#3567 [arrow] [arrow-flight] (alamb)Breaking changes:
arrow-json
#3449 [arrow] (JayjeetAtGithub)Implemented enhancements:
'2001-01-01'
to timestamp #3492 [arrow]build_compare
#3465concat_batches
to take non owned RecordBatch #3456 [arrow]30.0.1
(maintenance release for 30.0.0
) #3455get_slice_memory_size
or similar #3407 [arrow] [arrow-flight]Fixed bugs:
Merged pull requests:
31.0.0
and add changelog #3518 [parquet] [arrow] [arrow-flight] (iajoiner)concat_batches
parameter to non owned reference #3480 [arrow] (askoa)parquet-rewrite
CLI #3477 [parquet] (crepererum)build_compare
#3466 (viirya)FlightClient::{list_flights, list_actions, do_action, get_schema}
#3463 [arrow-flight] (alamb)column_name
based index access for RecordBatch
and StructArray
#3458 [arrow] (askoa)*like_dyn
, *like_utf8_scalar_dyn
and *like_dict
functions to macros #3411 [arrow] (askoa)FlightClient
#3402 [arrow-flight] (alamb)RecordBatch
<--> FlightData
encode/decode + tests #3391 [arrow] [arrow-flight] (alamb)into_builder
for bytearray #3326 [arrow] (viirya)Implemented enhancements:
ArrowFlight
Client #3371 [arrow-flight]Fixed bugs:
nullif
kernel no longer exported #3454 [arrow]Breaking changes:
Implemented enhancements:
ParquetObjectReader
#3381 [parquet]Fixed bugs:
try_to_type
#3350Documentation updates:
Merged pull requests:
ParquetObjectReader
#3382 [parquet] (kszlim)FlightClient
#3378 [arrow-flight] (alamb)Breaking changes:
Field::new
and Field::new_with_dict
to take existing String
as well as &str
#3288 [arrow] (alamb)&Option<T>
to Option<&T>
#3249 [parquet] [arrow] (Jimexist)*_dict_scalar
kernels behind *_dyn
kernels #3202 [arrow] (viirya)Implemented enhancements:
Fixed bugs:
compile-time-rng
kills reproducible builds #3271 [parquet]nullif
panics on empty array #3261 [arrow]can_cast_types
returns true #3221 [arrow]arrow-cast
crate #3219 [arrow]Documentation updates:
Merged pull requests:
ilike_scalar
and nilike_scalar
#3303 [arrow] (askoa)arrow-cast
to arrow
#3222 [arrow] (viirya)FixedSizeBinary
in Row format #3182 [arrow] (tustvold)Breaking changes:
Field::metadata
to HashMap
#3148 [parquet] [arrow] (crepererum)Field::metadata
#3091 [parquet] [arrow] (askoa)Implemented enhancements:
Field::size
and DataType::size
#3147 [parquet] [arrow]SortingColumn
when reading and writing parquet metadata #3090 [parquet]Fixed bugs:
concat_batches
#3123 [arrow]Closed issues:
27.0.0
(next release after 26.0.0
) #3045 [parquet] [arrow] [arrow-flight]Merged pull requests:
arrow-data
to arrow-schema
crate #3177 [arrow] (mbrobbel)RowSelection::iter()
, Into<Vec<RowSelector>>
and example #3173 [parquet] (alamb){Field,DataType}::size
#3149 [arrow] (crepererum)bloom
default feature, add reader properties #3119 [parquet] (Jimexist)SortingColumn
in parquet files #3103 [parquet] (askoa)Breaking changes:
RowConverter
#3080 [arrow] (crepererum)Implemented enhancements:
TimestampNanosecondArray
with timezones #3042 [arrow]RowSelection::intersection
#3003 [parquet]26.0.0
(next release after 25.0.0
) #2953 [parquet] [arrow] [arrow-flight]Fixed bugs:
Documentation updates:
Merged pull requests:
TimestampNanosecondArray
with timezones #3088 [arrow] (src255)OwnedRow
#3079 [arrow] (crepererum)intersect_row_selections
from datafusion to arrow-rs. #3047 [parquet] (Ted-Jiang)arrow::compute::kernels::temporal
should support nanoseconds #2996 [arrow] (comphead)RowSelection::from_selectors_and_combine
to merge RowSelectors #2994 [parquet] (Ted-Jiang)lexsort_to_indices
#2991 [arrow] (alamb)byte_size
from datafusion::physical_expr #2965 [arrow] (avantgardnerio)Breaking changes:
from_timestamp
by from_timestamp_opt
#2894 [arrow] (waitingkuo)Implemented enhancements:
true
and false
values in a BooleanArray #2963 [arrow]freeze_with_dictionary
API to MutableArrayData
#2914 [arrow][+/-]hhmm
and [+/-]hh
as fixedoffset timezone format #2910 [arrow]from_timestamp
by from_timestamp_opt
#2892 [arrow]Fixed bugs:
total_cmp
found for struct f16
in the current scope #2926 [arrow]and_then
method #2925 [parquet]RawPtrBox::new
#2882 [arrow]parquet::arrow::arrow_writer::ArrowWriter
ignores page size properties #2853 [parquet]Documentation updates:
Closed issues:
25.0.0
(next release after 24.0.0
) #2820 [parquet] [arrow] [arrow-flight]Merged pull requests:
WriterProperties
(#2941) #2942 [parquet] (tustvold)chrono-tz
feature #2879 [arrow] (viirya)Breaking changes:
Implemented enhancements:
ArrowNativeTypeOp
#2753 [arrow]Fixed bugs:
parquet::arrow::arrow_writer::ArrowWriter
ignores page size properties #2853 [parquet]the trait bound for<'de> arrow::datatypes::Schema: serde::de::Deserialize<'de> is not satisfied
#2825 [arrow]Closed issues:
Merged pull requests:
snap
option to README #2847 [parquet] (exyi)interleave
kernel (#1523) #2838 [arrow] (tustvold)ArrowNativeTypeOp
#2756 [arrow] (HaoYang670)Breaking changes:
ArrowNativeType
(#1918) #2793 [parquet] [arrow] (tustvold)ArrowNativeType::FromStr
#2775 [arrow] (tustvold)arrow-array
crate (#2594) #2769 [arrow] (tustvold)dyn_arith_dict
feature flag #2760 [arrow] (tustvold)arrow-data
into a separate crate #2746 [arrow] (tustvold)Implemented enhancements:
binary
function should not panic on unequaled array length. #2721 [arrow]Fixed bugs:
try_unary_dict
should check value type of dictionary array #2754 [arrow]Closed issues:
23.0.0
(next release after 22.0.0
) #2665 [parquet] [arrow] [arrow-flight]Merged pull requests:
verify_release_candidate.sh
for new arrow subcrates #2752 (alamb)Breaking changes:
Implemented enhancements:
RecordBatchOptions
#2728 [arrow]CHANGELOG-old.md
#2726DivideByZero
error for float modulus #2720 [arrow]try_binary
should not panic on unequaled array length. #2715 [arrow]DataType
function to detect nested types #2704 [arrow]RecordBatch::project
#2690 [arrow]min_max_helper
#2657 [arrow]downcast_primitive_array
in arithmetic kernels #2639 [arrow]Fixed bugs:
DivideByZero
in the Divide
kernel #2719 [arrow]arrow
crate does not build with features = ["ffi"]
and default_features = false
. #2670 [arrow]RowSelector
having row_count
of 0 #2669 [parquet]crate::array::layout
#2659 [arrow]CastOptions
#2648 [arrow]flight.proto
and fix schema to SchemaResult #2571 [arrow] [arrow-flight]SchemaResult
in IPC deviates from other implementations #2445 [arrow] [arrow-flight]Closed issues:
Merged pull requests:
CHANGELOG-old.md
#2732 (iajoiner)try_binary
and checked_ops
, and remove math_checked_op
#2717 [arrow] (HaoYang670)numeric_cast_with_error
#2661 [arrow] (liukun4515)test_fuzz_async_reader_selection
#2656 [parquet] (thinkharderdev)CastOption
for casting numeric #2649 [arrow] (liukun4515)downcast_primitive_array
in arithmetic kernels #2640 [arrow] (viirya)Breaking changes:
total_cmp
for floating value ordering and remove nan_ordering
feature flag #2614 [arrow] (viirya)dyn_cmp_dict
#2597 [arrow] (tustvold)Implemented enhancements:
total_cmp
for floating value ordering and remove nan_ordering
feature flag #2613 [arrow]dyn_cmp_dict
feature flag to gate dyn comparison of dictionary arrays #2596 [arrow]do_get()
#2581 [arrow-flight]eq_dyn
, neq_dyn
, lt_dyn
, lt_eq_dyn
, gt_dyn
, gt_eq_dyn
#2569 [arrow]sum
for dictionary array #2565 [arrow]lt_dyn
, lt_eq_dyn
, gt_dyn
, gt_eq_dyn
#2538 [arrow]eq_dyn
and neq_dyn
#2535 [arrow]like_utf8_scalar
for %pat%
#2519 [arrow]Fixed bugs:
ParquetFileArrowReader::get_record_reader[_by_column]
batch_size
overallocates #2321 [parquet]Documentation updates:
Closed issues:
Interval(DayTime)
to Timestamp(Nanosecond, None)
#2606 [arrow]length
field for Buffer
#2524 [arrow]const_generic
. #2390 [arrow]const_generic
#2384 [arrow]Merged pull requests:
length
field for Buffer and use more Buffer
in IPC reader to avoid memory copy. #2557 [arrow] [arrow-flight] (HaoYang670)Decimal
) when creating array in IPC reader #2547 [arrow] (HaoYang670)eq_dyn
and neq_dyn
#2533 [arrow] (viirya)create_array
in IPC reader. #2525 [arrow] (HaoYang670)%pat%
(>3x speedup) #2521 [arrow] (Dandandan)Breaking changes:
ColumnCloseResult
(#2465) #2466 [parquet] (tustvold)ChunkReader
into SerializedPageReader
(#2463) #2464 [parquet] (tustvold)Implemented enhancements:
ChunkReader
into SerializedPageReader
#2463 [parquet]RowFilter
withinParquetRecordBatchReader
#2431 [parquet]StructBuilder::len
#2429 [arrow]ParquetRecordBatchReaderBuilder
) #2427 [parquet]OffsetIndex
to Prune IO in ParquetRecordBatchStream
#2426 [parquet]peek_next_page
and skip_next_page
in InMemoryPageReader
#2406 [parquet]Utf8
/LargeUtf8
to Binary
/LargeBinary
#2402 [arrow]Decimal128
and Decimal256
arrays #2375 [arrow]skip_records
#2358 [parquet]read_buffer
Arrow IPC API to return Result<_>
#2342 [arrow]skip_records
in GenericColumnReader
to skip across row groups #2331 [parquet]Decimal256
#2320 [arrow]DeltaBitPackDecoder
#2281 [parquet]ParquetRecordBatchStream
to support row filtering in DataFusion #2270 [parquet]ArrayReader::skip_records
API #2197 [parquet]Fixed bugs:
DoPutUpdateResult
#2403 [arrow-flight]AsyncFileReader
No Longer Object-Safe #2372 [parquet]Closed issues:
Merged pull requests:
MapArrayReader
(#2484) (#1699) (#1561) #2500 [parquet] (tustvold)split_second
function #2436 [arrow] (viirya)Decimal
and DecimalArray
using const_generic
#2383 [parquet] [arrow] (HaoYang670)Decimal128
and Decimal256
arrays #2376 [arrow] (viirya)Box<dyn AsyncFileReader>
#2368 [parquet] (tustvold)Decimal256
validation based on bytes comparison and add benchmark test #2360 [parquet] [arrow] (liukun4515)field.rs
and add more tests #2345 [arrow] (HaoYang670)Breaking changes:
GenericBinaryArray
and GenericListArray
: add PREFIX
and data type constructor #2327 [parquet] [arrow] (HaoYang670)safe
(#2302) #2303 [arrow] (tustvold)test_utils
from default features (#2298) #2299 [arrow] (tustvold)DataType::Decimal
to DataType::Decimal128
#2229 [parquet] [arrow] (viirya)Decimal128Iter
and Decimal256Iter
and do maximum precision/scale check #2140 [arrow] (viirya)Implemented enhancements:
ListArray
#2311 [arrow]FlightSqlService
trait to pass session info along #2308 [arrow-flight]take_bits
for non-null indices #2306 [arrow]ffi
#2302 [arrow]ffi::ArrowArray::try_new
is safe #2301 [arrow]JsonEqual
trait #2296 [arrow]with_precision_and_scale
to Decimal
array traits #2291 [arrow]ArrayReader::next_batch
with ArrayReader::read_records
and ArrayReader::consume_batch
#2236 [parquet]DataType::Decimal
to DataType::Decimal128
#2228 [arrow]append_option
support to Decimal128Builder
and Decimal256Builder
#2224 [arrow]FixedSizeBinaryArray
and FixedSizeListArray
from array_binary.rs
and array_list.rs
#2217 [arrow]Box
Values in PrimitiveDictionaryBuilder
#2215 [arrow]Hash
for Schema
#2182 [arrow]GenericStringBuilder
should use GenericBinaryBuilder
#2156 [arrow]Decimal128
and Decimal256
#2139 [arrow]ArrayAccessor
in Decimal128Iter
and Decimal256Iter
#2138 [arrow]ArrayAccessor
and FromIterator
in Cast Kernels #2137 [arrow]TypedDictionaryArray
for more ergonomic interaction with DictionaryArray
#2136 [arrow]ArrayAccessor
in Comparison Kernels #2135 [arrow]peek_next_page()
and skip_next_page
in InMemoryColumnChunkReader
#2129 [parquet]Decimal256
#2112 [arrow]skip_def_levels
for ColumnLevelDecoder
#2107 [parquet]Time32
/ Time64
#2053 [arrow]Fixed bugs:
Decimal128
or Decimal256
#2256 [arrow]IntervalMonthDayNanoType::make_value()
does not match C implementation #2234 [arrow]FlightSqlService
trait does not allow impl
s to do handshake #2210 [arrow-flight]EnabledStatistics::None
not working #2185 [parquet]Documentation updates:
Performance improvements:
Closed issues:
Merged pull requests:
Request<FlightDescriptor>
to FlightSqlService
impl
s #2309 [parquet] [arrow-flight] (avantgardnerio)with_precision_and_scale
to BasicDecimalArray
trait #2292 [parquet] [arrow] (viirya)fn get_data_type
by const DATA_TYPE
in BinaryArray and StringArray #2289 [arrow] (HaoYang670)from_list
function. #2277 [arrow] (HaoYang670)like_utf8
/ nlike_utf8
kernels #2258 [arrow] (daniel-martinez-maqueda-sap)FixedSizeList
to array_fixed_size_list.rs
#2250 [arrow] (HaoYang670)Schema::fields
and Schema::metadata
pub
(public) #2239 [arrow] (alamb)IntervalMonthDayNanoType::make_value()
to conform to specifications #2235 [arrow] (avantgardnerio)arrow
CI on changes to arrow #2227 (alamb)FixedSizeBinaryArray
to array_fixed_size_binary.rs
#2218 [arrow] (HaoYang670)FlightSqlService
trait to proxy handshake #2211 [arrow-flight] (avantgardnerio)serde_json
feature name #2209 [parquet] (flisky)Hash
for Schema
#2183 [arrow] (crepererum)StringBuilder
use BinaryBuilder
#2181 [arrow] (HaoYang670)BYTE_ARRAY
type #2160 [parquet] (liukun4515)peek\_next\_page
and skip\_next\_page
for `InMemoryColumnCh… #2155 [parquet] (thinkharderdev)arrow
changes #2152 (alamb)object_store
code from object_store_rs to arrow-rs #2081 (alamb)validate_utf8
performance #2048 [arrow] (tfeda)Breaking changes:
DecimalArray``/DecimalBuilder
to Decimal128Array
/Decimal128Builder
#2101 [arrow]append
methods to be infallible where possible #2103 [parquet] [arrow] (jhorstmann)UnionArray::child
(#2035) #2099 [arrow] (tustvold)preserve_order
feature from serde_json
dependency (#2095) #2098 [parquet] [arrow] (tustvold)weekday
and weekday0
kernels to to num_days_from_monday
and num_days_since_sunday
#2066 [arrow] (alamb)null_count
from write_batch_with_statistics
#2047 [parquet] (tustvold)Implemented enhancements:
total_cmp
from std #2130 [arrow]ParquetRecordBatchStream
#2110 [parquet]GenericBinaryBuilder
should use buffer builders directly. #2104 [arrow]generate_decimal256_case
arrow integration test #2093 [arrow]weekday
and weekday0
kernels to to num_days_from_monday
and days_since_sunday
#2065 [arrow]filter_dict
#2062 [arrow]set_bits
#2060 [arrow]BooleanBuilder
#2058 [arrow]BooleanArray::from_iter
should omit validity buffer if all values are valid #2055 [arrow]DICTIONARY_ORDERED
flag if a field's dictionary is ordered #2049 [arrow]peek_next_page()
and skip_next_page
in SerializedPageReader
#2043 [parquet]MapType
#2037 [arrow]DecimalArrayBuilder
should use FixedSizedBinaryBuilder
#2026 [arrow]serialized_reader
read specific Page by passing row ranges. #1976 [parquet]Fixed bugs:
type_id
and value_offset
are incorrect for sliced UnionArray
#2086 [arrow]take
kernel does not handle null indices correctly #2057 [arrow]write_batch_with_statistics
#2046 [parquet]WriterProperties
#2014 [parquet]Documentation updates:
Closed issues:
serde_json
specify the preserve_order
feature in arrow
package #2095 [arrow]skip_values
in DictionaryDecoder #2079 [parquet]skip_values
in ByteArrayColumnValueDecoder
#2072 [parquet]Builder::append
methods returning results even though they are infallible #2071UnionArray::child
#2035Merged pull requests:
total_cmp
from std #2131 [arrow] (Dandandan)match
arms have incompatible types #2121 (alamb)GenericBinaryBuilder
to use buffer builders directly. #2117 [arrow] (HaoYang670)DataType::Decimal256
#2094 [parquet] [arrow] (viirya)DecimalBuilder
should use FixedSizeBinaryBuilder
#2092 [arrow] (HaoYang670)FixedSizeBinaryArray
to DecimalArray
#2041 [arrow] (HaoYang670)Breaking changes:
EnabledStatistics
to control level of statistics generated #2022 [parquet] (tustvold)Decimal256Builder
and Decimal256Array
; Decimal arrays now implement BasicDecimalArray
trait #2000 [parquet] [arrow] (viirya)ColumnReader::read_batch
#1995 [parquet] [arrow] (tustvold)PrimitiveBuilder::finish_dict
(#1978) #1980 [arrow] (tustvold)NullType
#1942 [arrow] (liukun4515)Implemented enhancements:
DataType::Dictionary
support to subtract_scalar
, multiply_scalar
, divide_scalar
#2019 [arrow]add_scalar
kernel #2017 [arrow]FixedSizeBinaryBuilder
#2007 [arrow]Decimal256Builder
and Decimal256Array
#1999 [arrow]DictionaryArray
in unary
kernel #1989 [arrow]Array
s #1987 [arrow]DictionaryArray
in divide
kernel #1982 [arrow]Into<ArrayData>
for T: Array
#1979 [arrow]DictionaryArray
in multiply
kernel #1972 [arrow]DictionaryArray
in subtract
kernel #1970 [arrow]DecimalArray::length
as a constant #1967 [arrow]DictionaryArray
in add
kernel #1950 [arrow]Field
#1934 [arrow]StringDictionaryBuilder
faster #1851 [arrow]concat_elements_utf8
should accept arbitrary number of input arrays #1748 [arrow]Fixed bugs:
ColumnWriterImpl::write_batch_with_statistics
incorrect distinct count in statistics #2016 [parquet]ColumnWriterImpl::write_batch_with_statistics
can write incorrect page statistics #2015 [parquet]RowFormatter
is not part of the public api #2008 [parquet]ColumnReader::read_batch
For Corrupted Files #1997 [parquet]PrimitiveBuilder::finish_dict
does not validate dictionary offsets #1978 [arrow]n_buffers
in FFI_ArrowArray
#1959 [arrow]DecimalArray::from_fixed_size_list_array
fails when offset > 0
#1958 [arrow]Send
+ Sync
impl for Allocation
may not be sound unless Allocation
is Send
+ Sync
as well #1944 [arrow]NullType
#1923 [arrow]Documentation updates:
Closed issues:
min_bytes
and max_bytes
return wrong size #2021 [parquet]Decimal
s by using constant generic. #2001DecimalArray
to a new file #1985 [arrow]DictionaryArray
in multiply
kernel #1974null_count
of DictionaryArray #1962 [arrow]TIMESTAMP
in schema #1920 [parquet]Merged pull requests:
Field
: with_name
, with_data_type
and with_nullable
#2024 [arrow] (alamb)FixedSizeBinaryBuilder
#2013 [arrow] (HaoYang670)Into<ArrayData>
for T: Array
#1992 [parquet] [arrow] (heyrutvik)FixedSizeListBuilder
#1988 [arrow] (HaoYang670)DecimalArray
to array_decimal.rs #1986 [arrow] (HaoYang670)const
#1968 [arrow] (HaoYang670)from_fixed_size_list
when offset > 0 #1964 [arrow] (HaoYang670)FixedSizeListArray::value_length
#1957 [arrow] (HaoYang670)ColumnMetadata
after the column chunk data, not the ColumnChunk
#1947 [parquet] (liukun4515)Breaking changes:
RecordBatch
for non-nullable fields containing null values #1890 [arrow] (andygrove)ArrayData::validate_dict_offsets
to ArrayData::validate_values
#1889 [arrow] (frolovdev)Decimal128
API and use it in DecimalArray and DecimalBuilder #1871 [parquet] [arrow] (viirya)safe
(#996) (#1027) #1866 [parquet] [arrow] (tustvold)Implemented enhancements:
ArrowWriter
being used with a cursor #1927 [parquet]cast
to/from NULL
and DataType::Decimal
#1921 [arrow]Decimal256
API #1913 [arrow]DictionaryArray::key
function #1911 [arrow]ListArrays
in MutableArrayData
#1884 [arrow]PrimitiveArray::from_iter
should omit validity buffer if all values are valid #1856 [arrow]from(v: Vec<Option<&[u8]>>)
and from(v: Vec<&[u8]>)
for FixedSizedBInaryArray
#1852 [arrow]Vec
-inspired APIs to BufferBuilder
#1850 [arrow]nilike
support in comparison
#1845 [arrow]arrow::array::builder
module #1843 [arrow]quarter
support in temporal
kernels #1835 [arrow]ArrayData::validate_dictionary_offset
to ArrayData::validate_values
#1812 [arrow]substring
kernel #1801 [arrow]substring_by_char
kernel #1800 [arrow]Fixed bugs:
dynamic_types
example does not print the projection #1902 [arrow]log2(0)
panicked at 'attempt to subtract with overflow', parquet/src/util/bit_util.rs:148:5
#1901 [parquet]combine_option_bitmap
needs to use bit slices #1899 [arrow]RecordBatch
with null values in non-nullable fields does not cause an error #1888 [arrow]regex
dependency #1874 [arrow]Documentation updates:
DictionaryArray::lookup_key
#1910 [arrow] (alamb)Closed issues:
Merged pull requests:
arrow::array::builder
module (#1843) #1928 [arrow] (tustvold)checked_add/sub().unwrap()
with +/-
#1924 [arrow] (HaoYang670)NULL
to/from Decimal
#1922 [arrow] (liukun4515)Decimal256
API #1914 [arrow] (viirya)DictionaryArray::key
function #1912 [arrow] (alamb)bit_util
of Parquet. #1905 [parquet] (HaoYang670)weekday
#1894 [arrow] (alamb)MutableArrayData
#1885 [arrow] (jhorstmann)regex
dependency to 1.5.6 #1875 [arrow] (martin-g)BitSliceIterator
and BitIndexIterator
(#1864) #1865 [arrow] (tustvold)from
methods for FixedSizeBinaryArray
#1854 [arrow] (HaoYang670)substring
kernel. #1853 [arrow] (HaoYang670)nilike
support in comparison
#1846 [arrow] (MazterQyou)test_command_help
#1844 [parquet] (viirya)quarter
support in temporal
#1836 [arrow] (MazterQyou)substring_by_char
by about 2.5x #1832 [arrow] (HaoYang670)Breaking changes:
ArrowNativeType
and OffsetSizeTrait
for safety (#1028) #1819 [arrow] (tustvold)csv::infer_file_schema
by removing redundant ref #1776 [arrow] (tustvold)Implemented enhancements:
ListArray
#1817 [arrow]UnionArray
in V4 IPC message #1793 [arrow]combine_option_bitmap
should accept arbitrary number of input arrays. #1780 [arrow]substring_by_char
kernels for slicing on character boundaries #1768 [arrow]PageIndex
from column metadata #1761 [parquet]DataType::Utf8
to DataType::Boolean
#1740 [arrow]FileWriter
. #1691 [parquet]stdout
#1687 [parquet]Fixed bugs:
flight_data_to_arrow_batch
does not support RecordBatch
es with no columns #1783 [arrow-flight]features=["zstd"]
#1630 [parquet]Documentation updates:
CHANGELOG-old.md
#1759 (alamb)Closed issues:
DataType::Decimal
Non-Compliant? #1779 [arrow]Parquet
is a part of Arrow
? #1715 [parquet] [arrow]Merged pull requests:
PageIterator
Documentation #1831 [parquet] (Ted-Jiang)resolver v2
, test more feature flag combinations in CI, fix errors (#1630) #1822 [parquet] [arrow] (tustvold)arrow/benches/string_kernels.rs
to arrow/benches/substring_kernels.rs
#1805 [arrow] (HaoYang670)Substring_by_char
#1784 [arrow] (HaoYang670)ParquetFileArrowReader::try_new
#1782 [parquet] (tustvold)ChunkReader
for Bytes
, deprecate SliceableCursor
#1775 [parquet] (tustvold)validate_decimal_precision
check in DecimalBuilder.append_value
for interop test #1767 [arrow] (viirya)Utf8
to Boolean
#1738 [arrow] (MazterQyou)Breaking changes:
ArrayDataBuilder::null_bit_buffer
to accept Option<Buffer>
rather than Buffer
#1739 [arrow] (HaoYang670)null_count
from ArrayData::try_new()
#1721 [arrow] (HaoYang670)std:io::Write
rather custom ParquetWriter
trait (#1717) (#1163) #1719 [parquet] (tustvold)ProjectionMask
(#1701) #1716 [parquet] (tustvold)Implemented enhancements:
string
kernel to concatenate_elements
#1747 [arrow]ArrayDataBuilder::null_bit_buffer
should accept Option<Buffer>
as input type #1737 [arrow]BinaryArray
#1724 [arrow]generate_unions_case
integration test #1712 [arrow]leaf_columns
argument to ArrowReader::get_record_reader_by_columns
#1653 [parquet]string_concat
kernel #1540 [arrow]Fixed bugs:
ArrayData::try_new
cannot always return expected error. #1707 [arrow]substring
kernel): The null buffer is not aligned when offset != 0
#1639 [arrow]Documentation updates:
Merged pull requests:
string_concat
to concat_elements_utf8
#1754 [arrow] (alamb)string
kernel to concat_elements
. #1752 [arrow] (HaoYang670)cargo install
installs not globally #1732 [parquet] (kazuk)min_binary
and max_binary
aggregate kernels #1725 [arrow] (HaoYang670)null_bit_buffer
in ArrayData::try_new()
#1714 [arrow] (HaoYang670)generate_unions_case
integration test #1713 [arrow] (viirya)offset
in substring
kernel. #1704 [arrow] (HaoYang670)OffsetSizeTrait
to extend num::Integer
#1702 [arrow] (HaoYang670)Breaking changes:
bytes
in parquet rather than custom Buffer implementation (#1474) #1683 [parquet] (tustvold)OffsetSize::fn is_large
to const OffsetSize::IS_LARGE
#1664 [parquet] [arrow] (HaoYang670)StringOffsetTrait
and BinaryOffsetTrait
#1645 [arrow] (HaoYang670)generate_nested_dictionary_case
integration test failure #1636 [arrow] [arrow-flight] (viirya)Implemented enhancements:
DataType::Duration
in ffi interface #1688 [arrow]generate_unions_case
integration test #1676 [arrow]DictionaryArray
support for bit_length
kernel #1673 [arrow]DictionaryArray
support for length
kernel #1672 [arrow]Field
#1646 [arrow]StringOffsetTrait
and BinaryOffsetTrait
#1644 [arrow]UnionArray::from(data: ArrayData)
#1643 [arrow]pub fn offsets_buffer
, pub fn types_ids_buffer
and pub fn data_buffer
for ArrayDataBuilder
#1640 [arrow]generate_nested_dictionary_case
integration test failure for Rust cases #1635 [arrow]ArrowWriter
row group flush in public API #1626 [parquet]substring
support for FixedSizeBinaryArray
#1618 [arrow]UnionArray
s #1594 [arrow]length
kernel #1489 [arrow]DecimalType
support in new_null_array
#1659 [arrow] (yjshen)Fixed bugs:
UnionArray::is_null
incorrect #1625 [arrow]arrow::async_reader
#1617 [parquet]Documentation updates:
Closed issues:
OffsetSizeTrait::IS_LARGE
as a const value #1658OffsetSizeTrait
s? #1638UnionArray
Changes DataType #1595Merged pull requests:
GenericListArray::get_type
#1650 [arrow] (HaoYang670)UnionArray
s #1648 [arrow] (tfeda)dict_id
and dict_is_ordered
from equality comparison of Field
#1647 [arrow] (viirya)substring
support for FixedSizeBinaryArray
#1633 [arrow] (HaoYang670)async
into doc features #1349 [parquet] (HaoYang670)Breaking changes:
parquet::basic::LogicalType
to be more idomatic #1612 [parquet] (tfeda)ArrayData
, UnionArray
, and MapArray
#1589 [arrow] (tustvold)&Option<T>
with Option<&T>
in several arrow
and parquet
APIs #1571 [parquet] [arrow] (tfeda)Implemented enhancements:
BinaryArray
in substring
kernel #1593 [arrow]Clone
for JSON DecoderOptions
#1580 [arrow]substring
kernel #1575 [arrow]DataType::Null
in cast
kernel #1572 [arrow] (WinkerDu)Fixed bugs:
force_validate
is active) #1587 [parquet]ipc::reader::tests::projection_should_work
fails validation #1548 [arrow]Documentation updates:
Closed issues:
&Option<T>
with Option<&T>
in some APIs #1556 [parquet] [arrow]parquet::basic::LogicalType
#1554 [parquet]substring
function as unsafe
and rename it. #1541 [arrow]Merged pull requests:
div_ceil
#1629 [arrow] (HaoYang670)substring
support for binary #1608 [arrow] (HaoYang670)ParquetTypeConverter
#1592 [parquet] (viirya)Clone
and PartialEq
for json DecoderOptions
#1581 [arrow] (alamb)substring
#1577 [arrow] (HaoYang670)Option<T>
rather than Option<&T>
for copy types in substring kernel #1576 [arrow] (tustvold)projection_should_work
#1573 [arrow] (viirya)Breaking changes:
ArrowReaderOptions
to ParquetFileArrowReader
, add option to skip decoding arrow metadata from parquet (#1459) #1558 [parquet] (tustvold)RecordBatch
with zero columns but non zero row count, add field to RecordBatchOptions
(#1536) #1552 [arrow] (tustvold)DecoderOptions
#1539 [arrow] (alamb)prost
, prost-derive
and prost-types
to 0.10, tonic
, and tonic-build
to 0.7
#1510 [arrow-flight] (alamb)DecoderOptions
and support custom format_string
for each field #1451 [arrow] (sum12)Implemented enhancements:
FixedSizeBinary
in the Arrow C data interface #1553 [arrow]ParquetRecordBatchReader
#1537 [parquet]RecordBatch
with zero columns but non zero row count #1536 [arrow]Date32
/Date64
<--> String
/LargeString
in cast
kernel #1535 [arrow]Vec
or String
#1516 [arrow]substring
kernel #1511 [arrow]Fixed bugs:
filter
produces invalid sparse UnionArray
s #1547 [arrow]GenericListBuilder
is not exposed. #1518 [arrow]substring
kernel panics when chars > U+0x007F #1478 [arrow]Documentation updates:
substring
kernel #1529 [arrow] (HaoYang670)GenericListBuilder
#1525 [arrow] (comath)take
kernel documentation #1524 [arrow] (alamb)Closed issues:
min_max_helper
#1400Merged pull requests:
into_buffers
for UnionArray #1567 [arrow] (viirya)MapArray
into separate module (#1483) #1562 [parquet] (tustvold)ParquetRecordBatchReader
#1560 [parquet] (tustvold)StreamReader
#1550 [arrow] (dispanser)cast
kernel #1534 [arrow] (yjshen)remove-old-releases.sh
executable #1522 (alamb)sort
kernel #1519 [arrow] (HaoYang670)substring
kernel by about 2x #1512 [arrow] (HaoYang670)new_from_strings
to create MapArrays
#1507 [arrow] (viirya)Implemented enhancements:
size_hint
and ExactSizedIterator
for DecimalArray #1505 [arrow]StringArray
#1493 [arrow]length
kernel support for ListArray
#1470 [arrow]BinaryArray
s #1464 [arrow]DictionaryArray::try_new()
#1313 [arrow]Fixed bugs:
Documentation updates:
BinaryArray
and LargeBinaryArray
#1455 [arrow]GenericBinaryArray::take_iter_unchecked
#1454 [arrow]BinaryOffsetSizeTrait
#1453 [arrow]FixedSizeBinaryArray
is confusing. #1452 [arrow]BinaryArray
and LargeBinaryArray
#1471 [arrow] (HaoYang670)Closed issues:
packed_simd
v.s. portable_simd
, which should be used? #1492Merged pull requests:
size_hint
and ExactSizedIterator
for DecimalArray
#1506 [arrow] (alamb)StringArray::num_chars
for calculating number of characters #1503 [arrow] (HaoYang670)try_from_trusted_len_iter
#1497 [arrow] (jhorstmann)length
kernel support for List Array #1488 [arrow] (HaoYang670)Decimal
data type #1487 [arrow] (yjshen)length
kernel on Binary Array #1465 [arrow] (HaoYang670)Breaking changes:
filter_row_groups
with ReadOptions
in parquet SerializedFileReader #1389 [parquet] (yjshen)IPC Reader
file / streams #1339 [arrow] [arrow-flight] (Dandandan)Implemented enhancements:
ListArray
and LargeListArray
more readable #1433if
and abs
in shift()
#1427FixedSizeBinaryArray
#1419shift
#1408UnionArray
s #1394 [arrow]FixedSizeListArrays
s #1393 [arrow]rust-version
to 1.59 #1377Fixed bugs:
GenericListArray
is confusing #1424Documentation updates:
GenericListArray
#1450 [arrow] (HaoYang670)Merged pull requests:
ListArray
and LargeListArray
#1447 [arrow] (HaoYang670)filter
kernel should work with FixedSizeListArrays #1434 [arrow] (viirya)FixedSizeListArray
#1426 [arrow] (HaoYang670)default-features = false
for zstd
in the parquet crate to support wasm32-unknown-unknown
#1414 [parquet] (kylebarron)UnionArray
infilter
kernel #1412 [arrow] (viirya)shift
#1409 [arrow] (HaoYang670)FixedSizeBinaryArray
input all none #1405 [arrow] (jackwener)Breaking changes:
ColumnChunkMetaData
#1346 [parquet] (shanisolomon)has_
methods in ColumnChunkMetaData
#1345 [parquet] (shanisolomon)Implemented enhancements:
IpcWriteOptions
#1381 [arrow]MapArray
in filter
kernel #1378 [arrow]week
temporal kernel #1375 [arrow]compare_dict_op
#1371 [arrow]arrow::array::builder::MapBuilder
public #1354 [arrow]StructArray::from
#1351 [arrow]RecordBatch::validate_new_batch
#1350 [arrow]write
method to JsonWriter #1340 [arrow]Bitmap::new
#1337 [arrow]compare_dict_op
#1329 [arrow]as_decimal_array(arr: &dyn Array) -> &DecimalArray
#1312 [arrow]eq_dyn
, neq_dyn
, lt_dyn
, lt_eq_dyn
, gt_dyn
, gt_eq_dyn
#1201 [arrow]Fixed bugs:
cargo clippy
fails on the master
branch #1362 [arrow]ArrowArray::try_from_raw
should not assume the pointers are from Arc #1333 [arrow]with_datetime_format()
to CSV WriterBuilder #1272 [arrow]Performance improvements:
Closed issues:
ColumnChunkMetadata
#1332Merged pull requests:
day
and month
in temporal.rs #1388 [arrow] (Ted-Jiang)Clone
for IpcWriteOptions
#1382 [arrow] (matthewmturner)week
in temporal.rs #1376 [arrow] (Ted-Jiang)min_max_string
#1374 [arrow] (HaoYang670)take_iter_unchecked
#1372 [arrow] (viirya)Option
) #1367 [arrow] (viirya)RecordBatch::validate_new_batch
#1361 [arrow] (HaoYang670)StructArray::from
#1360 [arrow] (HaoYang670)as_decimal_array
function #1356 [arrow] (liukun4515)Bitmap::new
#1343 [arrow] (HaoYang670)ArrowArray::try_from_raw
should not assume the pointers are from Arc #1334 [arrow] (viirya)compare_dict_op
#1330 [arrow] (viirya)Implemented enhancements:
DeltaBitPackDecoder
#1281 [parquet]ColumnChunkMetadata
#1322 [parquet] (shanisolomon)ColumnChunkMetadata
#1318 [parquet] (shanisolomon)ColumnChunkMetadata
#1309 [parquet] (shanisolomon)DictionaryArray::try_new()
to create dictionaries from pre existing arrays #1300 [arrow] (alamb)DictionaryArray::keys_iter
, and take_iter
for other array types #1296 [arrow] (viirya)rle
decoder public under experimental
feature #1271 [parquet] (zeevm)DictionaryArray
support in eq_dyn
kernel #1263 [arrow] (viirya)Fixed bugs:
len
is not a parameter of MutableArrayData::extend
#1316data_type
is private in Rust Parquet 8.0.0 #1302 [parquet]Documentation updates:
Performance improvements:
compute
module (up to 10x faster) #1248 [parquet] [arrow] (tustvold)Closed issues:
DictionaryArrays
from Key
and Value
arrays #1299DictionaryArray
#1295 [arrow]BitReader
#1282Merged pull requests:
MutableArrayData::extend
#1336 [arrow] (HaoYang670)dead_code
#1324 [arrow] (gsserge)to_thrift
#1320 [parquet] (shanisolomon)clippy::type_complexity
#1310 [arrow] (gsserge)clippy::float_equality_without_abs
#1305 [arrow] (gsserge)clippy::vec_init_then_push
lint #1303 [arrow] (gsserge)LevelEncoder
variants to conform to Rust standards #1273 [parquet] (HaoYang670)DecimalArray
API ergonomics: add iter()
, FromIterator
, with_precision_and_scale
#1223 [arrow] (alamb)Breaking changes:
Send
+ Sync
to DataType
, RowGroupReader
, FileReader
, ChunkReader
. #1264Bitmap::len
to Bitmap::bit_len
to clarify its meaning #1242 [parquet] [arrow] (HaoYang670)memory-check
feature #1222 [arrow] (jhorstmann)RecordBatches
before writing a parquet row group in ArrowWriter
#1214 [parquet] [arrow] (tustvold)Implemented enhancements:
async
arrow parquet reader #1154 [parquet] [arrow] (tustvold)Bitmap::len
to Bitmap::bit_len
#1233eq_dyn
etc. #1202 [arrow]BinaryArray
#1108impl ArrowNativeType for i128
#1098Copy
trait bound from dyn scalar kernels #1243 [arrow] (matthewmturner)into_inner
for IPC FileWriter
#1236 [arrow] (yjshen)array::builder::make_builder
to make it available for downstream #1235 [arrow] (yjshen)Fixed bugs:
Unknown configuration option rust-version
when running the rust format command #1240Bitmap
Length Validation is Incorrect #1231 [arrow]ListArray
or MapArray
ignore offsets #1226 [parquet]memory-tracking
crate feature #1171parquet::data_type
and parquet::arrow::schema
experimental #1244 [parquet] (tustvold)Documentation updates:
Performance improvements:
simd
feature enabled (except for division/modulo) #1221 [arrow] (jhorstmann)Closed issues:
UnalignedBitChunkIterator
to that iterates through already aligned u64
blocks #1227ArrowArrayReader
in parquet #1197 [parquet]Merged pull requests:
Breaking changes:
ArrowSignedNumericType
to Simplify and reduce code duplication in arithmetic kernels #1161 [arrow] (jhorstmann)Implemented enhancements:
arbitrary_precision
feature flag #1174 [arrow]Err
from JSON writer rather than panic!
for unsupported types #1157 [arrow]scalar
mathematics kernels for Array
and scalar value #1153 [arrow]DecimalArray
in sort kernel #1137from_iter_values
for binary array #1188 [arrow] (Jimexist)MapArray
in json writer #1149 [arrow] (helgikrs)Fixed bugs:
RecordBatch
containing Float16
triggers a panic #1193 [arrow]GenericStringArray::from_iter_values
if reported iterator upper bound is incorrect #1144 [arrow]simd
feature asserts #1136 [arrow]Security fixes:
Documentation updates:
from_trusted_len_iter
#1148 [arrow] (alamb)Performance improvements:
Closed issues:
Merged pull requests:
Float16Array
#1194 [arrow] (helgikrs)sort
and take
kernels #1172 [arrow] (liukun4515)GenericStringArray::from_vec
and GenericStringArray::from_opt_vec
#1147 [arrow] (alamb)Breaking changes:
pretty_format_batches
now returns Result<impl Display>
rather than String
: #975MutableBuffer::typed_data_mut
is marked unsafe
: #1029UnionMode
, UnionArray::new()
marked unsafe
: #885New Features:
Float16Array
types #888UnionArray
#654eq_dyn_scalar
), including DictionaryArray
: #1113Enhancements:
Schema::with_metadata
and Field::with_metadata
#1092Array
for ArrayRef
for easier use #1129FixedSizeBinaryArray
#1097pyo3
, parquet-format
, prost
, tonic
lexicographical_partition_ranges
#998Fixed bugs:
take
kernel (#1046) (#1057)rc
feature of serde (#990) (#1016)Changes:
Features / Fixes:
Features / Fixes:
Other:
Breaking changes:
ArrayData::new()
with ArrayData::try_new()
and unsafe ArrayData::new_unchecked
#822 [parquet] [arrow] (alamb)Implemented enhancements:
ArrowWriter
#788Fixed bugs:
minute
and second
temporal kernels do not respect timezone #500Documentation updates:
Implemented enhancements:
Fixed bugs:
RowColumnIter
#762Implemented enhancements:
append_nulls
and append_trusted_len_iter
for PrimitiveBuilder #725Fixed bugs:
unary
kernel doesn't respect offset #738new_null_array
creates invalid struct arrays #734Bitmap::len
returns the number of bytes, not bits. #730Closed issues:
Implemented enhancements:
Fixed bugs:
Closed issues:
Implemented enhancements:
value
method of boolean and primitive arrays #645Fixed bugs:
Closed issues:
Implemented enhancements:
limit
if sorted unstably #553Fixed bugs:
~/.cargo
is in a git checkout #589array::ffi::tests::test_struct
and other ffi tests #580Documentation updates:
Closed issues:
Breaking changes:
Implemented enhancements:
RecordBatch::concat
#537 [arrow] (silathdiir)BitReader
/DeltaBitPackDecoder
#325 [parquet] (kornholi)Fixed bugs:
ahash v0.7.4
depends on itself. Cycle #544default-features=false
#529Documentation updates:
Merged pull requests:
StringArray
creation from String Vec #522 [arrow] (silathdiir)ArrayData
equality and update unit tests #472 (Jimexist)SliceableCursor
to be constructed from an Arc
directly #369 (crepererum)take
#281 (Dandandan)Breaking changes:
Implemented enhancements:
Fixed bugs:
read_scalar
and read_scalar_at
allow transmuting values without unsafe
blocks #476Security fixes:
Closed issues:
Implemented enhancements:
Fixed bugs:
not
kernel does not take offset of null buffer into account #417WriterProperties.max_row_group_size
not wired up #257Breaking changes:
Implemented enhancements:
Arc
-based constructor for parquet::util::cursor::SliceableCursor
#368shrink_to
/ shrink_to_fit
to MutableBuffer
#297Fixed bugs:
Implemented enhancements:
Fixed bugs:
select_nth_unstable_by
found for mutable reference &mut [T]
#283metadata
when deserialize #241 [arrow]Security fixes:
Documentation updates:
Closed issues: