| [{"path":"/LICENSE.html","id":null,"dir":"","previous_headings":"","what":"Apache License","title":"Apache License","text":"Version 2.0, January 2004 <http://www.apache.org/licenses/>","code":""},{"path":[]},{"path":"/LICENSE.html","id":"id_1-definitions","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"1. Definitions","title":"Apache License","text":"“License” shall mean terms conditions use, reproduction, distribution defined Sections 1 9 document. “Licensor” shall mean copyright owner entity authorized copyright owner granting License. “Legal Entity” shall mean union acting entity entities control, controlled , common control entity. purposes definition, “control” means () power, direct indirect, cause direction management entity, whether contract otherwise, (ii) ownership fifty percent (50%) outstanding shares, (iii) beneficial ownership entity. “” (“”) shall mean individual Legal Entity exercising permissions granted License. “Source” form shall mean preferred form making modifications, including limited software source code, documentation source, configuration files. “Object” form shall mean form resulting mechanical transformation translation Source form, including limited compiled object code, generated documentation, conversions media types. “Work” shall mean work authorship, whether Source Object form, made available License, indicated copyright notice included attached work (example provided Appendix ). “Derivative Works” shall mean work, whether Source Object form, based (derived ) Work editorial revisions, annotations, elaborations, modifications represent, whole, original work authorship. purposes License, Derivative Works shall include works remain separable , merely link (bind name) interfaces , Work Derivative Works thereof. “Contribution” shall mean work authorship, including original version Work modifications additions Work Derivative Works thereof, intentionally submitted Licensor inclusion Work copyright owner individual Legal Entity authorized submit behalf copyright owner. purposes definition, “submitted” means form electronic, verbal, written communication sent Licensor representatives, including limited communication electronic mailing lists, source code control systems, issue tracking systems managed , behalf , Licensor purpose discussing improving Work, excluding communication conspicuously marked otherwise designated writing copyright owner “Contribution.” “Contributor” shall mean Licensor individual Legal Entity behalf Contribution received Licensor subsequently incorporated within Work.","code":""},{"path":"/LICENSE.html","id":"id_2-grant-of-copyright-license","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"2. Grant of Copyright License","title":"Apache License","text":"Subject terms conditions License, Contributor hereby grants perpetual, worldwide, non-exclusive, -charge, royalty-free, irrevocable copyright license reproduce, prepare Derivative Works , publicly display, publicly perform, sublicense, distribute Work Derivative Works Source Object form.","code":""},{"path":"/LICENSE.html","id":"id_3-grant-of-patent-license","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"3. Grant of Patent License","title":"Apache License","text":"Subject terms conditions License, Contributor hereby grants perpetual, worldwide, non-exclusive, -charge, royalty-free, irrevocable (except stated section) patent license make, made, use, offer sell, sell, import, otherwise transfer Work, license applies patent claims licensable Contributor necessarily infringed Contribution(s) alone combination Contribution(s) Work Contribution(s) submitted. institute patent litigation entity (including cross-claim counterclaim lawsuit) alleging Work Contribution incorporated within Work constitutes direct contributory patent infringement, patent licenses granted License Work shall terminate date litigation filed.","code":""},{"path":"/LICENSE.html","id":"id_4-redistribution","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"4. Redistribution","title":"Apache License","text":"may reproduce distribute copies Work Derivative Works thereof medium, without modifications, Source Object form, provided meet following conditions: () must give recipients Work Derivative Works copy License; (b) must cause modified files carry prominent notices stating changed files; (c) must retain, Source form Derivative Works distribute, copyright, patent, trademark, attribution notices Source form Work, excluding notices pertain part Derivative Works; (d) Work includes “NOTICE” text file part distribution, Derivative Works distribute must include readable copy attribution notices contained within NOTICE file, excluding notices pertain part Derivative Works, least one following places: within NOTICE text file distributed part Derivative Works; within Source form documentation, provided along Derivative Works; , within display generated Derivative Works, wherever third-party notices normally appear. contents NOTICE file informational purposes modify License. may add attribution notices within Derivative Works distribute, alongside addendum NOTICE text Work, provided additional attribution notices construed modifying License. may add copyright statement modifications may provide additional different license terms conditions use, reproduction, distribution modifications, Derivative Works whole, provided use, reproduction, distribution Work otherwise complies conditions stated License.","code":""},{"path":"/LICENSE.html","id":"id_5-submission-of-contributions","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"5. Submission of Contributions","title":"Apache License","text":"Unless explicitly state otherwise, Contribution intentionally submitted inclusion Work Licensor shall terms conditions License, without additional terms conditions. Notwithstanding , nothing herein shall supersede modify terms separate license agreement may executed Licensor regarding Contributions.","code":""},{"path":"/LICENSE.html","id":"id_6-trademarks","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"6. Trademarks","title":"Apache License","text":"License grant permission use trade names, trademarks, service marks, product names Licensor, except required reasonable customary use describing origin Work reproducing content NOTICE file.","code":""},{"path":"/LICENSE.html","id":"id_7-disclaimer-of-warranty","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"7. Disclaimer of Warranty","title":"Apache License","text":"Unless required applicable law agreed writing, Licensor provides Work (Contributor provides Contributions) “” BASIS, WITHOUT WARRANTIES CONDITIONS KIND, either express implied, including, without limitation, warranties conditions TITLE, NON-INFRINGEMENT, MERCHANTABILITY, FITNESS PARTICULAR PURPOSE. solely responsible determining appropriateness using redistributing Work assume risks associated exercise permissions License.","code":""},{"path":"/LICENSE.html","id":"id_8-limitation-of-liability","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"8. Limitation of Liability","title":"Apache License","text":"event legal theory, whether tort (including negligence), contract, otherwise, unless required applicable law (deliberate grossly negligent acts) agreed writing, shall Contributor liable damages, including direct, indirect, special, incidental, consequential damages character arising result License use inability use Work (including limited damages loss goodwill, work stoppage, computer failure malfunction, commercial damages losses), even Contributor advised possibility damages.","code":""},{"path":"/LICENSE.html","id":"id_9-accepting-warranty-or-additional-liability","dir":"","previous_headings":"Terms and Conditions for use, reproduction, and distribution","what":"9. Accepting Warranty or Additional Liability","title":"Apache License","text":"redistributing Work Derivative Works thereof, may choose offer, charge fee , acceptance support, warranty, indemnity, liability obligations /rights consistent License. However, accepting obligations, may act behalf sole responsibility, behalf Contributor, agree indemnify, defend, hold Contributor harmless liability incurred , claims asserted , Contributor reason accepting warranty additional liability. END TERMS CONDITIONS","code":""},{"path":"/LICENSE.html","id":"appendix-how-to-apply-the-apache-license-to-your-work","dir":"","previous_headings":"","what":"APPENDIX: How to apply the Apache License to your work","title":"Apache License","text":"apply Apache License work, attach following boilerplate notice, fields enclosed brackets [] replaced identifying information. (Don’t include brackets!) text enclosed appropriate comment syntax file format. also recommend file class name description purpose included “printed page” copyright notice easier identification within third-party archives.","code":"Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."},{"path":"/authors.html","id":null,"dir":"","previous_headings":"","what":"Authors","title":"Authors and Citation","text":"Dewey Dunnington. Author, maintainer. Apache Arrow. Author, copyright holder. Apache Software Foundation. Copyright holder.","code":""},{"path":"/authors.html","id":"citation","dir":"","previous_headings":"","what":"Citation","title":"Authors and Citation","text":"Dunnington D, Apache Arrow (2024). nanoarrow: Interface 'nanoarrow' 'C' Library. R package version 0.5.0.9000, https://github.com/apache/arrow-nanoarrow, https://arrow.apache.org/nanoarrow/latest/r/.","code":"@Manual{, title = {nanoarrow: Interface to the 'nanoarrow' 'C' Library}, author = {Dewey Dunnington and {Apache Arrow}}, year = {2024}, note = {R package version 0.5.0.9000, https://github.com/apache/arrow-nanoarrow}, url = {https://arrow.apache.org/nanoarrow/latest/r/}, }"},{"path":"/index.html","id":"nanoarrow","dir":"","previous_headings":"","what":"Interface to the nanoarrow C Library","title":"Interface to the nanoarrow C Library","text":"goal nanoarrow provide minimal useful bindings Arrow C Data Arrow C Stream interfaces using nanoarrow C library.","code":""},{"path":"/index.html","id":"installation","dir":"","previous_headings":"","what":"Installation","title":"Interface to the nanoarrow C Library","text":"can install released version nanoarrow CRAN : can install development version nanoarrow GitHub : can load package, ’re good go!","code":"install.packages(\"nanoarrow\") # install.packages(\"remotes\") remotes::install_github(\"apache/arrow-nanoarrow/r\") library(nanoarrow)"},{"path":"/index.html","id":"example","dir":"","previous_headings":"","what":"Example","title":"Interface to the nanoarrow C Library","text":"Arrow C Data Arrow C Stream interfaces comprised three structures: ArrowSchema represents data type array, ArrowArray represents values array, ArrowArrayStream, represents zero ArrowArrays common ArrowSchema. three can wrapped R objects using nanoarrow R package.","code":""},{"path":"/index.html","id":"schemas","dir":"","previous_headings":"Example","what":"Schemas","title":"Interface to the nanoarrow C Library","text":"Use infer_nanoarrow_schema() get ArrowSchema object corresponds given R vector type; use as_nanoarrow_schema() convert object data type representation (e.g., arrow R package DataType like arrow::int32()); use na_XXX() functions construct .","code":"infer_nanoarrow_schema(1:5) #> <nanoarrow_schema int32> #> $ format : chr \"i\" #> $ name : chr \"\" #> $ metadata : list() #> $ flags : int 2 #> $ children : list() #> $ dictionary: NULL as_nanoarrow_schema(arrow::schema(col1 = arrow::float64())) #> <nanoarrow_schema struct> #> $ format : chr \"+s\" #> $ name : chr \"\" #> $ metadata : list() #> $ flags : int 0 #> $ children :List of 1 #> ..$ col1:<nanoarrow_schema double> #> .. ..$ format : chr \"g\" #> .. ..$ name : chr \"col1\" #> .. ..$ metadata : list() #> .. ..$ flags : int 2 #> .. ..$ children : list() #> .. ..$ dictionary: NULL #> $ dictionary: NULL na_int64() #> <nanoarrow_schema int64> #> $ format : chr \"l\" #> $ name : chr \"\" #> $ metadata : list() #> $ flags : int 2 #> $ children : list() #> $ dictionary: NULL"},{"path":"/index.html","id":"arrays","dir":"","previous_headings":"Example","what":"Arrays","title":"Interface to the nanoarrow C Library","text":"Use as_nanoarrow_array() convert object ArrowArray object: can use .vector() .data.frame() get R representation object back: Even though C level ArrowArray distinct ArrowSchema, R level attach schema wherever possible. can access attached schema using infer_nanoarrow_schema():","code":"as_nanoarrow_array(1:5) #> <nanoarrow_array int32[5]> #> $ length : int 5 #> $ null_count: int 0 #> $ offset : int 0 #> $ buffers :List of 2 #> ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> ..$ :<nanoarrow_buffer data<int32>[5][20 b]> `1 2 3 4 5` #> $ dictionary: NULL #> $ children : list() as_nanoarrow_array(data.frame(col1 = c(1.1, 2.2))) #> <nanoarrow_array struct[2]> #> $ length : int 2 #> $ null_count: int 0 #> $ offset : int 0 #> $ buffers :List of 1 #> ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> $ children :List of 1 #> ..$ col1:<nanoarrow_array double[2]> #> .. ..$ length : int 2 #> .. ..$ null_count: int 0 #> .. ..$ offset : int 0 #> .. ..$ buffers :List of 2 #> .. .. ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> .. .. ..$ :<nanoarrow_buffer data<double>[2][16 b]> `1.1 2.2` #> .. ..$ dictionary: NULL #> .. ..$ children : list() #> $ dictionary: NULL array <- as_nanoarrow_array(data.frame(col1 = c(1.1, 2.2))) as.data.frame(array) #> col1 #> 1 1.1 #> 2 2.2 infer_nanoarrow_schema(array) #> <nanoarrow_schema struct> #> $ format : chr \"+s\" #> $ name : chr \"\" #> $ metadata : list() #> $ flags : int 0 #> $ children :List of 1 #> ..$ col1:<nanoarrow_schema double> #> .. ..$ format : chr \"g\" #> .. ..$ name : chr \"col1\" #> .. ..$ metadata : list() #> .. ..$ flags : int 2 #> .. ..$ children : list() #> .. ..$ dictionary: NULL #> $ dictionary: NULL"},{"path":"/index.html","id":"array-streams","dir":"","previous_headings":"Example","what":"Array Streams","title":"Interface to the nanoarrow C Library","text":"easiest way create ArrowArrayStream list arrays objects can converted array using as_nanoarrow_array(): can pull batches stream using $get_next() method. last batch return NULL. can pull batches data.frame() calling .data.frame() .vector(): consuming stream, call release method soon can. lets implementation stream release resources (like open files) may holding predictable way waiting garbage collector clean object.","code":"stream <- basic_array_stream( list( data.frame(col1 = c(1.1, 2.2)), data.frame(col1 = c(3.3, 4.4)) ) ) stream$get_next() #> <nanoarrow_array struct[2]> #> $ length : int 2 #> $ null_count: int 0 #> $ offset : int 0 #> $ buffers :List of 1 #> ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> $ children :List of 1 #> ..$ col1:<nanoarrow_array double[2]> #> .. ..$ length : int 2 #> .. ..$ null_count: int 0 #> .. ..$ offset : int 0 #> .. ..$ buffers :List of 2 #> .. .. ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> .. .. ..$ :<nanoarrow_buffer data<double>[2][16 b]> `1.1 2.2` #> .. ..$ dictionary: NULL #> .. ..$ children : list() #> $ dictionary: NULL stream$get_next() #> <nanoarrow_array struct[2]> #> $ length : int 2 #> $ null_count: int 0 #> $ offset : int 0 #> $ buffers :List of 1 #> ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> $ children :List of 1 #> ..$ col1:<nanoarrow_array double[2]> #> .. ..$ length : int 2 #> .. ..$ null_count: int 0 #> .. ..$ offset : int 0 #> .. ..$ buffers :List of 2 #> .. .. ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> .. .. ..$ :<nanoarrow_buffer data<double>[2][16 b]> `3.3 4.4` #> .. ..$ dictionary: NULL #> .. ..$ children : list() #> $ dictionary: NULL stream$get_next() #> NULL stream <- basic_array_stream( list( data.frame(col1 = c(1.1, 2.2)), data.frame(col1 = c(3.3, 4.4)) ) ) as.data.frame(stream) #> col1 #> 1 1.1 #> 2 2.2 #> 3 3.3 #> 4 4.4"},{"path":"/index.html","id":"integration-with-the-arrow-package","dir":"","previous_headings":"","what":"Integration with the arrow package","title":"Interface to the nanoarrow C Library","text":"nanoarrow package implements as_nanoarrow_schema(), as_nanoarrow_array(), as_nanoarrow_array_stream() arrow package types. Similarly, implements arrow::as_arrow_array(), arrow::as_record_batch(), arrow::as_arrow_table(), arrow::as_record_batch_reader(), arrow::infer_type(), arrow::as_data_type(), arrow::as_schema() nanoarrow objects can pass equivalent nanoarrow objects many arrow functions vice versa.","code":""},{"path":"/reference/array_stream_set_finalizer.html","id":null,"dir":"Reference","previous_headings":"","what":"Register an array stream finalizer — array_stream_set_finalizer","title":"Register an array stream finalizer — array_stream_set_finalizer","text":"cases, R functions return nanoarrow_array_stream may require scope object outlive array stream. need object released deterministically (e.g., close open files), can register function run stream's release callback invoked R thread. Note finalizer run stream's release callback invoked non-R thread. case, finalizer chain environments garbage-collected nanoarrow:::preserved_empty() run.","code":""},{"path":"/reference/array_stream_set_finalizer.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Register an array stream finalizer — array_stream_set_finalizer","text":"","code":"array_stream_set_finalizer(array_stream, finalizer)"},{"path":"/reference/array_stream_set_finalizer.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Register an array stream finalizer — array_stream_set_finalizer","text":"array_stream nanoarrow_array_stream finalizer function called zero arguments.","code":""},{"path":"/reference/array_stream_set_finalizer.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Register an array stream finalizer — array_stream_set_finalizer","text":"newly allocated array_stream whose release callback call supplied finalizer.","code":""},{"path":"/reference/array_stream_set_finalizer.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Register an array stream finalizer — array_stream_set_finalizer","text":"","code":"stream <- array_stream_set_finalizer( basic_array_stream(list(1:5)), function() message(\"All done!\") ) stream$release() #> All done!"},{"path":"/reference/as_nanoarrow_array.html","id":null,"dir":"Reference","previous_headings":"","what":"Convert an object to a nanoarrow array — as_nanoarrow_array","title":"Convert an object to a nanoarrow array — as_nanoarrow_array","text":"nanoarrow 'array' refers struct ArrowArray definition Arrow C data interface. R level, attach schema functionally nanoarrow_array class can used similar way arrow::Array. Note nanoarrow arrow::RecordBatch non-nullable arrow::StructArray represented identically.","code":""},{"path":"/reference/as_nanoarrow_array.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Convert an object to a nanoarrow array — as_nanoarrow_array","text":"","code":"as_nanoarrow_array(x, ..., schema = NULL)"},{"path":"/reference/as_nanoarrow_array.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Convert an object to a nanoarrow array — as_nanoarrow_array","text":"x object convert array ... Passed S3 methods schema optional schema used enforce conversion particular type. Defaults infer_nanoarrow_schema().","code":""},{"path":"/reference/as_nanoarrow_array.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Convert an object to a nanoarrow array — as_nanoarrow_array","text":"object class 'nanoarrow_array'","code":""},{"path":"/reference/as_nanoarrow_array.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Convert an object to a nanoarrow array — as_nanoarrow_array","text":"","code":"(array <- as_nanoarrow_array(1:5)) #> <nanoarrow_array int32[5]> #> $ length : int 5 #> $ null_count: int 0 #> $ offset : int 0 #> $ buffers :List of 2 #> ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> ..$ :<nanoarrow_buffer data<int32>[5][20 b]> `1 2 3 4 5` #> $ dictionary: NULL #> $ children : list() as.vector(array) #> [1] 1 2 3 4 5 (array <- as_nanoarrow_array(data.frame(x = 1:5))) #> <nanoarrow_array struct[5]> #> $ length : int 5 #> $ null_count: int 0 #> $ offset : int 0 #> $ buffers :List of 1 #> ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> $ children :List of 1 #> ..$ x:<nanoarrow_array int32[5]> #> .. ..$ length : int 5 #> .. ..$ null_count: int 0 #> .. ..$ offset : int 0 #> .. ..$ buffers :List of 2 #> .. .. ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> .. .. ..$ :<nanoarrow_buffer data<int32>[5][20 b]> `1 2 3 4 5` #> .. ..$ dictionary: NULL #> .. ..$ children : list() #> $ dictionary: NULL as.data.frame(array) #> x #> 1 1 #> 2 2 #> 3 3 #> 4 4 #> 5 5"},{"path":"/reference/as_nanoarrow_array_stream.html","id":null,"dir":"Reference","previous_headings":"","what":"Convert an object to a nanoarrow array_stream — as_nanoarrow_array_stream","title":"Convert an object to a nanoarrow array_stream — as_nanoarrow_array_stream","text":"nanoarrow, 'array stream' corresponds struct ArrowArrayStream defined Arrow C Stream interface. object used represent stream arrays common schema. similar arrow::RecordBatchReader except can used represent stream type (just record batches). Note stream record batches stream non-nullable struct arrays represented identically. Also note array streams mutable objects passed reference value.","code":""},{"path":"/reference/as_nanoarrow_array_stream.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Convert an object to a nanoarrow array_stream — as_nanoarrow_array_stream","text":"","code":"as_nanoarrow_array_stream(x, ..., schema = NULL)"},{"path":"/reference/as_nanoarrow_array_stream.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Convert an object to a nanoarrow array_stream — as_nanoarrow_array_stream","text":"x object convert array_stream ... Passed S3 methods schema optional schema used enforce conversion particular type. Defaults infer_nanoarrow_schema().","code":""},{"path":"/reference/as_nanoarrow_array_stream.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Convert an object to a nanoarrow array_stream — as_nanoarrow_array_stream","text":"object class 'nanoarrow_array_stream'","code":""},{"path":"/reference/as_nanoarrow_array_stream.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Convert an object to a nanoarrow array_stream — as_nanoarrow_array_stream","text":"","code":"(stream <- as_nanoarrow_array_stream(data.frame(x = 1:5))) #> <nanoarrow_array_stream struct<x: int32>> #> $ get_schema:function () #> $ get_next :function (schema = x$get_schema(), validate = TRUE) #> $ release :function () stream$get_schema() #> <nanoarrow_schema struct> #> $ format : chr \"+s\" #> $ name : chr \"\" #> $ metadata : list() #> $ flags : int 0 #> $ children :List of 1 #> ..$ x:<nanoarrow_schema int32> #> .. ..$ format : chr \"i\" #> .. ..$ name : chr \"x\" #> .. ..$ metadata : list() #> .. ..$ flags : int 2 #> .. ..$ children : list() #> .. ..$ dictionary: NULL #> $ dictionary: NULL stream$get_next() #> <nanoarrow_array struct[5]> #> $ length : int 5 #> $ null_count: int 0 #> $ offset : int 0 #> $ buffers :List of 1 #> ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> $ children :List of 1 #> ..$ x:<nanoarrow_array int32[5]> #> .. ..$ length : int 5 #> .. ..$ null_count: int 0 #> .. ..$ offset : int 0 #> .. ..$ buffers :List of 2 #> .. .. ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> .. .. ..$ :<nanoarrow_buffer data<int32>[5][20 b]> `1 2 3 4 5` #> .. ..$ dictionary: NULL #> .. ..$ children : list() #> $ dictionary: NULL # The last batch is returned as NULL stream$get_next() #> NULL # Release the stream stream$release()"},{"path":"/reference/as_nanoarrow_buffer.html","id":null,"dir":"Reference","previous_headings":"","what":"Convert an object to a nanoarrow buffer — as_nanoarrow_buffer","title":"Convert an object to a nanoarrow buffer — as_nanoarrow_buffer","text":"Convert object nanoarrow buffer","code":""},{"path":"/reference/as_nanoarrow_buffer.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Convert an object to a nanoarrow buffer — as_nanoarrow_buffer","text":"","code":"as_nanoarrow_buffer(x, ...)"},{"path":"/reference/as_nanoarrow_buffer.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Convert an object to a nanoarrow buffer — as_nanoarrow_buffer","text":"x object convert buffer ... Passed S3 methods","code":""},{"path":"/reference/as_nanoarrow_buffer.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Convert an object to a nanoarrow buffer — as_nanoarrow_buffer","text":"object class 'nanoarrow_buffer'","code":""},{"path":"/reference/as_nanoarrow_buffer.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Convert an object to a nanoarrow buffer — as_nanoarrow_buffer","text":"","code":"array <- as_nanoarrow_array(c(NA, 1:4)) array$buffers #> [[1]] #> <nanoarrow_buffer validity<bool>[8][1 b]> `FALSE TRUE TRUE TRUE TRUE TRUE TR...` #> #> [[2]] #> <nanoarrow_buffer data<int32>[5][20 b]> `NA 1 2 3 4` #> as.raw(array$buffers[[1]]) #> [1] fe as.raw(array$buffers[[2]]) #> [1] 00 00 00 80 01 00 00 00 02 00 00 00 03 00 00 00 04 00 00 00 convert_buffer(array$buffers[[1]]) #> [1] FALSE TRUE TRUE TRUE TRUE TRUE TRUE TRUE convert_buffer(array$buffers[[2]]) #> [1] NA 1 2 3 4"},{"path":"/reference/as_nanoarrow_schema.html","id":null,"dir":"Reference","previous_headings":"","what":"Convert an object to a nanoarrow schema — as_nanoarrow_schema","title":"Convert an object to a nanoarrow schema — as_nanoarrow_schema","text":"nanoarrow 'schema' refers struct ArrowSchema defined Arrow C Data interface. data structure can used represent arrow::schema(), arrow::field(), arrow::DataType. Note nanoarrow, arrow::schema() non-nullable arrow::struct() represented identically.","code":""},{"path":"/reference/as_nanoarrow_schema.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Convert an object to a nanoarrow schema — as_nanoarrow_schema","text":"","code":"as_nanoarrow_schema(x, ...) infer_nanoarrow_schema(x, ...) nanoarrow_schema_parse(x, recursive = FALSE) nanoarrow_schema_modify(x, new_values, validate = TRUE)"},{"path":"/reference/as_nanoarrow_schema.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Convert an object to a nanoarrow schema — as_nanoarrow_schema","text":"x object convert schema ... Passed S3 methods recursive Use TRUE include children member parsing schemas. new_values New schema component assign validate Use FALSE skip schema validation","code":""},{"path":"/reference/as_nanoarrow_schema.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Convert an object to a nanoarrow schema — as_nanoarrow_schema","text":"object class 'nanoarrow_schema'","code":""},{"path":"/reference/as_nanoarrow_schema.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Convert an object to a nanoarrow schema — as_nanoarrow_schema","text":"","code":"infer_nanoarrow_schema(integer()) #> <nanoarrow_schema int32> #> $ format : chr \"i\" #> $ name : chr \"\" #> $ metadata : list() #> $ flags : int 2 #> $ children : list() #> $ dictionary: NULL infer_nanoarrow_schema(data.frame(x = integer())) #> <nanoarrow_schema struct> #> $ format : chr \"+s\" #> $ name : chr \"\" #> $ metadata : list() #> $ flags : int 0 #> $ children :List of 1 #> ..$ x:<nanoarrow_schema int32> #> .. ..$ format : chr \"i\" #> .. ..$ name : chr \"x\" #> .. ..$ metadata : list() #> .. ..$ flags : int 2 #> .. ..$ children : list() #> .. ..$ dictionary: NULL #> $ dictionary: NULL"},{"path":"/reference/as_nanoarrow_vctr.html","id":null,"dir":"Reference","previous_headings":"","what":"Experimental Arrow encoded arrays as R vectors — as_nanoarrow_vctr","title":"Experimental Arrow encoded arrays as R vectors — as_nanoarrow_vctr","text":"experimental vctr class allows zero Arrow arrays present R vector without converting . useful arrays types non-lossy R equivalent, helps provide intermediary object type default conversion prohibitively expensive (e.g., nested list data frames). objects survive many vctr transformations; however, can sliced without copying underlying arrays.","code":""},{"path":"/reference/as_nanoarrow_vctr.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Experimental Arrow encoded arrays as R vectors — as_nanoarrow_vctr","text":"","code":"as_nanoarrow_vctr(x, ..., schema = NULL, subclass = character()) nanoarrow_vctr(schema = NULL, subclass = character())"},{"path":"/reference/as_nanoarrow_vctr.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Experimental Arrow encoded arrays as R vectors — as_nanoarrow_vctr","text":"x object works as_nanoarrow_array_stream(). ... Passed as_nanoarrow_array_stream() schema optional schema subclass optional subclass nanoarrow_vctr prepend final class name.","code":""},{"path":"/reference/as_nanoarrow_vctr.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Experimental Arrow encoded arrays as R vectors — as_nanoarrow_vctr","text":"vctr class 'nanoarrow_vctr'","code":""},{"path":"/reference/as_nanoarrow_vctr.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Experimental Arrow encoded arrays as R vectors — as_nanoarrow_vctr","text":"nanoarrow_vctr currently implemented similarly factor(): storage type integer() sequence along total length vctr attributes required resolve indices array + offset. Sequences typically compact representation recent versions R cheap storage footprint even large arrays. attributes currently: schema: nanoarrow_schema shared chunk. chunks: list() nanoarrow_array. offsets: integer() vector beginning 0 followed cumulative length chunk. allows chunk index + offset resolved logical index log(n) complexity. implementation preliminary may change; however, result as_nanoarrow_array_stream(some_vctr[begin:end]) remain stable.","code":""},{"path":"/reference/as_nanoarrow_vctr.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Experimental Arrow encoded arrays as R vectors — as_nanoarrow_vctr","text":"","code":"array <- as_nanoarrow_array(1:5) as_nanoarrow_vctr(array) #> <nanoarrow_vctr int32[5]> #> [1] 1 2 3 4 5"},{"path":"/reference/basic_array_stream.html","id":null,"dir":"Reference","previous_headings":"","what":"Create ArrayStreams from batches — basic_array_stream","title":"Create ArrayStreams from batches — basic_array_stream","text":"Create ArrayStreams batches","code":""},{"path":"/reference/basic_array_stream.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Create ArrayStreams from batches — basic_array_stream","text":"","code":"basic_array_stream(batches, schema = NULL, validate = TRUE)"},{"path":"/reference/basic_array_stream.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Create ArrayStreams from batches — basic_array_stream","text":"batches list() nanoarrow_array objects objects can coerced via as_nanoarrow_array(). schema nanoarrow_schema NULL guess based first schema. validate Use FALSE skip validation step (.e., know arrays valid).","code":""},{"path":"/reference/basic_array_stream.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Create ArrayStreams from batches — basic_array_stream","text":"nanoarrow_array_stream","code":""},{"path":"/reference/basic_array_stream.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Create ArrayStreams from batches — basic_array_stream","text":"","code":"(stream <- basic_array_stream(list(data.frame(a = 1, b = 2)))) #> <nanoarrow_array_stream struct<a: double, b: double>> #> $ get_schema:function () #> $ get_next :function (schema = x$get_schema(), validate = TRUE) #> $ release :function () as.data.frame(stream$get_next()) #> a b #> 1 1 2 stream$get_next() #> NULL"},{"path":"/reference/convert_array.html","id":null,"dir":"Reference","previous_headings":"","what":"Convert an Array into an R vector — convert_array","title":"Convert an Array into an R vector — convert_array","text":"Converts array type specified . low-level interface; users use .data.frame() .vector() unless finer-grained control needed conversion. function S3 generic dispatching : developers may implement S3 methods custom vector types.","code":""},{"path":"/reference/convert_array.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Convert an Array into an R vector — convert_array","text":"","code":"convert_array(array, to = NULL, ...)"},{"path":"/reference/convert_array.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Convert an Array into an R vector — convert_array","text":"array nanoarrow_array. target prototype object describing type array converted, NULL use default conversion returned infer_nanoarrow_ptype(). Alternatively, function can passed perform alternative calculation default ptype function array default inference prototype. ... Passed S3 methods","code":""},{"path":"/reference/convert_array.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Convert an Array into an R vector — convert_array","text":"R vector type .","code":""},{"path":"/reference/convert_array.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Convert an Array into an R vector — convert_array","text":"Conversions implemented following R vector types: logical(): numeric type can converted logical() addition bool type. numeric types, non-zero value considered TRUE. integer(): numeric type can converted integer(); however, warning signaled value outside range 32-bit integer. double(): numeric type can converted double(). conversion currently warn values may roundtrip floating-point double (e.g., large uint64 int64 values). character(): String large string types can converted character(). conversion check valid UTF-8: need finer-grained control encodings, use = blob::blob(). factor(): Dictionary-encoded arrays strings can converted factor(); however, must specified explicitly (.e., convert_array(array, factor())) arrays arriving chunks can dictionaries contain different levels. Use convert_array(array, factor(levels = c(...))) materialize array vector known levels. Date: date32 type can converted R Date vector. hms::hms(): Time32 time64 types can converted hms::hms(). difftime(): Time32, time64, duration types can converted R difftime() vectors. value converted match units() attribute . blob::blob(): String, large string, binary, large binary types can converted blob::blob(). vctrs::list_of(): List, large list, fixed-size list types can converted vctrs::list_of(). data.frame(): Struct types can converted data.frame(). vctrs::unspecified(): type can converted vctrs::unspecified(); however, warning raised non-null values encountered. addition conversions, null array may converted target prototype except data.frame(). Extension arrays currently converted storage type.","code":""},{"path":"/reference/convert_array.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Convert an Array into an R vector — convert_array","text":"","code":"array <- as_nanoarrow_array(data.frame(x = 1:5)) str(convert_array(array)) #> 'data.frame':\t5 obs. of 1 variable: #> $ x: int 1 2 3 4 5 str(convert_array(array, to = data.frame(x = double()))) #> 'data.frame':\t5 obs. of 1 variable: #> $ x: num 1 2 3 4 5"},{"path":"/reference/convert_array_stream.html","id":null,"dir":"Reference","previous_headings":"","what":"Convert an Array Stream into an R vector — convert_array_stream","title":"Convert an Array Stream into an R vector — convert_array_stream","text":"Converts array_stream type specified . low-level interface; users use .data.frame() .vector() unless finer-grained control needed conversion. See convert_array() details conversion process; see infer_nanoarrow_ptype() default inferences .","code":""},{"path":"/reference/convert_array_stream.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Convert an Array Stream into an R vector — convert_array_stream","text":"","code":"convert_array_stream(array_stream, to = NULL, size = NULL, n = Inf) collect_array_stream(array_stream, n = Inf, schema = NULL, validate = TRUE)"},{"path":"/reference/convert_array_stream.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Convert an Array Stream into an R vector — convert_array_stream","text":"array_stream nanoarrow_array_stream. target prototype object describing type array converted, NULL use default conversion returned infer_nanoarrow_ptype(). Alternatively, function can passed perform alternative calculation default ptype function array default inference prototype. size exact size output, known. specified, slightly efficient implementation may used collect output. n maximum number batches pull array stream. schema nanoarrow_schema NULL guess based first schema. validate Use FALSE skip validation step (.e., know arrays valid).","code":""},{"path":"/reference/convert_array_stream.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Convert an Array Stream into an R vector — convert_array_stream","text":"convert_array_stream(): R vector type . collect_array_stream(): list() nanoarrow_array","code":""},{"path":"/reference/convert_array_stream.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Convert an Array Stream into an R vector — convert_array_stream","text":"","code":"stream <- as_nanoarrow_array_stream(data.frame(x = 1:5)) str(convert_array_stream(stream)) #> 'data.frame':\t5 obs. of 1 variable: #> $ x: int 1 2 3 4 5 str(convert_array_stream(stream, to = data.frame(x = double()))) #> 'data.frame':\t0 obs. of 1 variable: #> $ x: num stream <- as_nanoarrow_array_stream(data.frame(x = 1:5)) collect_array_stream(stream) #> [[1]] #> <nanoarrow_array struct[5]> #> $ length : int 5 #> $ null_count: int 0 #> $ offset : int 0 #> $ buffers :List of 1 #> ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> $ children :List of 1 #> ..$ x:<nanoarrow_array int32[5]> #> .. ..$ length : int 5 #> .. ..$ null_count: int 0 #> .. ..$ offset : int 0 #> .. ..$ buffers :List of 2 #> .. .. ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> .. .. ..$ :<nanoarrow_buffer data<int32>[5][20 b]> `1 2 3 4 5` #> .. ..$ dictionary: NULL #> .. ..$ children : list() #> $ dictionary: NULL #>"},{"path":"/reference/infer_nanoarrow_ptype.html","id":null,"dir":"Reference","previous_headings":"","what":"Infer an R vector prototype — infer_nanoarrow_ptype","title":"Infer an R vector prototype — infer_nanoarrow_ptype","text":"Resolves default value use convert_array() convert_array_stream(). default conversions :","code":""},{"path":"/reference/infer_nanoarrow_ptype.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Infer an R vector prototype — infer_nanoarrow_ptype","text":"","code":"infer_nanoarrow_ptype(x)"},{"path":"/reference/infer_nanoarrow_ptype.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Infer an R vector prototype — infer_nanoarrow_ptype","text":"x nanoarrow_schema, nanoarrow_array, nanoarrow_array_stream.","code":""},{"path":"/reference/infer_nanoarrow_ptype.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Infer an R vector prototype — infer_nanoarrow_ptype","text":"R vector zero size describing target array materialized.","code":""},{"path":"/reference/infer_nanoarrow_ptype.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Infer an R vector prototype — infer_nanoarrow_ptype","text":"null vctrs::unspecified() boolean logical() int8, uint8, int16, uint16, int13 integer() uint32, int64, uint64, float, double double() string large string character() struct data.frame() binary large binary blob::blob() list, large_list, fixed_size_list vctrs::list_of() time32 time64 hms::hms() duration difftime() date32 .Date() timestamp .POSIXct() Additional conversions possible specifying explicit value . details conversion, see convert_array().","code":""},{"path":"/reference/infer_nanoarrow_ptype.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Infer an R vector prototype — infer_nanoarrow_ptype","text":"","code":"infer_nanoarrow_ptype(as_nanoarrow_array(1:10)) #> integer(0)"},{"path":"/reference/infer_nanoarrow_ptype_extension.html","id":null,"dir":"Reference","previous_headings":"","what":"Implement Arrow extension types — infer_nanoarrow_ptype_extension","title":"Implement Arrow extension types — infer_nanoarrow_ptype_extension","text":"Implement Arrow extension types","code":""},{"path":"/reference/infer_nanoarrow_ptype_extension.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Implement Arrow extension types — infer_nanoarrow_ptype_extension","text":"","code":"infer_nanoarrow_ptype_extension( extension_spec, x, ..., warn_unregistered = TRUE ) convert_array_extension( extension_spec, array, to, ..., warn_unregistered = TRUE ) as_nanoarrow_array_extension(extension_spec, x, ..., schema = NULL)"},{"path":"/reference/infer_nanoarrow_ptype_extension.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Implement Arrow extension types — infer_nanoarrow_ptype_extension","text":"extension_spec extension specification inheriting 'nanoarrow_extension_spec'. x, array, , schema, ... Passed infer_nanoarrow_ptype(), convert_array(), as_nanoarrow_array(), /as_nanoarrow_array_stream(). warn_unregistered Use FALSE infer/convert based storage type without warning.","code":""},{"path":"/reference/infer_nanoarrow_ptype_extension.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Implement Arrow extension types — infer_nanoarrow_ptype_extension","text":"infer_nanoarrow_ptype_extension(): R vector prototype used default conversion target. convert_array_extension(): R vector type . as_nanoarrow_array_extension(): nanoarrow_array type schema.","code":""},{"path":"/reference/na_type.html","id":null,"dir":"Reference","previous_headings":"","what":"Create type objects — na_type","title":"Create type objects — na_type","text":"nanoarrow, types, fields, schemas represented nanoarrow_schema. functions convenience constructors create objects readable way. Use na_type() construct types based constructor name, also name prints/returned nanoarrow_schema_parse().","code":""},{"path":"/reference/na_type.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Create type objects — na_type","text":"","code":"na_type( type_name, byte_width = NULL, unit = NULL, timezone = NULL, column_types = NULL, item_type = NULL, key_type = NULL, value_type = NULL, index_type = NULL, ordered = NULL, list_size = NULL, keys_sorted = NULL, storage_type = NULL, extension_name = NULL, extension_metadata = NULL, nullable = NULL ) na_na(nullable = TRUE) na_bool(nullable = TRUE) na_int8(nullable = TRUE) na_uint8(nullable = TRUE) na_int16(nullable = TRUE) na_uint16(nullable = TRUE) na_int32(nullable = TRUE) na_uint32(nullable = TRUE) na_int64(nullable = TRUE) na_uint64(nullable = TRUE) na_half_float(nullable = TRUE) na_float(nullable = TRUE) na_double(nullable = TRUE) na_string(nullable = TRUE) na_large_string(nullable = TRUE) na_binary(nullable = TRUE) na_large_binary(nullable = TRUE) na_fixed_size_binary(byte_width, nullable = TRUE) na_date32(nullable = TRUE) na_date64(nullable = TRUE) na_time32(unit = c(\"ms\", \"s\"), nullable = TRUE) na_time64(unit = c(\"us\", \"ns\"), nullable = TRUE) na_duration(unit = c(\"ms\", \"s\", \"us\", \"ns\"), nullable = TRUE) na_interval_months(nullable = TRUE) na_interval_day_time(nullable = TRUE) na_interval_month_day_nano(nullable = TRUE) na_timestamp(unit = c(\"us\", \"ns\", \"s\", \"ms\"), timezone = \"\", nullable = TRUE) na_decimal128(precision, scale, nullable = TRUE) na_decimal256(precision, scale, nullable = TRUE) na_struct(column_types = list(), nullable = FALSE) na_sparse_union(column_types = list()) na_dense_union(column_types = list()) na_list(item_type, nullable = TRUE) na_large_list(item_type, nullable = TRUE) na_fixed_size_list(item_type, list_size, nullable = TRUE) na_map(key_type, item_type, keys_sorted = FALSE, nullable = TRUE) na_dictionary(value_type, index_type = na_int32(), ordered = FALSE) na_extension(storage_type, extension_name, extension_metadata = \"\")"},{"path":"/reference/na_type.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Create type objects — na_type","text":"type_name name type (e.g., \"int32\"). form constructor useful writing tests loop many types. byte_width na_fixed_size_binary(), number bytes occupied item. unit One 's' (seconds), 'ms' (milliseconds), 'us' (microseconds), 'ns' (nanoseconds). timezone string representing timezone name. empty string \"\" represents naive point time (.e., one associated timezone). column_types list() nanoarrow_schemas. item_type na_list(), na_large_list(), na_fixed_size_list(), na_map(), nanoarrow_schema representing item type. key_type nanoarrow_schema representing na_map() key type. value_type nanoarrow_schema representing na_dictionary() na_map() value type. index_type nanoarrow_schema representing na_dictionary() index type. ordered Use TRUE assert order values dictionary meaningful. list_size number elements item na_fixed_size_list(). keys_sorted Use TRUE assert keys sorted. storage_type na_extension(), underlying value type. extension_name na_extension(), extension name. typically namespaced separated dots (e.g., arrow.r.vctrs). extension_metadata string raw vector defining extension metadata. Arrow extension types define extension metadata JSON object. nullable Use FALSE assert field contain null values. precision total number digits representable decimal type scale number digits decimal point decimal type","code":""},{"path":"/reference/na_type.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Create type objects — na_type","text":"nanoarrow_schema","code":""},{"path":"/reference/na_type.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Create type objects — na_type","text":"","code":"na_int32() #> <nanoarrow_schema int32> #> $ format : chr \"i\" #> $ name : chr \"\" #> $ metadata : list() #> $ flags : int 2 #> $ children : list() #> $ dictionary: NULL na_struct(list(col1 = na_int32())) #> <nanoarrow_schema struct> #> $ format : chr \"+s\" #> $ name : chr \"\" #> $ metadata : list() #> $ flags : int 0 #> $ children :List of 1 #> ..$ col1:<nanoarrow_schema int32> #> .. ..$ format : chr \"i\" #> .. ..$ name : chr \"col1\" #> .. ..$ metadata : list() #> .. ..$ flags : int 2 #> .. ..$ children : list() #> .. ..$ dictionary: NULL #> $ dictionary: NULL"},{"path":"/reference/na_vctrs.html","id":null,"dir":"Reference","previous_headings":"","what":"Vctrs extension type — na_vctrs","title":"Vctrs extension type — na_vctrs","text":"Arrow format provides rich type system can handle R vector types; however, many R vector types roundtrip perfectly Arrow memory. vctrs extension type uses vctrs::vec_data(), vctrs::vec_restore(), vctrs::vec_ptype() calls as_nanoarrow_array() convert_array() ensure roundtrip fidelity.","code":""},{"path":"/reference/na_vctrs.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Vctrs extension type — na_vctrs","text":"","code":"na_vctrs(ptype, storage_type = NULL)"},{"path":"/reference/na_vctrs.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Vctrs extension type — na_vctrs","text":"ptype vctrs prototype returned vctrs::vec_ptype(). prototype can arbitrary size, zero-size vector sufficient . storage_type na_extension(), underlying value type.","code":""},{"path":"/reference/na_vctrs.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Vctrs extension type — na_vctrs","text":"nanoarrow_schema.","code":""},{"path":"/reference/na_vctrs.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Vctrs extension type — na_vctrs","text":"","code":"vctr <- as.POSIXlt(\"2000-01-02 03:45\", tz = \"UTC\") array <- as_nanoarrow_array(vctr, schema = na_vctrs(vctr)) infer_nanoarrow_ptype(array) #> POSIXlt of length 0 convert_array(array) #> [1] \"2000-01-02 03:45:00 UTC\""},{"path":"/reference/nanoarrow-package.html","id":null,"dir":"Reference","previous_headings":"","what":"nanoarrow: Interface to the 'nanoarrow' 'C' Library — nanoarrow-package","title":"nanoarrow: Interface to the 'nanoarrow' 'C' Library — nanoarrow-package","text":"Provides 'R' interface 'nanoarrow' 'C' library 'Apache Arrow' application binary interface. Functions import export 'ArrowArray', 'ArrowSchema', 'ArrowArrayStream' 'C' structures 'R' objects provided alongside helpers facilitate zero-copy data transfer among 'R' bindings libraries implementing 'Arrow' 'C' data interface.","code":""},{"path":[]},{"path":"/reference/nanoarrow-package.html","id":"author","dir":"Reference","previous_headings":"","what":"Author","title":"nanoarrow: Interface to the 'nanoarrow' 'C' Library — nanoarrow-package","text":"Maintainer: Dewey Dunnington dewey@dunnington.ca (ORCID) Authors: Apache Arrow dev@arrow.apache.org [copyright holder] contributors: Apache Software Foundation dev@arrow.apache.org [copyright holder]","code":""},{"path":"/reference/nanoarrow_array_init.html","id":null,"dir":"Reference","previous_headings":"","what":"Modify nanoarrow arrays — nanoarrow_array_init","title":"Modify nanoarrow arrays — nanoarrow_array_init","text":"Create new array existing array, modify one parameters. importing array elsewhere, nanoarrow_array_set_schema() useful attach data type information array (without information little nanoarrow can array since content otherwise interpreted). nanoarrow_array_modify() can create shallow copy modify various parameters create new array, including setting children buffers recursively. functions power $<- operator, can modify one parameter time.","code":""},{"path":"/reference/nanoarrow_array_init.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Modify nanoarrow arrays — nanoarrow_array_init","text":"","code":"nanoarrow_array_init(schema) nanoarrow_array_set_schema(array, schema, validate = TRUE) nanoarrow_array_modify(array, new_values, validate = TRUE)"},{"path":"/reference/nanoarrow_array_init.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Modify nanoarrow arrays — nanoarrow_array_init","text":"schema nanoarrow_schema attach array. array nanoarrow_array. validate Use FALSE skip validation. Skipping validation may result creating array crash R. new_values named list() values replace.","code":""},{"path":"/reference/nanoarrow_array_init.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Modify nanoarrow arrays — nanoarrow_array_init","text":"nanoarrow_array_init() returns possibly invalid initialized array given schema. nanoarrow_array_set_schema() returns array, invisibly. Note array modified place reference. nanoarrow_array_modify() returns shallow copy array modified parameters original array remains valid.","code":""},{"path":"/reference/nanoarrow_array_init.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Modify nanoarrow arrays — nanoarrow_array_init","text":"","code":"nanoarrow_array_init(na_string()) #> <nanoarrow_array string[0]> #> $ length : int 0 #> $ null_count: int 0 #> $ offset : int 0 #> $ buffers :List of 3 #> ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> ..$ :<nanoarrow_buffer data_offset<int32>[0][0 b]> `` #> ..$ :<nanoarrow_buffer data<string>[0 b]> `` #> $ dictionary: NULL #> $ children : list() # Modify an array using $ and <- array <- as_nanoarrow_array(1:5) array$length <- 4 as.vector(array) #> [1] 1 2 3 4 # Modify potentially more than one component at a time array <- as_nanoarrow_array(1:5) as.vector(nanoarrow_array_modify(array, list(length = 4))) #> [1] 1 2 3 4 # Attach a schema to an array array <- as_nanoarrow_array(-1L) nanoarrow_array_set_schema(array, na_uint32()) as.vector(array) #> [1] 4294967295"},{"path":"/reference/nanoarrow_buffer_init.html","id":null,"dir":"Reference","previous_headings":"","what":"Create and modify nanoarrow buffers — nanoarrow_buffer_init","title":"Create and modify nanoarrow buffers — nanoarrow_buffer_init","text":"Create modify nanoarrow buffers","code":""},{"path":"/reference/nanoarrow_buffer_init.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Create and modify nanoarrow buffers — nanoarrow_buffer_init","text":"","code":"nanoarrow_buffer_init() nanoarrow_buffer_append(buffer, new_buffer) convert_buffer(buffer, to = NULL)"},{"path":"/reference/nanoarrow_buffer_init.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Create and modify nanoarrow buffers — nanoarrow_buffer_init","text":"buffer, new_buffer nanoarrow_buffers. target prototype object describing type array converted, NULL use default conversion returned infer_nanoarrow_ptype(). Alternatively, function can passed perform alternative calculation default ptype function array default inference prototype.","code":""},{"path":"/reference/nanoarrow_buffer_init.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Create and modify nanoarrow buffers — nanoarrow_buffer_init","text":"nanoarrow_buffer_init(): object class 'nanoarrow_buffer' nanoarrow_buffer_append(): Returns buffer, invisibly. Note buffer modified place reference.","code":""},{"path":"/reference/nanoarrow_buffer_init.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Create and modify nanoarrow buffers — nanoarrow_buffer_init","text":"","code":"buffer <- nanoarrow_buffer_init() nanoarrow_buffer_append(buffer, 1:5) array <- nanoarrow_array_modify( nanoarrow_array_init(na_int32()), list(length = 5, buffers = list(NULL, buffer)) ) as.vector(array) #> [1] 1 2 3 4 5"},{"path":"/reference/nanoarrow_extension_array.html","id":null,"dir":"Reference","previous_headings":"","what":"Create Arrow extension arrays — nanoarrow_extension_array","title":"Create Arrow extension arrays — nanoarrow_extension_array","text":"Create Arrow extension arrays","code":""},{"path":"/reference/nanoarrow_extension_array.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Create Arrow extension arrays — nanoarrow_extension_array","text":"","code":"nanoarrow_extension_array( storage_array, extension_name, extension_metadata = NULL )"},{"path":"/reference/nanoarrow_extension_array.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Create Arrow extension arrays — nanoarrow_extension_array","text":"storage_array nanoarrow_array. extension_name na_extension(), extension name. typically namespaced separated dots (e.g., arrow.r.vctrs). extension_metadata string raw vector defining extension metadata. Arrow extension types define extension metadata JSON object.","code":""},{"path":"/reference/nanoarrow_extension_array.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Create Arrow extension arrays — nanoarrow_extension_array","text":"nanoarrow_array attached extension schema.","code":""},{"path":"/reference/nanoarrow_extension_array.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Create Arrow extension arrays — nanoarrow_extension_array","text":"","code":"nanoarrow_extension_array(1:10, \"some_ext\", '{\"key\": \"value\"}') #> <nanoarrow_array some_ext{int32}[10]> #> $ length : int 10 #> $ null_count: int 0 #> $ offset : int 0 #> $ buffers :List of 2 #> ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> `` #> ..$ :<nanoarrow_buffer data<int32>[10][40 b]> `1 2 3 4 5 6 7 8 9 10` #> $ dictionary: NULL #> $ children : list()"},{"path":"/reference/nanoarrow_extension_spec.html","id":null,"dir":"Reference","previous_headings":"","what":"Register Arrow extension types — nanoarrow_extension_spec","title":"Register Arrow extension types — nanoarrow_extension_spec","text":"Register Arrow extension types","code":""},{"path":"/reference/nanoarrow_extension_spec.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Register Arrow extension types — nanoarrow_extension_spec","text":"","code":"nanoarrow_extension_spec(data = list(), subclass = character()) register_nanoarrow_extension(extension_name, extension_spec) unregister_nanoarrow_extension(extension_name) resolve_nanoarrow_extension(extension_name)"},{"path":"/reference/nanoarrow_extension_spec.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Register Arrow extension types — nanoarrow_extension_spec","text":"data Optional data include extension type specification subclass subclass extension type specification. Extension methods dispatch object. extension_name Arrow extension type name (e.g., arrow.r.vctrs) extension_spec extension specification inheriting 'nanoarrow_extension_spec'.","code":""},{"path":"/reference/nanoarrow_extension_spec.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Register Arrow extension types — nanoarrow_extension_spec","text":"nanoarrow_extension_spec() returns object class 'nanoarrow_extension_spec'. register_nanoarrow_extension() returns extension_spec, invisibly. unregister_nanoarrow_extension() returns extension_name, invisibly. resolve_nanoarrow_extension() returns object class 'nanoarrow_extension_spec' NULL extension type registered.","code":""},{"path":"/reference/nanoarrow_extension_spec.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Register Arrow extension types — nanoarrow_extension_spec","text":"","code":"nanoarrow_extension_spec(\"mynamespace.mytype\", subclass = \"mypackage_mytype_spec\") #> [1] \"mynamespace.mytype\" #> attr(,\"class\") #> [1] \"mypackage_mytype_spec\" \"nanoarrow_extension_spec\""},{"path":"/reference/nanoarrow_pointer_is_valid.html","id":null,"dir":"Reference","previous_headings":"","what":"Danger zone: low-level pointer operations — nanoarrow_pointer_is_valid","title":"Danger zone: low-level pointer operations — nanoarrow_pointer_is_valid","text":"nanoarrow_schema, nanoarrow_array, nanoarrow_array_stream classes represented R external pointers (EXTPTRSXP). objects go scope (.e., garbage collected shortly thereafter), underlying object's release() callback called underlying pointer non-null release() callback non-null.","code":""},{"path":"/reference/nanoarrow_pointer_is_valid.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Danger zone: low-level pointer operations — nanoarrow_pointer_is_valid","text":"","code":"nanoarrow_pointer_is_valid(ptr) nanoarrow_pointer_addr_dbl(ptr) nanoarrow_pointer_addr_chr(ptr) nanoarrow_pointer_addr_pretty(ptr) nanoarrow_pointer_release(ptr) nanoarrow_pointer_move(ptr_src, ptr_dst) nanoarrow_pointer_export(ptr_src, ptr_dst) nanoarrow_allocate_schema() nanoarrow_allocate_array() nanoarrow_allocate_array_stream() nanoarrow_pointer_set_protected(ptr_src, protected)"},{"path":"/reference/nanoarrow_pointer_is_valid.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Danger zone: low-level pointer operations — nanoarrow_pointer_is_valid","text":"ptr, ptr_src, ptr_dst external pointer struct ArrowSchema, struct ArrowArray, struct ArrowArrayStream. protected object whose scope must outlive ptr. useful array streams since least two specifications involving array stream specify stream valid lifecycle another object (e.g., AdbcStatement OGRDataset).","code":""},{"path":"/reference/nanoarrow_pointer_is_valid.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Danger zone: low-level pointer operations — nanoarrow_pointer_is_valid","text":"nanoarrow_pointer_is_valid() returns TRUE pointer non-null non-null release callback. nanoarrow_pointer_addr_dbl() nanoarrow_pointer_addr_chr() return pointer representations may helpful facilitate moving exporting nanoarrow objects libraries. nanoarrow_pointer_addr_pretty() gives pointer representation suitable printing error messages. nanoarrow_pointer_release() returns ptr, invisibly. nanoarrow_pointer_move() nanoarrow_pointer_export() reeturn ptr_dst, invisibly. nanoarrow_allocate_array(), nanoarrow_allocate_schema(), nanoarrow_allocate_array_stream() return array, schema, array stream, respectively.","code":""},{"path":"/reference/nanoarrow_pointer_is_valid.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Danger zone: low-level pointer operations — nanoarrow_pointer_is_valid","text":"interacting C Data Interface implementations, important keep mind R object wrapping pointers always passed reference (external pointer) may referred another R object (e.g., element list() variable assigned user's environment). importing schema, array, array stream nanoarrow problem: R object takes ownership lifecycle memory released R object garbage collected. case, one can use nanoarrow_pointer_move() ptr_dst created using nanoarrow_allocate_*(). case exporting complicated dedicated function, nanoarrow_pointer_export(), implements different logic schemas, arrays, array streams: Schema objects (deep) copied fresh copy schema exported made responsibility C data interface implementation. Array objects exported shell around original array preserves reference R object. ensures buffers children pointed array copied references original array invalidated. Array stream objects moved: responsibility object transferred C data interface implementation references original R object invalidated. objects mutable, typically want (.e., pulling arrays stream accidentally two places). know lifecycle object (.e., created R object never passed references elsewhere), can slightly efficiently call nanoarrow_pointer_move() three pointer types.","code":""},{"path":"/reference/nanoarrow_version.html","id":null,"dir":"Reference","previous_headings":"","what":"Underlying 'nanoarrow' C library build — nanoarrow_version","title":"Underlying 'nanoarrow' C library build — nanoarrow_version","text":"Underlying 'nanoarrow' C library build","code":""},{"path":"/reference/nanoarrow_version.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Underlying 'nanoarrow' C library build — nanoarrow_version","text":"","code":"nanoarrow_version(runtime = TRUE)"},{"path":"/reference/nanoarrow_version.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Underlying 'nanoarrow' C library build — nanoarrow_version","text":"runtime Compare TRUE FALSE values detect possible ABI mismatch.","code":""},{"path":"/reference/nanoarrow_version.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Underlying 'nanoarrow' C library build — nanoarrow_version","text":"string identifying version nanoarrow package compiled .","code":""},{"path":"/reference/nanoarrow_version.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Underlying 'nanoarrow' C library build — nanoarrow_version","text":"","code":"nanoarrow_version() #> [1] \"0.6.0-SNAPSHOT\""},{"path":"/reference/read_nanoarrow.html","id":null,"dir":"Reference","previous_headings":"","what":"Read serialized streams of Arrow data — read_nanoarrow","title":"Read serialized streams of Arrow data — read_nanoarrow","text":"Reads connections, file paths, URLs, raw vectors serialized Arrow data. Arrow documentation typically refers format \"Arrow IPC\", since origin means transmit tables processes (e.g., multiple R sessions). format can also written read files URLs essentially high performance equivalent CSV file better job maintaining types.","code":""},{"path":"/reference/read_nanoarrow.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Read serialized streams of Arrow data — read_nanoarrow","text":"","code":"read_nanoarrow(x, ..., lazy = FALSE) example_ipc_stream()"},{"path":"/reference/read_nanoarrow.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Read serialized streams of Arrow data — read_nanoarrow","text":"x raw() vector, connection, file path read binary data. Common extensions indicating compression (.gz, .bz2, .zip) automatically uncompressed. ... Currently unused. lazy default, read_nanoarrow() read discard copy reader's schema ensure invalid streams discovered soon possible. Use lazy = TRUE defer check reader actually consumed.","code":""},{"path":"/reference/read_nanoarrow.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Read serialized streams of Arrow data — read_nanoarrow","text":"nanoarrow_array_stream","code":""},{"path":"/reference/read_nanoarrow.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Read serialized streams of Arrow data — read_nanoarrow","text":"nanoarrow package currently ability write serialized IPC data: use arrow::write_ipc_stream() write data R, use equivalent writer another Arrow implementation Python, C++, Rust, JavaScript, Julia, C#, beyond. media type Arrow stream application/vnd.apache.arrow.stream recommended file extension .arrows.","code":""},{"path":"/reference/read_nanoarrow.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Read serialized streams of Arrow data — read_nanoarrow","text":"","code":"as.data.frame(read_nanoarrow(example_ipc_stream())) #> some_col #> 1 1 #> 2 2 #> 3 3"},{"path":"/news/index.html","id":"nanoarrow-040","dir":"Changelog","previous_headings":"","what":"nanoarrow 0.4.0","title":"nanoarrow 0.4.0","text":"CRAN release: 2024-02-01 Fix source links pkgdown site (#315). Provide LinkingTo headers extension packages (#332). Add nanoarrow_array_stream generics (#349). Add conversion integer type character() (#345). Ensure simple list()s can converted without arrow installed (#344).","code":""},{"path":"/news/index.html","id":"nanoarrow-0301","dir":"Changelog","previous_headings":"","what":"nanoarrow 0.3.0.1","title":"nanoarrow 0.3.0.1","text":"CRAN release: 2023-12-08 Ensure wrapper array stream eagerly releases wrapped array stream (#333).","code":""},{"path":"/news/index.html","id":"nanoarrow-030","dir":"Changelog","previous_headings":"","what":"nanoarrow 0.3.0","title":"nanoarrow 0.3.0","text":"CRAN release: 2023-09-29 Use classed warnings signal lossy conversion occurred (#298) Add support bit64::integer64() conversions (#293) Implement extension type registration/conversion (#288) Implement dictionary conversion (#285) Ensure ordered reflected na_dictionary() (#299) Warn possibly range int64 -> double conversions (#294) Support map conversion R vector (#282) Don’t link arrow package R6 class pages (#269) Use basic_array_stream() improve array stream data.frame conversion (#279)","code":""},{"path":"/news/index.html","id":"nanoarrow-020-1","dir":"Changelog","previous_headings":"","what":"nanoarrow 0.2.0-1","title":"nanoarrow 0.2.0-1","text":"Don’t link arrow package R6 class pages (#269)","code":""},{"path":[]},{"path":"/news/index.html","id":"new-features-0-2-0","dir":"Changelog","previous_headings":"","what":"New features","title":"nanoarrow 0.2.0","text":"Improve printing conversion buffers (#208) Add enum ArrowType buffer_data_type member struct ArrowLayout (#207) Implement ListChildOffset function (#197) Add ability deterministically run finalizer array stream (#196) Union array support (#195) Add ArrowArrayStream implementation support keeping dependent object scope (#194) Add as_nanoarrow_array() implementation fall back arrow::as_arrow_array() everywhere (#108) Create nanoarrow_array objects buffers (#105) Implement infer schema methods (#104) Create modify nanoarrow_schema objects (#101)","code":""},{"path":"/news/index.html","id":"bugfixes-0-2-0","dir":"Changelog","previous_headings":"","what":"Bugfixes","title":"nanoarrow 0.2.0","text":"Fix convert_array_stream() non-record batch stream zero batches (#212) clear release EmptyArrayStream::release_wrapper (#204) Release streams calling .vector() .data.frame() (#202) Don’t invoke undefined behaviour conversions /Arrow (#167) Use strict prototypes internal C functions (#151) Don’t memcpy NULL converting buffer raw (#149)","code":""}] |