| commit | 6d683edd806baf538378ef1203a472e299b66e7f | [log] [tgz] |
|---|---|---|
| author | dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> | Fri Aug 15 15:40:13 2025 +0200 |
| committer | GitHub <noreply@github.com> | Fri Aug 15 15:40:13 2025 +0200 |
| tree | 7e82eb0e3090d62ed8a87ce76bb981e02888f9a8 | |
| parent | 3953d64636fe8fc9c1c7e08f881adfa35ca74665 [diff] |
Bump github.com/apache/arrow-go/v18 from 18.3.0 to 18.4.0 Bumps [github.com/apache/arrow-go/v18](https://github.com/apache/arrow-go) from 18.3.0 to 18.4.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/apache/arrow-go/releases">github.com/apache/arrow-go/v18's releases</a>.</em></p> <blockquote> <h2>v18.4.0</h2> <h2>What's Changed</h2> <ul> <li>feat(arrow/cdata): Add ReleaseCArrowArrayStream function by <a href="https://github.com/karsov"><code>karsov</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/373">apache/arrow-go#373</a></li> <li>chore: Bump golang.org/x/sync from 0.13.0 to 0.14.0 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/368">apache/arrow-go#368</a></li> <li>fix: TestDeltaByteArray implementation and fix by <a href="https://github.com/MetalBlueberry"><code>MetalBlueberry</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/369">apache/arrow-go#369</a></li> <li>chore: Bump golang.org/x/tools from 0.32.0 to 0.33.0 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/376">apache/arrow-go#376</a></li> <li>chore: Bump actions/setup-go from 5.4.0 to 5.5.0 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/377">apache/arrow-go#377</a></li> <li>chore: move .github/ISSUE_TEMPLATE/config.yaml to config.yml as currently does not work by <a href="https://github.com/raulcd"><code>raulcd</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/383">apache/arrow-go#383</a></li> <li>chore: Bump google.golang.org/grpc from 1.72.0 to 1.72.1 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/381">apache/arrow-go#381</a></li> <li>fix: list_columns.parquet testing by <a href="https://github.com/MetalBlueberry"><code>MetalBlueberry</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/378">apache/arrow-go#378</a></li> <li>feat: Extend arrow csv writter by <a href="https://github.com/MetalBlueberry"><code>MetalBlueberry</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/375">apache/arrow-go#375</a></li> <li>feat(parquet/pqarrow): parallelize SeekToRow by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/380">apache/arrow-go#380</a></li> <li>chore: Use apache/arrow-js for JS in integration test by <a href="https://github.com/kou"><code>kou</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/389">apache/arrow-go#389</a></li> <li>chore: Bump github.com/apache/thrift from 0.21.0 to 0.22.0 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/387">apache/arrow-go#387</a></li> <li>feat(parquet): add variant encoder/decoder by <a href="https://github.com/sfc-gh-mbojanczyk"><code>sfc-gh-mbojanczyk</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/344">apache/arrow-go#344</a></li> <li>chore: remove extra binary by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/391">apache/arrow-go#391</a></li> <li>CI: add benchmark workflow and script by <a href="https://github.com/singh1203"><code>singh1203</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/250">apache/arrow-go#250</a></li> <li>chore: Bump actions/setup-python from 4 to 5 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/396">apache/arrow-go#396</a></li> <li>feat(arrow/extensions): Add Variant extension type, array, and builder by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/395">apache/arrow-go#395</a></li> <li>fix(parquet/pqarrow): Fix propagation of field-ids for Lists by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/397">apache/arrow-go#397</a></li> <li>feat(arrow/_examples): enhance library examples by <a href="https://github.com/kris-gaudel"><code>kris-gaudel</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/394">apache/arrow-go#394</a></li> <li>chore: bump Windows GitHub hosted runner to windows-2022 by <a href="https://github.com/raulcd"><code>raulcd</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/407">apache/arrow-go#407</a></li> <li>chore: Bump golang.org/x/tools from 0.33.0 to 0.34.0 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/398">apache/arrow-go#398</a></li> <li>chore: Bump golang.org/x/sync from 0.14.0 to 0.15.0 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/400">apache/arrow-go#400</a></li> <li>chore: Bump github.com/pterm/pterm from 0.12.80 to 0.12.81 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/402">apache/arrow-go#402</a></li> <li>chore: Bump google.golang.org/grpc from 1.72.1 to 1.73.0 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/401">apache/arrow-go#401</a></li> <li>ci(benchmark): Fix benchmark runs by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/409">apache/arrow-go#409</a></li> <li>ci: fix flaky test by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/411">apache/arrow-go#411</a></li> <li>ci: make additional checks to prevent flaky EOF by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/414">apache/arrow-go#414</a></li> <li>refactor: update linter and run it by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/419">apache/arrow-go#419</a></li> <li>chore: Bump github.com/stoewer/go-strcase from 1.3.0 to 1.3.1 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/423">apache/arrow-go#423</a></li> <li>chore: Bump github.com/hamba/avro/v2 from 2.28.0 to 2.29.0 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/422">apache/arrow-go#422</a></li> <li>feat: expose Payload.WritePayload to allow serializing into IPC format by <a href="https://github.com/alvarowolfx"><code>alvarowolfx</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/421">apache/arrow-go#421</a></li> <li>feat(parquet/variant): Parse JSON into variant by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/426">apache/arrow-go#426</a></li> <li>chore: Bump github.com/go-viper/mapstructure/v2 from 2.2.1 to 2.3.0 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/427">apache/arrow-go#427</a></li> <li>refactor(parquet/internal/encoding): Refactor parquet logic to use generics by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/428">apache/arrow-go#428</a></li> <li>feat(arrrow/compute/expr): support substrait timestamp and decimal properly by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/418">apache/arrow-go#418</a></li> <li>chore: Bump github.com/klauspost/cpuid/v2 from 2.2.10 to 2.2.11 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/431">apache/arrow-go#431</a></li> <li>chore: Bump github.com/andybalholm/brotli from 1.1.1 to 1.2.0 by <a href="https://github.com/dependabot"><code>dependabot</code></a>[bot] in <a href="https://redirect.github.com/apache/arrow-go/pull/432">apache/arrow-go#432</a></li> <li>feat(parquet/examples): enhance library examples by <a href="https://github.com/milden6"><code>milden6</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/429">apache/arrow-go#429</a></li> <li>feat(arrow/compute): support some float16 casts by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/430">apache/arrow-go#430</a></li> <li>feat(parquet/pqarrow): Correctly handle Variant types in schema by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/433">apache/arrow-go#433</a></li> <li>fix(arrow/avro-reader): bunch of types that didn't work by <a href="https://github.com/Willem-J-an"><code>Willem-J-an</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/416">apache/arrow-go#416</a></li> <li>feat(parquet/pqarrow): read/write variant by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/434">apache/arrow-go#434</a></li> <li>build(deps): update to substrait-go v4.3.0 by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/438">apache/arrow-go#438</a></li> <li>fix(arrow/flight/flightsql): drain channel in flightSqlServer.DoGet by <a href="https://github.com/arnoldwakim"><code>arnoldwakim</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/437">apache/arrow-go#437</a></li> <li>chore(arrow): Update PkgVersion by <a href="https://github.com/zeroshade"><code>zeroshade</code></a> in <a href="https://redirect.github.com/apache/arrow-go/pull/439">apache/arrow-go#439</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/karsov"><code>karsov</code></a> made their first contribution in <a href="https://redirect.github.com/apache/arrow-go/pull/373">apache/arrow-go#373</a></li> </ul> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/arrow-go/commit/abad137427940ef65d5d5c433a329a5d737a013d"><code>abad137</code></a> chore(arrow): Update PkgVersion (<a href="https://redirect.github.com/apache/arrow-go/issues/439">#439</a>)</li> <li><a href="https://github.com/apache/arrow-go/commit/f2773ada1df41f55fb66a4d13eda59b7a0567b31"><code>f2773ad</code></a> fix(arrow/flight/flightsql): drain channel in flightSqlServer.DoGet (<a href="https://redirect.github.com/apache/arrow-go/issues/437">#437</a>)</li> <li><a href="https://github.com/apache/arrow-go/commit/542afb689c7b20149945a9be5107ecf690efa642"><code>542afb6</code></a> build(deps): update to substrait-go v4.3.0 (<a href="https://redirect.github.com/apache/arrow-go/issues/438">#438</a>)</li> <li><a href="https://github.com/apache/arrow-go/commit/7ce3c03fbc268cb67dee53676c48e9bd74be1684"><code>7ce3c03</code></a> feat(parquet/pqarrow): read/write variant (<a href="https://redirect.github.com/apache/arrow-go/issues/434">#434</a>)</li> <li><a href="https://github.com/apache/arrow-go/commit/b196d3b316d09f63786f021d4f1baa1fdd7620d2"><code>b196d3b</code></a> fix(arrow/avro-reader): bunch of types that didn't work (<a href="https://redirect.github.com/apache/arrow-go/issues/416">#416</a>)</li> <li><a href="https://github.com/apache/arrow-go/commit/2cf2b297672f37895c83a42bfaa1ac08352efc16"><code>2cf2b29</code></a> feat(parquet/pqarrow): Correctly handle Variant types in schema (<a href="https://redirect.github.com/apache/arrow-go/issues/433">#433</a>)</li> <li><a href="https://github.com/apache/arrow-go/commit/8598fb3bd433b713424c4390fc9b785fc33fa5ee"><code>8598fb3</code></a> feat(arrow/compute): support some float16 casts (<a href="https://redirect.github.com/apache/arrow-go/issues/430">#430</a>)</li> <li><a href="https://github.com/apache/arrow-go/commit/45e47c7cc8183c8cb2b3162d9e548480e5778592"><code>45e47c7</code></a> feat(parquet/examples): enhance library examples (<a href="https://redirect.github.com/apache/arrow-go/issues/429">#429</a>)</li> <li><a href="https://github.com/apache/arrow-go/commit/084371ed663ee877561f64749e5f4f7d8cf454d3"><code>084371e</code></a> chore: Bump github.com/andybalholm/brotli from 1.1.1 to 1.2.0 (<a href="https://redirect.github.com/apache/arrow-go/issues/432">#432</a>)</li> <li><a href="https://github.com/apache/arrow-go/commit/450dcd770cd5f23ccddc957731841b2e67f87db8"><code>450dcd7</code></a> chore: Bump github.com/klauspost/cpuid/v2 from 2.2.10 to 2.2.11 (<a href="https://redirect.github.com/apache/arrow-go/issues/431">#431</a>)</li> <li>Additional commits viewable in <a href="https://github.com/apache/arrow-go/compare/v18.3.0...v18.4.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Closes #156 from dependabot[bot]/dependabot/go_modules/github.com/apache/arrow-go/v18-18.4.0. Authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Martin Grund <martin.grund@databricks.com>
This project houses the experimental client for Spark Connect for Apache Spark written in Golang.
Currently, the Spark Connect client for Golang is highly experimental and should not be used in any production setting. In addition, the PMC of the Apache Spark project reserves the right to withdraw and abandon the development of this project if it is not sustainable.
This section explains how to run Spark Connect Go locally.
Step 1: Install Golang: https://go.dev/doc/install.
Step 2: Ensure you have installed buf CLI installed, more info here
Step 3: Run the following commands to setup the Spark Connect client.
Building with Spark in case you need to re-generate the source files from the proto sources.
git clone https://github.com/apache/spark-connect-go.git git submodule update --init --recursive make gen && make test
Building without Spark
git clone https://github.com/apache/spark-connect-go.git make && make test
Step 4: Setup the Spark Driver on localhost.
Download Spark distribution (4.0.0+), unzip the package.
Start the Spark Connect server with the following command (make sure to use a package version that matches your Spark distribution):
sbin/start-connect-server.sh
Step 5: Run the example Go application.
go run cmd/spark-connect-example-spark-session/main.go
To run the Spark Connect Go application in a Spark Cluster, you need to build the Go application and submit it to the Spark Cluster. You can find a more detailed example runner and wrapper script in the java directory.
See the guide here: Sample Spark-Submit Wrapper.
The overall goal of the design is to find a good balance of principle of the least surprise for develoeprs that are familiar with the APIs of Apache Spark and idiomatic Go usage. The high-level structure of the packages follows roughly the PySpark giudance but with Go idioms.
Please review the Contribution to Spark guide for information on how to get started contributing to the project.