| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| --> |
| |
| # Updating the DataFusion / protobuf schema version |
| |
| Three things must move together when bumping DataFusion: |
| |
| 1. `Cargo.toml` (workspace root) — the `datafusion`, `datafusion-proto`, |
| `datafusion-spark`, and `datafusion-substrait` entries in |
| `[workspace.dependencies]`. Members inherit from there. |
| 2. `pom.xml` — the `<datafusion.version>` Maven property. **Must equal |
| the Cargo version**; a mismatch means JVM-built protobuf plans won't |
| deserialize on the native side. |
| 3. `pom.xml` — the `<sha512>` checksums on the two `download-maven-plugin` |
| executions. These pin the downloaded `.proto` files; the build fails |
| if upstream silently re-tags them, which is the desired behavior. |
| |
| ## Recipe |
| |
| ```sh |
| # 1. Bump the workspace dep |
| $EDITOR Cargo.toml # set datafusion = "<new>" in [workspace.dependencies] |
| cargo update -p datafusion |
| |
| # 2. Bump the Maven property to match |
| $EDITOR pom.xml # set <datafusion.version> |
| |
| # 3. Compute the new SHA-512 hashes for both `.proto` files from the |
| # upstream tag you just set in step 2, then paste them into the two |
| # <sha512> elements in pom.xml. |
| NEW=$(grep -m1 -oE '<datafusion.version>[^<]+' pom.xml | cut -d'>' -f2) |
| curl -sL "https://raw.githubusercontent.com/apache/datafusion/$NEW/datafusion/proto-common/proto/datafusion_common.proto" | shasum -a 512 | awk '{print $1}' |
| curl -sL "https://raw.githubusercontent.com/apache/datafusion/$NEW/datafusion/proto/proto/datafusion.proto" | shasum -a 512 | awk '{print $1}' |
| $EDITOR pom.xml # paste the two hashes into the <sha512> elements |
| |
| # Drop the local download cache so the next build re-downloads against |
| # the new hashes. |
| rm -rf ~/.m2/repository/.cache/download-maven-plugin target/proto |
| |
| # 4. Verify |
| make && make test |
| ``` |
| |
| ## Why the protobuf runtime version is separate |
| |
| The protobuf runtime version (`<protobuf.version>` in `pom.xml`) tracks |
| the Java ecosystem (security and JDK compatibility), not DataFusion. |
| Bump it independently when there is a reason. |