Policy on source versus distribution

Apache Burr is an apache-incubating project. As such, we intend to follow all Apache guidelines to both the spirit (and when applicable) the letter.

That said, there is occasional ambiguity. Thus we aim to clarify with a reasonable and consistently maintained approach. The question that we found most ambiguous when determining our release process is

What counts as source code, and should thus be included in the “sdist” (the source-only distribution)
What should be included in the build?

Specifically, we set the following guidelines:

	source (to vote on) -- tar.gz	sdist -- source used to build	whl file	Reasoning
Build Scripts	✓	✓	✗	Included in tar.gz and sdist as they are needed to reproduce the build, but not in the whl. These are only meant to be consumed by developers/pod members.
Library Source code	✓	✓	✓	Core library source code is included in all three distributions: tar.gz, sdist, and whl.
Tests (integration and unit)	✓	✓	✗	We expect users/PMC to download the source distribution, build from source, run the tests, and validate. Thus we include in the tar.gz and sdist, but not in the whl.
READMEs	✓	✓	✓	Standard project metadata files (README.md, LICENSE, NOTICE, DISCLAIMER) are included in all three distributions: tar.gz, sdist, and whl.
Documentation	✓	✗	✗	Documentation source is included in the tar.gz for voters to review, but not in the sdist or whl as it is not needed for building or using the package.
Deployment templates	✓	✓	✓	Convenience deployment templates are included in tar.gz, sdist, and whl as they are referred to by specific utility commands for deploying that are included in source.
Built artifacts (UI, etc...)	✗	✗	✓	These are not source code and are only included in the whl. They are created through a build process from the UI source. Notable examples include the built npm packages.
Examples (by default required for demo server)	✓	✓	✓	We have four examples (see pyproject.toml) required by the demo server which can be run by a single command. These are included in tar.gz, sdist, and whl as they are needed for the demo functionality.
Other examples	✓	✗	✗	These are included in the tar.gz for voters to review but not included in the sdist or whl as they are not needed to build or run the package. They serve more as documentation.

Release Process

Note: This is a work in progress and subject to change.

Environment Setup

Prerequisites:

Python 3.9+
flit for building (pip install flit)
twine for package validation (pip install twine)
GPG key configured for signing
Node.js + npm for UI builds
Apache RAT jar for license checking (optional)

# Install build dependencies
pip install flit
pip install -e ".[cli]"  # Installs burr-admin-build-ui command

# Verify GPG setup
gpg --list-secret-keys

# Build UI assets (one-time or when UI changes)
cd telemetry/ui && npm install && npm run build && cd ../..

Building Artifacts

Creates the three required distributions: git archive (voting artifact), sdist (source distribution), and wheel (binary distribution). All artifacts are automatically signed with GPG and checksummed with SHA512. The all command is the typical workflow - it builds everything in sequence.

Main release script: scripts/apache_release.py

# Full release build (creates all artifacts, signs, checksums, and generates vote email)
# Note: version, rc_num, and apache_id are POSITIONAL arguments
python scripts/apache_release.py all 0.41.0 0 your_apache_id

# Individual steps
python scripts/apache_release.py archive 0.41.0 0      # Git archive
python scripts/apache_release.py sdist 0.41.0 0        # Source dist
python scripts/apache_release.py wheel 0.41.0 0        # Wheel dist

# Upload to SVN
python scripts/apache_release.py upload 0.41.0 0 your_apache_id
python scripts/apache_release.py upload 0.41.0 0 your_apache_id --dry-run  # Test first

# Promote a voted RC from dist/dev to dist/release
python scripts/apache_release.py promote 0.41.0-RC0 your_apache_id
python scripts/apache_release.py promote 0.41.0-RC0 your_apache_id --dry-run
python scripts/apache_release.py promote 0.41.0-RC0 your_apache_id --release-svn-root https://dist.apache.org/repos/dist/release/burr  # TLP path override

# Verify artifacts locally
python scripts/apache_release.py verify 0.41.0 0

# Skip upload step in 'all' command
python scripts/apache_release.py all 0.41.0 0 your_apache_id --no-upload

# Generate release emails from templates
python scripts/apache_release.py vote-email --version 0.41.0 --rc 0
python scripts/apache_release.py result-email --version 0.41.0 --rc 0 --binding-yes 3 --non-binding-yes 2 --binding-no 0 --non-binding-no 1
python scripts/apache_release.py announce-email --version 0.41.0

Output: dist/ directory with tar.gz (archive + sdist), whl, plus .asc and .sha512 files. The wheel is validated with twine check to ensure metadata correctness before signing. Install from the whl file to test it out after running the wheel subcommand.

Promoting a voted RC

After an RC vote passes, promote the exact voted artifacts from Apache SVN dist/dev into dist/release with:

python scripts/apache_release.py promote 0.41.0-RC0 your_apache_id

What it does:

checks out the RC directory from dist/dev to validate the expected source archive, sdist, wheel, and matching .asc / .sha512 files
refuses to continue if the target release directory already exists
copies the voted RC directory server-side into a new per-version release directory (e.g. dist/release/incubator/burr/0.41.0) with a single atomic svn cp commit
leaves any existing release directories and the shared KEYS file untouched (additive)
prints the final PyPI upload command for the sdist and wheel

Use --dry-run to preview the actions without committing:

python scripts/apache_release.py promote 0.41.0-RC0 your_apache_id --dry-run

For post-incubation path changes, override the default SVN roots:

python scripts/apache_release.py promote 0.41.0-RC0 your_apache_id \
  --dev-svn-root https://dist.apache.org/repos/dist/dev/burr \
  --release-svn-root https://dist.apache.org/repos/dist/release/burr

For Voters: Verifying a Release

If you're voting on a release, follow these steps to verify the release candidate:

Complete Verification Workflow

# Set version and RC number (example: 0.41.0 RC3)
export VERSION=0.41.0
export RC=3

# 1. Download all artifacts from SVN
svn export https://dist.apache.org/repos/dist/dev/incubator/burr/${VERSION}-incubating-RC${RC}/ burr-rc${RC}
cd burr-rc${RC}

# 2. Import KEYS file and verify all GPG signatures
wget https://downloads.apache.org/incubator/burr/KEYS
gpg --import KEYS

# Verify git archive signature
gpg --verify apache-burr-${VERSION}-incubating-src.tar.gz.asc apache-burr-${VERSION}-incubating-src.tar.gz

# Verify sdist signature
gpg --verify apache-burr-${VERSION}-incubating-sdist.tar.gz.asc apache-burr-${VERSION}-incubating-sdist.tar.gz

# Verify wheel signature
gpg --verify apache_burr-${VERSION}-py3-none-any.whl.asc apache_burr-${VERSION}-py3-none-any.whl

# 3. Verify all SHA512 checksums
sha512sum -c apache-burr-${VERSION}-incubating-src.tar.gz.sha512
sha512sum -c apache-burr-${VERSION}-incubating-sdist.tar.gz.sha512
sha512sum -c apache_burr-${VERSION}-py3-none-any.whl.sha512

# 4. Extract the source archive
tar -xzf apache-burr-${VERSION}-incubating-src.tar.gz
cd apache-burr-${VERSION}-incubating-src/

# 5. Install build dependencies
pip install flit

# 6. Build the wheel from source (this also builds the UI)
python scripts/apache_release.py wheel ${VERSION} ${RC}

# 7. Install and test the wheel you just built -- play with the UI
pip install "dist/apache_burr-${VERSION}-py3-none-any.whl[learn]"
burr

Note that the script currently signs the file with a signature. We will be adding the ability to bypass that, but if you want to build from scratch you can follow this process:

# 1. Build UI
cd telemetry/ui
npm install
npm run build
cd ../..

# 2. Copy UI build to the right place
mkdir -p burr/tracking/server/build
cp -a telemetry/ui/build/. burr/tracking/server/build/

# 3. Handle examples (replace symlink with actual files)
rm burr/examples  # Remove symlink
mkdir burr/examples
cp examples/__init__.py burr/examples/
cp -r examples/email-assistant burr/examples/
cp -r examples/multi-modal-chatbot burr/examples/
cp -r examples/streaming-fastapi burr/examples/
cp -r examples/deep-researcher burr/examples/

# 4. Build wheel
flit build --format wheel

# 5. Restore symlink
rm -rf burr/examples
ln -s ../examples burr/examples

Then use the wheel at dist/apache_burr-0.41.0-py3-none-any.whl

See the “Verification” section below for more detailed verification steps including license checking.

Verification

Validate artifacts before uploading or voting. Checks GPG signatures, SHA512 checksums, archive integrity, and license compliance with Apache RAT. The list-contents command is useful for inspecting what's actually packaged in each artifact.

Verification script: scripts/verify_apache_artifacts.py

Prerequisites

For license verification, you'll need Apache RAT. Download it from:

# Download Apache RAT jar
curl -O https://repo1.maven.org/maven2/org/apache/rat/apache-rat/0.15/apache-rat-0.15.jar

Or download manually from: https://repo1.maven.org/maven2/org/apache/rat/apache-rat/0.15/

Running Verification

# Verify signatures and checksums
python scripts/verify_apache_artifacts.py signatures

# Verify licenses (requires Apache RAT)
python scripts/verify_apache_artifacts.py licenses --rat-jar apache-rat-0.15.jar

# Verify everything
python scripts/verify_apache_artifacts.py all --rat-jar apache-rat-0.15.jar

# Inspect artifact contents
python scripts/verify_apache_artifacts.py list-contents dist/apache-burr-0.41.0-incubating-src.tar.gz
python scripts/verify_apache_artifacts.py list-contents dist/apache_burr-0.41.0-py3-none-any.whl

Local Development

Simpler workflow for building wheels during development without signing or creating full release artifacts. Useful for testing packaging changes or building wheels to install locally.

For local wheel building/testing (simpler, no signing):

python scripts/build_artifacts.py build-ui    # Build UI only
python scripts/build_artifacts.py wheel       # Build wheel with UI

Milestone Management

Merged PRs are tracked against release milestones so changelog generation can tell which changes shipped in which release.

Milestone titles use the X.Y.Z naming convention, for example 0.43.0
PRs merged to main are assigned the earliest open X.Y.Z milestone
PRs merged to release/test branches use the milestone implied by the branch name, including:
- release-X.Y.Z
- apache-burr-X.Y.Z-release
- vX.Y-test or vX.Y.Z-test

The .github/workflows/milestone-management.yml workflow also supports manual milestone creation via workflow_dispatch. Provide one or more comma- or newline-separated X.Y.Z versions in the versions input to create or re-open upcoming release milestones.