Bulk CouchDB operations and concurrent API calls across 7 agents

Replace serial per-key get/set loops with get_many/set_many and
sequential HTTP calls with asyncio.gather + Semaphore(10). Saves
roughly 9-10 minutes per full pipeline run.

gha_security: bulk-read all cached YAML at init via get_many (was
~5,000 serial gets); accumulate findings in dict, flush with set_many
(was ~1,200 serial sets); bulk-read cached findings in redacted mode.

gha_publishing_detail: 3x get_many for prefetch/classification/YAML
keys (was ~7,400 serial gets); concurrent GitHub API run-history
fetches with Semaphore(10) (was ~200 serial calls); set_many for
report writes.

gha_artifact_verify: concurrent registry API queries with
Semaphore(10) (was ~200 serial calls); set_many for results.

gha_brief: get_many for all finding keys (was ~1,200 serial gets).

gha_review: same get_many fix as brief.

gha_json_export: get_many for classification keys and finding keys
(was ~3,700 serial gets).

gha_prefetch: fetch_single_yaml returns content instead of writing
per-file; set_many batches all YAML + metadata per repo; composites
and extras batched into single set_many per repo.
7 files changed
tree: 998996ca962452f580ea96d47c64283c3397de72
  1. ASVS/
  2. docs/
  3. gh-helper/
  4. gha-review/
  5. scrutineer/
  6. util/
  7. .asf.yaml
  8. .gitignore
  9. LICENSE
  10. README.md
README.md

Apache Tooling Agents

AI-driven security auditing and code review for ASF projects

Pipelines

ASVS Security Audit

Automated OWASP ASVS v5.0.0 compliance auditing for any GitHub-hosted codebase. Downloads source code, discovers the architecture, runs per-requirement security analysis with Claude, and produces a consolidated report with deduplicated findings and GitHub issues. In production — piloted on ATR and Apache Steve.

GitHub Actions Review

Automated security scan of GitHub Actions workflows across an entire GitHub organization. Combines LLM classification (which repos publish what, where) with static pattern matching (12 check types from CRITICAL to INFO) to identify exploitable workflows, supply chain risks, and policy violations. Scanned 2,500+ Apache repos.

Both pipelines run on Gofannon — see docs/gofannon for platform setup.

Repository Structure

├── ASVS/                  # ASVS security audit pipeline
│   ├── agents/            # Pipeline agent code (6 agents)
│   ├── audit_guidance/    # Project-specific false positive guidance
│   ├── reports/           # Audit output organized by project and commit
│   └── rerun-sections.sh  # QA: re-run failed sections, re-consolidate
├── gha-review/            # GitHub Actions security review
│   ├── agents/            # Review pipeline agents (7 agents + tests)
│   └── reports/           # Review output
├── docs/
│   ├── gofannon/          # Gofannon setup and agent development guide
│   ├── tooling/           # Security tooling landscape and comparisons
│   ├── roadmap/           # Eval framework, multi-spec expansion plans
│   └── how-to-contribute.md
└── util/                  # Utility scripts

Documentation

Getting Involved

Join the Conversation

  1. Mailing list: Say hello at 📧 dev@tooling.apache.org (Subscribe by sending an email with empty subject and body to dev-subscribe@tooling.apache.org and replying to the automated response, per the ASF mailing list how-to)

  2. Slack: #tooling-discuss on the ASF Slack

  3. Issues: Use GitHub Issues to ask questions, suggest approaches, or report bugs

Contribute

  • How to contribute
  • Request an audit: Just ask on the mailing list or Slack — we handle everything. No tokens, no setup needed.
  • Write audit guidance: Help reduce false positives for your project — see audit_guidance/README.md

Note: Please introduce yourself on the mailing list before submitting a PR; this helps us deter spam and means your contribution won't be overlooked.

License

This project is licensed under the Apache License 2.0.


Part of the Apache Tooling Initiative.