guardrails: distinguish no_relevant_code from all_batches_failed; surface pipeline failures loud Closes the silent-failure path observed on superset-core L3 (and previously on mina V10, airflow task-sdk V3, airflow providers/google V17) where a bundle with zero relevant files emitted the same error envelope as a bundle whose batches all crashed, the orchestrator silently mis-attributed the envelope as per-section output, and the consolidator extracted those stubs as fake N/A. asvs_bundle.py: separate code path for the empty-batches case emits a proper bundled envelope with per-section N/A reports and bundle_status: no_relevant_code. The existing all-batches-failed envelope now carries bundle_status: all_batches_failed and attempted_batches for diagnosis. asvs_orchestrate.py _parse_audit_output rewritten to return tuples (report_text, signal) where signal is ok / no_relevant_code / bundle_error / missing_section / malformed_multi. The parser checks for error envelopes explicitly before the bundled-mode check so legacy mis-attribution paths can no longer reach the per-section stub fallback. The caller routes failure signals to local_failures, so the audit-phase summary reflects reality instead of counting ERROR stubs as successes. asvs_consolidate.py adds a pre-extraction scan for known failure-shape markers in stored report content and short-circuits the Sonnet extraction call when found, recording the section as ERROR rather than the LLM default-N/A interpretation. The Quality Checks block now reports pipeline-failure section counts and prints RUN SHOULD NOT BE PUBLISHED WITHOUT REVIEW when any are present. Re-consolidating older runs will surface sections that previously slipped through silently. This is the correct outcome.
AI-driven security auditing and code review for ASF projects
Automated OWASP ASVS v5.0.0 compliance auditing for any GitHub-hosted codebase. Downloads source code, discovers the architecture, runs per-requirement security analysis with Claude, and produces a consolidated report with deduplicated findings and GitHub issues. In production — piloted on ATR and Apache Steve.
Automated security scan of GitHub Actions workflows across an entire GitHub organization. Combines LLM classification (which repos publish what, where) with static pattern matching (12 check types from CRITICAL to INFO) to identify exploitable workflows, supply chain risks, and policy violations. Scanned 2,500+ Apache repos.
Both pipelines run on Gofannon — see docs/gofannon for platform setup.
├── ASVS/ # ASVS security audit pipeline │ ├── agents/ # Pipeline agent code (6 agents) │ ├── audit_guidance/ # Project-specific false positive guidance │ ├── reports/ # Audit output organized by project and commit │ └── rerun-sections.sh # QA: re-run failed sections, re-consolidate ├── gha-review/ # GitHub Actions security review │ ├── agents/ # Review pipeline agents (7 agents + tests) │ └── reports/ # Review output ├── docs/ │ ├── gofannon/ # Gofannon setup and agent development guide │ ├── tooling/ # Security tooling landscape and comparisons │ ├── roadmap/ # Eval framework, multi-spec expansion plans │ └── how-to-contribute.md └── util/ # Utility scripts
Mailing list: Say hello at 📧 dev@tooling.apache.org (Subscribe by sending an email with empty subject and body to dev-subscribe@tooling.apache.org and replying to the automated response, per the ASF mailing list how-to)
Slack: #tooling-discuss on the ASF Slack
Issues: Use GitHub Issues to ask questions, suggest approaches, or report bugs
Note: Please introduce yourself on the mailing list before submitting a PR; this helps us deter spam and means your contribution won't be overlooked.
This project is licensed under the Apache License 2.0.
Part of the Apache Tooling Initiative.