Project

Knowledge-base updates

The scheduled workflows that keep the vulnerability knowledge bases current.

A scheduled GitHub Action (.github/workflows/update-knowledge-base.yml) keeps the sh-kb-* knowledge bases fresh. Every alternate day (and on manual workflow_dispatch), it runs an agent to distill new, reputable public security research - OWASP, PortSwigger Research, CWE/CAPEC, NIST, MDN, curated GitHub repos, and public HackerOne disclosures - into small, well-sourced improvements. A second, adversarial reviewer agent then scans the resulting diff for malicious/injected content, and the PR is auto-merged only if that reviewer approves.

Two workflows, three jobs

PR creation is deliberately separated from review and merge, so the thing that writes the diff is never the thing that decides to ship it.

Stage 1 - update-knowledge-base.yml (scheduled or manual). One job, create-pr:

1. Generate - the agent edits the KB from allowlisted sources. No commit, no push.
2. Open PR - a deterministic step opens (or updates) a PR on the automated/kb-update branch, labeled awaiting-review.
3. Hand off - on a successful PR creation it dispatches stage 2 with the PR number.

Stage 2 - kb-review-and-merge.yml (dispatched by stage 1, or run by hand against any automated PR). Two jobs:

• review - a separate agent run inspects the diff adversarially for prompt-injection artifacts, out-of-scope edits, secrets/exfil, PII, weaponized exploits, off-allowlist sourcing, or house-style violations. It has no web and no shell, and fails closed: anything suspicious, any uncertainty, or a missing verdict file → REJECT. The verdict is posted as a PR comment and drives the label.
• merge - runs only on APPROVE, and merges the PR. A REJECT skips it and the blocked job reports why.

Why a dispatch rather than a pull_request trigger: a PR opened by GITHUB_TOKEN does not trigger pull_request workflows. workflow_dispatch is one of the two events exempt from that recursion guard, so stage 1 can hand off reliably.

Auto-merge means an approving agent lands code in main. The controls on that:

• The merge job refuses any PR that is closed, from a fork, or whose head branch is outside automated/* (ALLOWED_HEAD_PREFIX in the workflow).
• It prefers GitHub's own auto-merge, so branch protection still applies. With a rule on main requiring an approving review, the PR queues and waits for a human instead of merging. It falls back to an immediate merge only on repos where auto-merge is off.
• Set the auto_merge input to false on a manual run to review without merging.
• The reviewer prompt tells the agent its verdict is binding, not advisory.

Requires the repo setting "Allow GitHub Actions to create and approve pull requests" (Settings → Actions → General → Workflow permissions) so the workflow can open the PR. If you want a human in the loop despite auto-merge, protect main with a branch-protection rule requiring a pull request and at least one approving review - the auto-merge path honours it.

Pluggable agents

Both stages run through .github/actions/ai-agent, a composite action that dispatches to whichever agent you configure. Claude Code, OpenAI Codex, Gemini CLI, and an escape hatch for anything else:

Pick per run from the workflow_dispatch inputs, or set repo variables to change the default: KB_AGENT and KB_MODEL for the generator, KB_REVIEW_AGENT and KB_REVIEW_MODEL for the reviewer. Running the generator and the reviewer on different agents is a meaningful hardening step: an injection tuned for one model is less likely to land on a second, independent one.

For agent: custom, set KB_AGENT_COMMAND to a shell command. The prompt is written to the file named by $AGENT_PROMPT_FILE, and $AGENT_MODEL carries the model input.

Prompt-injection defenses, since the generator reads the open web:

• Domain allowlist. WebFetch is restricted to the trusted domains in .github/kb-update/trusted-sources.md (mirrored in the workflow's --allowedTools). WebSearch can discover URLs, but only allowlisted domains can actually be fetched.
• Content is data, not commands. The task prompt (.github/kb-update/prompt.md) instructs Claude to treat every fetched byte as untrusted reference material and to ignore any instructions embedded in a page - HackerOne report bodies (user-generated) are flagged as the highest-risk tier.
• No shell, no push on the generator; the reviewer is the gate. The generator can only edit files. The independent reviewer (.github/kb-update/review-prompt.md) is what stands between fetched content and main - nothing merges without its explicit approval.
• Different agents for generator and reviewer. Optional, and the strongest version of the gate: set KB_AGENT and KB_REVIEW_AGENT to two different engines.

Setup:

• Add the credential for whichever agent you use (Settings → Secrets and variables → Actions): CLAUDE_CODE_OAUTH_TOKEN (default), ANTHROPIC_API_KEY, OPENAI_API_KEY, or GEMINI_API_KEY. The OAuth token authenticates against your Claude subscription's usage limits rather than a metered API key - generate it locally with claude setup-token (requires an active Claude Pro/Max subscription) and paste the result.
• Enable "Allow GitHub Actions to create and approve pull requests" (Settings → Actions → General → Workflow permissions) so the workflow can open its PR. Recommended: add a branch-protection rule on main requiring a PR and an approving review, so no automated change can land without a human even with auto-merge on.
• To change which sources are allowed, edit the allowlist in trusted-sources.md and the matching WebFetch(domain:...) entries in the workflow - keep the two in sync.

Each run records what it did in .github/kb-update/last-run-summary.md.