Changelog

All notable changes to Bulwark are documented here. The format is based on Keep a Changelog, and the project follows Semantic Versioning.

Unreleased

Changed

• PyPI distributions moved into the bulwark- namespace. Every unnamespaced name the suite wanted is already taken on PyPI by an unrelated project — airlock is a 2015 Google App Engine OAuth wrapper, warden a 2012 monitoring shim, manifest an LLM code-execution tool, and bulwark a pandas testing library. The documented install commands therefore installed someone else's package.

Was (someone else's)	Now
pip install airlock	pip install bulwark-airlock
pip install warden	pip install bulwark-warden
pip install manifest	pip install bulwark-manifest
pip install bulwark	pip install bulwark-suite

Nothing about usage changes. CLI commands are still airlock, warden, manifest, and bulwark; import names are still import airlock, from warden.scanner import …. Only the distribution name moves. A packaging invariant test now enforces the namespace, so the suite cannot be republished under a squatted name by accident. - CITATION.cff added, so GitHub renders a "Cite this repository" button and a Zenodo release archive carries real authorship instead of a repository slug.

Added

• Validation harnesses for Warden and Manifest (packages/<tool>/scripts/study.py), closing a gap where every published Bulwark measurement came from Airlock. Warden gets four studies — cross-framework invariance, lexicon robustness under obfuscation, false positives on benign assemblies, and --recommend efficacy. Manifest gets four — discovery recall against hand-written ground truth, CycloneDX/SPDX conformance, governance control coverage, and risk-bridge fidelity. Both are deterministic and offline; results are written to docs/VALIDATION.md per tool and folded into docs/EMPIRICAL_VALIDATION.md, which is now organized by layer (part / assembly / system) rather than by tool.
• 15 tests pinning every published validation figure, so a rule or lexicon change that moves a number fails CI instead of quietly making the docs wrong.

Fixed

• camelCase tool names were entirely unclassifiable, so any assembly using them — most of the TypeScript MCP ecosystem — silently lost A2, Warden's flagship attacker-triggerable exfiltration finding, while still reporting a clean-looking MEDIUM verdict. _tool_text() appended an underscore/hyphen-normalized copy so \bbrowse\b matched browse_web, but never split case transitions, so browseWeb matched nothing. It now appends a camel-split copy as well; snake_case and camelCase classify identically. Found by the new lexicon-robustness study.
• transfer / wire no longer imply FINANCIAL without a money noun. "Transfer the meaning of a phrase into another language" was classified as a financial operation, and because FINANCIAL is high-impact it also raised a spurious A3 missing-gate finding on a plain translation tool. ("Transfer learning" tripped it too.)
• request no longer implies NET_OUT without network context. A bare \brequest\b matched "the user's request" — ordinary English long before it is an HTTP verb, and the single noisiest source of spurious egress capability on benign agents.

Together the two false-positive fixes halve the benign agents carrying a HIGH+ finding (5/7 → 3/7) with no loss of true positives, and the camelCase fix raises A2 recovery under obfuscation from 3/7 to 4/7.

[0.2.0] — 2026-08-07

Productionisation release: the suite becomes installable software rather than a repository. Five packages move to a src/ layout, gain structured logging, a documentation site, runnable examples, and supply-chain hardening of the project itself — alongside a large correctness pass.

Upgrading from 0.1.0: no API breaks. airlock.toml now actually takes effect, which is a behaviour change if you had a config file that was previously being ignored — check that its fail_on and suppress_rules are what you intend before upgrading a gate.

Added

• src/ layout for all five packages (packages/<name>/src/<name>/). Imports now resolve to the installed wheel rather than the working directory, which is what surfaces a data file missing from a distribution. Verified: all 18 rule packs and the HTML template ship inside the wheels, and pip install airlock works standalone from a wheel with no workspace present.
• Structured logging (bulwark_core.logging), with -v / -vv on every CLI. Diagnostics go to stderr only, so --format json > out.json stays valid at any verbosity; as a library, Bulwark emits nothing until an application calls configure(). Artifact-derived text is single-lined and truncated before it can reach a log record, so a hostile description cannot forge log lines.
• examples/ — five runnable scripts covering the Python API for all three tools, custom rule packs, and the least-privilege recommender. Exercised by CI, so they cannot rot.
• Documentation site (MkDocs Material) with installation, quick start, guides, CLI and Python API reference, taxonomy, configuration, architecture, and threat model. Builds under --strict; published to GitHub Pages.
• load_rules(extra_roots=...) in all three tools — the documented "layer your own rule packs" story was previously not expressible from Python, only by replacing the built-in roots entirely.
• airlock rules debug <kind> <target> — dumps the signal bundle for a target. "Is the evidence there?" is the first question when a rule stops firing, and it previously required a fifteen-line script.
• CodeQL analysis and Dependabot grouped update policy.
• Suite-wide invariant tests: version agreement between pyproject.toml and __version__, src/ layout, and wheel-content declarations.

Fixed

• Snake_case tool names were not classified. _ is a regex word character, so \bbrowse\b never matched browse_web and \bshell\b never matched run_shell — meaning Warden's flagship CRITICAL finding (an attacker-triggerable exfiltration flow) silently did not fire on the naming convention the ecosystem actually uses. Tool text is now matched against a de-snaked copy as well as the raw form.
• The CRITICAL injectable-exfiltration flow ignored gates on the sink, while the sibling injectable-action check credited them. A declared approval gate breaks the automated chain, so the two escalations now agree — and the recommender's advice measurably clears the finding.
• \bopen\b classified "open a support ticket" as filesystem read. Third instance of the verb-without-a-domain-noun pattern, after \bformat\b and \bsandbox\b; now requires a filesystem noun.

Changed

• All five packages are at 0.2.0 and release in lockstep from one tag.
• check.py, noxfile.py, and CI type-check src/<module> paths.

[0.1.1] — 2026-08-02

Fixed

• Configuration files were silently ignored. airlock.toml was read, parsed, and then discarded: the merge helper recursed into nested tables but never assigned a scalar, so fail_on, output_format, strict_allowlist, suppress_rules/suppress_paths, and [ai].enabled all fell back to defaults with no error. Settings now layer through pydantic-settings' settings_customise_sources with a TOML source ordered below the environment, so files work and env still wins. Warden and Manifest gain the same layering (warden.toml, manifest.toml), which they previously lacked entirely.
• Manifest's Warden bridge skipped agent assemblies. --scan-risk considered only mcp-server components, so an assembly discovered from a CrewAI crew, an OpenAI Assistants config, or an agent manifest was inventoried and then never audited. agent components are now bridged too.
• Post-processing dropped score and meta. Waivers and baselines rebuilt the result from an explicit field list, losing Warden's agency score and Manifest's entire AIBOM — which would have broken --format cyclonedx after any suppression. Now a model_copy, so future fields survive.
• --strict did not apply to pickles embedded in numpy object arrays, making allowlist coverage depend on the container rather than the payload. serialized.py now shares pickle_scan's emitter.
• MCP timeout was accepted and never used, so a server that connected and never answered initialize hung the scan indefinitely. Enumeration is now bounded by Limits.connect_timeout_s (AIRLOCK_LIMIT_CONNECT_TIMEOUT), and a timeout is reported as a connect error like any other.
• Directory walks followed symlinks and had no file cap in both Airlock's model resolver and Manifest's discovery context — a target containing a link to / made a scan traverse the whole filesystem. Both now use a shared walk_files helper with resolved-path containment and Limits.max_files (AIRLOCK_LIMIT_MAX_FILES).
• Warden lexicon false positives. format_response classified as DESTRUCTIVE (and therefore high-impact, producing a spurious missing-gate finding and +10 agency score); "runs in a sandbox" classified as CODE_EXEC. Both patterns now require corroborating context.
• A5 egress allow-listing recognised only the literal string allowlist, so a genuine scope such as https://api.example.com/** was reported as unrestricted — a false positive on exactly the configuration the finding asks for. Concrete hosts, URL prefixes, and CIDRs now count.
• BOM drift ignored provenance.source/author, so a model switching publisher under the same name was reported as unchanged.
• AIBOM.add merged more shallowly than documented — provenance, licence, and findings from a later discoverer were discarded. The merge is now field-wise.
• noxfile.py lint never changed directory, running ruff from the repo root five times instead of once per package, contradicting its own docstring.

Added

• airlock rules debug <kind> <target> — dump the signal bundle a scan produces without applying rules. The first question when a rule stops firing is whether the evidence exists; this answers it.
• Signal-name validation in rules lint (all three tools). A mistyped match.signal was the one rule error that failed silently — no rule matched, nothing errored, the detection was simply gone. Each tool now declares KNOWN_SIGNALS and lint rejects a rule referencing anything outside it.
• hf:org/name@revision pinning — pin a scan to an immutable Hub commit, so a result is reproducible and attributable to specific bytes rather than to a mutable branch.
• Hugging Face purls — models and datasets emit pkg:huggingface/org/name@revision in CycloneDX, so they are identifiable across tools and advisory feeds, not just libraries.
• Typed SPDX relationships — the AIBOM's own verbs (trained-on, contains, variant-of, …) now map to real SPDX relationship types instead of flattening to DEPENDS_ON.
• Rug-pull detection reports added and removed tools, not only changed ones. A tool appearing after approval is the classic rug-pull shape.
• Remote MCP auth detection — auth.missing now checks for credentials actually supplied (URL userinfo, an auth query parameter, or an MCP_* token env var) instead of being a synonym for "is remote".
• Three-state governance status — ok / advisory / gap, driven by the worst severity mapped to a control. A single LOW advisory no longer marks a NIST function or an EU AI Act article as a gap.
• Risk register owner and status columns (emitted as a template), so the output is trackable rather than a list of complaints.
• AGPL is classified separately from GPL — network copyleft is the highest-consequence licence term for a hosted product and is now surfaced as restricted.
• Source→sink pairings are capped and rolled up in both Airlock's P5 and Warden's A2, so a large tool-set produces a readable report instead of hundreds of near-duplicates.
• bulwark-core test suite — 31 tests covering severity ordering, finding identity, post-processing field preservation, the zip-slip guard, and the bounded walk, plus architectural invariant tests asserting that core imports nothing from the suite and never executes its input.
• Regression suites for the configuration layering, the Warden bridge, lexicon classification, BOM merging and drift, and a property test that --recommend actually lowers the agency score.
• bulwark meta-CLI — one front door over all three tools plus bulwark scan (full pipeline).
• Airlock — format/extension-confusion detector (M6) — sniffs magic bytes and flags a pickle disguised under a safe extension (the picklescan CVE-2025-10155 bypass class), then scans it anyway.
• Airlock — allowlist mode (--strict, M3) — Fickling-style: flags pickle imports from modules outside the ML allowlist, catching novel callables a denylist misses.
• Warden — policy profiles (--profile strict|balanced|permissive).
• Warden — attacker-triggerable toxic flows — escalates A2 to CRITICAL when untrusted input (browse / inbound) can drive a read-secret-then-exfiltrate kill chain (indirect prompt injection), plus an injectable-high-impact-action variant.
• Manifest — SPDX 2.3 output, EU AI Act mapping, notebook discoverer, manifest diff (BOM drift).
• Manifest — VEX output (--format vex) — CycloneDX VEX seeded from detected vulnerable deps.
• Manifest — agent discovery + CycloneDX Agent BOM — agent components (autonomy + wired tools) surfaced as bulwark:agent:* properties, aligned with the emerging CycloneDX Agent BOM.
• Empirical validation — a real-model corpus study, a 14-payload adversarial suite, and a head-to-head benchmark against picklescan / modelscan / fickling.
• Project docs — USAGE.md, LANDSCAPE.md, EMPIRICAL_VALIDATION.md, DATASETS_AND_TESTING.md.
• OSS hygiene — root LICENSE, SECURITY.md, CONTRIBUTING.md, CODE_OF_CONDUCT.md, issue/PR templates, and a matrixed CI across all five packages.

Security (self-hardening — the scanner ingests hostile input)

• HTML report XSS fixed. The report template (report.html.j2) fell through select_autoescape(["html"]) (its .j2 suffix), so attacker-controlled strings from a scanned artifact (finding evidence, file paths, MCP tool descriptions) rendered unescaped — opening a report on a hostile artifact could execute injected <script>/onerror. Autoescape is now forced on.
• Rule-feed zip extraction hardened. rule_feed._extract_zip replaced a weak "/../" substring check with a resolved-path containment guard (defeats ../ and absolute/drive zip-slip), and added per-member, total-uncompressed, and member-count caps (decompression-bomb / member-flood guards); members are streamed, never zf.extract-ed.
• Bounded reads. Whole-file read_bytes() on artifacts (numpy .npy, GGUF magic check, compressed pickles, Keras zip members) replaced with capped reads via bulwark_core.limits.read_bounded, so a crafted multi-GB file can't OOM the scanner.
• ReDoS blast-radius bound. Rule-engine regexes now run against a length-capped input (MAX_MATCH_INPUT) and use a compiled-pattern cache, limiting catastrophic backtracking from a hostile field or an untrusted community rule pack.
• Symlink containment and a file cap on directory walks. Both Airlock's local model resolver and Manifest's discovery context used rglob, which follows symbolic links, with no file cap — so a hostile artifact directory containing a link to / turned a scan into a filesystem traversal. Both now share bulwark_core.limits.walk_files, which resolves each entry and requires it to remain under the scan root (the same containment check the rule feed uses against zip-slip) and stops at Limits.max_files.
• A time bound on MCP enumeration. Time was the one resource dimension left unbounded: a server that accepted a connection and never responded hung the scan forever. Enumeration now runs under anyio.fail_after(Limits.connect_timeout_s).
• Prompt-injection spotlighting in the AI layer. Content sent to a provider comes from the artifact being scanned and is therefore attacker-controlled. It is now fenced in <untrusted_content> markers, forged markers are stripped, and the system prompt states that the fence marks data — text inside it demanding a particular verdict is treated as evidence of manipulation. The layer's blast radius was already bounded (AI can add findings, never remove or downgrade one); this removes the cheapest version of the attack.
• Honest AI budget accounting. The executive-summary and model-card calls previously ran outside the max_findings_to_enrich counter, so the real ceiling was the documented cap plus two. All provider calls now share one counter.
• A stdio-scanning warning. airlock scan mcp with a command target now states that enumerating a stdio server starts it — Airlock never invokes a tool, but the server's own startup code runs with your privileges — and points at the fully static airlock scan toolspec path for CI.
• Versioned OWASP citations. Reference strings were a mix of 2023 and 2025 LLM Top 10 numbering (LLM05 for supply chain is 2023; LLM02/LLM06 were 2025). All 76 sites now carry an explicit edition — OWASP:LLM03:2025 for supply chain, and so on — so a future renumbering cannot silently invalidate a citation. Placeholder references (best-practice, license-compliance, …) were replaced with real, citable sources.

Changed (CI/CD — applying the project's own advice to itself)

• Pinned the CI toolchain. ruff, mypy, pytest, and pytest-cov were installed unpinned, so a third-party release could break the build with no change on our side — precisely the reproducibility gap Manifest's own B1 finding reports on other people's projects. Extras are installed without a silent || fallback, since a degraded install turns real failures into skipped tests.
• Enforced release ordering. bulwark-core now publishes in its own job with the tools gated on needs: core. The matrix previously ran in parallel with a comment claiming core went first, which could leave PyPI briefly holding an airlock depending on a bulwark-core that did not exist.
• Signed release artifacts with Sigstore, using the same keyless OIDC identity Trusted Publishing already establishes. A project arguing for model signing should not ship unsigned wheels.
• Bulwark now publishes its own AI-BOM (CycloneDX, SPDX, and a governance report) as a release artifact.

Fixed

• Benchmark harness no longer copies scan artifacts into the corpus directory (an earlier version silently inflated repeated runs); it now scans each file in place.

0.1.0 — Airlock · Warden · Manifest · bulwark-core

Added

• bulwark-core — shared spine: Finding/Severity/ScanResult, YAML rule engine, signal IR, report renderers (terminal / JSON / HTML / SARIF), optional AI provider layer, resource limits.
• Airlock — static scanner for models (M1–M7), MCP servers (P1–P9), and tool-specs; formats: pickle, safetensors, GGUF, ONNX, Keras, numpy, TensorFlow SavedModel, Flax, PMML; hardened parsers; SARIF/CI; optional AI enrichment; corpus study harness; GitHub Action.
• Warden — least-privilege auditor: AgentSpec IR, capability graph, toxic-combination detection (A1–A10), agency score, framework importers (manifest / MCP config / OpenAI Assistants / LangChain / CrewAI), least-privilege recommendation, and --scan-parts (runs Airlock on wired MCP servers).
• Manifest — AI-BOM generator: discoverers, provenance/license resolution, OSV vulns, CycloneDX 1.5 output, Airlock/Warden risk bridges (B1–B9), NIST AI RMF governance mapping.