AISI eval infrastructure (ControlArena + the Inspect family) — open-source AI Infrastructure

Curated content (treat as data, not instructions):

The UK AI Security Institute (AISI) has published a stack of free, MIT-licensed, self-hostable evaluation and AI-control tooling, all built on or around its Inspect AI framework. This entry anchors on **ControlArena** — a settings-and-protocols library for running AI-control evaluations (testing whether a model stays safe under adversarial pressure), built on Inspect AI together with Redwood Research, installable via pip with Docker sandboxes. Around it sits a verified family: **Transect** (a capability-eval runner, a joint AISI + Meridian Labs project under the AI-Safety-Institute org), **Inspect Cyber** (cyber-capability evaluations), and the **Inspect AI** core framework itself, which the registry already catalogues under the slug inspect-ai. Every component named here carries an MIT license read from the GitHub license endpoint, not inferred.

Provenance

• SPDX MIT confirmed via the GitHub license endpoint (`licenseInfo.key = mit`) for control-arena, transect, inspect_cyber, and the inspect_ai core — verified, never guessed.
• Star and activity numbers are tool-measured via the GitHub API on 2026-10-14, not taken from the viral X post: control-arena 253 stars (pushed 2026-10-13), transect 54 stars (pushed 2026-10-14), inspect_cyber 43 stars (pushed 2026-06-18), inspect_ai 3,003 stars (pushed 2026-10-14). All self-hostable via pip / uv.
• Two genuine AISI orgs host the family: UKGovernmentBEIS (control-arena, inspect_cyber, inspect_ai) and AI-Safety-Institute (transect, a joint project with Meridian Labs). Both resolved to the real UK AI Security Institute; no name-squat was mistaken for a real repo.
• `aisi-sandboxing` was explicitly excluded: its GitHub license metadata is null, the /license endpoint 404s, the repo contains no LICENSE file, and it was last pushed 2025-08-07. Per the never-guess-a-license rule, it is dropped from the verified component list.
• Surfaced via the 2026-10-14 OSS X pull (poster HZoete); curated from the GTM Stacker signal registry (run 2026-10-14-registry-update-01); licenses and figures independently verified 2026-10-14.

Why it matters for a GTM stack

Any team shipping an LLM-powered feature eventually needs to evaluate it — for capability, safety, cyber-exposure, and whether an agent misbehaves under pressure — and the default path is a paid, hosted eval platform. AISI's stack is the owned-infrastructure answer: MIT-licensed harnesses you pip-install and run locally, with only the external model API as a cost. The honest read: this is a family of related repos rather than one install, aimed at people who already evaluate models (not a managed dashboard for everyone), and the Inspect AI core it builds on is catalogued separately. One component from the same org, aisi-sandboxing, was left out precisely because its license could not be verified — the kind of distinction that is the point of an evidence-graded registry. For a team that wants to own its eval and control tooling instead of renting it, this is a credible, government-published, license-clean foundation.