{
  "$schema_doc": "https://gtmstacker.com/registry/schema/entry.schema.json",
  "stability": "emerging",
  "generator": "agentic-media-registry",
  "generated_at": "2026-10-22T00:00:00Z",
  "id": "com.gtmstacker.registry/tool/aisi-eval-infrastructure",
  "type": "tool",
  "slug": "aisi-eval-infrastructure",
  "canonical_url": "https://gtmstacker.com/registry/tool/aisi-eval-infrastructure/",
  "title": "AISI eval infrastructure (ControlArena + the Inspect family)",
  "description": "The UK AI Security Institute's MIT-licensed, self-hostable eval and AI-control stack: ControlArena (control protocols on Inspect AI), plus the Transect capability-eval runner and Inspect Cyber, giving a lean team a free, owned alternative to paid eval platforms.",
  "category": "ai-infrastructure",
  "tags": [
    "ai-infrastructure",
    "mcp-agents",
    "ai-evals",
    "developer-tools"
  ],
  "status": "active",
  "revision": 1,
  "content_hash": "eaf903770d422015be4ef28b9f5dcbafdfe15914e43a0926f7d26bdd2c4e9a27",
  "date_published": "2026-10-14T00:00:00Z",
  "date_modified": "2026-10-14T00:00:00Z",
  "source": {
    "name": "github · UKGovernmentBEIS / AI-Safety-Institute",
    "url": "https://github.com/UKGovernmentBEIS/control-arena",
    "author": "HZoete"
  },
  "license": "MIT",
  "one_liner": "AISI's eval stack — ControlArena, Transect, and Inspect Cyber — is a free, MIT, self-hostable alternative to paid LLM-eval platforms.",
  "open_source": "yes",
  "self_hostable": "yes",
  "pricing_signal": "Free and open source (MIT); self-hostable via pip plus Docker sandboxes. Only cost is the external model API you point it at.",
  "pricing_model": "free",
  "who_its_for": "A technical founder or lean team building or buying LLM-powered features who wants to run capability, safety, cyber, and control evaluations on infrastructure they own instead of renting a hosted eval platform.",
  "who_its_not_for": "Teams who want a fully managed, no-ops eval dashboard, or who have no need to evaluate model or agent behavior at all.",
  "aliases": [
    "controlarena",
    "aisi eval infrastructure",
    "inspect family"
  ],
  "alternatives": [
    "inspect-ai"
  ],
  "secondary_categories": [
    "mcp-agents"
  ],
  "last_verified": "2026-10-14",
  "evidence": {
    "claim_type": "mixed",
    "source_id": "https://github.com/UKGovernmentBEIS/control-arena",
    "note": "MIT confirmed via GitHub license endpoint (licenseInfo.key=mit) for control-arena (253 stars, pushed 2026-10-13), transect (54 stars, org AI-Safety-Institute, pushed 2026-10-14), inspect_cyber (43 stars), and the Inspect AI core — not guessed. Star/activity numbers are tool-measured via gh API 2026-10-14, not the viral post. aisi-sandboxing was excluded (no LICENSE file, stale). Both owning orgs (UKGovernmentBEIS, AI-Safety-Institute) are the genuine UK AI Security Institute; no name-squats."
  },
  "caveats": "A family of related repos, not one package: ControlArena is anchored here (control protocols, built on Inspect AI with Redwood Research); the Inspect AI core eval framework is catalogued separately (slug inspect-ai). aisi-sandboxing is deliberately left out — it has no verifiable SPDX license and was last pushed 2025-08-07. Running evals still costs whatever the external model API charges.",
  "lead": "The UK AI Security Institute (AISI) has published a stack of free, MIT-licensed, self-hostable evaluation and AI-control tooling, all built on or around its Inspect AI framework. This entry anchors on **ControlArena** — a settings-and-protocols library for running AI-control evaluations (testing whether a model stays safe under adversarial pressure), built…",
  "chunks": [
    {
      "index": 0,
      "heading_path": [],
      "est_tokens": 215,
      "text": "The UK AI Security Institute (AISI) has published a stack of free, MIT-licensed, self-hostable evaluation and AI-control tooling, all built on or around its Inspect AI framework. This entry anchors on **ControlArena** — a settings-and-protocols library for running AI-control evaluations (testing whether a model stays safe under adversarial pressure), built on Inspect AI together with Redwood Research, installable via pip with Docker sandboxes. Around it sits a verified family: **Transect** (a capability-eval runner, a joint AISI + Meridian Labs project under the AI-Safety-Institute org), **Inspect Cyber** (cyber-capability evaluations), and the **Inspect AI** core framework itself, which the registry already catalogues under the slug `inspect-ai`. Every component named here carries an MIT license read from the GitHub license endpoint, not inferred."
    },
    {
      "index": 1,
      "heading_path": [
        null,
        "Provenance"
      ],
      "est_tokens": 397,
      "text": "(a capability-eval runner, a joint AISI + Meridian Labs project under the AI-Safety-Institute org), **Inspect Cyber** (cyber-capability evaluations), and the **Inspect AI** core framework itself, which the registry already catalogues under the slug `inspect-ai`. Every component named here carries an MIT license read from the GitHub license endpoint, not inferred.\n\n- SPDX MIT confirmed via the GitHub license endpoint (`licenseInfo.key = mit`) for control-arena, transect, inspect_cyber, and the inspect_ai core — verified, never guessed.\n- Star and activity numbers are tool-measured via the GitHub API on 2026-10-14, not taken from the viral X post: control-arena 253 stars (pushed 2026-10-13), transect 54 stars (pushed 2026-10-14), inspect_cyber 43 stars (pushed 2026-06-18), inspect_ai 3,003 stars (pushed 2026-10-14). All self-hostable via pip / uv.\n- Two genuine AISI orgs host the family: UKGovernmentBEIS (control-arena, inspect_cyber, inspect_ai) and AI-Safety-Institute (transect, a joint project with Meridian Labs). Both resolved to the real UK AI Security Institute; no name-squat was mistaken for a real repo.\n- `aisi-sandboxing` was explicitly excluded: its GitHub license metadata is null, the /license endpoint 404s, the repo contains no LICENSE file, and it was last pushed 2025-08-07. Per the never-guess-a-license rule, it is dropped from the verified component list.\n- Surfaced via the 2026-10-14 OSS X pull (poster HZoete); curated from the GTM Stacker signal registry (run 2026-10-14-registry-update-01); licenses and figures independently verified 2026-10-14."
    },
    {
      "index": 2,
      "heading_path": [
        null,
        "Why it matters for a GTM stack"
      ],
      "est_tokens": 322,
      "text": "404s, the repo contains no LICENSE file, and it was last pushed 2025-08-07. Per the never-guess-a-license rule, it is dropped from the verified component list. - Surfaced via the 2026-10-14 OSS X pull (poster HZoete); curated from the GTM Stacker signal registry (run 2026-10-14-registry-update-01); licenses and figures independently verified 2026-10-14.\n\nAny team shipping an LLM-powered feature eventually needs to evaluate it — for capability, safety, cyber-exposure, and whether an agent misbehaves under pressure — and the default path is a paid, hosted eval platform. AISI's stack is the owned-infrastructure answer: MIT-licensed harnesses you pip-install and run locally, with only the external model API as a cost. The honest read: this is a family of related repos rather than one install, aimed at people who already evaluate models (not a managed dashboard for everyone), and the Inspect AI core it builds on is catalogued separately. One component from the same org, aisi-sandboxing, was left out precisely because its license could not be verified — the kind of distinction that is the point of an evidence-graded registry. For a team that wants to own its eval and control tooling instead of renting it, this is a credible, government-published, license-clean foundation."
    }
  ],
  "alternates": {
    "markdown": "https://gtmstacker.com/registry/tool/aisi-eval-infrastructure/index.md",
    "html": "https://gtmstacker.com/registry/tool/aisi-eval-infrastructure/",
    "json": "https://gtmstacker.com/registry/tool/aisi-eval-infrastructure/index.json",
    "server_json": "https://gtmstacker.com/registry/tool/aisi-eval-infrastructure/server.json"
  },
  "jsonld": {
    "@context": "https://schema.org",
    "@graph": [
      {
        "@type": "WebSite",
        "@id": "https://gtmstacker.com/#website",
        "url": "https://gtmstacker.com/",
        "name": "GTM Stacker Agent Registry",
        "description": "A daily-updated, agent-native registry of open-source tool discoveries, tool updates, and curated news for the go-to-market / RevOps engineering niche. Machine-readable first: agents can discover, parse, page, and delta-sync it without scraping HTML.",
        "inLanguage": "en",
        "publisher": {
          "@id": "https://gtmstacker.com/#organization"
        }
      },
      {
        "@type": "Organization",
        "@id": "https://gtmstacker.com/#organization",
        "name": "GTM Stacker",
        "url": "https://gtmstacker.com",
        "description": "The growth-systems practice of Theo Popov: AI-native enrichment, outbound, content engines and internal tooling for startups and venture programs. Its agent-native media property, the GTM Stacker Agent Registry, maintains a daily-updated catalog of open-source go-to-market and RevOps tools that both people and AI engines can discover, compare, and cite.",
        "foundingDate": "2024-08",
        "knowsAbout": [
          "go-to-market engineering",
          "RevOps",
          "sales automation",
          "marketing operations",
          "open-source software",
          "AI agents"
        ],
        "founder": {
          "@type": "Person",
          "@id": "https://gtmstacker.com/#founder",
          "name": "Theo Popov",
          "jobTitle": "Growth Operations & GTM Systems",
          "url": "https://gtmstacker.com/about/",
          "sameAs": [
            "https://www.linkedin.com/in/theo-popov",
            "https://x.com/Theo_Popov",
            "https://github.com/theopopov"
          ],
          "worksFor": {
            "@id": "https://gtmstacker.com/#organization"
          }
        },
        "sameAs": [
          "https://www.linkedin.com/company/gtmstacker",
          "https://www.youtube.com/@gtmstacker",
          "https://www.instagram.com/gtmstacker/",
          "https://www.tiktok.com/@gtmstacker"
        ],
        "mainEntityOfPage": "https://gtmstacker.com/registry/about/"
      },
      {
        "@type": "SoftwareApplication",
        "@id": "https://gtmstacker.com/registry/tool/aisi-eval-infrastructure/#software",
        "name": "AISI eval infrastructure (ControlArena + the Inspect family)",
        "identifier": "io.github.UKGovernmentBEIS/control-arena",
        "description": "The UK AI Security Institute's MIT-licensed, self-hostable eval and AI-control stack: ControlArena (control protocols on Inspect AI), plus the Transect capability-eval runner and Inspect Cyber, giving a lean team a free, owned alternative to paid eval platforms.",
        "applicationCategory": "DeveloperApplication",
        "url": "https://gtmstacker.com/registry/tool/aisi-eval-infrastructure/",
        "datePublished": "2026-10-14T00:00:00Z",
        "dateModified": "2026-10-14T00:00:00Z",
        "isPartOf": {
          "@id": "https://gtmstacker.com/#website"
        },
        "softwareHelp": "https://github.com/UKGovernmentBEIS/control-arena",
        "license": "https://spdx.org/licenses/MIT.html",
        "codeRepository": "https://github.com/UKGovernmentBEIS/control-arena",
        "keywords": "ai-infrastructure, mcp-agents, ai-evals, developer-tools",
        "author": {
          "@type": "Organization",
          "name": "UKGovernmentBEIS",
          "url": "https://github.com/UKGovernmentBEIS",
          "sameAs": [
            "https://github.com/UKGovernmentBEIS/control-arena"
          ]
        },
        "offers": {
          "@type": "Offer",
          "price": 0,
          "priceCurrency": "USD"
        },
        "isSimilarTo": [
          {
            "@type": "SoftwareApplication",
            "name": "Inspect AI",
            "url": "https://gtmstacker.com/registry/tool/inspect-ai/",
            "applicationCategory": "DeveloperApplication",
            "offers": {
              "@type": "Offer",
              "price": 0,
              "priceCurrency": "USD"
            }
          }
        ]
      },
      {
        "@type": "BreadcrumbList",
        "@id": "https://gtmstacker.com/registry/tool/aisi-eval-infrastructure/#breadcrumb",
        "itemListElement": [
          {
            "@type": "ListItem",
            "position": 1,
            "name": "GTM Stacker Registry",
            "item": "https://gtmstacker.com/registry/"
          },
          {
            "@type": "ListItem",
            "position": 2,
            "name": "AI Infrastructure",
            "item": "https://gtmstacker.com/registry/category/ai-infrastructure/"
          },
          {
            "@type": "ListItem",
            "position": 3,
            "name": "AISI eval infrastructure (ControlArena + the Inspect family)",
            "item": "https://gtmstacker.com/registry/tool/aisi-eval-infrastructure/"
          }
        ]
      }
    ]
  },
  "tool": {
    "name": "io.github.UKGovernmentBEIS/control-arena",
    "repository": {
      "url": "https://github.com/UKGovernmentBEIS/control-arena",
      "source": "github"
    },
    "homepage": "https://github.com/UKGovernmentBEIS/control-arena"
  }
}