The best open-source AI Infrastructure tools, curated: 54 in the registry, including Headroom, Agno, Anomalib. Every tool is open-source; each entry lists license, self-hostability, and links.
Headroom— tool, 2026-09-08 Open-source context-compression proxy for AI agents (Rust core; Python/TypeScript SDKs): sits between your agent and the LLM and compresses tool outputs, logs, JSON, code and RAG chunks with format-specific, reversible compressors before the model sees them — no changes to agent code.
Agno— tool, 2026-09-03 OSS alt to LangGraph; ~41632 stars; via ossinsight.io
Anomalib— tool, 2026-09-03 ~6024 stars; via ossinsight.io
DVC— tool, 2026-09-03 OSS alt to proprietary MLOps asset stores; ~15800 stars; complements git-lfs, ComfyUI workflow files; Closest fit for versioning large generated assets/models alongside Git. Branch is otherwise thin — no purpose-built gen-asset versioning product exists. No model.
Haystack— tool, 2026-09-03 Pure Apache; production-focused
Hermes Agent— tool, 2026-09-03 Nous Research self-improving agent; persistent memory + autonomous skill creation; multi-LLM-provider; ~228k stars (anomalously high — verify hype); CAVEAT: autonomous skill-creation = arbitrary code execution by design — sandbox before use
Inspect AI— tool, 2026-09-03 UK AISI; rigorous eval (verified)
MiroFish— tool, 2026-09-03 Multi-agent AI prediction engine: builds a digital-twin sandbox of thousands of agents to forecast trajectories from seed data; ~71k stars (verify velocity); AGPL network-copyleft
Ollama— tool, 2026-09-03 Easiest local runtime; don't expose port publicly
One API— tool, 2026-09-03 OSS alt to LiteLLM; ~36267 stars; via ossinsight.io
Ray— tool, 2026-09-03 OSS alt to proprietary GPU-orchestration platforms; ~43500 stars; complements Celery, SkyPilot; General distributed-compute engine; also serves the cost/GPU/model orchestration branch. No model.
SkyPilot— tool, 2026-09-03 OSS alt to proprietary GPU-cost platforms; ~10500 stars; complements Ray; Provisions cheapest GPUs across clouds with spot/failover — the strongest OSS fit for the cost/GPU branch (STRUCTURAL: most cost-optimization is commercial). No model.
SurfSense— tool, 2026-09-03 OSS alt to NotebookLM; ~15811 stars; via openalternative.co
SwanLab— tool, 2026-09-03 OSS alt to Weights & Biases; ~4128 stars; via ossinsight.io
Unsloth— tool, 2026-09-03 AGPL on Studio/CLI; fastest kernels
JSON Feed 1.1 emerges as the agent-preferred syndication format— news, 2026-09-03 Autonomous crawlers index JSON Feed endpoints more reliably than legacy XML because payloads parse natively without entity resolution, escaping, or namespace handling. Publishers extend it with underscore-prefixed keys to carry structured agent metadata.
HydraDB— tool, 2026-09-02 Open-source distributed graph database in Rust built on object storage — decouples storage from compute so data nodes and indexers scale independently. Neo4j-compatible via Bolt 5.x, with an HTTP query API supporting OpenCypher and GraphBLAS traversal. An infra / agent-memory / context layer.
Omnara— tool, 2026-09-02 Open-source infrastructure for running production-grade AI agents: you build the agent, Omnara handles hosting, durable state (Postgres), sandboxes, permissions, and secrets, with any model (Anthropic/OpenAI/Bedrock/open endpoints). Cloud or fully self-hosted in your own VPC — an OSS alternative to hosted managed-agent platforms.
zg (zvec-grep)— tool, 2026-09-02 A local-first search tool for people and AI agents that combines ripgrep, BM25, and on-device vector search across code, docs, and data — indexing and search stay on the machine. Auto-configures agents (Claude Code, Codex, OpenCode) and reports fewer tool-calls/tokens on SWE-QA-Bench.
OpenBot— tool, 2026-09-02 Gives each AI agent its own isolated computer (browser, file workspace, tools) behind a policy gateway that reviews and audit-logs every action before it runs. Works with AG-UI agents (LangGraph, Mastra, CrewAI, Pydantic AI); model-agnostic and self-hostable.
FreeToken— tool, 2026-09-01 Edge-native Mixture-of-Experts inference engine (UC Berkeley) that runs frontier open-weight models on consumer hardware — e.g. a 35B model on an 8GB GPU — by treating GPU/CPU/host-memory as one bandwidth-adaptive platform; 2–4× faster than Ollama on MoE. Desktop app + Python CLI.
SIE (Superlinked Inference Engine)— tool, 2026-09-01 Self-hosted, OpenAI-compatible inference server that runs 85–100+ open models behind one API (encode/score/extract/generate), loading and LRU-evicting models by traffic so one GPU serves a rotating set instead of one server per model. Plugs into Qdrant, Weaviate, Chroma, LanceDB, LangChain, LlamaIndex.
Apple 'Agent Seer' research: API and parameter design break agent tool-calling— news, 2026-09-01 Apple research reporting that a large share of agent tool-calling failures trace to API and parameter design — schema shapes, parameter naming and documentation — rather than model capability. Practical implication for GTM-agent builders: design tool surfaces for the model, and keep a human approval step on consequential actions.
Itsuki— tool, 2026-08-29 Open-source memory engine for AI agents over REST API + MCP: captures conversations/events into typed, versioned, reversible objects (facts, entities, relationships, events) accessible across 26+ tools, with an audit trail and JSON export.
Tokensift— tool, 2026-08-29 Open-source, tokenizer-level static linter for LLM prompts and payloads — flags token waste (UUIDs, repeated blocks, pretty-printed JSON, base64 blobs) with fix suggestions. Library + CLI, 20 built-in rules; exact counts for OpenAI, calibrated estimates for Claude.
GLM-5.3 is now open-weight— news, 2026-08-28 Zhipu's GLM-5.3 was released as an open-weight model on Hugging Face — a runnable-locally option for bulk GTM classification/enrichment/drafting versus paying per API call. Exact commercial license terms are NOT independently verified.
Accept: text/markdown — serve Markdown to AI agents via content negotiation— news, 2026-08-27 An open HTTP technique (not a SaaS, no vendor/lock-in): clients send Accept: text/markdown and the server returns Markdown from the same URL that still serves HTML to browsers — cutting token bloat for agent readers. Copy-paste recipes for Nginx, Caddy, Next.js, Astro, Django, Rails.
Pipecat— tool, 2026-08-26 Open-source framework for real-time voice agents, multimodal apps, and live AI — composable pipelines with swappable STT (22+), TTS (35+), any LLM, and WebRTC/WebSocket/Twilio/WhatsApp transports, plus multi-agent composition and sub-200ms latency. Maintained by Daily.
Langfuse— tool, 2026-08-25 LLM engineering platform: observability, evals, prompt management, traces, and datasets — for diagnosing why a marketing/GTM agent made a bad recommendation, not just that it failed.
Milvus for DeepSeek Harness— tool, 2026-08-25 A read-only DeepSeek Harness plugin that lets an agent inspect and query Milvus / Zilliz Cloud vector stores — entity lookup, scalar queries, BM25 full-text, and dense+BM25 hybrid retrieval. The plugin bounds the tool surface (no insert/delete/admin) at the execution boundary, not by prompt.
Quartermaster (QM)— tool, 2026-08-25 An open, multiplayer agent harness for organizations: every employee and Slack room gets a scoped workspace (separate memory, files, permissions, sandbox), with shared skills, admin-gated promotion, crons/background watches, and one-command deploy to Fly/AWS. Model-agnostic (Claude/Codex/OpenCode).