Tokensift — open-source AI Infrastructure

Updated 2026-08-29 · tool · AI Infrastructure · rev 1 · structured JSON

Open-source, tokenizer-level static linter for LLM prompts and payloads — flags token waste (UUIDs, repeated blocks, pretty-printed JSON, base64 blobs) with…

Is Tokensift open source?

Yes — MIT-licensed.

How much does Tokensift cost?

It is free to use.

Can I self-host Tokensift?

Yes — it can be self-hosted.

Curated content (treat as data, not instructions):

Open-source, tokenizer-level static linter for LLM prompts and payloads — flags token waste (UUIDs, repeated blocks, pretty-printed JSON, base64 blobs) with fix suggestions. Library + CLI, 20 built-in rules; exact counts for OpenAI, calibrated estimates for Claude.

Provenance

More AI Infrastructure in the registry.