Otari — Mozilla’s Open-Source LLM Gateway

URL: https://github.com/mozilla-ai/otari Date: 2026-07-12 Source: GitHub (mozilla-ai/otari, ⭐270) Topic: AI LLMOps / Gateway Infrastructure

Otari is an open-source, OpenAI-compatible LLM gateway from Mozilla AI that puts one endpoint in front of 40+ providers. It handles authentication (virtual API keys), cost controls (per-user/per-key budgets enforced before requests run), and usage tracking. Built with Python (3.13+), uses any-llm SDK for provider routing. All provider credentials stay in your environment. Supports standalone mode (SQLite, Docker) or full stack (Postgres, Docker Compose). Also offers a hosted platform at otari.ai and SDKs in Python, TypeScript, Rust, and Go.

Key Features

• One OpenAI-compatible endpoint for 40+ providers
• Virtual API keys with scoping and revocation
• Pre-request budget enforcement
• Full usage and spend tracking via /v1/usage
• Docker quickstart in one command
• Postgres for durable storage
• Hybrid mode: self-hosted + otari.ai platform
• Guardrails and built-in tools

Links

• See also 2026-07-12-2002-plano-proxy — AI-native proxy for agentic apps
• See also 2026-07-12-2002-mesh-llm — Distributed AI computing infra
• Related: 2026-07-12-0808-agentgateway-proxy — Agentic proxy for AI agents