liter-llm
One API for every LLM — in your language, without the rewrites.
What and Why?
liter-llm gives you a single, consistent way to call any large language model. Reach 174 providers — OpenAI, Anthropic, Google, Bedrock, and more — through one client, and switch models by changing a name instead of your code. The same client ships to 14 languages, all built on one Rust core, so you get identical behavior everywhere. Need a gateway? Point your existing OpenAI SDK at the built-in, drop-in proxy — no rewrites, no separate service to run.
- Work with any provider without rewrites — 174 providers behind one API; change the model name to switch, your code stays the same.
- Use it from your language — Python, TypeScript, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, and WebAssembly, all built on the same Rust core, plus a C/FFI surface for everything else.
- Secure by default — API keys are wrapped, redacted from logs, and never serialized; cloud auth for Azure, AWS, and Vertex refreshes and rotates for you.
- One small binary — the client, a drop-in OpenAI-compatible proxy, and an MCP server for AI agents ship in a single 35 MB binary with no interpreter and no dependency tree to compromise.
- Stay up when a provider fails — route by cost, latency, or weight and fall back automatically, with built-in rate limiting, caching, and cost tracking.
Deeper details — observability, middleware, and the full endpoint list — live in the documentation.
We credit litellm for proving the category; our provider registry was bootstrapped from theirs. See ATTRIBUTIONS.md.
Features
Quick Start
Language Packages
pip install liter-llm
See Python README for full documentation.
pnpm add @xberg-io/liter-llm
See Node.js README for full documentation.
cargo add liter-llm
See Rust crate for full documentation.
go get github.com/xberg-io/liter-llm/packages/go/v2
See Go README for full documentation.
Available on Maven Central as io.xberg.literllm:liter-llm. See Java README for the dependency snippet and current version.
gem install liter_llm
See Ruby README for full documentation.
composer require xberg-io/liter-llm
See PHP README for full documentation.
dotnet add package LiterLlm
See .NET README for full documentation.
Add {:liter_llm, "~> 1.6"} to your mix.exs dependencies. See Elixir README for full documentation.
dart pub add liter_llm
See Dart README for full documentation.
Add via Swift Package Manager. See Swift README for full documentation.
Available on Maven Central as io.xberg.literllm:liter-llm-android. See Kotlin README for the dependency snippet and current version.
See Zig README for installation and usage.
pnpm add @xberg-io/liter-llm-wasm
See WebAssembly README for full documentation.
Build from source as part of this workspace. See FFI crate for full documentation.
The liter-llm CLI ships both the OpenAI-compatible proxy and the MCP tool server. Install it any of these ways:
brew install xberg-io/tap/liter-llm
cargo install liter-llm-cli # from crates.io
cargo binstall liter-llm-cli # prebuilt binary from GitHub Releases
pnpm dlx @xberg-io/liter-llm-cli --help # npm (self-installs the binary)
docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-your-key ghcr.io/xberg-io/liter-llm
Windows users can install the same binary through Scoop:
scoop bucket add xberg https://github.com/xberg-io/scoop-bucket
scoop install liter-llm
Then run the proxy or the MCP server:
liter-llm api --config liter-llm-proxy.toml # OpenAI-compatible proxy (22 endpoints)
liter-llm mcp --transport stdio # MCP tool server (stdio)
liter-llm mcp --transport http --port 3001 # MCP tool server (Streamable HTTP)
See the MCP server guide and the proxy guide for transports, routing, virtual keys, and budgets. To use the MCP server inside a coding agent, install the liter-llm plugin (below) — it registers the server for your MCP client. The CLI release compatibility requirements below also apply to auto-installed binaries.
AI Coding Assistants
Install the liter-llm plugin from xberg-io/liter-llm. It ships the liter-llm agent skills (chat, streaming, tools, embeddings across 174 providers) — select your runtime below.
The bundle targets Claude Code, Codex, Cursor, Gemini CLI, Kimi, OpenCode, Factory Droid, Hermes, Pi, Agent Plugins and GitHub Copilot CLI. Read the plugin guide for runtime resources, CLI prerequisites and configuration. MCP-capable adapters expose the product's tools; Hermes uses a local stdio bridge and explicitly selected skills. Pi loads skills and prompt templates and requires the product CLI for executable operations.
Install only the public Agent Skills, without a plugin or MCP registration, using the skills CLI:
npx skills add xberg-io/liter-llm --list
npx skills add xberg-io/liter-llm
Choose the skills and target agents interactively. Installing instructions does not install the product CLI or configure provider credentials.
Published CLI 2.2.3 predates the local-stdio trust and deferred-provider-credential fixes. Until a CLI release includes them, use a source build for startup without an extra auth config:
cargo install --git https://github.com/xberg-io/liter-llm \
--rev f14fb8862f9cc73e2c4ab38a2876efe77fab8bf4 --locked liter-llm-cli
Put the installed binary on the assistant's PATH. If a plugin has already cached an older binary,
remove that plugin's bin/liter-llm (or bin/liter-llm.exe) before retrying so it selects the source build.
The released 2.2.3 binary requires mcp.stdio_trust_local = true or a configured mcp.stdio_key_id;
configured providers may also require credentials at startup. The source build defers missing provider
credentials until a request uses that provider. HTTP authentication remains configured separately.
The source-built stdio MCP server trusts its local host by default and starts without a provider key.
Set provider credentials such as OPENAI_API_KEY or ANTHROPIC_API_KEY before sending
requests to that provider; missing credentials fail the request. HTTP MCP access retains
its configured authentication. Use liter-llm api for the proxy and liter-llm mcp for MCP;
chat and streaming are SDK, proxy or MCP operations rather than CLI subcommands.
/plugin marketplace add xberg-io/liter-llm
/plugin install liter-llm@liter-llm
/plugins add https://github.com/xberg-io/liter-llm
Then search for liter-llm and select Install Plugin.
Settings → Plugins → Add from URL → https://github.com/xberg-io/liter-llm, then select liter-llm.
From a checkout of this repository:
gemini extensions install ./plugin
The manifest is inside plugin/. See the Gemini extension reference
for local installation and updates.
droid plugin marketplace add https://github.com/xberg-io/liter-llm
droid plugin install liter-llm@liter-llm
copilot plugin marketplace add https://github.com/xberg-io/liter-llm
copilot plugin install liter-llm@liter-llm
Install from a local checkout now:
pi install ./plugin
After the next plugin release, install the same package used by OpenCode:
pi install npm:@xberg-io/opencode-liter-llm
Pi loads the bundled skills and prompt templates. Use /liter-llm-chat and /liter-llm-stream
to start a task. Install the CLI separately as
shown in the plugin README.
This package does not register MCP servers, hooks, or executable extensions in Pi.
Add the package to opencode.json:
{
"$schema": "https://opencode.ai/config.json",
"plugin": ["@xberg-io/opencode-liter-llm"]
}
Documentation
Full guides, the unified chat() API for every binding, multimodal I/O, the proxy/gateway, and the complete provider list live at docs.liter-llm.xberg.io.
Contributing
Contributions are welcome! See CONTRIBUTING.md for guidelines.
Join our Discord community for questions and discussion.
Part of Xberg.io
- Xberg — document intelligence: text, tables, metadata from 101 formats with optional OCR.
- Xberg Enterprise — managed extraction API with SDKs, dashboards, and observability.
- crawlberg — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
- html-to-markdown — fast, lossless HTML→Markdown engine.
- liter-llm — universal LLM API client with native bindings for 14 languages and 174 providers.
- tree-sitter-language-pack — tree-sitter grammars and code-intelligence primitives.
- alef — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.
License
MIT — see LICENSE for details.
