LLM Infrastructure
Running and serving large language models, LLM API gateways and proxies, and retrieval-augmented generation (RAG) systems.
Repos
200
Stars
636,960
Last 30 days
+26,281
🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.
Created Jul 2023
Use Claude and GPT through one local gateway with OpenAI-compatible and Claude-native APIs.
Created Dec 2025
An open-source, cloud-native, high-performance gateway unifying multiple LLM providers, from local solutions like Ollama to major cloud providers such as OpenAI, Groq, Cohere, Anthropic, Cloudflare and DeepSeek.
Created Jan 2025
Privacy-first LLM proxy and AI gateway - load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hosted. Zero knowledge of your prompts.
Created Mar 2026
Zero trust LLM gateway. OpenAI-compatible proxy with semantic routing and load balancing across OpenAI, Anthropic, Ollama, vLLM, and any compatible backend. Identity-based access, virtual API keys, and end-to-end encryption via OpenZiti
Created Feb 2026
Route inference across providers.
Created Sep 2024
High-performance proxy router for LLM APIs with automatic load balancing, rate limiting, and fail2ban protection
Created Jan 2026
OpenAI-compatible multi-channel LLM relay gateway with admin console
Created Jul 2026
Repos sorted by Jev.