Categories

LLM Infrastructure

Running and serving large language models, LLM API gateways and proxies, and retrieval-augmented generation (RAG) systems.

Repos

200

Stars

636,960

30 days

+26,281

QuantumNous

Open-source AI gateway. Self-hosted LLM gateway with OpenAI-, Claude- and Gemini-compatible APIs, model routing, access control, usage tracking and cost accounting.

Created Nov 2023

ai-gateway api-management claude deepseek gemini
Total 52,337
30 days +2,082
higress-group

🤖 AI Gateway | AI Native API Gateway

Created Oct 2022

ai-gateway ai-native api-gateway cloud-native envoy
Total 9,810
30 days +206
maximhq

Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode. Works with any SDK or agent and adds under 200µs of overhead at 5k RPS.

Created Mar 2025

agent-gateway ai-gateway ai-governance api-gateway claude-code
Total 9,797
30 days +835
tbphp

Self-hosted AI gateway for multi-channel, multi-credential setups — API keys and subscription accounts, scheduling, failover, request logs and usage. 自托管 AI 网关:多渠道多凭据统一接入,含密钥与订阅账号、调度容错、日志与用量。

Created Jun 2025

ai-gateway anthropic api-gateway claude claude-code
Total 7,233
30 days +105
vllm-project

An open, programmable decision layer for models and compute.

Created Aug 2025

ai-gateway guardrails inference kubernetes llm
Total 6,518
30 days +285
theagentrouter

Manages Unified Access to Generative AI Services built on Envoy Gateway

Created Oct 2024

ai ai-gateway api-gateway envoy envoyproxy
Total 2,430
30 days +175
astaxie

TokenHub gives enterprises a private gateway to unify AI model access and governance, making every request controllable, traceable, and attributable.

Created Jun 2026

ai ai-gateway enterprise gateway
Total 1,450
30 days +64
kittors

Self-hosted AI gateway for coding CLIs — one OpenAI/Claude/Gemini/Codex-compatible endpoint, with a multi-tenant web console, request logs, and spend quotas.

Created Feb 2026

ai-gateway anthropic api-gateway claude-code gemini-cli
Total 1,061
30 days +27
Calcium-Ion

A local AI gateway for AI agents, unifying your subscriptions and API providers with smart routing and on-device privacy protection.

Created Sep 2026

ai-gateway
Total 588
30 days +287
WizisCool

Modern web console for CLIProxyAPI v8+ with provider and OAuth management, live routing, request analytics, pricing, and MCP.

Created Sep 2026

ai-gateway cliproxyapi cpa dashboard golang
Total 466
30 days +252
dataiku

Privacy proxy for your OpenAI requests

Created Oct 2025

ai-gateway data-anonymization data-masking llm-security pii-detection
Total 456
30 days +17
caidaoli

Self-hosted AI API gateway for Claude Code, Codex and Gemini: pool API keys and subscription accounts behind one endpoint with failover, protocol conversion and cost tracking.

Created Sep 2025

ai ai-gateway anthropic api-proxy claude-api
Total 446
30 days +19
yolorouter

A self-hosted, OpenAI-compatible LLM gateway with multi-provider failover, key rotation, and a built-in admin console.

Created Jul 2026

ai-gateway api-key-management claude-code-proxy litellm-alternative literouter
Total 390
30 days +184
packyme

LLM privacy gateway in Go — millisecond-latency PII and secret redaction. Used in production by PackyCode.

Created May 2026

ai-gateway data-redaction gitleaks golang grpc
Total 342
30 days +8
atopos31

Unified LLM gateway with weighted load balancing, observability & cost tracking. 统一的 LLM 网关,提供权重负载均衡、可观测性与费用追踪。

Created Aug 2025

ai ai-gateway claude claude-ai claude-code
Total 323
30 days +7
lij768423-svg

Grok2API & CPA egress quality guard with proxy recovery, quarantine, migration, and operations UI

Created Aug 2026

ai-gateway circuit-breaker grok2api patch proxy
Total 320
30 days +5
yanstd888

9RouterX (9R Next Go Generation): Single-binary Go AI router, LLM gateway & 20-40% token saver. Route Claude Code, Cursor & Codex across 155 providers, 28 OAuth & 248 free models (24 keyless) — OpenAI, Claude, Gemini, DeepSeek, Groq, Ollama & MCP.

Created Nov 2026

9router 9routerx ai-gateway ai-proxy ai-routing
Total 255
30 days +255
mydisha

Your friendly, blazing-fast, self-hostable AI gateway

Created May 2026

ai ai-gateway ai-router anthropic caveman
Total 167
30 days +17
voidmind-io

Privacy-first LLM proxy and AI gateway - load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hosted. Zero knowledge of your prompts.

Created Mar 2026

ai ai-gateway anthropic api-gateway docker
Total 149
30 days +5
yreidev

把腾讯 WorkBuddy 账号变成 OpenAI 兼容 API 的多账号网关。请求头照官方桌面端逐项对齐,主备入口 + HTTPDNS 自动切换,国内 / 国际双域合池,企业号可用 Claude,签到与成长任务全自动,React 管理面板支持一键更新。原版 workbuddy2api-panel 的持续同步增强版,Docker / 单文件部署。

Created Sep 2026

account-pool ai-gateway api-gateway claude codebuddy
Total 120
30 days +108
openziti

Zero trust LLM gateway. OpenAI-compatible proxy with semantic routing and load balancing across OpenAI, Anthropic, Ollama, vLLM, and any compatible backend. Identity-based access, virtual API keys, and end-to-end encryption via OpenZiti

Created Feb 2026

agentic-ai ai-gateway ai-infrastructure ai-security anthropic
Total 109
30 days +9
murongg

A lightweight, self-hosted AI subscription gateway for internal teams, with account pooling, access control, and usage tracking.

Created Sep 2026

account-pooling ai-gateway claude cliproxyapi codex
Total 96
30 days +11
ShadowSafin

AI inference gateway and control plane for multi-provider routing, policies, caching, tools, and observability.

Created Oct 2026

ai ai-gateway api-gateway inference llm
Total 93
30 days +3
greynewell

Route inference across providers.

Created Sep 2024

ai-gateway ai-infrastructure anthropic api-gateway cost-tracking
Total 93
30 days +0
adrianliechti

Inference Hub for AI at Scale

Created Sep 2023

ai-gateway azure-openai claude function-calling groq
Total 82
30 days +1
loxilb-io

Inference-aware L4/L7 load balancer for LLM serving fleets, forked from loxilb

Created Jul 2026

ai-gateway ebpf kubernetes kv-cache llm-inference
Total 71
30 days +66
PIXEL-API

基于 Sub2API 二开的 AI API 网关:多平台账号接入、账号共享与广场调度、Token 级计费与分账

Created May 2026

account-sharing ai-gateway api-gateway claude gemini
Total 65
30 days +1
qianfree

Team-API 专为团队、组织、企业打造的多租户大模型 API 网关。具备完整权限管理与审计能力,助力团队更好地使用AI大模型。支持Claude/OpenAI/Gemini等主力模。QQ交流群:1095286563

Created Apr 2026

ai-gateway claude gemini goframe openai
Total 64
30 days +7
DouDOU-start

插件化 AI 中转网关,平台能力以 gRPC 插件独立装载,支持多账号调度、计费与拼车共享,docker compose 一键自部署

Created Mar 2026

2api account-sharing ai-gateway claude-2api kiro-2api
Total 63
30 days +1
ZiChuanLan

OpenAI-compatible multi-channel LLM relay gateway with admin console

Created Jul 2026

ai-gateway anthropic api-gateway gemini golang
Total 55
30 days +14
OpenCSGs

CSGLite is a lightweight local LLM inference platform. One command downloads, loads, and chats with models. It ships a web UI, OpenAI-compatible API, llama.cpp inference, resumable downloads, marketplace browsing, and one-click AI app and coding-agent setup—all in a single cross-platform binary.

Created Mar 2026

ai-gateway cc-switch inference ollama-api
Total 53
30 days +4
LAGcomcom

Lamp Guide:zen-gate 的 Windows 桌面重构版。统一模型与渠道管理、聚合轮换、Agent 接入、日夜主题,提供完整安装向导。

Created Nov 2026

ai-gateway golang model-routing webview2 windows
Total 51
30 days +51

Repos sorted by Jev.