Categories

LLM Infrastructure

Running and serving large language models, LLM API gateways and proxies, and retrieval-augmented generation (RAG) systems.

Repos

200

Stars

636,933

30 days

+26,248

QuantumNous

Open-source AI gateway. Self-hosted LLM gateway with OpenAI-, Claude- and Gemini-compatible APIs, model routing, access control, usage tracking and cost accounting.

Created Nov 2023

ai-gateway api-management claude deepseek gemini
Total 52,334
30 days +2,080
tbphp

Self-hosted AI gateway for multi-channel, multi-credential setups — API keys and subscription accounts, scheduling, failover, request logs and usage. 自托管 AI 网关:多渠道多凭据统一接入,含密钥与订阅账号、调度容错、日志与用量。

Created Jun 2025

ai-gateway anthropic api-gateway claude claude-code
Total 7,233
30 days +105
beam-cloud

Ultrafast serverless GPU inference, sandboxes, and background jobs

Created Nov 2023

autoscaler cloudrun cuda developer-productivity distributed-computing
Total 1,860
30 days +32
bricks-cloud

🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.

Created Jul 2023

ai anthropic api artificial-intelligence azure
Total 1,234
30 days +6
kittors

Self-hosted AI gateway for coding CLIs — one OpenAI/Claude/Gemini/Codex-compatible endpoint, with a multi-tenant web console, request logs, and spend quotas.

Created Feb 2026

ai-gateway anthropic api-gateway claude-code gemini-cli
Total 1,061
30 days +27
zexadev

Gemini web → OpenAI-compatible + native Gemini API, self-hosted, no API key. Gemini 4 Argon from a paid account cookie, strict_model (429 instead of a silent downgrade), cookie pool with auto-renewal, proxy pool, admin panel, image & video generation, single Go binary · 把 Gemini 网页反代成 OpenAI 兼容 / Gemini 原生 API,付费号 cookie 直接用 4 Argon

Created May 2026

api-proxy chrome-fingerprint free-gemini-api gemini gemini-4-argon
Total 940
30 days +289
WizisCool

Modern web console for CLIProxyAPI v8+ with provider and OAuth management, live routing, request analytics, pricing, and MCP.

Created Sep 2026

ai-gateway cliproxyapi cpa dashboard golang
Total 466
30 days +252
caidaoli

Self-hosted AI API gateway for Claude Code, Codex and Gemini: pool API keys and subscription accounts behind one endpoint with failover, protocol conversion and cost tracking.

Created Sep 2025

ai ai-gateway anthropic api-proxy claude-api
Total 446
30 days +19
matrixhub-ai

An Open-source, self-hosted AI model hub with Hugging Face compatibility, accelerating vLLM/SGLang performance.

Created Jan 2026

artificial-intelligence dynamo huggingface kubernetes llm
Total 366
30 days +11
thushan

High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.

Created May 2025

ai amd golang intel llama-cpp
Total 338
30 days +20
T-Crypt

Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and how that access is handed off.

Created Oct 2026

comfyui golang gpu inference llama-cpp
Total 300
30 days +186
raketenkater

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

Created Mar 2026

cuda gguf golang inference-server llama-cpp
Total 284
30 days +3
inference-gateway

An open-source, cloud-native, high-performance gateway unifying multiple LLM providers, from local solutions like Ollama to major cloud providers such as OpenAI, Groq, Cohere, Anthropic, Cloudflare and DeepSeek.

Created Jan 2025

agnostic anthropic api deepseek-v4-pro gateway
Total 208
30 days +7
lordmathis

Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.

Created Jul 2025

llama-cpp llama-server llamacpp llm llm-inference
Total 162
30 days +2
voidmind-io

Privacy-first LLM proxy and AI gateway - load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hosted. Zero knowledge of your prompts.

Created Mar 2026

ai ai-gateway anthropic api-gateway docker
Total 149
30 days +5
yreidev

把腾讯 WorkBuddy 账号变成 OpenAI 兼容 API 的多账号网关。请求头照官方桌面端逐项对齐,主备入口 + HTTPDNS 自动切换,国内 / 国际双域合池,企业号可用 Claude,签到与成长任务全自动,React 管理面板支持一键更新。原版 workbuddy2api-panel 的持续同步增强版,Docker / 单文件部署。

Created Sep 2026

account-pool ai-gateway api-gateway claude codebuddy
Total 120
30 days +108
openziti

Zero trust LLM gateway. OpenAI-compatible proxy with semantic routing and load balancing across OpenAI, Anthropic, Ollama, vLLM, and any compatible backend. Identity-based access, virtual API keys, and end-to-end encryption via OpenZiti

Created Feb 2026

agentic-ai ai-gateway ai-infrastructure ai-security anthropic
Total 109
30 days +9
raiyanyahya

Selfhost modern LLM stacks. Run the whole fleet from your terminal

Created Jun 2026

ai-agents bubbletea cli docker fastapi
Total 105
30 days +0
murongg

A lightweight, self-hosted AI subscription gateway for internal teams, with account pooling, access control, and usage tracking.

Created Sep 2026

account-pooling ai-gateway claude cliproxyapi codex
Total 96
30 days +11
hypit-ai

Multi-tenant LLM API gateway for Claude and Codex — single Go binary, USD wallet, Alipay top-up, per-token caps, status-page health, fumadocs-style /docs.

Created May 2026

alipay anthropic api-gateway claude codex
Total 84
30 days +25
ZiChuanLan

OpenAI-compatible multi-channel LLM relay gateway with admin console

Created Jul 2026

ai-gateway anthropic api-gateway gemini golang
Total 55
30 days +14
localcloud-sh

Stop paying for AI APIs during development. LocalCloud runs everything locally - GPT-level models, databases, all free.

Created Jun 2025

ai ai-app ai-development ai-development-tools ai-infrastructure
Total 51
30 days +0

Repos sorted by Jev.