LLM Infrastructure
Running and serving large language models, LLM API gateways and proxies, and retrieval-augmented generation (RAG) systems.
Repos
200
Stars
636,909
Last 30 days
+27,516
Top 100 of 200
RAGFlow is an agent-native database: a Retrieval-Augmented Generation (RAG) engine that grew into context infrastructure.
Created Dec 2023
Wrap Antigravity, ChatGPT Codex, Claude Code, Grok Build, Muse Code, Devin as an OpenAI/Gemini/Claude/Codex compatible API service, allowing you to enjoy the free Gemini Series, GPT Series, Grok Series, Claude model through API
Created Jul 2025
Open-source AI gateway. Self-hosted LLM gateway with OpenAI-, Claude- and Gemini-compatible APIs, model routing, access control, usage tracking and cost accounting.
Created Nov 2023
An open-source engine that turns continuous content streams into search and monitoring. Durable ingestion, hybrid search, alerts, and plugins for formats, models and business rules.
Created May 2023
🤖 AI Gateway | AI Native API Gateway
Created Oct 2022
Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode. Works with any SDK or agent and adds under 200µs of overhead at 5k RPS.
Created Mar 2025
An open, programmable decision layer for models and compute.
Created Aug 2025
Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don't need it.
Created Oct 2024
Cost-efficient and pluggable Infrastructure components for GenAI inference
Created Jun 2024
把腾讯WorkBuddy账号变成 OpenAI 兼容 API 的多账号网关,同时自动完成任务中心全部任务,附 Web 管理面板(账号池可视化 / 积分任务 / 配置热更新)。基于 Sliverkiss/workbuddy2api 的增强分支
Created Sep 2026
代理Discord对话Coze-Bot,实现以API形式请求GPT4模型,提供对话、文生图、图生文、知识库检索等功能。
Created Jan 2024
Manages Unified Access to Generative AI Services built on Envoy Gateway
Created Oct 2024
OpenAI 接口接入适配,支持千帆大模型平台、讯飞星火大模型、腾讯混元以及MiniMax、Deep-Seek,等兼容OpenAI接口,仅单可执行文件,配置超级简单,一键部署,开箱即用. Seamlessly integrate with OpenAI and compatible APIs using a single executable for quick setup and deployment.
Created May 2024
Ultrafast serverless GPU inference, sandboxes, and background jobs
Created Nov 2023
A Go-based OpenCode Zen / Zen Go API proxy with OpenAI & Anthropic API compatibility, key/proxy pooling, automatic routing, and WebUI.
Created Aug 2026
把腾讯WorkBuddy账号变成 OpenAI 兼容 API 的多账号网关,同时自动完成任务中心全部任务,附 Web 管理面板(账号池可视化 / 积分任务 / 配置热更新)。基于 Sliverkiss/workbuddy2api 的增强分支
Created Sep 2026
Azure OpenAI Service Proxy. Convert OpenAI official API request to Azure OpenAI API request. Support GPT-4,Embeddings,Langchain. Adapter from OpenAI to Azure OpenAI.
Created Mar 2023
🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.
Created Jul 2023
将AI Studio Playground和Build反代成API | AI Studio to OpenAI, Anthropic & Gemini Compatible API
Created Aug 2025
Gemini web → OpenAI-compatible + native Gemini API, self-hosted, no API key. Gemini 4 Argon from a paid account cookie, strict_model (429 instead of a silent downgrade), cookie pool with auto-renewal, proxy pool, admin panel, image & video generation, single Go binary · 把 Gemini 网页反代成 OpenAI 兼容 / Gemini 原生 API,付费号 cookie 直接用 4 Argon
Created May 2026
Moon Bridge 是一个用 Go 编写的协议转换与模型路由代理。对外暴露 OpenAI Responses API(/v1/responses),对内支持 Anthropic Messages、Google Gemini(GenAI)、OpenAI Chat Completions 等多种上游协议。客户端指定不同模型别名时,自动将请求路由到对应上游 Provider 并在协议间自动转换。
Created Apr 2026
A proxy for converting the OpenAI API protocol to the Google Gemini Pro protocol.
Created Dec 2023
oneapi-pro is an enterprise-grade AI API Gateway built on oneapi(one-api). It has undergone a full architectural redesign, adding support for subscription plans, active-active clusters, precise billing. 本项目与new-api | sub2api不同之处是oneapi-pro是面向企业AI网关的应用场景,并不适用于中转站
Created Jul 2026
✨Reverse-engineered API for Gemini web app. It can be used as a genuine API key from OpenAI, Gemini, and Claude.
Created Dec 2025
A proxy for Azure OpenAI API that can convert an OpenAI request into an Azure OpenAI request.
Created Mar 2023
A local AI gateway for AI agents, unifying your subscriptions and API providers with smart routing and on-device privacy protection.
Created Sep 2026
AI Proxy is a high performance AI gateway using OpenAI / Claude / Gemini protocol as the entry point. It features intelligent error handling, multi-channel management, and comprehensive monitoring. With support for multiple models, rate limiting, and multi-tenant isolation.
Created Mar 2025
🏗️ Fine-tune, build, and deploy open-source LLMs easily!
Created Sep 2023
基于 Go 实现的 OpenAI HTTP 代理(云函数、lambda 函数、自己部署均可)
Created Mar 2023
Unified large model proxy gateway, providing load balancing, fault transfer, current limiting, budget management, content auditing, caching, and MCP gateway capabilities.
Created May 2026
大模型API网关-全新AI大模型接口管理与API聚合分发系统 , 支持将多种大模型转换成统一的OpenAI兼容接口,Claude接口,Gemini接口,可供个人或者企业内部大模型API 统一管理和渠道分发使用(key管理与二次分发),支持国际国内所有主流大模型,gemini,claude,qwen3,kimi-k2,豆包等,提供单可执行文件, docker镜像,一键部署,开箱即用,完全开源,自主可控!本项目基于New-API和One-API,整合了NewAPI,OneAPI所有功能及众多第三方插件为一身,功能超强!
Created Sep 2025
llm-d Router: The intelligent entry point for inference requests
Created May 2025
Privacy proxy for your OpenAI requests
Created Oct 2025
LightningRAG is a full-stack Vue + Gin starter with a decoupled frontend and backend, plus built-in, extensible RAG (retrieval-augmented generation): knowledge bases, vector search, and integrations with many LLM and vector-store providers
Created Mar 2026
Cut LLM costs by up to 80% and unlock sub-millisecond responses with intelligent semantic caching.A drop-in, provider-agnostic LLM proxy written in Go with sub-millisecond response
Created Nov 2025
MAX API Next 社区是由来自科研机构和高校的 AGI 爱好者组织发起、维护和运营的研究驱动型技术社区,聚焦 AI Models 与 Agents 治理方向,致力于建设面向 AGI 应用时代的开放基础设施。社区关注多模型与多平台接入、国产模型持续适配、AgentOps 工程实践、成本审计、权限与安全边界、私有化部署和长期运营优化,目标是把模型服务、Agent 应用、用户组织和上游平台之间的共性治理问题沉淀为稳定、可复用、可持续演进的工程能力。
Created Jun 2026
A self-hosted, OpenAI-compatible LLM gateway with multi-provider failover, key rotation, and a built-in admin console.
Created Jul 2026
一个本地优先的AI知识库系统(RAG),用于把本地文档接入辅导搜索与大模型对话流程。目前支持md、txt、pdf(文本)、xlsx、cvs类型。支持mcp服务
Created Mar 2026
LLM-And-More is a professional, plug-and-play, llm trainer and application builder that guides you through the complete LLM workflow from data to evaluation, from training to deployment, from idea to sevice. / LLM-And-More 是一个专业、开箱即用的大模型训练及应用构建一站式解决方案,包含从数据到评估、从训练到部署、从想法到服务的全流程最佳实践。
Created Feb 2024
Access models from OpenAI, Groq, local Ollama, and others by setting llm-router as Cursor's Base URL
Created Apr 2024
An Open-source, self-hosted AI model hub with Hugging Face compatibility, accelerating vLLM/SGLang performance.
Created Jan 2026
LLM privacy gateway in Go — millisecond-latency PII and secret redaction. Used in production by PackyCode.
Created May 2026
Use Claude and GPT through one local gateway with OpenAI-compatible and Claude-native APIs.
Created Dec 2025
Grok2API & CPA egress quality guard with proxy recovery, quarantine, migration, and operations UI
Created Aug 2026
A workload for deploying LLM inference services on Kubernetes
Created Aug 2025
High-performance Go proxy gateway for 9Router LLM routing — 32K RPS, 42MB RAM, CGO-free
Created Jul 2026
llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.
Created Mar 2026
Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
Created Jul 2023
高度自定义的LLM API 增强器,支持模型与渠道级规则控制,PII过滤,非流式转伪流式,超时控制,SystemPrompt修改注入,多代理出口,自定义参数覆盖与请求头请求体修改
Created Jul 2025
企业级 LLM API 快速集成系统,支持OpenAI、Azure、文心一言、讯飞星火、通义千问、智谱GLM、Gemini、DeepSeek、Anthropic Claude以及OpenAI格式的模型等,简洁的页面风格,轻量高效且稳定,支持Docker一键部署。
Created Nov 2023
9RouterX (9R Next Go Generation): Single-binary Go AI router, LLM gateway & 20-40% token saver. Route Claude Code, Cursor & Codex across 155 providers, 28 OAuth & 248 free models (24 keyless) — OpenAI, Claude, Gemini, DeepSeek, Groq, Ollama & MCP.
Created Nov 2026
A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.
Created Apr 2025
An Open Standard for Packaging, Distributing and Running LLMs in Cloud-Native Environments
Created Aug 2024
A lightweight, production-ready RAG (Retrieval Augmented Generation) library in Go.
Created Jul 2024
Прозрачный обратный прокси между вашими клиентами и LLM-провайдером: убирает чувствительные данные из запросов к модели и восстанавливает их в ответах.
Created Jul 2026
JoyCode API 代理 — 协议翻译成 Anthropic/OpenAI 兼容格式,让 Claude Code 直接调用 JoyCode支持的模型,比如GLM 5.1等
Created Apr 2026
AI API 中转站监控、推荐运营与 OpenAI 兼容专属网关系统,支持分层探测、健康评分、用量计量、告警审计和 Docker 自托管。
Created Jul 2026
An open-source, cloud-native, high-performance gateway unifying multiple LLM providers, from local solutions like Ollama to major cloud providers such as OpenAI, Groq, Cohere, Anthropic, Cloudflare and DeepSeek.
Created Jan 2025
这是一个为 Zai 提供 OpenAI API 兼容接口的代理服务,允许开发者通过标准的 OpenAI API 格式访问 Zai 的 GLM-4.5 模型。
Created Sep 2025
Claude2API 是基于 Go + Docker 构建的 Claude.ai API 兼容网关、账号池与网页镜像服务,支持 OpenAI Chat Completions、Responses 和 Anthropic Messages 接口,可接 入 Claude Code、Codex CLI 等客户端,并提供账号轮询、流式输出、多轮对话、多模态图片、Thinking、Function Calling、Tool Use、API Key 鉴权、调用日志和后台管理。
Created Aug 2026
Distributed KV cache scheduling & offloading libraries
Created Apr 2025
🚀 Autocache - Intelligent Anthropic API Cache Proxy Automatically inject cache-control fields into Claude API requests to reduce costs by up to 90% and latency by up to 85%. Works as a transparent drop-in replacement for popular AI platforms like n8n, Flowise, Make.com, LangChain, and LlamaIndex—no code changes required
Created Oct 2025
RogerAI - a two-way radio for GPUs: a marketplace for self-hosted LLMs (CLI + broker + site).
Created Jun 2026
go基于向量数据库与GPT3.5的通用本地知识库方案(A universal local knowledge base solution based on vector database and GPT3.5)
Created Mar 2023
Codex API 代理服务,提供 OpenAI / Claude 多协议兼容接口 用户交流群 : 1081291958
Created Mar 2026
Kiro 账号池管理系统 |自动注册Kiro账号 | 将 Amazon Q Developer API 转换为 OpenAI 兼容格式 | 支持多账号池、OIDC 自动认证、令牌自动刷新、Web 管理控制台 |AI账号管理工具
Created Feb 2026
Repos sorted by Jev.