Categories

LLM Infrastructure

Running and serving large language models, LLM API gateways and proxies, and retrieval-augmented generation (RAG) systems.

Repos

200

Stars

636,933

30 days

+26,248

infiniflow

RAGFlow is an agent-native database: a Retrieval-Augmented Generation (RAG) engine that grew into context infrastructure.

Created Dec 2023

agent-harness agent-native-database agentic-ai agentic-retrieval agentic-search
Total 94,563
30 days +2,085
The-Vibe-Company

An open-source engine that turns continuous content streams into search and monitoring. Durable ingestion, hybrid search, alerts, and plugins for formats, models and business rules.

Created May 2023

ai api chatbot chatgpt database
Total 39,691
30 days +123
Tencent

Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.

Created Jul 2025

agent agentic ai chatbot dsh-plugin
Total 37,868
30 days +3,034
mostlygeek

Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don't need it.

Created Oct 2024

ai golang llama llm localllama
Total 6,284
30 days +255
looplj

⚡️ Open-source AI Gateway — Use any SDK to call 100+ LLMs. Built-in failover, load balancing, cost control & end-to-end tracing.

Created Sep 2025

agent agents ai anthropic anthropic-api
Total 5,487
30 days +129
theagentrouter

Manages Unified Access to Generative AI Services built on Envoy Gateway

Created Oct 2024

ai ai-gateway api-gateway envoy envoyproxy
Total 2,430
30 days +175
astaxie

TokenHub gives enterprises a private gateway to unify AI model access and governance, making every request controllable, traceable, and attributable.

Created Jun 2026

ai ai-gateway enterprise gateway
Total 1,450
30 days +64
xllm-go

集成了openai-api、coze、deepseek、cursor、windsurf、qodo、blackbox、you、grok、bing 绘画 多款AI的聊天逆向接口适配到 OpenAI API 标准接口服务端。

Created Jun 2023

ai bing claude coze cursor
Total 1,247
30 days +6
bricks-cloud

🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.

Created Jul 2023

ai anthropic api artificial-intelligence azure
Total 1,234
30 days +6
VoAPI

🎉 全新下一代高颜值、高性能、高扩展的智能AI大模型API聚合分发系统 | A new next-generation high-value, high-performance, and highly scalable intelligent AI large-model API aggregation and distribution

Created Sep 2024

ai ai-agents api-gateway chatgpt claude-code
Total 1,095
30 days +11
ntthanh2603

✨Reverse-engineered API for Gemini web app. It can be used as a genuine API key from OpenAI, Gemini, and Claude.

Created Dec 2025

ai api chatbot claude docker
Total 676
30 days +78
diemus

A proxy for Azure OpenAI API that can convert an OpenAI request into an Azure OpenAI request.

Created Mar 2023

ai azure azure-openai chatgpt gpt3
Total 630
30 days +1
kaito-project

🏗️ Fine-tune, build, and deploy open-source LLMs easily!

Created Sep 2023

ai buildkit chatgpt docker fine-tuning
Total 544
30 days +6
llm-d

llm-d Router: The intelligent entry point for inference requests

Created May 2025

ai gateway-api inference kubernetes networking
Total 472
30 days +50
LightningRAG

LightningRAG is a full-stack Vue + Gin starter with a decoupled frontend and backend, plus built-in, extensible RAG (retrieval-augmented generation): knowledge bases, vector search, and integrations with many LLM and vector-store providers

Created Mar 2026

agent ai deepseek dify gin
Total 454
30 days +1
caidaoli

Self-hosted AI API gateway for Claude Code, Codex and Gemini: pool API keys and subscription accounts behind one endpoint with failover, protocol conversion and cost tracking.

Created Sep 2025

ai ai-gateway anthropic api-proxy claude-api
Total 446
30 days +19
messkan

Cut LLM costs by up to 80% and unlock sub-millisecond responses with intelligent semantic caching.A drop-in, provider-agnostic LLM proxy written in Go with sub-millisecond response

Created Nov 2025

ai badgerdb cache claude cost-optimization
Total 427
30 days +4
veyliss

一个本地优先的AI知识库系统(RAG),用于把本地文档接入辅导搜索与大模型对话流程。目前支持md、txt、pdf(文本)、xlsx、cvs类型。支持mcp服务

Created Mar 2026

ai go knowledge-base mcp ollama
Total 389
30 days +7
thushan

High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.

Created May 2025

ai amd golang intel llama-cpp
Total 338
30 days +20
atopos31

Unified LLM gateway with weighted load balancing, observability & cost tracking. 统一的 LLM 网关,提供权重负载均衡、可观测性与费用追踪。

Created Aug 2025

ai ai-gateway claude claude-ai claude-code
Total 323
30 days +7
modelpack

An Open Standard for Packaging, Distributing and Running LLMs in Cloud-Native Environments

Created Aug 2024

ai model oci oci-artifacts specification
Total 241
30 days +3
teilomillet

A lightweight, production-ready RAG (Retrieval Augmented Generation) library in Go.

Created Jul 2024

ai chromadb document-search embeddings golang
Total 228
30 days +2
llm-d

Distributed KV cache scheduling & offloading libraries

Created Apr 2025

ai incubating
Total 197
30 days +10
montevive

🚀 Autocache - Intelligent Anthropic API Cache Proxy Automatically inject cache-control fields into Claude API requests to reduce costs by up to 90% and latency by up to 85%. Works as a transparent drop-in replacement for popular AI platforms like n8n, Flowise, Make.com, LangChain, and LlamaIndex—no code changes required

Created Oct 2025

agent agentic-ai ai cache claude
Total 178
30 days +2
mydisha

Your friendly, blazing-fast, self-hostable AI gateway

Created May 2026

ai ai-gateway ai-router anthropic caveman
Total 167
30 days +17
pyadav

[deprecated] AI Gateway - core infrastructure stack for building production-ready AI Applications

Created Jan 2024

ai aistudio api gateway generative-ai
Total 162
30 days +0
EinStack

🐦 A open blazing-fast simple model gateway for rapid development of production GenAI apps

Created Dec 2023

ai gateway gateway-api genai generative-ai
Total 160
30 days +0
7sen1ot

ZCode(智谱、Zai)毒鸡蛋反代,支持免费额度以及领取免费token与自定义提示词注入

Created Oct 2026

ai api free glm token
Total 154
30 days +154
voidmind-io

Privacy-first LLM proxy and AI gateway - load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hosted. Zero knowledge of your prompts.

Created Mar 2026

ai ai-gateway anthropic api-gateway docker
Total 149
30 days +5
go-taas

Open-source Token as a Service (TaaS) platform on Kubernetes — deploy LLMs, price by model × GPU, and meter/bill usage via an OpenAI-compatible API.

Created Aug 2026

ai ai-infra taas
Total 104
30 days +45
GaeaRuiW

Created Mar 2026

ai deep-learning fine-tuning grafana helm
Total 102
30 days +0
llmariner

Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.

Created Mar 2024

ai autoscaling fine-tuning gpu inference
Total 97
30 days +0
ShadowSafin

AI inference gateway and control plane for multi-provider routing, policies, caching, tools, and observability.

Created Oct 2026

ai ai-gateway api-gateway inference llm
Total 93
30 days +3
drbh

💬 modular open source LLM server, web and desktop app

Created May 2023

ai chatbot containers gpt llm
Total 63
30 days +0
avilum

A client/server for LLaMA (Large Language Model Meta AI) that can run ANYWHERE.

Created Mar 2023

ai client-server facebook llama llm
Total 62
30 days +0
hugalafutro

Multi-Provider AI Gateway, "Because we have LiteLLM at home" No personal logs by design. Model autodiscovery, Failover groups, High availability, Android companion app, and more!

Created Apr 2026

ai ai-tools android-app anthropic-api discovery
Total 58
30 days +0
MiXaiLL76

High-performance proxy router for LLM APIs with automatic load balancing, rate limiting, and fail2ban protection

Created Jan 2026

ai ai-router air anthropic claude
Total 58
30 days +3
pgEdge

A simple API server for performing Retrieval-Augmented Generation (RAG) of text based on content from a PostgreSQL database using pgvector.

Created Nov 2025

ai api postgres postgresql rag
Total 57
30 days +1
D3-vin

OpenAI-compatible proxy for Zcode

Created Sep 2026

ai ai-agent ai-agents ai-coding ai-tools
Total 55
30 days +29
canonical

Local inference, optimized for your hardware

Created Jun 2025

ai inference llm local-ai model-serving
Total 55
30 days +2
localcloud-sh

Stop paying for AI APIs during development. LocalCloud runs everything locally - GPT-level models, databases, all free.

Created Jun 2025

ai ai-app ai-development ai-development-tools ai-infrastructure
Total 51
30 days +0
scitix

Arks is a cloud-native inference framework running on Kubernetes

Created Apr 2025

ai cloudnative-services dynamo inference kubernetes
Total 51
30 days +0
MegaGrindStone

A Go library implementation of LightRAG - an advanced Retrieval-Augmented Generation (RAG) system that uniquely combines vector databases with graph database relationships to enhance knowledge retrieval.

Created Mar 2025 Archived

ai go library lightrag llm
Total 62
30 days +0

Repos sorted by Jev.