Categories / LLM Infrastructure

High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.

Last push 1 week ago

ai amd golang intel llama-cpp llamacpp llm-inference llm-proxy llm-router llm-routing lmstudio local-ai mlx nvidia ollama proxy self-hosted self-hosted-ai sglang vllm

Created

May 2025

Total stars

338

7 days

+1

30 days

+20

12 months

+221

Star trends

README badge

Best of Go badge for thushan/olla
[![Best of Go](https://bestofgo.dev/badge/thushan/olla.svg)](https://bestofgo.dev/thushan/olla)