Categories / LLM Infrastructure
High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.
Last push 1 week ago
Created
May 2025
Total stars
338
Last 7 days
+1
Last 30 days
+20
Last 12 months
+221
README badge
[](https://bestofgo.dev/thushan/olla)