Categories / LLM Infrastructure

☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!

Last push 2 months ago

huggingface inference inference-platform kubernetes llamacpp llm modelscope ollama sglang text-generation-inference vllm

Created

Nov 2023

Total stars

315

7 days

+1

30 days

+1

12 months

+54

Star trends

README badge

Best of Go badge for InftyAI/llmaz
[![Best of Go](https://bestofgo.dev/badge/InftyAI/llmaz.svg)](https://bestofgo.dev/InftyAI/llmaz)