Categories / LLM Infrastructure

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

Last push 7 hours ago

cuda gguf golang inference-server llama-cpp llamacpp llm local-llm localllama metal moe multi-gpu ollama-alternative openai-api self-hosted speculative-decoding vulkan

Created

Mar 2026

Total stars

284

7 days

+1

30 days

+3

12 months

+284

Star trends

README badge

Best of Go badge for raketenkater/ggrun
[![Best of Go](https://bestofgo.dev/badge/raketenkater/ggrun.svg)](https://bestofgo.dev/raketenkater/ggrun)