GitHub topics

llama-cpp

Mostly in LLM Infrastructure 53%

Repositories

13

Total stars

3,806

Trending

T-Crypt

Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and how that access is handed off.

Created Oct 2026

comfyui golang gpu inference llama-cpp
Total 300
30 days +275
silirim

Self-hosted media center for live TV, a Jellyfin and Plex alternative: M3U and Xtream Codes (IPTV) playlists you are entitled to use, EPG guide, timeshift, DVR, movies and series. Apps for Android, Android TV, Fire TV and LG webOS.

Created Oct 2026

ai-assistant android-tv dvr epg fire-tv
Total 79
30 days +79
Dharshaneshwaran

Pools consumer laptops into one local runtime for Meta Llama models. Measures the free VRAM each machine reports, splits the model across CUDA and Metal devices through llama.cpp RPC, and rebuilds the pipeline when a worker drops out.

Created Oct 2026

cuda distributed-inference fault-tolerance gguf golang
Total 56
30 days +56
defilantech

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

Created Nov 2025

ai apple-silicon autoscaling edge-computing gguf
Total 267
30 days +30
ifocus9

PopTrans — Windows 本地离线翻译工具。选中文本 / OCR 截图后一键翻译,基于 Go + Wails + llama.cpp 纯 CPU 推理,无需 GPU。

Created Jun 2026

go hy-mt2 llama-cpp ocr offline
Total 193
30 days +26
thushan

High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.

Created May 2025

ai amd golang intel llama-cpp
Total 338
30 days +20
gpustack

Review/Check GGUF files and estimate the memory usage and maximum tokens per second.

Created May 2024

gguf go llama-box llama-cpp stable-diffusion-cpp
Total 314
30 days +7
raketenkater

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

Created Mar 2026

cuda gguf golang inference-server llama-cpp
Total 284
30 days +3
BalanceBalls

A portable terminal AI interface

Created Nov 2023

chatgpt-api gemini-api go llama-cpp llm
Total 202
30 days +3
lordmathis

Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.

Created Jul 2025

llama-cpp llama-server llamacpp llm llm-inference
Total 162
30 days +2

Top

gotzmann

llama.go is like llama.cpp in pure Golang!

Created Mar 2023

alpaca chatgpt dalai gpt gpt3
Total 1,404
thushan

High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.

Created May 2025

ai amd golang intel llama-cpp
Total 338
gpustack

Review/Check GGUF files and estimate the memory usage and maximum tokens per second.

Created May 2024

gguf go llama-box llama-cpp stable-diffusion-cpp
Total 314
T-Crypt

Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and how that access is handed off.

Created Oct 2026

comfyui golang gpu inference llama-cpp
Total 300
raketenkater

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

Created Mar 2026

cuda gguf golang inference-server llama-cpp
Total 284
defilantech

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

Created Nov 2025

ai apple-silicon autoscaling edge-computing gguf
Total 267
BalanceBalls

A portable terminal AI interface

Created Nov 2023

chatgpt-api gemini-api go llama-cpp llm
Total 202
ifocus9

PopTrans — Windows 本地离线翻译工具。选中文本 / OCR 截图后一键翻译,基于 Go + Wails + llama.cpp 纯 CPU 推理,无需 GPU。

Created Jun 2026

go hy-mt2 llama-cpp ocr offline
Total 193
lordmathis

Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.

Created Jul 2025

llama-cpp llama-server llamacpp llm llm-inference
Total 162
raiyanyahya

Selfhost modern LLM stacks. Run the whole fleet from your terminal

Created Jun 2026

ai-agents bubbletea cli docker fastapi
Total 105
silirim

Self-hosted media center for live TV, a Jellyfin and Plex alternative: M3U and Xtream Codes (IPTV) playlists you are entitled to use, EPG guide, timeshift, DVR, movies and series. Apps for Android, Android TV, Fire TV and LG webOS.

Created Oct 2026

ai-assistant android-tv dvr epg fire-tv
Total 79
Dharshaneshwaran

Pools consumer laptops into one local runtime for Meta Llama models. Measures the free VRAM each machine reports, splits the model across CUDA and Metal devices through llama.cpp RPC, and rebuilds the pipeline when a worker drops out.

Created Oct 2026

cuda distributed-inference fault-tolerance gguf golang
Total 56

Related

All topics →