Repositories

10

Total stars

5,631

Trending

Dharshaneshwaran

Pools consumer laptops into one local runtime for Meta Llama models. Measures the free VRAM each machine reports, splits the model across CUDA and Metal devices through llama.cpp RPC, and rebuilds the pipeline when a worker drops out.

Created Oct 2026

cuda distributed-inference fault-tolerance gguf golang
Total 56
30 days +56
defilantech

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

Created Nov 2025

ai apple-silicon autoscaling edge-computing gguf
Total 267
30 days +30
kitops-ml

An open source DevOps tool from the CNCF for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI Artifact.

Created Feb 2024

ai code datasets devops devops-tools
Total 1,452
30 days +21
hybridgroup

Go with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows, & WebAssembly.

Created Sep 2025

arduino-uno-q cuda gguf golang jetson-orin-nano
Total 676
30 days +20
sammcj

Go manage your Ollama models

Created May 2024

ai ggml gguf linux llm
Total 1,849
30 days +11
anthony-chaudhary

Agentic Runtime

Created Jun 2026

agent-kernel agentic-ai ai-agents ai-gateway claude-code
Total 50
30 days +7
gpustack

Review/Check GGUF files and estimate the memory usage and maximum tokens per second.

Created May 2024

gguf go llama-box llama-cpp stable-diffusion-cpp
Total 314
30 days +7
mattn

A tiny neural-network framework in pure Go with AVX2 SIMD kernels (GOEXPERIMENT=simd)

Created Aug 2026

autograd avx2 deep-learning gguf golang
Total 120
30 days +5
raketenkater

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

Created Mar 2026

cuda gguf golang inference-server llama-cpp
Total 284
30 days +3
kelindar

Go library for embedded vector search and semantic embeddings using llama.cpp

Created Sep 2024

ai bert embeddings gguf gpu
Total 563
30 days +2

Top

sammcj

Go manage your Ollama models

Created May 2024

ai ggml gguf linux llm
Total 1,849
kitops-ml

An open source DevOps tool from the CNCF for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI Artifact.

Created Feb 2024

ai code datasets devops devops-tools
Total 1,452
hybridgroup

Go with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows, & WebAssembly.

Created Sep 2025

arduino-uno-q cuda gguf golang jetson-orin-nano
Total 676
kelindar

Go library for embedded vector search and semantic embeddings using llama.cpp

Created Sep 2024

ai bert embeddings gguf gpu
Total 563
gpustack

Review/Check GGUF files and estimate the memory usage and maximum tokens per second.

Created May 2024

gguf go llama-box llama-cpp stable-diffusion-cpp
Total 314
raketenkater

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

Created Mar 2026

cuda gguf golang inference-server llama-cpp
Total 284
defilantech

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

Created Nov 2025

ai apple-silicon autoscaling edge-computing gguf
Total 267
mattn

A tiny neural-network framework in pure Go with AVX2 SIMD kernels (GOEXPERIMENT=simd)

Created Aug 2026

autograd avx2 deep-learning gguf golang
Total 120
Dharshaneshwaran

Pools consumer laptops into one local runtime for Meta Llama models. Measures the free VRAM each machine reports, splits the model across CUDA and Metal devices through llama.cpp RPC, and rebuilds the pipeline when a worker drops out.

Created Oct 2026

cuda distributed-inference fault-tolerance gguf golang
Total 56
anthony-chaudhary

Agentic Runtime

Created Jun 2026

agent-kernel agentic-ai ai-agents ai-gateway claude-code
Total 50

Related

All topics →