Categories / LLM Infrastructure

perk11

perk11 / large-model-proxy

Run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources by exposing them on different ports and loading/unloading them on demand

Last push 3 days ago

large-models llms localllm proxy

Created

Jul 2024

Total stars

88

7 days

0

30 days

0

12 months

+7

Star trends

README badge

Best of Go badge for perk11/large-model-proxy
[![Best of Go](https://bestofgo.dev/badge/perk11/large-model-proxy.svg)](https://bestofgo.dev/perk11/large-model-proxy)