Run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources by exposing them on different ports and loading/unloading them on demand
Created Jul 2024
large-models
llms
localllm
proxy
Total
88