Download, Chat & Serve Models from One Binary
Fetch verified GGUF presets with vllm download, chat interactively with KV-cache reuse via vllm chat, or start an OpenAI-compatible streaming server (vllm serve) without installing Python, CUDA, or Docker.
/v1/chat/completions SSE server • Multi-turn KV-cache reuse • Model & GPU diagnostics.