inference-serving-vllm
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
[vLLMInferenceServingPagedAttention
Orchestra Research
33,1943,891
serving-llms-vllm
vLLM: high-throughput LLM serving, OpenAI API, quantization.
[vLLMInferenceServingPagedAttention
delorenj
176
serving-llms-vllm
vLLM: high-throughput LLM serving, OpenAI API, quantization.
[vLLMInferenceServingPagedAttention
poorrican
0
serving-llms-vllm
vLLM: high-throughput LLM serving, OpenAI API, quantization.
[vLLMInferenceServingPagedAttention
PoorRican
0