Back to tags
Tag

Agent Skills with tag: Throughput

7 skills match this tag. Use tags to discover related Agent Skills and explore similar workflows.

inference-serving-tensorrt-llm

Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.

[InferenceServingTensorRT-LLMNVIDIA
Orchestra Research
Orchestra Research
33,1943,891

inference-serving-vllm

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

[vLLMInferenceServingPagedAttention
Orchestra Research
Orchestra Research
33,1943,891

serving-llms-vllm

vLLM: high-throughput LLM serving, OpenAI API, quantization.

[vLLMInferenceServingPagedAttention
delorenj
delorenj
176

serving-llms-vllm

vLLM: high-throughput LLM serving, OpenAI API, quantization.

[vLLMInferenceServingPagedAttention
poorrican
poorrican
0

tensorrt-llm

Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.

[InferenceServingTensorRT-LLMNVIDIA
poorrican
poorrican
0

serving-llms-vllm

vLLM: high-throughput LLM serving, OpenAI API, quantization.

[vLLMInferenceServingPagedAttention
PoorRican
PoorRican
0

tensorrt-llm

Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.

[InferenceServingTensorRT-LLMNVIDIA
PoorRican
PoorRican
0