Agent-Skills.md

Agent Skills: Compile TensorRT-LLM (Local / Compute Node)

Compile TensorRT-LLM on a compute node inside a Docker container. Use

UncategorizedID: autohandai/community-skills/nvidia-tensorrt-llm-exec-local-compile

Repository

autohandai/community-skills
autohandaiLicense: Apache-2.0
103

Install this agent skill to your local

pnpm dlx add-skill https://github.com/autohandai/community-skills/tree/HEAD/nvidia-tensorrt-llm-exec-local-compile

Skill Files

Browse the full folder contents for nvidia-tensorrt-llm-exec-local-compile.

Download Skill

Loading file tree…

nvidia-tensorrt-llm-exec-local-compile/SKILL.md

Skill Metadata

Name
nvidia-tensorrt-llm-exec-local-compile
Description
Compile TensorRT-LLM on a compute node inside a Docker container. Use

Compile TensorRT-LLM (Local / Compute Node)

Compile TensorRT-LLM from source on a compute node inside a Docker container.

When to Use

| Scenario | Use This Skill? | |----------|----------------| | On a compute node with GPUs visible (nvidia-smi works) | Yes | | On a SLURM login node (no GPUs) | No — use exec-slurm-compile instead |

Prerequisites

  • You are inside a Docker/enroot container on a compute node
  • nvidia-smi succeeds (GPUs visible)
  • /usr/local/tensorrt exists (TensorRT installation in the container)

Instructions

Step 1: Verify Environment

Run nvidia-smi to confirm you are on a compute node with GPU access.

Step 2: Locate the Codebase

cd to the TensorRT-LLM repository. If the path is not provided by the user, ask for it.

Step 3: (Optional) Checkout Branch

If the user specifies a branch (e.g., "compile ToT"), checkout and pull:

git checkout main && git pull

Step 4: Build

Run the build command (incremental by default — omit -c/--clean unless explicitly requested or the incremental build fails):

./scripts/build_wheel.py --trt_root /usr/local/tensorrt --benchmarks --use_ccache -a "<arch>" -f --nvtx

Replace <arch> with the target GPU architecture (see Architecture Reference below). If not specified by the user, auto-detect from nvidia-smi.

Step 5: Install

pip install -e .[devel]

Step 6: Verify

python3 -c "import tensorrt_llm; print(tensorrt_llm.__version__)"

Build Flags

| Flag | Description | |------|-------------| | --trt_root /usr/local/tensorrt | TensorRT installation path (standard in NVIDIA containers) | | --benchmarks | Build the C++ benchmarks | | -a "<arch>" | Target GPU architecture(s) | | --nvtx | Enable NVTX markers for profiling | | --use_ccache | Use ccache for faster recompilation | | -f / --fast_build | Skip some kernels for faster dev compilation. Always use for dev builds. | | -c / --clean | Clean build directory before building. Only when needed (see below). | | --skip_building_wheel | Build in-place without creating a wheel file | | --no-venv | Skip virtual environment creation |

Architecture Reference

| Value | GPU Family | |-------|-----------| | "100-real" | Blackwell (B200, GB200) | | "90-real" | Hopper (H100, H200) | | "89-real" | Ada Lovelace (L40S) | | "80-real" | Ampere (A100) | | "90;100-real" | Multiple architectures |

Incremental vs. Clean Builds

Default to incremental builds — CMake only recompiles changed files, saving significant time.

Use a clean build (-c) only when:

  • The user explicitly requests a clean/fresh build
  • An incremental build fails with linker errors, stale object files, or CMake cache issues
  • Major branch changes (e.g., rebasing across many commits) that may invalidate the build cache
  • Build system files changed (CMakeLists.txt, *.cmake)