Download skills/deepstream-import-vision-model/SKILL.md from the publisher, or copy the path and open the repository. PlugMCP lists the skill and does not host the file.
Where does deepstream-import-vision-model live?
The publisher keeps it at https://github.com/NVIDIA/skills.
Which agents can use deepstream-import-vision-model?
Claude, Cursor, and Codex can read a SKILL.md you add from the publisher's repository.
>
name: deepstream-import-vision-model
description: >
Use this skill to bring a supported object-detection vision model from HuggingFace or NVIDIA NGC into
an NVIDIA DeepStream pipeline with end-to-end automation: ONNX download,
SafeTensors export, TRT engine build, custom nvinfer bbox parser, multi-stream
benchmark, and PDF report. Object detection models only.
license: CC-BY-4.0 AND Apache-2.0
metadata:
author: "Tushar Khinvasara [email protected]"
owner: "Tushar Khinvasara [email protected]"
service: "deepstream"
version: "1.5.2"
reviewed: "2026-08-04"
team: deepstream-sdk
tags:
- deepstream
- tensorrt
- object-detection
- import-vision-model
languages:
- bash
- python
- cpp
domain: computer-vision
DeepStream Import Vision Model
When this skill is active, . Do not rely on memory — reference documents contain exact script paths, bash variable conventions, log filename contracts, and critical parsing rules.
read the relevant reference document before starting each phase
Current scope: Object detection models only. Fail fast on classification, segmentation, or other architectures detected in config.json.
Model choice — always offer two options
Before preflight, browsing, downloads, or file creation, present exactly these two choices. Do not
start with only an open-ended model-source prompt. If the user's request already clearly selects a
model, confirm the matching choice instead of asking redundantly.
Hugging Face model ID (organization/model) or full model URL.
NVIDIA NGC catalog model URL including its version.
Explain that the skill currently rejects classification, segmentation, and other non-detection
architectures after inspecting config.json. Do not invent or silently substitute a model when the
custom source is missing or unsupported.
For a dry run, present the same two choices and simulate discovery, build, benchmark, and report
stages without browsing, downloading, launching Docker, writing files, or starting processes.
Run the full pipeline autonomously without pausing for confirmation at each step.
Runs entirely through Docker (no host packages)
Every step runs INSIDE the DeepStream container. The host needs only Docker + the NVIDIA
driver — no host python/venv/torch/trtexec/make/wkhtmltopdf. This works identically on Linux and
Windows (Docker Desktop + WSL2 backend, required for --gpus). The per-shell bind-mount
token is the only OS difference — -v "$PWD":/work (bash), -v "${PWD}:/work" (PowerShell),
-v "%cd%:/work" (cmd); full guide in references/windows.md. All venv/ONNX/
engine/parser/config/report artifacts live under the mounted working root and persist between the
ephemeral --rm containers.
Pre-flight — bootstrap + verify (through the container)
1. One-time bootstrap — builds build/.venv_optimum (torch/onnx/onnxruntime/report deps; the
venv name is historical, optimum is no longer used) +
installs wkhtmltopdf, all in-container. From the working root:
docker run --rm -it --gpus all --shm-size=16g -v "$PWD":/work -w /work \
--entrypoint bash nvcr.io/nvidia/deepstream:9.1-triton-multiarch \
.claude/skills/deepstream-import-vision-model/setup.sh
2. Preflight — GPU + venv + trtexec, run THROUGH the container (container-mode auto-detects):
docker run --rm --gpus all -v "$PWD":/work -w /work \
--entrypoint bash nvcr.io/nvidia/deepstream:9.1-triton-multiarch \
.claude/skills/deepstream-import-vision-model/scripts/preflight.sh # proceed only on PASS
Every subsequent phase runs the same way — issue the model's commands via
docker run … --entrypoint bash … -lc '<commands>' (or the
.claude/skills/deepstream-import-vision-model/scripts/dsrun.sh wrapper:
bash .claude/skills/deepstream-import-vision-model/scripts/dsrun.sh '<in-container command>'),
using PY=build/.venv_optimum/bin/python and
trtexec at /usr/src/tensorrt/bin/trtexec inside the container. deepstream-app,
gst-launch-1.0, and /opt/nvidia/deepstream/… sample paths all exist in the image.
TensorRT build+runtime share one image, so there is no version skew (the concern the old
"build on the host" rule tried to avoid — see references/engine-build.md).
sample_720p.mp4 ships in the image; set DS_VIDEO only to override.
Mandatory Output Structure
Create once MODEL_NAME is known (Step 1). Never dump files flat.
KITTI validation gate — do NOT proceed to Step 7 if KITTI frame count is zero or detection rate < 90%.
Shared venv — build/.venv_optimum reused across all models. Never create per-model venvs.
trtexec --noDataTransfers — GPU-only compute matches DeepStream's GPU-to-GPU data flow.
Report HTML+PDF — always use .claude/skills/deepstream-import-vision-model/scripts/report/md-to-html-pdf.py. Never write a custom HTML generator or call wkhtmltopdf directly.
Object detection only — reject non-detection architectures from config.json before building anything.
Encoder fallback (MANDATORY) — x264enc and openh264enc are prohibited. On NVENC-unavailable systems, use theoraenc + oggmux (LGPL; ships in gst-plugins-base; output is .ogv). If theoraenc/oggmux are absent, skip video creation (DS_SINGLE_STREAM_MODE=skipped). Report which mode was used: nvv4l2h264enc / theoraenc-fallback / skipped.
Video source (MANDATORY) — default is always sample_720p.mp4 (1280×720). Never autonomously substitute sample_1080p_h264.mp4 or any other file. Only use a different video when the user explicitly provides a path (via DS_VIDEO env var or script argument).
Examples
Default model, end to end. Bootstrap once, then run the full pipeline:
docker run --rm -it --gpus all --shm-size=16g -v "$PWD":/work -w /work \
--entrypoint bash nvcr.io/nvidia/deepstream:9.1-triton-multiarch \
.claude/skills/deepstream-import-vision-model/setup.sh
# then: "Use deepstream-import-vision-model to run PekingU/rtdetr_r50vd"
SafeTensors model with no published ONNX. Step 2b exports it first; the wrapper reports which
backend produced the graph and fails loudly if the batch dimension was baked in:
bash .claude/skills/deepstream-import-vision-model/scripts/model/safetensors-to-onnx.sh \
models/$MODEL_NAME/hf_model models/$MODEL_NAME/onnx_export/
# [export] backend=dynamo
# [export] dynamo produced a static batch dimension; trying the next backend
# [export] backend=legacy-torchscript
# [export] pixel_values shape=['batch', 3, 640, 640]
Pin a Hub revision for a reproducible build — any exporter flag passes straight through:
STEP_START=$(date +%s.%N)
# ... step commands ...
STEP_END=$(date +%s.%N)
STEP_DURATION=$(python3 -c "print(round($STEP_END - $STEP_START, 2))") # bc is not in the container; python3 always is
echo "[Step N] completed in ${STEP_DURATION}s"
Track PIPELINE_START (before Step 1) and PIPELINE_END (after Step 8). Report all durations in the benchmark report.
benchmark_report.html — styled HTML (charts base64-inlined, no local file access)
benchmark_report_{model_name}.pdf — via md-to-html-pdf.py; verify charts are embedded by counting data:image/png occurrences in the HTML output: grep -o 'data:image/png' benchmark_report.html | wc -l should equal 5
Run charts and report scripts with the shared venv active: source build/.venv_optimum/bin/activate.
Reference Documents
IMPORTANT: Read the relevant reference before starting each phase. Do NOT generate code from memory.
Step 8: benchmark_data.json, 5 charts, 12-section markdown report, HTML + PDF
Scripts
Installed into .claude/skills/deepstream-import-vision-model/scripts/ by install.sh.
Script
Phase
Purpose
model/hf-list-files.sh
1–3
List HuggingFace repo files
model/hf-download-config.sh
1–3
Download config.json from HF
model/ngc-list-files.sh
1–3
List NGC model files
model/ngc-download.sh
1–3
Download NGC model archive
model/safetensors-to-onnx.sh
1–3
Export SafeTensors → ONNX via torch.onnx.export (wrapper)
model/safetensors_to_onnx.py
1–3
The exporter — dynamo backend, TorchScript fallback, verifies dynamic batch
model/inspect-onnx.py
1–5
Inspect ONNX input/output shapes
model/make-static-batch-onnx.py
4–5
Bake batch dim into ONNX
model/cleanup.sh
Any
Remove staging dirs, preserve shared venv
engine/benchmark-trtexec.sh
4–5
Run trtexec with standard flags
deepstream/ds-single-stream.sh
6–7
Single-stream visual validation (NVENC primary; theoraenc+oggmux fallback; skip if neither)
deepstream/ds-sweep.sh
6–7
2-phase batch size sweep
deepstream/benchmark-ds.sh
6–7
Fixed-stream DS benchmark
deepstream/ds-kitti-dump.sh
6–7
KITTI detection dump via deepstream-app
deepstream/ds-perf-run.sh
7
Step 7c two-run benchmark — wraps deepstream-app with enable-perf-measurement=1, writes fixed-name log for the report parser
deepstream/extract-frame.sh
6–7
Extract sample frames from output video (.mp4 NVENC path or .ogv theoraenc fallback)
report/generate-benchmark-charts.py
8
Generate 5 benchmark PNG charts
report/md-to-html-pdf.py
8
Markdown → styled HTML → PDF (canonical benchmark report path)
report/md-to-pdf.sh
Any
Markdown → PDF via pandoc/pdflatex — for design docs and references only, NOT for benchmark reports (use md-to-html-pdf.py for those)
report/report-style.css
8
CSS for HTML report
report/render-mermaid-for-pdf.py
8
Mermaid diagram → PNG
report/mermaid-puppeteer.json
8
Vetted Puppeteer config for Mermaid (sandboxed; non-root)
report/mermaid-puppeteer-root.json
8
Vetted Puppeteer config for Mermaid (used when running as root)
Quick Error Reference
Error
Fix
Tilted/diagonal bounding boxes
Parser struct not zero-initialized — use NvDsInferObjectDetectionInfo obj = {};
Zero KITTI files
gie-kitti-output-dir not read by nvinfer — use ds-kitti-dump.sh (wraps deepstream-app)
Engine rebuilds every DS run
model-engine-file path wrong — check relative path from config/ dir
setDimensions negative dims
Add infer-dims=3;H;W to nvinfer config for dynamic ONNX models
--memPoolSize workspace 0.03 MiB
Use M suffix not MiB — e.g. --memPoolSize=workspace:32768M
ForeignNode build failure (DETR)
Run onnxsim — see references/engine-build.md. Not reproduced on TRT 10.16 with either export backend
ONNX has a static batch dim
Both export backends specialized it — see the gotchas in references/model-acquire.md
Zero detections
Wrong net-scale-factor — check model family table in references/pipeline-run.md
No module named 'pyservicemaker'
Install into venv: pip install /opt/nvidia/deepstream/.../pyservicemaker*.whl
How do I install deepstream-import-vision-model?
Download skills/deepstream-import-vision-model/SKILL.md from the publisher, or copy the path and open the repository. PlugMCP lists the skill and does not host the file.
Where does deepstream-import-vision-model live?
The publisher keeps it at https://github.com/NVIDIA/skills.
Which agents can use deepstream-import-vision-model?
Claude, Cursor, and Codex can read a SKILL.md you add from the publisher's repository.