Agent-Skills.md

Agent Skills: FFmpeg + OpenCV Integration Guide

Complete FFmpeg + OpenCV + Python integration guide for video processing pipelines. PROACTIVELY activate for: (1) FFmpeg to OpenCV frame handoff, (2) cv2.VideoCapture vs ffmpeg subprocess, (3) BGR/RGB color format conversion gotchas, (4) Frame dimension order img[y,x] vs img[x,y], (5) ffmpegcv GPU-accelerated video I/O, (6) VidGear multi-threaded streaming, (7) Decord batch video loading for ML, (8) PyAV frame-level processing, (9) Audio stream preservation with video filters, (10) Memory-efficient frame generators, (11) OpenCV + FFmpeg + Modal parallel processing, (12) Pipe frames between FFmpeg and OpenCV. Provides: Color format conversion patterns, coordinate system gotchas, library selection guide, memory management, subprocess pipe patterns, GPU-accelerated alternatives to cv2.VideoCapture. Ensures: Correct integration between FFmpeg and OpenCV without color/coordinate bugs. See also: ffmpeg-python-integration-reference for type-safe parameter mappings.

UncategorizedID: josiahsiegel/claude-plugin-marketplace/ffmpeg-opencv-integration

Repository

josiahsiegel/claude-plugin-marketplace
JosiahSiegelLicense: MIT
213

Install this agent skill to your local

pnpm dlx add-skill https://github.com/JosiahSiegel/claude-plugin-marketplace/tree/HEAD/plugins/ffmpeg-python/skills/ffmpeg-opencv-integration

Skill Files

Browse the full folder contents for ffmpeg-opencv-integration.

Download Skill

Loading file tree…

plugins/ffmpeg-python/skills/ffmpeg-opencv-integration/SKILL.md

Skill Metadata

Name
ffmpeg-opencv-integration
Description
|

FFmpeg + OpenCV Integration Guide

Use this skill when FFmpeg handles video I/O and OpenCV handles image processing. This SKILL is a lean orchestrator; full pipe patterns, library examples, and Modal recipes are preserved in references/opencv-pipelines-and-libraries.md.

When to Use

  • Decode with FFmpeg and process frames with OpenCV
  • Encode OpenCV-generated/processed frames with FFmpeg
  • Compare cv2.VideoCapture, subprocess pipes, PyAV, ffmpegcv, VidGear, and Decord
  • Fix RGB/BGR color bugs or (height, width) dimension-order bugs
  • Preserve audio while replacing processed video frames
  • Build GPU-assisted video I/O around CPU OpenCV processing

Library Selection

| Need | Best option | Why | |---|---|---| | Simple local file | cv2.VideoCapture | Built-in and simple | | Full FFmpeg format/protocol support | subprocess pipe | Exact CLI behavior | | GPU video I/O | ffmpegcv | NVDEC/NVENC with OpenCV-like API | | Network/RTSP streaming | VidGear | Threaded capture and stream helpers | | ML batch loading | Decord | Fast random/batch frame access | | Frame-level libav control | PyAV | Direct FFmpeg library access |

Critical Gotchas

  1. OpenCV is BGR. FFmpeg/PyAV/PIL/Decord often produce RGB. Convert explicitly.
  2. NumPy dimensions are (height, width, channels). Pixel access is img[y, x], not img[x, y].
  3. Video filters can drop audio. Preserve or remux original audio intentionally.
  4. Release resources. Always close VideoCapture, pipes, writers, and PyAV containers.
  5. Rawvideo pipes require exact frame size. width * height * channels must match -pix_fmt.

Minimal Pipe Patterns

FFmpeg to OpenCV using BGR frames:

cmd = [
    "ffmpeg", "-i", input_path,
    "-f", "rawvideo", "-pix_fmt", "bgr24", "-"
]

OpenCV to FFmpeg using BGR frames:

cmd = [
    "ffmpeg", "-y",
    "-f", "rawvideo", "-vcodec", "rawvideo",
    "-s", f"{width}x{height}", "-pix_fmt", "bgr24",
    "-r", str(fps), "-i", "-",
    "-c:v", "libx264", "-preset", "fast", "-crf", "23",
    "-pix_fmt", "yuv420p", output_path
]

Core Workflow

  1. Probe input dimensions, fps, duration, and audio streams.
  2. Pick the I/O library based on the selection table.
  3. Lock pixel format at boundaries (bgr24 for OpenCV pipes; yuv420p for final H.264 output).
  4. Process frames in generators/batches; avoid loading full videos unless they are small.
  5. Preserve or re-encode audio explicitly.
  6. Verify output duration, fps, resolution, codec, and A/V sync.

Reference Map

  • references/opencv-pipelines-and-libraries.md - Full preserved reference: color/dimension gotchas, cleanup patterns, FFmpeg-to-OpenCV and OpenCV-to-FFmpeg pipes, bidirectional pipeline, ffmpegcv, VidGear, Decord, PyAV, Modal.com examples, GPU pipeline, cheat sheets, sources.

Related Skills

  • ffmpeg-python-integration-reference - Type-safe parameters, colors, time units
  • ffmpeg-pyav-integration - PyAV API details
  • ffmpeg-hardware-acceleration - GPU decode/encode and filter pipelines