Image Algorithm Validator Skill
Purpose
The Image Algorithm Validator Skill supports validation of medical image processing algorithms, including segmentation, detection, and analysis algorithms, ensuring performance meets clinical requirements.
Capabilities
- Ground truth dataset curation guidance
- Performance metric calculation (Dice, IoU, sensitivity, specificity)
- Inter-observer variability analysis
- Statistical comparison methods
- Validation dataset stratification
- Multi-reader multi-case study design
- FDA AI/ML guidance alignment
- Failure case analysis
- Edge case identification
- Performance boundary testing
- Cross-validation methodology
Usage Guidelines
When to Use
- Validating image analysis algorithms
- Curating validation datasets
- Designing reader studies
- Preparing regulatory submissions
Prerequisites
- Algorithm development complete
- Ground truth established
- Validation dataset available
- Performance criteria defined
Best Practices
- Use representative, diverse datasets
- Establish robust ground truth methodology
- Assess performance across subgroups
- Document failure modes
Process Integration
This skill integrates with the following processes:
- Medical Image Processing Algorithm Development
- AI/ML Medical Device Development
- Clinical Evaluation Report Development
- Software Verification and Validation
Dependencies
- SimpleITK library
- scikit-image
- MONAI framework
- Evaluation frameworks
- Statistical analysis tools
Configuration
image-algorithm-validator:
algorithm-types:
- segmentation
- detection
- classification
- registration
- quantification
metrics:
- Dice
- IoU
- sensitivity
- specificity
- AUC
- Hausdorff-distance
validation-methods:
- holdout
- cross-validation
- external-validation
Output Artifacts
- Dataset curation protocols
- Ground truth documentation
- Performance reports
- Statistical analyses
- Reader study results
- Failure mode catalogs
- Regulatory submission sections
- Validation summaries
Quality Criteria
- Ground truth methodology validated
- Metrics appropriate for algorithm type
- Dataset representative of intended use
- Statistical analysis rigorous
- Subgroup performance assessed
- Documentation supports regulatory review