Regression Test Skill | Agent Skills

Regression Test

Comprehensive testing checklist for voxtype releases.

Note: For detailed manual smoke tests (recording cycles, GPU isolation, output drivers, etc.), see docs/SMOKE_TESTS.md. Run those tests for thorough pre-release validation.

Quick Smoke Test

# Build and basic checks
cargo build --release
./target/release/voxtype --version
./target/release/voxtype --help
./target/release/voxtype setup --help

Unit Tests

cargo test

Key test modules:

text:: - Spoken punctuation and replacements
cli:: - Command-line argument parsing
state:: - State machine transitions

CLI Command Tests

Info and Setup Commands

# List models per engine and which are installed (replaces the removed
# `setup --list-models`)
./target/release/voxtype info models

# Engines, variants, styles, acceleration
./target/release/voxtype info engines
./target/release/voxtype info variants
./target/release/voxtype info styles
./target/release/voxtype info accel

# Resolved configuration (replaces the removed `setup --show-config`)
./target/release/voxtype config get
./target/release/voxtype config schema --json | python3 -c "import json,sys; print(len(json.load(sys.stdin)['keys']), 'keys')"

# System check and GPU detection
./target/release/voxtype setup check
./target/release/voxtype setup gpu --status

Status Commands

# Check daemon status (will fail if not running, that's ok)
timeout 2 ./target/release/voxtype status || echo "Daemon not running (expected)"

# JSON output format
timeout 2 ./target/release/voxtype status --format json || echo "Daemon not running (expected)"

Transcription Test

# Test with a sample audio file (if available)
./target/release/voxtype transcribe test.wav

Configuration Tests

Default Config Loading

Never touch ~/.config/voxtype/config.toml on a dev machine - the old version of this test deleted the real config. Isolate with XDG variables or --config instead:

# Should run on built-in defaults with no config at all
XDG_CONFIG_HOME=$(mktemp -d) ./target/release/voxtype info engines

# Should load the shipped default config without errors
./target/release/voxtype --config config/default.toml config get

Config Salvage (#646, since 1.1.0)

One bad value must not stop the daemon; syntax errors and duplicate keys must. Full matrix: docs/smoke_tests/config-validation.md.

Config Backwards Compatibility

Test that old config files still work:

# Create minimal old-style config
cat > /tmp/test-config.toml << 'EOF'
[hotkey]
key = "SCROLLLOCK"

[whisper]
model = "base.en"
EOF

# Should not error
./target/release/voxtype --config /tmp/test-config.toml --help

Binary Variant Tests

For each binary variant, verify version and help, and always download the release assets and check them against the CI-signed sums first:

VERSION=1.1.0
gh release download v${VERSION} -p 'SHA256SUMS.txt' -p 'voxtype-*-linux-x86_64-*'
grep -E "x86_64-(baseline|avx2|avx512|vulkan)$" SHA256SUMS.txt | sha256sum -c -

for v in baseline avx2 avx512 vulkan; do
  B=voxtype-${VERSION}-linux-x86_64-$v
  chmod +x "$B" && ./$B --version && ./$B --help > /dev/null
done

Then run /validate-binaries for the instruction-set and glibc gates. For the baseline variant, --version is NOT sufficient: the #740 class of bug passes every startup path and crashes at first inference. Run the behavioral floor test in docs/smoke_tests/baseline-v2-floor.md.

Integration Tests

Daemon Lifecycle

# Start daemon
./target/release/voxtype &
DAEMON_PID=$!
sleep 2

# Check it's running
./target/release/voxtype status

# Stop daemon
kill $DAEMON_PID

Signal Handling

./target/release/voxtype &
DAEMON_PID=$!
sleep 1

# SIGUSR1 should start recording (will fail without audio, ok)
kill -USR1 $DAEMON_PID

# SIGTERM should graceful shutdown
kill -TERM $DAEMON_PID

Package Tests

Debian Package

# Validate structure
dpkg-deb --info releases/${VERSION}/voxtype_${VERSION}-1_amd64.deb

# List contents
dpkg-deb --contents releases/${VERSION}/voxtype_${VERSION}-1_amd64.deb

# Check for required files
dpkg-deb --contents releases/${VERSION}/voxtype_${VERSION}-1_amd64.deb | grep -E 'voxtype-avx2|voxtype-avx512|config.toml|voxtype.service'

RPM Package

rpm -qp --info releases/${VERSION}/voxtype-${VERSION}-1.x86_64.rpm
rpm -qp --list releases/${VERSION}/voxtype-${VERSION}-1.x86_64.rpm

Checklist for Major Releases

[ ] cargo test passes
[ ] cargo clippy has no warnings
[ ] All binary variants build successfully
[ ] Binary instruction validation passes (no AVX-512 in AVX2/Vulkan)
[ ] Version numbers match across all binaries
[ ] CLI --help output is correct
[ ] Default config loads without errors
[ ] Old configs still work (backwards compatibility)
[ ] Packages install correctly on target distros
[ ] Daemon starts and stops cleanly
[ ] Recording/transcription works end-to-end (manual test)

Known Test Limitations

Audio capture requires real audio hardware (can't fully test in CI)
evdev hotkey detection requires /dev/input access
Transcription requires downloaded Whisper model
GPU acceleration requires Vulkan-capable hardware

Agent Skills: Regression Test

Install this agent skill to your local

Skill Files