Debugging Methodology Skill

Debugging Methodology

Systematic approach to finding and fixing bugs.

Core Principles

Occam's Razor - Start with the simplest explanation
Binary Search - Isolate the problem area systematically
Preserve Evidence - Understand state before making changes
Document Hypotheses - Track what was tried and didn't work

Debugging Workflow

1. Understand → What is expected vs actual behavior?
2. Reproduce → Can you trigger the bug reliably?
3. Locate → Where in the code does it happen?
4. Diagnose → Why does it happen? (root cause)
5. Fix → Minimal change to resolve
6. Verify → Confirm fix works, no regressions

Common Bug Patterns

| Symptom | Likely Cause | Check First | |---------|--------------|-------------| | TypeError/null | Missing null check | Input validation | | Off-by-one | Loop bounds, array index | Boundary conditions | | Race condition | Async timing | Await/promise handling | | Import error | Path/module resolution | File paths, exports | | Type mismatch | Wrong type passed | Function signatures | | Flaky test | Timing, shared state | Test isolation |

System-Level Tools

Memory Analysis

# Valgrind (C/C++/Rust)
valgrind --leak-check=full --show-leak-kinds=all ./program
valgrind --tool=massif ./program  # Heap profiling

# Python
python -m memory_profiler script.py

Performance Profiling

# Linux perf
perf record -g ./program
perf report
perf top  # Real-time CPU usage

# Python
python -m cProfile -s cumtime script.py

System Tracing (Traditional)

# System calls (ptrace-based, high overhead)
strace -f -e trace=all -p PID

# Library calls
ltrace -f -S ./program

# Open files/sockets
lsof -p PID

# Memory mapping
pmap -x PID

eBPF Tracing (Modern, Production-Safe)

eBPF is the modern replacement for strace/ptrace-based tracing. Key advantages:

Low overhead: Safe for production use
No recompilation: Works on running binaries
Non-intrusive: Doesn't stop program execution
Kernel-verified: Bounded execution, can't crash the system

# BCC tools (install: apt install bpfcc-tools)
# Trace syscalls with timing (like strace but faster)
sudo syscount -p PID              # Count syscalls
sudo opensnoop -p PID             # Trace file opens
sudo execsnoop                    # Trace new processes
sudo tcpconnect                   # Trace TCP connections
sudo funccount 'vfs_*'            # Count kernel function calls

# bpftrace (install: apt install bpftrace)
# One-liner tracing scripts
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_open { printf("%s %s\n", comm, str(args->filename)); }'
sudo bpftrace -e 'uprobe:/bin/bash:readline { printf("readline\n"); }'

# Trace function arguments in Go/other languages
sudo bpftrace -e 'uprobe:./myapp:main.handleRequest { printf("called\n"); }'

eBPF Tool Hierarchy: | Level | Tool | Use Case | |-------|------|----------| | High | BCC tools | Pre-built tracing scripts | | Medium | bpftrace | One-liner custom traces | | Low | libbpf/gobpf | Custom eBPF programs |

When to use eBPF over strace:

Production systems (strace adds 10-100x overhead)
Long-running traces
High-frequency syscalls
When you can't afford to slow down the process

Network Debugging

# Packet capture
tcpdump -i any port 8080

# Connection status
ss -tuln
netstat -tuln

Language-Specific Debugging

Python

# Quick debug
import pdb; pdb.set_trace()

# Better: ipdb or pudb
import ipdb; ipdb.set_trace()

# Print with context
print(f"{var=}")  # Python 3.8+

JavaScript/TypeScript

// Browser/Node
debugger;

// Structured logging
console.log({ var1, var2, context: 'function_name' });

Rust

// Debug print
dbg!(&variable);

// Backtrace on panic
RUST_BACKTRACE=1 cargo run

Debugging Questions

When stuck, ask:

What changed recently that could cause this?
Does it happen in all environments or just one?
Is the bug in my code or a dependency?
What assumptions am I making that might be wrong?
Can I write a minimal reproduction?

Effective Debugging Practices

Targeted changes: Form a hypothesis, change one thing at a time
Use proper debuggers: Step through code with breakpoints when possible
Find root causes: Trace issues to their origin, fix the source
Reproduce first: Create a minimal reproduction before attempting a fix
Verify the fix: Confirm the fix resolves the issue and passes tests

Agent Skills: Debugging Methodology

Install this agent skill to your local

Skill Files