Agent-Skills.md

Agent Skills: A/B Testing Skill

A/B testing patterns for lead generation sites. Client-side experiments, Google Optimize alternative, analytics integration. Use for conversion optimization.

UncategorizedID: Soborbo/claudeskills/ab-testing

Repository

Soborbo/claudeskills
Soborbo
2

Install this agent skill to your local

pnpm dlx add-skill https://github.com/Soborbo/claudeskills/tree/HEAD/ab-testing

Skill Files

Browse the full folder contents for ab-testing.

Download Skill

Loading file tree…

ab-testing/SKILL.md

Skill Metadata

Name
ab-testing
Description
A/B testing patterns for lead generation sites. Client-side experiments, Google Optimize alternative, analytics integration. Use for conversion optimization.

A/B Testing Skill

Purpose

Enables experimentation on lead generation sites to optimize conversion rates through simple, privacy-friendly client-side testing with statistical rigor.

Core Rules

  1. One test at a time — Per page, avoid interaction effects
  2. Statistical significance — Don't call winners too early (p < 0.05)
  3. Track conversions — Not just clicks or exposures
  4. Minimize flicker — Apply variants before paint using visibility hidden
  5. Respect privacy — No PII in experiments, localStorage only
  6. Minimum duration — Never less than 1 week (day-of-week effects)
  7. Sample size first — Calculate required n before starting
  8. Persist assignments — User sees same variant across sessions
  9. Track exposures — Log every variant view to GA4
  10. Conversion-focused — Optimize for business metrics, not vanity metrics

Test Duration Guidelines

| Traffic/Day | Min Duration | Sample Size | |-------------|--------------|-------------| | 100 | 8+ weeks | ~400 per variant | | 500 | 2 weeks | ~400 per variant | | 1000+ | 1 week | ~400 per variant |

Never call a test in less than 1 week — Day-of-week effects matter.

Common Tests for Lead Gen

| Test | Variants | |------|----------| | CTA Text | "Get Quote" vs "Get Free Quote" vs "Start Now" | | CTA Color | Primary vs Accent vs Contrasting | | Form Length | 3 fields vs 5 fields | | Social Proof | With reviews vs Without | | Urgency | None vs "Limited slots" | | Hero Image | Photo A vs Photo B | | Headline | Benefit-focused vs Problem-focused |

GA4 Custom Dimensions

Set up in GA4 for tracking:

| Dimension | Scope | Description | |-----------|-------|-------------| | ab_test_id | Event | Test identifier | | ab_variant | Event | Variant name |

References

Detailed implementation guides and code examples:

  • Variant System — ABTest and ABVariant components, usage examples
  • GA4 Integration — Conversion tracking, GTM configuration, custom dimensions
  • Statistics — Sample size calculator, results analysis, significance testing

Forbidden

  • ❌ Calling winners without significance (p < 0.05)
  • ❌ Running less than 1 week
  • ❌ Multiple tests on same element
  • ❌ Changing tests mid-run
  • ❌ Not tracking actual conversions
  • ❌ Flicker (variant change visible to user)
  • ❌ Storing PII in localStorage or events
  • ❌ Skipping sample size calculation

Definition of Done

  • [ ] A/B test component implemented (ABTest.astro)
  • [ ] Variant content component implemented (ABVariant.astro)
  • [ ] Variant assignment persisted in localStorage
  • [ ] Exposures tracked in GA4 (ab_test_exposure event)
  • [ ] Conversions tracked in GA4 (ab_test_conversion event)
  • [ ] No visible flicker (visibility: hidden until variant applied)
  • [ ] Sample size calculated based on baseline conversion rate
  • [ ] Test runs minimum 1 week before analysis
  • [ ] GA4 custom dimensions configured (ab_test_id, ab_variant)
  • [ ] Statistical significance verified (p < 0.05) before declaring winner