Agent Skills: exploring-llm-evaluations
>
ID: posthog/posthog/exploring-llm-evaluations
39,9873,435
Install this agent skill to your local
Skill Files
The files that make up exploring-llm-evaluations.
More agent skills
- clickhouse-migrationsClickHouse migration patterns and rules. Use when creating or modifying ClickHouse migrations.
- auditing-experiments-flagsAudit PostHog experiments and feature flags for configuration issues, staleness, and best-practice violations. Read when the user asks to audit, health-check, or review experiments or feature flags, check flag hygiene, or verify experiment setup.
- implementing-mcp-ui-appsGuide for adding MCP UI apps — interactive visualizations that render tool results in MCP clients like Claude Desktop. Use when adding a new detail or list view for an MCP tool, creating view components in products/*/mcp/apps/, or linking tools to UI apps via YAML.
- extending-hobby-smoke-testsDesign, extend, review, or debug PostHog Hobby end-to-end smoke tests in bin/hobby-ci.py and .github/workflows/ci-hobby.yml. Use when adding an ingestion round trip, deciding whether a product belongs in Hobby CI, changing the CI Hobby service topology or API-key scopes, or diagnosing a smoke test that captures data but cannot query it.
- adding-activity-loggingAdds or changes activity logging (the audit trail) for a Django model in PostHog. Use when a model's writes must show in the Activity side panel or the advanced activity logs, when adding ModelActivityMixin, an ActivityScope, a model_activity_signal receiver, an activity describer, or field exclusions, when auditing which write paths of a model are logged, or when a change is missing from the activity log. Covers the receiver-module convention, writes the signal cannot see (QuerySet.update, bulk_create), the actor outside requests, and product models on a separate database. Trigger terms - activity log, audit log, audit trail, ModelActivityMixin, log_activity, changes_between, activity describer, who changed this.
- scanning-experiments-with-replay-visionProvisions a Replay Vision scanner scoped to one experiment's exposed sessions: sets `experiment_targeting` so the API derives the person-scoped exposure filter server-side, templates a prompt that stays comparable across variants, sizes credit spend against the experiment's own population, and creates the scanner disabled so its prompt can be previewed on real sessions before it sweeps.\nTRIGGER when: user wants Replay Vision to watch an experiment, asks to scan or analyze an experiment's recordings with AI, asks \"what are users actually doing in the test variant\", or wants a scanner scoped to an experiment's exposed sessions.\nDO NOT TRIGGER when: creating a general-purpose scanner not tied to an experiment (use creating-replay-vision-scanners), reading observations a scanner already produced (use exploring-replay-vision-observations), or manually browsing an experiment's recordings without AI analysis (use analyzing-experiment-session-replays).