Agent Skills: Skill: Run Tests

Skill: Run Tests

Run the landable tree

• Check out the exact PR head that will land. After any rebase or cherry-pick, discard the old verdict and run again.
• Run one test at a time so each failure and ambient test evidence has one owner.

Choose the execution environment

pnpm evals:e2e <slug>

The CLI prints the placement and reason; copy that line into the report. Use --local only when the user asks for local. --daytona requires Daytona. Never switch lanes to turn a red Daytona run green.

Know what each placement runs before reading a verdict:

• Daytona (the default when the daytona CLI is logged in) builds OPENWORK_EVAL_REF, which defaults to dev, not your working tree. To test a branch, push it and export OPENWORK_EVAL_REF=$(git rev-parse HEAD); add --strict-ref so a mismatch fails instead of warning.
• --local runs your working tree. A spec whose world uses appWeb stays in headless Chrome. A spec whose world uses desktop (seed.desktop, app()) opens an Electron window that takes focus on this computer; run those on Daytona unless the user asks for a local window.

Run the core journey

The test every PR runs. Its world is a Freestyle VM, so this needs only the evals install and the Freestyle key:

FREESTYLE_API_KEY="$(infisical secrets get FREESTYLE_API_KEY --env dev --path /openwork-ops --plain --silent)" \
  node evals/bin/evals.mjs core/core-chat.e2e.test.ts --local --engine v1 --surface web

It runs against the pushed HEAD commit; push first. After changing the world itself, run node evals/scripts/check-freestyle-world.ts --base <sha> instead.

Prepare local fallback

pnpm --filter @openwork/types build
pnpm --filter @openwork-ee/den-db build
pnpm --filter @openwork/email build
pnpm dev:den:mysql

• Local server() requires MySQL at 127.0.0.1:3306.
• Build those workspace dependencies before local Den; otherwise den-api imports can fail.
• If the checkout path contains spaces, set OPENWORK_EVAL_SURFACES_DIR to a space-free path before E2E tests. node-gyp and electron-rebuild require it.
• Local Electron profiles, and the electron.log inside them, are removed when a journey ends. Set OPENWORK_EVAL_SURFACE_LOGS_DIR to a directory outside the surfaces directory to keep a copy of each log; the packaged smoke does.

Choose one lane

• Run one app/Den-driving E2E test:

pnpm evals:e2e <name>

• The CLI owns placement and prints placement: <daytona|local> (<reason>).

Match the runtime

Check what runtime the changed code ships on before trusting a green run. apps/server runs on Bun in evals; Desktop runs that same code on Electron's Node (undici). If the change touches fetch, streams, signals, GC, or timers, run it on the shipping runtime too:

ELECTRON_RUN_AS_NODE=1 apps/desktop/node_modules/electron/dist/Electron.app/Contents/MacOS/Electron --expose-gc <script>

A green run on the wrong runtime is not evidence.

Read the verdict

• Record the exact command, exit code, and passed/failed/skipped counts.
• Report each skip as skipped — needs: X; never call it passed. A green command containing skips makes the overall verdict Incomplete.
• Use Passed, Incomplete, or Failed for the overall result.

Iterate, then cold-boot

• While iterating, reuse a warm Den with OPENWORK_EVAL_DEN_API_URL.
• Before declaring Passed, remove the reuse override and cold-boot through server() on the same commit.
• Inject secrets with infisical run --silent --; never print or echo values.