{{!--
  JUDGE Template - LLM-as-Judge Prompt

  Purpose: Generate configurable judge prompts with reasoning-first pattern.

  Key Principles (from research):
  - Reasoning BEFORE score (13%+ accuracy improvement)
  - 1-5 scale most reliable (not 0-100)
  - Different model than generator
  - Position swapping for bias mitigation

  Input Data Structure:
  ```yaml
  judge:
    name: string
    focus: string (accuracy | style | completeness | custom)
    scale:
      type: string (1-5 | binary | custom)
      labels: object (optional)
    criteria:
      - name: string
        description: string
        weight: number (0-1)
    reasoning_required: boolean
    position_swap: boolean (for A/B comparisons)
  context:
    task_description: string
    golden_output: string (optional)
  output:
    format: string (json | structured)
  ```

  Usage:
  bun run RenderTemplate.ts -t Evals/Judge.hbs -d Data/AccuracyJudge.yaml
--}}

# {{judge.name}}

You are an expert AI evaluator specializing in {{judge.focus}} assessment.

## Your Task

Evaluate the following output against the specified criteria. You must provide thorough reasoning BEFORE giving any scores.

{{#if context.task_description}}
### Original Task

{{context.task_description}}
{{/if}}

{{#if context.golden_output}}
### Reference Output (Gold Standard)

{{context.golden_output}}
{{/if}}

### Output to Evaluate

```
{OUTPUT_TO_EVALUATE}
```

---

## Evaluation Criteria

{{#each judge.criteria}}
### {{@index}}. {{name}} (Weight: {{percent weight 1}}%)

{{description}}

{{/each}}

---

## Scoring Scale

{{#eq judge.scale.type "1-5"}}
| Score | Meaning |
|-------|---------|
| 5 | Excellent - Exceeds expectations |
| 4 | Good - Meets all requirements |
| 3 | Acceptable - Meets most requirements |
| 2 | Poor - Significant issues |
| 1 | Unacceptable - Fails requirements |
{{/eq}}

{{#eq judge.scale.type "binary"}}
| Score | Meaning |
|-------|---------|
| PASS | Meets the criterion |
| FAIL | Does not meet the criterion |
{{/eq}}

{{#if judge.scale.labels}}
| Score | Meaning |
|-------|---------|
{{#each judge.scale.labels}}
| {{@key}} | {{this}} |
{{/each}}
{{/if}}

---

## Instructions

{{#if judge.reasoning_required}}
**CRITICAL: You MUST provide detailed reasoning for each criterion BEFORE giving any scores.**

For each criterion:
1. Quote specific parts of the output that are relevant
2. Explain how well the output meets the criterion
3. Identify any gaps or issues
4. Only THEN assign a score
{{/if}}

{{#if judge.position_swap}}
**Note:** This evaluation may be run multiple times with content in different positions. Evaluate purely on merit, not position.
{{/if}}

---

## Required Output Format

{{#eq output.format "json"}}
You MUST respond with valid JSON in exactly this format:

```json
{
  "reasoning": {
{{#each judge.criteria}}
    "{{lowercase name}}": "Detailed reasoning for this criterion..."{{#unless @last}},{{/unless}}
{{/each}}
  },
  "scores": {
{{#each judge.criteria}}
    "{{lowercase name}}": <score>{{#unless @last}},{{/unless}}
{{/each}}
  },
  "overall_score": <weighted average>,
  "passed": <true/false based on threshold>,
  "summary": "Brief overall assessment"
}
```
{{/eq}}

{{#eq output.format "structured"}}
Provide your evaluation in this structure:

## Reasoning

{{#each judge.criteria}}
### {{name}}
[Your detailed reasoning here]

**Score:** [Your score]

{{/each}}

## Overall Assessment

**Weighted Score:** [Calculate the weighted average]
**Result:** [PASS/FAIL based on threshold]
**Summary:** [Brief overall assessment]
{{/eq}}

---

**Begin your evaluation now.**
