Agent Skills: LLM Adaptation

LLM Adaptation

Change a model's weights so it does a task better: supervised fine-tuning (SFT), parameter-efficient fine-tuning (PEFT: LoRA, QLoRA, DoRA), full fine-tuning, distillation and compression recovery, plus the dataset and tokenizer work that decides whether any of it succeeds.

Deciding whether to adapt at all (prompt vs thinking vs RAG vs fine-tune vs post-train) is not this skill. Use the single build ladder in ai-architecture-advisor first, and come here once it says "fine-tune" or "distill".

When to Use This Skill

• Configuring LoRA or QLoRA: rank, alpha, alpha convention, target modules, learning rate
• Choosing full fine-tuning vs PEFT, or escalating when LoRA plateaus
• Building an SFT dataset: chat templates, prompt-token loss masking, packing, contamination control
• Distilling a large teacher into a smaller student, or recovering quality after pruning
• Diagnosing tokenizer fragmentation, fertility, or glitch tokens on a subset of inputs
• Deciding whether a fine-tune pays back (break-even on volume and stability)

Scope Boundaries

| Need | Go to | |---|---| | Which approach at all; model and provider choice; MoE vs dense | ai-architecture-advisor | | Preference and RL post-training: RLHF/PPO, DPO, GRPO, RLVR, reward models | ai-post-training | | Eval design, judge calibration, thresholds, benchmark harnesses | ai-evals | | Prompts, structured outputs, schema enforcement | ai-prompt-engineering | | Context assembly and memory | ai-context-layer | | RAG and agents | ai-rag, ai-agents | | Serving, quantization formats, pruning runtime support | ai-llm-inference | | Local runs and GGUF packaging | ai-local-model-ops | | Deployment, rollout, monitoring, model/provider migration | ai-mlops | | API cost model, caching, self-host breakeven | ops-cost-optimization | | Pretraining, tokenizer building, scaling laws | ai-pretraining, ai-scaling-laws | | Multi-GPU parallelism | ai-distributed-training | | Hugging Face TRL trainer specifics | the huggingface-skills: plugin (external) |

Default Workflow

1. Confirm the gap is behavior, not knowledge or reasoning. Missing facts belong in retrieval and multi-step logic in the thinking budget (see the advisor ladder). Fine-tune for a stable behavior, format, or style gap that survives prompt and RAG fixes.
2. Build the eval before the dataset. Hold out a task set by slice, capture a same-seed baseline of the untuned model, and add an out-of-domain forgetting set.
3. Build the dataset. Group duplicate and near-duplicate source examples, then assign whole groups to train or eval before augmentation or synthetic generation; check for residual overlap. Apply the model's own chat template, and use the same template at train and at serve (generation prompt and end-of-turn token included): render one example through the training pipeline and through the serving engine and diff the token IDs, because a mismatch raises no error and looks like a fine-tune that did nothing (§5E). Compute loss on assistant tokens only (ignore_index=-100 on prompt tokens). Packing must not let attention or loss cross example boundaries. See dataset-formatting-guide.md.
4. Start with PEFT. LoRA or QLoRA on all linear layers; record rank, alpha, alpha convention, target modules, and effective batch (micro × accumulation × world size). Trainable share = Σ r·(d_in + d_out) over adapted matrices ÷ total parameters; compute it for your model instead of quoting a percentage.
5. Escalate only on evidence. If LoRA plateaus on a hard, far-from-pretraining domain, follow the escalation order in fine-tuning-recipes.md and keep the forgetting eval.
6. Distillation step. Before any teacher call, confirm the teacher's terms permit training a competing model, and record the licence. Then pick the supervision signal your access gives you (logits, generated text, or rationales) per advanced-llm-patterns.md.
7. Gate the release. Run the gate on the exact serving artifact and configuration: test the adapter loaded separately if that is how it will serve, or test the merged and quantized artifact if that is the deployment path. A delta that appears only after quantization is not the fine-tune's (merge and quantize). Pick the release checkpoint on the task and forgetting slices; validation loss only ranks checkpoints for early stopping. Compare against the baseline by slice, schema-valid and tool-call rates, and the forgetting set. Hand deployment to ai-mlops.

Quick Reference

| Decision | Default | Change when | Avoid | |---|---|---|---| | Method | LoRA/QLoRA, all-linear targets | Plateau on a hard domain after rank and DoRA tries → GaLore, then full FT | Jumping to rank 256 | | Memory | QLoRA when the base does not fit in BF16 | Base fits → plain LoRA | Sizing MoE by active params (all experts load) | | Multi-GPU | Data parallel when the frozen base fits one GPU | Base does not fit → FSDP / ZeRO-3 (memory model) | Sharding by reflex | | Loss | Assistant tokens only | Pretraining-style continued training on raw text | Training on system and user tokens | | Distillation | Sequence KD from teacher outputs | Logit access → soft-label KD; reasoning traces → trace KD | Any teacher whose terms forbid training a competitor | | Payback | Use the ROI calculator | Volume is high and the domain is stable | Fine-tuning for knowledge that changes |

Known Traps

• fine-tuning for information that should live in retrieval, or for a reasoning gap that a larger thinking budget closes
• quoting a LoRA parameter percentage instead of computing Σ r·(d_in + d_out) for the actual model
• computing loss on prompt tokens, or packing examples so attention crosses boundaries
• splitting near-duplicate source examples across train/eval, or augmenting before assigning source groups to a split
• distilling from a hosted model whose terms forbid training a competing model
• gating a compressed or tuned model on perplexity; reasoning, schema-valid, and tool-call slices break first
• treating a too-good eval score as success instead of a leakage alarm
• sizing an MoE fine-tune by active parameters, or ignoring router drift toward a few experts

Trainer features, supported losses, and framework maintenance status change often.

1. Check the trainer's own docs for supported methods and losses before promising one (for example, whether a preference loss is built in).
2. Check the model card and config.json for architecture facts (layers, hidden size, KV heads, MoE experts) instead of recalling them.
3. Check a framework's repository status before starting new work on it; some fine-tuning libraries have wound down.

Scripts

This skill ships no scripts. The per-call cost estimator is in ops-cost-optimization (scripts/cost_estimator.py), and the offline prompt regression runner is in ai-evals (scripts/prompt_eval_runner.py).

Navigation: References

• Fine-Tuning Recipes: SFT, instruction tuning, LoRA/QLoRA configuration, PEFT method family, full-FT escalation, multi-GPU and MoE notes, safety data, mid-training, data feedback loops, contamination control
• Dataset Formatting Guide: instruction, chat, and transformation formats; dataset hygiene; loss masking (ignore_index=-100, first EOS unmasked); packing inside example boundaries. ai-pretraining delegates SFT formatting here.
• Adaptation Patterns: task-specific tuning, synthetic data, knowledge distillation (logit, sequence, on-policy, rationale, prune-then-distill, reasoning-trace) with the teacher licence gate, compression checklist, multi-task learning. ai-architecture-advisor owns whether to distill; ai-pretraining delegates distillation here.
• Tokenizer Diagnostics: fertility and parity, glitch and undertrained tokens, domain fragmentation, number tokenization, typo brittleness. Building a BPE tokenizer is ai-pretraining.
• Multimodal Patterns: vision, audio, and document workflows. Outside the adaptation scope; kept here until it has an owner.

Moved out of this skill: post-training → ai-post-training; eval patterns → ai-evals; cost economics → ai-api-cost-guide; deployment, migration, and monitoring → ai-mlops; the build ladder and model selection → ai-architecture-advisor.

Templates

• Fine-Tuning Config: reproducible SFT/LoRA config record
• SFT Chat Dataset and Instruction Dataset: minimal record shapes
• Data Quality: dedupe, PII, and quality checks for training data
• Fine-Tuning ROI Calculator: break-even analysis for an adaptation investment

External Sources

See data/sources.json for the papers and docs behind the recipes (LoRA, QLoRA, DoRA, rsLoRA, IA³, LoRA-vs-full-FT, distillation) and provider lookup pages for current-fact checks.

Learnings Loop

When prior decisions or pitfalls are relevant, consult learnings.consolidated.md if present; use learnings.md only for needed history or as the available fallback. Otherwise skip both.

After applying it, if you encountered a pattern worth remembering, a mistake worth preventing, or a domain fact that surprised you, append one dated bullet to learnings.md via agents-skills-feedback-loop/scripts/append_learning.py. Do not modify SKILL.md itself.