Agent Skills: MLflow 3

MLflow 3

Library-reference skill for open-source MLflow 3 — 24 rules across 6 categories. MLflow 3 restructured the library around the model as a first-class entity, deprecated the registry-stage vocabulary, replaced the serving stack, and changed storage and serialization defaults; a model trained on the vast MLflow 2 corpus reproduces the old idioms fluently, which is exactly why each of these rules exists. There is no rule for things a capable model already gets right.

Scope is classic-ML MLOps on self-hosted OSS MLflow. GenAI features (mlflow.genai, tracing, prompt registry, AI Gateway) appear only where confusing them with the classic APIs is itself the trap. Databricks/Unity-Catalog-only features (Deployment Jobs) are flagged as out of scope where a model might scaffold them against OSS.

Pinned to mlflow 3.15.1 (Python ≥ 3.10). API claims were verified against the unpacked mlflow / mlflow-skinny 3.15.1 wheels.

When to Apply

• Writing or reviewing training code that logs models, metrics, params, or datasets with MLflow
• Registering model versions and wiring promotion across dev/staging/prod (aliases, copy_model_version, tags, webhooks)
• Standing up or hardening an mlflow server — backend store, artifact store, auth
• Evaluating candidate models and gating promotion on thresholds
• Serving models — mlflow models serve, build-docker, /invocations clients, pre-deploy validation
• Migrating an MLflow 2-era codebase (stages, artifact_path, mlflow.evaluate, ./mlruns) to MLflow 3

Rule Categories

| # | Category | Prefix | Covers | |---|----------|--------|--------| | 1 | Model Logging & LoggedModel | log- | name= not artifact_path, models decoupled from runs, input-example-driven signatures, register-at-log-time, skops/torch.export serialization defaults, model-linked metrics and search_logged_models | | 2 | Model Registry & Promotion | reg- | Aliases replacing stages, alias-based lookup, per-environment registered models with copy_model_version, gate state in tags, OSS webhooks vs Databricks-only Deployment Jobs | | 3 | Tracking Backend & Server | track- | sqlite:///mlflow.db default, database-only server backends and migrate-filestore, proxied artifacts topology, telemetry opt-out, autolog input-example default | | 4 | Serving | serve- | FastAPI scoring server (MLServer removed), /invocations payload contract, mlflow.models.predict pre-deploy validation, build-docker for clusters | | 5 | Evaluation & Gates | eval- | mlflow.models.evaluate vs mlflow.genai.evaluate, threshold gating with validate_evaluation_results after baseline_model's removal | | 6 | Environment & Reproducibility | env- | Generated environment files as the serving source of truth, dependency pinning and uv capture, bundling custom code with code_paths |

Quick Reference

1. Model Logging & LoggedModel

• log-name-not-artifact-path — name= creates a searchable LoggedModel; artifact_path is deprecated and warns
• log-models-are-not-run-artifacts — no start_run required; artifacts live under models/<model_id>/, addressed by model_uri
• log-input-example-infers-signature — input_example= infers the signature and validates serving input at log time
• log-register-at-log-time — registered_model_name= or register_model(model_info.model_uri), never composed runs:/ paths
• log-serialization-defaults-changed — sklearn/lightgbm write skops, pytorch writes torch.export, xgboost writes UBJSON
• log-link-metrics-search-models — log_metric(model_id=, dataset=) and search_logged_models rank models without run bookkeeping

2. Model Registry & Promotion

• reg-aliases-not-stages — set_registered_model_alias + models:/name@alias replace the deprecated stage APIs
• reg-resolve-by-alias-not-latest-versions — deployment code resolves an assigned alias, never get_latest_versions or /latest
• reg-per-environment-registered-models — dev.*/staging.*/prod.* models with copy_model_version as the promotion primitive
• reg-gate-state-in-tags — validation_status tags carry the review state stages used to imply
• reg-webhooks-oss-deployment-jobs-not — registry webhooks are OSS; Deployment Jobs are Databricks-only

3. Tracking Backend & Server

• track-default-is-sqlite-not-mlruns — the local default is sqlite:///mlflow.db, not ./mlruns
• track-server-needs-database-backend — the file store raises at server startup since 3.13; migrate-filestore + db upgrade
• track-proxied-artifacts-one-credential — server proxies artifacts by default; clients need only MLFLOW_TRACKING_URI
• track-telemetry-on-by-default — anonymized telemetry ships on; disable explicitly in production images
• track-autolog-input-examples-off — autolog(log_input_examples=True) for deployment-candidate models

4. Serving

• serve-fastapi-scoring-server-only — FastAPI/uvicorn is the only scoring server; MLServer and Flask/gunicorn are gone
• serve-invocations-payload-keys — /invocations takes dataframe_split/dataframe_records/inputs/instances (+ signature-declared params)
• serve-predict-before-deploy — mlflow.models.predict rebuilds the real env; validate_serving_input is deprecated
• serve-build-docker-for-clusters — build-docker is the container path; the Helm chart deploys the tracking server, not models

5. Evaluation & Gates

• eval-models-evaluate-split — classic ML uses mlflow.models.evaluate; mlflow.genai.evaluate is a different API
• eval-gate-with-validate-evaluation-results — baseline_model is removed; gate with validate_evaluation_results + MetricThreshold

6. Environment & Reproducibility

• env-generated-files-drive-serving — serving envs rebuild from the generated files; pin via pip_requirements/extra_pip_requirements/uv
• env-bundle-custom-code-paths — code_paths/infer_code_paths ship the import graph with the model

How to Use

Read a reference file when its decision comes up. Each rule names the wrong default it corrects, then shows the canonical way (with an incorrect/correct contrast only where the wrong way is a real trap).

• Section definitions — category structure
• Rule template — for adding new rules
• AGENTS.md — auto-built table of contents across all rules

Related Skills

• mlflow-mlops-migration — the sibling composition workflow that takes an arbitrary ML codebase through assessment, restructuring, and a dev/staging/prod MLflow 3 setup, citing these rules at each phase

Reference Files

| File | Description | |------|-------------| | references/_sections.md | Category definitions and ordering | | assets/templates/_template.md | Template for new rules | | metadata.json | Version and source references |