Agent Skills: Experimental Django Recommender + Search Backend Best Practices

Experimental Django Recommender + Search Backend Best Practices

Implementation patterns for a Django backend serving mixed-results recommendations (Personalize / Databricks / microservice fan-out) and OpenSearch-backed search/feeds. 48 rules across 8 categories, ordered by execution lifecycle impact — earlier categories cascade through everything downstream.

This is the backend peer of the react-fetch-cache-patterns skill. React handles client-side waterfalls and caching; this skill handles server-side fan-out, downstream protection, OpenSearch query design, and ML-blend orchestration.

When to Apply

• Building or reviewing Django views that fan out to AWS Personalize, Databricks Model Serving, internal microservices, or any ML inference downstream
• Designing OpenSearch query endpoints (search results, infinite feeds, faceted search)
• Implementing a recommendations endpoint that blends multiple ranker outputs
• Investigating "Django backend slow when downstream is degraded" or "Personalize quota exhausted"
• Adding caching, retry, circuit breakers, or rate limiting to outbound calls
• Choosing between sync and async Django views, configuring uvicorn vs gunicorn
• Designing DRF response shapes for paginated feeds, partial results, or degraded paths

Rule Categories by Priority

| # | Category | Impact | Prefix | Rules | |---|----------|--------|--------|-------| | 1 | Fan-out Orchestration | CRITICAL | orch- | 8 | | 2 | External Service Protection | CRITICAL | protect- | 7 | | 3 | OpenSearch Query Patterns | CRITICAL | search- | 8 | | 4 | Result Blending & Personalization | HIGH | blend- | 5 | | 5 | Caching Strategy | HIGH | cache- | 5 | | 6 | Resilience & Partial Results | HIGH | resilience- | 5 | | 7 | Async & Concurrency | MEDIUM-HIGH | async- | 5 | | 8 | API Response Design | MEDIUM | api- | 5 |

Quick Reference

1. Fan-out Orchestration (CRITICAL)

• orch-parallel-fanout-asyncio-gather — Use asyncio.gather for independent downstream calls; never await sequentially
• orch-return-exceptions-on-fanout — return_exceptions=True so one failure doesn't poison the whole gather
• orch-propagate-request-deadline — Pass a deadline through every downstream call to bound whole-request latency
• orch-reuse-async-clients — One httpx.AsyncClient per downstream at module scope; never per-request
• orch-bounded-fanout-concurrency — Cap per-request fan-out with asyncio.Semaphore to protect the pool
• orch-no-blocking-in-async-view — Never block the event loop with sync ORM/IO in async views
• orch-avoid-await-in-loop — for item in items: await ... is serial; use asyncio.gather with comprehension
• orch-batch-with-bulk-endpoint — Bulk endpoint over N parallel calls; DataLoader pattern for batchers

2. External Service Protection (CRITICAL)

• protect-per-downstream-timeout-budget — Different timeouts per service matched to each downstream's p99
• protect-circuit-breaker-per-downstream — One breaker per downstream so failures stay isolated
• protect-jittered-retry-backoff — Full-jitter exponential backoff to prevent thundering-herd recovery
• protect-no-retry-on-4xx — Skip retry on 4xx and non-idempotent failures; distinguish connect vs read errors
• protect-bulkhead-connection-pool — One connection pool per downstream so one slow service doesn't starve others
• protect-client-side-rate-limit — Token bucket toward each downstream to stay under their quota
• protect-honor-retry-after-header — Parse Retry-After (seconds or HTTP-date) on 429/503

3. OpenSearch Query Patterns (CRITICAL)

• search-use-search-after-not-from — search_after cursor instead of from/size for any paginated endpoint
• search-filter-source-fields — Restrict _source to fields you render; use docvalue_fields for sortable
• search-bool-filter-vs-must — Non-scoring clauses in filter (cacheable), scoring clauses in must
• search-function-score-for-blending — Use function_score to blend personalization signals in-engine
• search-stable-tiebreaker-sort — Always end sort with _id (or unique numeric field) for stable cursors
• search-alias-for-blue-green-reindex — Query through aliases; never direct index names
• search-enable-request-cache — request_cache=true for hit-returning queries; canonicalize request body
• search-shard-aware-routing — Use routing keys to limit per-query shard fan-out

4. Result Blending & Personalization (HIGH)

• blend-normalize-scores-across-sources — Min-max or RRF normalize before blending Personalize/Databricks/OpenSearch
• blend-mmr-for-diversity — Maximal Marginal Relevance to avoid monocultures in top-K
• blend-dedup-across-sources — Canonical item ID dedup; bonus for cross-source corroboration
• blend-cold-start-fallback — Popular/editorial fallback for new users; tiered personalization
• blend-anonymous-vs-personalized-paths — Cheap segment-keyed cache for anon traffic; ML only for logged-in

5. Caching Strategy (HIGH)

• cache-redis-with-stampede-protection — SETNX lock + jittered TTL + probabilistic early refresh
• cache-version-on-model-deploy — Bake model version into cache keys; no flush needed on retrain
• cache-segment-keyed-isolation — Include auth/role/locale/segment in keys to prevent cross-context leakage
• cache-two-tier-process-and-redis — Process LRU in front of Redis for the hottest keys
• cache-negative-results — Cache absences and empty results with shorter TTL

6. Resilience & Partial Results (HIGH)

• resilience-partial-response-envelope — partial: true + sources_used + failed_sources in response
• resilience-serve-stale-from-redis — Two TTLs (fresh + stale); serve stale on origin failure
• resilience-default-ranking-fallback — Precomputed default ranking when all ML sources are down
• resilience-per-source-observability — Tag every downstream call with structured source/outcome metadata
• resilience-degrade-search-gracefully — Tier 1 → tier 2 → tier 3 fallback for OpenSearch outages

7. Async & Concurrency (MEDIUM-HIGH)

• async-sync-to-async-orm — Use Django 4.1+ async ORM (aget, afilter) or sync_to_async with thread_sensitive=True
• async-worker-model-uvicorn-vs-gunicorn — Run ASGI (uvicorn or gunicorn+UvicornWorker) for true async concurrency
• async-fire-and-forget-with-create-task — create_task for analytics/audit; add error handler; hold task references
• async-context-vars-for-request-scope — contextvars.ContextVar for per-request state; not threading.local
• async-cancel-on-client-disconnect — Check await request.is_disconnected(); propagate cancellation

8. API Response Design (MEDIUM)

• api-cursor-pagination-in-drf — Cursor pagination over page-number; opaque base64 cursors
• api-serializer-perf-select-related — select_related/prefetch_related/only to eliminate N+1
• api-etag-and-cache-control-headers — ETag + Cache-Control + Vary for CDN/client reuse
• api-compression-and-payload-shaping — gzip/brotli; sparse fieldsets; msgpack for internal APIs
• api-throttle-per-user-and-endpoint — DRF throttle classes per user/anon and per expensive endpoint

How to Use

1. Open references/_sections.md for category definitions and impact rationale
2. Read individual rule files for incorrect-vs-correct code examples (each ~150-300 lines with Python code)
3. For ready-to-use scaffolds, see scaffolding templates
4. The AGENTS.md navigation document (auto-generated) provides a TOC for browsing

Scaffolding Templates

Five ready-to-adapt Python templates under assets/templates/:

| Template | Purpose | |----------|---------| | fanout_recommender_service.py.template | Async fan-out client to Personalize/Databricks/microservice with per-downstream circuit breaker, bounded timeout, partial-result return | | opensearch_search_view.py.template | DRF view + OpenSearch search_after cursor + function_score blending + _source filtering | | result_blender.py.template | Score normalization + MMR diversity + canonical-ID dedup + cold-start fallback | | redis_cache_with_stampede.py.template | Stampede-safe cached function decorator with SETNX lock and jittered TTL | | degraded_response.py.template | Partial-results envelope with per-source status flags + tier-based fallback |

Reference Files

| File | Description | |------|-------------| | references/_sections.md | Category definitions, ordering, impact rationale, tier definitions | | assets/templates/_template.md | Template for authoring new rules | | metadata.json | Version, references, abstract |

Related Skills

• react-fetch-cache-patterns — Client-side peer covering React data fetching/caching (Suspense, query libraries, prefetch)
• io-bound-data-processing — Python async patterns for batch and pipeline workloads
• inngest-nextjs-patterns — Workflow patterns for server-side step functions