Technology Assessment Skill

Technology Assessment

Overview

Technology assessment evaluates technologies for strategic fit, technical feasibility, and competitive advantage. This skill covers frameworks for making informed technology decisions.

Required Frameworks

| Framework | Output Section | Required | Condition | |-----------|---------------|----------|-----------| | TRL Levels | Technology Readiness | yes | — | | Hype Cycle Mapping | Hype Cycle Position | yes | — | | Build vs Buy Matrix | Build vs Buy Analysis | yes | — | | Domain-Specific Due Diligence | Due Diligence | conditional | Applicable domain detected (AI/ML, Fintech, Healthcare, Infrastructure) |

Trend Indicators: Load and apply the trend indicator definitions from protocols/TREND-INDICATORS.md.

Critical Assessment Rules

Follow these rules to produce assessments that are honest, grounded, and actionable:

Contextual Grounding: Always tie scores, recommendations, and analysis back to the user's specific context (team size, scale, budget, timeline). A technology that scores 5/5 for Google may score 2/5 for a 10-person startup.
Honest Scoring: Do not inflate scores. A score of 2 or 3 is normal and valuable — it tells the user where the risks are. If a technology has known weaknesses, say so. Scores above 4 require strong justification.
Decisive Recommendations: End every assessment with a clear recommendation. "It depends" is not a recommendation. If the answer genuinely depends, specify what it depends ON and give a recommendation for each scenario.
Trend Indicators Everywhere: Apply INC/DEC/CONST trend indicators to every technology mentioned, not just the primary subject. Trend context helps the user understand trajectory.
Domain Tailoring: When the user's domain has specific considerations (AI/ML, fintech, healthcare, infrastructure), extend the standard framework with domain-specific evaluation criteria rather than using a generic checklist.
Always Consider Alternatives: Every assessment should mention at least 2-3 alternatives to the technology being assessed, with brief comparative notes. The user needs to know what else they could choose.
Quantify When Possible: Prefer concrete numbers over vague qualifiers. "50k msgs/sec" is better than "high throughput." "$23k/year" is better than "affordable." "3-6 months" is better than "some time."

Assessment Dimensions

Technical Maturity

Proven vs. emerging
Stability and reliability
Community/ecosystem size
Documentation quality

Strategic Fit

Alignment with business goals
Competitive differentiation potential
Integration with existing systems
Scalability trajectory

Risk Profile

Technology risk (will it work?)
Adoption risk (will people use it?)
Vendor risk (will they survive?)
Security risk (vulnerabilities)

Economic Factors

Total cost of ownership
Time to value
Resource requirements
Opportunity cost

Technology Readiness Levels (TRL)

| Level | Description | Characteristics | |-------|-------------|-----------------| | TRL 1-3 | Research | Concept only, no implementation | | TRL 4-6 | Development | Prototype, limited deployment | | TRL 7-8 | Production | Proven at scale, enterprise-ready | | TRL 9 | Mature | Widely adopted, commoditized |

Gartner Hype Cycle Mapping

Position technologies on the hype cycle:

Innovation Trigger: New technology emerges
Peak of Inflated Expectations: Hype exceeds reality
Trough of Disillusionment: Reality sets in
Slope of Enlightenment: Practical use cases emerge
Plateau of Productivity: Mainstream adoption

Build vs. Buy Framework

Build When:

Core competitive differentiator
Unique requirements not met by market
Long-term cost advantage
Internal expertise available
Control is critical

Buy When:

Commodity functionality
Time-to-market critical
Proven solutions exist
Maintenance burden undesirable
Lower total cost of ownership

Build vs. Buy Matrix

| Factor | Weight | Build Score | Buy Score | |--------|--------|-------------|-----------| | Strategic importance | 30% | 1-5 | 1-5 | | Differentiation | 25% | 1-5 | 1-5 | | Time to market | 20% | 1-5 | 1-5 | | Total cost | 15% | 1-5 | 1-5 | | Risk | 10% | 1-5 | 1-5 | | Weighted Total | 100% | X.X | X.X |

Show the math: For each factor, multiply the score by the weight. Sum the weighted scores for Build and Buy separately. The option with the higher weighted total wins. Example: Strategic importance Build=3, weight=30% → 0.90. Display both the individual weighted contributions and the totals.

Technology Stack Analysis

Layers to Assess

Infrastructure

Cloud provider (AWS, Azure, GCP)
Compute (containers, serverless, VMs)
Storage and databases
Networking

Platform

Application frameworks
Runtime environments
Development tools
CI/CD pipeline

Application

Core product technologies
Third-party integrations
APIs and services
Security stack

Stack Evaluation Criteria

| Criterion | Question | Weight | |-----------|----------|--------| | Performance | Does it meet requirements? | High | | Scalability | Can it grow with us? | High | | Maintainability | Easy to support? | Medium | | Security | Meets compliance needs? | High | | Cost | Fits budget? | Medium | | Talent | Can we hire for it? | Medium | | Community | Active ecosystem? | Low |

Scoring Calibration

Use these anchor definitions to ensure consistent scoring across assessments:

| Score | Meaning | Example | |-------|---------|---------| | 1 | Critical weakness, likely blocker | No production deployments exist; vendor may not survive 12 months | | 2 | Significant concern, requires mitigation | Works in limited scenarios but known scaling issues; small talent pool | | 3 | Adequate, meets minimum requirements | Production-ready with caveats; moderate community; reasonable cost | | 4 | Strong, clear advantage | Battle-tested at scale; large ecosystem; good cost/value ratio | | 5 | Exceptional, best-in-class | Industry standard; massive community; proven at extreme scale |

Inflation check: If your overall score is above 4.0, re-examine each dimension. Technologies scoring 4+ across all dimensions are rare — most have at least one weak area (usually cost, talent availability, or operational complexity). An assessment with all 4s and 5s is almost certainly inflated.

Trend Indicators for Technologies

Apply three-valued logic:

INC (Increasing adoption)

Growing community
Increasing job postings
Rising search interest
New investment/funding

DEC (Decreasing adoption)

Shrinking community
Declining job postings
Migration to alternatives
Vendor instability

CONST (Stable)

Mature, established
Steady usage patterns
No major changes

Competitive Technology Analysis

Assess how competitors use technology:

| Competitor | Key Technologies | Strengths | Weaknesses | Trend | |------------|------------------|-----------|------------|-------| | A | [Stack] | [Advantage] | [Gap] | INC/DEC/CONST | | B | [Stack] | [Advantage] | [Gap] | INC/DEC/CONST |

Technical Due Diligence

For M&A or partnership evaluation:

Architecture Review

System design and patterns
Scalability approach
Technical debt level
Documentation quality

Code Quality

Testing coverage
Code review practices
Security practices
Performance benchmarks

Operations

Deployment processes
Monitoring and observability
Incident response
SLA track record

Team

Skills and expertise
Key person dependencies
Knowledge documentation
Retention risk

Domain-Specific Due Diligence Extensions

When the target company operates in a specialized domain, extend the standard four-area framework with domain-specific evaluation criteria. Every sub-item within Architecture Review, Code Quality, Operations, and Team must be covered — do not skip sub-items.

AI/ML Companies

Model quality and defensibility (proprietary data, novel architecture, data moat)
Training pipeline reproducibility (can a new engineer retrain from docs alone?)
AI ethics and bias testing across protected classes
Data licensing and transferability (do rights survive acquisition?)
Regulatory exposure (AI Act, sector-specific AI regulations)

Fintech Companies

Regulatory compliance (PCI-DSS, SOX, PSD2, banking licenses)
Transaction processing integrity (double-entry, reconciliation)
Fraud detection and prevention systems
Audit trail completeness and cryptographic integrity

Healthcare Companies

HIPAA/HITECH compliance (PHI handling, BAAs, breach notification)
Clinical validation and FDA clearance status
Interoperability standards (HL7 FHIR, DICOM)
Patient data anonymization and de-identification pipelines

Infrastructure/DevTools Companies

Multi-tenancy isolation guarantees
API stability and backward compatibility track record
Self-hosted vs. SaaS deployment model sustainability
Open-source licensing compliance and contribution model

Output Structure

## Technology Assessment Summary

### Technology: [Name]
**TRL**: [Level]
**Hype Cycle Position**: [Stage]
**Trend**: INC/DEC/CONST

### Evaluation Matrix
| Dimension | Score (1-5) | Notes |
|-----------|-------------|-------|
| Technical Maturity | X | [Note] |
| Strategic Fit | X | [Note] |
| Risk Profile | X | [Note] |
| Economic Factors | X | [Note] |
| **Overall** | X.X | |

### Build vs. Buy Recommendation
[Recommendation with rationale]

### Competitive Position
[How this tech compares to alternatives]

### Risk Assessment
- **Primary Risk**: [Risk and mitigation]
- **Secondary Risk**: [Risk and mitigation]

### Recommendations
1. [Recommendation]
2. [Recommendation]

### Monitoring Indicators
- [What to watch]
- [What to watch]

Assessment Methodology

Follow this 4-step process for every assessment:

Step 1: Understand Context

Before evaluating anything, extract the user's constraints:

Team size: How many engineers will implement/maintain this?
Scale: What volume/throughput/user count are they at now and targeting?
Budget: What can they spend (time and money)?
Timeline: When do they need this working?
Existing stack: What are they already using that this must integrate with?

If the user doesn't provide these, state your assumptions explicitly.

Step 2: Classify the Request

Determine which assessment type to produce:

Single technology assessment → Use full output template (TRL, Hype Cycle, Evaluation Matrix, Risk, Recommendations)
Build vs. Buy analysis → Use Build vs. Buy Matrix with weighted scoring and show the math
Tech stack analysis → Use three-layer framework (Infrastructure, Platform, Application) with Stack Evaluation Criteria
Competitive comparison → Use Competitive Technology Analysis table with trend indicators for each option
Technical due diligence → Use four-area framework (Architecture, Code Quality, Operations, Team) plus domain extensions

Step 3: Score and Analyze

Apply the relevant framework completely — do not skip sections
Score honestly — justify every score above 3 with specific evidence
Apply trend indicators to all technologies mentioned
Reference the user's specific constraints in scoring rationale

Step 4: Recommend Decisively

State your recommendation clearly in the first sentence
Support with 2-3 key reasons tied to the user's constraints
Acknowledge trade-offs honestly
Provide a conditional alternative ("If X changes, then consider Y instead")

Common Anti-Patterns

Avoid these failure modes that undermine assessment quality:

Resume-Driven Assessment: Recommending a technology because it's trendy rather than because it fits the user's constraints. Ask: "Would I recommend this if nobody was watching?"
Score Inflation: Giving everything 4s and 5s. If the average score across dimensions is above 3.5, verify each score has concrete justification. A realistic assessment of most technologies will have at least one dimension scoring 2 or 3.
Framework Without Recommendation: Presenting a beautifully structured analysis that ends with "it depends." The user hired you to make a call, not to organize their confusion.
Ignoring Constraints: Recommending Kubernetes to a 2-person team, or a $50k/year SaaS to a bootstrapped startup. Always check the user's stated constraints (team size, budget, timeline) and ensure recommendations are feasible within them.
Hype Cycle Blindness: Failing to position technologies on the hype cycle. Every technology has a hype trajectory. Ignoring it leaves the user vulnerable to adopting over-hyped tech or dismissing maturing tech.
Missing the "Don't" Recommendation: Sometimes the right answer is "don't adopt this technology." If the assessment reveals poor fit (overall score below 2.5, or critical dimension below 2), say so clearly.

Best Practices

Evaluate against actual use cases, not theoretical
Include non-technical stakeholders in assessment
Consider long-term maintenance, not just implementation
Document assumptions and update as they change
Prototype before committing to major decisions

Additional Resources

For detailed frameworks, see:

references/tech-evaluation-matrix.md - Complete evaluation template
references/build-buy-analysis.md - Detailed build vs. buy
examples/tech-assessment-report.md - Sample assessment

Orchestration Hints

Confidence tiers (universal scale):

High: 3+ independent, recent (<12mo) sources that converge
Medium: 2 sources OR sources >12mo old OR indirect evidence
Low: Single source, inference, or extrapolation

Dimension-specific confidence criteria below REFINE (not replace) these universal definitions.

Cross-reference dimensions: trends (technology adoption curves), competitive (competitor tech stacks and capabilities)
Alert triggers:
- Disruptive technology at TRL 7+ (system prototype demonstrated)
- Technology obsolescence risk for current market leaders
- Build vs buy cost differential >3x
Confidence rules:
- High: Technology demonstrated at scale with public benchmarks
- Medium: Multiple credible demonstrations or strong theoretical basis
- Low: Early-stage or single-vendor claims only
Conflict detection:
- Technology readiness vs trend dimension's adoption timeline
- Feasibility assessment vs competitive dimension's claimed capabilities
- Cost estimates vs financial dimension's unit economics

Agent Skills: Technology Assessment

Install this agent skill to your local

Skill Files