Agent-Skills.md

Agent Skills: Scientific Critical Thinking

Evaluate research rigor. Assess methodology, experimental design, statistical validity, biases, confounding, evidence quality (GRADE, Cochrane ROB), for critical analysis of scientific claims.

UncategorizedID: drshailesh88/integrated_content_OS/scientific-critical-thinking
2

Install this agent skill to your local

pnpm dlx add-skill https://github.com/drshailesh88/integrated_content_OS/tree/HEAD/skills/cardiology/scientific-critical-thinking

Skill Files

Browse the full folder contents for scientific-critical-thinking.

Download Skill

Loading file tree…

skills/cardiology/scientific-critical-thinking/SKILL.md

Scientific Critical Thinking

Systematic evaluation of research rigor through methodology assessment, bias detection, and evidence quality frameworks.

Triggers

  • User asks to evaluate a study's quality
  • User needs to assess evidence strength
  • User is reviewing trial methodology
  • User wants to identify limitations or biases
  • User is critiquing research for an editorial

Core Capabilities

1. Methodology Critique

Validity Assessment: | Type | Question | Red Flags | |------|----------|-----------| | Internal | Did the study measure what it intended? | Confounders, selection bias | | External | Can results generalize? | Narrow population, artificial setting | | Construct | Do measures capture the concept? | Surrogate endpoints, proxy measures | | Statistical | Are conclusions supported by data? | Underpowered, multiple testing |

Study Design Hierarchy:

  1. Systematic reviews/meta-analyses of RCTs
  2. Individual RCTs
  3. Cohort studies
  4. Case-control studies
  5. Cross-sectional studies
  6. Case series/reports
  7. Expert opinion

2. Bias Detection

Cognitive Biases in Research:

  • Confirmation bias: Interpreting data to support hypothesis
  • HARKing: Hypothesizing after results known
  • Publication bias: Positive results published more
  • Spin: Overstating or misrepresenting findings

Selection Biases:

  • Sampling bias (non-representative)
  • Volunteer bias (healthier participants)
  • Attrition bias (differential dropout)
  • Survivorship bias (only studying survivors)

Measurement Biases:

  • Observer/detection bias
  • Recall bias
  • Social desirability bias
  • Hawthorne effect

Analysis Biases:

  • P-hacking (multiple testing)
  • Outcome switching
  • Selective reporting
  • Data dredging

3. Statistical Evaluation Checklist

  • [ ] Sample size adequate? (power analysis done?)
  • [ ] Statistical test appropriate for data type?
  • [ ] Multiple comparison correction applied?
  • [ ] Effect sizes reported (not just p-values)?
  • [ ] Confidence intervals provided?
  • [ ] Missing data handled appropriately?
  • [ ] Assumptions of tests verified?

4. Evidence Quality Assessment (GRADE)

Quality Levels: | Level | Meaning | Implications | |-------|---------|--------------| | High | Very confident in estimate | Strong recommendation | | Moderate | Moderately confident | Conditional recommendation | | Low | Limited confidence | Further research likely | | Very Low | Little confidence | Estimate highly uncertain |

Downgrade Factors:

  • Risk of bias
  • Inconsistency across studies
  • Indirectness (surrogate outcomes)
  • Imprecision (wide CIs)
  • Publication bias

Upgrade Factors:

  • Large effect size
  • Dose-response relationship
  • Residual confounding would reduce effect

5. Logical Fallacy Detection

Causation Fallacies:

  • Post hoc ergo propter hoc (after = because of)
  • Correlation ≠ causation
  • Reverse causation
  • Confounding as causation

Generalization Errors:

  • Hasty generalization (small sample)
  • Ecological fallacy (group to individual)
  • Exception fallacy (individual to group)

Statistical Fallacies:

  • Texas sharpshooter (finding patterns in noise)
  • Base rate neglect
  • Regression to mean confusion
  • Multiple endpoints fishing

6. Research Design Questions

When evaluating a study, ask:

  1. Question: Is the research question clear and answerable?
  2. Design: Is the study design appropriate for the question?
  3. Population: Is the sample representative of target population?
  4. Intervention: Was the intervention clearly defined and consistent?
  5. Comparison: Was the control group appropriate?
  6. Outcome: Were outcomes clinically meaningful and measured reliably?
  7. Follow-up: Was follow-up long enough and complete enough?
  8. Analysis: Was the analysis appropriate and pre-specified?

7. Claim Evaluation Framework

For any scientific claim:

  1. Identify the assertion - What exactly is being claimed?
  2. Evaluate supporting evidence - What studies support it?
  3. Check logical connection - Does evidence actually support claim?
  4. Assess proportionality - Is strength of claim proportional to evidence?
  5. Detect overgeneralization - Are limits of findings respected?
  6. Flag red flags - Conflicts of interest, spin, p-hacking?

Application to Cardiology Content

Evaluating Trial Results

  1. Check randomization and blinding adequacy
  2. Assess primary endpoint clinical relevance
  3. Evaluate intention-to-treat vs per-protocol
  4. Look for protocol changes mid-trial
  5. Examine subgroup analyses critically
  6. Consider funding source influence

For Editorials/Newsletters

  • Acknowledge study limitations explicitly
  • Don't overstate findings
  • Note where evidence is weak
  • Distinguish association from causation
  • Highlight what questions remain

Critique Output Format

When critiquing research:

  1. Summary: Brief overview of what study did
  2. Strengths: What was done well
  3. Critical concerns: Major methodological issues
  4. Important limitations: Secondary concerns
  5. Minor issues: Small points for completeness
  6. Overall assessment: Balanced conclusion on reliability