
Write an honest model-capability brief
You'll have a 1-2 page assessment of a specific AI model's real strengths and weaknesses in your domain, including tasks it should not touch and how you verified that claim.
What you'll make
Model Capability Brief
- Model Context: what you tested and why
- Verified Strengths: tasks where it consistently succeeds
- Systematic Failures: patterns where it breaks down
- Boundary Conditions: edge cases and limitations
- Deployment Recommendations: when to use vs avoid
Write as a 1-2 page brief you'd actually send to a colleague making a similar decision
The 3 parts
Each part ends with a check you can see.
Test the Happy Path
Run the model on 5 straightforward tasks from your domain and document what good performance looks like
Done when
5 completed test cases with concrete examples of the model's output and your assessment of quality
Hunt for Systematic Failures
Design tests specifically to reveal where the model breaks down consistently and document the failure patterns
Done when
At least 3 adversarial test cases that reliably produce poor output, with documented patterns of how the model fails
Write the Capability Brief
Synthesize your testing into a practical brief that documents verified capabilities and clear recommendations for use
Done when
A complete 1-2 page brief with specific use cases, documented failure modes, and actionable deployment recommendations
Capstone: you put together your Model Capability Brief.
Where this helps
An elective: a single lesson that sits outside rtwo's job certificates and doesn't count toward one.
About this lesson
What you'll be able to do
- Test a specific AI model on 3-5 real tasks from your work and document where it succeeds vs fails
- Identify at least 2 systematic failure modes where the model consistently gets it wrong
- Have a written brief you'd actually share with colleagues about when to use vs avoid this model
Who it's for
- Anyone evaluating whether to adopt a specific AI model for their team or workflow
- People who need to make honest recommendations about AI capabilities to colleagues or stakeholders
- Professionals tired of vendor demos and marketing claims who want real capability data
What it won't cover
- Technical model architecture or training details
- Comparative benchmarking across multiple models
- Cost-benefit financial analysis or ROI calculations
Skills you'll practise
- Adversarial evaluation design
- Failure mode documentation
- Capability boundary identification
- Evidence-based model assessment