
Evaluate and switch your agent's underlying model based on your actual performance needs
You'll have a comparison document showing inference speed, accuracy, cost-per-run, and context window trade-offs between two different model options running against your real data, with a justified ch
What you'll make
Agent Model Selection Report
- Performance Benchmark Results: what to include
- Cost Analysis: what to include
- Technical Constraints Assessment: what to include
- Switch Decision and Implementation Plan: what to include
Write your analysis in a document format with specific numbers and your final model choice with reasoning
The 4 parts
Each part ends with a check you can see.
Benchmark Against Your Tasks
Run your actual agent tasks against two different models and measure the performance differences
Done when
You have timing, accuracy, and cost data from running the same 3-5 agent tasks on two different models
Calculate Your Total Cost
Factor in all costs - API fees, compute time, infrastructure, and your time - to get real cost comparison
Done when
You have a complete cost analysis showing total cost per task including hidden costs like setup time and infrastructure
Evaluate Your Constraints
Assess technical and business constraints that might override pure performance considerations
Done when
You have a constraint analysis that identifies deal-breakers and determines which model options are actually feasible
Make Your Switch Decision
Synthesize your performance, cost, and constraint analysis into a defensible model choice with implementation plan
Done when
You have a clear decision on which model to use with specific reasoning and a plan for implementing the switch
Capstone: you put together your Agent Model Selection Report.
Where this helps
An elective: a single lesson that sits outside rtwo's job certificates and doesn't count toward one.
About this lesson
What you'll be able to do
- Switch your agent's underlying model based on measured performance against your specific tasks
- Compare local vs hosted model trade-offs using real cost and latency data
- Make defensible model selection decisions that account for your deployment constraints
Who it's for
- Developers running agents in production who need to optimize model choice
- Teams evaluating local vs hosted models for agent workloads
- Anyone building agents where model performance directly affects user experience
What it won't cover
- How to build agents from scratch
- Model fine-tuning or training
- Detailed prompt engineering techniques
- Hardware procurement decisions
Skills you'll practise
- Performance benchmarking
- Cost-benefit analysis
- Technical constraint evaluation
- Model switching implementation