
Build a model-picker script for your workflow
You'll have a working script that tests your typical tasks against 2-3 AI models, logs cost and quality, and recommends which to use based on your priorities.
What you'll make
Model-Picker Script
- test_task() — function to run one task against multiple models: what to include
- log_results() — cost and quality tracking: what to include
- analyze_results() — recommendation engine: what to include
- main() — orchestration and user interface: what to include
Paste your working Python script that can be run from command line
The 2 parts
Each part ends with a check you can see.
Test Models on Real Tasks
Pick 5 tasks you actually use AI for, run them against 2-3 different models, and score the outputs based on your quality standards
Done when
Raw test results for 5 tasks across 2-3 models with cost data, response time, and your quality scores (1-5) for each output, plus notes on where each model succeeded or failed
Code the Decision Engine
Write the recommendation logic that weighs cost vs quality based on your priorities and build the complete model-picker script
Done when
A working model-picker script with test_task(), log_results(), analyze_results(), and main() functions that recommends models based on configurable cost-quality preferences
Capstone: you put together your Model-Picker Script.
Where this helps
An elective: a single lesson that sits outside rtwo's job certificates and doesn't count toward one.
About this lesson
What you'll be able to do
- Run a working model-picker script on at least 3 of your typical work tasks
- Have cost and quality data for 2-3 models on your actual workload
- Know which model to choose for different task types based on your priorities
- Have a documented method for testing new models when they launch
Who it's for
- Anyone spending money on AI API calls who wants data on which model gives the best value
- People who switch between models based on guesswork and want to test systematically
- Teams evaluating AI tools who need concrete performance data for budget decisions
What it won't cover
- Setting up API credentials (assume you can call the models)
- Advanced benchmark interpretation or statistical significance testing
- Building production model-switching infrastructure
Skills you'll practise
- Task-specific model evaluation
- Cost-quality tradeoff analysis
- Benchmark score interpretation
- Systematic model testing