
Build a cost-per-token calculator script for your inference workload
You'll have a working Python script that inputs your batch size, sequence length, and hardware specs, then outputs the actual cost per token and compute utilization rate for your deployment.
What you'll make
Cost-Per-Token Calculator Script
- hardware_specs() — input validation for GPU memory, compute specs, and pricing: what to include
- workload_params() — batch size, sequence length, and model size configuration: what to include
- bottleneck_analysis() — arithmetic intensity calculation and constraint identification: what to include
- cost_calculator() — per-token cost breakdown and utilization rates: what to include
- scenario_projector() — diffusion vs autoregressive cost comparison: what to include
- main() — orchestration with real parameter input and formatted output: what to include
Paste your complete Python script that runs end-to-end on your actual deployment parameters
The 4 parts
Each part ends with a check you can see.
Measure Your Current Bottleneck
Gather real utilization data from your inference deployment and calculate arithmetic intensity to identify whether you're memory-bound or compute-bound.
Done when
You have measured GPU utilization, memory bandwidth usage, and calculated your actual FLOP/byte ratio for at least one representative workload.
Build Cost Calculation Functions
Code the core cost-per-token calculator with hardware specs, utilization rates, and bottleneck analysis functions.
Done when
You have working Python functions that take hardware specs and workload parameters as input and return cost-per-token and utilization breakdowns.
Model Architectural Scenarios
Add scenario projection functions that compare your current autoregressive costs against projected diffusion model costs under different adoption timelines.
Done when
You have a scenario_projector() function that models cost changes from architectural shifts and can run 'what-if' analyses for different transition timelines.
Integrate and Validate Script
Wire all functions into a complete script, test it on your real deployment parameters, and validate outputs against your observed costs and utilization patterns.
Done when
You have a complete, tested Python script that runs end-to-end on your actual deployment parameters and produces cost-per-token calculations you trust for decision-making.
Capstone: you put together your Cost-Per-Token Calculator Script.
Where this helps
An elective: a single lesson that sits outside rtwo's job certificates and doesn't count toward one.
About this lesson
What you'll be able to do
- Run a working Python script that calculates cost-per-token for your specific hardware and workload parameters
- Identify whether your deployment is memory-bound or compute-bound and quantify the bottleneck severity
- Estimate how diffusion model adoption would change your infrastructure costs over the next 18 months
- Have a reusable framework for evaluating new model architectures against your current cost baseline
Who it's for
- ML engineers running inference workloads who need to justify hardware choices with real numbers
- Infrastructure teams evaluating whether to invest in high-memory vs high-compute hardware
- Anyone spending >$1000/month on inference who wants to understand where their money actually goes
What it won't cover
- Model training cost optimization or distributed training economics
- Specific cloud provider pricing negotiations or enterprise contract terms
- Production deployment orchestration or serving framework selection
- Diffusion model implementation details or algorithmic improvements
Skills you'll practise
- Hardware bottleneck identification from utilization metrics
- Cost modeling with realistic deployment constraints
- Scenario analysis for architectural transitions
- Quantitative hardware-software tradeoff evaluation