Skip to main content
Skip to content
Menu

AI pretraining Beginner 4 parts

Build a Python script that quantizes and benchmarks open-weight models for your hardware

You'll have a working script that downloads an open-source model, applies quantization techniques, measures inference speed on your machine, and logs performance metrics in a CSV file.

What you'll make

Model Quantization Benchmark Script

  1. load_model() — downloads and caches the target model: what to include
  2. apply_quantization() — implements different quantization levels: what to include
  3. benchmark_inference() — measures speed and memory usage: what to include
  4. quality_check() — tests output quality degradation: what to include
  5. log_results() — writes performance data to CSV: what to include
  6. main() — orchestrates the full benchmark run: what to include

Paste your complete Python script that runs end-to-end on your machine

The 4 parts

Each part ends with a check you can see.

  1. Map Your Hardware Limits

    Profile your machine's memory, compute capacity, and identify which models you can actually run.

    Done when

    You have specific numbers: available RAM, GPU memory (if any), and at least 2 candidate models that fit your constraints.

  2. Build the Basic Loading Script

    Create a working script that downloads and loads one model without quantization to establish a baseline.

    Done when

    You have a Python script that successfully downloads, loads, and runs inference on one model, with timing and memory usage logged.

  3. Test Multiple Quantization Levels

    Extend your script to apply different quantization techniques and measure how each affects speed, memory, and output quality.

    Done when

    You have tested at least 3 different quantization configurations and can show specific trade-offs between speed, memory usage, and output quality for each.

  4. Build the Complete Benchmark Framework

    Create the final script that automates testing multiple models and quantization levels, with systematic logging and comparison features.

    Done when

    You have a complete benchmarking script that can test any compatible model with multiple quantization levels and outputs organized comparison data in CSV format.

    Capstone: you put together your Model Quantization Benchmark Script.

Where this helps

An elective: a single lesson that sits outside rtwo's job certificates and doesn't count toward one.

About this lesson

What you'll be able to do

  • Run a complete quantization benchmark script on at least one open-weight model on your local machine
  • Measure and log performance metrics (inference speed, memory usage, accuracy) across different quantization levels
  • Identify which quantization techniques work best for your specific hardware constraints
  • Have a reusable benchmarking framework for evaluating future model deployments

Who it's for

  • Developers who want to run large models locally but need to optimize for their hardware
  • ML engineers evaluating deployment options for open-source models
  • Anyone with basic Python skills who wants hands-on experience with model quantization

What it won't cover

  • Training models from scratch or fine-tuning techniques
  • Cloud deployment or distributed inference setups
  • Advanced quantization theory or implementing new quantization algorithms
  • Production security or model serving infrastructure

Skills you'll practise

  • Model loading and quantization configuration
  • Performance benchmarking with real hardware constraints
  • Trade-off analysis between speed, memory, and quality
  • Systematic testing methodology for model optimization