
Build an AI compliance audit script
You'll have a working Python or SQL script that scans your AI tool logs and outputs a report flagging any instances where the system made a recommendation in a regulated domain without documenting its
What you'll make
AI Compliance Audit Script
- scan_logs() — the pattern detector: what to include
- flag_regulatory_risks() — your domain-specific rules: what to include
- generate_audit_report() — formatted compliance output: what to include
- main() — orchestration and filtering: what to include
Paste your Python or SQL script with sample output from your actual logs
The 4 parts
Each part ends with a check you can see.
Extract Your Domain Rules
Hand-code the regulatory rules your script must check, based on your actual compliance requirements
Done when
5-7 written compliance rules in if-then format that your script will check, with specific trigger conditions from your regulatory context
Find Risk Patterns in Your Logs
Mine your actual AI tool logs to identify patterns that could indicate regulatory violations
Done when
A documented list of 3-5 log patterns that correlate with regulatory risk, with examples from your real data
Calibrate Your Detection Thresholds
Test different flagging configurations against your real logs to find the sweet spot between catching issues and avoiding alert fatigue
Done when
Documented threshold settings that catch high-risk scenarios while keeping false positive rate under 20%, with test results on at least 100 log entries
Design Your Complete Workflow
Map out the full audit script architecture and code the core components yourself, using AI only for syntax help
Done when
Working scan_logs(), flag_regulatory_risks(), and generate_audit_report() functions that you wrote, with documented data flow between components
Capstone: you put together your AI Compliance Audit Script.
Where this helps
An elective: a single lesson that sits outside rtwo's job certificates and doesn't count toward one.
About this lesson
What you'll be able to do
- Run a working compliance audit script on your actual AI tool logs
- Identify 3-5 specific regulatory risk patterns your AI systems create
- Have a documented override list for the compliance flags AI gets wrong in your domain
Who it's for
- Anyone using AI tools in regulated industries who needs to document compliance
- Teams that have to answer 'what did our AI recommend?' during audits
- People comfortable with basic Python/SQL who need to operationalize AI governance
What it won't cover
- Legal advice on specific regulations
- Production-grade security or multi-tenant deployment
- Replacing human compliance review — this flags issues for human judgment
Skills you'll practise
- Regulatory impact analysis
- Log pattern recognition
- Compliance documentation
- Risk threshold calibration