
Build a classifier script that enforces your team's content policy without needing labeled training
You'll have a working Python script that takes your policy description and a handful of unlabeled examples, then automatically flags content that violates your rules—reducing dependence on expensive f
What you'll make
Content Policy Classifier
- policy_parser() — converts policy text into structured classification criteria: what to include
- classify_content() — the zero-shot LLM classifier with confidence scoring: what to include
- batch_moderate() — processes multiple items and flags borderline cases: what to include
- main() — orchestration with output formatting: what to include
Paste your complete Python script with example policy text and at least 3 test cases showing classification results
The 3 parts
Each part ends with a check you can see.
Break Down Your Policy
Convert your content policy from human-readable guidelines into specific, testable classification criteria
Done when
You have 5-8 specific policy criteria with concrete examples of violations and borderline cases for each
Code the Zero-Shot Classifier
Build the classify_content() function that applies your policy criteria to new content with confidence scoring
Done when
You have a working classifier function that processes content against your criteria and returns violation flags with confidence scores (0-100)
Test on Real Content and Build Edge Cases
Run your classifier on a batch of real content, identify systematic failures, and document refinement patterns
Done when
You have tested the classifier on at least 20 real content examples and documented 3-5 specific edge cases where AI classification fails predictably
Capstone: you put together your Content Policy Classifier.
Where this helps
An elective: a single lesson that sits outside rtwo's job certificates and doesn't count toward one.
About this lesson
What you'll be able to do
- Run a working classifier script on real content from your domain that flags policy violations without labeled training data
- Identify which policy rules AI handles reliably versus where human judgment is still required
- Have a documented prompt refinement process for improving classification accuracy over time
- Understand when zero-shot classification works versus when you need human-in-the-loop validation
Who it's for
- Content moderators, community managers, or compliance teams who need to scale policy enforcement
- Anyone managing user-generated content who's wondered if AI could pre-screen for violations
- Teams with clear content policies but limited budget for training custom models
What it won't cover
- Training supervised ML models with labeled datasets
- Setting up production infrastructure or API rate limiting
- Legal compliance review of automated moderation decisions
- Handling appeals or contested moderation decisions
Skills you'll practise
- Zero-shot prompt design for classification
- Policy decomposition into AI-testable rules
- Confidence calibration for automated decisions
- Iterative classifier refinement from real examples