Calibration - asking for uncertainty and getting something usable
Lesson 3 of 6 · Run your controls
1 · Learn the move · Enhance the Output
Ask a model how confident it is and you get performed confidence - a number invented to sound calibrated. The usable version is structural: pin an output format that carries uncertainty per claim, with tiers you define and a column for what would change the answer. Three tiers are enough. HIGH: standard reference material, checkable in any handbook. MEDIUM: probably right, depends on specifics the model cannot see. LOW: pattern-matched, could be invented. Treat the tiers as a sorting tool for your checking time, never as probabilities - a model's stated confidence is not calibrated, but its relative ranking of its own claims is usually worth having. Check LOW first, MEDIUM second, and spot-check HIGH the way you spot-check anything.
Answer the question below. Output format: a table with columns CLAIM, TIER, WHAT WOULD CHANGE THIS. Tiers: HIGH = standard reference knowledge, checkable in a handbook; MEDIUM = depends on specifics of my system you cannot see; LOW = pattern-matched, could be invented. Every row gets a tier. No prose outside the table except one line stating what you were not able to assess. Question: [question]
2 · Your turn. You write the prompt
You are choosing a detergent for solubilizing a membrane protein, and the model's advice sounds equally sure about everything - CMC values, downstream assay compatibility, and what your particular protein will tolerate. Those are not equally knowable. Write a prompt that forces the answer into tiers you can allocate checking time against.
Remember: the AI sees only your prompt, not this page. If the situation isn't in your prompt, it doesn't exist.
Optional. These shape the output when you run your prompt below, not your score.