When to distrust - novelty, specificity, numbers, citations
Lesson 5 of 6 · Run your controls
1 · Learn the move · Fact-check list
Distrust is not a mood, it is a gradient you can read off the text. Risk climbs with four features. Novelty: the newer or narrower the topic, the thinner the training data under the fluent prose. Specificity: a mechanism described in general terms is usually sound; a specific catalog number, kit step, or buffer recipe is where invention hides. Numbers: every unsourced number is a candidate fabrication, and precise-looking numbers are worse, because precision is cheap to generate. Citations: the highest-risk object a model produces - plausible authors, real journal, invented paper. The move: extract every high-risk item into a checklist, and give each one a verification state. VERIFIED with a source you opened, UNVERIFIED, or FAILED. No reference leaves the list without a state.
Extract from the text below every item in these classes: numbers with units, named reagents or catalog items, protocol-specific steps, and citations. Output a checklist: ITEM (quoted), CLASS, RISK (higher for novel, specific, numeric, cited - one sentence why), STATE. Initial state is UNVERIFIED for everything; VERIFIED is a state only I can assign after opening a source. Order by risk, highest first. Text: [paste output]
2 · Your turn. You write the prompt
A model drafted the background section for your fellowship application, covering a fast-moving subfield with only eighteen months of literature. It cites nine papers and quotes three effect sizes. You have one afternoon to check it. Write a prompt that turns the draft into a ranked checklist so the afternoon goes where the risk is.
Remember: the AI sees only your prompt, not this page. If the situation isn't in your prompt, it doesn't exist.
Optional. These shape the output when you run your prompt below, not your score.