The audit trail - recording every filter you applied

1 · Learn the move · Template pattern

The area's done-check is a written filter log that reproduces your cleaned dataset from the raw one, and the log is a template with no optional fields: for every filter, in order - what it removed, by what criterion, with what threshold, how many rows in, how many out, who decided, and why the reason predates the result. The order matters because filters compose: exclude-then-normalize and normalize-then-exclude are different datasets. The arithmetic matters because it is the proof: raw count, minus each step's removals, equals the clean count, or the log is wrong. Kept beside the data, the log is what makes cleaned mean something - it is the difference between a dataset and an artifact of undocumented choices; it is also, not incidentally, most of your methods section's data-processing paragraph, already written. Where a past filter went unrecorded, the entry is UNKNOWN - and a log that has to say UNKNOWN is telling you something about your process worth fixing.

Build my filter log from this session's cleaning work, as a template with all fields required per filter: STEP # / FILTER (what it removes) / CRITERION AND THRESHOLD (exact) / ROWS IN -> ROWS OUT (with the subtraction shown) / DECIDED BY / REASON (stated so a reader can tell it predates the results). Order = order applied. Close with the arithmetic check: raw total minus all removals equals final count. Any step we did but did not fully record: fill what is known, mark the rest UNKNOWN - never reconstruct from memory of what we probably did.