"I tweaked the prompt and it feels better now" is the most common form of prompt iteration, and it's also close to worthless - a handful of manual re-reads can't tell you whether a change actually improved results or just happened to fix the one example you were staring at. Evaluating prompts properly doesn't require a research lab; it requires a test set, a way to score outputs consistently, and the discipline to change one thing at a time.
Build a small test set before you start tweaking
A test set is just a fixed collection of representative inputs - 15 to 30 real or realistic examples covering the easy cases, the edge cases, and the ones that have actually gone wrong before. The point isn't statistical rigor, it's consistency: running every prompt version against the exact same inputs is what makes "version B is better than version A" a meaningful claim instead of a vibe.
Score outputs with a rubric, not a gut check
A rubric turns "does this look right" into a small checklist you can apply the same way every time: did it include the required fields, did it stay within the length limit, did it avoid the specific mistake you're trying to prevent, would a domain expert accept this without edits. For simple, checkable criteria - format compliance, required fields, length - you can often score automatically with a script. For judgment calls - tone, correctness of a nuanced answer - a second model call scoring against your rubric, or a quick human pass, works well at small scale.
The loop that makes iteration actual iteration: same test set every time, scored the same way, one change per revision.
Change one variable at a time
It's tempting to rewrite half the prompt at once when something isn't working, but that makes it impossible to know which change actually mattered - or whether one change helped while another quietly made things worse. Treat prompt revisions the way you'd treat any other experiment: change the role, or the format instruction, or add one example, run the full test set, compare scores, and only then decide whether to keep the change.
Know when to stop iterating
Diminishing returns set in fast - going from "works 60% of the time" to "works 90% of the time" is usually a few focused iterations; going from 90% to 99% often costs disproportionately more effort for a use case that may not need it. Decide upfront what "good enough" looks like for the actual stakes of the task, and stop there rather than chasing a perfect score a lower-stakes task doesn't need.
Where this leaves you
With a test set, a rubric, and the discipline of changing one variable at a time, you have a real loop for turning a first rough prompt into one you can trust - not by guessing, but by measuring. The last guide in this series covers what happens once a prompt built this way is ready to run in production: version control, system prompts, and change management as your prompt library grows.