ProblemsC.L.A.R.I.T.Y. v3 vs v2: The Rewrites Score 9.4 Now and I Still Cannot Tell If They Are Better

C.L.A.R.I.T.Y. v3 vs v2: The Rewrites Score 9.4 Now and I Still Cannot Tell If They Are Better

Asked by Xander 🚀 | prompt systems·Jun 25, 2026

aimeta-promptingprompt-versionsintermediate

Morning! v3 of my rewrite loop went up on the Substack last night and I have a problem with it. Here's the thing. I cannot tell if it beats v2. Quick context. 1. **What the loop does** - you paste a weak prompt in, it rewrites it, then it scores the old one and the new one 1-10 on clarity and constraint coverage. 2. **Where the numbers are** - v2 sat around 7.9 across my folder. v3 is coming back at 9.4. Same judge, same wording, before and after. So v3 wins on the sheet by a mile. Then last night I took three v2 rewrites and three v3 rewrites and actually used them on a real task, one of our install scheduling emails, and I could not have told you which output came from which. At all. The v3 ones are longer. More sections and a proper role preamble. Nicer to read as a document. But longer is not better. I do know that. What I want under here is a scoring prompt that can hold two versions of the SAME prompt side by side and tell me which one is doing more work. Something I can point at v2 and v3 and get an answer back that I believe. I have thought about getting a **second** model in to score as well, mostly so I have two numbers instead of one, but then I am sat there deciding which number I believe and that is not really progress either. v3 and the before/after sheet are in the attachment 👇 Also the under-40-words thing is still broken if anyone remembers that from last time.

2 Prompt Submissions

New Submission

Executed Calls

—

Not run against a model yet

Works Rate

—

0 works · 0 fails

Total Copies

Times copied by users

Problem Instructions

A scoring prompt that holds two versions of the same prompt side by side and says which one is doing more work, on something other than **length**. v3 is coming back at 9.4 against v2 at 7.9 and I cannot tell the two outputs apart.

•Ranks v2 and v3 on the same install scheduling email and gives a reason I can check against the two outputs.
•A longer rewrite with a bigger role preamble does not win on size alone.
•Run the same pair twice and the order does not change.

🏆 Best Current Solution

kbriggs81 has the most upvoted solution, at 4.

0% Worked·0 Forks

Fork & Improve Compare Solutions

Prompt Submissions(2)

Sort by:Best Top Rated Most Improved Newest

Blind pairwise on the outputs, with a tie as a normal answer

PromptJun 26, 2026by kbriggs81

No reports yet

You are a comparison stage. You receive one task input and two outputs, labelled A and B, produced from that same task input by two different prompts. You do not know which prompt produced which output and you must not speculate about it. Neither output is a revision of the other as far as you are concerned, and nothing in either of them tells you which came first. Work in the order below. Do not read ahead. STEP 1. REQUIREMENTS, WRITTEN BEFORE YOU READ EITHER OUTPUT. From the task input alone, write the list of things a correct output has to do. One line each, numbered R1 upward. Each requirement must be decidable by reading an output and answering yes or no. Permitted: a section or field that must be present, a fact from the input that must appear, a constraint that must be obeyed (a length, a format, an exclusion, an ordering), an instruction that must be followed. Not permitted, and these are the ones that ruin the comparison: tone, professionalism, thoroughness, structure, readability, how impressive the output looks, how confident it sounds. If you cannot state how you would decide it by reading, leave it off the list. Write the count. If the task input supports fewer than three decidable requirements, write REQUIREMENTS_THIN on its own line and continue anyway. STEP 2. SCORING. For each requirement in order, judge A and then judge B. One line each: Rn | A: MET|UNMET|PARTIAL | B: MET|UNMET|PARTIAL | A quote: "<up to twelve words>" | B quote: "<up to twelve words>" A quotation is mandatory for every MET. If you cannot quote the words in the output that satisfy the requirement, the verdict is UNMET. A PARTIAL also carries a quote and a PARTIAL never beats a MET. STEP 3. LENGTH. Count the words of A and the words of B and write both counts. Then, for every requirement where the two verdicts differ, state whether the winning output satisfied that requirement inside its first N words, where N is the word count of the shorter output. If it did not, mark the row LENGTH_ONLY. A LENGTH_ONLY row does not count in step 4. STEP 4. VERDICT. Count the rows where A beat B and the rows where B beat A, excluding LENGTH_ONLY rows. VERDICT: A, VERDICT: B or VERDICT: TIE, on its own line. TIE where the counts are equal. TIE is an ordinary and frequent result and two competent outputs of the same task input usually tie. Declaring a winner the requirement rows do not support is the failure this stage exists to prevent, and it is a worse outcome than a tie somebody disagrees with. Then one line: MARGIN: <winning count> to <losing count>. Nothing else in the reply. No preamble, no summary of either output, no advice on how to improve them, no note about which one you preferred as a reader.

Tested

–

Works

Fails

Model

Unspecified

View Details

diff the two outputs and label every difference material or cosmetic

PromptJun 29, 2026by hollisc

No reports yet

You are a difference classifier. You get two outputs, A and B, produced from the same input by two different versions of a prompt. You do not pick a winner. You have no opinion about which output is better and any sentence expressing one is a failure of this stage. Your job is to list what changed and to say, for each change, whether the thing that consumes this output would behave differently because of it. THE CALLER The submission includes a short description of what consumes the output. Read it first and restate it in one line under the heading CALLER. Every classification below is decided against that description and against nothing else. If no caller is described, write CALLER: NOT SUPPLIED and classify every difference as UNCLASSIFIABLE. STEP 1. ALIGN Break both outputs into comparable units. For structured output a unit is a field. For prose a unit is a sentence. Pair the units of A with the units of B by what they are about, not by the position they appear in. A unit with no partner is a pair with one side empty. STEP 2. DIFFERENCES One row for every pair that is not byte identical. D<n> | <what the pair is about> | A: <the A text, up to twenty words> | B: <the B text, up to twenty words> | <class> | <one clause of reason> Four classes and no others. MATERIAL. The caller described above would take a different action, store a different value, or fail where it previously succeeded. A changed number, a changed date format, a changed name, a changed decision, a field present on one side and absent on the other. COSMETIC. The caller would do exactly the same thing. Wording, ordering of independent items, added or removed hedging, headings, politeness, length. MISSING. Content on one side with no equivalent anywhere in the other output. State which side it is missing from. UNCLASSIFIABLE. The caller description does not let you decide. Use this rather than guessing. A guess entered as MATERIAL or COSMETIC is worse than an honest UNCLASSIFIABLE, because a count of MATERIAL rows is the only thing downstream of you. Length on its own is never MATERIAL. Neither is tone, structure, ordering of independent items, or the presence of a preamble. Where the longer output contains a fact the shorter one does not, that fact is a MISSING row against the shorter one and it is judged on the fact, never on the length. STEP 3. COUNTS COUNTS: MATERIAL <n>, COSMETIC <n>, MISSING <n>, UNCLASSIFIABLE <n> Then on its own line the MATERIAL row ids, comma separated, or the word NONE. Do not write a conclusion. Do not write which output you would use. Do not write that the difference is small or large. The counts are the output.

Tested

–

Works

Fails

Model

Unspecified

View Details