Problem: 8 Rules, 3 Pairs That Cancel Each Other Out, and the Model Never Says a Word 🔥
Ok here's the thing. I inherited our support system prompt and it is about 40 lines now, because every time somebody has a bad ticket they append a rule to the bottom of it. Nobody reads the top. I did. Once. Here is the section I pulled out to test, cleaned up, 8 rules: ``` You are a support assistant for an industrial supplier. 1. Always respond in British English. 2. Always include the ticket reference at the start of your reply. 3. Never quote a delivery date without checking stock. 4. Keep replies to at most three sentences. 5. Always sign off with the agent name. 6. Use American English spelling throughout. 7. Never include a ticket reference, as it confuses customers. 8. Be thorough and explain the reasoning behind any delay in full detail. ``` Now read 1 against 6. Then 2 against 7. Then 4 against 8, which I nearly walked past. Three pairs, every one of them a flat contradiction, sitting inside the same 8 lines and going out to customers for eighteen months! Test input was: "Ticket 88213. When will my bearing order ship? It has been two weeks." Two models. Both of them just **picked a side** on each pair and handed back a lovely confident reply. No flag, nothing. 🔥 **Model A** went American and kept the ticket ref. Then it wrote four paragraphs about warehouse scheduling. ✅ **Model B** went British and dropped the ticket ref. Three sentences, done. Same prompt. Opposite behaviour on all three pairs, and this thing is like 3x the size it was when I got here. Then I asked it to score its own reply 1-10 on rule compliance. It gave itself a **9.4**. I asked it which rules it had broken. It said NONE. So what I am after is two things: 1. **A resolution** - some way to make the outcome the same on every run and every model, because right now we are shipping a coin flip. 2. **A signal** - and genuinely this is the one I care about more. If my prompt contradicts itself I want it failing in CI, not arriving as a customer who got two different answers. Rule 3 has to survive whatever you do, by the way. Stock is not checked in that turn so nobody gets a date. Both replies pasted in the comments 👇
Real model calls, transcripts kept
0 works · 2 fails
Times copied by users
Problem Instructions
Get one deterministic reply out of a support prompt that contradicts itself in three places, and get the collisions named before the reply goes out instead of resolved silently. C.L.A.R.I.T.Y. covers the rewrite side and does nothing on detection, so this is the gap.
- •All three colliding pairs named: 1 against 6, 2 against 7, and 4 against 8.
- •Rule 3 survives whatever the resolution is, so no delivery date in a turn where stock was not checked.
- •Same reply on the bearing ticket across two models, three runs each.
- •The **contradiction** report comes back as a field a build step can read, not as prose I have to skim.
Response contract
Response contract, added 10 August 2026 so answers to this problem can be checked mechanically instead of read by eye. Solutions published before that date were not written against it and are not judged by it.
Reply with one JSON object and nothing else:
{
"conflicts": [[int, int]], // each colliding pair, lower rule number first
"reply": string // the reply to send the customer
}