ProblemsAsked for one JSON object with nothing round it. Three of four models found a way not to.

Asked for one JSON object with nothing round it. Three of four models found a way not to.

Asked by hollisc·
devjsonoutput-formatexpert

We turn about 40,000 inbound faxes and emails into rows a day. Support tickets are a slice of that. The prompt asks for one JSON object and nothing else, no prose, no markdown, no fence, nothing after the closing brace. I've had that line in caps for six months. Here is a real one, name and domain changed. From: dana.r@northwind-logistics.example Subject: Re: Re: FW: still broken (ticket 88213?) Hi, following up again. Since the 4.2 upgrade the route optimizer and the driver app both crash when we export a manifest. I've attached the log: ```json {"err": "NullRef", "module": "manifest.export"} ``` Your rep said someone would call me back by the 14th but nobody did. At this point I want the March invoice refunded. Please confirm. This is costing us maybe 5 hours a day, I'd call it a 4 out of 5 on your severity thing but honestly it feels like a 5. Out of that I want `ticket_id` 88213, category bug, severity 4 or 5 (he waffles and we take the higher one), `refund_requested` true. Four models, three of them failed it. One wrapped the whole object in a ```json fence with the instrucion sitting two lines above it. One came back with the object and a sentence carrying on after the brace, so there wasn't anything to parse. the third one is the one I care about. it handed back this: ```json {"err": "NullRef", "module": "manifest.export"} ``` which is the block the customer pasted in himself. valid json, parses first time. the writer takes it, the row goes in, nothign throws, because the parse check passes and it's a fine object. we have a check for the fence and a check for the parse and neither has an opinion about which object it got handed. I don't have one for this. the fence is about one in thirty on the live queue. the customers own block i have seen twice, both on mail with a log pasted in the middle of it, which is most of them. so I'm after wording that gets one object back with nothing round it, off a document with its own json sat inside it. one model of the four is clean and its not the one we run. manual queue is one person on nights and she is reading for the refund line, she isnt checking the ticket number against anything.

7 Prompt Submissions
Executed Calls
42

Real model calls, transcripts kept

Works Rate
50%

1 works · 1 fails

Total Copies
0

Times copied by users

Problem Instructions

One JSON object back with nothing round it, off a mail that has its own JSON sat in the middle of it. The object has to be the one built from the ticket, not the log block the customer pasted.

  • Parses first time, no fence, nothing carrying on after the closing brace.
  • The object has our fields in it. A parse check that passes on the customer's own block is no use to me.
  • On the mail above it comes back with ticket_id 88213, category bug, severity 5, refund_requested true.

🏆 Best Current Solution

crustyloaf has the most upvoted solution, at 11.

0% Worked·0 Forks

Prompt Submissions(7)

Loading...