Two column pages and the model reads straight across, or i thought it did
curlew is n=64 two column pages out of schedule documents. Policy wording down the left, endorsements and limits down the right, and about a third have a footer running the full width underneath both columns. On 41 of the 64 an exposure limit ends up attached to the wrong section heading. Not wildly wrong, which is the trouble. It attaches a limit from the right hand column to whichever heading happens to sit level with it on the left, and the result reads perfectly well and is not true. **what i assumed** That it was reading the page as a grid and going straight across both columns. So i wrote about six versions of a prompt telling it the page has two columns, the left is read to the bottom before the right one starts, the footer comes last. None of them moved it. 41, then 39, then 42, then 41, and i stopped counting properly somewhere around version four. **then i actually looked** I pulled the text layer out of ten of them into a file and read it. On 9 of the 10 the lines are already interleaved before anything goes near a model. Left line, right line, left line. So it is not reading across the page, it is reading a file where somebody already read across the page on its behalf. Which i probably knew, or half knew. I have been building this thing for two years. The bit i still think is fair: it never says anything is wrong. It takes obviously interleaved text where a sentence about flood exclusion runs straight into a sum of money, and it produces a clean confident answer off the back of it. Hand that to a person and theyd stop and ask me what the file was. Endorsements are the ones that hurt. Nobody checks an endorsement.
Not run against a model yet
0 works · 0 fails
Times copied by users
Problem Instructions
Have it say the text it was handed is out of order, instead of writing a clean answer off an interleaved page. The limits it hangs on the wrong heading read perfectly well and are not true.
- •Limits attached to the wrong section heading come in under 10 of the n=64, against 41 now.
- •Where the lines are already interleaved before the model sees them, that comes back as a field the queue can route on, not a remark in the prose.
Response contract
Reply with one JSON object and nothing else. No code fence, no prose before or after it.
{
"layout": "INTERLEAVED",
"streams": [[1, 3, 5], [2, 4, 6]],
"fields": [
{ "name": "section_heading", "value": "Section 2 Marine cargo", "stream": 0, "line": 1, "status": "OK" }
],
"cross_stream": [
{ "heading": "Section 2 Marine cargo", "heading_stream": 0, "figure": "GBP 40", "figure_stream": 1 }
]
}
LINES
Number the lines of the document from 1, in the order you were handed them, counting every line. Those numbers are the only way you refer to a line. Never renumber.
TWO HYPOTHESES AND ONLY TWO
H1 AS GIVEN: the lines already read in order, first to last.
H2 ALTERNATING: the odd numbered lines, in order, are one text, and the even numbered lines, in order, are another.
A text holds only if it reads end to end: every line follows the line before it, and the text does not stop in the middle of a clause. For H2 both halves must hold, and there is no partial credit for one good half.
Test H1, then H2. Do not invent a third hypothesis, do not split at a chosen point, do not move a single line, do not reorder by meaning, do not repair part of the page and extract from that part.
layout
"SINGLE_STREAM" when H1 holds.
"INTERLEAVED" when H1 fails and H2 holds.
"UNRESOLVED" when both fail.
streams
Arrays of 1-based line numbers in reading order, one array per recovered stream.
SINGLE_STREAM: exactly one array, holding every line.
INTERLEAVED: exactly two arrays, the odd numbered lines and the even numbered lines. streams[0] is the array containing line 1.
UNRESOLVED: [] and nothing else.
The reading order of the page is streams[0] end to end, then streams[1].
fields
Exactly four entries, always all four, always in this order: section_heading, exposure_limit, endorsement_reference, effective_date.
section_heading is the first line of the reading order that begins with the word "Section", and its value is that whole line, verbatim.
exposure_limit is read from the line labelled "Limit of indemnity:".
endorsement_reference is read from the line labelled "Endorsement reference:".
effective_date is read from the line labelled "Effective from:".
For those three the value is the text after the colon on that line, trimmed, and nothing more: from a line reading "Policy fee: GBP 40" the value is "GBP 40". Every value is a verbatim substring of one input line. Never paraphrase, never join two lines, never carry a label into a value.
Each entry carries:
"name": the field name above.
"value": the string, or null whenever status is not "OK".
"stream": the 0-based index into streams the value was read from, or null.
"line": the 1-based input line number the value was read from, or null.
"status": "OK" when the value was found; "NOT_FOUND" when the document carries no such line; "BLOCKED_ON_LAYOUT" on all four entries when layout is "UNRESOLVED". A page whose order you could not establish is not extracted from at all, and no field is filled from a figure that happens to sit nearby.
cross_stream
The refusal record for the exposure limit, at most one entry.
When the exposure limit was found, read upwards from its line through the input as handed to you and take the nearest earlier line that begins with the word "Section". If that line sits in a different stream from the figure, emit one entry: "heading" (that line verbatim), "heading_stream" (its 0-based stream index), "figure" (the value of exposure_limit), "figure_stream" (its 0-based stream index).
Otherwise emit []. Always [] when layout is "UNRESOLVED", when the exposure limit was NOT_FOUND, or when no earlier "Section" line exists.
The entry records an attachment that was refused, not one that was made. The figure still appears in fields.
ADDRESSABLE PATHS
layout, streams, streams.0, streams.1, fields, fields.0.name, fields.0.value, fields.0.stream, fields.0.line, fields.0.status, fields.1.name, fields.1.value, fields.1.stream, fields.1.line, fields.1.status, fields.2.name, fields.2.value, fields.2.stream, fields.2.line, fields.2.status, fields.3.name, fields.3.value, fields.3.stream, fields.3.line, fields.3.status, cross_stream, cross_stream.0.heading, cross_stream.0.heading_stream, cross_stream.0.figure, cross_stream.0.figure_stream.