Repository navigation
Conversation
parse_json_markdown picked the first marker *kind* that occurred anywhere in the text, with backticks ahead of braces, and closed the slice at the last end marker via rfind. A single backtick after a valid payload therefore hijacked the start offset, and the valid JSON at offset 0 was reported as "could not find json block". Two payloads in one response failed the same way. Find the payload start by preferring braces over backticks, and let the JSON decoder parse exactly one value instead of slicing to the last end marker.
|
All contributors on this pull request have signed the CLA. |
Author
|
I have read the CLA Document and I hereby sign the CLA |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Symptom
parse_json_markdownrejects a valid payload when a backtick appears anywhere after it — an inline code span is enough:The same payload alone parses fine, so the failure depends purely on unrelated text elsewhere in the response.
Fixes #380
Root cause
Two independent defects, both from slicing by text search instead of parsing:
1. Start offset is chosen by marker kind, not by position. The loop
keeps the first marker kind that occurs anywhere, and
`is tried before{. A backtick later in the text therefore wins over a brace that appears earlier, and the offset lands on the backtick — the payload is never considered.2. End offset takes the last closing character.
rfind(end_marker, start_index)scans to the end of the string, so two payloads (or a stray}in trailing prose) make the slice run past the first value andjson.loadsfail on the extra data:Fix
_payload_startprefers braces over backticks (fences keep priority, so a fenced block still beats prose braces before it), and skips whitespace after a fence marker to land on the payload itself.json.JSONDecoder().raw_decode, which parses exactly one value and stops: trailing prose, a second payload, or a}inside a string can no longer extend the slice.No change to the error contract: a missing payload still raises
ValueError("could not find json block in the output."), and malformed JSON still raisesJSONDecodeError, soparse_and_check_json_markdownkeeps wrapping it intoOutputParserError.Scope
'[{"a": 1}]'still returns{"a": 1}— the brace is found before the bracket, andtest_parse_json_markdown_extracts_raw_json_arrayasserts exactly that. Whether a bare top-level array should be returned as a list is a separate contract question; this PR does not touch it.Tests
Five cases added to
tests/utils/test_json_in_md_parser.py:..._ignores_a_backtick_after_the_payload'{"a": 1} andcode'→{"a": 1}..._ignores_a_lone_backtick_after_the_payloadValueError, now parses..._ignores_an_inline_code_span_before_the_payload'Usecodethen {"a": 1}'→{"a": 1}..._returns_the_first_of_two_payloads..._keeps_braces_inside_strings'{"a": "```"}'→{"a": "```"}All seven pre-existing tests are unchanged and still pass.
Verification
The module is stdlib-only (
json), so I loaded it by path and ran the seven existing assertions plus the new cases as a standalone harness: 15/15 pass. I did not run the repo's pytest suite locally — importinggraphon.utilspulls the package__init__, which needs the full dependency set (transformers,pandas,unstructured) that is not installed here — so CI will be the first pytest run. The assertions themselves are reproduced one-for-one in the harness, including theOutputParserErrorwrapping cases.