The return value of json_sax::parse_error() was documented both as "must return false" and as "whether the parsing should continue", and the code just passed it on. For JSON text, parsing stopped anyway, but sax_parse() could report success for invalid input. The binary readers read on after the error, looping forever on a CBOR indefinite-length array without its end. Now false stops parsing, and true recovers from the error: - JSON text is repaired with the smallest local edit (insert a missing ',' or ':', remove a stray token, keep the readable part of a broken string or number, null for a value that cannot be read, close the innermost container at a wrong closing bracket and all of them at the end of the input), and parsing continues. The SAX events stay balanced, every key is followed by exactly one value, and each token is reported at most once. - The binary formats cannot resynchronize, so they stop, but complete the value read so far. sax_parse() returns false after any error. parse(), accept(), and the from_*() functions never recover and compile to the same code as before. Supersedes #4522. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
7.0 KiB
Error Recovery
By default, parsing stops at the first error. With the SAX interface, you can instead ask the parser to recover: to repair the error and continue, so that you get as much as possible out of malformed input, for instance a file that was cut off, JSON edited by hand, or the output of a language model.
Recovering from errors
The SAX parser's parse_error function is called for every error. Its return value
decides what happens next:
#!cpp falsestops parsing. This is what the SAX parsers of the library do, soparseandacceptnever recover.#!cpp truerepairs the error and continues parsing.
When recovering, the SAX parser still receives well-formed events: every start_object or start_array is followed by
the matching end_object or end_array, and every key is followed by exactly one value. A SAX parser that creates a
JSON value, such as the one in the example below, therefore gets a complete value. Parsing always ends, and
sax_parse returns #!cpp false for input that is not valid JSON, even if every
error was repaired. Each token is reported at most once, and the SAX parser can stop at any error by returning
#!cpp false.
!!! example
The example below derives a SAX parser from the library's parser for `json` values (`json_sax_dom_parser`),
and recovers from all errors.
```cpp
--8<-- "examples/sax_parse__error_recovery.cpp"
```
Output:
```
--8<-- "examples/sax_parse__error_recovery.output"
```
How errors are repaired
Each error is repaired with the smallest local edit: a missing separator is inserted, a stray token is removed, what can
be read of a broken string or number is kept, and a value that cannot be read at all becomes #!json null.
| Mistake | Repair | Example | Result |
|---|---|---|---|
missing , or : |
inserted | #!json [1 2], #!json {"a" 1} |
[1,2], {"a":1} |
| missing value | #!json null for an object key or between commas in an array |
#!json {"a":}, #!json [1,,2] |
{"a":null}, [1,null,2] |
| trailing comma | removed | #!json [1,2,] |
[1,2] |
| broken string | invalid escapes and bytes are replaced (see below); a line break ends the string | #!json ["a\qb"] |
["aqb"] |
| broken number | the longest valid beginning is kept | #!json [1., 2e+] |
[1,2] |
| unreadable value | #!json null |
#!json [1, NaN, tru] |
[1,null,null] |
| number too large | passed as infinity, together with its text | #!json [1e999] |
infinity (see below) |
stray : |
removed | #!json ["a":1] |
["a",1] |
| member without a key | skipped up to the next , or } |
#!json {1:2, "b":3} |
{"b":3} |
| wrong closing bracket | closes the innermost array or object | #!json {"a":[1,2}, "b":3} |
{"a":[1,2],"b":3} |
| input ends too early | all open arrays and objects are closed | #!json {"a":[1,2 |
{"a":[1,2]} |
| text before the value | skipped | #!json )]}'{"a":1} |
{"a":1} |
In a string, an unknown escape like \q stands for the escaped character (q), as in JavaScript. An invalid \u
escape, a lone surrogate, and ill-formed UTF-8 are each replaced by U+FFFD (REPLACEMENT CHARACTER), and control
characters are kept. A string without its closing quote ends at the next line break or at the end of the input.
The input after the top-level value is not repaired: as without recovery, it is reported as an error, and parsing stops.
Binary formats
The binary formats (BJData, BON8,
BSON, CBOR, MessagePack,
and UBJSON) cannot be repaired: a value's size is stored before its content, and every
byte is a valid type marker, so after an error there is no way to tell where the next value begins. Parsing therefore
always stops at the first error. If parse_error returns #!cpp true, the value read so far is completed before
parsing stops: a key that waits for its value gets #!json null, and all open arrays and objects are closed. This keeps
everything before the error of an input that was cut off.
Limitations
- A repair is a guess. For example,
#!json {"a" "b": 1}could be meant as#!json {"a": "b"}or as#!json {"a": null, "b": 1}; it is repaired to the former. Treat recovered values as a best effort, and check the reported errors. - A closing bracket always closes the innermost array or object. If a bracket is missing rather than wrong, the
repair differs from the intention:
#!json {"a": {"b": [1, 2}, "c": 3}is repaired to#!json {"a": {"b": [1, 2], "c": 3}}, although#!json {"a": {"b": [1, 2]}, "c": 3}may have been meant. - Keys without quotes, and strings in single quotes, are not supported; such members are skipped.
- A number that is too large for
number_float_tis passed as positive or negative infinity. The SAX parser'snumber_floatalso gets the number's text, but a JSON value cannot store it, anddumpserializes infinity as#!json null. - When parsing is not strict (see
sax_parse), a repair may read parts of the input after the value, for instance of the next value in a stream of concatenated values.
See also
- SAX interface - implement a custom SAX handler
parse_error- the SAX event for parse errorssax_parse- generate SAX events- parsing and exceptions - control error handling