When the SAX parser asks to recover, the binary readers now repair an
item whose end is known and read on after it, as RFC 8949, Section 5.3
describes for CBOR:
- CBOR: tags are ignored, and simple values other than false, true, and
null become null (RFC 8949, Section 6.1); a negative integer below the
range of number_integer_t becomes the nearest floating-point number.
- Strings that are not valid UTF-8 get U+FFFD for each ill-formed
sequence, as in JSON text; so does a UBJSON/BJData char above 0x7F.
- UBJSON/BJData high-precision numbers keep their longest valid
beginning (via the lexer's recover_token()), or become infinity.
- Members whose key is not a string are skipped (CBOR, MessagePack,
BON8), like members without a key in JSON text.
- BSON elements of types the library does not read (ObjectId, datetime,
decimal128, ...) become null; a string without its terminator and a
document whose size does not match are kept.
Where the end of an item is unknown, reading stops as before, except
that BSON skips to the end of the document, whose size it knows.
The value read before such an error is now completed by the reader from
its container stack, as the JSON parser does, instead of by a proxy SAX
parser, which is removed. Like the parser, binary_reader gets an
AllowRecovery template parameter, so that from_*() compile without the
new code.
Tests: a table of repairs, numbers out of range, errors that stop, and
a sweep over changed and removed bytes of eight encodings that checks
balanced events and that the first error is the one from_*() reports.
All fuzzers now run a recovering checker; the binary ones also check
that it reports an error exactly when from_*() fails.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The fuzzer drivers check their round trips with assert(), which NDEBUG
compiles away. The OSS-Fuzz build keeps assertions on today, but nothing
pins that: a build change that adds NDEBUG would silently turn every
round-trip check into a mere "does not crash" check. Each driver now
stops the build with an #error instead, and includes <cassert> itself
rather than relying on json.hpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Niels Lohmann
