starred/json
1
0
Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-01 14:15:50 +01:00
Code Issues Packages Projects Releases 10 Wiki Activity

Validate UTF-8 in CBOR/MessagePack/BSON/UBJSON/BJData text strings on develop

Browse Source 
PR #5531 fixed the UTF-8-validation gap described in #5529, but it was
merged onto the still-unmerged bson-sizes branch rather than develop, so
develop was left with the original bug for all affected formats. A
follow-up comment on #5529 reproduced this on develop and additionally
found that UBJSON (and, by the same code path, BJData) has the identical
gap, undocumented.

Port the same fix directly onto develop: extract the UTF-8 DFA decoder
out of serializer<>::decode() into a shared detail::decode()/is_valid_utf8()
in string_utils.hpp, and call it from binary_reader::get_string() - the
single choke point shared by all five binary readers - so malformed text
strings are rejected at decode time (parse_error.113) instead of only
failing later on dump() (type_error.316). Byte/binary payloads are
unaffected. Add matching decode-time tests for CBOR, MessagePack, BSON,
UBJSON, and BJData, and document the new behavior on all five binary
format pages (the two UBJSON/BJData pages didn't get this note in #5531).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017sdieJCn6BHxzRMaXP49sP
This commit is contained in:
Claude
2026-09-16 05:20:00 +00:00
parent c72f37a40d
commit 5e5a087cb3
15 changed files with 448 additions and 180 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/src/unit-ubjson.cpp
+17
View File
@@ -1927,6 +1927,23 @@ TEST_CASE("UBJSON")
std::vector<uint8_t> const v0 = {'S', 'i', 0};
CHECK(json::from_ubjson(v0) == json(""));
}
SECTION("invalid UTF-8 in string (see #5529)")
{
// a UBJSON string of length 2 whose bytes are not valid
// UTF-8 (0xC0 0xAE is an overlong encoding of '.') must be
// rejected at decode time, matching every other kind of
// malformed binary input, rather than only failing later
// when the resulting value is dumped
std::vector<uint8_t> const v = {'S', 'i', 0x02, 0xc0, 0xae};
json _;
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v), "[json.exception.parse_error.113] parse error at byte 5: syntax error while parsing UBJSON string: invalid string: ill-formed UTF-8 byte", json::parse_error&);
CHECK(json::from_ubjson(v, true, false).is_discarded());
// valid UTF-8 must still round-trip
const json j = "h\xc3\xa9llo, w\xc3\xb6rld! \xe6\x97\xa5\xe6\x9c\xac\xe8\xaa\x9e"; // héllo, wörld! 日本語
CHECK(json::from_ubjson(json::to_ubjson(j)) == j);
}
}
SECTION("array")