From 9e1a09eec0242380339b43691fa6f9174caf8ada Mon Sep 17 00:00:00 2001 From: Niels Lohmann Date: Mon, 28 Sep 2026 17:51:07 +0200 Subject: [PATCH] Name the key type when rejecting non-string CBOR/MessagePack map keys (#5594) * Name the key type when rejecting non-string CBOR/MessagePack map keys CBOR and MessagePack allow map keys of any type, but JSON object keys are always strings, so such maps are rejected. The error so far was the one for a malformed string (e.g. "expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xC0" for a nil key), which does not tell the user what went wrong. Report the type of the key instead: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0 The exception id (parse_error.113) and type are unchanged. Malformed string keys and a missing key keep their previous messages. Document the restriction on the CBOR and MessagePack pages. Refs #2766, #3381 Signed-off-by: Niels Lohmann * Point the MessagePack key note to the spec's profile section The note linked to "Serialization: type to format conversion", which says nothing about key types. Restricting map keys to strings is only mentioned in the "Profile" section (under "Future discussion") as an example of a JSON-compatible profile, so link there and describe it as such instead of as a permission. Signed-off-by: Niels Lohmann --------- Signed-off-by: Niels Lohmann --- docs/mkdocs/docs/api/basic_json/from_cbor.md | 4 +- .../docs/api/basic_json/from_msgpack.md | 4 +- .../docs/features/binary_formats/cbor.md | 15 +- .../features/binary_formats/messagepack.md | 15 ++ docs/mkdocs/docs/home/exceptions.md | 11 +- .../nlohmann/detail/input/binary_reader.hpp | 170 +++++++++++++++++- single_include/nlohmann/json.hpp | 170 +++++++++++++++++- tests/src/unit-cbor.cpp | 45 ++++- tests/src/unit-msgpack.cpp | 61 ++++++- tests/src/unit-regression1.cpp | 6 +- 10 files changed, 484 insertions(+), 17 deletions(-) diff --git a/docs/mkdocs/docs/api/basic_json/from_cbor.md b/docs/mkdocs/docs/api/basic_json/from_cbor.md index b72f55280..8c1062da8 100644 --- a/docs/mkdocs/docs/api/basic_json/from_cbor.md +++ b/docs/mkdocs/docs/api/basic_json/from_cbor.md @@ -80,8 +80,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va the end of the file was not reached when `strict` was set to true - Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if unsupported features from CBOR were used in the given input or if the input is not valid CBOR -- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a string was expected as a map key, - but not found +- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a map key is not a string (keys of other + types are not supported, as JSON object keys are always strings) or a string is malformed ## Complexity diff --git a/docs/mkdocs/docs/api/basic_json/from_msgpack.md b/docs/mkdocs/docs/api/basic_json/from_msgpack.md index 2f4b7bb3b..e41edfe7e 100644 --- a/docs/mkdocs/docs/api/basic_json/from_msgpack.md +++ b/docs/mkdocs/docs/api/basic_json/from_msgpack.md @@ -73,8 +73,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va the end of the file was not reached when `strict` was set to true - Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if unsupported features from MessagePack were used in the given input or if the input is not valid MessagePack -- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a string was expected as a map key, - but not found +- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a map key is not a string (keys of other + types are not supported, as JSON object keys are always strings) or a string is malformed ## Complexity diff --git a/docs/mkdocs/docs/features/binary_formats/cbor.md b/docs/mkdocs/docs/features/binary_formats/cbor.md index 8e6acf0fb..a488466d4 100644 --- a/docs/mkdocs/docs/features/binary_formats/cbor.md +++ b/docs/mkdocs/docs/features/binary_formats/cbor.md @@ -174,7 +174,20 @@ The library maps CBOR types to JSON value types as follows: !!! warning "Object keys" - CBOR allows map keys of any type, whereas JSON only allows strings as keys in object values. Therefore, CBOR maps with keys other than UTF-8 strings are rejected. + CBOR allows map keys of any type, whereas JSON only allows strings as keys in object values. Therefore, CBOR maps + with keys other than text strings (major type 3) are rejected with a + [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with `allow_exceptions` set + to `false`, a discarded value) naming the type of the key that was found, for instance: + + ``` + [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an unsigned integer; last byte: 0x01 + ``` + + This applies to the [SAX interface](../parsing/sax_interface.md) as well, as the key is read before it is passed + on. This is a deliberate restriction of the library's JSON value model, not an oversight: formats built on CBOR + maps with integer keys, such as COSE ([RFC 9052](https://www.rfc-editor.org/rfc/rfc9052.html)) or CWT + ([RFC 8392](https://www.rfc-editor.org/rfc/rfc8392.html)), cannot be read with this library and need a + general-purpose CBOR library instead. !!! warning "UTF-8 validation of text strings" diff --git a/docs/mkdocs/docs/features/binary_formats/messagepack.md b/docs/mkdocs/docs/features/binary_formats/messagepack.md index 0ca82c145..3ce5f7620 100644 --- a/docs/mkdocs/docs/features/binary_formats/messagepack.md +++ b/docs/mkdocs/docs/features/binary_formats/messagepack.md @@ -138,6 +138,21 @@ The library maps MessagePack types to JSON value types as follows: Any MessagePack output created by `to_msgpack` can be successfully parsed by `from_msgpack`. +!!! warning "Object keys" + + MessagePack allows map keys of any type, whereas JSON only allows strings as keys in object values. Like the + JSON-compatible [profile](https://github.com/msgpack/msgpack/blob/master/spec.md#profile) sketched in the + MessagePack specification, this library restricts map keys to `str` values. Maps with keys of any other type are + rejected with a [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with + `allow_exceptions` set to `false`, a discarded value) naming the type of the key that was found, for instance: + + ``` + [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0 + ``` + + This applies to the [SAX interface](../parsing/sax_interface.md) as well, as the key is read before it is passed + on. Such input needs a general-purpose MessagePack library instead. + !!! warning "UTF-8 validation of string values" The MessagePack specification requires `str` values (`fixstr`, `str 8`, `str 16`, `str 32`) to be valid UTF-8. diff --git a/docs/mkdocs/docs/home/exceptions.md b/docs/mkdocs/docs/home/exceptions.md index bf18baab1..407f3c3f1 100644 --- a/docs/mkdocs/docs/home/exceptions.md +++ b/docs/mkdocs/docs/home/exceptions.md @@ -343,13 +343,20 @@ A string could not be read from a [binary format](../features/binary_formats/ind string was read where one was required (for instance as a map key), the string's length specification is invalid, or the string's bytes are not valid UTF-8. +CBOR and MessagePack allow map keys of any type, but JSON object keys are always strings. Maps with keys of any other +type (for instance integers or `null`) are therefore not supported; see the notes on +[CBOR](../features/binary_formats/cbor.md) and [MessagePack](../features/binary_formats/messagepack.md). + !!! failure "Example messages" ``` - [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF + [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an unsigned integer; last byte: 0x01 ``` ``` - [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xFF + [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0 + ``` + ``` + [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x7C ``` ``` [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON char: byte after 'C' must be in range 0x00..0x7F; last byte: 0x82 diff --git a/include/nlohmann/detail/input/binary_reader.hpp b/include/nlohmann/detail/input/binary_reader.hpp index b0675c626..b9e6b304b 100644 --- a/include/nlohmann/detail/input/binary_reader.hpp +++ b/include/nlohmann/detail/input/binary_reader.hpp @@ -1324,6 +1324,80 @@ class binary_reader } } + /*! + @brief reads a CBOR object key + + RFC 8949 allows any data item as a map key, but only strings have a + counterpart in JSON. A key of any other type is rejected with a message + naming that type, rather than the one @ref get_cbor_string gives for a + malformed string. + + @param[out] result created key + + @return whether key creation completed + */ + bool get_cbor_object_key(string_t& result) + { + // EOF and major type 3 (text string) are left to get_cbor_string + if (current == char_traits::eof() || (static_cast(current) & 0xE0u) == 0x60u) + { + return get_cbor_string(result); + } + + const char* found = nullptr; + switch (static_cast(current) >> 5u) + { + case 0: + found = "an unsigned integer"; + break; + case 1: + found = "a negative integer"; + break; + case 2: + found = "a byte string"; + break; + case 4: + found = "an array"; + break; + case 5: + found = "a map"; + break; + case 6: + found = "a tag"; + break; + default: // major type 7 + switch (current) + { + case 0xF4: + case 0xF5: + found = "a boolean"; + break; + case 0xF6: + found = "null"; + break; + case 0xF7: + found = "undefined"; + break; + case 0xF9: + case 0xFA: + case 0xFB: + found = "a floating-point number"; + break; + case 0xFF: + found = "a break stop code"; + break; + default: + found = "a simple value"; + break; + } + break; + } + + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, + exception_message(input_format_t::cbor, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); + } + /*! @brief reads a definite-length CBOR byte array @@ -1568,7 +1642,7 @@ class binary_reader if (top.is_object) { key.clear(); - if (JSON_HEDLEY_UNLIKELY(!get_cbor_string(key) || !sax->key(key))) + if (JSON_HEDLEY_UNLIKELY(!get_cbor_object_key(key) || !sax->key(key))) { return false; } @@ -2069,6 +2143,98 @@ class binary_reader } } + /*! + @brief reads a MessagePack object key + + The MessagePack specification allows any type as a map key, but only + strings have a counterpart in JSON. A key of any other type is rejected + with a message naming that type, rather than the one @ref + get_msgpack_string gives for a malformed string. + + @param[out] result created key + + @return whether key creation completed + */ + bool get_msgpack_object_key(string_t& result) + { + const char* found = nullptr; + switch (current) + { + case 0xC0: + found = "nil"; + break; + case 0xC2: + case 0xC3: + found = "a boolean"; + break; + case 0xCA: + case 0xCB: + found = "a float"; + break; + case 0xC4: + case 0xC5: + case 0xC6: + found = "a bin"; + break; + case 0xC7: + case 0xC8: + case 0xC9: + case 0xD4: + case 0xD5: + case 0xD6: + case 0xD7: + case 0xD8: + found = "an ext"; + break; + case 0xCC: + case 0xCD: + case 0xCE: + case 0xCF: + case 0xD0: + case 0xD1: + case 0xD2: + case 0xD3: + found = "an integer"; + break; + case 0xDC: + case 0xDD: + found = "an array"; + break; + case 0xDE: + case 0xDF: + found = "a map"; + break; + default: + // fixint, fixmap, and fixarray; strings, EOF, and the unused + // byte 0xC1 are left to get_msgpack_string + if (current == char_traits::eof()) + { + return get_msgpack_string(result); + } + if (current <= 0x7F || current >= 0xE0) + { + found = "an integer"; + } + else if (current <= 0x8F) + { + found = "a map"; + } + else if (current <= 0x9F) + { + found = "an array"; + } + else + { + return get_msgpack_string(result); + } + break; + } + + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, + exception_message(input_format_t::msgpack, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); + } + /*! @brief reads a MessagePack byte array @@ -2231,7 +2397,7 @@ class binary_reader { get(); key.clear(); - if (JSON_HEDLEY_UNLIKELY(!get_msgpack_string(key) || !sax->key(key))) + if (JSON_HEDLEY_UNLIKELY(!get_msgpack_object_key(key) || !sax->key(key))) { return false; } diff --git a/single_include/nlohmann/json.hpp b/single_include/nlohmann/json.hpp index 0e3cae486..591a00b74 100644 --- a/single_include/nlohmann/json.hpp +++ b/single_include/nlohmann/json.hpp @@ -14059,6 +14059,80 @@ class binary_reader } } + /*! + @brief reads a CBOR object key + + RFC 8949 allows any data item as a map key, but only strings have a + counterpart in JSON. A key of any other type is rejected with a message + naming that type, rather than the one @ref get_cbor_string gives for a + malformed string. + + @param[out] result created key + + @return whether key creation completed + */ + bool get_cbor_object_key(string_t& result) + { + // EOF and major type 3 (text string) are left to get_cbor_string + if (current == char_traits::eof() || (static_cast(current) & 0xE0u) == 0x60u) + { + return get_cbor_string(result); + } + + const char* found = nullptr; + switch (static_cast(current) >> 5u) + { + case 0: + found = "an unsigned integer"; + break; + case 1: + found = "a negative integer"; + break; + case 2: + found = "a byte string"; + break; + case 4: + found = "an array"; + break; + case 5: + found = "a map"; + break; + case 6: + found = "a tag"; + break; + default: // major type 7 + switch (current) + { + case 0xF4: + case 0xF5: + found = "a boolean"; + break; + case 0xF6: + found = "null"; + break; + case 0xF7: + found = "undefined"; + break; + case 0xF9: + case 0xFA: + case 0xFB: + found = "a floating-point number"; + break; + case 0xFF: + found = "a break stop code"; + break; + default: + found = "a simple value"; + break; + } + break; + } + + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, + exception_message(input_format_t::cbor, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); + } + /*! @brief reads a definite-length CBOR byte array @@ -14303,7 +14377,7 @@ class binary_reader if (top.is_object) { key.clear(); - if (JSON_HEDLEY_UNLIKELY(!get_cbor_string(key) || !sax->key(key))) + if (JSON_HEDLEY_UNLIKELY(!get_cbor_object_key(key) || !sax->key(key))) { return false; } @@ -14804,6 +14878,98 @@ class binary_reader } } + /*! + @brief reads a MessagePack object key + + The MessagePack specification allows any type as a map key, but only + strings have a counterpart in JSON. A key of any other type is rejected + with a message naming that type, rather than the one @ref + get_msgpack_string gives for a malformed string. + + @param[out] result created key + + @return whether key creation completed + */ + bool get_msgpack_object_key(string_t& result) + { + const char* found = nullptr; + switch (current) + { + case 0xC0: + found = "nil"; + break; + case 0xC2: + case 0xC3: + found = "a boolean"; + break; + case 0xCA: + case 0xCB: + found = "a float"; + break; + case 0xC4: + case 0xC5: + case 0xC6: + found = "a bin"; + break; + case 0xC7: + case 0xC8: + case 0xC9: + case 0xD4: + case 0xD5: + case 0xD6: + case 0xD7: + case 0xD8: + found = "an ext"; + break; + case 0xCC: + case 0xCD: + case 0xCE: + case 0xCF: + case 0xD0: + case 0xD1: + case 0xD2: + case 0xD3: + found = "an integer"; + break; + case 0xDC: + case 0xDD: + found = "an array"; + break; + case 0xDE: + case 0xDF: + found = "a map"; + break; + default: + // fixint, fixmap, and fixarray; strings, EOF, and the unused + // byte 0xC1 are left to get_msgpack_string + if (current == char_traits::eof()) + { + return get_msgpack_string(result); + } + if (current <= 0x7F || current >= 0xE0) + { + found = "an integer"; + } + else if (current <= 0x8F) + { + found = "a map"; + } + else if (current <= 0x9F) + { + found = "an array"; + } + else + { + return get_msgpack_string(result); + } + break; + } + + auto last_token = get_token_string(); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, + exception_message(input_format_t::msgpack, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); + } + /*! @brief reads a MessagePack byte array @@ -14966,7 +15132,7 @@ class binary_reader { get(); key.clear(); - if (JSON_HEDLEY_UNLIKELY(!get_msgpack_string(key) || !sax->key(key))) + if (JSON_HEDLEY_UNLIKELY(!get_msgpack_object_key(key) || !sax->key(key))) { return false; } diff --git a/tests/src/unit-cbor.cpp b/tests/src/unit-cbor.cpp index 6bd792f8a..fe0fb2644 100644 --- a/tests/src/unit-cbor.cpp +++ b/tests/src/unit-cbor.cpp @@ -1830,10 +1830,51 @@ TEST_CASE("CBOR") SECTION("invalid string in map") { json _; - CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector({0xa1, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF", json::parse_error&); + CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector({0xa1, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found a break stop code; last byte: 0xFF", json::parse_error&); CHECK(json::from_cbor(std::vector({0xa1, 0xff, 0x01}), true, false).is_discarded()); } + SECTION("non-string key (see #2766 and #3381)") + { + // only text strings map to JSON object keys; any other key is + // rejected with a message naming its type + const std::vector, std::string>> cases = + { + {{0xA1, 0x01, 0x01}, "an unsigned integer; last byte: 0x01"}, + {{0xA1, 0x20, 0x01}, "a negative integer; last byte: 0x20"}, + {{0xA1, 0x41, 0x61, 0x01}, "a byte string; last byte: 0x41"}, + {{0xA1, 0x80, 0x01}, "an array; last byte: 0x80"}, + {{0xA1, 0xA0, 0x01}, "a map; last byte: 0xA0"}, + {{0xA1, 0xC0, 0x61, 0x61, 0x01}, "a tag; last byte: 0xC0"}, + {{0xA1, 0xF4, 0x01}, "a boolean; last byte: 0xF4"}, + {{0xA1, 0xF5, 0x01}, "a boolean; last byte: 0xF5"}, + {{0xA1, 0xF6, 0x01}, "null; last byte: 0xF6"}, + {{0xA1, 0xF7, 0x01}, "undefined; last byte: 0xF7"}, + {{0xA1, 0xF9, 0x3C, 0x00, 0x01}, "a floating-point number; last byte: 0xF9"}, + {{0xA1, 0xFA, 0x3F, 0x80, 0x00, 0x00, 0x01}, "a floating-point number; last byte: 0xFA"}, + {{0xA1, 0xFB, 0x3F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "a floating-point number; last byte: 0xFB"}, + {{0xA1, 0xE0, 0x01}, "a simple value; last byte: 0xE0"}, + {{0xA1, 0xF8, 0x20, 0x01}, "a simple value; last byte: 0xF8"}, + // indefinite-length map + {{0xBF, 0x01, 0x01, 0xFF}, "an unsigned integer; last byte: 0x01"}, + }; + + for (const auto& c : cases) + { + CAPTURE(c.first) + const std::string expected = "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found " + c.second; + json _; + CHECK_THROWS_WITH_AS(_ = json::from_cbor(c.first), expected.c_str(), json::parse_error&); + CHECK(json::from_cbor(c.first, true, false).is_discarded()); + } + + // a key of major type 3 with a reserved length is still reported as + // a malformed string, and a missing key as the end of input + json _; + CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector({0xA1})), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing CBOR string: unexpected end of input", json::parse_error&); + CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector({0xA1, 0x7C, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x7C", json::parse_error&); + } + SECTION("invalid UTF-8 in string (see #5529)") { // a two-character text string (major type 3) whose bytes are not @@ -2284,7 +2325,7 @@ TEST_CASE("CBOR indefinite-length strings do not recurse per chunk") SECTION("a break marker outside an indefinite-length string is not a string") { // 0xFF only closes a string that was opened; on its own it is not one - CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector({0xA1, 0xFF, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF", json::parse_error&); + CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector({0xA1, 0xFF, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found a break stop code; last byte: 0xFF", json::parse_error&); } } diff --git a/tests/src/unit-msgpack.cpp b/tests/src/unit-msgpack.cpp index de4255b4a..498dec859 100644 --- a/tests/src/unit-msgpack.cpp +++ b/tests/src/unit-msgpack.cpp @@ -1551,10 +1551,69 @@ TEST_CASE("MessagePack") SECTION("invalid string in map") { json _; - CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector({0x81, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xFF", json::parse_error&); + CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector({0x81, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found an integer; last byte: 0xFF", json::parse_error&); CHECK(json::from_msgpack(std::vector({0x81, 0xff, 0x01}), true, false).is_discarded()); } + SECTION("non-string key (see #3381)") + { + // only strings map to JSON object keys; any other key is rejected + // with a message naming its type + const std::vector, std::string>> cases = + { + {{0x81, 0xC0, 0x01}, "nil; last byte: 0xC0"}, + {{0x81, 0xC2, 0x01}, "a boolean; last byte: 0xC2"}, + {{0x81, 0xC3, 0x01}, "a boolean; last byte: 0xC3"}, + {{0x81, 0xCA, 0x3F, 0x80, 0x00, 0x00, 0x01}, "a float; last byte: 0xCA"}, + {{0x81, 0xCB, 0x3F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "a float; last byte: 0xCB"}, + {{0x81, 0xC4, 0x00, 0x01}, "a bin; last byte: 0xC4"}, + {{0x81, 0xC5, 0x00, 0x00, 0x01}, "a bin; last byte: 0xC5"}, + {{0x81, 0xC6, 0x00, 0x00, 0x00, 0x00, 0x01}, "a bin; last byte: 0xC6"}, + {{0x81, 0xC7, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC7"}, + {{0x81, 0xC8, 0x00, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC8"}, + {{0x81, 0xC9, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC9"}, + {{0x81, 0xD4, 0x01, 0x00, 0x01}, "an ext; last byte: 0xD4"}, + {{0x81, 0xD5, 0x01, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD5"}, + {{0x81, 0xD6, 0x01, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD6"}, + {{0x81, 0xD7, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD7"}, + {{0x81, 0xD8, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD8"}, + {{0x81, 0xCC, 0x01, 0x01}, "an integer; last byte: 0xCC"}, + {{0x81, 0xCD, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCD"}, + {{0x81, 0xCE, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCE"}, + {{0x81, 0xCF, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCF"}, + {{0x81, 0xD0, 0x01, 0x01}, "an integer; last byte: 0xD0"}, + {{0x81, 0xD1, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD1"}, + {{0x81, 0xD2, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD2"}, + {{0x81, 0xD3, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD3"}, + {{0x81, 0x00, 0x01}, "an integer; last byte: 0x00"}, + {{0x81, 0x7F, 0x01}, "an integer; last byte: 0x7F"}, + {{0x81, 0xE0, 0x01}, "an integer; last byte: 0xE0"}, + {{0x81, 0x80, 0x01}, "a map; last byte: 0x80"}, + {{0x81, 0x8F, 0x01}, "a map; last byte: 0x8F"}, + {{0x81, 0xDE, 0x00, 0x00, 0x01}, "a map; last byte: 0xDE"}, + {{0x81, 0xDF, 0x00, 0x00, 0x00, 0x00, 0x01}, "a map; last byte: 0xDF"}, + {{0x81, 0x90, 0x01}, "an array; last byte: 0x90"}, + {{0x81, 0x9F, 0x01}, "an array; last byte: 0x9F"}, + {{0x81, 0xDC, 0x00, 0x00, 0x01}, "an array; last byte: 0xDC"}, + {{0x81, 0xDD, 0x00, 0x00, 0x00, 0x00, 0x01}, "an array; last byte: 0xDD"}, + }; + + for (const auto& c : cases) + { + CAPTURE(c.first) + const std::string expected = "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found " + c.second; + json _; + CHECK_THROWS_WITH_AS(_ = json::from_msgpack(c.first), expected.c_str(), json::parse_error&); + CHECK(json::from_msgpack(c.first, true, false).is_discarded()); + } + + json _; + // the unused byte 0xC1 is still reported as a malformed string + CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector({0x81, 0xC1, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xC1", json::parse_error&); + // a missing key is still reported as the end of input + CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector({0x81})), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing MessagePack string: unexpected end of input", json::parse_error&); + } + SECTION("invalid UTF-8 in string (see #5529)") { // a fixstr of length 2 (0xA0 | 2) whose bytes are not valid UTF-8 diff --git a/tests/src/unit-regression1.cpp b/tests/src/unit-regression1.cpp index 0529f83dd..43cd18438 100644 --- a/tests/src/unit-regression1.cpp +++ b/tests/src/unit-regression1.cpp @@ -1018,7 +1018,7 @@ TEST_CASE("regression tests 1") }; json _; - CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x98", json::parse_error&); + CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an array; last byte: 0x98", json::parse_error&); // related test case: nonempty UTF-8 string (indefinite length) std::vector const vec1 {0x7f, 0x61, 0x61}; @@ -1065,7 +1065,7 @@ TEST_CASE("regression tests 1") }; json _; - CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec1), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xB4", json::parse_error&); + CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec1), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR object key: only string keys are supported, but found a map; last byte: 0xB4", json::parse_error&); // related test case: double-precision std::vector const vec2 @@ -1077,7 +1077,7 @@ TEST_CASE("regression tests 1") 0x96, 0x96, 0xb4, 0xb4, 0xfa, 0x94, 0x94, 0x61, 0x61, 0x61, 0x61, 0x61, 0x61, 0x61, 0x61, 0xfb }; - CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec2), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xB4", json::parse_error&); + CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec2), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR object key: only string keys are supported, but found a map; last byte: 0xB4", json::parse_error&); } SECTION("issue #452 - Heap-buffer-overflow (OSS-Fuzz issue 585)")