diff --git a/docs/mkdocs/docs/api/basic_json/from_cbor.md b/docs/mkdocs/docs/api/basic_json/from_cbor.md index b72f55280..8c1062da8 100644 --- a/docs/mkdocs/docs/api/basic_json/from_cbor.md +++ b/docs/mkdocs/docs/api/basic_json/from_cbor.md @@ -80,8 +80,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va the end of the file was not reached when `strict` was set to true - Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if unsupported features from CBOR were used in the given input or if the input is not valid CBOR -- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a string was expected as a map key, - but not found +- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a map key is not a string (keys of other + types are not supported, as JSON object keys are always strings) or a string is malformed ## Complexity diff --git a/docs/mkdocs/docs/api/basic_json/from_msgpack.md b/docs/mkdocs/docs/api/basic_json/from_msgpack.md index 2f4b7bb3b..e41edfe7e 100644 --- a/docs/mkdocs/docs/api/basic_json/from_msgpack.md +++ b/docs/mkdocs/docs/api/basic_json/from_msgpack.md @@ -73,8 +73,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va the end of the file was not reached when `strict` was set to true - Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if unsupported features from MessagePack were used in the given input or if the input is not valid MessagePack -- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a string was expected as a map key, - but not found +- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a map key is not a string (keys of other + types are not supported, as JSON object keys are always strings) or a string is malformed ## Complexity diff --git a/docs/mkdocs/docs/features/binary_formats/cbor.md b/docs/mkdocs/docs/features/binary_formats/cbor.md index 8e6acf0fb..a488466d4 100644 --- a/docs/mkdocs/docs/features/binary_formats/cbor.md +++ b/docs/mkdocs/docs/features/binary_formats/cbor.md @@ -174,7 +174,20 @@ The library maps CBOR types to JSON value types as follows: !!! warning "Object keys" - CBOR allows map keys of any type, whereas JSON only allows strings as keys in object values. Therefore, CBOR maps with keys other than UTF-8 strings are rejected. + CBOR allows map keys of any type, whereas JSON only allows strings as keys in object values. Therefore, CBOR maps + with keys other than text strings (major type 3) are rejected with a + [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with `allow_exceptions` set + to `false`, a discarded value) naming the type of the key that was found, for instance: + + ``` + [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an unsigned integer; last byte: 0x01 + ``` + + This applies to the [SAX interface](../parsing/sax_interface.md) as well, as the key is read before it is passed + on. This is a deliberate restriction of the library's JSON value model, not an oversight: formats built on CBOR + maps with integer keys, such as COSE ([RFC 9052](https://www.rfc-editor.org/rfc/rfc9052.html)) or CWT + ([RFC 8392](https://www.rfc-editor.org/rfc/rfc8392.html)), cannot be read with this library and need a + general-purpose CBOR library instead. !!! warning "UTF-8 validation of text strings" diff --git a/docs/mkdocs/docs/features/binary_formats/messagepack.md b/docs/mkdocs/docs/features/binary_formats/messagepack.md index 0ca82c145..3ce5f7620 100644 --- a/docs/mkdocs/docs/features/binary_formats/messagepack.md +++ b/docs/mkdocs/docs/features/binary_formats/messagepack.md @@ -138,6 +138,21 @@ The library maps MessagePack types to JSON value types as follows: Any MessagePack output created by `to_msgpack` can be successfully parsed by `from_msgpack`. +!!! warning "Object keys" + + MessagePack allows map keys of any type, whereas JSON only allows strings as keys in object values. Like the + JSON-compatible [profile](https://github.com/msgpack/msgpack/blob/master/spec.md#profile) sketched in the + MessagePack specification, this library restricts map keys to `str` values. Maps with keys of any other type are + rejected with a [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with + `allow_exceptions` set to `false`, a discarded value) naming the type of the key that was found, for instance: + + ``` + [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0 + ``` + + This applies to the [SAX interface](../parsing/sax_interface.md) as well, as the key is read before it is passed + on. Such input needs a general-purpose MessagePack library instead. + !!! warning "UTF-8 validation of string values" The MessagePack specification requires `str` values (`fixstr`, `str 8`, `str 16`, `str 32`) to be valid UTF-8. diff --git a/docs/mkdocs/docs/home/exceptions.md b/docs/mkdocs/docs/home/exceptions.md index bf18baab1..407f3c3f1 100644 --- a/docs/mkdocs/docs/home/exceptions.md +++ b/docs/mkdocs/docs/home/exceptions.md @@ -343,13 +343,20 @@ A string could not be read from a [binary format](../features/binary_formats/ind string was read where one was required (for instance as a map key), the string's length specification is invalid, or the string's bytes are not valid UTF-8. +CBOR and MessagePack allow map keys of any type, but JSON object keys are always strings. Maps with keys of any other +type (for instance integers or `null`) are therefore not supported; see the notes on +[CBOR](../features/binary_formats/cbor.md) and [MessagePack](../features/binary_formats/messagepack.md). + !!! failure "Example messages" ``` - [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF + [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an unsigned integer; last byte: 0x01 ``` ``` - [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xFF + [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0 + ``` + ``` + [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x7C ``` ``` [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON char: byte after 'C' must be in range 0x00..0x7F; last byte: 0x82 diff --git a/include/nlohmann/detail/input/binary_reader.hpp b/include/nlohmann/detail/input/binary_reader.hpp index 16f687d28..711038c3a 100644 --- a/include/nlohmann/detail/input/binary_reader.hpp +++ b/include/nlohmann/detail/input/binary_reader.hpp @@ -1401,14 +1401,12 @@ class binary_reader into the same string. @param[out] result string the bytes are appended to - @param[in] whole_string whether the chunk is a string of its own rather - than a chunk of an indefinite-length string @return whether string creation completed @pre @a current is not EOF */ - bool get_cbor_string_chunk(string_t& result, const bool whole_string) + bool get_cbor_string_chunk(string_t& result) { switch (current) { @@ -1468,15 +1466,8 @@ class binary_reader default: { auto last_token = get_token_string(); - // a string value begins with one of the bytes above, so only an - // object key can begin with another one; if it begins an item, - // the member is skipped when recovering (see skip_member) - if (report_error_repairable_if(whole_string && is_cbor_item_head(current), chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::cbor, concat("expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x", last_token), "string"), nullptr))) - { - skip_requested = true; - } - return false; + return report_error(chars_read, last_token, parse_error::create(113, chars_read, + exception_message(input_format_t::cbor, concat("expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x", last_token), "string"), nullptr)); } } } @@ -1528,7 +1519,7 @@ class binary_reader continue; } - if (JSON_HEDLEY_UNLIKELY(!get_cbor_string_chunk(result, open == 0))) + if (JSON_HEDLEY_UNLIKELY(!get_cbor_string_chunk(result))) { return false; } @@ -1542,6 +1533,87 @@ class binary_reader } } + /*! + @brief reads a CBOR object key + + RFC 8949 allows any data item as a map key, but only strings have a + counterpart in JSON. A key of any other type is rejected with a message + naming that type, rather than the one @ref get_cbor_string gives for a + malformed string. When recovering, a key that is a complete item is + skipped with its value (see @ref skip_member). + + @param[out] result created key + + @return whether key creation completed + */ + bool get_cbor_object_key(string_t& result) + { + // EOF and major type 3 (text string) are left to get_cbor_string + if (current == char_traits::eof() || (static_cast(current) & 0xE0u) == 0x60u) + { + return get_cbor_string(result); + } + + const char* found = nullptr; + switch (static_cast(current) >> 5u) + { + case 0: + found = "an unsigned integer"; + break; + case 1: + found = "a negative integer"; + break; + case 2: + found = "a byte string"; + break; + case 4: + found = "an array"; + break; + case 5: + found = "a map"; + break; + case 6: + found = "a tag"; + break; + default: // major type 7 + switch (current) + { + case 0xF4: + case 0xF5: + found = "a boolean"; + break; + case 0xF6: + found = "null"; + break; + case 0xF7: + found = "undefined"; + break; + case 0xF9: + case 0xFA: + case 0xFB: + found = "a floating-point number"; + break; + case 0xFF: + found = "a break stop code"; + break; + default: + found = "a simple value"; + break; + } + break; + } + + // a break stop code or a reserved byte begins no item that could be + // skipped + auto last_token = get_token_string(); + if (report_error_repairable_if(is_cbor_item_head(current), chars_read, last_token, parse_error::create(113, chars_read, + exception_message(input_format_t::cbor, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr))) + { + skip_requested = true; + } + return false; + } + /*! @brief reads a definite-length CBOR byte array @@ -1786,7 +1858,7 @@ class binary_reader if (top.is_object) { key.clear(); - if (JSON_HEDLEY_UNLIKELY(!get_cbor_string(key))) + if (JSON_HEDLEY_UNLIKELY(!get_cbor_object_key(key))) { if (!skip_member(std::integral_constant {})) { @@ -2420,20 +2492,109 @@ class binary_reader default: { auto last_token = get_token_string(); - // a string value begins with one of the bytes above, so only an - // object key can begin with another one; unless it is the - // unused byte 0xC1, it begins an item, and the member is - // skipped when recovering (see skip_member) - if (report_error_repairable_if(current != 0xC1, chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::msgpack, concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", last_token), "string"), nullptr))) - { - skip_requested = true; - } - return false; + return report_error(chars_read, last_token, parse_error::create(113, chars_read, + exception_message(input_format_t::msgpack, concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", last_token), "string"), nullptr)); } } } + /*! + @brief reads a MessagePack object key + + The MessagePack specification allows any type as a map key, but only + strings have a counterpart in JSON. A key of any other type is rejected + with a message naming that type, rather than the one @ref + get_msgpack_string gives for a malformed string. When recovering, the key + is skipped with its value (see @ref skip_member). + + @param[out] result created key + + @return whether key creation completed + */ + bool get_msgpack_object_key(string_t& result) + { + const char* found = nullptr; + switch (current) + { + case 0xC0: + found = "nil"; + break; + case 0xC2: + case 0xC3: + found = "a boolean"; + break; + case 0xCA: + case 0xCB: + found = "a float"; + break; + case 0xC4: + case 0xC5: + case 0xC6: + found = "a bin"; + break; + case 0xC7: + case 0xC8: + case 0xC9: + case 0xD4: + case 0xD5: + case 0xD6: + case 0xD7: + case 0xD8: + found = "an ext"; + break; + case 0xCC: + case 0xCD: + case 0xCE: + case 0xCF: + case 0xD0: + case 0xD1: + case 0xD2: + case 0xD3: + found = "an integer"; + break; + case 0xDC: + case 0xDD: + found = "an array"; + break; + case 0xDE: + case 0xDF: + found = "a map"; + break; + default: + // fixint, fixmap, and fixarray; strings, EOF, and the unused + // byte 0xC1 are left to get_msgpack_string + if (current == char_traits::eof()) + { + return get_msgpack_string(result); + } + if (current <= 0x7F || current >= 0xE0) + { + found = "an integer"; + } + else if (current <= 0x8F) + { + found = "a map"; + } + else if (current <= 0x9F) + { + found = "an array"; + } + else + { + return get_msgpack_string(result); + } + break; + } + + auto last_token = get_token_string(); + if (report_repairable_error(chars_read, last_token, parse_error::create(113, chars_read, + exception_message(input_format_t::msgpack, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr))) + { + skip_requested = true; + } + return false; + } + /*! @brief reads a MessagePack byte array @@ -2596,7 +2757,7 @@ class binary_reader { get(); key.clear(); - if (JSON_HEDLEY_UNLIKELY(!get_msgpack_string(key))) + if (JSON_HEDLEY_UNLIKELY(!get_msgpack_object_key(key))) { if (!skip_member(std::integral_constant {})) { diff --git a/single_include/nlohmann/json.hpp b/single_include/nlohmann/json.hpp index 6f3fdbf9b..a90aa2562 100644 --- a/single_include/nlohmann/json.hpp +++ b/single_include/nlohmann/json.hpp @@ -14795,14 +14795,12 @@ class binary_reader into the same string. @param[out] result string the bytes are appended to - @param[in] whole_string whether the chunk is a string of its own rather - than a chunk of an indefinite-length string @return whether string creation completed @pre @a current is not EOF */ - bool get_cbor_string_chunk(string_t& result, const bool whole_string) + bool get_cbor_string_chunk(string_t& result) { switch (current) { @@ -14862,15 +14860,8 @@ class binary_reader default: { auto last_token = get_token_string(); - // a string value begins with one of the bytes above, so only an - // object key can begin with another one; if it begins an item, - // the member is skipped when recovering (see skip_member) - if (report_error_repairable_if(whole_string && is_cbor_item_head(current), chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::cbor, concat("expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x", last_token), "string"), nullptr))) - { - skip_requested = true; - } - return false; + return report_error(chars_read, last_token, parse_error::create(113, chars_read, + exception_message(input_format_t::cbor, concat("expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x", last_token), "string"), nullptr)); } } } @@ -14922,7 +14913,7 @@ class binary_reader continue; } - if (JSON_HEDLEY_UNLIKELY(!get_cbor_string_chunk(result, open == 0))) + if (JSON_HEDLEY_UNLIKELY(!get_cbor_string_chunk(result))) { return false; } @@ -14936,6 +14927,87 @@ class binary_reader } } + /*! + @brief reads a CBOR object key + + RFC 8949 allows any data item as a map key, but only strings have a + counterpart in JSON. A key of any other type is rejected with a message + naming that type, rather than the one @ref get_cbor_string gives for a + malformed string. When recovering, a key that is a complete item is + skipped with its value (see @ref skip_member). + + @param[out] result created key + + @return whether key creation completed + */ + bool get_cbor_object_key(string_t& result) + { + // EOF and major type 3 (text string) are left to get_cbor_string + if (current == char_traits::eof() || (static_cast(current) & 0xE0u) == 0x60u) + { + return get_cbor_string(result); + } + + const char* found = nullptr; + switch (static_cast(current) >> 5u) + { + case 0: + found = "an unsigned integer"; + break; + case 1: + found = "a negative integer"; + break; + case 2: + found = "a byte string"; + break; + case 4: + found = "an array"; + break; + case 5: + found = "a map"; + break; + case 6: + found = "a tag"; + break; + default: // major type 7 + switch (current) + { + case 0xF4: + case 0xF5: + found = "a boolean"; + break; + case 0xF6: + found = "null"; + break; + case 0xF7: + found = "undefined"; + break; + case 0xF9: + case 0xFA: + case 0xFB: + found = "a floating-point number"; + break; + case 0xFF: + found = "a break stop code"; + break; + default: + found = "a simple value"; + break; + } + break; + } + + // a break stop code or a reserved byte begins no item that could be + // skipped + auto last_token = get_token_string(); + if (report_error_repairable_if(is_cbor_item_head(current), chars_read, last_token, parse_error::create(113, chars_read, + exception_message(input_format_t::cbor, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr))) + { + skip_requested = true; + } + return false; + } + /*! @brief reads a definite-length CBOR byte array @@ -15180,7 +15252,7 @@ class binary_reader if (top.is_object) { key.clear(); - if (JSON_HEDLEY_UNLIKELY(!get_cbor_string(key))) + if (JSON_HEDLEY_UNLIKELY(!get_cbor_object_key(key))) { if (!skip_member(std::integral_constant {})) { @@ -15814,20 +15886,109 @@ class binary_reader default: { auto last_token = get_token_string(); - // a string value begins with one of the bytes above, so only an - // object key can begin with another one; unless it is the - // unused byte 0xC1, it begins an item, and the member is - // skipped when recovering (see skip_member) - if (report_error_repairable_if(current != 0xC1, chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::msgpack, concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", last_token), "string"), nullptr))) - { - skip_requested = true; - } - return false; + return report_error(chars_read, last_token, parse_error::create(113, chars_read, + exception_message(input_format_t::msgpack, concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", last_token), "string"), nullptr)); } } } + /*! + @brief reads a MessagePack object key + + The MessagePack specification allows any type as a map key, but only + strings have a counterpart in JSON. A key of any other type is rejected + with a message naming that type, rather than the one @ref + get_msgpack_string gives for a malformed string. When recovering, the key + is skipped with its value (see @ref skip_member). + + @param[out] result created key + + @return whether key creation completed + */ + bool get_msgpack_object_key(string_t& result) + { + const char* found = nullptr; + switch (current) + { + case 0xC0: + found = "nil"; + break; + case 0xC2: + case 0xC3: + found = "a boolean"; + break; + case 0xCA: + case 0xCB: + found = "a float"; + break; + case 0xC4: + case 0xC5: + case 0xC6: + found = "a bin"; + break; + case 0xC7: + case 0xC8: + case 0xC9: + case 0xD4: + case 0xD5: + case 0xD6: + case 0xD7: + case 0xD8: + found = "an ext"; + break; + case 0xCC: + case 0xCD: + case 0xCE: + case 0xCF: + case 0xD0: + case 0xD1: + case 0xD2: + case 0xD3: + found = "an integer"; + break; + case 0xDC: + case 0xDD: + found = "an array"; + break; + case 0xDE: + case 0xDF: + found = "a map"; + break; + default: + // fixint, fixmap, and fixarray; strings, EOF, and the unused + // byte 0xC1 are left to get_msgpack_string + if (current == char_traits::eof()) + { + return get_msgpack_string(result); + } + if (current <= 0x7F || current >= 0xE0) + { + found = "an integer"; + } + else if (current <= 0x8F) + { + found = "a map"; + } + else if (current <= 0x9F) + { + found = "an array"; + } + else + { + return get_msgpack_string(result); + } + break; + } + + auto last_token = get_token_string(); + if (report_repairable_error(chars_read, last_token, parse_error::create(113, chars_read, + exception_message(input_format_t::msgpack, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr))) + { + skip_requested = true; + } + return false; + } + /*! @brief reads a MessagePack byte array @@ -15990,7 +16151,7 @@ class binary_reader { get(); key.clear(); - if (JSON_HEDLEY_UNLIKELY(!get_msgpack_string(key))) + if (JSON_HEDLEY_UNLIKELY(!get_msgpack_object_key(key))) { if (!skip_member(std::integral_constant {})) { diff --git a/tests/src/unit-cbor.cpp b/tests/src/unit-cbor.cpp index 6bd792f8a..fe0fb2644 100644 --- a/tests/src/unit-cbor.cpp +++ b/tests/src/unit-cbor.cpp @@ -1830,10 +1830,51 @@ TEST_CASE("CBOR") SECTION("invalid string in map") { json _; - CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector({0xa1, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF", json::parse_error&); + CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector({0xa1, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found a break stop code; last byte: 0xFF", json::parse_error&); CHECK(json::from_cbor(std::vector({0xa1, 0xff, 0x01}), true, false).is_discarded()); } + SECTION("non-string key (see #2766 and #3381)") + { + // only text strings map to JSON object keys; any other key is + // rejected with a message naming its type + const std::vector, std::string>> cases = + { + {{0xA1, 0x01, 0x01}, "an unsigned integer; last byte: 0x01"}, + {{0xA1, 0x20, 0x01}, "a negative integer; last byte: 0x20"}, + {{0xA1, 0x41, 0x61, 0x01}, "a byte string; last byte: 0x41"}, + {{0xA1, 0x80, 0x01}, "an array; last byte: 0x80"}, + {{0xA1, 0xA0, 0x01}, "a map; last byte: 0xA0"}, + {{0xA1, 0xC0, 0x61, 0x61, 0x01}, "a tag; last byte: 0xC0"}, + {{0xA1, 0xF4, 0x01}, "a boolean; last byte: 0xF4"}, + {{0xA1, 0xF5, 0x01}, "a boolean; last byte: 0xF5"}, + {{0xA1, 0xF6, 0x01}, "null; last byte: 0xF6"}, + {{0xA1, 0xF7, 0x01}, "undefined; last byte: 0xF7"}, + {{0xA1, 0xF9, 0x3C, 0x00, 0x01}, "a floating-point number; last byte: 0xF9"}, + {{0xA1, 0xFA, 0x3F, 0x80, 0x00, 0x00, 0x01}, "a floating-point number; last byte: 0xFA"}, + {{0xA1, 0xFB, 0x3F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "a floating-point number; last byte: 0xFB"}, + {{0xA1, 0xE0, 0x01}, "a simple value; last byte: 0xE0"}, + {{0xA1, 0xF8, 0x20, 0x01}, "a simple value; last byte: 0xF8"}, + // indefinite-length map + {{0xBF, 0x01, 0x01, 0xFF}, "an unsigned integer; last byte: 0x01"}, + }; + + for (const auto& c : cases) + { + CAPTURE(c.first) + const std::string expected = "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found " + c.second; + json _; + CHECK_THROWS_WITH_AS(_ = json::from_cbor(c.first), expected.c_str(), json::parse_error&); + CHECK(json::from_cbor(c.first, true, false).is_discarded()); + } + + // a key of major type 3 with a reserved length is still reported as + // a malformed string, and a missing key as the end of input + json _; + CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector({0xA1})), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing CBOR string: unexpected end of input", json::parse_error&); + CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector({0xA1, 0x7C, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x7C", json::parse_error&); + } + SECTION("invalid UTF-8 in string (see #5529)") { // a two-character text string (major type 3) whose bytes are not @@ -2284,7 +2325,7 @@ TEST_CASE("CBOR indefinite-length strings do not recurse per chunk") SECTION("a break marker outside an indefinite-length string is not a string") { // 0xFF only closes a string that was opened; on its own it is not one - CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector({0xA1, 0xFF, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF", json::parse_error&); + CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector({0xA1, 0xFF, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found a break stop code; last byte: 0xFF", json::parse_error&); } } diff --git a/tests/src/unit-msgpack.cpp b/tests/src/unit-msgpack.cpp index de4255b4a..498dec859 100644 --- a/tests/src/unit-msgpack.cpp +++ b/tests/src/unit-msgpack.cpp @@ -1551,10 +1551,69 @@ TEST_CASE("MessagePack") SECTION("invalid string in map") { json _; - CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector({0x81, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xFF", json::parse_error&); + CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector({0x81, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found an integer; last byte: 0xFF", json::parse_error&); CHECK(json::from_msgpack(std::vector({0x81, 0xff, 0x01}), true, false).is_discarded()); } + SECTION("non-string key (see #3381)") + { + // only strings map to JSON object keys; any other key is rejected + // with a message naming its type + const std::vector, std::string>> cases = + { + {{0x81, 0xC0, 0x01}, "nil; last byte: 0xC0"}, + {{0x81, 0xC2, 0x01}, "a boolean; last byte: 0xC2"}, + {{0x81, 0xC3, 0x01}, "a boolean; last byte: 0xC3"}, + {{0x81, 0xCA, 0x3F, 0x80, 0x00, 0x00, 0x01}, "a float; last byte: 0xCA"}, + {{0x81, 0xCB, 0x3F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "a float; last byte: 0xCB"}, + {{0x81, 0xC4, 0x00, 0x01}, "a bin; last byte: 0xC4"}, + {{0x81, 0xC5, 0x00, 0x00, 0x01}, "a bin; last byte: 0xC5"}, + {{0x81, 0xC6, 0x00, 0x00, 0x00, 0x00, 0x01}, "a bin; last byte: 0xC6"}, + {{0x81, 0xC7, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC7"}, + {{0x81, 0xC8, 0x00, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC8"}, + {{0x81, 0xC9, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC9"}, + {{0x81, 0xD4, 0x01, 0x00, 0x01}, "an ext; last byte: 0xD4"}, + {{0x81, 0xD5, 0x01, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD5"}, + {{0x81, 0xD6, 0x01, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD6"}, + {{0x81, 0xD7, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD7"}, + {{0x81, 0xD8, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD8"}, + {{0x81, 0xCC, 0x01, 0x01}, "an integer; last byte: 0xCC"}, + {{0x81, 0xCD, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCD"}, + {{0x81, 0xCE, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCE"}, + {{0x81, 0xCF, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCF"}, + {{0x81, 0xD0, 0x01, 0x01}, "an integer; last byte: 0xD0"}, + {{0x81, 0xD1, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD1"}, + {{0x81, 0xD2, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD2"}, + {{0x81, 0xD3, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD3"}, + {{0x81, 0x00, 0x01}, "an integer; last byte: 0x00"}, + {{0x81, 0x7F, 0x01}, "an integer; last byte: 0x7F"}, + {{0x81, 0xE0, 0x01}, "an integer; last byte: 0xE0"}, + {{0x81, 0x80, 0x01}, "a map; last byte: 0x80"}, + {{0x81, 0x8F, 0x01}, "a map; last byte: 0x8F"}, + {{0x81, 0xDE, 0x00, 0x00, 0x01}, "a map; last byte: 0xDE"}, + {{0x81, 0xDF, 0x00, 0x00, 0x00, 0x00, 0x01}, "a map; last byte: 0xDF"}, + {{0x81, 0x90, 0x01}, "an array; last byte: 0x90"}, + {{0x81, 0x9F, 0x01}, "an array; last byte: 0x9F"}, + {{0x81, 0xDC, 0x00, 0x00, 0x01}, "an array; last byte: 0xDC"}, + {{0x81, 0xDD, 0x00, 0x00, 0x00, 0x00, 0x01}, "an array; last byte: 0xDD"}, + }; + + for (const auto& c : cases) + { + CAPTURE(c.first) + const std::string expected = "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found " + c.second; + json _; + CHECK_THROWS_WITH_AS(_ = json::from_msgpack(c.first), expected.c_str(), json::parse_error&); + CHECK(json::from_msgpack(c.first, true, false).is_discarded()); + } + + json _; + // the unused byte 0xC1 is still reported as a malformed string + CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector({0x81, 0xC1, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xC1", json::parse_error&); + // a missing key is still reported as the end of input + CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector({0x81})), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing MessagePack string: unexpected end of input", json::parse_error&); + } + SECTION("invalid UTF-8 in string (see #5529)") { // a fixstr of length 2 (0xA0 | 2) whose bytes are not valid UTF-8 diff --git a/tests/src/unit-regression1.cpp b/tests/src/unit-regression1.cpp index 0529f83dd..43cd18438 100644 --- a/tests/src/unit-regression1.cpp +++ b/tests/src/unit-regression1.cpp @@ -1018,7 +1018,7 @@ TEST_CASE("regression tests 1") }; json _; - CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x98", json::parse_error&); + CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an array; last byte: 0x98", json::parse_error&); // related test case: nonempty UTF-8 string (indefinite length) std::vector const vec1 {0x7f, 0x61, 0x61}; @@ -1065,7 +1065,7 @@ TEST_CASE("regression tests 1") }; json _; - CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec1), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xB4", json::parse_error&); + CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec1), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR object key: only string keys are supported, but found a map; last byte: 0xB4", json::parse_error&); // related test case: double-precision std::vector const vec2 @@ -1077,7 +1077,7 @@ TEST_CASE("regression tests 1") 0x96, 0x96, 0xb4, 0xb4, 0xfa, 0x94, 0x94, 0x61, 0x61, 0x61, 0x61, 0x61, 0x61, 0x61, 0x61, 0xfb }; - CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec2), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xB4", json::parse_error&); + CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec2), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR object key: only string keys are supported, but found a map; last byte: 0xB4", json::parse_error&); } SECTION("issue #452 - Heap-buffer-overflow (OSS-Fuzz issue 585)")