From 6b4b825af2c487991538ca01c40aa6421f12de78 Mon Sep 17 00:00:00 2001 From: Niels Lohmann Date: Sun, 4 Oct 2026 17:53:29 +0200 Subject: [PATCH] Handle numbers that do not fit narrow number types in the binary readers (#5607) * Handle numbers that do not fit narrow number types in the binary readers With custom number types narrower than the values in a binary document, for example basic_json<..., std::int32_t, std::uint32_t, float>, every binary reader (CBOR, MessagePack, UBJSON, BJData, BSON, BON8) passed the decoded number to the SAX interface with an implicit conversion: the integer 5000000000 silently became 705032704, and a finite double such as 1e300 became infinity. The lexer handles the same values in JSON text: an integer that fits neither integer type is stored as number_float_t, and a finite number that overflows number_float_t is rejected with out_of_range.406. Pass every number read from binary input through three helpers that apply the lexer's rules: - emit_signed(): number_integer_t, else number_unsigned_t for a non-negative value, else number_float_t - emit_unsigned(): number_unsigned_t, else number_float_t - emit_float(): out_of_range.406 if a finite value overflows number_float_t; infinity and NaN are passed on For consistency, a CBOR negative integer below the range of number_integer_t is now stored as number_float_t, like a too small integer in JSON text, instead of being rejected with parse_error.112. With the default number types, this is the only change in behavior. Signed-off-by: Niels Lohmann * Fix MSVC and clang 3.5 in the narrow number type test MSVC types 3000000000 and 5000000000 as unsigned long, so json(-3000000000) triggered C4146 (unary minus on an unsigned type), which /WX turns into an error. Use LL literals, as elsewhere in the tests. clang 3.5 cannot convert the lambdas in the braced initializer of the format table to function pointers. Use named functions instead. Signed-off-by: Niels Lohmann * Check integer-to-float fallbacks for overflow in the binary readers emit_signed, emit_unsigned, and the CBOR negative integer fallback now pass their number_float_t fallback through emit_float, so a value that overflows number_float_t is rejected with out_of_range.406 like a floating-point value, instead of silently becoming infinity. This only matters for a number_float_t that cannot represent 2^64, such as a half-precision type. The CBOR value -1 - n is computed as long double so that emit_float sees a finite value. Signed-off-by: Niels Lohmann * Use the input_format member instead of passing the format to binary_reader helpers The helpers (get_number, get_to, get_string, get_binary, get_bytes, emit_signed, emit_unsigned, emit_float, unexpect_eof, exception_message) are members of binary_reader, which already stores the format it was constructed with, so the parameter was redundant. Signed-off-by: Niels Lohmann --------- Signed-off-by: Niels Lohmann --- .../docs/api/basic_json/number_float_t.md | 4 + .../docs/api/basic_json/number_integer_t.md | 5 +- .../docs/api/basic_json/number_unsigned_t.md | 5 +- .../docs/features/binary_formats/cbor.md | 6 +- docs/mkdocs/docs/home/exceptions.md | 12 +- .../nlohmann/detail/input/binary_reader.hpp | 494 ++++++++++-------- single_include/nlohmann/json.hpp | 494 ++++++++++-------- tests/src/unit-binary_formats.cpp | 141 +++++ tests/src/unit-cbor.cpp | 30 +- 9 files changed, 743 insertions(+), 448 deletions(-) diff --git a/docs/mkdocs/docs/api/basic_json/number_float_t.md b/docs/mkdocs/docs/api/basic_json/number_float_t.md index 8419a392d..282bb6123 100644 --- a/docs/mkdocs/docs/api/basic_json/number_float_t.md +++ b/docs/mkdocs/docs/api/basic_json/number_float_t.md @@ -55,6 +55,10 @@ This implementation does exactly follow this approach, as it uses double precisi smaller than `-1.79769313486232e+308` and values greater than `1.79769313486232e+308` will be stored as NaN internally and be serialized to `null`. +During deserialization (from JSON text or any of the binary formats), a finite number that does not fit into +`number_float_t` is rejected with [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406), for +example a double-precision number in a binary format when `number_float_t` is `#!cpp float`. + ### Storage Floating-point number values are stored directly inside a `basic_json` type. diff --git a/docs/mkdocs/docs/api/basic_json/number_integer_t.md b/docs/mkdocs/docs/api/basic_json/number_integer_t.md index d77c6a24d..48d2873ed 100644 --- a/docs/mkdocs/docs/api/basic_json/number_integer_t.md +++ b/docs/mkdocs/docs/api/basic_json/number_integer_t.md @@ -47,8 +47,9 @@ With the default values for `NumberIntegerType` (`std::int64_t`), the default va When the default type is used, the maximal integer number that can be stored is `9223372036854775807` (INT64_MAX) and the minimal integer number that can be stored is `-9223372036854775808` (INT64_MIN). Integer numbers that are out of -range will yield over/underflow when used in a constructor. During deserialization, too large or small integer numbers -will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md) or [`number_float_t`](number_float_t.md). +range will yield over/underflow when used in a constructor. During deserialization (from JSON text or any of the binary +formats), too large or small integer numbers will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md) +or [`number_float_t`](number_float_t.md). [RFC 8259](https://tools.ietf.org/html/rfc8259) further states: > Note that when such software is used, numbers that are integers and are in the range [-253+1, 253-1] are diff --git a/docs/mkdocs/docs/api/basic_json/number_unsigned_t.md b/docs/mkdocs/docs/api/basic_json/number_unsigned_t.md index 774fda638..81cc00f77 100644 --- a/docs/mkdocs/docs/api/basic_json/number_unsigned_t.md +++ b/docs/mkdocs/docs/api/basic_json/number_unsigned_t.md @@ -48,8 +48,9 @@ With the default values for `NumberUnsignedType` (`std::uint64_t`), the default When the default type is used, the maximal integer number that can be stored is `18446744073709551615` (UINT64_MAX) and the minimal integer number that can be stored is `0`. Integer numbers that are out of range will yield over/underflow -when used in a constructor. During deserialization, too large or small integer numbers will automatically be stored -as [`number_integer_t`](number_integer_t.md) or [`number_float_t`](number_float_t.md). +when used in a constructor. During deserialization (from JSON text or any of the binary formats), too large or small +integer numbers will automatically be stored as [`number_integer_t`](number_integer_t.md) or +[`number_float_t`](number_float_t.md). [RFC 8259](https://tools.ietf.org/html/rfc8259) further states: > Note that when such software is used, numbers that are integers and are in the range [-253+1, 253-1] are diff --git a/docs/mkdocs/docs/features/binary_formats/cbor.md b/docs/mkdocs/docs/features/binary_formats/cbor.md index 7b5be4631..1b226cd15 100644 --- a/docs/mkdocs/docs/features/binary_formats/cbor.md +++ b/docs/mkdocs/docs/features/binary_formats/cbor.md @@ -168,9 +168,9 @@ The library maps CBOR types to JSON value types as follows: !!! warning "Negative integer overflow" CBOR negative integers (major type 1) are decoded as `-1 - n`. If the encoded magnitude `n` is too large for the - result to fit into `number_integer_t` (`std::int64_t` by default), parsing fails with a - [`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) exception rather than overflowing - silently. + result to fit into `number_integer_t` (`std::int64_t` by default), the result is stored as `number_float_t`, like + a too small integer in JSON text. For example, `-18446744073709551616` (`0x3B` followed by eight `0xFF` bytes) is + stored as `-1.8446744073709552e+19`. !!! warning "Object keys" diff --git a/docs/mkdocs/docs/home/exceptions.md b/docs/mkdocs/docs/home/exceptions.md index e7eb8fd03..a4706dfdd 100644 --- a/docs/mkdocs/docs/home/exceptions.md +++ b/docs/mkdocs/docs/home/exceptions.md @@ -331,9 +331,6 @@ An unexpected byte was read in a [binary format](../features/binary_formats/inde [json.exception.parse_error.112] parse error at byte 15: syntax error while parsing BSON binary: byte array length cannot be negative, is -1 ``` ``` - [json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow - ``` - ``` [json.exception.parse_error.112] parse error at byte 5: syntax error while parsing BSON document: document size 6 does not match the number of bytes read (5) ``` @@ -863,13 +860,18 @@ The JSON Patch operations 'remove' and 'add' cannot be applied to the root eleme ### json.exception.out_of_range.406 -A parsed number could not be stored as without changing it to NaN or INF. +A parsed number could not be stored without changing it to NaN or INF. For the binary formats, this happens when a +finite floating-point number does not fit into [`number_float_t`](../api/basic_json/number_float_t.md), for example a +double-precision number when `number_float_t` is `#!cpp float`. -!!! failure "Example message" +!!! failure "Example messages" ``` number overflow parsing '10E1000' ``` + ``` + [json.exception.out_of_range.406] syntax error while parsing CBOR value: number overflow + ``` ### json.exception.out_of_range.407 diff --git a/include/nlohmann/detail/input/binary_reader.hpp b/include/nlohmann/detail/input/binary_reader.hpp index d6b45909a..369e274fb 100644 --- a/include/nlohmann/detail/input/binary_reader.hpp +++ b/include/nlohmann/detail/input/binary_reader.hpp @@ -197,7 +197,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(current != char_traits::eof())) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(110, chars_read, - exception_message(input_format, concat("expected end of input; last byte: 0x", get_token_string()), "value"), nullptr)); + exception_message(concat("expected end of input; last byte: 0x", get_token_string()), "value"), nullptr)); } } @@ -320,7 +320,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(document_size < 0 || static_cast(document_size) != chars_read - document_start)) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(112, chars_read, - exception_message(input_format_t::bson, concat("document size ", std::to_string(document_size), " does not match the number of bytes read (", std::to_string(chars_read - document_start), ")"), "document"), nullptr)); + exception_message(concat("document size ", std::to_string(document_size), " does not match the number of bytes read (", std::to_string(chars_read - document_start), ")"), "document"), nullptr)); } return true; } @@ -335,7 +335,7 @@ class binary_reader // check_bson_document_size() measures the document from here const std::size_t document_start = chars_read; std::int32_t document_size{}; - if (!get_number(input_format_t::bson, document_size)) + if (!get_number(document_size)) { return false; } @@ -400,7 +400,7 @@ class binary_reader continue; } - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bson, "element list"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("element list"))) { return false; } @@ -444,7 +444,7 @@ class binary_reader while (true) { get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bson, "cstring"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("cstring"))) { return false; } @@ -514,10 +514,10 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bson, concat("string length must be at least 1, is ", std::to_string(len)), "string"), nullptr)); + exception_message(concat("string length must be at least 1, is ", std::to_string(len)), "string"), nullptr)); } - if (JSON_HEDLEY_UNLIKELY(!get_string(input_format_t::bson, len - static_cast(1), result))) + if (JSON_HEDLEY_UNLIKELY(!get_string(len - static_cast(1), result))) { return false; } @@ -526,8 +526,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bson, - "BSON string is not null-terminated", + exception_message("BSON string is not null-terminated", "string"), nullptr)); } @@ -550,18 +549,18 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bson, concat("byte array length cannot be negative, is ", std::to_string(len)), "binary"), nullptr)); + exception_message(concat("byte array length cannot be negative, is ", std::to_string(len)), "binary"), nullptr)); } // All BSON binary values have a subtype std::uint8_t subtype{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::bson, subtype))) + if (JSON_HEDLEY_UNLIKELY(!get_number(subtype))) { return false; } result.set_subtype(subtype); - return get_binary(input_format_t::bson, len, result); + return get_binary(len, result); } /*! @@ -582,14 +581,14 @@ class binary_reader case 0x01: // double { double number{}; - return get_number(input_format_t::bson, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0x02: // string { std::int32_t len{}; string_t value; - return get_number(input_format_t::bson, len) && get_bson_string(len, value) && sax->string(value); + return get_number(len) && get_bson_string(len, value) && sax->string(value); } case 0x03: // object @@ -606,13 +605,13 @@ class binary_reader { std::int32_t len{}; binary_t value; - return get_number(input_format_t::bson, len) && get_bson_binary(len, value) && sax->binary(value); + return get_number(len) && get_bson_binary(len, value) && sax->binary(value); } case 0x08: // boolean { std::uint8_t value{}; - return get_number(input_format_t::bson, value) && sax->boolean(value != 0); + return get_number(value) && sax->boolean(value != 0); } case 0x0A: // null @@ -623,19 +622,19 @@ class binary_reader case 0x10: // int32 { std::int32_t value{}; - return get_number(input_format_t::bson, value) && sax->number_integer(conditional_static_cast(value)); + return get_number(value) && emit_signed(value); } case 0x12: // int64 { std::int64_t value{}; - return get_number(input_format_t::bson, value) && sax->number_integer(conditional_static_cast(value)); + return get_number(value) && emit_signed(value); } case 0x11: // uint64 { std::uint64_t value{}; - return get_number(input_format_t::bson, value) && sax->number_unsigned(value); + return get_number(value) && emit_unsigned(value); } default: // anything else is not supported (yet) @@ -657,18 +656,23 @@ class binary_reader bool get_cbor_negative_integer() { NumberType number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } - const auto max_val = static_cast((std::numeric_limits::max)()); - if (number > max_val) + + // the value is -1 - number, which fits into number_integer_t + // whenever number does + if (JSON_HEDLEY_LIKELY(value_in_range_of(number))) { - return sax->parse_error(chars_read, get_token_string(), - parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr)); + return sax->number_integer(static_cast(-1) - static_cast(number)); } - return sax->number_integer(conditional_static_cast(static_cast(-1) - static_cast(number))); + + // like the lexer does for JSON text, store a value too small for + // number_integer_t as number_float_t; compute it as long double so + // that emit_float sees a finite value and can detect an overflow of + // number_float_t + return emit_float(static_cast(-1) - static_cast(number)); } /*! @@ -693,7 +697,7 @@ class binary_reader { // EOF case char_traits::eof(): - return unexpect_eof(input_format_t::cbor, "value"); + return unexpect_eof("value"); // Integer 0x00..0x17 (0..23) case 0x00: @@ -725,25 +729,25 @@ class binary_reader case 0x18: // Unsigned integer (one-byte uint8_t follows) { std::uint8_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0x19: // Unsigned integer (two-byte uint16_t follows) { std::uint16_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0x1A: // Unsigned integer (four-byte uint32_t follows) { std::uint32_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0x1B: // Unsigned integer (eight-byte uint64_t follows) { std::uint64_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } // Negative integer -1-0x00..-1-0x17 (-1..-24) @@ -964,7 +968,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } // ignore and store: the tag value is already in the head, so @@ -986,7 +990,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } case cbor_tag_handler_t::ignore: @@ -1044,25 +1048,25 @@ class binary_reader return sax->null(); case 0xF9: // Half-Precision Float (two-byte IEEE 754) - return get_half_float(input_format_t::cbor, false); + return get_half_float(false); case 0xFA: // Single-Precision Float (four-byte IEEE 754) { float number{}; - return get_number(input_format_t::cbor, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xFB: // Double-Precision Float (eight-byte IEEE 754) { double number{}; - return get_number(input_format_t::cbor, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } default: // anything else (0xFF is handled inside the other types) { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } } } @@ -1111,38 +1115,38 @@ class binary_reader case 0x76: case 0x77: { - return get_string(input_format_t::cbor, static_cast(current) & 0x1Fu, result); + return get_string(static_cast(current) & 0x1Fu, result); } case 0x78: // UTF-8 string (one-byte uint8_t for n follows) { std::uint8_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } case 0x79: // UTF-8 string (two-byte uint16_t for n follow) { std::uint16_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } case 0x7A: // UTF-8 string (four-byte uint32_t for n follow) { std::uint32_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } case 0x7B: // UTF-8 string (eight-byte uint64_t for n follow) { std::uint64_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } default: { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::cbor, concat("expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x", last_token), "string"), nullptr)); + exception_message(concat("expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x", last_token), "string"), nullptr)); } } } @@ -1170,7 +1174,7 @@ class binary_reader while (true) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "string"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("string"))) { return false; } @@ -1279,7 +1283,7 @@ class binary_reader auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::cbor, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); + exception_message(concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); } /*! @@ -1326,42 +1330,42 @@ class binary_reader case 0x56: case 0x57: { - return get_binary(input_format_t::cbor, static_cast(current) & 0x1Fu, result); + return get_binary(static_cast(current) & 0x1Fu, result); } case 0x58: // Binary data (one-byte uint8_t for n follows) { std::uint8_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } case 0x59: // Binary data (two-byte uint16_t for n follow) { std::uint16_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } case 0x5A: // Binary data (four-byte uint32_t for n follow) { std::uint32_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } case 0x5B: // Binary data (eight-byte uint64_t for n follow) { std::uint64_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } default: { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::cbor, concat("expected length specification (0x40-0x5B) or indefinite binary array type (0x5F); last byte: 0x", last_token), "binary"), nullptr)); + exception_message(concat("expected length specification (0x40-0x5B) or indefinite binary array type (0x5F); last byte: 0x", last_token), "binary"), nullptr)); } } } @@ -1385,7 +1389,7 @@ class binary_reader while (true) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "binary"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("binary"))) { return false; } @@ -1443,7 +1447,7 @@ class binary_reader case 0x18: // 1 byte { std::uint8_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -1454,7 +1458,7 @@ class binary_reader case 0x19: // 2 bytes { std::uint16_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -1465,7 +1469,7 @@ class binary_reader case 0x1A: // 4 bytes { std::uint32_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -1476,7 +1480,7 @@ class binary_reader case 0x1B: // 8 bytes { std::uint64_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -1509,7 +1513,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(!value_in_range_of(len) || len == detail::unknown_size())) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format_t::cbor, concat("excessive ", context, " size"), "size"), nullptr)); + exception_message(concat("excessive ", context, " size"), "size"), nullptr)); } result = conditional_static_cast(len); return true; @@ -1638,7 +1642,7 @@ class binary_reader { // EOF case char_traits::eof(): - return unexpect_eof(input_format_t::msgpack, "value"); + return unexpect_eof("value"); // positive fixint case 0x00: @@ -1878,85 +1882,85 @@ class binary_reader case 0xCA: // float 32 { float number{}; - return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xCB: // float 64 { double number{}; - return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xCC: // uint 8 { std::uint8_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xCD: // uint 16 { std::uint16_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xCE: // uint 32 { std::uint32_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xCF: // uint 64 { std::uint64_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xD0: // int 8 { std::int8_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xD1: // int 16 { std::int16_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xD2: // int 32 { std::int32_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xD3: // int 64 { std::int64_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xDC: // array 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && enter_array(static_cast(len)); + return get_number(len) && enter_array(static_cast(len)); } case 0xDD: // array 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && enter_array(conditional_static_cast(len)); + return get_number(len) && enter_array(conditional_static_cast(len)); } case 0xDE: // map 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && enter_object(static_cast(len)); + return get_number(len) && enter_object(static_cast(len)); } case 0xDF: // map 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && enter_object(conditional_static_cast(len)); + return get_number(len) && enter_object(conditional_static_cast(len)); } // negative fixint @@ -1998,7 +2002,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::msgpack, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } } } @@ -2015,7 +2019,7 @@ class binary_reader */ bool get_msgpack_string(string_t& result, const char* context = "string") { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::msgpack, "string"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("string"))) { return false; } @@ -2056,32 +2060,32 @@ class binary_reader case 0xBE: case 0xBF: { - return get_string(input_format_t::msgpack, static_cast(current) & 0x1Fu, result) && check_string_utf8(result, context); + return get_string(static_cast(current) & 0x1Fu, result) && check_string_utf8(result, context); } case 0xD9: // str 8 { std::uint8_t len{}; - return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 0xDA: // str 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 0xDB: // str 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } default: { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::msgpack, concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", last_token), "string"), nullptr)); + exception_message(concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", last_token), "string"), nullptr)); } } } @@ -2175,7 +2179,7 @@ class binary_reader auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::msgpack, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); + exception_message(concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); } /*! @@ -2202,31 +2206,31 @@ class binary_reader case 0xC4: // bin 8 { std::uint8_t len{}; - return get_number(input_format_t::msgpack, len) && - get_binary(input_format_t::msgpack, len, result); + return get_number(len) && + get_binary(len, result); } case 0xC5: // bin 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && - get_binary(input_format_t::msgpack, len, result); + return get_number(len) && + get_binary(len, result); } case 0xC6: // bin 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && - get_binary(input_format_t::msgpack, len, result); + return get_number(len) && + get_binary(len, result); } case 0xC7: // ext 8 { std::uint8_t len{}; std::int8_t subtype{}; - return get_number(input_format_t::msgpack, len) && - get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, len, result) && + return get_number(len) && + get_number(subtype) && + get_binary(len, result) && assign_and_return_true(subtype); } @@ -2234,9 +2238,9 @@ class binary_reader { std::uint16_t len{}; std::int8_t subtype{}; - return get_number(input_format_t::msgpack, len) && - get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, len, result) && + return get_number(len) && + get_number(subtype) && + get_binary(len, result) && assign_and_return_true(subtype); } @@ -2244,49 +2248,49 @@ class binary_reader { std::uint32_t len{}; std::int8_t subtype{}; - return get_number(input_format_t::msgpack, len) && - get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, len, result) && + return get_number(len) && + get_number(subtype) && + get_binary(len, result) && assign_and_return_true(subtype); } case 0xD4: // fixext 1 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 1, result) && + return get_number(subtype) && + get_binary(1, result) && assign_and_return_true(subtype); } case 0xD5: // fixext 2 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 2, result) && + return get_number(subtype) && + get_binary(2, result) && assign_and_return_true(subtype); } case 0xD6: // fixext 4 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 4, result) && + return get_number(subtype) && + get_binary(4, result) && assign_and_return_true(subtype); } case 0xD7: // fixext 8 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 8, result) && + return get_number(subtype) && + get_binary(8, result) && assign_and_return_true(subtype); } case 0xD8: // fixext 16 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 16, result) && + return get_number(subtype) && + get_binary(16, result) && assign_and_return_true(subtype); } @@ -2485,7 +2489,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(len < 0)) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, - exception_message(input_format, "string length must not be negative", "string"), nullptr)); + exception_message("string length must not be negative", "string"), nullptr)); } return true; } @@ -2515,7 +2519,7 @@ class binary_reader get(); } - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "value"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("value"))) { return false; } @@ -2525,31 +2529,31 @@ class binary_reader case 'U': { std::uint8_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 'i': { std::int8_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'I': { std::int16_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'l': { std::int32_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'L': { std::int64_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'u': @@ -2559,7 +2563,7 @@ class binary_reader break; } std::uint16_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 'm': @@ -2569,7 +2573,7 @@ class binary_reader break; } std::uint32_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 'M': @@ -2579,7 +2583,7 @@ class binary_reader break; } std::uint64_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } default: @@ -2596,7 +2600,7 @@ class binary_reader { message = "expected length type specification (U, i, u, I, m, l, M, L); last byte: 0x" + last_token; } - return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format, message, "string"), nullptr)); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(message, "string"), nullptr)); } /*! @@ -2675,19 +2679,19 @@ class binary_reader bool get_ubjson_signed_count(std::size_t& result) { SignedType number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } if (JSON_HEDLEY_UNLIKELY(number < 0)) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, - exception_message(input_format, "count in an optimized container must be positive", "size"), nullptr)); + exception_message("count in an optimized container must be positive", "size"), nullptr)); } if (JSON_HEDLEY_UNLIKELY(!value_in_range_of(number))) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format, "integer value overflow", "size"), nullptr)); + exception_message("integer value overflow", "size"), nullptr)); } result = static_cast(number); // NOLINT(bugprone-signed-char-misuse,cert-str34-c): number is not a char return true; @@ -2716,7 +2720,7 @@ class binary_reader case 'U': { std::uint8_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } @@ -2743,7 +2747,7 @@ class binary_reader break; } std::uint16_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } @@ -2758,7 +2762,7 @@ class binary_reader break; } std::uint32_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } @@ -2773,14 +2777,14 @@ class binary_reader break; } std::uint64_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } if (!value_in_range_of(number)) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format, "integer value overflow", "size"), nullptr)); + exception_message("integer value overflow", "size"), nullptr)); } result = detail::conditional_static_cast(number); return true; @@ -2794,7 +2798,7 @@ class binary_reader } if (is_ndarray) // ndarray dimensional vector can only contain integers and cannot embed another array { - return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, exception_message(input_format, "ndarray dimensional vector is not allowed", "size"), nullptr)); + return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, exception_message("ndarray dimensional vector is not allowed", "size"), nullptr)); } std::vector dim; if (JSON_HEDLEY_UNLIKELY(!get_ubjson_ndarray_size(dim))) @@ -2834,7 +2838,7 @@ class binary_reader // or SIZE_MAX. if (JSON_HEDLEY_UNLIKELY(result > (std::numeric_limits::max)() / i)) { - return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr)); + return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message("excessive ndarray size caused overflow", "size"), nullptr)); } result *= i; // the pre-check above already rules out result becoming 0 @@ -2843,9 +2847,9 @@ class binary_reader // unknown-size container (see get_ubjson_size_type()) if (result == npos) { - return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr)); + return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message("excessive ndarray size caused overflow", "size"), nullptr)); } - if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast(i)))) + if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(i))) { return false; } @@ -2871,7 +2875,7 @@ class binary_reader { message = "expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x" + last_token; } - return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format, message, "size"), nullptr)); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(message, "size"), nullptr)); } /*! @@ -2906,10 +2910,10 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, concat("marker 0x", last_token, " is not a permitted optimized array type"), "type"), nullptr)); + exception_message(concat("marker 0x", last_token, " is not a permitted optimized array type"), "type"), nullptr)); } - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "type"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("type"))) { return false; } @@ -2917,13 +2921,13 @@ class binary_reader get_ignore_noop(); if (JSON_HEDLEY_UNLIKELY(current != '#')) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "value"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("value"))) { return false; } auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, concat("expected '#' after type information; last byte: 0x", last_token), "size"), nullptr)); + exception_message(concat("expected '#' after type information; last byte: 0x", last_token), "size"), nullptr)); } const bool is_error = get_ubjson_size_value(result.first, is_ndarray); @@ -2943,7 +2947,7 @@ class binary_reader if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(112, chars_read, - exception_message(input_format, "ndarray requires both type and size", "size"), nullptr)); + exception_message("ndarray requires both type and size", "size"), nullptr)); } return is_error; } @@ -2960,7 +2964,7 @@ class binary_reader switch (prefix) { case char_traits::eof(): // EOF - return unexpect_eof(input_format, "value"); + return unexpect_eof("value"); case 'T': // true return sax->boolean(true); @@ -2977,37 +2981,37 @@ class binary_reader break; } std::uint8_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'U': { std::uint8_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'i': { std::int8_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'I': { std::int16_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'l': { std::int32_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'L': { std::int64_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'u': @@ -3017,7 +3021,7 @@ class binary_reader break; } std::uint16_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'm': @@ -3027,7 +3031,7 @@ class binary_reader break; } std::uint32_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'M': @@ -3037,7 +3041,7 @@ class binary_reader break; } std::uint64_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'h': @@ -3046,19 +3050,19 @@ class binary_reader { break; } - return get_half_float(input_format, true); + return get_half_float(true); } case 'd': { float number{}; - return get_number(input_format, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 'D': { double number{}; - return get_number(input_format, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 'H': @@ -3069,7 +3073,7 @@ class binary_reader case 'C': // char { get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "char"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("char"))) { return false; } @@ -3077,7 +3081,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format, concat("byte after 'C' must be in range 0x00..0x7F; last byte: 0x", last_token), "char"), nullptr)); + exception_message(concat("byte after 'C' must be in range 0x00..0x7F; last byte: 0x", last_token), "char"), nullptr)); } string_t s(1, static_cast(current)); return sax->string(s); @@ -3099,7 +3103,7 @@ class binary_reader break; } auto last_token = get_token_string(); - return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message(input_format, "invalid byte: 0x" + last_token, "value"), nullptr)); + return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message("invalid byte: 0x" + last_token, "value"), nullptr)); } /*! @@ -3125,7 +3129,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, "invalid byte: 0x" + last_token, "type"), nullptr)); + exception_message("invalid byte: 0x" + last_token, "type"), nullptr)); } string_t type = type_name; // sax->string() takes a reference @@ -3160,7 +3164,7 @@ class binary_reader if (input_format == input_format_t::bjdata && size_and_type.first != npos && size_and_type.second == 'B') { binary_t result; - return get_binary(input_format, size_and_type.first, result) && sax->binary(result); + return get_binary(size_and_type.first, result) && sax->binary(result); } if (size_and_type.first != npos) @@ -3173,7 +3177,7 @@ class binary_reader && size_and_type.first > max_valueless_container_size)) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format, "excessive array size", "size"), nullptr)); + exception_message("excessive array size", "size"), nullptr)); } if (JSON_HEDLEY_UNLIKELY(!enter_array(size_and_type.first, size_and_type.second))) @@ -3210,7 +3214,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, "BJData object does not support ND-array size in optimized format", "object"), nullptr)); + exception_message("BJData object does not support ND-array size in optimized format", "object"), nullptr)); } if (size_and_type.first != npos) @@ -3240,7 +3244,7 @@ class binary_reader for (std::size_t i = 0; i < size; ++i) { get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } @@ -3259,7 +3263,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(result_remainder != token_type::end_of_input)) { return sax->parse_error(chars_read, number_string, parse_error::create(115, chars_read, - exception_message(input_format, concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); + exception_message(concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); } switch (result_number) @@ -3299,7 +3303,7 @@ class binary_reader case token_type::literal_or_value: default: return sax->parse_error(chars_read, number_string, parse_error::create(115, chars_read, - exception_message(input_format, concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); + exception_message(concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); } } @@ -3368,7 +3372,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bon8, concat(detail, ": 0x", last_token), context), nullptr)); + exception_message(concat(detail, ": 0x", last_token), context), nullptr)); } /*! @@ -3472,7 +3476,7 @@ class binary_reader if (byte == char_traits::eof()) { - return unexpect_eof(input_format_t::bon8, "value"); + return unexpect_eof("value"); } // string: ASCII character @@ -3509,25 +3513,25 @@ class binary_reader case 0x8C: // int32 { std::int32_t number{}; - return get_number(input_format_t::bon8, number) && emit_bon8_integer(number); + return get_number(number) && emit_bon8_integer(number); } case 0x8D: // int64 { std::int64_t number{}; - return get_number(input_format_t::bon8, number) && emit_bon8_integer(number); + return get_number(number) && emit_bon8_integer(number); } case 0x8E: // binary32 { float number{}; - return get_number(input_format_t::bon8, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0x8F: // binary64 { double number{}; - return get_number(input_format_t::bon8, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xF8: @@ -3593,7 +3597,9 @@ class binary_reader @brief pass an integer to the SAX parser Non-negative integers are passed as unsigned, negative integers as signed - numbers, like the other binary formats do. + numbers, like the other binary formats do. A value that does not fit the + number type is passed as described for @ref emit_unsigned and + @ref emit_signed. @param[in] number the integer @return whether the SAX parser accepted the value @@ -3602,9 +3608,9 @@ class binary_reader { if (number >= 0) { - return sax->number_unsigned(static_cast(number)); + return emit_unsigned(static_cast(number)); } - return sax->number_integer(static_cast(number)); + return emit_signed(number); } /*! @@ -3624,7 +3630,7 @@ class binary_reader */ bool get_bon8_integer(const char_int_type lead, const char_int_type second) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bon8, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } @@ -3656,13 +3662,12 @@ class binary_reader { if (JSON_HEDLEY_UNLIKELY(get_bon8() == char_traits::eof())) { - return unexpect_eof(input_format_t::bon8, "number"); + return unexpect_eof("number"); } value = (value << 8) | static_cast(current); } - return negative ? sax->number_integer(static_cast(-(value + offset))) - : sax->number_unsigned(static_cast(value + offset)); + return emit_bon8_integer(negative ? -(value + offset) : value + offset); } /*! @@ -3680,7 +3685,7 @@ class binary_reader if (byte == char_traits::eof()) { - return unexpect_eof(input_format_t::bon8, "key"); + return unexpect_eof("key"); } if (byte == 0xFF) @@ -3700,7 +3705,7 @@ class binary_reader if (second == char_traits::eof()) { // the input ends inside a character or an integer - return unexpect_eof(input_format_t::bon8, "key"); + return unexpect_eof("key"); } unget_bon8(second); if (is_bon8_continuation(second)) @@ -3773,7 +3778,7 @@ class binary_reader if (byte == char_traits::eof()) { - return unexpect_eof(input_format_t::bon8, "string"); + return unexpect_eof("string"); } // end of string @@ -3804,7 +3809,7 @@ class binary_reader { // the input ends inside a character or an integer: either // way, the message is incomplete - return unexpect_eof(input_format_t::bon8, "string"); + return unexpect_eof("string"); } if (!is_bon8_continuation(second)) { @@ -3845,7 +3850,7 @@ class binary_reader { if (JSON_HEDLEY_UNLIKELY(get_bon8() == char_traits::eof())) { - return unexpect_eof(input_format_t::bon8, "string"); + return unexpect_eof("string"); } if (JSON_HEDLEY_UNLIKELY(!is_bon8_continuation(current))) { @@ -3884,7 +3889,7 @@ class binary_reader @return bool, whether the read was successful */ template - bool get_to(T& dest, const input_format_t format, const char* context) + bool get_to(T& dest, const char* context) { // false positive: new_chars_read is read on the next lines // @infer-ignore DEAD_STORE @@ -3894,7 +3899,7 @@ class binary_reader { // in case of failure, advance position by 1 to report the failing location ++chars_read; - sax->parse_error(chars_read, "", parse_error::create(110, chars_read, exception_message(format, "unexpected end of input", context), nullptr)); + sax->parse_error(chars_read, "", parse_error::create(110, chars_read, exception_message("unexpected end of input", context), nullptr)); return false; } return true; @@ -3945,7 +3950,6 @@ class binary_reader @brief read a number from the input @tparam NumberType the type of the number - @param[in] format the current format (for diagnostics) @param[out] result number of type @a NumberType @return whether conversion completed @@ -3957,42 +3961,119 @@ class binary_reader should reorder on big endian systems. */ template - bool get_number(const input_format_t format, NumberType& result) + bool get_number(NumberType& result) { // read in the original format - if (JSON_HEDLEY_UNLIKELY(!get_to(result, format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!get_to(result, "number"))) { return false; } - if (is_little_endian != (InputIsLittleEndian || format == input_format_t::bjdata)) + if (is_little_endian != (InputIsLittleEndian || input_format == input_format_t::bjdata)) { byte_swap(result); } return true; } + /*! + @brief pass a signed integer read from the input to the SAX parser + + Like the lexer does for JSON text, a value that does not fit into + number_integer_t is passed as number_unsigned_t if it is non-negative and + fits there, and as number_float_t otherwise. With the default number + types, every integer the binary formats can encode fits, so this only + matters for narrower custom number types. + + @tparam NumberType a signed integer type + @param[in] number the integer + @return whether the SAX parser accepted the value + + @throw out_of_range.406 if @a number overflows number_float_t (see + @ref emit_float) + */ + template + bool emit_signed(const NumberType number) + { + if (JSON_HEDLEY_LIKELY(value_in_range_of(number))) + { + return sax->number_integer(static_cast(number)); + } + if (value_in_range_of(number)) + { + return sax->number_unsigned(static_cast(number)); + } + return emit_float(number); + } + + /*! + @brief pass an unsigned integer read from the input to the SAX parser + + Like the lexer does for JSON text, a value that does not fit into + number_unsigned_t is passed as number_float_t. + + @tparam NumberType an unsigned integer type + @param[in] number the integer + @return whether the SAX parser accepted the value + + @throw out_of_range.406 if @a number overflows number_float_t (see + @ref emit_float) + */ + template + bool emit_unsigned(const NumberType number) + { + if (JSON_HEDLEY_LIKELY(value_in_range_of(number))) + { + return sax->number_unsigned(static_cast(number)); + } + return emit_float(number); + } + + /*! + @brief pass a floating-point number read from the input to the SAX parser + + Like the lexer does for JSON text, a finite value that overflows + number_float_t is rejected instead of silently becoming infinity. Infinity + and NaN in the input are passed on unchanged. Integers only overflow if + number_float_t cannot represent 2^64, e.g., a half-precision type. + + @tparam NumberType a floating-point or integer type + @param[in] number the number + @return whether the SAX parser accepted the value + + @throw out_of_range.406 if a finite @a number overflows number_float_t + */ + template + bool emit_float(const NumberType number) + { + const auto result = static_cast(number); + if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result))) + { + return sax->parse_error(chars_read, get_token_string(), + out_of_range::create(406, exception_message("number overflow", "value"), nullptr)); + } + return sax->number_float(result, ""); + } + /*! @brief read and decode an IEEE 754 half-precision (16-bit) float Used by CBOR (big endian) and BJData (little endian); the two formats only differ in the byte order of the two bytes that make up the half. - - @param[in] format the current format (for diagnostics) @param[in] little_endian whether the two bytes are little endian (BJData) or big endian (CBOR) @return whether reading and decoding succeeded */ - bool get_half_float(const input_format_t format, const bool little_endian) + bool get_half_float(const bool little_endian) { const auto byte1_raw = get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } const auto byte2_raw = get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } @@ -4038,7 +4119,6 @@ class binary_reader @brief create a string by reading characters from the input @tparam NumberType the type of the number - @param[in] format the current format (for diagnostics) @param[in] len number of characters to read @param[out] result string created by reading @a len bytes @@ -4049,8 +4129,7 @@ class binary_reader the input before we run out of string memory. */ template - bool get_string(const input_format_t format, - const NumberType len, + bool get_string(const NumberType len, string_t& result) { // Strings are taken as is by default: none of CBOR (RFC 8949 ยง3.1 @@ -4062,7 +4141,7 @@ class binary_reader // @ref error_handler, applied once the whole string (all chunks of // an indefinite-length CBOR string included) has been assembled, by // @ref check_string_utf8 at the call site. - return get_bytes(format, len, "string", result); + return get_bytes(len, "string", result); } /*! @@ -4093,7 +4172,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format, "invalid string: ill-formed UTF-8 byte", context), nullptr)); + exception_message("invalid string: ill-formed UTF-8 byte", context), nullptr)); } result = sanitize_utf8(result, error_handler); @@ -4104,7 +4183,6 @@ class binary_reader @brief create a byte array by reading bytes from the input @tparam NumberType the type of the number - @param[in] format the current format (for diagnostics) @param[in] len number of bytes to read @param[out] result byte array created by reading @a len bytes @@ -4115,11 +4193,10 @@ class binary_reader the input before we run out of memory. */ template - bool get_binary(const input_format_t format, - const NumberType len, + bool get_binary(const NumberType len, binary_t& result) { - return get_bytes(format, len, "binary", result); + return get_bytes(len, "binary", result); } /*! @@ -4127,7 +4204,6 @@ class binary_reader @tparam NumberType the type of the length @tparam ContainerType the destination container (string_t or binary_t) - @param[in] format the current format (for diagnostics) @param[in] len number of bytes to read @param[in] context further context information (for diagnostics) @param[out] result container the bytes are appended to @@ -4142,8 +4218,7 @@ class binary_reader detects a premature end of input. */ template - bool get_bytes(const input_format_t format, - NumberType len, + bool get_bytes(NumberType len, const char* context, ContainerType& result) { @@ -4174,7 +4249,7 @@ class binary_reader result.resize(old_size + bytes_read); ++chars_read; current = char_traits::eof(); - return unexpect_eof(format, context); + return unexpect_eof(context); } // a full chunk was read; get_elements() never returns more than requested JSON_ASSERT(bytes_read == wanted); @@ -4184,17 +4259,16 @@ class binary_reader } /*! - @param[in] format the current format (for diagnostics) @param[in] context further context information (for diagnostics) @return whether the last read character is not EOF */ - JSON_HEDLEY_NON_NULL(3) - bool unexpect_eof(const input_format_t format, const char* context) const + JSON_HEDLEY_NON_NULL(2) + bool unexpect_eof(const char* context) const { if (JSON_HEDLEY_UNLIKELY(current == char_traits::eof())) { return sax->parse_error(chars_read, "", - parse_error::create(110, chars_read, exception_message(format, "unexpected end of input", context), nullptr)); + parse_error::create(110, chars_read, exception_message("unexpected end of input", context), nullptr)); } return true; } @@ -4210,18 +4284,16 @@ class binary_reader } /*! - @param[in] format the current format @param[in] detail a detailed error message @param[in] context further context information @return a message string to use in the parse_error exceptions */ - std::string exception_message(const input_format_t format, - const std::string& detail, + std::string exception_message(const std::string& detail, const std::string& context) const { std::string error_msg = "syntax error while parsing "; - switch (format) + switch (input_format) { case input_format_t::cbor: error_msg += "CBOR"; diff --git a/single_include/nlohmann/json.hpp b/single_include/nlohmann/json.hpp index 75684866e..bea2305a2 100644 --- a/single_include/nlohmann/json.hpp +++ b/single_include/nlohmann/json.hpp @@ -13783,7 +13783,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(current != char_traits::eof())) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(110, chars_read, - exception_message(input_format, concat("expected end of input; last byte: 0x", get_token_string()), "value"), nullptr)); + exception_message(concat("expected end of input; last byte: 0x", get_token_string()), "value"), nullptr)); } } @@ -13906,7 +13906,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(document_size < 0 || static_cast(document_size) != chars_read - document_start)) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(112, chars_read, - exception_message(input_format_t::bson, concat("document size ", std::to_string(document_size), " does not match the number of bytes read (", std::to_string(chars_read - document_start), ")"), "document"), nullptr)); + exception_message(concat("document size ", std::to_string(document_size), " does not match the number of bytes read (", std::to_string(chars_read - document_start), ")"), "document"), nullptr)); } return true; } @@ -13921,7 +13921,7 @@ class binary_reader // check_bson_document_size() measures the document from here const std::size_t document_start = chars_read; std::int32_t document_size{}; - if (!get_number(input_format_t::bson, document_size)) + if (!get_number(document_size)) { return false; } @@ -13986,7 +13986,7 @@ class binary_reader continue; } - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bson, "element list"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("element list"))) { return false; } @@ -14030,7 +14030,7 @@ class binary_reader while (true) { get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bson, "cstring"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("cstring"))) { return false; } @@ -14100,10 +14100,10 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bson, concat("string length must be at least 1, is ", std::to_string(len)), "string"), nullptr)); + exception_message(concat("string length must be at least 1, is ", std::to_string(len)), "string"), nullptr)); } - if (JSON_HEDLEY_UNLIKELY(!get_string(input_format_t::bson, len - static_cast(1), result))) + if (JSON_HEDLEY_UNLIKELY(!get_string(len - static_cast(1), result))) { return false; } @@ -14112,8 +14112,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bson, - "BSON string is not null-terminated", + exception_message("BSON string is not null-terminated", "string"), nullptr)); } @@ -14136,18 +14135,18 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bson, concat("byte array length cannot be negative, is ", std::to_string(len)), "binary"), nullptr)); + exception_message(concat("byte array length cannot be negative, is ", std::to_string(len)), "binary"), nullptr)); } // All BSON binary values have a subtype std::uint8_t subtype{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::bson, subtype))) + if (JSON_HEDLEY_UNLIKELY(!get_number(subtype))) { return false; } result.set_subtype(subtype); - return get_binary(input_format_t::bson, len, result); + return get_binary(len, result); } /*! @@ -14168,14 +14167,14 @@ class binary_reader case 0x01: // double { double number{}; - return get_number(input_format_t::bson, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0x02: // string { std::int32_t len{}; string_t value; - return get_number(input_format_t::bson, len) && get_bson_string(len, value) && sax->string(value); + return get_number(len) && get_bson_string(len, value) && sax->string(value); } case 0x03: // object @@ -14192,13 +14191,13 @@ class binary_reader { std::int32_t len{}; binary_t value; - return get_number(input_format_t::bson, len) && get_bson_binary(len, value) && sax->binary(value); + return get_number(len) && get_bson_binary(len, value) && sax->binary(value); } case 0x08: // boolean { std::uint8_t value{}; - return get_number(input_format_t::bson, value) && sax->boolean(value != 0); + return get_number(value) && sax->boolean(value != 0); } case 0x0A: // null @@ -14209,19 +14208,19 @@ class binary_reader case 0x10: // int32 { std::int32_t value{}; - return get_number(input_format_t::bson, value) && sax->number_integer(conditional_static_cast(value)); + return get_number(value) && emit_signed(value); } case 0x12: // int64 { std::int64_t value{}; - return get_number(input_format_t::bson, value) && sax->number_integer(conditional_static_cast(value)); + return get_number(value) && emit_signed(value); } case 0x11: // uint64 { std::uint64_t value{}; - return get_number(input_format_t::bson, value) && sax->number_unsigned(value); + return get_number(value) && emit_unsigned(value); } default: // anything else is not supported (yet) @@ -14243,18 +14242,23 @@ class binary_reader bool get_cbor_negative_integer() { NumberType number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } - const auto max_val = static_cast((std::numeric_limits::max)()); - if (number > max_val) + + // the value is -1 - number, which fits into number_integer_t + // whenever number does + if (JSON_HEDLEY_LIKELY(value_in_range_of(number))) { - return sax->parse_error(chars_read, get_token_string(), - parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr)); + return sax->number_integer(static_cast(-1) - static_cast(number)); } - return sax->number_integer(conditional_static_cast(static_cast(-1) - static_cast(number))); + + // like the lexer does for JSON text, store a value too small for + // number_integer_t as number_float_t; compute it as long double so + // that emit_float sees a finite value and can detect an overflow of + // number_float_t + return emit_float(static_cast(-1) - static_cast(number)); } /*! @@ -14279,7 +14283,7 @@ class binary_reader { // EOF case char_traits::eof(): - return unexpect_eof(input_format_t::cbor, "value"); + return unexpect_eof("value"); // Integer 0x00..0x17 (0..23) case 0x00: @@ -14311,25 +14315,25 @@ class binary_reader case 0x18: // Unsigned integer (one-byte uint8_t follows) { std::uint8_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0x19: // Unsigned integer (two-byte uint16_t follows) { std::uint16_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0x1A: // Unsigned integer (four-byte uint32_t follows) { std::uint32_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0x1B: // Unsigned integer (eight-byte uint64_t follows) { std::uint64_t number{}; - return get_number(input_format_t::cbor, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } // Negative integer -1-0x00..-1-0x17 (-1..-24) @@ -14550,7 +14554,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } // ignore and store: the tag value is already in the head, so @@ -14572,7 +14576,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } case cbor_tag_handler_t::ignore: @@ -14630,25 +14634,25 @@ class binary_reader return sax->null(); case 0xF9: // Half-Precision Float (two-byte IEEE 754) - return get_half_float(input_format_t::cbor, false); + return get_half_float(false); case 0xFA: // Single-Precision Float (four-byte IEEE 754) { float number{}; - return get_number(input_format_t::cbor, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xFB: // Double-Precision Float (eight-byte IEEE 754) { double number{}; - return get_number(input_format_t::cbor, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } default: // anything else (0xFF is handled inside the other types) { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::cbor, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } } } @@ -14697,38 +14701,38 @@ class binary_reader case 0x76: case 0x77: { - return get_string(input_format_t::cbor, static_cast(current) & 0x1Fu, result); + return get_string(static_cast(current) & 0x1Fu, result); } case 0x78: // UTF-8 string (one-byte uint8_t for n follows) { std::uint8_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } case 0x79: // UTF-8 string (two-byte uint16_t for n follow) { std::uint16_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } case 0x7A: // UTF-8 string (four-byte uint32_t for n follow) { std::uint32_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } case 0x7B: // UTF-8 string (eight-byte uint64_t for n follow) { std::uint64_t len{}; - return get_number(input_format_t::cbor, len) && get_string(input_format_t::cbor, len, result); + return get_number(len) && get_string(len, result); } default: { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::cbor, concat("expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x", last_token), "string"), nullptr)); + exception_message(concat("expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x", last_token), "string"), nullptr)); } } } @@ -14756,7 +14760,7 @@ class binary_reader while (true) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "string"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("string"))) { return false; } @@ -14865,7 +14869,7 @@ class binary_reader auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::cbor, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); + exception_message(concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); } /*! @@ -14912,42 +14916,42 @@ class binary_reader case 0x56: case 0x57: { - return get_binary(input_format_t::cbor, static_cast(current) & 0x1Fu, result); + return get_binary(static_cast(current) & 0x1Fu, result); } case 0x58: // Binary data (one-byte uint8_t for n follows) { std::uint8_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } case 0x59: // Binary data (two-byte uint16_t for n follow) { std::uint16_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } case 0x5A: // Binary data (four-byte uint32_t for n follow) { std::uint32_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } case 0x5B: // Binary data (eight-byte uint64_t for n follow) { std::uint64_t len{}; - return get_number(input_format_t::cbor, len) && - get_binary(input_format_t::cbor, len, result); + return get_number(len) && + get_binary(len, result); } default: { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::cbor, concat("expected length specification (0x40-0x5B) or indefinite binary array type (0x5F); last byte: 0x", last_token), "binary"), nullptr)); + exception_message(concat("expected length specification (0x40-0x5B) or indefinite binary array type (0x5F); last byte: 0x", last_token), "binary"), nullptr)); } } } @@ -14971,7 +14975,7 @@ class binary_reader while (true) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::cbor, "binary"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("binary"))) { return false; } @@ -15029,7 +15033,7 @@ class binary_reader case 0x18: // 1 byte { std::uint8_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -15040,7 +15044,7 @@ class binary_reader case 0x19: // 2 bytes { std::uint16_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -15051,7 +15055,7 @@ class binary_reader case 0x1A: // 4 bytes { std::uint32_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -15062,7 +15066,7 @@ class binary_reader case 0x1B: // 8 bytes { std::uint64_t n{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format_t::cbor, n))) + if (JSON_HEDLEY_UNLIKELY(!get_number(n))) { return false; } @@ -15095,7 +15099,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(!value_in_range_of(len) || len == detail::unknown_size())) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format_t::cbor, concat("excessive ", context, " size"), "size"), nullptr)); + exception_message(concat("excessive ", context, " size"), "size"), nullptr)); } result = conditional_static_cast(len); return true; @@ -15224,7 +15228,7 @@ class binary_reader { // EOF case char_traits::eof(): - return unexpect_eof(input_format_t::msgpack, "value"); + return unexpect_eof("value"); // positive fixint case 0x00: @@ -15464,85 +15468,85 @@ class binary_reader case 0xCA: // float 32 { float number{}; - return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xCB: // float 64 { double number{}; - return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xCC: // uint 8 { std::uint8_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xCD: // uint 16 { std::uint16_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xCE: // uint 32 { std::uint32_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xCF: // uint 64 { std::uint64_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 0xD0: // int 8 { std::int8_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xD1: // int 16 { std::int16_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xD2: // int 32 { std::int32_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xD3: // int 64 { std::int64_t number{}; - return get_number(input_format_t::msgpack, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 0xDC: // array 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && enter_array(static_cast(len)); + return get_number(len) && enter_array(static_cast(len)); } case 0xDD: // array 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && enter_array(conditional_static_cast(len)); + return get_number(len) && enter_array(conditional_static_cast(len)); } case 0xDE: // map 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && enter_object(static_cast(len)); + return get_number(len) && enter_object(static_cast(len)); } case 0xDF: // map 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && enter_object(conditional_static_cast(len)); + return get_number(len) && enter_object(conditional_static_cast(len)); } // negative fixint @@ -15584,7 +15588,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::msgpack, concat("invalid byte: 0x", last_token), "value"), nullptr)); + exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr)); } } } @@ -15601,7 +15605,7 @@ class binary_reader */ bool get_msgpack_string(string_t& result, const char* context = "string") { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::msgpack, "string"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("string"))) { return false; } @@ -15642,32 +15646,32 @@ class binary_reader case 0xBE: case 0xBF: { - return get_string(input_format_t::msgpack, static_cast(current) & 0x1Fu, result) && check_string_utf8(result, context); + return get_string(static_cast(current) & 0x1Fu, result) && check_string_utf8(result, context); } case 0xD9: // str 8 { std::uint8_t len{}; - return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 0xDA: // str 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 0xDB: // str 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && get_string(input_format_t::msgpack, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } default: { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::msgpack, concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", last_token), "string"), nullptr)); + exception_message(concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", last_token), "string"), nullptr)); } } } @@ -15761,7 +15765,7 @@ class binary_reader auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format_t::msgpack, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); + exception_message(concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr)); } /*! @@ -15788,31 +15792,31 @@ class binary_reader case 0xC4: // bin 8 { std::uint8_t len{}; - return get_number(input_format_t::msgpack, len) && - get_binary(input_format_t::msgpack, len, result); + return get_number(len) && + get_binary(len, result); } case 0xC5: // bin 16 { std::uint16_t len{}; - return get_number(input_format_t::msgpack, len) && - get_binary(input_format_t::msgpack, len, result); + return get_number(len) && + get_binary(len, result); } case 0xC6: // bin 32 { std::uint32_t len{}; - return get_number(input_format_t::msgpack, len) && - get_binary(input_format_t::msgpack, len, result); + return get_number(len) && + get_binary(len, result); } case 0xC7: // ext 8 { std::uint8_t len{}; std::int8_t subtype{}; - return get_number(input_format_t::msgpack, len) && - get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, len, result) && + return get_number(len) && + get_number(subtype) && + get_binary(len, result) && assign_and_return_true(subtype); } @@ -15820,9 +15824,9 @@ class binary_reader { std::uint16_t len{}; std::int8_t subtype{}; - return get_number(input_format_t::msgpack, len) && - get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, len, result) && + return get_number(len) && + get_number(subtype) && + get_binary(len, result) && assign_and_return_true(subtype); } @@ -15830,49 +15834,49 @@ class binary_reader { std::uint32_t len{}; std::int8_t subtype{}; - return get_number(input_format_t::msgpack, len) && - get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, len, result) && + return get_number(len) && + get_number(subtype) && + get_binary(len, result) && assign_and_return_true(subtype); } case 0xD4: // fixext 1 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 1, result) && + return get_number(subtype) && + get_binary(1, result) && assign_and_return_true(subtype); } case 0xD5: // fixext 2 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 2, result) && + return get_number(subtype) && + get_binary(2, result) && assign_and_return_true(subtype); } case 0xD6: // fixext 4 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 4, result) && + return get_number(subtype) && + get_binary(4, result) && assign_and_return_true(subtype); } case 0xD7: // fixext 8 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 8, result) && + return get_number(subtype) && + get_binary(8, result) && assign_and_return_true(subtype); } case 0xD8: // fixext 16 { std::int8_t subtype{}; - return get_number(input_format_t::msgpack, subtype) && - get_binary(input_format_t::msgpack, 16, result) && + return get_number(subtype) && + get_binary(16, result) && assign_and_return_true(subtype); } @@ -16071,7 +16075,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(len < 0)) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, - exception_message(input_format, "string length must not be negative", "string"), nullptr)); + exception_message("string length must not be negative", "string"), nullptr)); } return true; } @@ -16101,7 +16105,7 @@ class binary_reader get(); } - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "value"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("value"))) { return false; } @@ -16111,31 +16115,31 @@ class binary_reader case 'U': { std::uint8_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 'i': { std::int8_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'I': { std::int16_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'l': { std::int32_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'L': { std::int64_t len{}; - return get_number(input_format, len) && check_ubjson_string_length(len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && check_ubjson_string_length(len) && get_string(len, result) && check_string_utf8(result, context); } case 'u': @@ -16145,7 +16149,7 @@ class binary_reader break; } std::uint16_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 'm': @@ -16155,7 +16159,7 @@ class binary_reader break; } std::uint32_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } case 'M': @@ -16165,7 +16169,7 @@ class binary_reader break; } std::uint64_t len{}; - return get_number(input_format, len) && get_string(input_format, len, result) && check_string_utf8(result, context); + return get_number(len) && get_string(len, result) && check_string_utf8(result, context); } default: @@ -16182,7 +16186,7 @@ class binary_reader { message = "expected length type specification (U, i, u, I, m, l, M, L); last byte: 0x" + last_token; } - return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format, message, "string"), nullptr)); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(message, "string"), nullptr)); } /*! @@ -16261,19 +16265,19 @@ class binary_reader bool get_ubjson_signed_count(std::size_t& result) { SignedType number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } if (JSON_HEDLEY_UNLIKELY(number < 0)) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, - exception_message(input_format, "count in an optimized container must be positive", "size"), nullptr)); + exception_message("count in an optimized container must be positive", "size"), nullptr)); } if (JSON_HEDLEY_UNLIKELY(!value_in_range_of(number))) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format, "integer value overflow", "size"), nullptr)); + exception_message("integer value overflow", "size"), nullptr)); } result = static_cast(number); // NOLINT(bugprone-signed-char-misuse,cert-str34-c): number is not a char return true; @@ -16302,7 +16306,7 @@ class binary_reader case 'U': { std::uint8_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } @@ -16329,7 +16333,7 @@ class binary_reader break; } std::uint16_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } @@ -16344,7 +16348,7 @@ class binary_reader break; } std::uint32_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } @@ -16359,14 +16363,14 @@ class binary_reader break; } std::uint64_t number{}; - if (JSON_HEDLEY_UNLIKELY(!get_number(input_format, number))) + if (JSON_HEDLEY_UNLIKELY(!get_number(number))) { return false; } if (!value_in_range_of(number)) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format, "integer value overflow", "size"), nullptr)); + exception_message("integer value overflow", "size"), nullptr)); } result = detail::conditional_static_cast(number); return true; @@ -16380,7 +16384,7 @@ class binary_reader } if (is_ndarray) // ndarray dimensional vector can only contain integers and cannot embed another array { - return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, exception_message(input_format, "ndarray dimensional vector is not allowed", "size"), nullptr)); + return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, exception_message("ndarray dimensional vector is not allowed", "size"), nullptr)); } std::vector dim; if (JSON_HEDLEY_UNLIKELY(!get_ubjson_ndarray_size(dim))) @@ -16420,7 +16424,7 @@ class binary_reader // or SIZE_MAX. if (JSON_HEDLEY_UNLIKELY(result > (std::numeric_limits::max)() / i)) { - return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr)); + return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message("excessive ndarray size caused overflow", "size"), nullptr)); } result *= i; // the pre-check above already rules out result becoming 0 @@ -16429,9 +16433,9 @@ class binary_reader // unknown-size container (see get_ubjson_size_type()) if (result == npos) { - return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr)); + return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message("excessive ndarray size caused overflow", "size"), nullptr)); } - if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast(i)))) + if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(i))) { return false; } @@ -16457,7 +16461,7 @@ class binary_reader { message = "expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x" + last_token; } - return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(input_format, message, "size"), nullptr)); + return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(message, "size"), nullptr)); } /*! @@ -16492,10 +16496,10 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, concat("marker 0x", last_token, " is not a permitted optimized array type"), "type"), nullptr)); + exception_message(concat("marker 0x", last_token, " is not a permitted optimized array type"), "type"), nullptr)); } - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "type"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("type"))) { return false; } @@ -16503,13 +16507,13 @@ class binary_reader get_ignore_noop(); if (JSON_HEDLEY_UNLIKELY(current != '#')) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "value"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("value"))) { return false; } auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, concat("expected '#' after type information; last byte: 0x", last_token), "size"), nullptr)); + exception_message(concat("expected '#' after type information; last byte: 0x", last_token), "size"), nullptr)); } const bool is_error = get_ubjson_size_value(result.first, is_ndarray); @@ -16529,7 +16533,7 @@ class binary_reader if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray) { return sax->parse_error(chars_read, get_token_string(), parse_error::create(112, chars_read, - exception_message(input_format, "ndarray requires both type and size", "size"), nullptr)); + exception_message("ndarray requires both type and size", "size"), nullptr)); } return is_error; } @@ -16546,7 +16550,7 @@ class binary_reader switch (prefix) { case char_traits::eof(): // EOF - return unexpect_eof(input_format, "value"); + return unexpect_eof("value"); case 'T': // true return sax->boolean(true); @@ -16563,37 +16567,37 @@ class binary_reader break; } std::uint8_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'U': { std::uint8_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'i': { std::int8_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'I': { std::int16_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'l': { std::int32_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'L': { std::int64_t number{}; - return get_number(input_format, number) && sax->number_integer(conditional_static_cast(number)); + return get_number(number) && emit_signed(number); } case 'u': @@ -16603,7 +16607,7 @@ class binary_reader break; } std::uint16_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'm': @@ -16613,7 +16617,7 @@ class binary_reader break; } std::uint32_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'M': @@ -16623,7 +16627,7 @@ class binary_reader break; } std::uint64_t number{}; - return get_number(input_format, number) && sax->number_unsigned(number); + return get_number(number) && emit_unsigned(number); } case 'h': @@ -16632,19 +16636,19 @@ class binary_reader { break; } - return get_half_float(input_format, true); + return get_half_float(true); } case 'd': { float number{}; - return get_number(input_format, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 'D': { double number{}; - return get_number(input_format, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 'H': @@ -16655,7 +16659,7 @@ class binary_reader case 'C': // char { get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "char"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("char"))) { return false; } @@ -16663,7 +16667,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format, concat("byte after 'C' must be in range 0x00..0x7F; last byte: 0x", last_token), "char"), nullptr)); + exception_message(concat("byte after 'C' must be in range 0x00..0x7F; last byte: 0x", last_token), "char"), nullptr)); } string_t s(1, static_cast(current)); return sax->string(s); @@ -16685,7 +16689,7 @@ class binary_reader break; } auto last_token = get_token_string(); - return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message(input_format, "invalid byte: 0x" + last_token, "value"), nullptr)); + return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message("invalid byte: 0x" + last_token, "value"), nullptr)); } /*! @@ -16711,7 +16715,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, "invalid byte: 0x" + last_token, "type"), nullptr)); + exception_message("invalid byte: 0x" + last_token, "type"), nullptr)); } string_t type = type_name; // sax->string() takes a reference @@ -16746,7 +16750,7 @@ class binary_reader if (input_format == input_format_t::bjdata && size_and_type.first != npos && size_and_type.second == 'B') { binary_t result; - return get_binary(input_format, size_and_type.first, result) && sax->binary(result); + return get_binary(size_and_type.first, result) && sax->binary(result); } if (size_and_type.first != npos) @@ -16759,7 +16763,7 @@ class binary_reader && size_and_type.first > max_valueless_container_size)) { return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, - exception_message(input_format, "excessive array size", "size"), nullptr)); + exception_message("excessive array size", "size"), nullptr)); } if (JSON_HEDLEY_UNLIKELY(!enter_array(size_and_type.first, size_and_type.second))) @@ -16796,7 +16800,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format, "BJData object does not support ND-array size in optimized format", "object"), nullptr)); + exception_message("BJData object does not support ND-array size in optimized format", "object"), nullptr)); } if (size_and_type.first != npos) @@ -16826,7 +16830,7 @@ class binary_reader for (std::size_t i = 0; i < size; ++i) { get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } @@ -16845,7 +16849,7 @@ class binary_reader if (JSON_HEDLEY_UNLIKELY(result_remainder != token_type::end_of_input)) { return sax->parse_error(chars_read, number_string, parse_error::create(115, chars_read, - exception_message(input_format, concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); + exception_message(concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); } switch (result_number) @@ -16885,7 +16889,7 @@ class binary_reader case token_type::literal_or_value: default: return sax->parse_error(chars_read, number_string, parse_error::create(115, chars_read, - exception_message(input_format, concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); + exception_message(concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr)); } } @@ -16954,7 +16958,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, - exception_message(input_format_t::bon8, concat(detail, ": 0x", last_token), context), nullptr)); + exception_message(concat(detail, ": 0x", last_token), context), nullptr)); } /*! @@ -17058,7 +17062,7 @@ class binary_reader if (byte == char_traits::eof()) { - return unexpect_eof(input_format_t::bon8, "value"); + return unexpect_eof("value"); } // string: ASCII character @@ -17095,25 +17099,25 @@ class binary_reader case 0x8C: // int32 { std::int32_t number{}; - return get_number(input_format_t::bon8, number) && emit_bon8_integer(number); + return get_number(number) && emit_bon8_integer(number); } case 0x8D: // int64 { std::int64_t number{}; - return get_number(input_format_t::bon8, number) && emit_bon8_integer(number); + return get_number(number) && emit_bon8_integer(number); } case 0x8E: // binary32 { float number{}; - return get_number(input_format_t::bon8, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0x8F: // binary64 { double number{}; - return get_number(input_format_t::bon8, number) && sax->number_float(static_cast(number), ""); + return get_number(number) && emit_float(number); } case 0xF8: @@ -17179,7 +17183,9 @@ class binary_reader @brief pass an integer to the SAX parser Non-negative integers are passed as unsigned, negative integers as signed - numbers, like the other binary formats do. + numbers, like the other binary formats do. A value that does not fit the + number type is passed as described for @ref emit_unsigned and + @ref emit_signed. @param[in] number the integer @return whether the SAX parser accepted the value @@ -17188,9 +17194,9 @@ class binary_reader { if (number >= 0) { - return sax->number_unsigned(static_cast(number)); + return emit_unsigned(static_cast(number)); } - return sax->number_integer(static_cast(number)); + return emit_signed(number); } /*! @@ -17210,7 +17216,7 @@ class binary_reader */ bool get_bon8_integer(const char_int_type lead, const char_int_type second) { - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(input_format_t::bon8, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } @@ -17242,13 +17248,12 @@ class binary_reader { if (JSON_HEDLEY_UNLIKELY(get_bon8() == char_traits::eof())) { - return unexpect_eof(input_format_t::bon8, "number"); + return unexpect_eof("number"); } value = (value << 8) | static_cast(current); } - return negative ? sax->number_integer(static_cast(-(value + offset))) - : sax->number_unsigned(static_cast(value + offset)); + return emit_bon8_integer(negative ? -(value + offset) : value + offset); } /*! @@ -17266,7 +17271,7 @@ class binary_reader if (byte == char_traits::eof()) { - return unexpect_eof(input_format_t::bon8, "key"); + return unexpect_eof("key"); } if (byte == 0xFF) @@ -17286,7 +17291,7 @@ class binary_reader if (second == char_traits::eof()) { // the input ends inside a character or an integer - return unexpect_eof(input_format_t::bon8, "key"); + return unexpect_eof("key"); } unget_bon8(second); if (is_bon8_continuation(second)) @@ -17359,7 +17364,7 @@ class binary_reader if (byte == char_traits::eof()) { - return unexpect_eof(input_format_t::bon8, "string"); + return unexpect_eof("string"); } // end of string @@ -17390,7 +17395,7 @@ class binary_reader { // the input ends inside a character or an integer: either // way, the message is incomplete - return unexpect_eof(input_format_t::bon8, "string"); + return unexpect_eof("string"); } if (!is_bon8_continuation(second)) { @@ -17431,7 +17436,7 @@ class binary_reader { if (JSON_HEDLEY_UNLIKELY(get_bon8() == char_traits::eof())) { - return unexpect_eof(input_format_t::bon8, "string"); + return unexpect_eof("string"); } if (JSON_HEDLEY_UNLIKELY(!is_bon8_continuation(current))) { @@ -17470,7 +17475,7 @@ class binary_reader @return bool, whether the read was successful */ template - bool get_to(T& dest, const input_format_t format, const char* context) + bool get_to(T& dest, const char* context) { // false positive: new_chars_read is read on the next lines // @infer-ignore DEAD_STORE @@ -17480,7 +17485,7 @@ class binary_reader { // in case of failure, advance position by 1 to report the failing location ++chars_read; - sax->parse_error(chars_read, "", parse_error::create(110, chars_read, exception_message(format, "unexpected end of input", context), nullptr)); + sax->parse_error(chars_read, "", parse_error::create(110, chars_read, exception_message("unexpected end of input", context), nullptr)); return false; } return true; @@ -17531,7 +17536,6 @@ class binary_reader @brief read a number from the input @tparam NumberType the type of the number - @param[in] format the current format (for diagnostics) @param[out] result number of type @a NumberType @return whether conversion completed @@ -17543,42 +17547,119 @@ class binary_reader should reorder on big endian systems. */ template - bool get_number(const input_format_t format, NumberType& result) + bool get_number(NumberType& result) { // read in the original format - if (JSON_HEDLEY_UNLIKELY(!get_to(result, format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!get_to(result, "number"))) { return false; } - if (is_little_endian != (InputIsLittleEndian || format == input_format_t::bjdata)) + if (is_little_endian != (InputIsLittleEndian || input_format == input_format_t::bjdata)) { byte_swap(result); } return true; } + /*! + @brief pass a signed integer read from the input to the SAX parser + + Like the lexer does for JSON text, a value that does not fit into + number_integer_t is passed as number_unsigned_t if it is non-negative and + fits there, and as number_float_t otherwise. With the default number + types, every integer the binary formats can encode fits, so this only + matters for narrower custom number types. + + @tparam NumberType a signed integer type + @param[in] number the integer + @return whether the SAX parser accepted the value + + @throw out_of_range.406 if @a number overflows number_float_t (see + @ref emit_float) + */ + template + bool emit_signed(const NumberType number) + { + if (JSON_HEDLEY_LIKELY(value_in_range_of(number))) + { + return sax->number_integer(static_cast(number)); + } + if (value_in_range_of(number)) + { + return sax->number_unsigned(static_cast(number)); + } + return emit_float(number); + } + + /*! + @brief pass an unsigned integer read from the input to the SAX parser + + Like the lexer does for JSON text, a value that does not fit into + number_unsigned_t is passed as number_float_t. + + @tparam NumberType an unsigned integer type + @param[in] number the integer + @return whether the SAX parser accepted the value + + @throw out_of_range.406 if @a number overflows number_float_t (see + @ref emit_float) + */ + template + bool emit_unsigned(const NumberType number) + { + if (JSON_HEDLEY_LIKELY(value_in_range_of(number))) + { + return sax->number_unsigned(static_cast(number)); + } + return emit_float(number); + } + + /*! + @brief pass a floating-point number read from the input to the SAX parser + + Like the lexer does for JSON text, a finite value that overflows + number_float_t is rejected instead of silently becoming infinity. Infinity + and NaN in the input are passed on unchanged. Integers only overflow if + number_float_t cannot represent 2^64, e.g., a half-precision type. + + @tparam NumberType a floating-point or integer type + @param[in] number the number + @return whether the SAX parser accepted the value + + @throw out_of_range.406 if a finite @a number overflows number_float_t + */ + template + bool emit_float(const NumberType number) + { + const auto result = static_cast(number); + if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result))) + { + return sax->parse_error(chars_read, get_token_string(), + out_of_range::create(406, exception_message("number overflow", "value"), nullptr)); + } + return sax->number_float(result, ""); + } + /*! @brief read and decode an IEEE 754 half-precision (16-bit) float Used by CBOR (big endian) and BJData (little endian); the two formats only differ in the byte order of the two bytes that make up the half. - - @param[in] format the current format (for diagnostics) @param[in] little_endian whether the two bytes are little endian (BJData) or big endian (CBOR) @return whether reading and decoding succeeded */ - bool get_half_float(const input_format_t format, const bool little_endian) + bool get_half_float(const bool little_endian) { const auto byte1_raw = get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } const auto byte2_raw = get(); - if (JSON_HEDLEY_UNLIKELY(!unexpect_eof(format, "number"))) + if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("number"))) { return false; } @@ -17624,7 +17705,6 @@ class binary_reader @brief create a string by reading characters from the input @tparam NumberType the type of the number - @param[in] format the current format (for diagnostics) @param[in] len number of characters to read @param[out] result string created by reading @a len bytes @@ -17635,8 +17715,7 @@ class binary_reader the input before we run out of string memory. */ template - bool get_string(const input_format_t format, - const NumberType len, + bool get_string(const NumberType len, string_t& result) { // Strings are taken as is by default: none of CBOR (RFC 8949 ยง3.1 @@ -17648,7 +17727,7 @@ class binary_reader // @ref error_handler, applied once the whole string (all chunks of // an indefinite-length CBOR string included) has been assembled, by // @ref check_string_utf8 at the call site. - return get_bytes(format, len, "string", result); + return get_bytes(len, "string", result); } /*! @@ -17679,7 +17758,7 @@ class binary_reader { auto last_token = get_token_string(); return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, - exception_message(input_format, "invalid string: ill-formed UTF-8 byte", context), nullptr)); + exception_message("invalid string: ill-formed UTF-8 byte", context), nullptr)); } result = sanitize_utf8(result, error_handler); @@ -17690,7 +17769,6 @@ class binary_reader @brief create a byte array by reading bytes from the input @tparam NumberType the type of the number - @param[in] format the current format (for diagnostics) @param[in] len number of bytes to read @param[out] result byte array created by reading @a len bytes @@ -17701,11 +17779,10 @@ class binary_reader the input before we run out of memory. */ template - bool get_binary(const input_format_t format, - const NumberType len, + bool get_binary(const NumberType len, binary_t& result) { - return get_bytes(format, len, "binary", result); + return get_bytes(len, "binary", result); } /*! @@ -17713,7 +17790,6 @@ class binary_reader @tparam NumberType the type of the length @tparam ContainerType the destination container (string_t or binary_t) - @param[in] format the current format (for diagnostics) @param[in] len number of bytes to read @param[in] context further context information (for diagnostics) @param[out] result container the bytes are appended to @@ -17728,8 +17804,7 @@ class binary_reader detects a premature end of input. */ template - bool get_bytes(const input_format_t format, - NumberType len, + bool get_bytes(NumberType len, const char* context, ContainerType& result) { @@ -17760,7 +17835,7 @@ class binary_reader result.resize(old_size + bytes_read); ++chars_read; current = char_traits::eof(); - return unexpect_eof(format, context); + return unexpect_eof(context); } // a full chunk was read; get_elements() never returns more than requested JSON_ASSERT(bytes_read == wanted); @@ -17770,17 +17845,16 @@ class binary_reader } /*! - @param[in] format the current format (for diagnostics) @param[in] context further context information (for diagnostics) @return whether the last read character is not EOF */ - JSON_HEDLEY_NON_NULL(3) - bool unexpect_eof(const input_format_t format, const char* context) const + JSON_HEDLEY_NON_NULL(2) + bool unexpect_eof(const char* context) const { if (JSON_HEDLEY_UNLIKELY(current == char_traits::eof())) { return sax->parse_error(chars_read, "", - parse_error::create(110, chars_read, exception_message(format, "unexpected end of input", context), nullptr)); + parse_error::create(110, chars_read, exception_message("unexpected end of input", context), nullptr)); } return true; } @@ -17796,18 +17870,16 @@ class binary_reader } /*! - @param[in] format the current format @param[in] detail a detailed error message @param[in] context further context information @return a message string to use in the parse_error exceptions */ - std::string exception_message(const input_format_t format, - const std::string& detail, + std::string exception_message(const std::string& detail, const std::string& context) const { std::string error_msg = "syntax error while parsing "; - switch (format) + switch (input_format) { case input_format_t::cbor: error_msg += "CBOR"; diff --git a/tests/src/unit-binary_formats.cpp b/tests/src/unit-binary_formats.cpp index ed6d89911..846a092fb 100644 --- a/tests/src/unit-binary_formats.cpp +++ b/tests/src/unit-binary_formats.cpp @@ -11,7 +11,12 @@ #include using nlohmann::json; +#include #include +#include +#include +#include +#include #include "make_test_data_available.hpp" TEST_CASE("Binary Formats" * doctest::skip()) @@ -224,3 +229,139 @@ TEST_CASE("Binary Formats" * doctest::skip()) CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(89.450)); } } + +namespace +{ +// the binary formats as function pointers for "Binary formats with narrow number types"; +// named functions rather than lambdas, because clang 3.5 cannot convert a lambda +// to a function pointer in the braced initializer of the format table +using narrow_json = nlohmann::basic_json; +using bytes = std::vector; + +bytes encode_cbor(const json& j) +{ + return json::to_cbor(j); +} +narrow_json decode_cbor(const bytes& v, bool allow_exceptions) +{ + return narrow_json::from_cbor(v, true, allow_exceptions); +} + +bytes encode_msgpack(const json& j) +{ + return json::to_msgpack(j); +} +narrow_json decode_msgpack(const bytes& v, bool allow_exceptions) +{ + return narrow_json::from_msgpack(v, true, allow_exceptions); +} + +bytes encode_ubjson(const json& j) +{ + return json::to_ubjson(j); +} +narrow_json decode_ubjson(const bytes& v, bool allow_exceptions) +{ + return narrow_json::from_ubjson(v, true, allow_exceptions); +} + +bytes encode_bjdata(const json& j) +{ + return json::to_bjdata(j); +} +narrow_json decode_bjdata(const bytes& v, bool allow_exceptions) +{ + return narrow_json::from_bjdata(v, true, allow_exceptions); +} + +// BSON can only store numbers as object members +bytes encode_bson(const json& j) +{ + return json::to_bson(json{{"a", j}}); +} +narrow_json decode_bson(const bytes& v, bool allow_exceptions) +{ + const auto result = narrow_json::from_bson(v, true, allow_exceptions); + return result.is_discarded() ? result : result.at("a"); +} + +bytes encode_bon8(const json& j) +{ + return json::to_bon8(j); +} +narrow_json decode_bon8(const bytes& v, bool allow_exceptions) +{ + return narrow_json::from_bon8(v, true, allow_exceptions); +} + +} // namespace + +TEST_CASE("Binary formats with narrow number types") +{ + // Numbers that do not fit the number types are handled like the lexer + // handles them in JSON text: an integer that fits neither integer type is + // stored as a floating-point number, and a finite floating-point number + // that overflows number_float_t is rejected with out_of_range.406. + struct binary_format + { + const char* name; + bytes (*encode)(const json&); + narrow_json (*decode)(const bytes&, bool); + }; + + const std::vector formats = + { + {"CBOR", encode_cbor, decode_cbor}, + {"MessagePack", encode_msgpack, decode_msgpack}, + {"UBJSON", encode_ubjson, decode_ubjson}, + {"BJData", encode_bjdata, decode_bjdata}, + {"BSON", encode_bson, decode_bson}, + {"BON8", encode_bon8, decode_bon8}, + }; + + for (const auto& format : formats) + { + const std::string name = format.name; + INFO("format := ", name); + const auto roundtrip = [&format](const json & j) + { + return format.decode(format.encode(j), true); + }; + + // integers that fit keep their type + CHECK(roundtrip(json(-5)).is_number_integer()); + CHECK(roundtrip(json(-5)).get() == -5); + CHECK(roundtrip(json(3000000000u)).is_number_unsigned()); + CHECK(roundtrip(json(3000000000u)).get() == 3000000000u); + + // integers that fit neither integer type are stored as float + CHECK(roundtrip(json(5000000000u)).is_number_float()); + CHECK(roundtrip(json(5000000000u)).get() == 5000000000.0f); + if (name != "BON8") // BON8 cannot encode integers above INT64_MAX + { + CHECK(roundtrip(json(10000000000000000000u)).is_number_float()); + CHECK(roundtrip(json(10000000000000000000u)).get() == 10000000000000000000.0f); + } + CHECK(roundtrip(json(-3000000000LL)).is_number_float()); + CHECK(roundtrip(json(-3000000000LL)).get() == -3000000000.0f); + CHECK(roundtrip(json(-5000000000LL)).is_number_float()); + CHECK(roundtrip(json(-5000000000LL)).get() == -5000000000.0f); + + // floating-point numbers that fit + CHECK(roundtrip(json(1.5)).get() == 1.5f); + const auto just_above_max = std::nextafter(static_cast((std::numeric_limits::max)()), + std::numeric_limits::infinity()); + CHECK(roundtrip(json(just_above_max)).get() == (std::numeric_limits::max)()); + + // infinity and NaN are passed on + CHECK(std::isinf(roundtrip(json(std::numeric_limits::infinity())).get())); + CHECK(std::isnan(roundtrip(json(std::numeric_limits::quiet_NaN())).get())); + + // finite floating-point numbers that overflow number_float_t are rejected + const std::string message = "[json.exception.out_of_range.406] syntax error while parsing " + name + + " value: number overflow"; + CHECK_THROWS_WITH_AS(roundtrip(json(1e300)), message.c_str(), narrow_json::out_of_range&); + CHECK_THROWS_WITH_AS(roundtrip(json(-1e300)), message.c_str(), narrow_json::out_of_range&); + CHECK(format.decode(format.encode(json(1e300)), false).is_discarded()); + } +} diff --git a/tests/src/unit-cbor.cpp b/tests/src/unit-cbor.cpp index f3f4301e5..546e256b1 100644 --- a/tests/src/unit-cbor.cpp +++ b/tests/src/unit-cbor.cpp @@ -3185,7 +3185,8 @@ TEST_CASE("Tagged values") // CBOR encodes negative integers as: result = -1 - n // For type 0x3B, n is an 8-byte uint64_t. Valid range for n with // the default int64_t is [0, INT64_MAX], producing results in [INT64_MIN, -1]. - // When n > INT64_MAX, the result exceeds int64_t range and is rejected. + // When n > INT64_MAX, the result exceeds int64_t range and is stored + // as a floating-point number, as the lexer does for JSON text. SECTION("n = 0 is valid (result = -1)") { @@ -3206,33 +3207,34 @@ TEST_CASE("Tagged values") CHECK(result.get() == (std::numeric_limits::min)()); } - SECTION("n = INT64_MAX + 1 is rejected (overflow)") + SECTION("n = INT64_MAX + 1 is stored as float") { // n = INT64_MAX + 1 (0x8000000000000000) - // result = -1 - n = -9223372036854775809, which exceeds int64_t range + // result = -1 - n = -9223372036854775809, which exceeds int64_t range; + // the nearest double is -9223372036854775808.0 const std::vector input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}; - json _; - CHECK_THROWS_WITH_AS(_ = json::from_cbor(input), - "[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow", - json::parse_error); + const auto result = json::from_cbor(input); + CHECK(result.is_number_float()); + CHECK(result.get() == -9223372036854775808.0); + CHECK(result == json::parse("-9223372036854775809")); } - SECTION("n = UINT64_MAX is rejected (overflow)") + SECTION("n = UINT64_MAX is stored as float") { // n = UINT64_MAX (0xFFFFFFFFFFFFFFFF) // result = -1 - n = -18446744073709551616, which exceeds int64_t range const std::vector input = {0x3B, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; - json _; - CHECK_THROWS_WITH_AS(_ = json::from_cbor(input), - "[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow", - json::parse_error); + const auto result = json::from_cbor(input); + CHECK(result.is_number_float()); + CHECK(result.get() == -18446744073709551616.0); + CHECK(result == json::parse("-18446744073709551616")); } - SECTION("overflow with allow_exceptions=false returns discarded") + SECTION("overflow with allow_exceptions=false is not an error") { const std::vector input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}; const auto result = json::from_cbor(input, true, false); - CHECK(result.is_discarded()); + CHECK(result.is_number_float()); } }