mirror of
https://github.com/nlohmann/json.git
synced 2026-11-11 18:08:22 +00:00
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Niels Lohmann
|
017a3ce001 |
No files matched your search
@@ -67,3 +67,4 @@ updates:
|
||||
- dependency-name: mcp
|
||||
- dependency-name: boltons
|
||||
- dependency-name: "opentelemetry-*"
|
||||
- dependency-name: importlib-metadata
|
||||
@@ -391,9 +391,9 @@ httpx-sse==0.4.3 \
|
||||
idna==3.20 \
|
||||
--hash=sha256:a7db850025b95ded1eae8a46181a1a6c56c92c96f0e2b005d9ff8dc0210cab44 \
|
||||
--hash=sha256:ab7ae7122974553370f0bdb919e1a960b2cd1bc1ef0276416d896db81c14582c
|
||||
importlib-metadata==10.1.0 \
|
||||
--hash=sha256:82b04047fa09792c1a8cfeddfe08bec79e4b13c29c8469b0d3949860f61017e8 \
|
||||
--hash=sha256:cf45ce66a929c64e994185a809c04840d0501db17e663810d993d0f4829265c7
|
||||
importlib-metadata==8.7.1 \
|
||||
--hash=sha256:49fef1ae6440c182052f407c8d34a68f72efc36db9ca90dc0113398f2fdde8bb \
|
||||
--hash=sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151
|
||||
jsonschema==4.25.1 \
|
||||
--hash=sha256:3fba0169e345c7175110351d456342c364814cfcf3b964ba4587f22915230a63 \
|
||||
--hash=sha256:e4a9655ce0da0c0b67a085847e00a3a51449e1157f4f75e9fb5aa545e122eb85
|
||||
|
||||
@@ -89,10 +89,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
|
||||
- Throws [out_of_range.406](../../home/exceptions.md#jsonexceptionout_of_range406) if a high-precision number, or a
|
||||
float64 when `number_float_t` is `float`, does not fit into `number_float_t`
|
||||
- Throws [out_of_range.408](../../home/exceptions.md#jsonexceptionout_of_range408) if the count of an optimized
|
||||
container exceeds the capacity of the container (or of `std::size_t`; an array of type `N` is empty, so only the
|
||||
latter applies to it), if an optimized array of `Z`, `T`, or `F` has more than 1,048,576 elements, or if the
|
||||
optimized arrays of `Z`, `T`, and `F` in the input together have more than 1,048,576 elements plus 4 for every byte
|
||||
read so far
|
||||
container exceeds the capacity of the container (or of `std::size_t`), or if an optimized array of `Z`, `T`, or `F` has
|
||||
more than 1,048,576 elements
|
||||
|
||||
## Complexity
|
||||
|
||||
@@ -134,9 +132,6 @@ Linear in the size of the input.
|
||||
- Extended container support (1) to include types with lvalue-only ADL `begin`/`end` (matching `std::begin`/`std::end` semantics) in version 3.13.0.
|
||||
- Extended overload (2) to accept heterogeneous iterator+sentinel pairs (C++20 ranges support) in version 3.13.0.
|
||||
- Added `error_handler` parameter in version 3.13.0.
|
||||
- Report `start_array(0)` to a SAX handler for an optimized array of type `N` (`[$N#n`), whatever its count, in version
|
||||
3.13.0.
|
||||
- Limited the total number of elements of optimized arrays of `Z`, `T`, and `F` in one input in version 3.13.0.
|
||||
|
||||
!!! warning "Deprecation"
|
||||
|
||||
|
||||
@@ -117,8 +117,5 @@ Linear in the size of the JSON value `j`.
|
||||
- Added `error_handler` parameter in version 3.13.0. Its default, `keep`, writes the bytes of a string or object key
|
||||
that is not valid UTF-8 unchanged, as before; `strict` (the default if
|
||||
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316`.
|
||||
- Writes arrays of `Z`, `T`, or `F` without the `$` type annotation (`[#n Z Z ...`) once their elements in one call
|
||||
add up to more than 1,048,576, in version 3.13.0, so that `from_ubjson` can read the output back. Such output is
|
||||
larger: 270,000 rows of 100 `null` values take 26.8 MB instead of 1.35 MB.
|
||||
- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
|
||||
array or object was silently skipped, producing invalid UBJSON.
|
||||
@@ -50,8 +50,8 @@ flowchart LR
|
||||
enabled explicitly. Invalid UTF-8 is rejected.
|
||||
- **Errors are reported, not ignored.** Malformed input results in a [`parse_error`](../home/exceptions.md#parse-errors)
|
||||
with the byte position of the error. Binary readers do not trust announced sizes: strings and binary values grow
|
||||
only as bytes are actually read, arrays reserve at most a fixed number of elements up front (and all of them
|
||||
together no more than the input read so far can back), and sizes that no container can hold are rejected.
|
||||
only as bytes are actually read, arrays reserve at most a fixed number of elements up front, and sizes that no
|
||||
container can hold are rejected.
|
||||
- **Memory is owned by values.** Each `basic_json` value owns its content, and there is no manual memory management in
|
||||
user code. The destructor does not recurse, so destroying a deeply nested value does not exhaust the stack.
|
||||
- **Bounded recursion.** The JSON parser and the binary readers keep their state in explicit stacks instead of
|
||||
|
||||
@@ -122,18 +122,9 @@ The library uses the following mapping from JSON values types to UBJSON types ac
|
||||
already is the value. Its declared count is therefore the only thing that decides how much memory the receiving side
|
||||
allocates, and a handful of bytes can describe billions of elements. `from_ubjson` rejects such an array with
|
||||
[`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408) when the count exceeds 1,048,576
|
||||
(`1 << 20`), or when the counts of all such arrays in the input add up to more than 1,048,576 plus 4 for every byte
|
||||
read so far. `to_ubjson` therefore writes an array of these types in the counted but untyped form when it is longer
|
||||
than that, or when it would bring the total of the typed arrays written so far above 1,048,576: the `$` type marker
|
||||
is dropped, the `#` count is kept, and every element is written with its own marker (`[#l 1048577 Z Z ...`), so
|
||||
everything it produces can be read back. Only input from another encoder, or written by version 3.12.0 and earlier of
|
||||
this library, can exceed the limit: for example, 1,100 rows of 1,000 `null` values or 210,000 rows of 25, each with
|
||||
a `$Z` annotation. Version 3.12.0 writes such a row in 5 bytes (6 above 255 elements), so rows of 20 or fewer such
|
||||
values are never rejected. Objects and untyped counted arrays are not subject to this limit.
|
||||
|
||||
An optimized array of type `N` (no-op) is empty whatever its count is, because a no-op is not a value. Its count is
|
||||
not passed on: `sax_parse` reports `start_array(0)` (versions before 3.13.0 reported the declared count), and the DOM
|
||||
parser reserves nothing for it.
|
||||
(`1 << 20`), and `to_ubjson` writes longer arrays of these types in the counted but untyped form: the `$` type marker is
|
||||
dropped, the `#` count is kept, and every element is written with its own marker (`[#l 1048577 Z Z ...`), so any value
|
||||
it produces can be read back. Objects and untyped counted arrays are not subject to this limit.
|
||||
|
||||
!!! info "Binary values"
|
||||
|
||||
@@ -189,11 +180,10 @@ The library maps UBJSON types to JSON value types as follows:
|
||||
|
||||
Every well-formed UBJSON value can be converted to a JSON value, with these exceptions: a high-precision number
|
||||
outside the range of `number_float_t` ([`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406)),
|
||||
optimized arrays of `Z`, `T`, or `F` with more elements than the limit allows (1,048,576 plus 4 per byte read),
|
||||
and a count that exceeds the capacity of the container
|
||||
([`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408)). High-precision numbers that do not
|
||||
fit an integer type are stored as floating-point numbers and may lose precision. Ill-formed input is rejected with a
|
||||
parse error.
|
||||
an optimized array of `Z`, `T`, or `F` with more than 1,048,576 elements, and a count that exceeds the capacity of the
|
||||
container ([`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408)). High-precision numbers that
|
||||
do not fit an integer type are stored as floating-point numbers and may lose precision. Ill-formed input is rejected
|
||||
with a parse error.
|
||||
|
||||
!!! info "Duplicate object keys"
|
||||
|
||||
|
||||
@@ -71,32 +71,11 @@ For the binary formats, an announced size is never trusted outright:
|
||||
actually consumed, rather than allocating the announced length up front — a truncated input runs out of bytes
|
||||
(reported as a parse error) instead of triggering an oversized allocation.
|
||||
- When an array announces its number of elements and the array container supports `reserve()` (as `#!cpp std::vector`,
|
||||
the default, does), the DOM parser reserves storage for them before the elements are read, but for at most 16384
|
||||
elements per array, regardless of how large the announced count is; further elements still grow the container
|
||||
normally as they are read. The arrays that are open at the same time hold their storage at the same time, and an
|
||||
announcement is not backed by input when it is read, so the elements reserved for all arrays of one input together
|
||||
may not exceed 65,536 plus 2 for every byte of input read so far; an array that announces more gets what is left
|
||||
of that and grows as its elements arrive. An element takes at least one byte of input, so the elements of the
|
||||
arrays that were read use up what was reserved for them, and only arrays that are nested deeply, with an array as
|
||||
their first element, can announce more than is left. For a truncated or hostile input, the storage that
|
||||
announcements can hold is therefore at most (65,536 + 2 per byte of input) values, that is about 1 MiB plus 32 bytes
|
||||
per byte of input with 16-byte values (a value is larger with `JSON_DIAGNOSTICS` or `JSON_DIAGNOSTIC_POSITIONS`, 24 to
|
||||
40 bytes, and so is the bound). For example, 100,000 nested CBOR array headers `99 40 00` (300,000 bytes) that
|
||||
announce 16384 elements each peaked at 23 MiB, against 1.65 GiB if every array reserved what it announced and 12 MiB
|
||||
with version 3.12.0, which reserved nothing.
|
||||
A [SAX](../../api/json_sax/index.md) handler is not affected: it is told the announced number of elements.
|
||||
the default, does), the library reserves storage for at most 16384 of them upfront, regardless of how large the
|
||||
announced count is; further elements still grow the container normally as they are read.
|
||||
- An announced array or object size that exceeds what the target container could ever hold (its `max_size()`) is
|
||||
rejected immediately as [`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408), without
|
||||
attempting to allocate anything.
|
||||
- A [UBJSON](../binary_formats/ubjson.md) array of the no-op type (`[$N#<n>`) is empty whatever count it declares: a
|
||||
SAX handler is told `start_array(0)` and the DOM parser reserves nothing.
|
||||
- A UBJSON array of type `Z`, `T` or `F` has no payload, so its announced count is not bounded by the input. One array
|
||||
may announce at most 1,048,576 elements, and all such arrays of one input together may contain at most 1,048,576
|
||||
elements plus 4 for every byte read so far (checked when each array's header has been read); beyond that the input is
|
||||
rejected as [`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408). For a DOM that bounds the memory these
|
||||
arrays can need to about 80 bytes per byte of input plus about 33 MB for the first 1,048,576 elements; a SAX handler
|
||||
receives at most that many events. Every other optimized array element, and every UBJSON object member (it has a
|
||||
key), takes at least one byte of input.
|
||||
|
||||
## Strings
|
||||
|
||||
|
||||
@@ -1050,17 +1050,11 @@ The size of an array or object in a [binary format](../features/binary_formats/i
|
||||
the size following `#` for [UBJSON](../features/binary_formats/ubjson.md)/[BJData](../features/binary_formats/bjdata.md),
|
||||
or the encoded length for [CBOR](../features/binary_formats/cbor.md).
|
||||
|
||||
The exception is also thrown for [UBJSON](../features/binary_formats/ubjson.md) arrays of a type that is encoded by
|
||||
its marker alone (`Z`, `T` or `F`). Such an array has no payload, so its count alone decides how much memory is
|
||||
allocated, and a handful of bytes would otherwise describe billions of values. It is thrown if the declared count of one
|
||||
array exceeds 1,048,576 (`1 << 20`), and if the counts of all such arrays read so far in one input add up to more than
|
||||
1,048,576 plus 4 for every byte of the input read when the header of the last array has been read.
|
||||
[`to_ubjson`](../api/basic_json/to_ubjson.md) writes an array of these types without the `$` type annotation (the `#`
|
||||
count and one marker per element are kept) when it is longer, or when it would bring the total of the typed arrays
|
||||
written so far above 1,048,576, so everything it produces can be read back. Only input written by another encoder, or
|
||||
by version 3.12.0 and earlier of this library, can exceed the limit: for example, 1,100 rows of 1,000 `null` values or
|
||||
210,000 rows of 25, each with a `$Z` annotation. Version 3.12.0 writes such a row in 5 bytes (6 above 255 elements),
|
||||
so rows of 20 or fewer such values are never rejected.
|
||||
The exception is also thrown for a [UBJSON](../features/binary_formats/ubjson.md) array of a type that is encoded by its
|
||||
marker alone (`Z`, `T` or `F`) whose declared count exceeds 1,048,576 (`1 << 20`). Such an array has no payload, so its
|
||||
count alone decides how much memory is allocated, and a handful of bytes would otherwise describe billions of values.
|
||||
[`to_ubjson`](../api/basic_json/to_ubjson.md) writes longer arrays of these types without the `$` type annotation (the `#`
|
||||
count and one marker per element are kept), so any value it produces can still be read back.
|
||||
|
||||
!!! failure "Example messages"
|
||||
|
||||
@@ -1076,9 +1070,6 @@ so rows of 20 or fewer such values are never rejected.
|
||||
```
|
||||
[json.exception.out_of_range.408] syntax error while parsing UBJSON size: excessive array size
|
||||
```
|
||||
```
|
||||
[json.exception.out_of_range.408] syntax error while parsing UBJSON size: excessive number of elements without payload
|
||||
```
|
||||
|
||||
### json.exception.out_of_range.409
|
||||
|
||||
|
||||
@@ -16,7 +16,6 @@
|
||||
#include <iterator> // back_inserter
|
||||
#include <limits> // numeric_limits
|
||||
#include <string> // char_traits, string
|
||||
#include <type_traits> // is_convertible, true_type, false_type
|
||||
#include <utility> // make_pair, move
|
||||
#include <vector> // vector
|
||||
#ifdef __cpp_lib_byteswap
|
||||
@@ -74,46 +73,13 @@ the input.
|
||||
|
||||
This is a sanity bound rather than a security boundary, and it is far above
|
||||
any container met in practice. @ref binary_writer falls back to the
|
||||
unoptimized encoding for longer containers, and for containers that together
|
||||
with the ones before them would exceed @ref valueless_elements_base, so that a
|
||||
value serialized by this library can always be read back.
|
||||
unoptimized encoding for longer containers, so that a value serialized by
|
||||
this library can always be read back.
|
||||
|
||||
@sa https://github.com/nlohmann/json/issues/2793
|
||||
*/
|
||||
JSON_INLINE_VARIABLE constexpr std::size_t max_valueless_container_size = 1 << 20;
|
||||
|
||||
/*!
|
||||
@brief number of elements of valueless UBJSON containers accepted on top of
|
||||
@ref valueless_elements_per_byte
|
||||
|
||||
The limit @ref max_valueless_container_size applies to each container on its
|
||||
own, so on its own it lets 9 bytes announce 1048576 values, and every further
|
||||
9 bytes another 1048576. The elements of all containers of a valueless type
|
||||
in one input are therefore added up, and the sum must stay below this base
|
||||
plus @ref valueless_elements_per_byte for every byte read so far. The base is
|
||||
the per-container limit, so a container that was accepted on its own before
|
||||
the sum was introduced is still accepted at the start of an input.
|
||||
*/
|
||||
JSON_INLINE_VARIABLE constexpr std::size_t valueless_elements_base = max_valueless_container_size;
|
||||
|
||||
/*!
|
||||
@brief number of elements of valueless UBJSON containers accepted per byte of
|
||||
input, in addition to @ref valueless_elements_base
|
||||
|
||||
An element of a value that has a payload costs at least one byte of input.
|
||||
Valueless elements are accepted at a rate of 4 per byte. A value takes 16
|
||||
bytes of memory in a DOM, so these arrays add about 80 bytes of memory per
|
||||
byte of input (measured: 77; one `Z` per byte needs about 32), beyond what the
|
||||
same bytes cost anyway, plus the base. This bounds the arrays of valueless
|
||||
type, not the memory of a whole document. Containers written by
|
||||
@ref binary_writer never need more than the base, because it writes an array
|
||||
in the unoptimized form once the typed arrays written so far and this one add
|
||||
up to more than that. The rate matters for input written by other encoders and
|
||||
by versions before 3.13.0 only; in the data this was measured on, no input has
|
||||
more than 0.17 elements per byte.
|
||||
*/
|
||||
JSON_INLINE_VARIABLE constexpr std::size_t valueless_elements_per_byte = 4;
|
||||
|
||||
///////////////////
|
||||
// binary reader //
|
||||
///////////////////
|
||||
@@ -181,8 +147,6 @@ class binary_reader
|
||||
sax = sax_;
|
||||
container_stack.clear();
|
||||
bon8_pushback_size = 0;
|
||||
// the position is only known to the DOM parser while this reader lives
|
||||
const position_binding binding(bound_parser(sax, std::is_convertible<SAX*, json_sax_dom_error_state*> {}), &chars_read);
|
||||
bool result = false;
|
||||
|
||||
switch (input_format)
|
||||
@@ -240,61 +204,6 @@ class binary_reader
|
||||
}
|
||||
|
||||
private:
|
||||
/// the DOM parser of @a s whose reserved storage is bounded by the input
|
||||
/// read so far; nullptr for any other SAX parser, which does not reserve,
|
||||
/// and for one that derives from a DOM parser privately or protectedly
|
||||
/// (or ambiguously), which the reader cannot reach
|
||||
static json_sax_dom_error_state* bound_parser(SAX* s, std::true_type /*is a DOM parser*/) noexcept
|
||||
{
|
||||
return s;
|
||||
}
|
||||
|
||||
static json_sax_dom_error_state* bound_parser(SAX* /*s*/, std::false_type /*is a DOM parser*/) noexcept
|
||||
{
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief tells a DOM parser where to find the number of characters read
|
||||
while sax_parse() runs, and puts back what it knew before when
|
||||
sax_parse() returns or throws
|
||||
|
||||
The reader is a local of the caller and is gone then, so the parser must
|
||||
not keep its position. A parser that was bound by a reader that is still
|
||||
running (a parse inside a parse) gets that binding back; that reader is
|
||||
still alive, as it encloses this one.
|
||||
*/
|
||||
class position_binding
|
||||
{
|
||||
public:
|
||||
position_binding(json_sax_dom_error_state* parser_, const std::size_t* position) noexcept
|
||||
: parser(parser_), saved()
|
||||
{
|
||||
if (parser != nullptr)
|
||||
{
|
||||
saved = parser->get_reserve_state();
|
||||
parser->set_input_position(position);
|
||||
}
|
||||
}
|
||||
|
||||
position_binding(const position_binding&) = delete;
|
||||
position_binding(position_binding&&) = delete;
|
||||
position_binding& operator=(const position_binding&) = delete;
|
||||
position_binding& operator=(position_binding&&) = delete;
|
||||
|
||||
~position_binding()
|
||||
{
|
||||
if (parser != nullptr)
|
||||
{
|
||||
parser->set_reserve_state(saved);
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
json_sax_dom_error_state* parser;
|
||||
json_sax_dom_error_state::reserve_state saved;
|
||||
};
|
||||
|
||||
////////////////////////
|
||||
// nested containers //
|
||||
////////////////////////
|
||||
@@ -3791,37 +3700,6 @@ class binary_reader
|
||||
return emit_unsigned(number);
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief add the elements of a container of a valueless type to the total of
|
||||
the input and check the total against the input read so far
|
||||
|
||||
A container of type 'Z', 'T' or 'F' has no payload, so its elements are the
|
||||
only ones for which the input does not bound the count. The check is made
|
||||
once per container, not per element.
|
||||
|
||||
@param[in] count number of elements the container declares; at most
|
||||
@ref max_valueless_container_size
|
||||
@return false if the total is more than @ref valueless_elements_base plus
|
||||
@ref valueless_elements_per_byte for every byte read so far; the
|
||||
total is not changed then
|
||||
*/
|
||||
bool charge_valueless_elements(const std::size_t count) noexcept
|
||||
{
|
||||
// the limit saturates instead of overflowing when size_t is 32 bits
|
||||
// wide and a very large input has been read; valueless_elements
|
||||
// never exceeds the limit, so the difference cannot underflow
|
||||
constexpr std::size_t max_size = (std::numeric_limits<std::size_t>::max)();
|
||||
const std::size_t limit = chars_read <= (max_size - valueless_elements_base) / valueless_elements_per_byte
|
||||
? valueless_elements_base + (valueless_elements_per_byte * chars_read)
|
||||
: max_size;
|
||||
if (JSON_HEDLEY_UNLIKELY(count > limit - valueless_elements))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
valueless_elements += count;
|
||||
return true;
|
||||
}
|
||||
|
||||
/*!
|
||||
@return whether array creation completed
|
||||
*/
|
||||
@@ -3907,27 +3785,24 @@ class binary_reader
|
||||
// declared count alone decides how much is allocated; the check is
|
||||
// made before the start event so that no container is opened that
|
||||
// is then abandoned. See @ref max_valueless_container_size.
|
||||
if (JSON_HEDLEY_UNLIKELY(size_and_type.second == 'Z' || size_and_type.second == 'T' || size_and_type.second == 'F'))
|
||||
if (JSON_HEDLEY_UNLIKELY((size_and_type.second == 'Z' || size_and_type.second == 'T' || size_and_type.second == 'F')
|
||||
&& size_and_type.first > max_valueless_container_size))
|
||||
{
|
||||
if (JSON_HEDLEY_UNLIKELY(size_and_type.first > max_valueless_container_size))
|
||||
{
|
||||
return out_of_range_error(exception_id::container_too_large, "excessive array size", "size");
|
||||
}
|
||||
if (JSON_HEDLEY_UNLIKELY(!charge_valueless_elements(size_and_type.first)))
|
||||
{
|
||||
return out_of_range_error(exception_id::container_too_large, "excessive number of elements without payload", "size");
|
||||
}
|
||||
return out_of_range_error(exception_id::container_too_large, "excessive array size", "size");
|
||||
}
|
||||
|
||||
// a no-op is not a value, so an array of them is empty whatever
|
||||
// count it declares: the SAX parser is told so, instead of being
|
||||
// made to reserve storage for a count that no input backs
|
||||
const std::size_t announced = size_and_type.second == 'N' ? 0 : size_and_type.first;
|
||||
if (JSON_HEDLEY_UNLIKELY(!enter_array(announced, size_and_type.second)))
|
||||
if (JSON_HEDLEY_UNLIKELY(!enter_array(size_and_type.first, size_and_type.second)))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
if (size_and_type.second == 'N')
|
||||
{
|
||||
// a no-op is not a value, so a container of them holds none;
|
||||
// the declared size has already been passed to the SAX parser
|
||||
container_stack.back().remaining = 0;
|
||||
}
|
||||
|
||||
return size_and_type.second == 0
|
||||
|| parse_ubjson_typed_run(container_stack.back().remaining, size_and_type.second, std::integral_constant<bool, bulk_scan> {});
|
||||
}
|
||||
@@ -5239,10 +5114,6 @@ class binary_reader
|
||||
/// the number of characters read
|
||||
std::size_t chars_read = 0;
|
||||
|
||||
/// UBJSON: the sum of the element counts of the containers of a valueless
|
||||
/// type read so far; see @ref charge_valueless_elements
|
||||
std::size_t valueless_elements = 0;
|
||||
|
||||
/// whether we can assume little endianness
|
||||
const bool is_little_endian = little_endianness();
|
||||
|
||||
|
||||
@@ -159,40 +159,6 @@ constexpr std::size_t unknown_size()
|
||||
return (std::numeric_limits<std::size_t>::max)();
|
||||
}
|
||||
|
||||
/// the largest number of elements the DOM parsers reserve for one array upfront
|
||||
constexpr std::size_t reserve_cap()
|
||||
{
|
||||
return 16384;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief number of array elements a DOM parser may reserve upfront on top of
|
||||
@ref reserved_elements_per_byte
|
||||
|
||||
An array that announces its size is given storage for that many elements when
|
||||
it opens (at most @ref reserve_cap of them), before any input backs the
|
||||
announcement, and the arrays that are open at the same time keep their storage
|
||||
until the input is complete: 100000 nested headers of three bytes each that
|
||||
announce 16384 elements would otherwise need 1.7 GB. The elements reserved for
|
||||
all arrays of an input are therefore added up, and the sum must stay below
|
||||
this base plus @ref reserved_elements_per_byte for every byte read so far; an
|
||||
array that would go beyond reserves what is left, and grows as its elements
|
||||
arrive. The base is four arrays of the maximum size, so the first nested
|
||||
arrays of an input are not affected.
|
||||
*/
|
||||
JSON_INLINE_VARIABLE constexpr std::size_t reserved_elements_base = 4 * reserve_cap();
|
||||
|
||||
/*!
|
||||
@brief number of array elements a DOM parser may reserve upfront per byte of
|
||||
input, in addition to @ref reserved_elements_base
|
||||
|
||||
An element of an array takes at least one byte of input, so an array that
|
||||
completes has used what was reserved for it. At a rate of 2 elements per
|
||||
byte, an array that announces more than the input provides keeps at most
|
||||
32 bytes of memory per byte of input, plus the base, until the parse fails.
|
||||
*/
|
||||
JSON_INLINE_VARIABLE constexpr std::size_t reserved_elements_per_byte = 2;
|
||||
|
||||
/*!
|
||||
@brief reserve capacity for @a len elements in array @a arr
|
||||
|
||||
@@ -208,7 +174,8 @@ template<typename ArrayType>
|
||||
auto reserve_array(ArrayType& arr, std::size_t len, priority_tag<1> /*unused*/)
|
||||
-> decltype(arr.reserve(len), void())
|
||||
{
|
||||
arr.reserve((std::min)(len, reserve_cap()));
|
||||
constexpr std::size_t reserve_cap = 16384;
|
||||
arr.reserve((std::min)(len, reserve_cap));
|
||||
}
|
||||
|
||||
template<typename ArrayType>
|
||||
@@ -375,13 +342,6 @@ struct diagnostic_positions
|
||||
Holds whether a parse error occurred and whether errors are thrown, and
|
||||
implements the parse_error() and is_errored() members of the SAX interface for
|
||||
json_sax_dom_parser and json_sax_dom_callback_parser.
|
||||
|
||||
It also bounds what the parsers reserve for arrays that announce their size
|
||||
(reserve_count()): the announced size is not backed by input yet when the
|
||||
array opens, and the arrays that are open at the same time each keep what was
|
||||
reserved for them, so the binary_reader tells the parser where to find the
|
||||
number of characters it has read (set_input_position()), which the bound
|
||||
is derived from.
|
||||
*/
|
||||
class json_sax_dom_error_state
|
||||
{
|
||||
@@ -415,103 +375,7 @@ class json_sax_dom_error_state
|
||||
return errored;
|
||||
}
|
||||
|
||||
/// bind the number of characters the input has provided so far; see
|
||||
/// @ref reserve_count
|
||||
/// (nullptr: not known, which bounds only the size of each array)
|
||||
void set_input_position(const std::size_t* position) noexcept
|
||||
{
|
||||
input_position = position;
|
||||
credit_position = 0;
|
||||
reserve_credit = position != nullptr ? reserved_elements_base : (std::numeric_limits<std::size_t>::max)();
|
||||
}
|
||||
|
||||
/// what set_input_position() changes, so that a binding can be undone
|
||||
struct reserve_state
|
||||
{
|
||||
const std::size_t* position;
|
||||
std::size_t position_at_refill;
|
||||
std::size_t credit;
|
||||
};
|
||||
|
||||
reserve_state get_reserve_state() const noexcept
|
||||
{
|
||||
return {input_position, credit_position, reserve_credit};
|
||||
}
|
||||
|
||||
void set_reserve_state(const reserve_state& state) noexcept
|
||||
{
|
||||
input_position = state.position;
|
||||
credit_position = state.position_at_refill;
|
||||
reserve_credit = state.credit;
|
||||
}
|
||||
|
||||
protected:
|
||||
/*!
|
||||
@brief the number of elements to reserve for an array that announces @a len
|
||||
|
||||
The announced number is not backed by input when the array opens, and the
|
||||
arrays that are open at the same time each keep what was reserved for
|
||||
them, so what is reserved for all arrays together is bounded by the
|
||||
input read so far: @ref reserved_elements_base plus
|
||||
@ref reserved_elements_per_byte for every character. An array that
|
||||
announces more than what is left of that is given what is left, and
|
||||
grows as its elements arrive. Without the position of the input (the
|
||||
parser is not driven by a binary_reader that is running) nothing is
|
||||
bounded but the cap of reserve_array().
|
||||
|
||||
The common case, an array that announces no more than the credit left,
|
||||
costs a comparison and a subtraction. It is charged what it announces, which
|
||||
for an array of more than reserve_cap() elements is more than reserve_array()
|
||||
reserves for it, which its own elements refill as they are read.
|
||||
*/
|
||||
std::size_t reserve_count(const std::size_t len) noexcept
|
||||
{
|
||||
if (JSON_HEDLEY_LIKELY(len <= reserve_credit))
|
||||
{
|
||||
reserve_credit -= len;
|
||||
return len;
|
||||
}
|
||||
return reserve_count_slowly(len);
|
||||
}
|
||||
|
||||
private:
|
||||
JSON_HEDLEY_NEVER_INLINE
|
||||
std::size_t reserve_count_slowly(const std::size_t len) noexcept
|
||||
{
|
||||
constexpr std::size_t max_size = (std::numeric_limits<std::size_t>::max)();
|
||||
if (input_position == nullptr)
|
||||
{
|
||||
reserve_credit = max_size;
|
||||
return len;
|
||||
}
|
||||
|
||||
// refill the credit with 'reserved_elements_per_byte' for every
|
||||
// character read since the last refill; it saturates instead of
|
||||
// overflowing when size_t is 32 bits wide and a very large input has
|
||||
// been read (the position goes back by up to two in BON8, which has
|
||||
// no sizes)
|
||||
const std::size_t position = *input_position;
|
||||
if (position > credit_position)
|
||||
{
|
||||
const std::size_t bytes = position - credit_position;
|
||||
reserve_credit = bytes <= (max_size - reserve_credit) / reserved_elements_per_byte
|
||||
? reserve_credit + (reserved_elements_per_byte * bytes)
|
||||
: max_size;
|
||||
credit_position = position;
|
||||
}
|
||||
|
||||
std::size_t count = (std::min)(len, reserve_cap());
|
||||
count = (std::min)(count, reserve_credit);
|
||||
reserve_credit -= count;
|
||||
return count;
|
||||
}
|
||||
|
||||
/// the number of characters the input has provided so far, if known
|
||||
const std::size_t* input_position = nullptr;
|
||||
/// the number of characters the input had provided when the credit was refilled
|
||||
std::size_t credit_position = 0;
|
||||
/// the number of elements that may still be reserved
|
||||
std::size_t reserve_credit = reserved_elements_base;
|
||||
/// whether a syntax error occurred
|
||||
bool errored = false;
|
||||
/// whether to throw exceptions in case of errors
|
||||
@@ -1346,7 +1210,7 @@ class json_sax_dom_parser : public json_sax_dom_error_state
|
||||
|
||||
if (len != detail::unknown_size())
|
||||
{
|
||||
reserve_array(*ref->m_data.m_value.array, reserve_count(len), priority_tag<1> {});
|
||||
reserve_array(*ref->m_data.m_value.array, len, priority_tag<1> {});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1694,7 +1558,7 @@ class json_sax_dom_binary_parser : public json_sax_dom_error_state
|
||||
|
||||
if (len != detail::unknown_size())
|
||||
{
|
||||
reserve_array(*ref_stack.back()->m_data.m_value.array, reserve_count(len), priority_tag<1> {});
|
||||
reserve_array(*ref_stack.back()->m_data.m_value.array, len, priority_tag<1> {});
|
||||
}
|
||||
|
||||
return true;
|
||||
@@ -1986,7 +1850,7 @@ class json_sax_dom_callback_parser : public json_sax_dom_error_state
|
||||
|
||||
if (len != detail::unknown_size())
|
||||
{
|
||||
reserve_array(*ref_stack.back()->m_data.m_value.array, reserve_count(len), priority_tag<1> {});
|
||||
reserve_array(*ref_stack.back()->m_data.m_value.array, len, priority_tag<1> {});
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1204,27 +1204,20 @@ class binary_writer
|
||||
|
||||
// an optimized array of a valueless type carries no payload, so a
|
||||
// reader has nothing but the declared count to bound the allocation
|
||||
// by. It refuses one array of more than max_valueless_container_size
|
||||
// elements, and all such arrays of one input together when they add
|
||||
// up to more than valueless_elements_base plus a number per byte
|
||||
// read. Write the unoptimized form, at one byte per element, for
|
||||
// an array that would exceed the base with the arrays written so
|
||||
// far, so that the result can be read back: the limit of a reader
|
||||
// is never below the base. Objects are not affected: every element
|
||||
// is preceded by its key.
|
||||
if (same_prefix && !(use_bjdata && is_bjdata_excluded_type_marker(first_prefix)))
|
||||
// by and refuses an excessive one. Write the unoptimized form for
|
||||
// those, at one byte per element, so the result can be read back.
|
||||
// Objects are not affected: every element is preceded by its key.
|
||||
const bool valueless_type = (first_prefix == 'Z' || first_prefix == 'T' || first_prefix == 'F');
|
||||
const bool excessive_valueless = IsArray
|
||||
&& valueless_type
|
||||
&& size > detail::max_valueless_container_size;
|
||||
|
||||
if (same_prefix && !excessive_valueless
|
||||
&& !(use_bjdata && is_bjdata_excluded_type_marker(first_prefix)))
|
||||
{
|
||||
const bool valueless_type = IsArray && (first_prefix == 'Z' || first_prefix == 'T' || first_prefix == 'F');
|
||||
if (!valueless_type || size <= detail::valueless_elements_base - valueless_elements_written)
|
||||
{
|
||||
prefix_required = false;
|
||||
oa.write_character(to_char_type('$'));
|
||||
oa.write_character(first_prefix);
|
||||
if (valueless_type)
|
||||
{
|
||||
valueless_elements_written += size;
|
||||
}
|
||||
}
|
||||
prefix_required = false;
|
||||
oa.write_character(to_char_type('$'));
|
||||
oa.write_character(first_prefix);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5448,10 +5441,6 @@ class binary_writer
|
||||
/// how to treat a string value or object key that is not valid UTF-8
|
||||
/// (CBOR, MessagePack, UBJSON, BJData, and BSON; not BON8)
|
||||
const error_handler_t error_handler = binary_writer_default_error_handler();
|
||||
|
||||
/// UBJSON: the number of elements written so far in optimized arrays of a
|
||||
/// valueless type, which a reader adds up and bounds
|
||||
std::size_t valueless_elements_written = 0;
|
||||
};
|
||||
|
||||
} // namespace detail
|
||||
|
||||
Generated
+31
-307
@@ -8866,7 +8866,6 @@ NLOHMANN_JSON_NAMESPACE_END
|
||||
#include <iterator> // back_inserter
|
||||
#include <limits> // numeric_limits
|
||||
#include <string> // char_traits, string
|
||||
#include <type_traits> // is_convertible, true_type, false_type
|
||||
#include <utility> // make_pair, move
|
||||
#include <vector> // vector
|
||||
#ifdef __cpp_lib_byteswap
|
||||
@@ -16057,40 +16056,6 @@ constexpr std::size_t unknown_size()
|
||||
return (std::numeric_limits<std::size_t>::max)();
|
||||
}
|
||||
|
||||
/// the largest number of elements the DOM parsers reserve for one array upfront
|
||||
constexpr std::size_t reserve_cap()
|
||||
{
|
||||
return 16384;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief number of array elements a DOM parser may reserve upfront on top of
|
||||
@ref reserved_elements_per_byte
|
||||
|
||||
An array that announces its size is given storage for that many elements when
|
||||
it opens (at most @ref reserve_cap of them), before any input backs the
|
||||
announcement, and the arrays that are open at the same time keep their storage
|
||||
until the input is complete: 100000 nested headers of three bytes each that
|
||||
announce 16384 elements would otherwise need 1.7 GB. The elements reserved for
|
||||
all arrays of an input are therefore added up, and the sum must stay below
|
||||
this base plus @ref reserved_elements_per_byte for every byte read so far; an
|
||||
array that would go beyond reserves what is left, and grows as its elements
|
||||
arrive. The base is four arrays of the maximum size, so the first nested
|
||||
arrays of an input are not affected.
|
||||
*/
|
||||
JSON_INLINE_VARIABLE constexpr std::size_t reserved_elements_base = 4 * reserve_cap();
|
||||
|
||||
/*!
|
||||
@brief number of array elements a DOM parser may reserve upfront per byte of
|
||||
input, in addition to @ref reserved_elements_base
|
||||
|
||||
An element of an array takes at least one byte of input, so an array that
|
||||
completes has used what was reserved for it. At a rate of 2 elements per
|
||||
byte, an array that announces more than the input provides keeps at most
|
||||
32 bytes of memory per byte of input, plus the base, until the parse fails.
|
||||
*/
|
||||
JSON_INLINE_VARIABLE constexpr std::size_t reserved_elements_per_byte = 2;
|
||||
|
||||
/*!
|
||||
@brief reserve capacity for @a len elements in array @a arr
|
||||
|
||||
@@ -16106,7 +16071,8 @@ template<typename ArrayType>
|
||||
auto reserve_array(ArrayType& arr, std::size_t len, priority_tag<1> /*unused*/)
|
||||
-> decltype(arr.reserve(len), void())
|
||||
{
|
||||
arr.reserve((std::min)(len, reserve_cap()));
|
||||
constexpr std::size_t reserve_cap = 16384;
|
||||
arr.reserve((std::min)(len, reserve_cap));
|
||||
}
|
||||
|
||||
template<typename ArrayType>
|
||||
@@ -16273,13 +16239,6 @@ struct diagnostic_positions
|
||||
Holds whether a parse error occurred and whether errors are thrown, and
|
||||
implements the parse_error() and is_errored() members of the SAX interface for
|
||||
json_sax_dom_parser and json_sax_dom_callback_parser.
|
||||
|
||||
It also bounds what the parsers reserve for arrays that announce their size
|
||||
(reserve_count()): the announced size is not backed by input yet when the
|
||||
array opens, and the arrays that are open at the same time each keep what was
|
||||
reserved for them, so the binary_reader tells the parser where to find the
|
||||
number of characters it has read (set_input_position()), which the bound
|
||||
is derived from.
|
||||
*/
|
||||
class json_sax_dom_error_state
|
||||
{
|
||||
@@ -16313,103 +16272,7 @@ class json_sax_dom_error_state
|
||||
return errored;
|
||||
}
|
||||
|
||||
/// bind the number of characters the input has provided so far; see
|
||||
/// @ref reserve_count
|
||||
/// (nullptr: not known, which bounds only the size of each array)
|
||||
void set_input_position(const std::size_t* position) noexcept
|
||||
{
|
||||
input_position = position;
|
||||
credit_position = 0;
|
||||
reserve_credit = position != nullptr ? reserved_elements_base : (std::numeric_limits<std::size_t>::max)();
|
||||
}
|
||||
|
||||
/// what set_input_position() changes, so that a binding can be undone
|
||||
struct reserve_state
|
||||
{
|
||||
const std::size_t* position;
|
||||
std::size_t position_at_refill;
|
||||
std::size_t credit;
|
||||
};
|
||||
|
||||
reserve_state get_reserve_state() const noexcept
|
||||
{
|
||||
return {input_position, credit_position, reserve_credit};
|
||||
}
|
||||
|
||||
void set_reserve_state(const reserve_state& state) noexcept
|
||||
{
|
||||
input_position = state.position;
|
||||
credit_position = state.position_at_refill;
|
||||
reserve_credit = state.credit;
|
||||
}
|
||||
|
||||
protected:
|
||||
/*!
|
||||
@brief the number of elements to reserve for an array that announces @a len
|
||||
|
||||
The announced number is not backed by input when the array opens, and the
|
||||
arrays that are open at the same time each keep what was reserved for
|
||||
them, so what is reserved for all arrays together is bounded by the
|
||||
input read so far: @ref reserved_elements_base plus
|
||||
@ref reserved_elements_per_byte for every character. An array that
|
||||
announces more than what is left of that is given what is left, and
|
||||
grows as its elements arrive. Without the position of the input (the
|
||||
parser is not driven by a binary_reader that is running) nothing is
|
||||
bounded but the cap of reserve_array().
|
||||
|
||||
The common case, an array that announces no more than the credit left,
|
||||
costs a comparison and a subtraction. It is charged what it announces, which
|
||||
for an array of more than reserve_cap() elements is more than reserve_array()
|
||||
reserves for it, which its own elements refill as they are read.
|
||||
*/
|
||||
std::size_t reserve_count(const std::size_t len) noexcept
|
||||
{
|
||||
if (JSON_HEDLEY_LIKELY(len <= reserve_credit))
|
||||
{
|
||||
reserve_credit -= len;
|
||||
return len;
|
||||
}
|
||||
return reserve_count_slowly(len);
|
||||
}
|
||||
|
||||
private:
|
||||
JSON_HEDLEY_NEVER_INLINE
|
||||
std::size_t reserve_count_slowly(const std::size_t len) noexcept
|
||||
{
|
||||
constexpr std::size_t max_size = (std::numeric_limits<std::size_t>::max)();
|
||||
if (input_position == nullptr)
|
||||
{
|
||||
reserve_credit = max_size;
|
||||
return len;
|
||||
}
|
||||
|
||||
// refill the credit with 'reserved_elements_per_byte' for every
|
||||
// character read since the last refill; it saturates instead of
|
||||
// overflowing when size_t is 32 bits wide and a very large input has
|
||||
// been read (the position goes back by up to two in BON8, which has
|
||||
// no sizes)
|
||||
const std::size_t position = *input_position;
|
||||
if (position > credit_position)
|
||||
{
|
||||
const std::size_t bytes = position - credit_position;
|
||||
reserve_credit = bytes <= (max_size - reserve_credit) / reserved_elements_per_byte
|
||||
? reserve_credit + (reserved_elements_per_byte * bytes)
|
||||
: max_size;
|
||||
credit_position = position;
|
||||
}
|
||||
|
||||
std::size_t count = (std::min)(len, reserve_cap());
|
||||
count = (std::min)(count, reserve_credit);
|
||||
reserve_credit -= count;
|
||||
return count;
|
||||
}
|
||||
|
||||
/// the number of characters the input has provided so far, if known
|
||||
const std::size_t* input_position = nullptr;
|
||||
/// the number of characters the input had provided when the credit was refilled
|
||||
std::size_t credit_position = 0;
|
||||
/// the number of elements that may still be reserved
|
||||
std::size_t reserve_credit = reserved_elements_base;
|
||||
/// whether a syntax error occurred
|
||||
bool errored = false;
|
||||
/// whether to throw exceptions in case of errors
|
||||
@@ -17244,7 +17107,7 @@ class json_sax_dom_parser : public json_sax_dom_error_state
|
||||
|
||||
if (len != detail::unknown_size())
|
||||
{
|
||||
reserve_array(*ref->m_data.m_value.array, reserve_count(len), priority_tag<1> {});
|
||||
reserve_array(*ref->m_data.m_value.array, len, priority_tag<1> {});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -17592,7 +17455,7 @@ class json_sax_dom_binary_parser : public json_sax_dom_error_state
|
||||
|
||||
if (len != detail::unknown_size())
|
||||
{
|
||||
reserve_array(*ref_stack.back()->m_data.m_value.array, reserve_count(len), priority_tag<1> {});
|
||||
reserve_array(*ref_stack.back()->m_data.m_value.array, len, priority_tag<1> {});
|
||||
}
|
||||
|
||||
return true;
|
||||
@@ -17884,7 +17747,7 @@ class json_sax_dom_callback_parser : public json_sax_dom_error_state
|
||||
|
||||
if (len != detail::unknown_size())
|
||||
{
|
||||
reserve_array(*ref_stack.back()->m_data.m_value.array, reserve_count(len), priority_tag<1> {});
|
||||
reserve_array(*ref_stack.back()->m_data.m_value.array, len, priority_tag<1> {});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -18489,46 +18352,13 @@ the input.
|
||||
|
||||
This is a sanity bound rather than a security boundary, and it is far above
|
||||
any container met in practice. @ref binary_writer falls back to the
|
||||
unoptimized encoding for longer containers, and for containers that together
|
||||
with the ones before them would exceed @ref valueless_elements_base, so that a
|
||||
value serialized by this library can always be read back.
|
||||
unoptimized encoding for longer containers, so that a value serialized by
|
||||
this library can always be read back.
|
||||
|
||||
@sa https://github.com/nlohmann/json/issues/2793
|
||||
*/
|
||||
JSON_INLINE_VARIABLE constexpr std::size_t max_valueless_container_size = 1 << 20;
|
||||
|
||||
/*!
|
||||
@brief number of elements of valueless UBJSON containers accepted on top of
|
||||
@ref valueless_elements_per_byte
|
||||
|
||||
The limit @ref max_valueless_container_size applies to each container on its
|
||||
own, so on its own it lets 9 bytes announce 1048576 values, and every further
|
||||
9 bytes another 1048576. The elements of all containers of a valueless type
|
||||
in one input are therefore added up, and the sum must stay below this base
|
||||
plus @ref valueless_elements_per_byte for every byte read so far. The base is
|
||||
the per-container limit, so a container that was accepted on its own before
|
||||
the sum was introduced is still accepted at the start of an input.
|
||||
*/
|
||||
JSON_INLINE_VARIABLE constexpr std::size_t valueless_elements_base = max_valueless_container_size;
|
||||
|
||||
/*!
|
||||
@brief number of elements of valueless UBJSON containers accepted per byte of
|
||||
input, in addition to @ref valueless_elements_base
|
||||
|
||||
An element of a value that has a payload costs at least one byte of input.
|
||||
Valueless elements are accepted at a rate of 4 per byte. A value takes 16
|
||||
bytes of memory in a DOM, so these arrays add about 80 bytes of memory per
|
||||
byte of input (measured: 77; one `Z` per byte needs about 32), beyond what the
|
||||
same bytes cost anyway, plus the base. This bounds the arrays of valueless
|
||||
type, not the memory of a whole document. Containers written by
|
||||
@ref binary_writer never need more than the base, because it writes an array
|
||||
in the unoptimized form once the typed arrays written so far and this one add
|
||||
up to more than that. The rate matters for input written by other encoders and
|
||||
by versions before 3.13.0 only; in the data this was measured on, no input has
|
||||
more than 0.17 elements per byte.
|
||||
*/
|
||||
JSON_INLINE_VARIABLE constexpr std::size_t valueless_elements_per_byte = 4;
|
||||
|
||||
///////////////////
|
||||
// binary reader //
|
||||
///////////////////
|
||||
@@ -18596,8 +18426,6 @@ class binary_reader
|
||||
sax = sax_;
|
||||
container_stack.clear();
|
||||
bon8_pushback_size = 0;
|
||||
// the position is only known to the DOM parser while this reader lives
|
||||
const position_binding binding(bound_parser(sax, std::is_convertible<SAX*, json_sax_dom_error_state*> {}), &chars_read);
|
||||
bool result = false;
|
||||
|
||||
switch (input_format)
|
||||
@@ -18655,61 +18483,6 @@ class binary_reader
|
||||
}
|
||||
|
||||
private:
|
||||
/// the DOM parser of @a s whose reserved storage is bounded by the input
|
||||
/// read so far; nullptr for any other SAX parser, which does not reserve,
|
||||
/// and for one that derives from a DOM parser privately or protectedly
|
||||
/// (or ambiguously), which the reader cannot reach
|
||||
static json_sax_dom_error_state* bound_parser(SAX* s, std::true_type /*is a DOM parser*/) noexcept
|
||||
{
|
||||
return s;
|
||||
}
|
||||
|
||||
static json_sax_dom_error_state* bound_parser(SAX* /*s*/, std::false_type /*is a DOM parser*/) noexcept
|
||||
{
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief tells a DOM parser where to find the number of characters read
|
||||
while sax_parse() runs, and puts back what it knew before when
|
||||
sax_parse() returns or throws
|
||||
|
||||
The reader is a local of the caller and is gone then, so the parser must
|
||||
not keep its position. A parser that was bound by a reader that is still
|
||||
running (a parse inside a parse) gets that binding back; that reader is
|
||||
still alive, as it encloses this one.
|
||||
*/
|
||||
class position_binding
|
||||
{
|
||||
public:
|
||||
position_binding(json_sax_dom_error_state* parser_, const std::size_t* position) noexcept
|
||||
: parser(parser_), saved()
|
||||
{
|
||||
if (parser != nullptr)
|
||||
{
|
||||
saved = parser->get_reserve_state();
|
||||
parser->set_input_position(position);
|
||||
}
|
||||
}
|
||||
|
||||
position_binding(const position_binding&) = delete;
|
||||
position_binding(position_binding&&) = delete;
|
||||
position_binding& operator=(const position_binding&) = delete;
|
||||
position_binding& operator=(position_binding&&) = delete;
|
||||
|
||||
~position_binding()
|
||||
{
|
||||
if (parser != nullptr)
|
||||
{
|
||||
parser->set_reserve_state(saved);
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
json_sax_dom_error_state* parser;
|
||||
json_sax_dom_error_state::reserve_state saved;
|
||||
};
|
||||
|
||||
////////////////////////
|
||||
// nested containers //
|
||||
////////////////////////
|
||||
@@ -22206,37 +21979,6 @@ class binary_reader
|
||||
return emit_unsigned(number);
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief add the elements of a container of a valueless type to the total of
|
||||
the input and check the total against the input read so far
|
||||
|
||||
A container of type 'Z', 'T' or 'F' has no payload, so its elements are the
|
||||
only ones for which the input does not bound the count. The check is made
|
||||
once per container, not per element.
|
||||
|
||||
@param[in] count number of elements the container declares; at most
|
||||
@ref max_valueless_container_size
|
||||
@return false if the total is more than @ref valueless_elements_base plus
|
||||
@ref valueless_elements_per_byte for every byte read so far; the
|
||||
total is not changed then
|
||||
*/
|
||||
bool charge_valueless_elements(const std::size_t count) noexcept
|
||||
{
|
||||
// the limit saturates instead of overflowing when size_t is 32 bits
|
||||
// wide and a very large input has been read; valueless_elements
|
||||
// never exceeds the limit, so the difference cannot underflow
|
||||
constexpr std::size_t max_size = (std::numeric_limits<std::size_t>::max)();
|
||||
const std::size_t limit = chars_read <= (max_size - valueless_elements_base) / valueless_elements_per_byte
|
||||
? valueless_elements_base + (valueless_elements_per_byte * chars_read)
|
||||
: max_size;
|
||||
if (JSON_HEDLEY_UNLIKELY(count > limit - valueless_elements))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
valueless_elements += count;
|
||||
return true;
|
||||
}
|
||||
|
||||
/*!
|
||||
@return whether array creation completed
|
||||
*/
|
||||
@@ -22322,27 +22064,24 @@ class binary_reader
|
||||
// declared count alone decides how much is allocated; the check is
|
||||
// made before the start event so that no container is opened that
|
||||
// is then abandoned. See @ref max_valueless_container_size.
|
||||
if (JSON_HEDLEY_UNLIKELY(size_and_type.second == 'Z' || size_and_type.second == 'T' || size_and_type.second == 'F'))
|
||||
if (JSON_HEDLEY_UNLIKELY((size_and_type.second == 'Z' || size_and_type.second == 'T' || size_and_type.second == 'F')
|
||||
&& size_and_type.first > max_valueless_container_size))
|
||||
{
|
||||
if (JSON_HEDLEY_UNLIKELY(size_and_type.first > max_valueless_container_size))
|
||||
{
|
||||
return out_of_range_error(exception_id::container_too_large, "excessive array size", "size");
|
||||
}
|
||||
if (JSON_HEDLEY_UNLIKELY(!charge_valueless_elements(size_and_type.first)))
|
||||
{
|
||||
return out_of_range_error(exception_id::container_too_large, "excessive number of elements without payload", "size");
|
||||
}
|
||||
return out_of_range_error(exception_id::container_too_large, "excessive array size", "size");
|
||||
}
|
||||
|
||||
// a no-op is not a value, so an array of them is empty whatever
|
||||
// count it declares: the SAX parser is told so, instead of being
|
||||
// made to reserve storage for a count that no input backs
|
||||
const std::size_t announced = size_and_type.second == 'N' ? 0 : size_and_type.first;
|
||||
if (JSON_HEDLEY_UNLIKELY(!enter_array(announced, size_and_type.second)))
|
||||
if (JSON_HEDLEY_UNLIKELY(!enter_array(size_and_type.first, size_and_type.second)))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
if (size_and_type.second == 'N')
|
||||
{
|
||||
// a no-op is not a value, so a container of them holds none;
|
||||
// the declared size has already been passed to the SAX parser
|
||||
container_stack.back().remaining = 0;
|
||||
}
|
||||
|
||||
return size_and_type.second == 0
|
||||
|| parse_ubjson_typed_run(container_stack.back().remaining, size_and_type.second, std::integral_constant<bool, bulk_scan> {});
|
||||
}
|
||||
@@ -23654,10 +23393,6 @@ class binary_reader
|
||||
/// the number of characters read
|
||||
std::size_t chars_read = 0;
|
||||
|
||||
/// UBJSON: the sum of the element counts of the containers of a valueless
|
||||
/// type read so far; see @ref charge_valueless_elements
|
||||
std::size_t valueless_elements = 0;
|
||||
|
||||
/// whether we can assume little endianness
|
||||
const bool is_little_endian = little_endianness();
|
||||
|
||||
@@ -30811,27 +30546,20 @@ class binary_writer
|
||||
|
||||
// an optimized array of a valueless type carries no payload, so a
|
||||
// reader has nothing but the declared count to bound the allocation
|
||||
// by. It refuses one array of more than max_valueless_container_size
|
||||
// elements, and all such arrays of one input together when they add
|
||||
// up to more than valueless_elements_base plus a number per byte
|
||||
// read. Write the unoptimized form, at one byte per element, for
|
||||
// an array that would exceed the base with the arrays written so
|
||||
// far, so that the result can be read back: the limit of a reader
|
||||
// is never below the base. Objects are not affected: every element
|
||||
// is preceded by its key.
|
||||
if (same_prefix && !(use_bjdata && is_bjdata_excluded_type_marker(first_prefix)))
|
||||
// by and refuses an excessive one. Write the unoptimized form for
|
||||
// those, at one byte per element, so the result can be read back.
|
||||
// Objects are not affected: every element is preceded by its key.
|
||||
const bool valueless_type = (first_prefix == 'Z' || first_prefix == 'T' || first_prefix == 'F');
|
||||
const bool excessive_valueless = IsArray
|
||||
&& valueless_type
|
||||
&& size > detail::max_valueless_container_size;
|
||||
|
||||
if (same_prefix && !excessive_valueless
|
||||
&& !(use_bjdata && is_bjdata_excluded_type_marker(first_prefix)))
|
||||
{
|
||||
const bool valueless_type = IsArray && (first_prefix == 'Z' || first_prefix == 'T' || first_prefix == 'F');
|
||||
if (!valueless_type || size <= detail::valueless_elements_base - valueless_elements_written)
|
||||
{
|
||||
prefix_required = false;
|
||||
oa.write_character(to_char_type('$'));
|
||||
oa.write_character(first_prefix);
|
||||
if (valueless_type)
|
||||
{
|
||||
valueless_elements_written += size;
|
||||
}
|
||||
}
|
||||
prefix_required = false;
|
||||
oa.write_character(to_char_type('$'));
|
||||
oa.write_character(first_prefix);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -35055,10 +34783,6 @@ class binary_writer
|
||||
/// how to treat a string value or object key that is not valid UTF-8
|
||||
/// (CBOR, MessagePack, UBJSON, BJData, and BSON; not BON8)
|
||||
const error_handler_t error_handler = binary_writer_default_error_handler();
|
||||
|
||||
/// UBJSON: the number of elements written so far in optimized arrays of a
|
||||
/// valueless type, which a reader adds up and bounds
|
||||
std::size_t valueless_elements_written = 0;
|
||||
};
|
||||
|
||||
} // namespace detail
|
||||
|
||||
@@ -1,86 +0,0 @@
|
||||
// __ _____ _____ _____
|
||||
// __| | __| | | | JSON for Modern C++ (supporting code)
|
||||
// | | |__ | | | | | | version 3.12.0
|
||||
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
|
||||
//
|
||||
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
|
||||
// SPDX-License-Identifier: MIT
|
||||
|
||||
#pragma once
|
||||
|
||||
#include <algorithm> // max
|
||||
#include <cstddef> // size_t
|
||||
#include <cstdint> // uint8_t
|
||||
#include <memory> // allocator
|
||||
#include <vector> // vector
|
||||
|
||||
#include <nlohmann/json.hpp>
|
||||
|
||||
namespace utils
|
||||
{
|
||||
/// the allocations of the values of a JSON type that uses counting_allocator
|
||||
struct allocation_counts
|
||||
{
|
||||
/// bytes allocated and not yet deallocated
|
||||
std::size_t current = 0;
|
||||
/// the largest value current had
|
||||
std::size_t peak = 0;
|
||||
/// number of allocations
|
||||
std::size_t allocations = 0;
|
||||
};
|
||||
|
||||
/// the counts of all counting_allocators; the tests are single-threaded
|
||||
inline allocation_counts& counts()
|
||||
{
|
||||
static allocation_counts c; // NOLINT(misc-const-correctness)
|
||||
return c;
|
||||
}
|
||||
|
||||
/// an allocator that counts the bytes and the number of allocations of a
|
||||
/// JSON type, so that a test can check how much memory a parse holds at any
|
||||
/// time without measuring the memory of the process
|
||||
template<class T>
|
||||
struct counting_allocator : std::allocator<T>
|
||||
{
|
||||
using std::allocator<T>::allocator;
|
||||
|
||||
counting_allocator() = default;
|
||||
template<class U> counting_allocator(const counting_allocator<U>& /*unused*/) noexcept {} // NOLINT(google-explicit-constructor,hicpp-explicit-conversions)
|
||||
|
||||
template <class U>
|
||||
struct rebind
|
||||
{
|
||||
using other = counting_allocator<U>;
|
||||
};
|
||||
|
||||
T* allocate(const std::size_t n)
|
||||
{
|
||||
T* const result = std::allocator<T>::allocate(n);
|
||||
allocation_counts& c = counts();
|
||||
c.current += n * sizeof(T);
|
||||
c.peak = (std::max)(c.peak, c.current);
|
||||
++c.allocations;
|
||||
return result;
|
||||
}
|
||||
|
||||
void deallocate(T* p, const std::size_t n)
|
||||
{
|
||||
counts().current -= n * sizeof(T);
|
||||
std::allocator<T>::deallocate(p, n);
|
||||
}
|
||||
};
|
||||
|
||||
using counted_json = nlohmann::json::with_allocator_t<counting_allocator>;
|
||||
|
||||
/// @a count times the bytes of @a unit
|
||||
inline std::vector<std::uint8_t> repeated(const std::vector<std::uint8_t>& unit, const std::size_t count)
|
||||
{
|
||||
std::vector<std::uint8_t> result;
|
||||
result.reserve(unit.size() * count);
|
||||
for (std::size_t i = 0; i < count; ++i)
|
||||
{
|
||||
result.insert(result.end(), unit.begin(), unit.end());
|
||||
}
|
||||
return result;
|
||||
}
|
||||
} // namespace utils
|
||||
@@ -31,7 +31,6 @@ using nlohmann::ordered_json;
|
||||
#include "make_test_data_available.hpp"
|
||||
#include "test_utils.hpp"
|
||||
#include "sax_countdown.hpp"
|
||||
#include "allocation_counter.hpp"
|
||||
#include "sax_event_loggers.hpp"
|
||||
using utils::SaxCountdown;
|
||||
using utils::SaxEventLogger;
|
||||
@@ -4031,60 +4030,6 @@ TEST_CASE("issue #5405 - array reserve for definite-length BJData arrays")
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("BJData - open arrays reserve only what the input has backed")
|
||||
{
|
||||
// an array that announces its size gets storage for (up to 16384 of) its
|
||||
// elements when it opens; the arrays that are open at the same time must
|
||||
// not hold more than the input read so far can back
|
||||
SECTION("nested headers that announce 16384 elements and are never completed")
|
||||
{
|
||||
// 200 levels would hold 200 * 16384 * sizeof(json),
|
||||
// if each of them reserved what it announces
|
||||
const auto input = utils::repeated({'[', '#', 'I', 0x00, 0x40}, 200);
|
||||
|
||||
utils::counts() = utils::allocation_counts{};
|
||||
const auto j = utils::counted_json::from_bjdata(input, true, false);
|
||||
CHECK(j.is_discarded());
|
||||
CAPTURE(utils::counts().peak)
|
||||
// the credit is 65,536 values; the base peaks at more than 3 million
|
||||
CHECK(utils::counts().peak < sizeof(utils::counted_json) * 128 * 1024);
|
||||
}
|
||||
|
||||
SECTION("a completed array is still reserved in one allocation")
|
||||
{
|
||||
json array = json::array();
|
||||
for (int i = 0; i < 10000; ++i)
|
||||
{
|
||||
array.push_back(i % 100);
|
||||
}
|
||||
|
||||
// an array without a size is not reserved, so only these two
|
||||
{
|
||||
const auto packed = json::to_bjdata(array, true);
|
||||
|
||||
utils::counts() = utils::allocation_counts{};
|
||||
const auto j = utils::counted_json::from_bjdata(packed);
|
||||
CHECK(j.size() == 10000);
|
||||
// the array and its storage; growing the storage as the elements
|
||||
// arrive would take more than a dozen allocations
|
||||
CAPTURE(utils::counts().allocations)
|
||||
CHECK(utils::counts().allocations <= 4);
|
||||
}
|
||||
|
||||
{
|
||||
const auto packed = json::to_bjdata(array, true, true);
|
||||
|
||||
utils::counts() = utils::allocation_counts{};
|
||||
const auto j = utils::counted_json::from_bjdata(packed);
|
||||
CHECK(j.size() == 10000);
|
||||
// the array and its storage; growing the storage as the elements
|
||||
// arrive would take more than a dozen allocations
|
||||
CAPTURE(utils::counts().allocations)
|
||||
CHECK(utils::counts().allocations <= 4);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("Universal Binary JSON Specification Examples 1")
|
||||
{
|
||||
SECTION("Null Value")
|
||||
|
||||
@@ -29,7 +29,6 @@ using nlohmann::json;
|
||||
#include "round_trip_corpus.hpp"
|
||||
#include "test_utils.hpp"
|
||||
#include "sax_countdown.hpp"
|
||||
#include "allocation_counter.hpp"
|
||||
#include "sax_event_loggers.hpp"
|
||||
using utils::SaxCountdown;
|
||||
using utils::SaxEventLogger;
|
||||
@@ -2710,289 +2709,6 @@ TEST_CASE("issue #5405 - array reserve for definite-length CBOR arrays")
|
||||
}
|
||||
}
|
||||
|
||||
namespace
|
||||
{
|
||||
// NOLINTBEGIN(bugprone-derived-method-shadowing-base-method)
|
||||
using dom_parser = nlohmann::detail::json_sax_dom_parser<json, nlohmann::detail::input_stream_adapter>;
|
||||
|
||||
// handlers that wrap a DOM parser, as users of the SAX interface do
|
||||
struct public_wrapper : dom_parser
|
||||
{
|
||||
explicit public_wrapper(json& result) : dom_parser(result, false)
|
||||
{}
|
||||
};
|
||||
|
||||
/// parses another input with itself while the first parse is still running
|
||||
struct reentrant_wrapper : dom_parser
|
||||
{
|
||||
explicit reentrant_wrapper(json& result) : dom_parser(result, false)
|
||||
{}
|
||||
|
||||
bool start_array(std::size_t len)
|
||||
{
|
||||
if (!inner_done)
|
||||
{
|
||||
inner_done = true;
|
||||
const std::vector<std::uint8_t> input = {0x80};
|
||||
json::sax_parse(input, this, json::input_format_t::cbor);
|
||||
}
|
||||
return dom_parser::start_array(len);
|
||||
}
|
||||
|
||||
bool inner_done = false;
|
||||
};
|
||||
|
||||
class private_wrapper : private dom_parser
|
||||
{
|
||||
public:
|
||||
explicit private_wrapper(json& result) : dom_parser(result, false)
|
||||
{}
|
||||
|
||||
using dom_parser::null;
|
||||
using dom_parser::boolean;
|
||||
using dom_parser::number_integer;
|
||||
using dom_parser::number_unsigned;
|
||||
using dom_parser::number_float;
|
||||
using dom_parser::string;
|
||||
using dom_parser::binary;
|
||||
using dom_parser::start_array;
|
||||
using dom_parser::end_array;
|
||||
using dom_parser::parse_error;
|
||||
|
||||
// the DOM parser has private overloads of these, which a using
|
||||
// declaration cannot expose; the input of the test has no objects
|
||||
static bool start_object(std::size_t /*len*/)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
static bool key(std::string& /*val*/)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
static bool end_object()
|
||||
{
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
class protected_wrapper : protected dom_parser
|
||||
{
|
||||
public:
|
||||
explicit protected_wrapper(json& result) : dom_parser(result, false)
|
||||
{}
|
||||
|
||||
using dom_parser::null;
|
||||
using dom_parser::boolean;
|
||||
using dom_parser::number_integer;
|
||||
using dom_parser::number_unsigned;
|
||||
using dom_parser::number_float;
|
||||
using dom_parser::string;
|
||||
using dom_parser::binary;
|
||||
using dom_parser::start_array;
|
||||
using dom_parser::end_array;
|
||||
using dom_parser::parse_error;
|
||||
|
||||
// the DOM parser has private overloads of these, which a using
|
||||
// declaration cannot expose; the input of the test has no objects
|
||||
static bool start_object(std::size_t /*len*/)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
static bool key(std::string& /*val*/)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
static bool end_object()
|
||||
{
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
// NOLINTEND(bugprone-derived-method-shadowing-base-method)
|
||||
|
||||
/// the capacity of the fifth array of five that are open
|
||||
std::size_t innermost_capacity(const json& result)
|
||||
{
|
||||
const json* node = &result;
|
||||
for (int level = 0; level < 4; ++level)
|
||||
{
|
||||
node = &node->front();
|
||||
}
|
||||
return node->get_ref<const json::array_t&>().capacity();
|
||||
}
|
||||
} // namespace
|
||||
|
||||
TEST_CASE("CBOR - open arrays reserve only what the input has backed")
|
||||
{
|
||||
// an array that announces its size gets storage for (up to 16384 of) its
|
||||
// elements when it opens; the arrays that are open at the same time must
|
||||
// not hold more than the input read so far can back
|
||||
SECTION("nested headers that announce 16384 elements and are never completed")
|
||||
{
|
||||
// 200 levels would hold 200 * 16384 * sizeof(json),
|
||||
// if each of them reserved what it announces
|
||||
const auto input = utils::repeated({0x99, 0x40, 0x00}, 200);
|
||||
|
||||
utils::counts() = utils::allocation_counts{};
|
||||
const auto j = utils::counted_json::from_cbor(input, true, false);
|
||||
CHECK(j.is_discarded());
|
||||
CAPTURE(utils::counts().peak)
|
||||
// the credit is 65,536 values; the base peaks at more than 3 million
|
||||
CHECK(utils::counts().peak < sizeof(utils::counted_json) * 128 * 1024);
|
||||
}
|
||||
|
||||
SECTION("a completed array is still reserved in one allocation")
|
||||
{
|
||||
json array = json::array();
|
||||
for (int i = 0; i < 10000; ++i)
|
||||
{
|
||||
array.push_back(i % 100);
|
||||
}
|
||||
|
||||
{
|
||||
const auto packed = json::to_cbor(array);
|
||||
|
||||
utils::counts() = utils::allocation_counts{};
|
||||
const auto j = utils::counted_json::from_cbor(packed);
|
||||
CHECK(j.size() == 10000);
|
||||
// the array and its storage; growing the storage as the elements
|
||||
// arrive would take more than a dozen allocations
|
||||
CAPTURE(utils::counts().allocations)
|
||||
CHECK(utils::counts().allocations <= 4);
|
||||
}
|
||||
}
|
||||
|
||||
SECTION("an array that is given less than it announces is still decoded")
|
||||
{
|
||||
// five levels that announce 16384 elements each, the first of which
|
||||
// is the next level: they are more than the input could back when
|
||||
// they open
|
||||
constexpr std::size_t levels = 5;
|
||||
std::vector<std::uint8_t> input;
|
||||
for (std::size_t level = 0; level < levels; ++level)
|
||||
{
|
||||
input.insert(input.end(), {0x99, 0x40, 0x00});
|
||||
}
|
||||
input.insert(input.end(), 16384, 0x01);
|
||||
for (std::size_t level = 1; level < levels; ++level)
|
||||
{
|
||||
input.insert(input.end(), 16383, 0x02);
|
||||
}
|
||||
|
||||
const auto j = json::from_cbor(input);
|
||||
const json* node = &j;
|
||||
for (std::size_t level = 0; level < levels; ++level)
|
||||
{
|
||||
REQUIRE(node->is_array());
|
||||
CHECK(node->size() == 16384);
|
||||
node = &node->front();
|
||||
}
|
||||
CHECK(*node == 1);
|
||||
CHECK(j.back() == 2);
|
||||
}
|
||||
|
||||
SECTION("a DOM parser that a parse has finished with is not bounded by the input of that parse")
|
||||
{
|
||||
// the DOM parser learns where the reader counts its characters, and
|
||||
// must forget it when the parse is over: a handler may be driven by
|
||||
// other code afterwards, while the reader no longer exists
|
||||
// five levels announce more than the credit of one input, which a
|
||||
// parse of one byte has hardly refilled
|
||||
const auto check_unbounded = [](dom_parser & parser, json & result)
|
||||
{
|
||||
for (int level = 0; level < 5; ++level)
|
||||
{
|
||||
CHECK(parser.start_array(16384));
|
||||
}
|
||||
|
||||
const json* node = &result;
|
||||
for (int level = 0; level < 4; ++level)
|
||||
{
|
||||
REQUIRE(node->is_array());
|
||||
REQUIRE(node->size() == 1);
|
||||
node = &node->front();
|
||||
}
|
||||
REQUIRE(node->is_array());
|
||||
CHECK(node->get_ref<const json::array_t&>().capacity() == 16384);
|
||||
};
|
||||
|
||||
SECTION("after a parse that succeeded")
|
||||
{
|
||||
json result;
|
||||
dom_parser parser(result, false);
|
||||
const std::vector<std::uint8_t> input = {0x80};
|
||||
CHECK(json::sax_parse(input, &parser, json::input_format_t::cbor));
|
||||
check_unbounded(parser, result);
|
||||
}
|
||||
|
||||
#if !defined(JSON_NOEXCEPTION)
|
||||
SECTION("after a parse that threw")
|
||||
{
|
||||
json result;
|
||||
dom_parser parser(result, true);
|
||||
const std::vector<std::uint8_t> input = {0x1C}; // not a CBOR value
|
||||
CHECK_THROWS_AS(json::sax_parse(input, &parser, json::input_format_t::cbor), json::parse_error&);
|
||||
check_unbounded(parser, result);
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
SECTION("a handler that wraps a DOM parser is bound if the reader can reach the parser")
|
||||
{
|
||||
// five arrays announce 16384 elements: the credit of one input covers
|
||||
// four of them, and a parse of 15 bytes has hardly refilled it
|
||||
const auto input = utils::repeated({0x99, 0x40, 0x00}, 5);
|
||||
|
||||
SECTION("public base")
|
||||
{
|
||||
json result;
|
||||
public_wrapper handler(result);
|
||||
CHECK_FALSE(json::sax_parse(input, &handler, json::input_format_t::cbor));
|
||||
CHECK(innermost_capacity(result) < 16384);
|
||||
}
|
||||
|
||||
SECTION("public base, a parse inside the parse")
|
||||
{
|
||||
json result;
|
||||
reentrant_wrapper handler(result);
|
||||
CHECK_FALSE(json::sax_parse(input, &handler, json::input_format_t::cbor));
|
||||
CHECK(handler.inner_done);
|
||||
CHECK(innermost_capacity(result) < 16384);
|
||||
}
|
||||
|
||||
// the reader cannot convert these to the DOM parser: they are not bound
|
||||
SECTION("private base")
|
||||
{
|
||||
json result;
|
||||
private_wrapper handler(result);
|
||||
CHECK_FALSE(json::sax_parse(input, &handler, json::input_format_t::cbor));
|
||||
CHECK(innermost_capacity(result) == 16384);
|
||||
}
|
||||
|
||||
SECTION("protected base")
|
||||
{
|
||||
json result;
|
||||
protected_wrapper handler(result);
|
||||
CHECK_FALSE(json::sax_parse(input, &handler, json::input_format_t::cbor));
|
||||
CHECK(innermost_capacity(result) == 16384);
|
||||
}
|
||||
}
|
||||
|
||||
SECTION("SAX events report the announced size")
|
||||
{
|
||||
SaxEventLogger logger;
|
||||
const std::vector<std::uint8_t> input = {0x99, 0x40, 0x00};
|
||||
CHECK_FALSE(json::sax_parse(input, &logger, json::input_format_t::cbor));
|
||||
REQUIRE(logger.events.size() == 2);
|
||||
CHECK(logger.events[0] == "start_array(16384)");
|
||||
CHECK(logger.events[1] == "parse_error(4)");
|
||||
}
|
||||
}
|
||||
|
||||
#if !defined(JSON_NOEXCEPTION) // corpus values that do not survive the round trip are skipped by catching the exception
|
||||
TEST_CASE("CBOR round-trip invariants")
|
||||
{
|
||||
|
||||
@@ -33,7 +33,6 @@ using nlohmann::json;
|
||||
#include "round_trip_corpus.hpp"
|
||||
#include "test_utils.hpp"
|
||||
#include "sax_countdown.hpp"
|
||||
#include "allocation_counter.hpp"
|
||||
#include "sax_event_loggers.hpp"
|
||||
using utils::SaxCountdown;
|
||||
using utils::SaxEventLogger;
|
||||
@@ -1760,47 +1759,6 @@ TEST_CASE("issue #5405 - array reserve for definite-length MessagePack arrays")
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("MessagePack - open arrays reserve only what the input has backed")
|
||||
{
|
||||
// an array that announces its size gets storage for (up to 16384 of) its
|
||||
// elements when it opens; the arrays that are open at the same time must
|
||||
// not hold more than the input read so far can back
|
||||
SECTION("nested headers that announce 16384 elements and are never completed")
|
||||
{
|
||||
// 200 levels would hold 200 * 16384 * sizeof(json),
|
||||
// if each of them reserved what it announces
|
||||
const auto input = utils::repeated({0xDC, 0x40, 0x00}, 200);
|
||||
|
||||
utils::counts() = utils::allocation_counts{};
|
||||
const auto j = utils::counted_json::from_msgpack(input, true, false);
|
||||
CHECK(j.is_discarded());
|
||||
CAPTURE(utils::counts().peak)
|
||||
// the credit is 65,536 values; the base peaks at more than 3 million
|
||||
CHECK(utils::counts().peak < sizeof(utils::counted_json) * 128 * 1024);
|
||||
}
|
||||
|
||||
SECTION("a completed array is still reserved in one allocation")
|
||||
{
|
||||
json array = json::array();
|
||||
for (int i = 0; i < 10000; ++i)
|
||||
{
|
||||
array.push_back(i % 100);
|
||||
}
|
||||
|
||||
{
|
||||
const auto packed = json::to_msgpack(array);
|
||||
|
||||
utils::counts() = utils::allocation_counts{};
|
||||
const auto j = utils::counted_json::from_msgpack(packed);
|
||||
CHECK(j.size() == 10000);
|
||||
// the array and its storage; growing the storage as the elements
|
||||
// arrive would take more than a dozen allocations
|
||||
CAPTURE(utils::counts().allocations)
|
||||
CHECK(utils::counts().allocations <= 4);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("regression test - MessagePack ext type rejects a subtype that doesn't fit a single byte")
|
||||
{
|
||||
// subtype 0-255 must still round-trip correctly (regression guard, pre-existing behavior)
|
||||
|
||||
@@ -19,19 +19,14 @@
|
||||
#include <nlohmann/json.hpp>
|
||||
using nlohmann::json;
|
||||
|
||||
#include <cstdio>
|
||||
#include <iostream>
|
||||
#include <sstream>
|
||||
#include <utility>
|
||||
#include <fstream>
|
||||
#include <limits>
|
||||
#include <list>
|
||||
#include <set>
|
||||
#include "make_test_data_available.hpp"
|
||||
#include "round_trip_corpus.hpp"
|
||||
#include "test_utils.hpp"
|
||||
#include "sax_countdown.hpp"
|
||||
#include "allocation_counter.hpp"
|
||||
#include "sax_event_loggers.hpp"
|
||||
using utils::SaxCountdown;
|
||||
using utils::SaxEventLogger;
|
||||
@@ -2441,341 +2436,6 @@ TEST_CASE("UBJSON optimized arrays of a valueless type are bounded")
|
||||
}
|
||||
}
|
||||
|
||||
namespace
|
||||
{
|
||||
/// SAX consumer that counts the values and keeps the id of a parse error
|
||||
struct ValuelessEventCounter : public nlohmann::json_sax<json>
|
||||
{
|
||||
bool null() override
|
||||
{
|
||||
return ++values < stop_after;
|
||||
}
|
||||
bool boolean(bool /*val*/) override
|
||||
{
|
||||
return ++values < stop_after;
|
||||
}
|
||||
bool number_integer(json::number_integer_t /*val*/) override
|
||||
{
|
||||
++values;
|
||||
return true;
|
||||
}
|
||||
bool number_unsigned(json::number_unsigned_t /*val*/) override
|
||||
{
|
||||
++values;
|
||||
return true;
|
||||
}
|
||||
bool number_float(json::number_float_t /*val*/, const std::string& /*s*/) override
|
||||
{
|
||||
++values;
|
||||
return true;
|
||||
}
|
||||
bool string(std::string& /*val*/) override
|
||||
{
|
||||
++values;
|
||||
return true;
|
||||
}
|
||||
bool binary(json::binary_t& /*val*/) override
|
||||
{
|
||||
++values;
|
||||
return true;
|
||||
}
|
||||
bool start_object(std::size_t /*elements*/) override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool key(std::string& /*val*/) override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool end_object() override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool start_array(std::size_t /*elements*/) override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool end_array() override
|
||||
{
|
||||
return true;
|
||||
}
|
||||
bool parse_error(std::size_t /*position*/, const std::string& /*last_token*/, const json::exception& ex) override
|
||||
{
|
||||
error_id = ex.id;
|
||||
return false;
|
||||
}
|
||||
|
||||
/// the handler refuses the value that makes this many
|
||||
std::size_t stop_after = (std::numeric_limits<std::size_t>::max)();
|
||||
std::size_t values = 0;
|
||||
int error_id = 0;
|
||||
};
|
||||
|
||||
/// appends `[$<marker>#l<count>` (nine bytes) to @a v
|
||||
void append_valueless_array(std::vector<std::uint8_t>& v, const char marker, const std::uint32_t count)
|
||||
{
|
||||
v.push_back('[');
|
||||
v.push_back('$');
|
||||
v.push_back(static_cast<std::uint8_t>(marker));
|
||||
v.push_back('#');
|
||||
v.push_back('l');
|
||||
v.push_back(static_cast<std::uint8_t>(count >> 24));
|
||||
v.push_back(static_cast<std::uint8_t>(count >> 16));
|
||||
v.push_back(static_cast<std::uint8_t>(count >> 8));
|
||||
v.push_back(static_cast<std::uint8_t>(count));
|
||||
}
|
||||
|
||||
/// `[` followed by one `[$<marker>#l<count>` per entry of @a counts and `]`
|
||||
std::vector<std::uint8_t> valueless_arrays(const char marker, const std::vector<std::uint32_t>& counts)
|
||||
{
|
||||
std::vector<std::uint8_t> v = {'['};
|
||||
for (const auto count : counts)
|
||||
{
|
||||
append_valueless_array(v, marker, count);
|
||||
}
|
||||
v.push_back(']');
|
||||
return v;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
TEST_CASE("UBJSON the valueless elements of a whole input are bounded")
|
||||
{
|
||||
// The per-container limit alone let nine bytes (`[$Z#l` and a count of
|
||||
// 1048576) announce a million values, and every further nine bytes another
|
||||
// million. The elements of all containers of type 'Z', 'T' or 'F' in one
|
||||
// input are now added up: they may not exceed 1048576 plus 4 for every
|
||||
// byte read when the container's header has been read.
|
||||
//
|
||||
// The tests count events with a SAX handler where they can: building and
|
||||
// comparing values of a million elements is what makes them slow under
|
||||
// Valgrind, so there is one rejection and one round trip with a DOM.
|
||||
json _;
|
||||
const char* const budget_message = "[json.exception.out_of_range.408] syntax error while parsing UBJSON size: excessive number of elements without payload";
|
||||
|
||||
// two arrays that exceed the limit: the second one, with its header read
|
||||
// at byte 19, may hold 4 * 19 = 76 values on top of the base of 1048576
|
||||
const std::uint32_t over_first = 524288;
|
||||
const std::uint32_t over_second = 1048576 + 76 - over_first + 1;
|
||||
|
||||
SECTION("the input of the original report is rejected")
|
||||
{
|
||||
// 362 bytes (40 arrays of 1048576 values) used to allocate 661 MB; the
|
||||
// second array is the one that is refused
|
||||
const auto input = valueless_arrays('Z', {over_first, over_second});
|
||||
CHECK_THROWS_WITH_AS(_ = json::from_ubjson(input), budget_message, json::out_of_range&);
|
||||
}
|
||||
|
||||
SECTION("every input adapter applies the budget")
|
||||
{
|
||||
const auto input = valueless_arrays('Z', {over_first, over_second});
|
||||
|
||||
// stream; not strict, which does not matter
|
||||
std::stringstream stream;
|
||||
stream.write(reinterpret_cast<const char*>(input.data()), static_cast<std::streamsize>(input.size()));
|
||||
ValuelessEventCounter from_stream;
|
||||
CHECK(!json::sax_parse(stream, &from_stream, json::input_format_t::ubjson, false));
|
||||
CHECK(from_stream.error_id == 408);
|
||||
|
||||
// a container that is not contiguous; the counts of arrays of
|
||||
// different types are added up
|
||||
std::vector<std::uint8_t> mixed = {'['};
|
||||
append_valueless_array(mixed, 'T', over_first);
|
||||
append_valueless_array(mixed, 'F', over_second);
|
||||
mixed.push_back(']');
|
||||
const std::list<std::uint8_t> as_list(mixed.begin(), mixed.end());
|
||||
ValuelessEventCounter from_list;
|
||||
CHECK(!json::sax_parse(as_list.begin(), as_list.end(), &from_list, json::input_format_t::ubjson));
|
||||
CHECK(from_list.error_id == 408);
|
||||
CHECK(from_list.values == over_first);
|
||||
|
||||
// FILE*
|
||||
FILE* const file = std::tmpfile(); // NOLINT(cppcoreguidelines-owning-memory)
|
||||
REQUIRE(file != nullptr);
|
||||
CHECK(std::fwrite(input.data(), 1, input.size(), file) == input.size());
|
||||
std::rewind(file);
|
||||
ValuelessEventCounter from_file;
|
||||
CHECK(!json::sax_parse(file, &from_file, json::input_format_t::ubjson));
|
||||
CHECK(from_file.error_id == 408);
|
||||
std::fclose(file); // NOLINT(cert-err33-c,cppcoreguidelines-owning-memory)
|
||||
}
|
||||
|
||||
SECTION("the limit is exact and the bytes read raise it")
|
||||
{
|
||||
// [ [$Z#l 524288] [$U#l 4096 + 4096 bytes] [$Z#l count] ]: chars_read
|
||||
// is 1 + 9 + (9 + 4096) + 9 = 4124 when the last header has been read,
|
||||
// so the last array may hold 4 * 4124 = 16496 values on top of the
|
||||
// base of 1048576 less the 524288 of the first array
|
||||
const std::uint32_t last_allowed = 1048576 + 16496 - 524288;
|
||||
for (const std::uint32_t count :
|
||||
{
|
||||
last_allowed, last_allowed + 1
|
||||
})
|
||||
{
|
||||
CAPTURE(count)
|
||||
std::vector<std::uint8_t> input = {'['};
|
||||
append_valueless_array(input, 'Z', 524288);
|
||||
input.insert(input.end(), {'[', '$', 'U', '#', 'l', 0x00, 0x00, 0x10, 0x00});
|
||||
input.insert(input.end(), 4096, 0x01);
|
||||
append_valueless_array(input, 'Z', count);
|
||||
input.push_back(']');
|
||||
ValuelessEventCounter sax;
|
||||
const bool accepted = count == last_allowed;
|
||||
CHECK(json::sax_parse(input, &sax, json::input_format_t::ubjson) == accepted);
|
||||
CHECK(sax.error_id == (accepted ? 0 : 408));
|
||||
CHECK(sax.values == 524288u + 4096u + (accepted ? count : 0u));
|
||||
}
|
||||
|
||||
// an array of 1048576 values on its own is accepted as before
|
||||
const std::vector<std::uint8_t> single = {'[', '$', 'Z', '#', 'l', 0x00, 0x10, 0x00, 0x00};
|
||||
// (the handler ends the parse at the first value, so that the million
|
||||
// values are not read)
|
||||
ValuelessEventCounter sax;
|
||||
sax.stop_after = 1;
|
||||
CHECK(!json::sax_parse(single, &sax, json::input_format_t::ubjson));
|
||||
CHECK(sax.error_id == 0);
|
||||
CHECK(sax.values == 1);
|
||||
}
|
||||
|
||||
SECTION("to_ubjson writes the typed form up to the base only")
|
||||
{
|
||||
// 1048 rows of 1000 values fit under the base of 1048576, the 1049th
|
||||
// would not: its elements are written one by one. Written all typed,
|
||||
// 1100 rows of 6 bytes would be refused by a reader (1075 rows are the
|
||||
// most it accepts). The rows hold null, true or false, and the
|
||||
// fallback does not depend on which
|
||||
std::vector<std::uint8_t> encoded;
|
||||
{
|
||||
json table = json::array();
|
||||
for (int row = 0; row < 1100; ++row)
|
||||
{
|
||||
const json cell = row % 3 == 0 ? json(nullptr) : json(row % 3 == 1);
|
||||
table.push_back(json(1000, cell));
|
||||
}
|
||||
encoded = json::to_ubjson(table, true, true);
|
||||
}
|
||||
// outer array: `[$[#I` and the count (7 bytes), then 1048 typed rows of 6
|
||||
// bytes (`$Z#I` and two bytes) and 52 rows of `#I`, two bytes, and 1000 `Z`
|
||||
CHECK(encoded.size() == 7 + (1048 * 6) + (52 * (4 + 1000)));
|
||||
|
||||
// the output is read back
|
||||
ValuelessEventCounter sax;
|
||||
CHECK(json::sax_parse(encoded, &sax, json::input_format_t::ubjson));
|
||||
CHECK(sax.error_id == 0);
|
||||
CHECK(sax.values == 1100u * 1000u);
|
||||
}
|
||||
|
||||
SECTION("BJData is not affected")
|
||||
{
|
||||
// BJData has no optimized arrays of these types, and the writer does
|
||||
// not charge them
|
||||
json table = json::array();
|
||||
for (int row = 0; row < 3; ++row)
|
||||
{
|
||||
table.push_back(json(10, nullptr));
|
||||
}
|
||||
const auto encoded = json::to_bjdata(table, true, true);
|
||||
CHECK(json::from_bjdata(encoded) == table);
|
||||
|
||||
const std::vector<std::uint8_t> input = {'[', '$', 'Z', '#', 'l', 0x00, 0x10, 0x00, 0x00};
|
||||
CHECK_THROWS_AS(_ = json::from_bjdata(input), json::parse_error&);
|
||||
}
|
||||
|
||||
SECTION("an object of a valueless type needs a key for every value")
|
||||
{
|
||||
// an object member costs a key, so the count is bounded by the input
|
||||
// and is not charged to the budget
|
||||
const std::vector<std::uint8_t> input = {'{', '$', 'Z', '#', 'l', 0x00, 0x10, 0x00, 0x00};
|
||||
ValuelessEventCounter sax;
|
||||
CHECK(!json::sax_parse(input, &sax, json::input_format_t::ubjson));
|
||||
CHECK(sax.error_id == 110);
|
||||
CHECK(sax.values == 0);
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("UBJSON an optimized array of no-ops is empty whatever its count")
|
||||
{
|
||||
// A no-op ('N') is not a value, so `[$N#<n>` is an array without elements
|
||||
// for every n. The count used to be passed on to the SAX parser, which
|
||||
// told it to expect n elements, and the DOM parser reserved storage for
|
||||
// them: 16384 values (256 KB) for a count of 2^31 in 9 bytes of input.
|
||||
SECTION("the SAX parser is told that the array is empty")
|
||||
{
|
||||
for (const std::vector<std::uint8_t>& input :
|
||||
{
|
||||
std::vector<std::uint8_t> {'[', '$', 'N', '#', 'i', 2},
|
||||
std::vector<std::uint8_t> {'[', '$', 'N', '#', 'I', 0x7F, 0xFF},
|
||||
std::vector<std::uint8_t> {'[', '$', 'N', '#', 'l', 0x7F, 0xFF, 0xFF, 0xFF},
|
||||
std::vector<std::uint8_t> {'[', '$', 'N', '#', 'i', 0}
|
||||
})
|
||||
{
|
||||
SaxEventLogger logger;
|
||||
CHECK(json::sax_parse(input, &logger, json::input_format_t::ubjson));
|
||||
CHECK(logger.events == std::vector<std::string>({"start_array(0)", "end_array()"}));
|
||||
}
|
||||
|
||||
// between other values, the next value is read as usual
|
||||
const std::vector<std::uint8_t> nested = {'[', '[', '$', 'N', '#', 'l', 0x7F, 0xFF, 0xFF, 0xFF, 'T', '[', '$', 'N', '#', 'i', 3, ']'};
|
||||
SaxEventLogger logger;
|
||||
CHECK(json::sax_parse(nested, &logger, json::input_format_t::ubjson));
|
||||
CHECK(logger.events == std::vector<std::string>({"start_array()", "start_array(0)", "end_array()", "boolean(true)", "start_array(0)", "end_array()", "end_array()"}));
|
||||
}
|
||||
|
||||
SECTION("the DOM parser reserves nothing")
|
||||
{
|
||||
// 2000 arrays of 9 bytes that announce 2^31 elements each: with a
|
||||
// reservation of 16384 values per array this took about 500 MB
|
||||
std::vector<std::uint8_t> input = {'['};
|
||||
for (int i = 0; i < 2000; ++i)
|
||||
{
|
||||
input.insert(input.end(), {'[', '$', 'N', '#', 'l', 0x7F, 0xFF, 0xFF, 0xFF});
|
||||
}
|
||||
input.push_back(']');
|
||||
|
||||
const json j = json::from_ubjson(input);
|
||||
REQUIRE(j.size() == 2000);
|
||||
std::size_t capacity = 0;
|
||||
for (const auto& element : j)
|
||||
{
|
||||
CHECK(element.is_array());
|
||||
CHECK(element.empty());
|
||||
capacity += element.get_ref<const json::array_t&>().capacity();
|
||||
}
|
||||
CHECK(capacity == 0);
|
||||
|
||||
// the 7-byte form inside an optimized array of arrays
|
||||
std::vector<std::uint8_t> typed = {'[', '$', '[', '#', 'I', 0x07, 0xD0};
|
||||
for (int i = 0; i < 2000; ++i)
|
||||
{
|
||||
typed.insert(typed.end(), {'$', 'N', '#', 'I', 0x7F, 0xFF});
|
||||
}
|
||||
const json k = json::from_ubjson(typed);
|
||||
REQUIRE(k.size() == 2000);
|
||||
capacity = 0;
|
||||
for (const auto& element : k)
|
||||
{
|
||||
CHECK(element.empty());
|
||||
capacity += element.get_ref<const json::array_t&>().capacity();
|
||||
}
|
||||
CHECK(capacity == 0);
|
||||
}
|
||||
|
||||
SECTION("a count that no container could hold is no longer an error")
|
||||
{
|
||||
// the array is empty, so nothing has to fit: 0x7FFFFFFF used to be
|
||||
// reported as out_of_range.408 where it exceeds the maximum size of
|
||||
// an array, and is accepted now
|
||||
const std::vector<std::uint8_t> input = {'[', '$', 'N', '#', 'l', 0x7F, 0xFF, 0xFF, 0xFF};
|
||||
CHECK(json::from_ubjson(input) == json::array());
|
||||
|
||||
// a count that does not fit into std::size_t still is
|
||||
#if SIZE_MAX == 0xffffffff
|
||||
const std::vector<std::uint8_t> too_large = {'[', '$', 'N', '#', 'L', 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00};
|
||||
json _;
|
||||
CHECK_THROWS_AS(_ = json::from_ubjson(too_large), json::out_of_range&);
|
||||
#endif
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("issue #5405 - array reserve for definite-length UBJSON arrays")
|
||||
{
|
||||
#if !defined(JSON_NOEXCEPTION)
|
||||
@@ -2881,60 +2541,6 @@ TEST_CASE("issue #5405 - array reserve for definite-length UBJSON arrays")
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("UBJSON - open arrays reserve only what the input has backed")
|
||||
{
|
||||
// an array that announces its size gets storage for (up to 16384 of) its
|
||||
// elements when it opens; the arrays that are open at the same time must
|
||||
// not hold more than the input read so far can back
|
||||
SECTION("nested headers that announce 16384 elements and are never completed")
|
||||
{
|
||||
// 200 levels would hold 200 * 16384 * sizeof(json),
|
||||
// if each of them reserved what it announces
|
||||
const auto input = utils::repeated({'[', '#', 'I', 0x40, 0x00}, 200);
|
||||
|
||||
utils::counts() = utils::allocation_counts{};
|
||||
const auto j = utils::counted_json::from_ubjson(input, true, false);
|
||||
CHECK(j.is_discarded());
|
||||
CAPTURE(utils::counts().peak)
|
||||
// the credit is 65,536 values; the base peaks at more than 3 million
|
||||
CHECK(utils::counts().peak < sizeof(utils::counted_json) * 128 * 1024);
|
||||
}
|
||||
|
||||
SECTION("a completed array is still reserved in one allocation")
|
||||
{
|
||||
json array = json::array();
|
||||
for (int i = 0; i < 10000; ++i)
|
||||
{
|
||||
array.push_back(i % 100);
|
||||
}
|
||||
|
||||
// an array without a size is not reserved, so only these two
|
||||
{
|
||||
const auto packed = json::to_ubjson(array, true);
|
||||
|
||||
utils::counts() = utils::allocation_counts{};
|
||||
const auto j = utils::counted_json::from_ubjson(packed);
|
||||
CHECK(j.size() == 10000);
|
||||
// the array and its storage; growing the storage as the elements
|
||||
// arrive would take more than a dozen allocations
|
||||
CAPTURE(utils::counts().allocations)
|
||||
CHECK(utils::counts().allocations <= 4);
|
||||
}
|
||||
|
||||
{
|
||||
const auto packed = json::to_ubjson(array, true, true);
|
||||
|
||||
utils::counts() = utils::allocation_counts{};
|
||||
const auto j = utils::counted_json::from_ubjson(packed);
|
||||
CHECK(j.size() == 10000);
|
||||
// the array and its storage; growing the storage as the elements
|
||||
// arrive would take more than a dozen allocations
|
||||
CAPTURE(utils::counts().allocations)
|
||||
CHECK(utils::counts().allocations <= 4);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
TEST_CASE("Universal Binary JSON Specification Examples 1")
|
||||
{
|
||||
|
||||
Reference in new issue
Block a user