Compare commits

..
Author SHA1 Message Date
Niels Lohmann b3dd34036a Make unflatten() linear in the length of the keys
unflatten() decides in a first pass which prefixes of the keys denote arrays
(those that have a reference token 0 among their children), and looks up
prefixes again in the second pass. A prefix was identified by hashing it, and
by comparing it byte by byte with the entry in the set, from its beginning.
A key with n tokens therefore cost n^2/2 bytes: 100,000 tokens "/0" took
2.4 s (3.12.0: 5 ms), and so did "/0" followed by "/1" tokens, because every
new level looks its prefix up. Keys in the order of an unsorted object type
(ordered_json) defeated the shortcut for keys that share a beginning with the
previous one: four keys were enough to be quadratic.

Prefixes of less than 64 bytes are kept in a hash table and compared as
before, which costs a bounded number of bytes. Longer prefixes are kept in a
tree whose nodes are looked up by their parent and the text of one token. The
node of a prefix is found by walking the tokens of the key from the root, so
all prefixes of a key together cost one pass over the key, and no prefix is
compared as a whole. The nodes along the key walked last are kept, so keys
that share a beginning do not walk it again. The array/object decision and
the independence from the order of the keys are unchanged.

The hash function is seeded with a value derived from the address of a local
variable, which address space layout randomization changes between
processes, so keys cannot be crafted to collide in advance: with the fixed
constants, 20,000 keys of 16 + 2 bytes with the same hash took 0.5 s (random
keys, and 3.12.0: 7 ms). Where addresses are not randomized, or when the seed
is known, k short prefixes can still collide and cost k^2 / 2 comparisons of
fewer than 64 bytes. Results do not depend on the seed.

Measurements (Apple clang -O2, arm64), unflatten() of one key with n tokens:

  tokens "/0"          n=25k  n=50k  n=100k  n=200k
  before              175 ms 617 ms 2437 ms 9729 ms
  after                 4 ms   6 ms   12 ms   27 ms
  3.12.0                1 ms   3 ms    5 ms   10 ms

  "/0" then "/1"       n=25k  n=50k  n=100k  n=200k
  before              153 ms 603 ms 2436 ms 9678 ms
  after                 1 ms   3 ms    5 ms   10 ms
  3.12.0                1 ms   3 ms    5 ms   11 ms

  ordered_json, 4 keys /0*m/a0, /10, /0*m/a1, /11
                       m=25k  m=50k  m=100k  m=200k
  before              397 ms 1402 ms 5191 ms 20723 ms
  after                 4 ms   7 ms   15 ms   33 ms
  3.12.0                2 ms   3 ms    8 ms   17 ms

Ordinary documents (flatten() then unflatten(), instructions retired): within
0.3 % of before for canada, citm_catalog, twitter, jeopardy, mesh,
update-center and random from the benchmark corpus.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-11 12:36:01 +01:00
32 changed files with 1090 additions and 1576 deletions

No files matched your search

+1 -1
View File
@@ -153,7 +153,7 @@ jobs:
container: ubuntu:24.04
strategy:
matrix:
target: [ci_cmake_flags, ci_test_diagnostics, ci_test_diagnostic_positions, ci_test_noexceptions, ci_test_noimplicitconversions, ci_test_legacycomparison, ci_test_noglobaludls, ci_test_disableenumserialization, ci_test_disabletuplereferenceconversion, ci_test_skiplibraryversioncheck, ci_test_simdutf, ci_test_strict_nul_handling, ci_test_delete_deprecated_functions, ci_test_no_thread_local, ci_test_recursion_depth_limit_0]
target: [ci_cmake_flags, ci_test_diagnostics, ci_test_diagnostic_positions, ci_test_noexceptions, ci_test_noimplicitconversions, ci_test_legacycomparison, ci_test_noglobaludls, ci_test_disableenumserialization, ci_test_disabletuplereferenceconversion, ci_test_skiplibraryversioncheck, ci_test_simdutf, ci_test_strict_nul_handling, ci_test_delete_deprecated_functions, ci_test_no_thread_local]
steps:
- name: Install build-essential
run: apt-get update ; apt-get install -y build-essential unzip wget git
-21
View File
@@ -342,27 +342,6 @@ add_custom_target(ci_test_no_thread_local
COMMENT "Compile and test without thread-local storage"
)
###############################################################################
# Change the number of levels the walks over a value recurse into.
###############################################################################
# JSON_RECURSION_DEPTH_LIMIT is 128 by default, which is what the whole test
# suite otherwise runs with. 0 sends every walk to its explicit stack at once,
# so this target runs the whole suite through the walks that the default only
# uses for values nested more deeply than 128 levels: results must not depend on
# the limit. The largest limit, 254, is covered by a build of
# tests/src/unit-recursion_depth_limit.cpp (see tests/CMakeLists.txt).
add_custom_target(ci_test_recursion_depth_limit_0
COMMAND ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Debug -GNinja
-DJSON_BuildTests=ON -DJSON_FastTests=ON
-DCMAKE_CXX_FLAGS=-DJSON_RECURSION_DEPTH_LIMIT=0
-S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_recursion_depth_limit_0
COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_recursion_depth_limit_0
COMMAND cd ${PROJECT_BINARY_DIR}/build_recursion_depth_limit_0 && ${CMAKE_CTEST_COMMAND} --parallel ${N} --output-on-failure
COMMENT "Compile and test with JSON_RECURSION_DEPTH_LIMIT=0"
)
###############################################################################
# Coverage.
###############################################################################
-2
View File
@@ -40,8 +40,6 @@ header. See also the [macro overview page](../../features/macros.md).
- [**JSON_NO_IO**](json_no_io.md) - switch off functions relying on certain C++ I/O headers
- [**JSON_NO_SIMD**](json_no_simd.md) - use only portable code instead of vector instructions (SSE2, SSSE3, NEON)
- [**JSON_NO_THREAD_LOCAL**](json_no_thread_local.md) - switch off the use of `thread_local` storage
- [**JSON_RECURSION_DEPTH_LIMIT**](json_recursion_depth_limit.md) - set how many nesting levels the operations on a value
recurse into on the call stack (default 128)
- [**JSON_SKIP_UNSUPPORTED_COMPILER_CHECK**](json_skip_unsupported_compiler_check.md) - do not warn about unsupported compilers
- [**JSON_USE_GLOBAL_UDLS**](json_use_global_udls.md) - place user-defined string literals (UDLs) into the global namespace
- [**JSON_USE_SIMDUTF**](json_use_simdutf.md) - use the simdutf library to accelerate UTF-8 validation
@@ -45,7 +45,6 @@ Copying and comparing fall back to working without the call stack there, as they
## See also
- [`JSON_RECURSION_DEPTH_LIMIT`](json_recursion_depth_limit.md) - set how many nesting levels the operations recurse into
- [FAQ: Thread safety](../../home/faq.md#thread-safety) - thread-safety guarantees of the library
- [Supported Macros](../../features/macros.md) - guide to all supported macros
@@ -1,307 +0,0 @@
# JSON_RECURSION_DEPTH_LIMIT
```cpp
#define JSON_RECURSION_DEPTH_LIMIT /* value */
```
This macro sets how many nesting levels the operations that walk a value recurse into on the call stack. Beyond that,
they continue with an explicit stack on the heap. The value is an integer constant from `0` to `254`; the default is
`128`.
The operations are copying a value, comparing two values, serializing with [`dump`](../basic_json/dump.md) or
[`operator<<`](../operator_ltlt.md), hashing with `std::hash`, [`merge_patch`](../basic_json/merge_patch.md),
[`update`](../basic_json/update.md) with `merge_objects`, [`diff`](../basic_json/diff.md),
[`merge_diff`](../basic_json/merge_diff.md), converting a value to another specialization of `basic_json`, destroying a
value, and the binary writers [`to_cbor`](../basic_json/to_cbor.md),
[`to_msgpack`](../basic_json/to_msgpack.md), [`to_ubjson`](../basic_json/to_ubjson.md),
[`to_bjdata`](../basic_json/to_bjdata.md), and [`to_bon8`](../basic_json/to_bon8.md). They share this one limit; none
has
a limit of its own. Recursing is the fastest way to walk a value, but a value nested deeply enough would exhaust the
call
stack, so the library recurses only as far as the limit and finishes the levels below with an explicit stack. This is
what makes all of them safe for untrusted input
(see [Nesting depth](../../features/parsing/untrusted_input.md#nesting-depth)). Parsing, the binary readers,
[`flatten`](../basic_json/flatten.md), [`unflatten`](../basic_json/unflatten.md), and
[`to_bson`](../basic_json/to_bson.md) always use an explicit stack and do not depend on the macro. The lockstep
comparison of two [`json_view`](../basic_json_view/index.md) documents has a fixed bound of its own, 48 levels,
and does not depend on it either.
The call stack that these operations need grows with the limit: roughly *a few kilobytes plus a cost per level times the
limit*, where the cost per level is up to about 2.3 KiB in an unoptimized and up to about 0.5 KiB in an optimized build
(see [Stack needed](#stack-needed)). The default of `128` levels is tuned for threads with ordinary stacks of a few
hundred kilobytes or more. Set a lower limit if your threads have small stacks (RTOS tasks, fibers, threads of the musl
C
library, which has a default stack of 128 KiB), or a higher limit to trade stack for speed on deeply nested values.
The macro changes only how much stack is used and how fast the operations are. It never changes a result: every
operation returns the same value for every setting of the limit, for values nested less deeply than the limit as well as
for values nested more deeply.
## Default definition
The default value is `128`.
```cpp
#define JSON_RECURSION_DEPTH_LIMIT 128
```
When the macro is not defined, the library defines it to its default value.
## Notes
!!! note "Allowed values"
The value must be an integer constant expression from `0` to `254`; any other value (a negative number, a number
above `254`, a floating-point number such as `254.0`, or `true` or `false`) is rejected at compile time with an
error message that names the macro. (A character such as `'a'` is an integer, too, and is accepted as its code.)
- `0` is allowed, and means that every operation uses its explicit stack from the first level on and no operation
recurses over the nesting of a value at all. That needs a constant amount of call stack (at most 16 KiB in the
measurements below, for a value nested 20,000 levels deep) and is the setting for tiny stacks, at the price of
speed (see [Speed](#speed)).
- `254` is the largest allowed value. Copying a value and comparing two values (with `<=>`, or with the operators on
objects and arrays of a custom type) cannot pass the depth down as an argument, because the signatures of the
copy constructor and the operators are fixed. They count the levels they have descended into in a variable of type
`#!cpp std::uint8_t` that exists once per thread (see [`JSON_NO_THREAD_LOCAL`](json_no_thread_local.md)). Only
the levels that are descended into are counted, so the count never exceeds the limit, not even when user code (a
copy constructor of a custom base class, say) copies a value while a copy is going on. A byte holds `255` at
most, and the limit stays below that, so that the count cannot wrap around. (More levels would hardly be
useful: `254` levels need up to 576 KiB of stack in an unoptimized build already, twice as much as the
default.)
### Stack needed
The tables show the smallest stack that an operation needs, in KiB, at several limits, on a value that is nested
more deeply than the limit (limit + 10 levels). Each cell shows the result for Clang (Apple clang 21) and for
GCC 16, separated by a slash; both measured on macOS with arm64, compiling in C++20 mode. The operations run in a
thread whose stack has the size given, and are not given any stack that they do not need: the value is built, and
the result destroyed, outside of that thread. Stacks can only be set in steps of 16 KiB (the page size), so `16`
means "16 KiB or less". A row stands for all operations listed; the table shows the largest value among them.
Unoptimized (`-O0`):
| Operation | limit 16 | limit 32 | limit 64 | limit 128 | limit 254 |
|---|---:|---:|---:|---:|---:|
| `dump()` and `dump(indent)`, `operator<<` | 48 / 32 | 80 / 64 | 144 / 128 | 288 / 256 | 576 / 512 |
| copy construction and assignment | 32 / 16 | 48 / 32 | 112 / 80 | 224 / 144 | 432 / 304 |
| `==` | 16 / 16 | 16 / 16 | 16 / 16 | 48 / 32 | 96 / 48 |
| `<=>` (C++20) | 16 / 16 | 16 / 16 | 32 / 32 | 64 / 64 | 144 / 128 |
| `std::hash` | 16 / 16 | 16 / 16 | 32 / 32 | 80 / 64 | 160 / 128 |
| `merge_patch`, `update(j, true)`, `patch` | 32 / 16 | 48 / 32 | 112 / 80 | 224 / 144 | 432 / 304 |
| `diff` | 16 / 16 | 16 / 16 | 32 / 32 | 64 / 48 | 144 / 96 |
| `merge_diff` | 16 / 16 | 48 / 32 | 80 / 48 | 176 / 112 | 352 / 224 |
| `to_cbor`, `to_msgpack` | 16 / 16 | 32 / 16 | 64 / 32 | 112 / 80 | 224 / 160 |
| `to_ubjson`, `to_bjdata` | 16 / 16 | 16 / 16 | 48 / 32 | 96 / 64 | 176 / 112 |
| `to_bon8` | 16 / 16 | 16 / 16 | 16 / 16 | 32 / 16 | 64 / 48 |
| conversion to another `basic_json` type | 32 / 16 | 64 / 32 | 112 / 80 | 224 / 144 | 448 / 304 |
| destruction | 16 / 16 | 16 / 16 | 16 / 16 | 16 / 16 | 32 / 32 |
Optimized (`-O2`):
| Operation | limit 16 | limit 32 | limit 64 | limit 128 | limit 254 |
|---|---:|---:|---:|---:|---:|
| `dump()` and `dump(indent)`, `operator<<` | 16 / 16 | 16 / 16 | 16 / 16 | 16 / 16 | 32 / 32 |
| copy construction and assignment | 16 / 16 | 16 / 16 | 16 / 16 | 32 / 32 | 64 / 48 |
| `==` | 16 / 16 | 16 / 16 | 16 / 16 | 16 / 16 | 16 / 16 |
| `<=>` (C++20) | 16 / 16 | 16 / 16 | 16 / 16 | 16 / 16 | 48 / 16 |
| `std::hash` | 16 / 16 | 16 / 16 | 16 / 16 | 32 / 48 | 80 / 80 |
| `merge_patch`, `update(j, true)`, `patch` | 16 / 16 | 16 / 16 | 16 / 16 | 32 / 32 | 64 / 48 |
| `diff` | 16 / 16 | 16 / 16 | 16 / 32 | 48 / 48 | 96 / 112 |
| `merge_diff` | 16 / 16 | 16 / 16 | 16 / 16 | 48 / 32 | 96 / 64 |
| `to_cbor`, `to_msgpack` | 16 / 16 | 16 / 16 | 16 / 16 | 16 / 16 | 16 / 16 |
| `to_ubjson`, `to_bjdata` | 16 / 16 | 16 / 16 | 16 / 16 | 32 / 32 | 64 / 64 |
| `to_bon8` | 16 / 16 | 16 / 16 | 16 / 16 | 16 / 32 | 32 / 64 |
| conversion to another `basic_json` type | 16 / 16 | 16 / 16 | 16 / 16 | 32 / 48 | 64 / 96 |
| destruction | 16 / 16 | 16 / 16 | 16 / 16 | 16 / 16 | 16 / 16 |
With a limit of `0`, every operation in the tables ran in a 16 KiB stack at `-O0` (Clang and GCC), on a value nested
20,000 levels deep. The operations that do not depend on the macro (`flatten`, `unflatten`, `to_bson`) needed
16 KiB or less at every limit.
The rule of thumb is
> stack needed ≈ a few KiB + cost per level × limit
with a cost per level of up to 2.3 KiB (`dump` at `-O0` with Clang; 2.1 KiB with GCC), 1.7 KiB for copying and merging
(1.3 KiB with GCC), and 0.1 to 0.5 KiB for most other operations at `-O0`; and 0.5 KiB or less at `-O2`. The numbers
come from the difference between the limits `128` and `254` in the tables, so they are rough: other compilers,
other targets (the frames of x86-64 code are about as large), other standard libraries, `JSON_DIAGNOSTICS`, and
sanitizers change them. Measure with your own tool chain (see the example below) rather than relying on the tables.
The stack that your own code and the operating system need comes on top of these numbers.
Which limit to choose, for the stack that you can spare for the library (the largest limit of those measured
for which the slowest operation needs at most that much):
| Stack for the library | optimized (`-O2`) | unoptimized (`-O0`) |
|---|---:|---:|
| 32 KiB | 64 | 0 |
| 64 KiB | 128 | 16 |
| 128 KiB (the default stack of threads on musl) | 254 | 32 |
| 256 KiB | 254 | 64 |
| 512 KiB | 254 | 128 |
| 1 MiB or more | 254 | 254 |
### Speed
Operations are faster the more of a value they handle with plain recursion. The levels below the limit are handled with
an explicit stack, which has to be kept on the heap and costs more per level. The macro therefore trades speed against
stack.
The table shows the time of copying a value (including destroying the copy), of `==` on two equal values, of `dump`, and
of `std::hash` for several limits, relative to the default of `128` (`1.00` means the same time; more is slower). It was
measured with Clang 21 at `-O2` in C++17 mode on macOS with arm64, as the best of seven runs of each setting (two
rounds, best of both). The documents are ordinary JSON files that are nested 7 (`canada.json`), 8
(`citm_catalog.json`), 10 (`twitter.json`), and 2 levels deep (`jeopardy.json`); the other values are one array or object
with one member per level, nested 1,000 and 100,000 levels deep. The absolute time at the limit `128` is given in the
third column. Differences of up to about 0.1 are noise: a different limit changes the layout of the compiled code, and
timings of the same binary vary by about as much. The slow columns (limits 0 and 1) vary more: repeating `std::hash` on
`canada.json` gave 1.27 to 1.30 for limit 0 and 1.4 to 1.9 for limit 1, so read the cells of those two columns as
accurate to about 0.3.
| Value | Operation | limit 128 (absolute) | limit 0 | limit 1 | limit 16 | limit 254 |
|---|---|---:|---:|---:|---:|---:|
| `canada.json` (2.1 MB) | copy | 3.5 ms | 1.80 | 1.80 | 0.99 | 1.00 |
| `canada.json` (2.1 MB) | `==` | 350.5 µs | 2.36 | 2.38 | 1.00 | 0.99 |
| `canada.json` (2.1 MB) | `dump` | 2.4 ms | 1.11 | 1.11 | 1.00 | 1.00 |
| `canada.json` (2.1 MB) | `std::hash` | 920.2 µs | 1.33 | 1.59 | 1.00 | 1.00 |
| `citm_catalog.json` (1.7 MB) | copy | 1.8 ms | 1.38 | 1.36 | 1.00 | 0.99 |
| `citm_catalog.json` (1.7 MB) | `==` | 275.5 µs | 1.31 | 1.31 | 0.99 | 1.00 |
| `citm_catalog.json` (1.7 MB) | `dump` | 314.6 µs | 1.24 | 1.23 | 1.00 | 1.00 |
| `citm_catalog.json` (1.7 MB) | `std::hash` | 309.1 µs | 1.77 | 1.42 | 1.06 | 1.05 |
| `twitter.json` (0.6 MB) | copy | 865.7 µs | 1.45 | 1.47 | 1.01 | 1.00 |
| `twitter.json` (0.6 MB) | `==` | 135.8 µs | 1.26 | 1.23 | 1.00 | 1.00 |
| `twitter.json` (0.6 MB) | `dump` | 224.6 µs | 1.12 | 1.12 | 1.01 | 1.00 |
| `twitter.json` (0.6 MB) | `std::hash` | 132.9 µs | 1.28 | 1.27 | 1.03 | 1.04 |
| `jeopardy.json` (53 MB) | copy | 111.1 ms | 1.22 | 1.25 | 1.00 | 1.00 |
| `jeopardy.json` (53 MB) | `==` | 23.5 ms | 1.29 | 1.53 | 1.07 | 1.00 |
| `jeopardy.json` (53 MB) | `dump` | 33.2 ms | 1.14 | 1.34 | 1.03 | 1.01 |
| `jeopardy.json` (53 MB) | `std::hash` | 19.9 ms | 1.17 | 1.52 | 1.05 | 1.00 |
| arrays, 1,000 levels | copy | 69.9 µs | 0.95 | 0.98 | 0.92 | 0.99 |
| arrays, 1,000 levels | `==` | 10.4 µs | 1.02 | 1.01 | 0.96 | 1.00 |
| arrays, 1,000 levels | `dump` | 9.6 µs | 0.97 | 0.96 | 0.97 | 1.06 |
| arrays, 1,000 levels | `std::hash` | 16.9 µs | 1.00 | 1.01 | 1.01 | 0.98 |
| objects, 1,000 levels | copy | 77.1 µs | 0.98 | 0.97 | 0.97 | 1.03 |
| objects, 1,000 levels | `==` | 12.0 µs | 0.96 | 0.95 | 0.95 | 1.07 |
| objects, 1,000 levels | `dump` | 12.6 µs | 0.97 | 0.95 | 0.98 | 1.01 |
| objects, 1,000 levels | `std::hash` | 17.3 µs | 0.99 | 1.01 | 0.98 | 1.00 |
| arrays, 100,000 levels | copy | 6.4 ms | 1.00 | 1.00 | 1.00 | 1.00 |
| arrays, 100,000 levels | `==` | 1.1 ms | 1.01 | 1.03 | 1.00 | 1.06 |
| arrays, 100,000 levels | `dump` | 917.9 µs | 1.01 | 1.01 | 1.02 | 1.01 |
| arrays, 100,000 levels | `std::hash` | 1.7 ms | 1.04 | 1.05 | 0.99 | 0.99 |
| objects, 100,000 levels | copy | 7.5 ms | 0.99 | 1.02 | 1.01 | 0.99 |
| objects, 100,000 levels | `==` | 1.3 ms | 1.02 | 1.09 | 1.08 | 0.96 |
| objects, 100,000 levels | `dump` | 1.2 ms | 1.06 | 0.99 | 1.01 | 0.99 |
| objects, 100,000 levels | `std::hash` | 1.8 ms | 1.00 | 1.00 | 1.00 | 1.00 |
The results are as follows.
- A limit of `16` or more does not change the speed of ordinary documents (differences are within noise), because they
are not nested more deeply than that anyway. A document nested more deeply than the limit is walked partly with the
explicit stack.
- A limit of `0` or `1` makes the operations on ordinary documents 1.1 to 2.4 times as slow, because even their first
levels are handled with the explicit stack. The slowdown is largest for `==` on a document of many small containers
(`canada.json`: 2.4 times) and for copying (up to 1.8 times).
- For values that are nested far more deeply than any limit (1,000 and 100,000 levels deep), the limit makes no
measurable difference: almost all of the levels are handled with the explicit stack for every setting.
Raising the limit gains nothing for them, and lowering it costs nothing.
!!! note "Other macros"
- With [`JSON_NO_THREAD_LOCAL`](json_no_thread_local.md) defined (or for Clang targeting MinGW, where the library
defines it by itself), copying and comparing never descend, so for them the limit does not matter. The other
operations still recurse as many levels as the limit says. The range of the limit is the same.
- The macro is not part of the [ABI tag](../../features/namespace.md), as `JSON_NO_THREAD_LOCAL` is not, and it is
not undefined at the end of `json.hpp`.
!!! warning "Use the same value in every translation unit"
All translation units of a program that include the library must define the same value. Otherwise, the program
violates the One Definition Rule: the function `nlohmann::detail::recursion_depth_limit()` and all the functions
that
use it are defined differently in different translation units. The linker keeps one definition of each function that
several translation units define, and every caller then recurses as far as that definition says, whatever value
its own translation unit defined. Each function uses the limit of whichever translation unit's copy was kept, and
which one that is is not specified.
Mixing values cannot corrupt the byte per thread that counts the levels of copying and comparing: each limit
is at most `254`, and all of them stop counting at their own limit, so the count never exceeds `254`, whichever
translation unit's definition runs. The counter exists once per thread and `basic_json` specialization, and is
shared by all translation units. Nor can mixing values change any result, as no result depends on the limit. What
changes is how much stack is used and how fast the operations are. Nevertheless, avoid it: the program has undefined
behavior as far as the C++ standard is concerned.
## Examples
??? example "Example: lower the limit"
The example nests an array 5,000 levels deep. With the limit set to 16, the library recurses over the first 16
levels
and over the rest uses an explicit stack. The results are those of the default limit.
```cpp
--8<-- "examples/json_recursion_depth_limit.cpp"
```
Output:
```
--8<-- "examples/json_recursion_depth_limit.output"
```
??? example "Example: find out how much stack an operation needs"
The program runs `dump()` on a value nested `depth` levels deep in a thread with a stack of `kib` KiB. Compile it
with the options and the limit that you use, e.g. `-DJSON_RECURSION_DEPTH_LIMIT=64`, run it with `depth` set to the
limit plus 10 and decreasing values of `kib` (the stack size must be a multiple of the page size, often 4 or 16
KiB),
and note the smallest stack with which it still finishes. That is how the tables above were measured.
```cpp
#include <cstdlib>
#include <iostream>
#include <string>
#include <utility>
#include <pthread.h>
#include <nlohmann/json.hpp>
static nlohmann::json value;
static void* run(void*)
{
std::cout << "dump() ok, " << value.dump().size() << " bytes\n";
return nullptr;
}
int main(int argc, char** argv)
{
const int depth = std::atoi(argv[1]);
const std::size_t kib = static_cast<std::size_t>(std::atoi(argv[2]));
// built on the main thread, without recursion
value = 1;
for (int i = 0; i < depth; ++i)
{
nlohmann::json outer = nlohmann::json::array();
outer.push_back(std::move(value));
value = std::move(outer);
}
pthread_attr_t attr;
pthread_attr_init(&attr);
pthread_attr_setstacksize(&attr, kib * 1024);
pthread_t thread;
pthread_create(&thread, &attr, run, nullptr);
pthread_join(thread, nullptr);
}
```
## See also
- [Nesting depth](../../features/parsing/untrusted_input.md#nesting-depth) - how the library handles deeply nested
values
- [`JSON_NO_THREAD_LOCAL`](json_no_thread_local.md) - switch off the use of `thread_local` storage
- [Supported Macros](../../features/macros.md) - guide to all supported macros
## Version history
- Added in version 3.13.0.
+2 -5
View File
@@ -53,15 +53,12 @@ flowchart LR
only as bytes are actually read, arrays reserve at most a fixed number of elements up front, and sizes that no
container can hold are rejected.
- **Memory is owned by values.** Each `basic_json` value owns its content, and there is no manual memory management in
user code. The destructor recurses only over a bounded number of nesting levels and then continues without the call
stack, so destroying a deeply nested value does not exhaust the stack.
user code. The destructor does not recurse, so destroying a deeply nested value does not exhaust the stack.
- **Bounded recursion.** The JSON parser and the binary readers keep their state in explicit stacks instead of
recursing per nesting level. Operations that walk a value, such as [`dump`](../api/basic_json/dump.md), copying,
comparison, hashing, [`merge_patch`](../api/basic_json/merge_patch.md), [`diff`](../api/basic_json/diff.md),
[`flatten`](../api/basic_json/flatten.md), and the binary writers, recurse only up to a fixed depth and continue with
an explicit stack below it. The depth is 128 levels by default and can be changed with
[`JSON_RECURSION_DEPTH_LIMIT`](../api/macros/json_recursion_depth_limit.md), for example to run on a small stack.
Applications that process untrusted input can limit its nesting depth with a
an explicit stack below it. Applications that process untrusted input can limit its nesting depth with a
[parser callback](../features/parsing/parser_callbacks.md).
- **Invariants are checked.** The class invariant (for instance, that the pointer for the stored type is never null) is
checked with runtime assertions throughout the test suite.
@@ -1,29 +0,0 @@
#include <functional>
#include <iostream>
#include <utility>
// walk over at most 16 nesting levels on the call stack, and over the rest with an explicit stack
#define JSON_RECURSION_DEPTH_LIMIT 16
#include <nlohmann/json.hpp>
using json = nlohmann::json;
int main()
{
// an array nested 5,000 levels deep, built without recursion
json j = 1;
for (int i = 0; i < 5000; ++i)
{
json outer = json::array();
outer.push_back(std::move(j));
j = std::move(outer);
}
// the results are those of any other limit
const json copy = j;
std::cout << std::boolalpha;
std::cout << "limit: " << JSON_RECURSION_DEPTH_LIMIT << '\n';
std::cout << "copy equals original: " << (copy == j) << '\n';
std::cout << "length of dump(): " << j.dump().size() << '\n';
std::cout << "same hash: " << (std::hash<json> {}(copy) == std::hash<json> {}(j)) << '\n';
}
@@ -1,4 +0,0 @@
limit: 16
copy equals original: true
length of dump(): 10001
same hash: true
-10
View File
@@ -198,16 +198,6 @@ this is planned to become the default in version 4.0.0.
See [full documentation of `JSON_PRECISE_STREAM_POSITION`](../api/macros/json_precise_stream_position.md).
## `JSON_RECURSION_DEPTH_LIMIT`
Sets the number of nesting levels that the operations walking a value (copying, comparing, serializing, hashing,
merging, diffing, converting, destroying, and the binary writers) recurse into on the call stack before they continue
with an explicit stack on the heap. The default is `128`; allowed are the integers from `0` to `254`. Lower it for
threads with small stacks, raise it to trade stack for speed. Results are the same for every setting; the value must be
the same in all translation units of a program.
See [full documentation of `JSON_RECURSION_DEPTH_LIMIT`](../api/macros/json_recursion_depth_limit.md).
## `JSON_SKIP_LIBRARY_VERSION_CHECK`
When defined, the library will not create a compiler warning when a different version of the library was already
@@ -54,32 +54,10 @@ Once a value has been parsed, operations that walk it recursively — serializin
[`dump`](../../api/basic_json/dump.md), hashing it, copying it, comparing two values with `#!cpp ==`, `#!cpp <`, or (in
C++20) `#!cpp <=>`, merging with [`update`](../../api/basic_json/update.md), applying a
[`merge_patch`](../../api/basic_json/merge_patch.md), creating a patch with [`diff`](../../api/basic_json/diff.md) or
[`merge_diff`](../../api/basic_json/merge_diff.md), converting it to another `basic_json` type, destroying it, and the
binary writers (`to_cbor`, `to_msgpack`, ...) — descend at most
[`JSON_RECURSION_DEPTH_LIMIT`](../../api/macros/json_recursion_depth_limit.md) levels on the call stack, 128 by default,
and continue below that with an explicit stack on the heap. So the call stack does not grow with the nesting depth of the
value, however deeply it is nested. ([`flatten`](../../api/basic_json/flatten.md),
[`unflatten`](../../api/basic_json/unflatten.md), and [`to_bson`](../../api/basic_json/to_bson.md) never use the call
stack for the nesting.)
The call stack still needs room for the levels that are recursed into, and the default limit assumes a stack of ordinary
size. The stack that an operation needs is roughly proportional to the limit; with the default of 128 levels, the most
demanding operation (`dump` in an unoptimized build) needed up to 288 KiB, and an optimized build needed 48 KiB or less,
in measurements on macOS with arm64. The most demanding operation needed this much stack at the limits `0`, `32`,
`128`, and `254`:
| Stack needed, in KiB | limit 0 | limit 32 | limit 128 (default) | limit 254 |
|---|---:|---:|---:|---:|
| unoptimized build (`-O0`) | 16 or less | 80 | 288 | 576 |
| optimized build (`-O2`) | 16 or less | 16 or less | 48 | 112 |
A thread with a smaller stack (a task of a real-time operating system, a fiber, or a thread of the musl C library, whose
default stack is 128 KiB) can still overflow its stack on deeply nested input, which is why the limit can be lowered:
define `JSON_RECURSION_DEPTH_LIMIT` to a smaller value, down to `0`, before including the library. Such a limit changes
only the stack that is needed and the speed (values nested more deeply than the limit are walked more slowly), never a
result. See the [documentation of the macro](../../api/macros/json_recursion_depth_limit.md) for the complete tables,
the speed, and the rules for choosing a value. Destroying a value (its destructor) recurses over the same number of
levels and then continues without the call stack, and needs 32 KiB or less at every limit.
[`merge_diff`](../../api/basic_json/merge_diff.md), [`flatten`](../../api/basic_json/flatten.md), and the binary
writers (`to_cbor`, `to_msgpack`, ...) — descend at most 128 levels on the call stack and continue
below that with an explicit stack instead, so none of them can exhaust the stack either, however deeply the value is
nested. Destroying a value (its destructor) never recurses at all, regardless of nesting depth, for the same reason.
## Input size
-1
View File
@@ -454,7 +454,6 @@ nav:
- 'JSON_NO_SIMD': api/macros/json_no_simd.md
- 'JSON_NO_THREAD_LOCAL': api/macros/json_no_thread_local.md
- 'JSON_PRECISE_STREAM_POSITION': api/macros/json_precise_stream_position.md
- 'JSON_RECURSION_DEPTH_LIMIT': api/macros/json_recursion_depth_limit.md
- 'JSON_SKIP_LIBRARY_VERSION_CHECK': api/macros/json_skip_library_version_check.md
- 'JSON_SKIP_UNSUPPORTED_COMPILER_CHECK': api/macros/json_skip_unsupported_compiler_check.md
- 'JSON_STRICT_BINARY_UTF8': api/macros/json_strict_binary_utf8.md
+351 -135
View File
@@ -852,139 +852,6 @@ class json_pointer
}
private:
/*!
@brief the pointer prefixes of a flattened object that denote arrays
@ref unflatten creates arrays exactly below the prefixes that have a
reference token 0 among their children, and objects everywhere else.
Deciding this up front keeps the result independent of the order in which
the flattened object is iterated, which is unspecified for some object
types.
A prefix is identified by the text of the key it is the beginning of:
escaping is unambiguous, so equal prefixes of valid keys are equal texts.
The set refers to the keys, which must outlive it.
*/
class array_prefix_set
{
public:
void insert(const char* data, std::size_t size)
{
// flattened objects are usually sorted, so most keys that have a
// 0 yield the prefix of the previous one
if (last_data != nullptr && last_size == size && std::memcmp(last_data, data, size) == 0)
{
return;
}
last_data = data;
last_size = size;
if (table.empty())
{
table.resize(16);
}
const std::size_t hash = hash_of(data, size);
const std::size_t mask = table.size() - 1;
std::size_t i = hash & mask;
while (table[i].data != nullptr)
{
if (table[i].hash == hash && table[i].size == size && std::memcmp(table[i].data, data, size) == 0)
{
return;
}
i = (i + 1) & mask;
}
table[i] = slot(hash, data, size);
// keep the load factor below 1/2
if (++used * 2 > table.size())
{
grow();
}
}
bool empty() const noexcept
{
return used == 0;
}
bool contains(const char* data, std::size_t size) const noexcept
{
if (used == 0)
{
return false;
}
const std::size_t hash = hash_of(data, size);
const std::size_t mask = table.size() - 1;
for (std::size_t i = hash & mask; table[i].data != nullptr; i = (i + 1) & mask)
{
if (table[i].hash == hash && table[i].size == size && std::memcmp(table[i].data, data, size) == 0)
{
return true;
}
}
return false;
}
private:
struct slot
{
slot() noexcept = default;
slot(std::size_t h, const char* d, std::size_t s) noexcept
: hash(h), data(d), size(s)
{}
std::size_t hash = 0;
const char* data = nullptr; // nullptr: the slot is empty
std::size_t size = 0;
};
static std::size_t hash_of(const char* data, std::size_t size) noexcept
{
std::uint64_t h = 0x9E3779B97F4A7C15ULL ^ size;
std::size_t i = 0;
for (; i + 8 <= size; i += 8)
{
std::uint64_t w = 0;
std::memcpy(&w, data + i, 8);
h = (h ^ w) * 0x100000001B3ULL;
h ^= h >> 32u;
}
std::uint64_t w = 0;
if (i < size)
{
std::memcpy(&w, data + i, size - i);
}
h = (h ^ w) * 0x9E3779B97F4A7C15ULL;
return detail::conditional_static_cast<std::size_t>(h ^ (h >> 29u));
}
void grow()
{
std::vector<slot> old(table.size() * 2);
old.swap(table);
const std::size_t mask = table.size() - 1;
for (const auto& s : old)
{
if (s.data != nullptr)
{
std::size_t i = s.hash & mask;
while (table[i].data != nullptr)
{
i = (i + 1) & mask;
}
table[i] = s;
}
}
}
std::vector<slot> table{};
std::size_t used = 0;
const char* last_data = nullptr;
std::size_t last_size = 0;
};
/// @brief the length of the longest common prefix of the first @a limit bytes of @a a and @a b
static std::size_t common_prefix_length(const char* a, const char* b, const std::size_t limit) noexcept
{
@@ -1007,6 +874,354 @@ class json_pointer
return i;
}
/*!
@brief the prefixes of a flattened object that are marked as arrays, for
prefixes that are not short
A prefix is identified by the tokens it consists of: a node of a tree
stands for the prefix of its parent followed by a token, and is looked up
by its parent and the text of the token. The node of a prefix is found by
walking the tokens of the key from the root, so identifying all prefixes
of a key together costs one pass over the key, however long the prefixes
are, and no prefix is ever compared byte by byte as a whole. The nodes
along the key that was walked last are kept, and the walk of the next key
starts below the beginning it has in common with that key. The tree refers
to the keys, which must outlive it.
*/
class prefix_tree
{
public:
/// @brief set the seed of the hash function, before anything is added
void reseed(const std::uint64_t seed_value) noexcept
{
seed = seed_value;
}
/// @brief mark the first @a prefix_size bytes of the key of @a key_size bytes at @a data
void mark(const char* data, std::size_t key_size, std::size_t prefix_size)
{
nodes[walk(data, key_size, prefix_size)].marked = true;
any_marked = true;
}
bool is_marked(const char* data, std::size_t key_size, std::size_t prefix_size)
{
return any_marked && nodes[walk(data, key_size, prefix_size)].marked;
}
bool empty() const noexcept
{
return !any_marked;
}
/// @brief the hash of the first @a size bytes at @a data, for the seed @a seed_value
static std::size_t hash_bytes(const char* data, std::size_t size, const std::uint64_t seed_value) noexcept
{
std::uint64_t h = seed_value ^ size;
std::size_t i = 0;
for (; i + 8 <= size; i += 8)
{
std::uint64_t w = 0;
std::memcpy(&w, data + i, 8);
h = (h ^ w) * 0x100000001B3ULL;
h ^= h >> 32u;
}
std::uint64_t w = 0;
if (i < size)
{
std::memcpy(&w, data + i, size - i);
}
h = (h ^ w) * 0x9E3779B97F4A7C15ULL;
return detail::conditional_static_cast<std::size_t>(h ^ (h >> 29u));
}
private:
struct node
{
std::size_t parent = 0;
std::size_t hash = 0;
const char* token = nullptr;
std::size_t token_size = 0;
bool marked = false;
};
/// @brief a slot of the hash table; the index of a node, 0 if the slot is empty
struct bucket
{
std::size_t index = 0;
};
/// @brief the node after a token of the key that was walked last, and where the token ends
struct step
{
std::size_t node = 0;
std::size_t end = 0;
};
/// @brief the index of the node of the prefix that ends at the slash (or the end) at @a prefix_size
std::size_t walk(const char* data, std::size_t key_size, std::size_t prefix_size)
{
if (nodes.empty())
{
nodes.emplace_back();
table.resize(16);
chain.emplace_back();
}
if (data != last_key)
{
// keep the nodes within the beginning that this key has in common
// with the key they were found for; a prefix that ends inside a
// token of this key is not one of its prefixes
const std::size_t same = common_prefix_length(data, last_key, (std::min)(key_size, last_key_size));
std::size_t keep = chain.size();
while (keep > 1 && (chain[keep - 1].end > same || (chain[keep - 1].end == same && same < key_size && data[same] != '/')))
{
--keep;
}
chain.resize(keep);
last_key = data;
last_key_size = key_size;
}
// walk the tokens that are not walked yet
while (chain.back().end < prefix_size && chain.back().end < key_size)
{
const std::size_t start = chain.back().end + 1;
const void* const slash = std::memchr(data + start, '/', key_size - start);
const std::size_t end = slash == nullptr ? key_size : static_cast<std::size_t>(static_cast<const char*>(slash) - data);
step next;
next.node = child(chain.back().node, data + start, end - start);
next.end = end;
chain.push_back(next);
}
if (chain.back().end == prefix_size)
{
return chain.back().node;
}
// an earlier call walked further
const auto it = std::lower_bound(chain.begin(), chain.end(), prefix_size, [](const step & s, std::size_t size)
{
return s.end < size;
});
return it->node;
}
std::size_t slot_of(std::size_t parent, std::size_t hash) const noexcept
{
std::uint64_t x = static_cast<std::uint64_t>(hash) ^ (static_cast<std::uint64_t>(parent) * 0x9E3779B97F4A7C15ULL);
x ^= x >> 32u;
x *= 0xD6E8FEB86659FD93ULL;
x ^= x >> 32u;
return static_cast<std::size_t>(x) & (table.size() - 1);
}
/// @brief the node of the prefix of the node @a parent followed by the token, which is added if it is not there
std::size_t child(std::size_t parent, const char* token, std::size_t token_size)
{
const std::size_t hash = hash_bytes(token, token_size, seed);
std::size_t i = slot_of(parent, hash);
while (table[i].index != 0)
{
const node& n = nodes[table[i].index];
if (n.parent == parent && n.hash == hash && n.token_size == token_size && std::memcmp(n.token, token, token_size) == 0)
{
return table[i].index;
}
i = (i + 1) & (table.size() - 1);
}
node created;
created.parent = parent;
created.hash = hash;
created.token = token;
created.token_size = token_size;
nodes.push_back(created);
table[i].index = nodes.size() - 1;
// keep the load factor below 1/2
if (nodes.size() * 2 > table.size())
{
std::vector<bucket> old(table.size() * 2);
old.swap(table);
for (std::size_t k = 1; k < nodes.size(); ++k)
{
std::size_t j = slot_of(nodes[k].parent, nodes[k].hash);
while (table[j].index != 0)
{
j = (j + 1) & (table.size() - 1);
}
table[j].index = k;
}
}
return nodes.size() - 1;
}
std::uint64_t seed = 0;
/// @brief the nodes; the first is the root, the empty prefix
std::vector<node> nodes{};
/// @brief the nodes (not the root) in a hash table
std::vector<bucket> table{};
/// @brief the root, and the node after each token of the key that was walked last
std::vector<step> chain{};
const char* last_key = nullptr;
std::size_t last_key_size = 0;
bool any_marked = false;
};
/*!
@brief the pointer prefixes of a flattened object that denote arrays
@ref unflatten creates arrays exactly below the prefixes that have a
reference token 0 among their children, and objects everywhere else.
Deciding this up front keeps the result independent of the order in which
the flattened object is iterated, which is unspecified for some object
types.
A prefix is identified by the text of the key it is the beginning of:
escaping is unambiguous, so equal prefixes of valid keys are equal texts.
The set refers to the keys, which must outlive it. Short prefixes are
kept in a hash table and compared as texts, which costs at most the length
of the bound. Longer prefixes are kept in a @ref prefix_tree, because
hashing and comparing every one of the prefixes of a key with many tokens
from its beginning would take quadratic time.
The hash function depends on a seed that is derived from the address of a
local variable, which address space layout randomization changes between
processes, so keys cannot be crafted to collide in advance; the result
does not depend on the seed. Where the addresses are not randomized, or
when the seed is known, keys can collide: k short prefixes then cost
k^2 / 2 comparisons of fewer than 64 bytes each. The long prefixes are not
hashed as a whole.
*/
class array_prefix_set
{
public:
/// @brief choose the seed of the hash function, before anything is added: it is derived from the address that is passed
void reseed(const void* address) noexcept
{
// NOLINTNEXTLINE(cppcoreguidelines-pro-type-reinterpret-cast): the address is only a source of entropy
std::uint64_t x = static_cast<std::uint64_t>(reinterpret_cast<std::uintptr_t>(address)) ^ 0x9E3779B97F4A7C15ULL;
x *= 0xBF58476D1CE4E5B9ULL;
seed = x ^ (x >> 31u);
tree.reseed(seed);
}
/// @brief add the first @a prefix_size bytes of the key of @a key_size bytes at @a data
void insert(const char* data, std::size_t key_size, std::size_t prefix_size)
{
if (prefix_size >= short_limit)
{
tree.mark(data, key_size, prefix_size);
return;
}
// flattened objects are usually sorted, so most keys that have a
// 0 yield the prefix of the previous one
if (last_data != nullptr && last_size == prefix_size && std::memcmp(last_data, data, prefix_size) == 0)
{
return;
}
last_data = data;
last_size = prefix_size;
const std::size_t hash = prefix_tree::hash_bytes(data, prefix_size, seed);
if (table.empty())
{
table.resize(16);
}
const std::size_t mask = table.size() - 1;
std::size_t i = hash & mask;
while (table[i].data != nullptr)
{
if (table[i].hash == hash && table[i].size == prefix_size && std::memcmp(table[i].data, data, prefix_size) == 0)
{
return;
}
i = (i + 1) & mask;
}
table[i] = slot(hash, data, prefix_size);
// keep the load factor below 1/2
if (++used * 2 > table.size())
{
grow();
}
}
bool empty() const noexcept
{
return used == 0 && tree.empty();
}
/// @brief whether the first @a prefix_size bytes of the key of @a key_size bytes at @a data were added
bool contains(const char* data, std::size_t key_size, std::size_t prefix_size)
{
if (prefix_size >= short_limit)
{
return tree.is_marked(data, key_size, prefix_size);
}
if (used == 0)
{
return false;
}
const std::size_t hash = prefix_tree::hash_bytes(data, prefix_size, seed);
const std::size_t mask = table.size() - 1;
for (std::size_t i = hash & mask; table[i].data != nullptr; i = (i + 1) & mask)
{
if (table[i].hash == hash && table[i].size == prefix_size && std::memcmp(table[i].data, data, prefix_size) == 0)
{
return true;
}
}
return false;
}
private:
/// @brief prefixes of fewer bytes are kept in the hash table
static constexpr std::size_t short_limit = 64;
struct slot
{
slot() noexcept = default;
slot(std::size_t h, const char* d, std::size_t s) noexcept
: hash(h), data(d), size(s)
{}
std::size_t hash = 0;
const char* data = nullptr; // nullptr: the slot is empty
std::size_t size = 0;
};
void grow()
{
std::vector<slot> old(table.size() * 2);
old.swap(table);
const std::size_t mask = table.size() - 1;
for (const auto& s : old)
{
if (s.data != nullptr)
{
std::size_t i = s.hash & mask;
while (table[i].data != nullptr)
{
i = (i + 1) & mask;
}
table[i] = s;
}
}
}
std::uint64_t seed = 0;
prefix_tree tree{};
std::vector<slot> table{};
std::size_t used = 0;
const char* last_data = nullptr;
std::size_t last_size = 0;
};
/*!
@brief check that @a key is a valid JSON pointer, and add the prefixes of
it that denote arrays to @a arrays
@@ -1052,7 +1267,7 @@ class json_pointer
// a token 0 makes the pointer before it an array
if (i - begin == 1 && data[begin] == '0')
{
arrays.insert(data, begin - 1);
arrays.insert(data, size, begin - 1);
}
begin = i + 1;
}
@@ -2062,6 +2277,7 @@ class json_pointer
// token 0 among their children; the values below them are arrays,
// all others are objects (see array_prefix_set)
array_prefix_set arrays;
arrays.reseed(&arrays);
const string_t* previous_key = nullptr;
for (const auto& element : *value.m_data.m_value.object)
{
@@ -2153,7 +2369,7 @@ class json_pointer
case detail::value_t::null:
{
// a first child 0 makes the value an array without a look-up
if ((token_size == 1 && data[pos + 1] == '0') || arrays.contains(data, pos))
if ((token_size == 1 && data[pos + 1] == '0') || arrays.contains(data, key.size(), pos))
{
// some reference token below this position is 0, so the
// value is an array
-6
View File
@@ -224,12 +224,6 @@
#define JSON_NO_THREAD_LOCAL 1
#endif
// JSON_RECURSION_DEPTH_LIMIT (the number of nesting levels the walks over a
// value recurse into before they continue with an explicit stack; 128 by
// default, 0 to 254 allowed) is defined and checked where it is used, in
// detail/recursion_depth_limit.hpp. Like JSON_NO_THREAD_LOCAL it is neither part
// of the ABI tag nor undefined at the end of json.hpp.
// allow disabling exceptions
#if (defined(__cpp_exceptions) || defined(__EXCEPTIONS) || defined(_CPPUNWIND)) && !defined(JSON_NOEXCEPTION)
#define JSON_THROW(exception) throw exception
@@ -9,22 +9,9 @@
#pragma once
#include <cstddef> // size_t
#include <type_traits> // is_integral, is_same
#include <nlohmann/detail/abi_macros.hpp>
// JSON_RECURSION_DEPTH_LIMIT: the number of nesting levels the operations that
// walk a value recurse into on the call stack before they continue with an
// explicit stack (see detail::recursion_depth_limit() below).
//
// May be defined by the user before the first include of the library; it has
// to be the same in every translation unit of a program (it is not part of the
// ABI tag, see https://json.nlohmann.me/api/macros/json_recursion_depth_limit/).
// Allowed: an integer constant expression from 0 to 254. The default is 128.
#ifndef JSON_RECURSION_DEPTH_LIMIT
#define JSON_RECURSION_DEPTH_LIMIT 128
#endif
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
@@ -33,59 +20,21 @@ namespace detail
@brief the number of nesting levels an operation recurses into
Operations that walk a value (copying, comparing, serializing, hashing, merging,
diffing, converting, destroying, and writing the binary formats CBOR,
MessagePack, UBJSON, BJData, and BON8) recurse once per nesting level, which is
fastest, but a value nested deeply enough would exhaust the call stack. So they
recurse only this many levels deep and finish whatever lies below with an
explicit stack that lives on the heap. All of them share this limit and none has
a constant of its own. It is the value of the macro JSON_RECURSION_DEPTH_LIMIT,
128 by default. (Parsing, the binary readers, flatten(), unflatten(), and
to_bson() always use an explicit stack and do not read it, and the lockstep
comparison of json_view documents has a fixed bound of its own, 48 levels.)
...) recurse once
per nesting level, which is fastest, but a value nested deeply enough would
exhaust the call stack. So they recurse only this many levels deep and finish
whatever lies below with an explicit stack. All of them share this limit.
Most of the operations pass the depth down as an argument. The copy constructor
and the comparison operators cannot, as their signatures are fixed, so they
count it in basic_json::nesting_depth() instead, a byte per thread. Only the
levels that are descended into are counted, so the count never exceeds the limit.
The limit must fit in the byte with some headroom, so that the count cannot wrap
around: it must be below 255, and 254 is the largest value allowed. (More levels
would hardly be useful: 254 levels need up to 576 KiB of stack in an unoptimized
build already, twice the stack of the default.)
Most of them pass the depth down as an argument. The copy constructor and the
comparison operators cannot, as their signatures are fixed, so they count it
in basic_json::nesting_depth() instead, a byte per thread; the limit must
therefore stay below 255.
The smallest value is 0: every operation then uses its explicit stack from the
first level on and no operation recurses over the nesting of a value at all.
That is slower, but needs a constant amount of call stack. It works because
every use of the limit compares a depth with it (`depth >= limit`, or `<` for
the opposite), which holds for 0 as for any other value; nothing computes
`limit - 1` or `limit + 1`, so nothing wraps around. Each operation's explicit
stack handles any nesting level, including the first, so none of them relies on
having recursed first.
The call stack that an operation needs is roughly a few KiB plus a cost per level
times the limit. At the default of 128 levels, an unoptimized build needs up to
about 290 KiB (`dump()` with Clang at -O0: about 2.3 KiB per level) and an
optimized build up to about 48 KiB; with 254 levels it is up to 576 KiB and
112 KiB, and with 0 it is 16 KiB or less. Lower the limit for threads with small
stacks (RTOS tasks, fibers, musl threads), raise it to trade stack for speed on
deeply nested values. All of this, with the measurements, is in the
documentation of the macro.
@sa https://json.nlohmann.me/api/macros/json_recursion_depth_limit/
@sa https://github.com/nlohmann/json/issues/5387
*/
constexpr std::size_t recursion_depth_limit() noexcept
{
// an integer: not a floating-point number, whose range check would pass for 1.5 or 254.0
// (bool is an integer type, too, but 1 and 0 spelled true and false are no way to say a limit)
static_assert(std::is_integral<decltype(JSON_RECURSION_DEPTH_LIMIT)>::value
&& !std::is_same<decltype(JSON_RECURSION_DEPTH_LIMIT), bool>::value,
"JSON_RECURSION_DEPTH_LIMIT must be an integer constant");
static_assert(JSON_RECURSION_DEPTH_LIMIT >= 0 && JSON_RECURSION_DEPTH_LIMIT <= 254,
"JSON_RECURSION_DEPTH_LIMIT must be between 0 and 254: the nesting depth is counted in a byte per thread "
"and may exceed the limit by one (see https://json.nlohmann.me/api/macros/json_recursion_depth_limit/)");
// (a value out of range has been reported by now; the remainder keeps it
// below 255 for any value, so that basic_json does not report it again)
return static_cast<std::size_t>(JSON_RECURSION_DEPTH_LIMIT) % 255;
return 128;
}
} // namespace detail
+7 -32
View File
@@ -1256,12 +1256,8 @@ public:
}
#ifndef JSON_NO_THREAD_LOCAL
// nesting_depth() is a byte that never exceeds the limit (JSON_RECURSION_DEPTH_LIMIT),
// which is at most 254: the largest value stays one below the byte's maximum, so
// that the count cannot wrap around to 0 and let a descent go on without a bound.
// (recursion_depth_limit() checks the macro itself, with a message that
// names it; this is the check that cannot be edited away unnoticed.)
static_assert(detail::recursion_depth_limit() < 255, "the nesting depth count must fit in a byte: JSON_RECURSION_DEPTH_LIMIT must not exceed 254");
// nesting_depth() is a byte and may exceed the limit by one level
static_assert(detail::recursion_depth_limit() < 255, "the nesting depth count must fit in a byte");
/*!
@brief how many levels the operation going on in this thread has descended into
@@ -1272,15 +1268,8 @@ public:
user code nests them anyway, sharing the count only ends a descent sooner
than it had to, which costs a little speed and is never wrong.
A byte is enough: only the levels that are descended into are counted
(@ref nesting_depth_guard), so the count never exceeds the limit, and the
limit is at most 254.
The count is per thread and not per value or per library configuration: all
translation units of a program share it, even if they were compiled with
different values of JSON_RECURSION_DEPTH_LIMIT (which is not supported, see
the documentation of the macro); each of them then compares the shared count
with its own limit.
A byte is enough: the count never exceeds the limit by more than the single
level that notices the limit has been reached.
*/
static std::uint8_t& nesting_depth() noexcept
{
@@ -1338,25 +1327,14 @@ public:
#endif
{
#ifndef JSON_NO_THREAD_LOCAL
// Only a level that is entered is counted, so that the count never
// exceeds the limit. A level that is not entered is finished without
// descending, and whatever runs while it is (a copy made by a custom
// base class, say) sees the same count as the level itself did and
// is not entered either.
if (m_okay)
{
++nesting_depth();
}
++nesting_depth();
#endif
}
~nesting_depth_guard()
{
#ifndef JSON_NO_THREAD_LOCAL
if (m_okay)
{
--nesting_depth();
}
--nesting_depth();
#endif
}
@@ -5268,10 +5246,7 @@ public:
*/
void update_members(const iterator& first, const iterator& last, const bool merge_objects, const std::size_t depth)
{
// without merge_objects the loop below never descends, so there is no
// call stack to save (and the explicit stack always merges); the
// distinction matters for a limit of 0, which reaches this at depth 0
if (JSON_HEDLEY_UNLIKELY(merge_objects && depth >= detail::recursion_depth_limit()))
if (JSON_HEDLEY_UNLIKELY(depth >= detail::recursion_depth_limit()))
{
update_members_iteratively(first, last);
return;
+367 -233
View File
@@ -2597,12 +2597,6 @@ JSON_HEDLEY_DIAGNOSTIC_POP
#define JSON_NO_THREAD_LOCAL 1
#endif
// JSON_RECURSION_DEPTH_LIMIT (the number of nesting levels the walks over a
// value recurse into before they continue with an explicit stack; 128 by
// default, 0 to 254 allowed) is defined and checked where it is used, in
// detail/recursion_depth_limit.hpp. Like JSON_NO_THREAD_LOCAL it is neither part
// of the ABI tag nor undefined at the end of json.hpp.
// allow disabling exceptions
#if (defined(__cpp_exceptions) || defined(__EXCEPTIONS) || defined(_CPPUNWIND)) && !defined(JSON_NOEXCEPTION)
#define JSON_THROW(exception) throw exception
@@ -8606,23 +8600,10 @@ NLOHMANN_JSON_NAMESPACE_END
#include <cstddef> // size_t
#include <type_traits> // is_integral, is_same
// #include <nlohmann/detail/abi_macros.hpp>
// JSON_RECURSION_DEPTH_LIMIT: the number of nesting levels the operations that
// walk a value recurse into on the call stack before they continue with an
// explicit stack (see detail::recursion_depth_limit() below).
//
// May be defined by the user before the first include of the library; it has
// to be the same in every translation unit of a program (it is not part of the
// ABI tag, see https://json.nlohmann.me/api/macros/json_recursion_depth_limit/).
// Allowed: an integer constant expression from 0 to 254. The default is 128.
#ifndef JSON_RECURSION_DEPTH_LIMIT
#define JSON_RECURSION_DEPTH_LIMIT 128
#endif
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
@@ -8631,59 +8612,21 @@ namespace detail
@brief the number of nesting levels an operation recurses into
Operations that walk a value (copying, comparing, serializing, hashing, merging,
diffing, converting, destroying, and writing the binary formats CBOR,
MessagePack, UBJSON, BJData, and BON8) recurse once per nesting level, which is
fastest, but a value nested deeply enough would exhaust the call stack. So they
recurse only this many levels deep and finish whatever lies below with an
explicit stack that lives on the heap. All of them share this limit and none has
a constant of its own. It is the value of the macro JSON_RECURSION_DEPTH_LIMIT,
128 by default. (Parsing, the binary readers, flatten(), unflatten(), and
to_bson() always use an explicit stack and do not read it, and the lockstep
comparison of json_view documents has a fixed bound of its own, 48 levels.)
...) recurse once
per nesting level, which is fastest, but a value nested deeply enough would
exhaust the call stack. So they recurse only this many levels deep and finish
whatever lies below with an explicit stack. All of them share this limit.
Most of the operations pass the depth down as an argument. The copy constructor
and the comparison operators cannot, as their signatures are fixed, so they
count it in basic_json::nesting_depth() instead, a byte per thread. Only the
levels that are descended into are counted, so the count never exceeds the limit.
The limit must fit in the byte with some headroom, so that the count cannot wrap
around: it must be below 255, and 254 is the largest value allowed. (More levels
would hardly be useful: 254 levels need up to 576 KiB of stack in an unoptimized
build already, twice the stack of the default.)
Most of them pass the depth down as an argument. The copy constructor and the
comparison operators cannot, as their signatures are fixed, so they count it
in basic_json::nesting_depth() instead, a byte per thread; the limit must
therefore stay below 255.
The smallest value is 0: every operation then uses its explicit stack from the
first level on and no operation recurses over the nesting of a value at all.
That is slower, but needs a constant amount of call stack. It works because
every use of the limit compares a depth with it (`depth >= limit`, or `<` for
the opposite), which holds for 0 as for any other value; nothing computes
`limit - 1` or `limit + 1`, so nothing wraps around. Each operation's explicit
stack handles any nesting level, including the first, so none of them relies on
having recursed first.
The call stack that an operation needs is roughly a few KiB plus a cost per level
times the limit. At the default of 128 levels, an unoptimized build needs up to
about 290 KiB (`dump()` with Clang at -O0: about 2.3 KiB per level) and an
optimized build up to about 48 KiB; with 254 levels it is up to 576 KiB and
112 KiB, and with 0 it is 16 KiB or less. Lower the limit for threads with small
stacks (RTOS tasks, fibers, musl threads), raise it to trade stack for speed on
deeply nested values. All of this, with the measurements, is in the
documentation of the macro.
@sa https://json.nlohmann.me/api/macros/json_recursion_depth_limit/
@sa https://github.com/nlohmann/json/issues/5387
*/
constexpr std::size_t recursion_depth_limit() noexcept
{
// an integer: not a floating-point number, whose range check would pass for 1.5 or 254.0
// (bool is an integer type, too, but 1 and 0 spelled true and false are no way to say a limit)
static_assert(std::is_integral<decltype(JSON_RECURSION_DEPTH_LIMIT)>::value
&& !std::is_same<decltype(JSON_RECURSION_DEPTH_LIMIT), bool>::value,
"JSON_RECURSION_DEPTH_LIMIT must be an integer constant");
static_assert(JSON_RECURSION_DEPTH_LIMIT >= 0 && JSON_RECURSION_DEPTH_LIMIT <= 254,
"JSON_RECURSION_DEPTH_LIMIT must be between 0 and 254: the nesting depth is counted in a byte per thread "
"and may exceed the limit by one (see https://json.nlohmann.me/api/macros/json_recursion_depth_limit/)");
// (a value out of range has been reported by now; the remainder keeps it
// below 255 for any value, so that basic_json does not report it again)
return static_cast<std::size_t>(JSON_RECURSION_DEPTH_LIMIT) % 255;
return 128;
}
} // namespace detail
@@ -27084,139 +27027,6 @@ class json_pointer
}
private:
/*!
@brief the pointer prefixes of a flattened object that denote arrays
@ref unflatten creates arrays exactly below the prefixes that have a
reference token 0 among their children, and objects everywhere else.
Deciding this up front keeps the result independent of the order in which
the flattened object is iterated, which is unspecified for some object
types.
A prefix is identified by the text of the key it is the beginning of:
escaping is unambiguous, so equal prefixes of valid keys are equal texts.
The set refers to the keys, which must outlive it.
*/
class array_prefix_set
{
public:
void insert(const char* data, std::size_t size)
{
// flattened objects are usually sorted, so most keys that have a
// 0 yield the prefix of the previous one
if (last_data != nullptr && last_size == size && std::memcmp(last_data, data, size) == 0)
{
return;
}
last_data = data;
last_size = size;
if (table.empty())
{
table.resize(16);
}
const std::size_t hash = hash_of(data, size);
const std::size_t mask = table.size() - 1;
std::size_t i = hash & mask;
while (table[i].data != nullptr)
{
if (table[i].hash == hash && table[i].size == size && std::memcmp(table[i].data, data, size) == 0)
{
return;
}
i = (i + 1) & mask;
}
table[i] = slot(hash, data, size);
// keep the load factor below 1/2
if (++used * 2 > table.size())
{
grow();
}
}
bool empty() const noexcept
{
return used == 0;
}
bool contains(const char* data, std::size_t size) const noexcept
{
if (used == 0)
{
return false;
}
const std::size_t hash = hash_of(data, size);
const std::size_t mask = table.size() - 1;
for (std::size_t i = hash & mask; table[i].data != nullptr; i = (i + 1) & mask)
{
if (table[i].hash == hash && table[i].size == size && std::memcmp(table[i].data, data, size) == 0)
{
return true;
}
}
return false;
}
private:
struct slot
{
slot() noexcept = default;
slot(std::size_t h, const char* d, std::size_t s) noexcept
: hash(h), data(d), size(s)
{}
std::size_t hash = 0;
const char* data = nullptr; // nullptr: the slot is empty
std::size_t size = 0;
};
static std::size_t hash_of(const char* data, std::size_t size) noexcept
{
std::uint64_t h = 0x9E3779B97F4A7C15ULL ^ size;
std::size_t i = 0;
for (; i + 8 <= size; i += 8)
{
std::uint64_t w = 0;
std::memcpy(&w, data + i, 8);
h = (h ^ w) * 0x100000001B3ULL;
h ^= h >> 32u;
}
std::uint64_t w = 0;
if (i < size)
{
std::memcpy(&w, data + i, size - i);
}
h = (h ^ w) * 0x9E3779B97F4A7C15ULL;
return detail::conditional_static_cast<std::size_t>(h ^ (h >> 29u));
}
void grow()
{
std::vector<slot> old(table.size() * 2);
old.swap(table);
const std::size_t mask = table.size() - 1;
for (const auto& s : old)
{
if (s.data != nullptr)
{
std::size_t i = s.hash & mask;
while (table[i].data != nullptr)
{
i = (i + 1) & mask;
}
table[i] = s;
}
}
}
std::vector<slot> table{};
std::size_t used = 0;
const char* last_data = nullptr;
std::size_t last_size = 0;
};
/// @brief the length of the longest common prefix of the first @a limit bytes of @a a and @a b
static std::size_t common_prefix_length(const char* a, const char* b, const std::size_t limit) noexcept
{
@@ -27239,6 +27049,354 @@ class json_pointer
return i;
}
/*!
@brief the prefixes of a flattened object that are marked as arrays, for
prefixes that are not short
A prefix is identified by the tokens it consists of: a node of a tree
stands for the prefix of its parent followed by a token, and is looked up
by its parent and the text of the token. The node of a prefix is found by
walking the tokens of the key from the root, so identifying all prefixes
of a key together costs one pass over the key, however long the prefixes
are, and no prefix is ever compared byte by byte as a whole. The nodes
along the key that was walked last are kept, and the walk of the next key
starts below the beginning it has in common with that key. The tree refers
to the keys, which must outlive it.
*/
class prefix_tree
{
public:
/// @brief set the seed of the hash function, before anything is added
void reseed(const std::uint64_t seed_value) noexcept
{
seed = seed_value;
}
/// @brief mark the first @a prefix_size bytes of the key of @a key_size bytes at @a data
void mark(const char* data, std::size_t key_size, std::size_t prefix_size)
{
nodes[walk(data, key_size, prefix_size)].marked = true;
any_marked = true;
}
bool is_marked(const char* data, std::size_t key_size, std::size_t prefix_size)
{
return any_marked && nodes[walk(data, key_size, prefix_size)].marked;
}
bool empty() const noexcept
{
return !any_marked;
}
/// @brief the hash of the first @a size bytes at @a data, for the seed @a seed_value
static std::size_t hash_bytes(const char* data, std::size_t size, const std::uint64_t seed_value) noexcept
{
std::uint64_t h = seed_value ^ size;
std::size_t i = 0;
for (; i + 8 <= size; i += 8)
{
std::uint64_t w = 0;
std::memcpy(&w, data + i, 8);
h = (h ^ w) * 0x100000001B3ULL;
h ^= h >> 32u;
}
std::uint64_t w = 0;
if (i < size)
{
std::memcpy(&w, data + i, size - i);
}
h = (h ^ w) * 0x9E3779B97F4A7C15ULL;
return detail::conditional_static_cast<std::size_t>(h ^ (h >> 29u));
}
private:
struct node
{
std::size_t parent = 0;
std::size_t hash = 0;
const char* token = nullptr;
std::size_t token_size = 0;
bool marked = false;
};
/// @brief a slot of the hash table; the index of a node, 0 if the slot is empty
struct bucket
{
std::size_t index = 0;
};
/// @brief the node after a token of the key that was walked last, and where the token ends
struct step
{
std::size_t node = 0;
std::size_t end = 0;
};
/// @brief the index of the node of the prefix that ends at the slash (or the end) at @a prefix_size
std::size_t walk(const char* data, std::size_t key_size, std::size_t prefix_size)
{
if (nodes.empty())
{
nodes.emplace_back();
table.resize(16);
chain.emplace_back();
}
if (data != last_key)
{
// keep the nodes within the beginning that this key has in common
// with the key they were found for; a prefix that ends inside a
// token of this key is not one of its prefixes
const std::size_t same = common_prefix_length(data, last_key, (std::min)(key_size, last_key_size));
std::size_t keep = chain.size();
while (keep > 1 && (chain[keep - 1].end > same || (chain[keep - 1].end == same && same < key_size && data[same] != '/')))
{
--keep;
}
chain.resize(keep);
last_key = data;
last_key_size = key_size;
}
// walk the tokens that are not walked yet
while (chain.back().end < prefix_size && chain.back().end < key_size)
{
const std::size_t start = chain.back().end + 1;
const void* const slash = std::memchr(data + start, '/', key_size - start);
const std::size_t end = slash == nullptr ? key_size : static_cast<std::size_t>(static_cast<const char*>(slash) - data);
step next;
next.node = child(chain.back().node, data + start, end - start);
next.end = end;
chain.push_back(next);
}
if (chain.back().end == prefix_size)
{
return chain.back().node;
}
// an earlier call walked further
const auto it = std::lower_bound(chain.begin(), chain.end(), prefix_size, [](const step & s, std::size_t size)
{
return s.end < size;
});
return it->node;
}
std::size_t slot_of(std::size_t parent, std::size_t hash) const noexcept
{
std::uint64_t x = static_cast<std::uint64_t>(hash) ^ (static_cast<std::uint64_t>(parent) * 0x9E3779B97F4A7C15ULL);
x ^= x >> 32u;
x *= 0xD6E8FEB86659FD93ULL;
x ^= x >> 32u;
return static_cast<std::size_t>(x) & (table.size() - 1);
}
/// @brief the node of the prefix of the node @a parent followed by the token, which is added if it is not there
std::size_t child(std::size_t parent, const char* token, std::size_t token_size)
{
const std::size_t hash = hash_bytes(token, token_size, seed);
std::size_t i = slot_of(parent, hash);
while (table[i].index != 0)
{
const node& n = nodes[table[i].index];
if (n.parent == parent && n.hash == hash && n.token_size == token_size && std::memcmp(n.token, token, token_size) == 0)
{
return table[i].index;
}
i = (i + 1) & (table.size() - 1);
}
node created;
created.parent = parent;
created.hash = hash;
created.token = token;
created.token_size = token_size;
nodes.push_back(created);
table[i].index = nodes.size() - 1;
// keep the load factor below 1/2
if (nodes.size() * 2 > table.size())
{
std::vector<bucket> old(table.size() * 2);
old.swap(table);
for (std::size_t k = 1; k < nodes.size(); ++k)
{
std::size_t j = slot_of(nodes[k].parent, nodes[k].hash);
while (table[j].index != 0)
{
j = (j + 1) & (table.size() - 1);
}
table[j].index = k;
}
}
return nodes.size() - 1;
}
std::uint64_t seed = 0;
/// @brief the nodes; the first is the root, the empty prefix
std::vector<node> nodes{};
/// @brief the nodes (not the root) in a hash table
std::vector<bucket> table{};
/// @brief the root, and the node after each token of the key that was walked last
std::vector<step> chain{};
const char* last_key = nullptr;
std::size_t last_key_size = 0;
bool any_marked = false;
};
/*!
@brief the pointer prefixes of a flattened object that denote arrays
@ref unflatten creates arrays exactly below the prefixes that have a
reference token 0 among their children, and objects everywhere else.
Deciding this up front keeps the result independent of the order in which
the flattened object is iterated, which is unspecified for some object
types.
A prefix is identified by the text of the key it is the beginning of:
escaping is unambiguous, so equal prefixes of valid keys are equal texts.
The set refers to the keys, which must outlive it. Short prefixes are
kept in a hash table and compared as texts, which costs at most the length
of the bound. Longer prefixes are kept in a @ref prefix_tree, because
hashing and comparing every one of the prefixes of a key with many tokens
from its beginning would take quadratic time.
The hash function depends on a seed that is derived from the address of a
local variable, which address space layout randomization changes between
processes, so keys cannot be crafted to collide in advance; the result
does not depend on the seed. Where the addresses are not randomized, or
when the seed is known, keys can collide: k short prefixes then cost
k^2 / 2 comparisons of fewer than 64 bytes each. The long prefixes are not
hashed as a whole.
*/
class array_prefix_set
{
public:
/// @brief choose the seed of the hash function, before anything is added: it is derived from the address that is passed
void reseed(const void* address) noexcept
{
// NOLINTNEXTLINE(cppcoreguidelines-pro-type-reinterpret-cast): the address is only a source of entropy
std::uint64_t x = static_cast<std::uint64_t>(reinterpret_cast<std::uintptr_t>(address)) ^ 0x9E3779B97F4A7C15ULL;
x *= 0xBF58476D1CE4E5B9ULL;
seed = x ^ (x >> 31u);
tree.reseed(seed);
}
/// @brief add the first @a prefix_size bytes of the key of @a key_size bytes at @a data
void insert(const char* data, std::size_t key_size, std::size_t prefix_size)
{
if (prefix_size >= short_limit)
{
tree.mark(data, key_size, prefix_size);
return;
}
// flattened objects are usually sorted, so most keys that have a
// 0 yield the prefix of the previous one
if (last_data != nullptr && last_size == prefix_size && std::memcmp(last_data, data, prefix_size) == 0)
{
return;
}
last_data = data;
last_size = prefix_size;
const std::size_t hash = prefix_tree::hash_bytes(data, prefix_size, seed);
if (table.empty())
{
table.resize(16);
}
const std::size_t mask = table.size() - 1;
std::size_t i = hash & mask;
while (table[i].data != nullptr)
{
if (table[i].hash == hash && table[i].size == prefix_size && std::memcmp(table[i].data, data, prefix_size) == 0)
{
return;
}
i = (i + 1) & mask;
}
table[i] = slot(hash, data, prefix_size);
// keep the load factor below 1/2
if (++used * 2 > table.size())
{
grow();
}
}
bool empty() const noexcept
{
return used == 0 && tree.empty();
}
/// @brief whether the first @a prefix_size bytes of the key of @a key_size bytes at @a data were added
bool contains(const char* data, std::size_t key_size, std::size_t prefix_size)
{
if (prefix_size >= short_limit)
{
return tree.is_marked(data, key_size, prefix_size);
}
if (used == 0)
{
return false;
}
const std::size_t hash = prefix_tree::hash_bytes(data, prefix_size, seed);
const std::size_t mask = table.size() - 1;
for (std::size_t i = hash & mask; table[i].data != nullptr; i = (i + 1) & mask)
{
if (table[i].hash == hash && table[i].size == prefix_size && std::memcmp(table[i].data, data, prefix_size) == 0)
{
return true;
}
}
return false;
}
private:
/// @brief prefixes of fewer bytes are kept in the hash table
static constexpr std::size_t short_limit = 64;
struct slot
{
slot() noexcept = default;
slot(std::size_t h, const char* d, std::size_t s) noexcept
: hash(h), data(d), size(s)
{}
std::size_t hash = 0;
const char* data = nullptr; // nullptr: the slot is empty
std::size_t size = 0;
};
void grow()
{
std::vector<slot> old(table.size() * 2);
old.swap(table);
const std::size_t mask = table.size() - 1;
for (const auto& s : old)
{
if (s.data != nullptr)
{
std::size_t i = s.hash & mask;
while (table[i].data != nullptr)
{
i = (i + 1) & mask;
}
table[i] = s;
}
}
}
std::uint64_t seed = 0;
prefix_tree tree{};
std::vector<slot> table{};
std::size_t used = 0;
const char* last_data = nullptr;
std::size_t last_size = 0;
};
/*!
@brief check that @a key is a valid JSON pointer, and add the prefixes of
it that denote arrays to @a arrays
@@ -27284,7 +27442,7 @@ class json_pointer
// a token 0 makes the pointer before it an array
if (i - begin == 1 && data[begin] == '0')
{
arrays.insert(data, begin - 1);
arrays.insert(data, size, begin - 1);
}
begin = i + 1;
}
@@ -28294,6 +28452,7 @@ class json_pointer
// token 0 among their children; the values below them are arrays,
// all others are objects (see array_prefix_set)
array_prefix_set arrays;
arrays.reseed(&arrays);
const string_t* previous_key = nullptr;
for (const auto& element : *value.m_data.m_value.object)
{
@@ -28385,7 +28544,7 @@ class json_pointer
case detail::value_t::null:
{
// a first child 0 makes the value an array without a look-up
if ((token_size == 1 && data[pos + 1] == '0') || arrays.contains(data, pos))
if ((token_size == 1 && data[pos + 1] == '0') || arrays.contains(data, key.size(), pos))
{
// some reference token below this position is 0, so the
// value is an array
@@ -40463,12 +40622,8 @@ public:
}
#ifndef JSON_NO_THREAD_LOCAL
// nesting_depth() is a byte that never exceeds the limit (JSON_RECURSION_DEPTH_LIMIT),
// which is at most 254: the largest value stays one below the byte's maximum, so
// that the count cannot wrap around to 0 and let a descent go on without a bound.
// (recursion_depth_limit() checks the macro itself, with a message that
// names it; this is the check that cannot be edited away unnoticed.)
static_assert(detail::recursion_depth_limit() < 255, "the nesting depth count must fit in a byte: JSON_RECURSION_DEPTH_LIMIT must not exceed 254");
// nesting_depth() is a byte and may exceed the limit by one level
static_assert(detail::recursion_depth_limit() < 255, "the nesting depth count must fit in a byte");
/*!
@brief how many levels the operation going on in this thread has descended into
@@ -40479,15 +40634,8 @@ public:
user code nests them anyway, sharing the count only ends a descent sooner
than it had to, which costs a little speed and is never wrong.
A byte is enough: only the levels that are descended into are counted
(@ref nesting_depth_guard), so the count never exceeds the limit, and the
limit is at most 254.
The count is per thread and not per value or per library configuration: all
translation units of a program share it, even if they were compiled with
different values of JSON_RECURSION_DEPTH_LIMIT (which is not supported, see
the documentation of the macro); each of them then compares the shared count
with its own limit.
A byte is enough: the count never exceeds the limit by more than the single
level that notices the limit has been reached.
*/
static std::uint8_t& nesting_depth() noexcept
{
@@ -40545,25 +40693,14 @@ public:
#endif
{
#ifndef JSON_NO_THREAD_LOCAL
// Only a level that is entered is counted, so that the count never
// exceeds the limit. A level that is not entered is finished without
// descending, and whatever runs while it is (a copy made by a custom
// base class, say) sees the same count as the level itself did and
// is not entered either.
if (m_okay)
{
++nesting_depth();
}
++nesting_depth();
#endif
}
~nesting_depth_guard()
{
#ifndef JSON_NO_THREAD_LOCAL
if (m_okay)
{
--nesting_depth();
}
--nesting_depth();
#endif
}
@@ -44475,10 +44612,7 @@ public:
*/
void update_members(const iterator& first, const iterator& last, const bool merge_objects, const std::size_t depth)
{
// without merge_objects the loop below never descends, so there is no
// call stack to save (and the explicit stack always merges); the
// distinction matters for a limit of 0, which reaches this at depth 0
if (JSON_HEDLEY_UNLIKELY(merge_objects && depth >= detail::recursion_depth_limit()))
if (JSON_HEDLEY_UNLIKELY(depth >= detail::recursion_depth_limit()))
{
update_members_iteratively(first, last);
return;
-14
View File
@@ -355,20 +355,6 @@ json_test_add_test_for(src/unit-comparison.cpp
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
# test the walks over deeply nested values with the extreme settings of
# JSON_RECURSION_DEPTH_LIMIT: 0 (every walk uses its explicit stack from the
# first level on), 1, and 254 (the largest limit that fits the byte that counts
# the nesting of a copy)
foreach(recursion_depth_limit 0 1 254)
json_test_set_test_options(test-recursion_depth_limit_${recursion_depth_limit}
COMPILE_DEFINITIONS JSON_RECURSION_DEPTH_LIMIT=${recursion_depth_limit}
)
json_test_add_test_for(src/unit-recursion_depth_limit.cpp
NAME test-recursion_depth_limit_${recursion_depth_limit}
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
endforeach()
# test the parser again with JSON_DIAGNOSTIC_POSITIONS enabled
json_test_set_test_options(test-class_parser_diagnostic_positions
COMPILE_DEFINITIONS JSON_DIAGNOSTIC_POSITIONS=1
+11 -15
View File
@@ -236,8 +236,7 @@ TEST_CASE("controlled bad_alloc")
next_construct_fails = false;
// deeper than the levels the copy constructor descends into
// (recursion_depth_limit(): 128 by default, 254 at most)
// deeper than the 128 levels the copy constructor descends into
const std::size_t depth = 300;
my_json j = 1;
@@ -301,9 +300,8 @@ struct nth_alloc_fails_allocator : std::allocator<T>
};
};
// builds a value nested more than recursion_depth_limit() (128 by default)
// levels deep - the bound the copy constructor descends into before it
// continues without the call stack - and
// builds a value nested more than 128 levels deep - the bound the copy
// constructor descends into before it continues without the call stack - and
// checks that a copy survives any single allocation of it failing: every
// attempt either throws std::bad_alloc, without crashing or leaving the
// source altered, or completes the copy
@@ -314,9 +312,9 @@ void check_deep_copy_survives_failing_allocation(bool nest_objects)
fail_at_alloc_call = -1;
// [[[ ... [1] ... ]]], or the same nesting with objects, two levels deeper than the limit
// [[[ ... [1] ... ]]], or the same nesting with objects, 130 levels deep
BasicJsonType src = 1;
for (std::size_t i = 0; i < nlohmann::detail::recursion_depth_limit() + 2; ++i)
for (std::size_t i = 0; i < 130; ++i)
{
if (nest_objects)
{
@@ -442,9 +440,8 @@ TEST_CASE("deep copy uses the provided allocator")
{
using counting_json = nlohmann::json::with_allocator_t<scratch_counting_allocator>;
// deeper than the levels the copy constructor descends into (128 by
// default, 254 at most), so the innermost objects are copied by the
// iterative deep copy
// deeper than the 128 levels the copy constructor descends into, so the
// innermost objects are copied by the iterative deep copy
counting_json j = 1;
for (std::size_t i = 0; i < 300; ++i)
{
@@ -502,12 +499,11 @@ TEST_CASE("converting a deeply nested value from another specialization fails cl
#if !(defined(_MSC_VER) && _MSC_VER < 1910 && defined(_ITERATOR_DEBUG_LEVEL) && _ITERATOR_DEBUG_LEVEL > 0)
using countdown_json = nlohmann::json::with_allocator_t<countdown_allocator>;
// deeper than the levels the converting constructor descends into
// (recursion_depth_limit(), 128 by default), so that failures land on both
// sides of the bound - or, built with JSON_NO_THREAD_LOCAL, all in the
// iterative conversion
// deeper than the 128 levels the converting constructor descends into, so
// that failures land on both sides of the bound - or, built with
// JSON_NO_THREAD_LOCAL, all in the iterative conversion
json j = {1, "two", {{"three", 3}}};
for (std::size_t i = 0; i < nlohmann::detail::recursion_depth_limit() + 22; ++i)
for (std::size_t i = 0; i < 150; ++i)
{
j = json{{"a", json::array({j, "sibling"})}};
}
+5 -15
View File
@@ -62,15 +62,6 @@ template<> struct StringMaker<std::partial_ordering>
namespace
{
// nesting depths on either side of detail::recursion_depth_limit(): none, the
// last one within the limit, the first one past it, and well past it (0, 127,
// 128, and 200 by default)
std::vector<std::size_t> depths_around_recursion_limit()
{
const std::size_t limit = nlohmann::detail::recursion_depth_limit();
return {0, limit > 0 ? limit - 1 : 0, limit, limit + 72};
}
// helper function to check std::less<json::value_t>
// see https://en.cppreference.com/w/cpp/utility/functional/less
template <typename A, typename B, typename U = std::less<json::value_t>>
@@ -977,7 +968,7 @@ TEST_CASE("copying an object preserves its comparator's state")
object["a"] = 3;
REQUIRE(object.dump() == R"({"B":2,"a":3,"b":1})");
for (const std::size_t depth : depths_around_recursion_limit())
for (const std::size_t depth : std::vector<std::size_t> {0, 127, 128, 200})
{
CAPTURE(depth)
@@ -1020,7 +1011,7 @@ TEST_CASE("equality of an object whose comparator treats different keys as equiv
// sanity check: the object type's own comparison already disagrees
CHECK_FALSE(a.get_ref<const ci_json::object_t&>() == b.get_ref<const ci_json::object_t&>());
for (const std::size_t depth : depths_around_recursion_limit())
for (const std::size_t depth : std::vector<std::size_t> {0, 127, 128, 200})
{
CAPTURE(depth)
@@ -1144,10 +1135,9 @@ TEST_CASE("operator<=> of binary values with a different subtype does not depend
return result;
};
// one level less than the recursion depth limit (127 by default) stays within it;
// the limit itself and more (128 and 200) do not, and must still agree with the
// levels that do
for (const std::size_t depth : depths_around_recursion_limit())
// 127 levels stay within nesting_depth_limit() (128); 128 and 200 do not,
// and must still agree with the levels that do
for (const std::size_t depth : std::vector<std::size_t> {0, 127, 128, 200})
{
CAPTURE(depth)
const json x = deep(a, depth);
+3 -5
View File
@@ -11,7 +11,6 @@
#include <nlohmann/json.hpp>
using nlohmann::json;
#include <cstddef>
#include <cstdint>
#include <deque>
#include <map>
@@ -218,17 +217,16 @@ TEST_CASE("array type with std::deque nested beyond the recursion limit")
// recursion_depth_limit() with an explicit stack of frames; with the
// iterators of std::deque, these frames are copied and moved through
// their own special members
const std::size_t depth = nlohmann::detail::recursion_depth_limit() + 72; // 200 by default
deque_json j = 1;
for (std::size_t i = 0; i < depth; ++i)
for (int i = 0; i < 200; ++i)
{
deque_json outer = deque_json::array();
outer.push_back(std::move(j));
j = std::move(outer);
}
// `depth` arrays of one element each (0x81) around the integer 1
std::vector<std::uint8_t> expected(depth, 0x81);
// 200 arrays of one element each (0x81) around the integer 1
std::vector<std::uint8_t> expected(200, 0x81);
expected.push_back(0x01);
CHECK(deque_json::to_cbor(j) == expected);
}
+2 -2
View File
@@ -475,10 +475,10 @@ TEST_CASE("Regression test for issue #5674 - copy construction must not require
SECTION("nested deeper than the copy constructor's descent bound")
{
// beyond recursion_depth_limit() (128 by default) levels, the copy constructor
// beyond nesting_depth_limit() (128) levels, the copy constructor
// copies without the call stack (copy_iteratively / copy_array_level),
// which used to assign the base class of every element it created
const std::size_t depth = nlohmann::detail::recursion_depth_limit() + 172;
const std::size_t depth = 300;
const json_with_const_base j = make_nested_array(depth);
const json_with_const_base copy = j; // NOLINT(performance-unnecessary-copy-initialization)
+9 -15
View File
@@ -128,19 +128,16 @@ TEST_CASE("Better diagnostics with positions")
check_copy(depth, true);
};
// around detail::recursion_depth_limit() (127, 128, and 129 by default)
const std::size_t limit = nlohmann::detail::recursion_depth_limit();
check_arrays(1);
check_arrays(limit > 0 ? limit - 1 : 0);
check_arrays(limit);
check_arrays(limit + 1);
check_arrays(127);
check_arrays(128);
check_arrays(129);
check_arrays(300);
check_objects(1);
check_objects(limit > 0 ? limit - 1 : 0);
check_objects(limit);
check_objects(limit + 1);
check_objects(127);
check_objects(128);
check_objects(129);
check_objects(300);
}
@@ -189,13 +186,10 @@ TEST_CASE("Better diagnostics with positions")
}
};
// around detail::recursion_depth_limit() (127, 128, and 129 by default)
const std::size_t limit = nlohmann::detail::recursion_depth_limit();
check_conversion(1);
check_conversion(limit > 0 ? limit - 1 : 0);
check_conversion(limit);
check_conversion(limit + 1);
check_conversion(127);
check_conversion(128);
check_conversion(129);
check_conversion(300);
}
+1 -3
View File
@@ -515,11 +515,9 @@ TEST_CASE("Regression tests for extended diagnostics")
// update(j, true) around its descent bound, where the nested vectors
// grow while the objects are merged without recursing
const std::size_t limit = nlohmann::detail::recursion_depth_limit();
// (the limit can be 0, where "one level less" is no level)
for (const std::size_t depth :
{
limit > 0 ? limit - 1 : 0, limit, limit + 2
nlohmann::detail::recursion_depth_limit() - 1, nlohmann::detail::recursion_depth_limit(), nlohmann::detail::recursion_depth_limit() + 2
})
{
ordered_json j = {{"z", {{"x", 1}}}};
+3 -4
View File
@@ -1975,15 +1975,14 @@ TEST_CASE("JSON patch: diff of deeply nested values")
SECTION("the diff reproduces the target at every depth")
{
// depths on either side of the nesting depth up to which diff()
// recurses (detail::recursion_depth_limit(), 128 by default); not every depth up
// recurses (detail::recursion_depth_limit(), 128); not every depth up
// to 300, as the test would then time out under Valgrind
std::vector<std::size_t> depths;
for (std::size_t depth = 0; depth <= 16; ++depth)
{
depths.push_back(depth);
}
const std::size_t limit = nlohmann::detail::recursion_depth_limit();
for (std::size_t depth = limit > 8 ? limit - 8 : 0; depth <= limit + 8; ++depth)
for (std::size_t depth = 120; depth <= 136; ++depth)
{
depths.push_back(depth);
}
@@ -2525,7 +2524,7 @@ TEST_CASE("JSON patch - diff of leaf values below the bound of the recursive des
// the values at the innermost level are diffed without recursion: equal
// leaves yield nothing, leaves of different types and different leaves
// are replaced
const std::size_t depth = nlohmann::detail::recursion_depth_limit() + 72; // 200 by default
const std::size_t depth = 200;
json source = {1, "text", 3, {{"x", 1}, {"y", 2}}, true, nullptr};
json target = {1, 2, 4, {{"x", 1}, {"y", 3}}, true, nullptr};
+293
View File
@@ -23,8 +23,10 @@ using nlohmann::json;
using namespace nlohmann::literals; // NOLINT(google-build-using-namespace)
#endif
#include <algorithm>
#include <cerrno>
#include <cstdlib>
#include <ctime>
#include <limits>
#include <map>
#include <random>
@@ -1560,3 +1562,294 @@ TEST_CASE("JSON patch with moves and copies")
CHECK_THROWS_AS(inplace.patch_inplace(json::parse(R"([{"op":"move","from":"","path":""}])")), json::out_of_range&);
CHECK(inplace == doc);
}
namespace
{
// the keys of a shape of flattened objects with long keys; `zero` is the
// token that is repeated `m` times
// 0: one key, 1: one `zero` followed by tokens 1, 2: the token 1 followed by
// tokens `zero`, 3 and 4: two keys with a long prefix, apart from other keys,
// 5: the same for two long branches, 6: more of the keys of 3
std::vector<std::string> shape_keys(const int shape, const std::size_t m, const std::string& zero)
{
std::string repeated;
for (std::size_t i = 0; i < m; ++i)
{
repeated += "/" + zero;
}
std::vector<std::string> keys;
switch (shape)
{
case 0:
keys.push_back(repeated);
break;
case 1:
{
std::string key = "/" + zero;
for (std::size_t i = 1; i < m; ++i)
{
key += "/1";
}
keys.push_back(key);
break;
}
case 2:
{
std::string key = "/1";
for (std::size_t i = 1; i < m; ++i)
{
key += "/" + zero;
}
keys.push_back(key);
break;
}
case 3:
case 4:
keys = {repeated + "/a0", "/10", repeated + "/a1", "/11"};
break;
case 5:
keys = {"/x" + repeated + "/a0", "/y" + repeated + "/a0", "/x" + repeated + "/a1", "/y" + repeated + "/a1"};
break;
case 6:
for (int j = 0; j < 4; ++j)
{
keys.push_back(repeated + "/a" + std::to_string(j));
keys.push_back("/1" + std::to_string(j));
}
break;
default:
break;
}
return keys;
}
// the best of `runs` times in seconds that std::clock() measures to unflatten the keys,
// which an ordered_json holds in the order they are given (shape 4 uses a
// json, which sorts them)
double unflatten_cpu_time(const int shape, const std::size_t m, const int runs, const std::string& zero)
{
const std::vector<std::string> keys = shape_keys(shape, m, zero);
nlohmann::ordered_json ordered = nlohmann::ordered_json::object();
json sorted = json::object();
for (const auto& key : keys)
{
ordered[key] = 1;
sorted[key] = 1;
}
double best = 1e30;
for (int i = 0; i < runs; ++i)
{
const std::clock_t start = std::clock();
if (shape == 4)
{
const json nested = sorted.unflatten();
}
else
{
const nlohmann::ordered_json nested = ordered.unflatten();
}
best = (std::min)(best, static_cast<double>(std::clock() - start) / static_cast<double>(CLOCKS_PER_SEC));
}
return best;
}
} // namespace
TEST_CASE("unflatten of keys with many reference tokens")
{
using nlohmann::ordered_json;
SECTION("which prefixes become arrays")
{
// sorted keys that share a long beginning
CHECK(json({{"/x/y/q/1", 5}, {"/x/y/w/0", 4}, {"/x/y/w/1", 3}, {"/x/y/z/0", 1}, {"/x/y/z/1", 2}}).unflatten() ==
json::parse(R"({"x":{"y":{"q":{"1":5},"w":[4,3],"z":[1,2]}}})"));
// prefixes that differ in the position of a slash only
CHECK(json({{"/a/0/x", 1}, {"/a/1/y", 3}, {"/ab/1", 2}}).unflatten() ==
json::parse(R"({"a":[{"x":1},{"y":3}],"ab":{"1":2}})"));
CHECK(json({{"/a/b/1", 1}, {"/ab/0", 2}}).unflatten() ==
json::parse(R"({"a":{"b":{"1":1}},"ab":[2]})"));
// tokens longer than one word, escapes, and a token 0 below a token 1
const std::string t = "token_with_more_than_eight_bytes";
CHECK(json({{"/" + t + "/1/b/0", 1}, {"/" + t + "/0", 2}, {"/" + t + "/2/a~1b", 3}}).unflatten() ==
json::parse(R"({"token_with_more_than_eight_bytes":[2,{"b":[1]},{"a/b":3}]})"));
// prefixes of more than 64 bytes, that share a long beginning
std::string deep;
json expected = {{"a", {1, 2}}, {"b", {{"1", 3}}}, {"c", {{"1", {{"x", {4}}}}}}};
for (int i = 0; i < 30; ++i)
{
deep += "/seg";
expected = {{"seg", expected}};
}
CHECK(json({{deep + "/c/1/x/0", 4}, {deep + "/a/1", 2}, {deep + "/b/1", 3}, {deep + "/a/0", 1}}).unflatten() == expected);
CHECK(json(ordered_json({{deep + "/c/1/x/0", 4}, {deep + "/a/1", 2}, {deep + "/b/1", 3}, {deep + "/a/0", 1}}).unflatten()) == expected);
CHECK(json(ordered_json({{deep + "/a/1", 2}, {deep + "/a/0", 1}, {deep + "/b/1", 3}, {deep + "/c/1/x/0", 4}}).unflatten()) == expected);
// prefixes of every length around the limit up to which they are hashed directly
for (std::size_t length = 40; length < 90; ++length)
{
const std::string name(length, 'x');
const json flat = {{"/" + name + "/a/0", 1}, {"/" + name + "/a/1", 2}, {"/" + name + "/b/1", 3}, {"/" + name + "/c/1/0", 4}};
CHECK(flat.unflatten() == json({{name, {{"a", {1, 2}}, {"b", {{"1", 3}}}, {"c", {{"1", {4}}}}}}}));
}
// only the parent of a token 0 is an array, however deep the key is
CHECK(json({{"/0/1/1/1", 1}}).unflatten() == json::parse(R"([{"1":{"1":{"1":1}}}])"));
CHECK(json({{"/1/1/1/0", 1}}).unflatten() == json::parse(R"({"1":{"1":{"1":[1]}}})"));
CHECK(json({{"/1/0/1/0", 1}}).unflatten() == json::parse(R"({"1":[{"1":[1]}]})"));
}
SECTION("the result does not depend on the order of the keys")
{
// random documents without empty containers and without the key "0" in
// an object: flatten() and unflatten() are inverse to each other
std::mt19937 gen(20261110); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible
const std::vector<std::string> names = {"a", "b", "ab", "1", "2", "10", "a/b", "a~b", "token_with_more_than_eight_bytes", "x"};
struct builder
{
std::mt19937& gen;
const std::vector<std::string>& names;
json operator()(int depth) const
{
const int kind = std::uniform_int_distribution<int>(0, 9)(gen);
if (depth == 0 || kind < 3)
{
return kind;
}
json result = kind < 6 ? json::array() : json::object();
const int size = std::uniform_int_distribution<int>(1, 4)(gen);
for (int i = 0; i < size; ++i)
{
if (kind < 6)
{
result.push_back((*this)(depth - 1));
}
else
{
result[names[std::uniform_int_distribution<std::size_t>(0, names.size() - 1)(gen)]] = (*this)(depth - 1);
}
}
return result;
}
};
const builder build{gen, names};
for (int round = 0; round < 200; ++round)
{
const json doc = build(6);
const json flat = doc.flatten();
CHECK(flat.unflatten() == doc);
// the same entries in shuffled order, in an object that keeps it
std::vector<std::pair<std::string, json>> entries;
for (const auto& el : flat.items())
{
entries.emplace_back(el.key(), el.value());
}
std::shuffle(entries.begin(), entries.end(), gen);
ordered_json shuffled = ordered_json::object();
for (const auto& e : entries)
{
shuffled[e.first] = e.second;
}
CHECK(json(shuffled.unflatten()) == doc);
}
}
SECTION("keys with long prefixes, in every order")
{
// prefixes of more than 64 bytes that share a long beginning, and
// continue with tokens that start the same way
std::string deep;
for (int i = 0; i < 30; ++i)
{
deep += "/seg";
}
const std::string other = deep.substr(0, 70) + "/x" + deep.substr(70);
std::vector<std::string> keys = {deep + "/a/0", deep + "/ab/1", deep + "/ab/x", deep + "/abc/0/y/0", other + "/b/1/0", deep + "/abc/1"};
std::sort(keys.begin(), keys.end());
// the result for the keys in sorted order, which json iterates in
json sorted = json::object();
for (const auto& key : keys)
{
sorted[key] = 1;
}
const json expected = sorted.unflatten();
CHECK(expected.is_object());
// ordered_json iterates in the order the keys were added
std::vector<std::size_t> order(keys.size());
for (std::size_t i = 0; i < order.size(); ++i)
{
order[i] = i;
}
std::size_t permutations = 0;
bool all_equal = true;
do
{
nlohmann::ordered_json flat = nlohmann::ordered_json::object();
for (const auto i : order)
{
flat[keys[i]] = 1;
}
all_equal = all_equal && json(flat.unflatten()) == expected;
++permutations;
}
while (std::next_permutation(order.begin(), order.end()));
CHECK(permutations == 720);
CHECK(all_equal);
}
SECTION("time is linear in the number of tokens")
{
// A bound in seconds would depend on the machine. Instead, the time is
// compared with the time for the same keys with the token 0 replaced by
// a token that is not a number: there is no array, so no prefix is
// looked up. The ratio is 1 to 3 for a linear algorithm, and between
// 12 and 100 when looking up a prefix takes time proportional to its
// length. The time that std::clock() measures is mostly the time of the
// processor, which a busy machine does not inflate (on MSVC, it is the
// time that has passed), and the best of two or three runs counts.
for (int shape = 0; shape < 7; ++shape)
{
// the shapes with several keys, in an order that does not sort them,
// need more tokens to be slow enough to tell
const bool many = shape == 3 || shape == 5 || shape == 6;
const std::size_t m = many ? 40000 : 10000;
const int runs = many ? 2 : 3;
// a busy machine may spoil one comparison, hardly two
bool linear = false;
double t_array = 0;
double t_object = 0;
for (int attempt = 0; attempt < 2 && !linear; ++attempt)
{
t_array = unflatten_cpu_time(shape, m, runs, "0");
t_object = (std::max)(unflatten_cpu_time(shape, m, runs, "a"), 1e-3);
linear = t_array < 6 * t_object;
}
CAPTURE(shape)
CAPTURE(t_array)
CAPTURE(t_object)
CHECK(linear);
}
// the nesting is as expected: an array exactly below each token 0
const json nested_deep = json::object({{shape_keys(0, 1000, "0")[0], 1}}).unflatten();
std::size_t depth = 0;
for (const json* p = &nested_deep; p->is_array(); p = &p->back())
{
++depth;
}
CHECK(depth == 1000);
const json nested_ones = json::object({{shape_keys(1, 1000, "0")[0], 1}}).unflatten();
CHECK(nested_ones.is_array());
CHECK(nested_ones.back().is_object());
}
}
+12 -17
View File
@@ -50,18 +50,13 @@ json nested_object(const std::size_t depth, json leaf)
TEST_CASE("issue #5392 - binary writers on deeply nested values")
{
// The limit (128 by default) of the levels that the writers recurse into.
const std::size_t limit = nlohmann::detail::recursion_depth_limit();
// 72 levels more than the limit (200 by default) is past the point where
// the writers stop recursing, and still shallow enough that from_* (which
// does not recurse) and operator== (which does, up to the limit) are quick.
const json deep_array = nested_array(limit + 72, json(0));
const json deep_object = nested_object(limit + 72, json("x"));
const json empty_array = nested_array(limit + 72, json::array());
const json empty_object = nested_object(limit + 72, json::object());
// 2 * (limit / 2 + 16) levels: 160 by default, and more than the limit for every limit
const json mixed = nested_object((limit / 2) + 16, nested_array((limit / 2) + 16, json(true)));
// 200 is past the point where the writers stop recursing, and still
// shallow enough that from_* and operator== (which still recurse) are fine.
const json deep_array = nested_array(200, json(0));
const json deep_object = nested_object(200, json("x"));
const json empty_array = nested_array(200, json::array());
const json empty_object = nested_object(200, json::object());
const json mixed = nested_object(80, nested_array(80, json(true)));
SECTION("roundtrip past the recursion bound")
{
@@ -100,7 +95,7 @@ TEST_CASE("issue #5392 - binary writers on deeply nested values")
SECTION("the two ways of writing a value meet at the bound")
{
for (std::size_t depth = limit > 8 ? limit - 8 : 0; depth <= limit + 12; ++depth)
for (std::size_t depth = 120; depth <= 140; ++depth)
{
CAPTURE(depth)
@@ -123,8 +118,8 @@ TEST_CASE("issue #5392 - binary writers on deeply nested values")
const json ndarray = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
const json invalid = json({{"_ArrayType_", "nope"}, {"_ArraySize_", {1}}, {"_ArrayData_", {1}}});
const json deep_ndarray = nested_array(limit + 12, ndarray);
const json deep_invalid = nested_array(limit + 12, invalid);
const json deep_ndarray = nested_array(140, ndarray);
const json deep_invalid = nested_array(140, invalid);
CHECK(json::from_bjdata(json::to_bjdata(deep_ndarray)) == deep_ndarray);
CHECK(json::from_bjdata(json::to_bjdata(deep_invalid)) == deep_invalid);
@@ -136,10 +131,10 @@ TEST_CASE("issue #5392 - binary writers on deeply nested values")
// nested one-element arrays around the recursion bound: the exact
// bytes a writer produces do not depend on whether it stayed on the
// call stack or moved to the heap one partway through
// (the limit can be 0, where "one level less" is no level)
for (const std::size_t depth :
{
limit > 0 ? limit - 1 : 0, limit, limit + 1, limit + 2
nlohmann::detail::recursion_depth_limit() - 1, nlohmann::detail::recursion_depth_limit(),
nlohmann::detail::recursion_depth_limit() + 1, nlohmann::detail::recursion_depth_limit() + 2
})
{
CAPTURE(depth)
+2 -2
View File
@@ -115,7 +115,7 @@ TEST_CASE("issue #5650 - stack overflow converting between specializations")
// members that the two object types enumerate in different orders.
const auto bury = [](nlohmann::ordered_json value)
{
for (std::size_t i = 0; i < nlohmann::detail::recursion_depth_limit() + 72; ++i)
for (std::size_t i = 0; i < 200; ++i)
{
value = nlohmann::ordered_json::array({std::move(value)});
}
@@ -124,7 +124,7 @@ TEST_CASE("issue #5650 - stack overflow converting between specializations")
const auto dig = [](const json & value)
{
const json* current = &value;
for (std::size_t i = 0; i < nlohmann::detail::recursion_depth_limit() + 72; ++i)
for (std::size_t i = 0; i < 200; ++i)
{
current = &current->at(0);
}
+2 -3
View File
@@ -930,15 +930,14 @@ TEST_CASE("JSON Merge Patch diff of deeply nested values")
SECTION("the patch is the same past the descent bound")
{
// depths on either side of the nesting depth up to which merge_diff()
// recurses (detail::recursion_depth_limit(), 128 by default); not every depth up
// recurses (detail::recursion_depth_limit(), 128); not every depth up
// to 300, to keep the test fast under Valgrind
std::vector<std::size_t> depths;
for (std::size_t depth = 0; depth <= 16; ++depth)
{
depths.push_back(depth);
}
const std::size_t limit = nlohmann::detail::recursion_depth_limit();
for (std::size_t depth = limit > 8 ? limit - 8 : 0; depth <= limit + 8; ++depth)
for (std::size_t depth = 120; depth <= 136; ++depth)
{
depths.push_back(depth);
}
+2 -2
View File
@@ -302,7 +302,7 @@ TEST_CASE("JSON Merge Patch on deeply nested values")
SECTION("patching past the descent bound gives the same result")
{
// every depth on either side of where the iterative version takes
// over (detail::recursion_depth_limit(), 128 by default)
// over (detail::recursion_depth_limit(), 128)
for (std::size_t depth = 0; depth <= 300; ++depth)
{
CAPTURE(depth)
@@ -396,7 +396,7 @@ TEST_CASE("merge_patch() with an argument that aliases *this (#5641)")
SECTION("a patch nested past the iterative descent bound aliases *this")
{
// every depth on either side of where the iterative version takes
// over (detail::recursion_depth_limit(), 128 by default); patching *this with
// over (detail::recursion_depth_limit(), 128); patching *this with
// itself is idempotent, aliased or not
for (const std::size_t depth :
{
+2 -2
View File
@@ -1124,7 +1124,7 @@ TEST_CASE("update() on deeply nested values")
SECTION("merging past the descent bound gives the same result")
{
// every depth on either side of where the iterative version takes
// over (detail::recursion_depth_limit(), 128 by default)
// over (detail::recursion_depth_limit(), 128)
for (std::size_t depth = 0; depth <= 300; ++depth)
{
CAPTURE(depth)
@@ -1215,7 +1215,7 @@ TEST_CASE("update() with an argument that aliases *this (#5641)")
SECTION("nested past the iterative descent bound aliases *this")
{
// every depth on either side of where the iterative version takes
// over (detail::recursion_depth_limit(), 128 by default); merging *this into
// over (detail::recursion_depth_limit(), 128); merging *this into
// itself is idempotent, aliased or not
for (const std::size_t depth :
{
-600
View File
@@ -1,600 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
using nlohmann::json;
#include <algorithm>
#include <cstddef>
#include <cstdint>
#include <functional>
#include <iomanip>
#include <map>
#include <memory>
#include <sstream>
#include <string>
#include <utility>
#include <vector>
// JSON_RECURSION_DEPTH_LIMIT changes only how much call stack the walks over a
// value use, never what they return. This file checks both: the macro is
// honoured (the number of levels the containers handle themselves is the limit)
// and every walk returns what it returns with the default limit. The expected
// results are built here, without the library, so that the same file passes for
// every limit; tests/CMakeLists.txt builds it with limits 0, 1, and 254 besides
// the default. Values are nested up to 300 levels (and past every limit, so that
// the explicit stack of every walk is exercised, even for the limit of 254) and
// are built and taken apart without recursion. Keep the depths moderate: walks
// like diff and dump with indentation are quadratic in the depth.
namespace
{
constexpr std::size_t limit()
{
return nlohmann::detail::recursion_depth_limit();
}
// a value nested as @a kinds says, outermost level first: 'a' is an array of
// one element, 'o' an object with the single key "a"; @a leaf is innermost
json build(const std::string& kinds, json leaf)
{
json j = std::move(leaf);
for (std::size_t i = kinds.size(); i-- > 0;)
{
if (kinds[i] == 'a')
{
json a = json::array();
a.push_back(std::move(j));
j = std::move(a);
}
else
{
json o = json::object();
o["a"] = std::move(j);
j = std::move(o);
}
}
return j;
}
std::string repeat(const std::string& s, std::size_t n)
{
std::string result;
for (std::size_t i = 0; i < n; ++i)
{
result += s;
}
return result;
}
// the nesting of the given shape and depth: arrays, objects, or both alternating
std::string make_kinds(char shape, std::size_t depth)
{
std::string kinds;
for (std::size_t i = 0; i < depth; ++i)
{
if (shape == 'm')
{
kinds += i % 2 == 0 ? 'a' : 'o';
}
else
{
kinds += shape;
}
}
return kinds;
}
std::string expected_compact(const std::string& kinds, const std::string& leaf)
{
std::string s;
for (const char kind : kinds)
{
s += kind == 'a' ? "[" : "{\"a\":";
}
s += leaf;
for (std::size_t i = kinds.size(); i-- > 0;)
{
s += kinds[i] == 'a' ? "]" : "}";
}
return s;
}
// dump(1)
std::string expected_indented(const std::string& kinds, const std::string& leaf)
{
std::string s;
for (std::size_t i = 0; i < kinds.size(); ++i)
{
s += kinds[i] == 'a' ? "[\n" : "{\n";
s += std::string(i + 1, ' ');
if (kinds[i] == 'o')
{
s += "\"a\": ";
}
}
s += leaf;
for (std::size_t i = kinds.size(); i-- > 0;)
{
s += "\n" + std::string(i, ' ') + (kinds[i] == 'a' ? "]" : "}");
}
return s;
}
// the JSON pointer to the innermost value
std::string pointer(const std::string& kinds)
{
std::string s;
for (const char kind : kinds)
{
s += kind == 'a' ? "/0" : "/a";
}
return s;
}
// the binary formats: the bytes that open and close a level, and the leaf 1
std::vector<std::uint8_t> expected_cbor(const std::string& kinds)
{
std::vector<std::uint8_t> bytes;
for (const char kind : kinds)
{
if (kind == 'a')
{
bytes.push_back(0x81);
}
else
{
bytes.push_back(0xA1);
bytes.push_back(0x61);
bytes.push_back('a');
}
}
bytes.push_back(0x01);
return bytes;
}
std::vector<std::uint8_t> expected_msgpack(const std::string& kinds)
{
std::vector<std::uint8_t> bytes;
for (const char kind : kinds)
{
if (kind == 'a')
{
bytes.push_back(0x91);
}
else
{
bytes.push_back(0x81);
bytes.push_back(0xA1);
bytes.push_back('a');
}
}
bytes.push_back(0x01);
return bytes;
}
std::vector<std::uint8_t> expected_ubjson(const std::string& kinds)
{
std::vector<std::uint8_t> bytes;
for (const char kind : kinds)
{
if (kind == 'a')
{
bytes.push_back('[');
}
else
{
bytes.push_back('{');
bytes.push_back('i');
bytes.push_back(1);
bytes.push_back('a');
}
}
bytes.push_back('i');
bytes.push_back(1);
for (std::size_t i = kinds.size(); i-- > 0;)
{
bytes.push_back(kinds[i] == 'a' ? ']' : '}');
}
return bytes;
}
// the depths worth trying for the limit: either side of it, twice it (if that
// is not too deep to be quick), and a value nested well past all limits
std::vector<std::size_t> depths()
{
std::vector<std::size_t> result;
result.push_back(1);
result.push_back(2);
result.push_back(limit() > 0 ? limit() - 1 : 3);
result.push_back(limit());
result.push_back(limit() + 1);
if ((2 * limit()) + 5 <= 300)
{
result.push_back((2 * limit()) + 5);
}
result.push_back(300);
std::sort(result.begin(), result.end());
result.erase(std::unique(result.begin(), result.end()), result.end());
result.erase(std::remove(result.begin(), result.end(), static_cast<std::size_t>(0)), result.end());
return result;
}
// the call for every shape and depth
void for_each_value(const std::function<void(const std::string&)>& check)
{
const std::string shapes = "aom";
for (const char shape : shapes)
{
for (const std::size_t depth : depths())
{
CAPTURE(shape)
CAPTURE(depth)
check(make_kinds(shape, depth));
}
}
}
// An array type that counts how many of its copies are being made at once. A
// copy of a value copies one level at a time through the array type until the
// limit has been reached, so the peak is the number of levels that were copied
// with the call stack.
std::size_t copies_going_on = 0;
std::size_t copies_peak = 0;
std::size_t equal_going_on = 0;
std::size_t equal_peak = 0;
template<class T, class Allocator = std::allocator<T>>
class counting_vector : public std::vector<T, Allocator>
{
public:
counting_vector() = default;
template<class InputIt>
counting_vector(InputIt first, InputIt last) : std::vector<T, Allocator>(first, last) {}
counting_vector(const counting_vector& other) : std::vector<T, Allocator>()
{
++copies_going_on;
copies_peak = (std::max)(copies_peak, copies_going_on);
this->std::vector<T, Allocator>::operator=(static_cast<const std::vector<T, Allocator>&>(other));
--copies_going_on;
}
counting_vector(counting_vector&&) = default;
counting_vector& operator=(const counting_vector&) = default;
counting_vector& operator=(counting_vector&&) = default;
~counting_vector() = default;
friend bool operator==(const counting_vector& lhs, const counting_vector& rhs)
{
++equal_going_on;
equal_peak = (std::max)(equal_peak, equal_going_on);
const bool result = static_cast<const std::vector<T, Allocator>&>(lhs) == static_cast<const std::vector<T, Allocator>&>(rhs);
--equal_going_on;
return result;
}
};
using counting_json = json::with_array_t<counting_vector>;
// A base class whose copy constructor copies a whole value of the same
// specialization when it copies a marked element, as user code may do from a
// hook. That copy runs while the copy that it is part of is going on.
struct copy_hook // NOLINT(cppcoreguidelines-special-member-functions,hicpp-special-member-functions)
{
int mark = 0;
copy_hook() = default;
copy_hook(const copy_hook& other);
copy_hook(copy_hook&&) = default;
copy_hook& operator=(const copy_hook&) = default;
copy_hook& operator=(copy_hook&&) = default;
};
using hook_json = nlohmann::basic_json<std::map, counting_vector, std::string, bool, std::int64_t, std::uint64_t, double,
std::allocator, nlohmann::adl_serializer, std::vector<std::uint8_t>, copy_hook>;
const hook_json* hook_copy_source = nullptr;
int hook_copies_going_on = 0;
copy_hook::copy_hook(const copy_hook& other) : mark(other.mark)
{
// the value that is copied is marked, too: its copy copies it again, once
if (other.mark == 1 && hook_copy_source != nullptr && hook_copies_going_on < 2)
{
++hook_copies_going_on;
const hook_json nested = *hook_copy_source; // NOLINT(performance-unnecessary-copy-initialization)
static_cast<void>(nested);
--hook_copies_going_on;
}
}
// an array nested @a depth levels around the integer 1, which is marked if @a marked
hook_json hook_nested(std::size_t depth, bool marked)
{
hook_json j = 1;
if (marked)
{
static_cast<copy_hook&>(j).mark = 1;
}
for (std::size_t i = 0; i < depth; ++i)
{
hook_json a = hook_json::array();
a.push_back(std::move(j));
j = std::move(a);
}
return j;
}
} // namespace
// the macro is the limit, and is in the range the library supports
static_assert(nlohmann::detail::recursion_depth_limit() == static_cast<std::size_t>(JSON_RECURSION_DEPTH_LIMIT),
"the library does not use JSON_RECURSION_DEPTH_LIMIT");
static_assert(nlohmann::detail::recursion_depth_limit() <= 254, "the nesting depth must fit in a byte");
TEST_CASE("JSON_RECURSION_DEPTH_LIMIT is honoured")
{
SECTION("copying and comparing stop descending at the limit")
{
// copy and compare a value nested deeper than the limit, with the array type that counts
counting_json inner = 1;
for (std::size_t i = 0; i < (2 * limit()) + 5; ++i)
{
counting_json a = counting_json::array();
a.push_back(std::move(inner));
inner = std::move(a);
}
copies_going_on = copies_peak = 0;
const counting_json copy = inner;
CHECK(copies_going_on == 0);
equal_going_on = equal_peak = 0;
CHECK((inner == copy));
CHECK(equal_going_on == 0);
#ifdef JSON_NO_THREAD_LOCAL
// without the counters of the levels, no level is descended into
CHECK(copies_peak == 0);
CHECK(equal_peak == 0);
#else
// copying descends into as many levels as the limit says, and nothing further
CHECK(copies_peak == limit());
CHECK(equal_peak == limit());
#endif
// a value that is not nested deeper than the limit is not cut short
const std::size_t shallow = limit() > 0 ? limit() : 1;
counting_json small = 1;
for (std::size_t i = 0; i < shallow; ++i)
{
counting_json a = counting_json::array();
a.push_back(std::move(small));
small = std::move(a);
}
copies_going_on = copies_peak = 0;
const counting_json small_copy = small;
CHECK(copies_going_on == 0);
#ifndef JSON_NO_THREAD_LOCAL
CHECK(copies_peak == limit());
#endif
CHECK(small_copy.dump() == expected_compact(make_kinds('a', shallow), "1"));
}
}
TEST_CASE("a copy made while a copy is going on is bounded, too")
{
// The innermost element of the value is marked, and copying it (which happens
// after limit() levels have been entered, when the copy continues without the
// call stack) copies the value again, and that copy once more. Those copies
// must not be given limit() levels of their own: the count of the levels entered
// never exceeds the limit, not even by one, which for the largest limit would
// wrap the byte that holds it around to 0.
const hook_json marked = hook_nested((2 * limit()) + 5, true);
hook_copy_source = &marked;
copies_going_on = copies_peak = 0;
const hook_json copy = marked; // NOLINT(performance-unnecessary-copy-initialization)
hook_copy_source = nullptr;
CHECK(copies_going_on == 0);
CHECK(static_cast<const copy_hook&>(copy).mark == 0); // (the top level is not marked)
#ifdef JSON_NO_THREAD_LOCAL
CHECK(copies_peak == 0);
#else
CHECK(copies_peak == limit());
#endif
}
TEST_CASE("walks over values nested around the recursion depth limit")
{
SECTION("copying and comparing")
{
for_each_value([](const std::string & kinds)
{
const json a = build(kinds, 1);
const json b = a; // NOLINT(performance-unnecessary-copy-initialization)
CHECK((a == b));
CHECK_FALSE((a != b));
CHECK_FALSE((a < b));
CHECK((a <= b));
CHECK((a >= b));
CHECK_FALSE((a > b));
json assigned;
assigned = a;
CHECK((assigned == a));
// differing only in the innermost value
const json c = build(kinds, 2);
CHECK((a != c));
CHECK((a < c));
CHECK((c > a));
CHECK_FALSE((c < a));
CHECK_FALSE((a == c));
// a same-shape array or object that is one level shallower
const json shallower = build(kinds.substr(1), 1);
CHECK((shallower != a));
const json outer = build(kinds.substr(0, 1), 1);
CHECK((outer != a) == (kinds.size() > 1));
const json moved = std::move(assigned);
CHECK((moved == a));
});
}
SECTION("serializing")
{
for_each_value([](const std::string & kinds)
{
const json a = build(kinds, 1);
const std::string compact = expected_compact(kinds, "1");
const std::string indented = expected_indented(kinds, "1");
CHECK(a.dump() == compact);
CHECK(a.dump(1) == indented);
std::ostringstream compact_stream;
compact_stream << a;
CHECK(compact_stream.str() == compact);
std::ostringstream indented_stream;
indented_stream << std::setw(1) << a;
CHECK(indented_stream.str() == indented);
CHECK((json::parse(compact) == a));
CHECK((json::parse(indented) == a));
});
}
SECTION("hashing")
{
for_each_value([](const std::string & kinds)
{
const json a = build(kinds, 1);
const std::hash<json> hash;
CHECK(hash(a) == hash(json(a)));
CHECK(hash(a) == hash(json::parse(expected_compact(kinds, "1"))));
CHECK(hash(a) != hash(build(kinds, 2)));
CHECK(hash(a) != hash(build(kinds.substr(1), 1)));
});
}
SECTION("diff and patch")
{
for_each_value([](const std::string & kinds)
{
const json a = build(kinds, 1);
const json b = build(kinds, 2);
const json difference = json::diff(a, b);
CHECK(difference.dump() == "[{\"op\":\"replace\",\"path\":\"" + pointer(kinds) + "\",\"value\":2}]");
CHECK((a.patch(difference) == b));
CHECK((json::diff(a, a) == json::array()));
});
}
SECTION("flatten and unflatten")
{
for_each_value([](const std::string & kinds)
{
const json a = build(kinds, 1);
const json flat = a.flatten();
CHECK(flat.dump() == "{\"" + pointer(kinds) + "\":1}");
CHECK((flat.unflatten() == a));
});
}
SECTION("merge_patch and update")
{
for (const std::size_t depth : depths())
{
CAPTURE(depth)
const std::string kinds = make_kinds('o', depth);
const std::string all_but_leaf = repeat("{\"a\":", depth - 1);
// the patch removes the innermost member, or replaces it
json target = build(kinds, 1);
target.merge_patch(build(kinds, json(nullptr)));
CHECK(target.dump() == all_but_leaf + "{}" + repeat("}", depth - 1));
target = build(kinds, 1);
target.merge_patch(build(kinds, 2));
CHECK(target.dump() == expected_compact(kinds, "2"));
// merging the objects of two values nested alike, and replacing the first level
const json base = build(kinds.substr(1), json::parse("{\"x\":1}"));
const json other = build(kinds.substr(1), json::parse("{\"y\":2}"));
json merged = json{{"a", base}};
merged.update(json{{"a", other}}, true);
CHECK(merged.dump() == expected_compact(kinds, "{\"x\":1,\"y\":2}"));
json replaced = json{{"a", base}};
replaced.update(json{{"a", other}});
CHECK(replaced.dump() == expected_compact(kinds, "{\"y\":2}"));
// update from a range
json from_range = json{{"a", base}};
const json source = json{{"a", other}};
from_range.update(source.begin(), source.end(), true);
CHECK((from_range == merged));
}
}
SECTION("binary formats")
{
for_each_value([](const std::string & kinds)
{
const json a = build(kinds, 1);
const std::vector<std::uint8_t> cbor = json::to_cbor(a);
CHECK(cbor == expected_cbor(kinds));
CHECK((json::from_cbor(cbor) == a));
const std::vector<std::uint8_t> msgpack = json::to_msgpack(a);
CHECK(msgpack == expected_msgpack(kinds));
CHECK((json::from_msgpack(msgpack) == a));
const std::vector<std::uint8_t> ubjson = json::to_ubjson(a);
CHECK(ubjson == expected_ubjson(kinds));
CHECK((json::from_ubjson(ubjson) == a));
CHECK((json::from_bjdata(json::to_bjdata(a)) == a));
CHECK((json::from_bon8(json::to_bon8(a)) == a));
// BSON documents are objects
const json document = json{{"a", a}};
CHECK((json::from_bson(json::to_bson(document)) == document));
});
}
SECTION("conversion to another specialization")
{
for_each_value([](const std::string & kinds)
{
const json a = build(kinds, 1);
const nlohmann::ordered_json ordered = a;
CHECK(ordered.dump() == expected_compact(kinds, "1"));
const json back = ordered;
CHECK((back == a));
});
}
SECTION("destroying")
{
for_each_value([](const std::string & kinds)
{
json a = build(kinds, 1);
a = nullptr;
CHECK(a.is_null());
});
}
}
+2 -4
View File
@@ -1554,10 +1554,8 @@ TEST_CASE("regression tests 1")
SECTION("a missing member of an object")
{
my_json j = make();
#if defined(JSON_NO_THREAD_LOCAL) || JSON_RECURSION_DEPTH_LIMIT < 2
// (set by the user, or by the library for clang with MinGW; or fewer than two
// levels are copied by the object type itself, as for a recursion depth limit
// of 0 or 1: the objects below the outermost one are the second level)
#ifdef JSON_NO_THREAD_LOCAL
// (set by the user, or by the library for clang with MinGW)
// the objects below the outermost one are not copied by their
// object_t here, but member by member, which keeps their
// timestamps correct: "c" is not mistaken for "b" and is inserted