mirror of
https://github.com/nlohmann/json.git
synced 2026-11-14 19:38:20 +00:00
Niels Lohmann
b38131ec40
Copying, comparing, serializing, hashing, merging, diffing, converting and destroying a value, and the CBOR, MessagePack, UBJSON, BJData and BON8 writers recurse a fixed number of nesting levels on the call stack and finish deeper levels with an explicit stack (#5387). The number was hard-coded to 128. That is too much for small stacks: a 128-level dump() needs about 288 KiB at -O0 with Clang (2.3 KiB per level; 256 KiB with GCC) and crashed in a thread with a 256 KiB stack, and musl threads have 128 KiB by default. Others may want the opposite. The macro JSON_RECURSION_DEPTH_LIMIT, defined before including the library, sets the number; the default stays 128. The value must be an integer constant from 0 to 254 (not a floating-point number or bool), otherwise a static_assert names the macro. 254 is the largest value because the copy constructor and the comparison operators cannot pass a depth down and count it in a byte per thread (nesting_depth()). The guard that counts now counts only the levels it enters, so the count never exceeds the limit. It used to count the level that finds the limit reached as well, which made the count 255 at limit 254: a copy made by a custom base class while a copy was going on then wrapped the byte to 0 and recursed another 254 levels (twice the stack). 0 is allowed: every walk then uses its explicit stack from the first level on and needs a constant amount of stack (16 KiB or less at -O0 with Clang and GCC, for a value nested 20,000 levels deep). All uses of the limit compare a depth with it; none computes limit - 1 or limit + 1, and no walk has a constant 128 of its own. The macro changes only stack use and speed, never a result. Like JSON_NO_THREAD_LOCAL it is not part of the ABI tag and must be the same in all translation units (ODR); the page of the macro says what happens otherwise. Supporting a limit of 0 needed one fix: update() without merge_objects reached the explicit-stack merge at depth 0, which always merges objects. update_members now takes that path only with merge_objects, which is the only case in which it recurses at all. Measurements (macOS arm64, Apple clang 21 and g++ 16, documented in the macro page with a table per operation and optimization level, and a program to reproduce them): the stack needed at the default of 128 levels is up to 288 KiB (dump at -O0, Clang), 224 KiB (copy), and 48 KiB or less at -O2; at 254 levels up to 576 KiB at -O0 and 112 KiB at -O2. Speed at -O2 relative to the default: ordinary documents (nested 2 to 10 levels) are 1.1 to 2.4 times as slow with a limit of 0 or 1 and unchanged (within about 0.1) with 16 or 254; values nested 1,000 or 100,000 levels deep show no difference for any limit. The default build does the same work as before: the instructions retired for copy and conversion (the walks whose guard changed; net of loading, canada, citm, twitter, jeopardy and deep values), ==, dump and hash differ from the base by at most 0.02 % for copy and 0.71 % in either direction for the others (noise), and the cycles for copy by at most 0.65 %. Tests: the existing tests that depended on 128 (comparison, allocator, custom base class, diagnostics, binary writers, patch, merge_diff, conversions, custom array type) derive their depths from the limit, and two assumed a limit of at least 1 (limit - 1 wrapped around). tests/src/unit-recursion_depth_limit.cpp checks that the macro is honoured (the array type of a value counts the levels that a copy and a comparison descend) and that all walks return the same results for values nested 1 to 300 levels deep as arrays, objects and both, against expected strings and bytes built independently of the library; CMake builds it with limits 0, 1 and 254 besides the default, and the CI target ci_test_recursion_depth_limit_0 runs the whole suite with a limit of 0. A test with a base class that copies the value from its copy constructor covers the wrap-around at 254. The new test does not compile against the previous header with -DJSON_RECURSION_DEPTH_LIMIT=1. Docs: new macro page (range, stack and speed tables, small-stack guidance, ODR), index, nav, macro overview, "Nesting depth" of the untrusted-input page (which claimed that no walk could exhaust the stack however deep a value is nested; the default assumes a stack of a few hundred KiB), the assurance case, and the page of JSON_NO_THREAD_LOCAL. No breaking changes: the default is unchanged, and the macro is new. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Documentation
Generate documentation
Note on documentation: The source files contain links to the online documentation at https://json.nlohmann.me.
This URL provides the most recent documentation and also applies to previous versions. Documentation for deprecated
functions is not removed; instead, it is marked as deprecated.
If you want to view the documentation for a specific tag or commit hash, you can generate it locally as follows (example
using tag v3.12.0):
git clone https://github.com/nlohmann/json.git
cd json
git checkout v3.12.0
make install_venv serve -C docs/mkdocs
Open http://127.0.0.1:8000/ in your browser. Replace any URL in the source code that points to
https://json.nlohmann.me with http://127.0.0.1:8000 to view the documentation for the selected tag or commit hash.