Files
json/tests/src/unit-merge_patch.cpp
T
Niels Lohmann b38131ec40 Add JSON_RECURSION_DEPTH_LIMIT to set how deep value walks recurse
Copying, comparing, serializing, hashing, merging, diffing, converting and
destroying a value, and the CBOR, MessagePack, UBJSON, BJData and BON8 writers
recurse a fixed number of nesting levels on the call stack and finish deeper
levels with an explicit stack (#5387). The number was hard-coded to 128. That is
too much for small stacks: a 128-level dump() needs about 288 KiB at -O0 with
Clang (2.3 KiB per level; 256 KiB with GCC) and crashed in a thread with a
256 KiB stack, and musl threads have 128 KiB by default. Others may want the
opposite.

The macro JSON_RECURSION_DEPTH_LIMIT, defined before including the library,
sets the number; the default stays 128. The value must be an integer constant
from 0 to 254 (not a floating-point number or bool), otherwise a static_assert
names the macro. 254 is the largest value because the copy constructor and the
comparison operators cannot pass a depth down and count it in a byte per thread
(nesting_depth()). The guard that counts now counts only the levels it enters,
so the count never exceeds the limit. It used to count the level that finds the
limit reached as well, which made the count 255 at limit 254: a copy made by a
custom base class while a copy was going on then wrapped the byte to 0 and
recursed another 254 levels (twice the stack).

0 is allowed: every walk then uses its explicit stack from the first level on
and needs a constant amount of stack (16 KiB or less at
-O0 with Clang and GCC, for a value nested 20,000 levels deep). All uses of the
limit compare a depth with it; none computes limit - 1 or limit + 1, and no
walk has a constant 128 of its own.

The macro changes only stack use and speed, never a result. Like
JSON_NO_THREAD_LOCAL it is not part of the ABI tag and must be the same in all
translation units (ODR); the page of the macro says what happens otherwise.

Supporting a limit of 0 needed one fix: update() without merge_objects reached
the explicit-stack merge at depth 0, which always merges objects. update_members
now takes that path only with merge_objects, which is the only case in which it
recurses at all.

Measurements (macOS arm64, Apple clang 21 and g++ 16, documented in the macro
page with a table per operation and optimization level, and a program to
reproduce them): the stack needed at the default of 128 levels is up to 288 KiB
(dump at -O0, Clang), 224 KiB (copy), and 48 KiB or less at -O2; at 254 levels
up to 576 KiB at -O0 and 112 KiB at -O2. Speed at -O2 relative to the default:
ordinary documents (nested 2 to 10 levels) are 1.1 to 2.4 times as slow with a
limit of 0 or 1 and unchanged (within about 0.1) with 16 or 254; values nested
1,000 or 100,000 levels deep show no difference for any limit. The default
build does the same work as before: the instructions retired for copy and conversion
(the walks whose guard changed; net of loading, canada, citm, twitter, jeopardy
and deep values), ==, dump and hash differ from the base by at most 0.02 % for copy
and 0.71 % in either direction for the others (noise), and the cycles for copy
by at most 0.65 %.

Tests: the existing tests that depended on 128 (comparison, allocator, custom
base class, diagnostics, binary writers, patch, merge_diff, conversions, custom
array type) derive their depths from the limit, and two assumed a limit of at
least 1 (limit - 1 wrapped around). tests/src/unit-recursion_depth_limit.cpp
checks that the macro is honoured (the array type of a value counts the levels
that a copy and a comparison descend) and that all walks return the same results
for values nested 1 to 300 levels deep as arrays, objects and both, against
expected strings and bytes built independently of the library; CMake builds it
with limits 0, 1 and 254 besides the default, and the CI target
ci_test_recursion_depth_limit_0 runs the whole suite with a limit of 0. A test
with a base class that copies the value from its copy constructor covers the
wrap-around at 254. The new test does not compile against the
previous header with -DJSON_RECURSION_DEPTH_LIMIT=1.

Docs: new macro page (range, stack and speed tables, small-stack guidance, ODR),
index, nav, macro overview, "Nesting depth" of the untrusted-input page (which
claimed that no walk could exhaust the stack however deep a value is nested; the
default assumes a stack of a few hundred KiB), the assurance case, and the page
of JSON_NO_THREAD_LOCAL.

No breaking changes: the default is unchanged, and the macro is new.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-11 18:32:55 +01:00

509 lines
16 KiB
C++

// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
using nlohmann::json;
#ifdef JSON_TEST_NO_GLOBAL_UDLS
using namespace nlohmann::literals; // NOLINT(google-build-using-namespace)
#endif
#include <string>
namespace
{
// RFC 7396's MergePatch, written recursively as in the RFC; only usable on
// values nested a few hundred levels deep
void reference_merge_patch(json& target, const json& patch)
{
if (!patch.is_object())
{
target = patch;
return;
}
if (!target.is_object())
{
target = json::object();
}
for (auto it = patch.begin(); it != patch.end(); ++it)
{
if (it.value().is_null())
{
target.erase(it.key());
}
else
{
reference_merge_patch(target[it.key()], it.value());
}
}
}
// objects nested `depth` levels deep under the key "a", with members that
// differ by `variant` on the way down
std::string nested_objects(const std::size_t depth, const int variant)
{
std::string text;
for (std::size_t i = 0; i < depth; ++i)
{
text += "{";
if ((i + static_cast<std::size_t>(variant)) % 3 == 0)
{
text += "\"s" + std::to_string(variant) + "\":" + std::to_string(i) + ",";
}
if (variant == 2 && i % 5 == 0)
{
text += "\"s0\":null,";
}
text += "\"a\":";
}
text += variant == 1 ? R"({"x":1,"y":null})" : "{\"y\":2}";
text.append(depth, '}');
return text;
}
} // namespace
TEST_CASE("JSON Merge Patch")
{
SECTION("examples from RFC 7396")
{
SECTION("Section 1")
{
json document = R"({
"a": "b",
"c": {
"d": "e",
"f": "g"
}
})"_json;
json const patch = R"({
"a": "z",
"c": {
"f": null
}
})"_json;
json expected = R"({
"a": "z",
"c": {
"d": "e"
}
})"_json;
document.merge_patch(patch);
CHECK(document == expected);
}
SECTION("Section 3")
{
json document = R"({
"title": "Goodbye!",
"author": {
"givenName": "John",
"familyName": "Doe"
},
"tags": [
"example",
"sample"
],
"content": "This will be unchanged"
})"_json;
json const patch = R"({
"title": "Hello!",
"phoneNumber": "+01-123-456-7890",
"author": {
"familyName": null
},
"tags": [
"example"
]
})"_json;
json expected = R"({
"title": "Hello!",
"author": {
"givenName": "John"
},
"tags": [
"example"
],
"content": "This will be unchanged",
"phoneNumber": "+01-123-456-7890"
})"_json;
document.merge_patch(patch);
CHECK(document == expected);
}
SECTION("Appendix A")
{
SECTION("Example 1")
{
json original = R"({"a":"b"})"_json;
json const patch = R"({"a":"c"})"_json;
json result = R"({"a":"c"})"_json;
original.merge_patch(patch);
CHECK(original == result);
}
SECTION("Example 2")
{
json original = R"({"a":"b"})"_json;
json const patch = R"({"b":"c"})"_json;
json result = R"({"a":"b", "b":"c"})"_json;
original.merge_patch(patch);
CHECK(original == result);
}
SECTION("Example 3")
{
json original = R"({"a":"b"})"_json;
json const patch = R"({"a":null})"_json;
json result = R"({})"_json;
original.merge_patch(patch);
CHECK(original == result);
}
SECTION("Example 4")
{
json original = R"({"a":"b","b":"c"})"_json;
json const patch = R"({"a":null})"_json;
json result = R"({"b":"c"})"_json;
original.merge_patch(patch);
CHECK(original == result);
}
SECTION("Example 5")
{
json original = R"({"a":["b"]})"_json;
json const patch = R"({"a":"c"})"_json;
json result = R"({"a":"c"})"_json;
original.merge_patch(patch);
CHECK(original == result);
}
SECTION("Example 6")
{
json original = R"({"a":"c"})"_json;
json const patch = R"({"a":["b"]})"_json;
json result = R"({"a":["b"]})"_json;
original.merge_patch(patch);
CHECK(original == result);
}
SECTION("Example 7")
{
json original = R"({"a":{"b": "c"}})"_json;
json const patch = R"({"a":{"b":"d","c":null}})"_json;
json result = R"({"a": {"b": "d"}})"_json;
original.merge_patch(patch);
CHECK(original == result);
}
SECTION("Example 8")
{
json original = R"({"a":[{"b":"c"}]})"_json;
json const patch = R"({"a":[1]})"_json;
json result = R"({"a":[1]})"_json;
original.merge_patch(patch);
CHECK(original == result);
}
SECTION("Example 9")
{
json original = R"(["a","b"])"_json;
json const patch = R"(["c","d"])"_json;
json result = R"(["c","d"])"_json;
original.merge_patch(patch);
CHECK(original == result);
}
SECTION("Example 10")
{
json original = R"({"a":"b"})"_json;
json const patch = R"(["c"])"_json;
json result = R"(["c"])"_json;
original.merge_patch(patch);
CHECK(original == result);
}
SECTION("Example 11")
{
json original = R"({"a":"foo"})"_json;
json const patch = R"(null)"_json;
json result = R"(null)"_json;
original.merge_patch(patch);
CHECK(original == result);
}
SECTION("Example 12")
{
json original = R"({"a":"foo"})"_json;
json const patch = R"("bar")"_json;
json result = R"("bar")"_json;
original.merge_patch(patch);
CHECK(original == result);
}
SECTION("Example 13")
{
json original = R"({"e":null})"_json;
json const patch = R"({"a":1})"_json;
json result = R"({"e":null,"a":1})"_json;
original.merge_patch(patch);
CHECK(original == result);
}
SECTION("Example 14")
{
json original = R"([1,2])"_json;
json const patch = R"({"a":"b","c":null})"_json;
json result = R"({"a":"b"})"_json;
original.merge_patch(patch);
CHECK(original == result);
}
SECTION("Example 15")
{
json original = R"({})"_json;
json const patch = R"({"a":{"bb":{"ccc":null}}})"_json;
json result = R"({"a":{"bb":{}}})"_json;
original.merge_patch(patch);
CHECK(original == result);
}
}
}
}
TEST_CASE("JSON Merge Patch on deeply nested values")
{
SECTION("patching past the descent bound gives the same result")
{
// every depth on either side of where the iterative version takes
// over (detail::recursion_depth_limit(), 128 by default)
for (std::size_t depth = 0; depth <= 300; ++depth)
{
CAPTURE(depth)
for (int variant = 0; variant < 3; ++variant)
{
CAPTURE(variant)
const json patch = json::parse(nested_objects(depth, variant));
json result = json::parse(nested_objects(depth, (variant + 1) % 3));
json expected = result;
result.merge_patch(patch);
reference_merge_patch(expected, patch);
CHECK(result == expected);
// a target that is not an object, and an empty one
json from_null;
from_null.merge_patch(patch);
json expected_from_null;
reference_merge_patch(expected_from_null, patch);
CHECK(from_null == expected_from_null);
}
}
}
SECTION("patches nested too deeply for the call stack (#5393)")
{
// applying a patch used to recurse once per nesting level. The result
// is only walked, never copied or compared, since those recurse too.
const std::size_t depth = 100000;
json target = json::parse(nested_objects(depth, 0));
target.merge_patch(json::parse(nested_objects(depth, 1)));
const json* p = &target;
for (std::size_t i = 0; i < depth; ++i)
{
p = &p->at("a");
}
// {"y":2} patched with {"x":1,"y":null}
CHECK(p->size() == 1);
CHECK(p->at("x") == 1);
}
}
TEST_CASE("JSON Merge Patch and update on ordered_json")
{
using nlohmann::ordered_json;
SECTION("merge_patch")
{
ordered_json target = ordered_json::parse(R"({"a": {"b": 1, "c": 2}, "d": 3, "e": [1]})");
target.merge_patch(ordered_json::parse(R"({"a": {"b": null, "f": 4}, "d": {"x": {"y": null}}, "e": null, "g": {"h": 5}})"));
CHECK(target == ordered_json::parse(R"({"a": {"c": 2, "f": 4}, "d": {"x": {}}, "g": {"h": 5}})"));
// a patch that is not an object replaces the target
target.merge_patch(ordered_json({1, 2}));
CHECK(target == ordered_json({1, 2}));
// an object patch turns a target that is not an object into one
target.merge_patch(ordered_json::parse(R"({"k": {"l": null}})"));
CHECK(target == ordered_json::parse(R"({"k": {}})"));
}
SECTION("update with merge_objects")
{
ordered_json target = ordered_json::parse(R"({"a": {"b": 1, "c": {"d": 2}}, "e": 3})");
target.update(ordered_json::parse(R"({"a": {"c": {"x": 1}, "f": 4}, "e": {"y": 5}, "g": 6})"), true);
CHECK(target == ordered_json::parse(R"({"a": {"b": 1, "c": {"d": 2, "x": 1}, "f": 4}, "e": {"y": 5}, "g": 6})"));
target.update(ordered_json::parse(R"({"a": 1})"), false);
CHECK(target == ordered_json::parse(R"({"a": 1, "e": {"y": 5}, "g": 6})"));
}
}
TEST_CASE("merge_patch() with an argument that aliases *this (#5641)")
{
SECTION("j.merge_patch(j): erasing a member destroys the node the loop's iterator points to")
{
// reproduces issue #5641, case 1
json j = {{"a", nullptr}, {"b", 1}};
j.merge_patch(j);
CHECK(j == json({{"b", 1}}));
}
SECTION("j.merge_patch(j[\"a\"]): removing \"a\" destroys the patch while it is iterated")
{
// reproduces issue #5641, case 2
json j = {{"a", {{"a", nullptr}, {"b", 2}}}};
j.merge_patch(j["a"]);
CHECK(j == json({{"b", 2}}));
}
SECTION("a patch nested past the iterative descent bound aliases *this")
{
// every depth on either side of where the iterative version takes
// over (detail::recursion_depth_limit(), 128 by default); patching *this with
// itself is idempotent, aliased or not
for (const std::size_t depth :
{
std::size_t{0}, std::size_t{127}, std::size_t{128}, std::size_t{300}
})
{
CAPTURE(depth)
json j = json::parse(nested_objects(depth, 0));
const json expected = j;
j.merge_patch(j);
CHECK(j == expected);
}
}
SECTION("ordered_json")
{
using nlohmann::ordered_json;
SECTION("merge_patch with a member of *this")
{
ordered_json j = {{"a", {{"a", nullptr}, {"b", 2}}}};
j.merge_patch(j["a"]);
CHECK(j == ordered_json({{"b", 2}}));
}
}
}
TEST_CASE("JSON Merge Patch on non-object, discarded, and binary values")
{
SECTION("an empty object patch turns a non-object target into an empty object")
{
json target = 1;
target.merge_patch(json::object());
CHECK(target == json::object());
json from_null;
from_null.merge_patch(json::object());
CHECK(from_null == json::object());
}
SECTION("a non-object patch replaces a deeply nested target")
{
// checked by type, since == recurses
SECTION("number")
{
json target = json::parse(nested_objects(1000, 0));
target.merge_patch(json(42));
CHECK(target.is_number_integer());
CHECK(target.get<int>() == 42);
}
SECTION("array")
{
json target = json::parse(nested_objects(1000, 0));
target.merge_patch(json({1, nullptr}));
CHECK(target.is_array());
CHECK(target.size() == 2);
CHECK(target[1].is_null());
}
SECTION("null")
{
json target = json::parse(nested_objects(1000, 0));
target.merge_patch(json(nullptr));
CHECK(target.is_null());
}
}
SECTION("discarded values")
{
const json discarded(json::value_t::discarded);
// a discarded patch is not an object and replaces the target
json target = {{"a", 1}};
target.merge_patch(discarded);
CHECK(target.is_discarded());
// a discarded target is replaced by an empty object first
json discarded_target(json::value_t::discarded);
discarded_target.merge_patch(json({{"a", 1}}));
CHECK(discarded_target == json({{"a", 1}}));
json both(json::value_t::discarded);
both.merge_patch(discarded);
CHECK(both.is_discarded());
}
SECTION("binary values")
{
const json bin = json::binary({1, 2, 3}, 7);
// a binary patch is not an object and replaces the target
json target = {{"a", 1}};
target.merge_patch(bin);
CHECK(target.is_binary());
CHECK(target.get_binary() == bin.get_binary());
// a binary target is replaced by an object
json binary_target = bin;
binary_target.merge_patch(json({{"a", 1}}));
CHECK(binary_target == json({{"a", 1}}));
// binary values nested in the patch are kept
json nested = json::object();
nested.merge_patch(json({{"a", bin}}));
CHECK(nested["a"].is_binary());
CHECK(nested["a"].get_binary() == bin.get_binary());
}
}