mirror of
https://github.com/nlohmann/json.git
synced 2026-11-14 03:18:20 +00:00
Copying, comparing, serializing, hashing, merging, diffing, converting and destroying a value, and the CBOR, MessagePack, UBJSON, BJData and BON8 writers recurse a fixed number of nesting levels on the call stack and finish deeper levels with an explicit stack (#5387). The number was hard-coded to 128. That is too much for small stacks: a 128-level dump() needs about 288 KiB at -O0 with Clang (2.3 KiB per level; 256 KiB with GCC) and crashed in a thread with a 256 KiB stack, and musl threads have 128 KiB by default. Others may want the opposite. The macro JSON_RECURSION_DEPTH_LIMIT, defined before including the library, sets the number; the default stays 128. The value must be an integer constant from 0 to 254 (not a floating-point number or bool), otherwise a static_assert names the macro. 254 is the largest value because the copy constructor and the comparison operators cannot pass a depth down and count it in a byte per thread (nesting_depth()). The guard that counts now counts only the levels it enters, so the count never exceeds the limit. It used to count the level that finds the limit reached as well, which made the count 255 at limit 254: a copy made by a custom base class while a copy was going on then wrapped the byte to 0 and recursed another 254 levels (twice the stack). 0 is allowed: every walk then uses its explicit stack from the first level on and needs a constant amount of stack (16 KiB or less at -O0 with Clang and GCC, for a value nested 20,000 levels deep). All uses of the limit compare a depth with it; none computes limit - 1 or limit + 1, and no walk has a constant 128 of its own. The macro changes only stack use and speed, never a result. Like JSON_NO_THREAD_LOCAL it is not part of the ABI tag and must be the same in all translation units (ODR); the page of the macro says what happens otherwise. Supporting a limit of 0 needed one fix: update() without merge_objects reached the explicit-stack merge at depth 0, which always merges objects. update_members now takes that path only with merge_objects, which is the only case in which it recurses at all. Measurements (macOS arm64, Apple clang 21 and g++ 16, documented in the macro page with a table per operation and optimization level, and a program to reproduce them): the stack needed at the default of 128 levels is up to 288 KiB (dump at -O0, Clang), 224 KiB (copy), and 48 KiB or less at -O2; at 254 levels up to 576 KiB at -O0 and 112 KiB at -O2. Speed at -O2 relative to the default: ordinary documents (nested 2 to 10 levels) are 1.1 to 2.4 times as slow with a limit of 0 or 1 and unchanged (within about 0.1) with 16 or 254; values nested 1,000 or 100,000 levels deep show no difference for any limit. The default build does the same work as before: the instructions retired for copy and conversion (the walks whose guard changed; net of loading, canada, citm, twitter, jeopardy and deep values), ==, dump and hash differ from the base by at most 0.02 % for copy and 0.71 % in either direction for the others (noise), and the cycles for copy by at most 0.65 %. Tests: the existing tests that depended on 128 (comparison, allocator, custom base class, diagnostics, binary writers, patch, merge_diff, conversions, custom array type) derive their depths from the limit, and two assumed a limit of at least 1 (limit - 1 wrapped around). tests/src/unit-recursion_depth_limit.cpp checks that the macro is honoured (the array type of a value counts the levels that a copy and a comparison descend) and that all walks return the same results for values nested 1 to 300 levels deep as arrays, objects and both, against expected strings and bytes built independently of the library; CMake builds it with limits 0, 1 and 254 besides the default, and the CI target ci_test_recursion_depth_limit_0 runs the whole suite with a limit of 0. A test with a base class that copies the value from its copy constructor covers the wrap-around at 254. The new test does not compile against the previous header with -DJSON_RECURSION_DEPTH_LIMIT=1. Docs: new macro page (range, stack and speed tables, small-stack guidance, ODR), index, nav, macro overview, "Nesting depth" of the untrusted-input page (which claimed that no walk could exhaust the stack however deep a value is nested; the default assumes a stack of a few hundred KiB), the assurance case, and the page of JSON_NO_THREAD_LOCAL. No breaking changes: the default is unchanged, and the macro is new. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
509 lines
16 KiB
C++
509 lines
16 KiB
C++
// __ _____ _____ _____
|
|
// __| | __| | | | JSON for Modern C++ (supporting code)
|
|
// | | |__ | | | | | | version 3.12.0
|
|
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
|
|
//
|
|
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
|
|
// SPDX-License-Identifier: MIT
|
|
|
|
#include "doctest_compatibility.h"
|
|
|
|
#include <nlohmann/json.hpp>
|
|
using nlohmann::json;
|
|
#ifdef JSON_TEST_NO_GLOBAL_UDLS
|
|
using namespace nlohmann::literals; // NOLINT(google-build-using-namespace)
|
|
#endif
|
|
|
|
#include <string>
|
|
|
|
namespace
|
|
{
|
|
// RFC 7396's MergePatch, written recursively as in the RFC; only usable on
|
|
// values nested a few hundred levels deep
|
|
void reference_merge_patch(json& target, const json& patch)
|
|
{
|
|
if (!patch.is_object())
|
|
{
|
|
target = patch;
|
|
return;
|
|
}
|
|
if (!target.is_object())
|
|
{
|
|
target = json::object();
|
|
}
|
|
for (auto it = patch.begin(); it != patch.end(); ++it)
|
|
{
|
|
if (it.value().is_null())
|
|
{
|
|
target.erase(it.key());
|
|
}
|
|
else
|
|
{
|
|
reference_merge_patch(target[it.key()], it.value());
|
|
}
|
|
}
|
|
}
|
|
|
|
// objects nested `depth` levels deep under the key "a", with members that
|
|
// differ by `variant` on the way down
|
|
std::string nested_objects(const std::size_t depth, const int variant)
|
|
{
|
|
std::string text;
|
|
for (std::size_t i = 0; i < depth; ++i)
|
|
{
|
|
text += "{";
|
|
if ((i + static_cast<std::size_t>(variant)) % 3 == 0)
|
|
{
|
|
text += "\"s" + std::to_string(variant) + "\":" + std::to_string(i) + ",";
|
|
}
|
|
if (variant == 2 && i % 5 == 0)
|
|
{
|
|
text += "\"s0\":null,";
|
|
}
|
|
text += "\"a\":";
|
|
}
|
|
text += variant == 1 ? R"({"x":1,"y":null})" : "{\"y\":2}";
|
|
text.append(depth, '}');
|
|
return text;
|
|
}
|
|
} // namespace
|
|
|
|
TEST_CASE("JSON Merge Patch")
|
|
{
|
|
SECTION("examples from RFC 7396")
|
|
{
|
|
SECTION("Section 1")
|
|
{
|
|
json document = R"({
|
|
"a": "b",
|
|
"c": {
|
|
"d": "e",
|
|
"f": "g"
|
|
}
|
|
})"_json;
|
|
|
|
json const patch = R"({
|
|
"a": "z",
|
|
"c": {
|
|
"f": null
|
|
}
|
|
})"_json;
|
|
|
|
json expected = R"({
|
|
"a": "z",
|
|
"c": {
|
|
"d": "e"
|
|
}
|
|
})"_json;
|
|
|
|
document.merge_patch(patch);
|
|
CHECK(document == expected);
|
|
}
|
|
|
|
SECTION("Section 3")
|
|
{
|
|
json document = R"({
|
|
"title": "Goodbye!",
|
|
"author": {
|
|
"givenName": "John",
|
|
"familyName": "Doe"
|
|
},
|
|
"tags": [
|
|
"example",
|
|
"sample"
|
|
],
|
|
"content": "This will be unchanged"
|
|
})"_json;
|
|
|
|
json const patch = R"({
|
|
"title": "Hello!",
|
|
"phoneNumber": "+01-123-456-7890",
|
|
"author": {
|
|
"familyName": null
|
|
},
|
|
"tags": [
|
|
"example"
|
|
]
|
|
})"_json;
|
|
|
|
json expected = R"({
|
|
"title": "Hello!",
|
|
"author": {
|
|
"givenName": "John"
|
|
},
|
|
"tags": [
|
|
"example"
|
|
],
|
|
"content": "This will be unchanged",
|
|
"phoneNumber": "+01-123-456-7890"
|
|
})"_json;
|
|
|
|
document.merge_patch(patch);
|
|
CHECK(document == expected);
|
|
}
|
|
|
|
SECTION("Appendix A")
|
|
{
|
|
SECTION("Example 1")
|
|
{
|
|
json original = R"({"a":"b"})"_json;
|
|
json const patch = R"({"a":"c"})"_json;
|
|
json result = R"({"a":"c"})"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
|
|
SECTION("Example 2")
|
|
{
|
|
json original = R"({"a":"b"})"_json;
|
|
json const patch = R"({"b":"c"})"_json;
|
|
json result = R"({"a":"b", "b":"c"})"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
|
|
SECTION("Example 3")
|
|
{
|
|
json original = R"({"a":"b"})"_json;
|
|
json const patch = R"({"a":null})"_json;
|
|
json result = R"({})"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
|
|
SECTION("Example 4")
|
|
{
|
|
json original = R"({"a":"b","b":"c"})"_json;
|
|
json const patch = R"({"a":null})"_json;
|
|
json result = R"({"b":"c"})"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
|
|
SECTION("Example 5")
|
|
{
|
|
json original = R"({"a":["b"]})"_json;
|
|
json const patch = R"({"a":"c"})"_json;
|
|
json result = R"({"a":"c"})"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
|
|
SECTION("Example 6")
|
|
{
|
|
json original = R"({"a":"c"})"_json;
|
|
json const patch = R"({"a":["b"]})"_json;
|
|
json result = R"({"a":["b"]})"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
|
|
SECTION("Example 7")
|
|
{
|
|
json original = R"({"a":{"b": "c"}})"_json;
|
|
json const patch = R"({"a":{"b":"d","c":null}})"_json;
|
|
json result = R"({"a": {"b": "d"}})"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
|
|
SECTION("Example 8")
|
|
{
|
|
json original = R"({"a":[{"b":"c"}]})"_json;
|
|
json const patch = R"({"a":[1]})"_json;
|
|
json result = R"({"a":[1]})"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
|
|
SECTION("Example 9")
|
|
{
|
|
json original = R"(["a","b"])"_json;
|
|
json const patch = R"(["c","d"])"_json;
|
|
json result = R"(["c","d"])"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
|
|
SECTION("Example 10")
|
|
{
|
|
json original = R"({"a":"b"})"_json;
|
|
json const patch = R"(["c"])"_json;
|
|
json result = R"(["c"])"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
|
|
SECTION("Example 11")
|
|
{
|
|
json original = R"({"a":"foo"})"_json;
|
|
json const patch = R"(null)"_json;
|
|
json result = R"(null)"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
|
|
SECTION("Example 12")
|
|
{
|
|
json original = R"({"a":"foo"})"_json;
|
|
json const patch = R"("bar")"_json;
|
|
json result = R"("bar")"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
|
|
SECTION("Example 13")
|
|
{
|
|
json original = R"({"e":null})"_json;
|
|
json const patch = R"({"a":1})"_json;
|
|
json result = R"({"e":null,"a":1})"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
|
|
SECTION("Example 14")
|
|
{
|
|
json original = R"([1,2])"_json;
|
|
json const patch = R"({"a":"b","c":null})"_json;
|
|
json result = R"({"a":"b"})"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
|
|
SECTION("Example 15")
|
|
{
|
|
json original = R"({})"_json;
|
|
json const patch = R"({"a":{"bb":{"ccc":null}}})"_json;
|
|
json result = R"({"a":{"bb":{}}})"_json;
|
|
|
|
original.merge_patch(patch);
|
|
CHECK(original == result);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
TEST_CASE("JSON Merge Patch on deeply nested values")
|
|
{
|
|
SECTION("patching past the descent bound gives the same result")
|
|
{
|
|
// every depth on either side of where the iterative version takes
|
|
// over (detail::recursion_depth_limit(), 128 by default)
|
|
for (std::size_t depth = 0; depth <= 300; ++depth)
|
|
{
|
|
CAPTURE(depth)
|
|
for (int variant = 0; variant < 3; ++variant)
|
|
{
|
|
CAPTURE(variant)
|
|
const json patch = json::parse(nested_objects(depth, variant));
|
|
|
|
json result = json::parse(nested_objects(depth, (variant + 1) % 3));
|
|
json expected = result;
|
|
result.merge_patch(patch);
|
|
reference_merge_patch(expected, patch);
|
|
CHECK(result == expected);
|
|
|
|
// a target that is not an object, and an empty one
|
|
json from_null;
|
|
from_null.merge_patch(patch);
|
|
json expected_from_null;
|
|
reference_merge_patch(expected_from_null, patch);
|
|
CHECK(from_null == expected_from_null);
|
|
}
|
|
}
|
|
}
|
|
|
|
SECTION("patches nested too deeply for the call stack (#5393)")
|
|
{
|
|
// applying a patch used to recurse once per nesting level. The result
|
|
// is only walked, never copied or compared, since those recurse too.
|
|
const std::size_t depth = 100000;
|
|
json target = json::parse(nested_objects(depth, 0));
|
|
target.merge_patch(json::parse(nested_objects(depth, 1)));
|
|
|
|
const json* p = ⌖
|
|
for (std::size_t i = 0; i < depth; ++i)
|
|
{
|
|
p = &p->at("a");
|
|
}
|
|
// {"y":2} patched with {"x":1,"y":null}
|
|
CHECK(p->size() == 1);
|
|
CHECK(p->at("x") == 1);
|
|
}
|
|
}
|
|
|
|
TEST_CASE("JSON Merge Patch and update on ordered_json")
|
|
{
|
|
using nlohmann::ordered_json;
|
|
|
|
SECTION("merge_patch")
|
|
{
|
|
ordered_json target = ordered_json::parse(R"({"a": {"b": 1, "c": 2}, "d": 3, "e": [1]})");
|
|
target.merge_patch(ordered_json::parse(R"({"a": {"b": null, "f": 4}, "d": {"x": {"y": null}}, "e": null, "g": {"h": 5}})"));
|
|
CHECK(target == ordered_json::parse(R"({"a": {"c": 2, "f": 4}, "d": {"x": {}}, "g": {"h": 5}})"));
|
|
|
|
// a patch that is not an object replaces the target
|
|
target.merge_patch(ordered_json({1, 2}));
|
|
CHECK(target == ordered_json({1, 2}));
|
|
// an object patch turns a target that is not an object into one
|
|
target.merge_patch(ordered_json::parse(R"({"k": {"l": null}})"));
|
|
CHECK(target == ordered_json::parse(R"({"k": {}})"));
|
|
}
|
|
|
|
SECTION("update with merge_objects")
|
|
{
|
|
ordered_json target = ordered_json::parse(R"({"a": {"b": 1, "c": {"d": 2}}, "e": 3})");
|
|
target.update(ordered_json::parse(R"({"a": {"c": {"x": 1}, "f": 4}, "e": {"y": 5}, "g": 6})"), true);
|
|
CHECK(target == ordered_json::parse(R"({"a": {"b": 1, "c": {"d": 2, "x": 1}, "f": 4}, "e": {"y": 5}, "g": 6})"));
|
|
|
|
target.update(ordered_json::parse(R"({"a": 1})"), false);
|
|
CHECK(target == ordered_json::parse(R"({"a": 1, "e": {"y": 5}, "g": 6})"));
|
|
}
|
|
}
|
|
|
|
TEST_CASE("merge_patch() with an argument that aliases *this (#5641)")
|
|
{
|
|
SECTION("j.merge_patch(j): erasing a member destroys the node the loop's iterator points to")
|
|
{
|
|
// reproduces issue #5641, case 1
|
|
json j = {{"a", nullptr}, {"b", 1}};
|
|
j.merge_patch(j);
|
|
CHECK(j == json({{"b", 1}}));
|
|
}
|
|
|
|
SECTION("j.merge_patch(j[\"a\"]): removing \"a\" destroys the patch while it is iterated")
|
|
{
|
|
// reproduces issue #5641, case 2
|
|
json j = {{"a", {{"a", nullptr}, {"b", 2}}}};
|
|
j.merge_patch(j["a"]);
|
|
CHECK(j == json({{"b", 2}}));
|
|
}
|
|
|
|
SECTION("a patch nested past the iterative descent bound aliases *this")
|
|
{
|
|
// every depth on either side of where the iterative version takes
|
|
// over (detail::recursion_depth_limit(), 128 by default); patching *this with
|
|
// itself is idempotent, aliased or not
|
|
for (const std::size_t depth :
|
|
{
|
|
std::size_t{0}, std::size_t{127}, std::size_t{128}, std::size_t{300}
|
|
})
|
|
{
|
|
CAPTURE(depth)
|
|
json j = json::parse(nested_objects(depth, 0));
|
|
const json expected = j;
|
|
j.merge_patch(j);
|
|
CHECK(j == expected);
|
|
}
|
|
}
|
|
|
|
SECTION("ordered_json")
|
|
{
|
|
using nlohmann::ordered_json;
|
|
|
|
SECTION("merge_patch with a member of *this")
|
|
{
|
|
ordered_json j = {{"a", {{"a", nullptr}, {"b", 2}}}};
|
|
j.merge_patch(j["a"]);
|
|
CHECK(j == ordered_json({{"b", 2}}));
|
|
}
|
|
}
|
|
}
|
|
|
|
TEST_CASE("JSON Merge Patch on non-object, discarded, and binary values")
|
|
{
|
|
SECTION("an empty object patch turns a non-object target into an empty object")
|
|
{
|
|
json target = 1;
|
|
target.merge_patch(json::object());
|
|
CHECK(target == json::object());
|
|
|
|
json from_null;
|
|
from_null.merge_patch(json::object());
|
|
CHECK(from_null == json::object());
|
|
}
|
|
|
|
SECTION("a non-object patch replaces a deeply nested target")
|
|
{
|
|
// checked by type, since == recurses
|
|
SECTION("number")
|
|
{
|
|
json target = json::parse(nested_objects(1000, 0));
|
|
target.merge_patch(json(42));
|
|
CHECK(target.is_number_integer());
|
|
CHECK(target.get<int>() == 42);
|
|
}
|
|
|
|
SECTION("array")
|
|
{
|
|
json target = json::parse(nested_objects(1000, 0));
|
|
target.merge_patch(json({1, nullptr}));
|
|
CHECK(target.is_array());
|
|
CHECK(target.size() == 2);
|
|
CHECK(target[1].is_null());
|
|
}
|
|
|
|
SECTION("null")
|
|
{
|
|
json target = json::parse(nested_objects(1000, 0));
|
|
target.merge_patch(json(nullptr));
|
|
CHECK(target.is_null());
|
|
}
|
|
}
|
|
|
|
SECTION("discarded values")
|
|
{
|
|
const json discarded(json::value_t::discarded);
|
|
|
|
// a discarded patch is not an object and replaces the target
|
|
json target = {{"a", 1}};
|
|
target.merge_patch(discarded);
|
|
CHECK(target.is_discarded());
|
|
|
|
// a discarded target is replaced by an empty object first
|
|
json discarded_target(json::value_t::discarded);
|
|
discarded_target.merge_patch(json({{"a", 1}}));
|
|
CHECK(discarded_target == json({{"a", 1}}));
|
|
|
|
json both(json::value_t::discarded);
|
|
both.merge_patch(discarded);
|
|
CHECK(both.is_discarded());
|
|
}
|
|
|
|
SECTION("binary values")
|
|
{
|
|
const json bin = json::binary({1, 2, 3}, 7);
|
|
|
|
// a binary patch is not an object and replaces the target
|
|
json target = {{"a", 1}};
|
|
target.merge_patch(bin);
|
|
CHECK(target.is_binary());
|
|
CHECK(target.get_binary() == bin.get_binary());
|
|
|
|
// a binary target is replaced by an object
|
|
json binary_target = bin;
|
|
binary_target.merge_patch(json({{"a", 1}}));
|
|
CHECK(binary_target == json({{"a", 1}}));
|
|
|
|
// binary values nested in the patch are kept
|
|
json nested = json::object();
|
|
nested.merge_patch(json({{"a", bin}}));
|
|
CHECK(nested["a"].is_binary());
|
|
CHECK(nested["a"].get_binary() == bin.get_binary());
|
|
}
|
|
}
|