diff() descended into both values once per nesting level, and compared
them with operator== on every level on the way, which recurses as well.
Values nested deeply enough - 25,000 levels on an 8 MiB stack - exhausted
the call stack and terminated the process, although parse() accepts
them without complaint. On such a chain the per-level comparisons and
path strings also made diff() quadratic in time and memory.
Both the recursion and operator== only descend as far as the source is
nested. So diff() first checks, recursing at most diff_depth_limit()
(128) levels, whether the source is nested more deeply than that. If not
- all but a vanishing minority of values - the recursive algorithm
diffs it exactly as before, now as diff_recursively(). Otherwise
diff_iteratively() walks the two values on an explicit stack, emitting
the same operations in the same order. It does not compare arrays and
objects with operator== up front (equal ones yield no operations
anyway), keeps the path in one buffer instead of a new string per
level, and hands every subtree that is not nested too deeply back to
diff_recursively(), so equal parts are still skipped quickly.
The check costs one pass over the source. On a 3,000-object document
that is about 30% of diffing two equal values (which is just an
operator== call), about 10% of diffing values that differ in a few
places, and noise when arrays change length. Once operator== no longer
recurses (#5390), the check can go.
Tests check that the patch reproduces the target at every depth up to
300, for json and ordered_json, including reordered members. They also
check the exact operation for a difference deep inside, and diff values
nested 100,000 levels deep.
Fixes#5393 for diff().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: cover JSON_NO_IO, JSON_THROW/TRY/CATCH_USER, JSON_SKIP_LIBRARY_VERSION_CHECK, and JSON_DisableEnumSerialization in CI (#5423)
These four supported configuration macros were never actually compiled
anywhere in the test matrix:
- JSON_NO_IO and the JSON_THROW_USER/JSON_TRY_USER/JSON_CATCH_USER trio
are exercised together in a new tests/src/unit-no_io_and_user_exceptions.cpp,
which is automatically picked up by the existing unit-*.cpp test glob and
thus built across the whole standard test matrix.
- JSON_SKIP_LIBRARY_VERSION_CHECK is exercised by a new, dedicated
tests/src/skip_library_version_check.cpp, compiled directly by the new
ci_test_skiplibraryversioncheck target in cmake/ci.cmake: the scenario it
simulates (mixing two differently-versioned inclusions of the library)
unavoidably triggers the compiler's own "macro redefined" warning, which
would fail under the library's own -Weverything/-Werror unit test matrix
for a reason unrelated to the macro under test.
- JSON_DisableEnumSerialization already had #if-guarded tests in several
unit-*.cpp files (from #4384), but no CMake target ever actually set the
JSON_DisableEnumSerialization CMake option, so that guarded code was never
compiled. Add ci_test_disableenumserialization, mirroring the existing
ci_test_noimplicitconversions/ci_test_noglobaludls targets. Building the
full test suite with this option on surfaced one real, narrow gap: get<T>()
on std::vector<std::byte> (used by unit-regression2.cpp's custom BinaryType
tests) relies on std::byte being handled via enum serialization, so add the
same #if-guard convention to the two affected SECTIONs there.
Both new CI targets are added to the ci_cmake_options matrix in
.github/workflows/ubuntu.yml, alongside the existing ci_test_* targets.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: cover multi-digit widths and bare alignment in std::formatter<json> (#5423)
Every existing std::formatter spec with a width used a single digit (e.g.
"{:2}"), so the width-parsing loop's accumulation of a second/third digit was
never exercised; add multi-digit width cases. Likewise, every existing spec
with an alignment character also had an explicit fill character, so the
bare-alignment branch (e.g. "{:<}", with no fill) was never exercised; add
cases asserting it keeps the default space indent character.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: add coverage for patch_inplace() (#5423)
patch_inplace() had no unit test at all. Add a happy-path case mirroring an
existing patch() example, and -- more importantly -- pin its distinguishing
contract versus patch(): when a multi-operation JSON Patch fails partway
through, patch_inplace() (which mutates the document directly, operation by
operation) leaves whatever operations already succeeded applied, whereas
patch() (which applies the patch to an internal copy that is discarded on
exception) leaves the original completely untouched either way. Verified
empirically against the current implementation before writing the assertions.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: fix duplicate TEST_CASE name in unit-no-mem-leak-on-adl-serialize.cpp (#5423)
Two distinct TEST_CASEs were both named "check_for_mem_leak_on_adl_to_json-2".
doctest allows duplicate names, so both still ran, but it makes
--test-case=<name> filtering and reporting ambiguous. Rename the second one
to "-3", continuing the existing "-1"/"-2" sequence.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: add direct coverage for the std::u8string to_json overload (#5423)
The ADL to_json overload for std::basic_string<char8_t, ...> was only ever
reached indirectly, via std::filesystem::path::u8string(). Add a test that
constructs a json value directly from a std::u8string, gated the same way as
the overload itself (include/nlohmann/detail/conversions/to_json.hpp): behind
both the std::filesystem::path feature guard and __cpp_lib_char8_t, since the
overload only exists when both are satisfied.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: verify move semantics of byte_container_with_subtype's rvalue constructors (#5423)
The two rvalue-reference constructors were never distinguished from their
const-lvalue-reference twins by any test. Add a "move semantics" section that
constructs from an rvalue std::vector, checks the resulting container keeps
the exact same buffer address as the source (a stronger check than just
observing the source ended up empty, since a copy-then-clear could do that
too), and confirms the source vector was left empty.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Guard patch_inplace() partial-application test against JSON_NOEXCEPTION
The "distinguishing contract vs patch(): partial application on
failure" test relies on doc.patch_inplace(patch) actually throwing so
the partially-applied state can be observed right after the throw
point. Under ci_test_noexceptions, JSON_THROW() calls std::abort()
instead of throwing, and doctest's --no-throw test filter (which that
CI job passes) makes CHECK_THROWS_AS() a no-op that never even
evaluates its expression -- so patch_inplace() is never called and the
follow-up assertions fail against the untouched original document.
Guard the whole SECTION with #if !defined(JSON_NOEXCEPTION), following
the same convention already used elsewhere in the test suite (e.g.
unit-class_parser.cpp) for exception-dependent tests.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MSVC C2220 in the std::u8string conversion test
MSVC's C5321 ("nonstandard extension used: encoding '\xNN' as a
multi-byte utf-8 character") is promoted to a hard error by our MSVC CI
configs. It fires because the test composed a non-ASCII UTF-8 sequence
inside a u8"" literal using raw \x byte escapes; MSVC treats that as
nonstandard and suggests using \u universal-character-names instead,
which every compiler agrees on and which compiles down to the exact
same encoded bytes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Guard the JSON_THROW_USER test against JSON_NOEXCEPTION and GCC's -Wunused-result
Two independent CI configurations failed to build/run this new test:
- ci_test_noexceptions runs the whole suite with -DJSON_NOEXCEPTION and
doctest's "--no-throw" filter, which compiles CHECK_THROWS_AS() down
to a no-op that never even invokes the guarded expression. Since this
test's whole point is to observe json_throw_user_call_count after
json::parse()/at() actually throw, it can't be meaningfully run under
that filter (our JSON_THROW_USER override still throws real
exceptions regardless of JSON_NOEXCEPTION, but the assertion never
gets a chance to run). Guard the TEST_CASE with
#if !defined(JSON_NOEXCEPTION), mirroring the existing precedent in
unit-json_patch.cpp.
- ci_test_gcc and ci_test_standards_gcc(11) failed with
-Werror=unused-result on the discarded json::parse() return value.
json::parse() is marked warn_unused_result, and unlike a real
[[nodiscard]] attribute, GCC does not consider that satisfied by
doctest's (void)-cast around the expression in C++11 mode. Assign the
result to a discarded local instead, matching the established
`json _ = json::parse(...)` idiom already used throughout
unit-class_parser.cpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Suppress a clang-tidy false positive on an intentional defensive copy
performance-unnecessary-copy-initialization suggests copy_for_patch
could be a reference since it's never modified -- but the copy is the
point: it guards against a hypothetical regression where patch()
mutates its receiver, which a reference could never catch (the
follow-up assertion would just compare `original` to itself).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy findings in the JSON_NO_IO/JSON_THROW_USER test
- bugprone-macro-parentheses: wrap the JSON_THROW_USER macro argument in
parentheses at the throw site.
- modernize-raw-string-literal: switch two escaped JSON string literals to
raw string literals.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Reject JSON Patch move when from is a proper prefix of path
RFC 6902 (section 4.4) forbids "from" from being a proper prefix of
"path" for a "move" operation: "a location cannot be moved into one
of its children." "move" is implemented as remove-then-add with no
check for this. For object targets, the subsequent "add" happened to
throw as a side effect of resolving through the now-removed parent,
but for array targets, removing the "from" element shifts subsequent
indices, so "path" silently re-resolves to a different element and
the operation "succeeds" with a silently corrupted document.
Add a check, before performing the remove/add, for whether "from" is
a proper prefix of "path" at the reference-token level. This compares
json_pointer's already-unescaped reference_tokens vectors (basic_json
is a friend of json_pointer) rather than the raw pointer strings, so
that tokens containing escaped '/' or '~' characters are compared
correctly, and a token that merely looks like a string prefix (e.g.
"/ab" vs "/abc/x") is not mistaken for a pointer-token prefix. When
"from" is a proper prefix of "path", throw out_of_range.414.
Fixes#5397.
Stacked on top of the fix for #5396 (branch
issue-5396-patch-remove-primitive-parent), since both touch the same
patch_inplace move/remove handling in include/nlohmann/json.hpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add root-pointer and array-append-token edge case tests for the move prefix check
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Replace std::equal with an explicitly-bounded loop in the move prefix check
The three-iterator std::equal(first1, last1, first2) form has no
explicit end iterator for the second range, which a static analyzer
(Flawfinder, CWE-126) flags as a potential over-read even though the
preceding size comparison already guarantees the second range is long
enough. Rather than argue the point, make the bound visible in the code
itself via an explicit loop -- every access to ptr.reference_tokens is
now guarded by the same index the loop condition bounds against
from_size.
(The C++14 four-iterator std::equal(first1, last1, first2, last2) form
was tried first as a more minimal fix, but this codebase targets C++11
and that overload is not safely usable under -std=c++11 with all
supported standard library implementations.)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Extract the move prefix check into a named helper lambda
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Account for JSON_DIAGNOSTIC_POSITIONS in the move-prefix-check error messages
out_of_range::create() includes a "(bytes X-Y)" position annotation when
JSON_DIAGNOSTIC_POSITIONS is enabled, which the ci_test_diagnostic_positions
CI job builds the whole suite with. The five new out_of_range.414
assertions only checked the annotation-free message. Confirmed
JSON_DIAGNOSTICS produces the same (annotation-free) message as the
default build for this particular throw site (its path-based annotation
is empty at the root, where &result always points here), so only two
message variants are needed, not three.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
RFC 6902 (section 4.2) requires the target location of a "remove"
operation to exist. operation_remove handled parent.is_object() and
parent.is_array(), but had no final else branch: when the resolved
parent was a primitive value or null, neither branch matched and the
operation silently did nothing instead of failing.
Add the missing else branch, throwing out_of_range.413 with wording
that matches the existing out_of_range.411 thrown by the analogous
"add" case (operation_add) for the same kind of invalid parent.
Fixes#5396.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move UDLs into nlohmann::literals::json_literals namespace
* Add 'using namespace' to unit tests
* Add 'using namespace' to examples
* Add 'using namespace' to README
* Move UDL mkdocs pages out of basic_json/
* Update documentation
* Update docset index
* Add JSON_GlobalUDLs CMake option
* Add unit test
* Build examples without global UDLs
* Add CI target
* Fix patch::add creating nonexistent parents
The previous behavior was not in accordance with RFC6902.
Add unit test.
Fixes#3134.
* Fix incorrect JSON patch unit test
Co-authored-by: Hudson00 <yagdhscdasg@gmail.com>
Niels Lohmann
Avionic Harshit
Harinath Nampally
