* docs: document that a NUL byte in the input is treated as end of input
A NUL byte anywhere in the input - trailing, or embedded ahead of more
otherwise well-formed JSON - is currently treated the same as genuine
end of input, so parsing silently stops there instead of raising the
parse_error.101 any other unexpected byte triggers. This mirrors the
NUL-terminated-C-string convention already used when no explicit input
length is given (json::parse(const char*) already stops at strlen()),
just applied uniformly rather than only when a length is genuinely
unavailable.
This behavior predates this change and is not being altered here -
changing it would be an observable, backwards-incompatible behavior
change for any caller that (knowingly or not) depends on it, which is
not something to do silently in a patch. Documenting the current,
verified behavior as a new FAQ entry instead, so it's an intentional
and discoverable part of the contract rather than a surprise.
Fixes#5530.
Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4RQ1Ahan5YAGbnAQGjZTY
* Add JSON_STRICT_NUL_HANDLING opt-in macro for issue #5530
A NUL byte anywhere in the input is currently treated the same as real
end of input, rather than raising parse_error.101 like any other
unexpected byte (documented in the previous commit's FAQ entry). A full
unconditional fix was tried in PR #5532 but rejected as too risky to
ship by default: any caller could depend on the current behavior, even
unknowingly (e.g. a zero-padded buffer). On PR #5534, gregmarr proposed
a compile-time opt-in flag instead, and the maintainer agreed, wanting
it available now and defaulting to the corrected behavior in 4.0.0.
This mirrors the existing JSON_BRACE_INIT_COPY_SEMANTICS precedent as
closely as sensible:
- JSON_STRICT_NUL_HANDLING defaults to 0 (off); the three lexer sites
that treat '\0' as EOF/comment-terminator are gated with
`#if !JSON_STRICT_NUL_HANDLING` so the default-off behavior is
byte-for-byte identical to today's.
- input_adapters.hpp's `T (&array)[N]` overload additionally trims a
single trailing '\0' from a `char` array (e.g. a string literal like
`json::parse("123")`) when the macro is on, so that case keeps
working; every other element type (unsigned char, std::uint8_t, ...)
always keeps its full extent. This intentionally does *not* reuse the
existing strlen()-based pointer overload via SFINAE-excluding `char`
from the array overload, as originally sketched for this change: that
approach is ambiguous against the newer generic container overload
added since PR #5532, and even where it compiles, strlen()-scanning a
`char` array that is not NUL-terminated within its bounds reads past
the end of the array (confirmed with AddressSanitizer). Trimming only
a single trailing byte, without scanning, avoids both problems.
- Documented via docs/mkdocs/docs/api/macros/json_strict_nul_handling.md,
linked from the macros index/nav/features page, the FAQ entry, and
the parse/accept/operator>> reference pages.
- Tested in unit-class_parser.cpp and unit-deserialization.cpp, default
state unguarded and opt-in state guarded. Since the library itself
#undefs the macro at the end of json.hpp (as JSON_BRACE_INIT_COPY_SEMANTICS
already does), a plain `#if defined(JSON_STRICT_NUL_HANDLING)` guard
after the include never actually triggers; the tests instead capture
the command-line value into a test-local macro before including the
header. A few pre-existing fixtures elsewhere (std::array<uint8_t, N>
sized one larger than their literal, relying on value-initialization
to silently add a trailing zero byte) needed the same one-byte
adjustment to keep passing under the opt-in behavior.
Unlike the precedent, this adds a proper `JSON_StrictNulHandling` CMake
option (rather than a raw -DCMAKE_CXX_FLAGS injection) and wires its
ci_test_strict_nul_handling target into the ci_cmake_options job matrix
in .github/workflows/ubuntu.yml, so the opt-in build is actually
exercised in CI -- closing the one gap in the precedent's own CI setup
(ci_test_brace_init_copy_semantics is defined but never referenced by
any workflow, so it has never actually run).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Clarify where JSON_STRICT_NUL_HANDLING does not reject NUL bytes
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Add iterator+sentinel tests and docs for binary deserializers
This commit extends the C++20 ranges support (iterator+sentinel pairs) to the
binary format deserializers from_cbor, from_msgpack, from_ubjson, from_bjdata,
and from_bson, matching what was already done for parse(), accept(), and
sax_parse().
Changes:
- Add istreambuf_sentinel helper to test_utils.hpp for EOF detection in tests
- Add 5 new test cases that read binary files directly via
std::istreambuf_iterator<char> + sentinel, without pre-buffering
- Update documentation for all 5 from_* functions to document overload (3)
with SentinelType parameter
- All tests pass; verified against existing test suite data
- Fix potential buffer over-read warning in heterogeneous iterator test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Merge iterator+sentinel overloads and fix ambiguity/CI issues
Address PR review feedback and CI failures:
- Merge the separate same-type and sentinel-type iterator overloads of
parse(), accept(), sax_parse(), and the five from_* binary deserializers
into a single overload with SentinelType defaulted to IteratorType,
as suggested in review. Applied the same simplification to the
detail::input_adapter() free functions.
- Fix a latent ambiguity: some compilers (e.g. GCC 4.8) unreliably SFINAE
the operator!= detection for std::nullptr_t against container/string
types, making calls like parse(s, nullptr, ...) ambiguous with the
compatible-input overload. can_compare_ne now explicitly excludes
std::nullptr_t as a SentinelType.
- Use a named enable_if_t template parameter instead of an unnamed
function parameter for the SFINAE guard, fixing a clang-tidy
hicpp-named-parameter/readability-named-parameter failure.
- Update parse.md, accept.md, sax_parse.md, and the five from_*.md pages
to document the merged overload instead of separate (2)/(3) overloads,
also fixing an over-160-char line that broke the documentation
style_check CI job.
- Rework the BSON iterator+sentinel test to parse a BSON file already
present in the test suite instead of writing/deleting a temp file.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix -Wunneeded-internal-declaration for CustomSentinel in test
CustomSentinel lives in an anonymous namespace (internal linkage), and
the library's parse loop only ever evaluates the iterator-first
direction (it != last), so the reversed-order friend operator!= was
never referenced. Clang's -Weverything flags such unused internal
declarations as an error. Drop the unused overload; the used direction
is enough to satisfy can_compare_ne's either-order detection.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy hicpp-named-parameter and misc-const-correctness
- Drop the unused reversed-order operator!= overload from
utils::istreambuf_sentinel (only iterator != sentinel is ever
evaluated) and name the remaining friend's sentinel parameter, fixing
hicpp-named-parameter/readability-named-parameter.
- Mark the istreambuf_iterator first/last helper variable const in the
five binary-format sentinel tests, fixing misc-const-correctness.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy misc-const-correctness in heterogeneous sentinel test
json_str is only read via .data()/.size() and never reassigned, so
clang-tidy correctly flags it as const-able. Verified against the exact
CI job (silkeh/clang:dev, ci_clang_tidy target) by running clang-tidy
directly on this file plus the five binary-format sentinel tests
touched by prior commits; all are now clean.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix container input_adapter SFINAE for lvalue-only ADL begin/end (#111)
The container overload of json::parse(c) / accept(c) / sax_parse(c, ...)
silently dropped from overload resolution for user types whose ADL
begin(T&) / end(T&) accepted only non-const lvalue references
(a legitimate pattern matching std::begin semantics). This was because
the detection code used std::declval<ContainerType>() which synthesized
an rvalue, and the rvalue failed to bind to lvalue-only ADL functions.
Fix by making both the outer input_adapter(ContainerType&&) and the
factory's create(ContainerType&&) forwarding references, preserving the
caller's value category and constness via reference collapsing. This
ensures detection (std::declval) and actual use (std::forward) always
match without needing decay/remove_reference.
- Rewrite input_adapters.hpp container overload with forwarding refs
- Add regression tests for lvalue-only non-const ADL begin/end
- Add regression test for rvalue containers (no breakage)
- Update API docs (parse, accept, sax_parse, from_*) to clarify
that begin/end must match std::begin/std::end semantics
- Add version history notes for 3.13.0
- Regenerate amalgamation
Second-order effect: binary_reader.hpp's internal call to
input_adapter(number_vector) now deduces iterator vs const_iterator
based on the lvalue; functionally harmless (iterator_input_adapter is
iterator-type-agnostic), verified via unit-ubjson/unit-bjdata tests.
Closes remaining limitation from #4354 / PR #5218 (todo 106).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid strlen() in test container to fix Codacy CWE-126 flag
Suppressing the strlen()-based CWE-126 warning with NOLINT/nosec
comments only silenced clang-tidy and the standalone Flawfinder
Action; Codacy's own analysis (which also flags this pattern and
doesn't honor those suppression comments) still reported it as a new
issue, plus flagged the near-duplicate begin/end pair as cloned code.
Store the buffer's size explicitly in MyContainerNonConstADL instead
of computing it via strlen() in end(), which removes the flagged
pattern outright and also de-duplicates the struct from the existing
MyContainer's char*-based begin/end pair.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid trailing return type to satisfy clang-tidy fuchsia-trailing-return
The forwarding-reference input_adapter(ContainerType&&) entry point was
written with an auto/trailing-decltype return type, but this project's
ci_clang_tidy job enables the fuchsia-trailing-return check as an
error, which rejects it. The return type only depends on the template
parameter ContainerType, not on the runtime parameter, so it can be
written as an ordinary leading return type instead - no functional
change.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid C-style array in test to satisfy clang-tidy avoid-c-arrays
clang-tidy's cppcoreguidelines/hicpp/modernize-avoid-c-arrays checks
flagged the char raw_data[] declaration used to reproduce the
lvalue-only non-const ADL begin/end scenario. Use std::string instead
and take a mutable pointer via &raw_data[0], which is the standard
way to get a non-const char* into a string's buffer under C++11
(std::string::data() only returns non-const in C++17 and later).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 📝 Fix documentation gaps for 3.13.0 release (todos 138-142)
- Todo 138: Add "Known issues" section to modules.md with compiler-specific troubleshooting (GCC redefinition, MSVC symbol export). Add pointer note to quality_assurance.md.
- Todo 139: Document CBOR/MessagePack half-precision float encoding for NaN/Infinity (0xF9/0xCA with exact byte sequences). Explain pre-3.13.0 double-precision bug mechanism without issue citations.
- Todo 140: Document CBOR negative-integer-overflow rejection (parse_error.112) for magnitudes exceeding int64_t range (already implemented in rev 1).
- Todo 141: Update version history in value.md and operator[].md with behavior-change details, removing issue citations per citation policy (prose is self-contained).
- Todo 142: Global sed replace of 3.12.x → 3.13.0 placeholder across all 20 documentation files.
Revision 2 incorporates feedback to reduce changelog-like issue citations. Only citations that add unique troubleshooting value are retained (#5103 for GCC workaround, #3970 for MSVC symbol export). "Known issues" section follows PR #5252's visual pattern (info admonition with bold-bullet format).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 📝 Document integer type selection, type_name() invalid value, and std::optional get() fix
- number_handling.md: clarify that positive/negative integers select
unsigned/signed storage based on the leading minus sign (todo 143).
- type_name.md: document the new "invalid" return value for corrupted
JSON values (todo 145).
- get.md: note that get<std::optional<T>>() was unreachable in every
configuration prior to 3.13.0 due to an internal macro-guard bug,
unrelated to JSON_USE_IMPLICIT_CONVERSIONS's actual effect (todo 144).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add operator<< for json_pointer
* Deprecate json_pointer::operator string_t()
* Update documentation
* Move operator<<(basic_json) example
* Add example
* Add mkdocs-redirects
* Move operator<< and operator>> doc pages out of basic_json/
* Rename JSON pointer operator_string to operator_string_t
* Add unit test
Niels LohmannandClaude Sonnet 5
chirszandNiels Lohmann
Florian Albrechtskirchinger