Files
json/docs/mkdocs/docs/home/faq.md
T
Niels LohmannandClaude Sonnet 5 ed513715a8 Document that a NUL byte in the input is treated as end of input (#5534)
* docs: document that a NUL byte in the input is treated as end of input

A NUL byte anywhere in the input - trailing, or embedded ahead of more
otherwise well-formed JSON - is currently treated the same as genuine
end of input, so parsing silently stops there instead of raising the
parse_error.101 any other unexpected byte triggers. This mirrors the
NUL-terminated-C-string convention already used when no explicit input
length is given (json::parse(const char*) already stops at strlen()),
just applied uniformly rather than only when a length is genuinely
unavailable.

This behavior predates this change and is not being altered here -
changing it would be an observable, backwards-incompatible behavior
change for any caller that (knowingly or not) depends on it, which is
not something to do silently in a patch. Documenting the current,
verified behavior as a new FAQ entry instead, so it's an intentional
and discoverable part of the contract rather than a surprise.

Fixes #5530.

Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4RQ1Ahan5YAGbnAQGjZTY

* Add JSON_STRICT_NUL_HANDLING opt-in macro for issue #5530

A NUL byte anywhere in the input is currently treated the same as real
end of input, rather than raising parse_error.101 like any other
unexpected byte (documented in the previous commit's FAQ entry). A full
unconditional fix was tried in PR #5532 but rejected as too risky to
ship by default: any caller could depend on the current behavior, even
unknowingly (e.g. a zero-padded buffer). On PR #5534, gregmarr proposed
a compile-time opt-in flag instead, and the maintainer agreed, wanting
it available now and defaulting to the corrected behavior in 4.0.0.

This mirrors the existing JSON_BRACE_INIT_COPY_SEMANTICS precedent as
closely as sensible:
- JSON_STRICT_NUL_HANDLING defaults to 0 (off); the three lexer sites
  that treat '\0' as EOF/comment-terminator are gated with
  `#if !JSON_STRICT_NUL_HANDLING` so the default-off behavior is
  byte-for-byte identical to today's.
- input_adapters.hpp's `T (&array)[N]` overload additionally trims a
  single trailing '\0' from a `char` array (e.g. a string literal like
  `json::parse("123")`) when the macro is on, so that case keeps
  working; every other element type (unsigned char, std::uint8_t, ...)
  always keeps its full extent. This intentionally does *not* reuse the
  existing strlen()-based pointer overload via SFINAE-excluding `char`
  from the array overload, as originally sketched for this change: that
  approach is ambiguous against the newer generic container overload
  added since PR #5532, and even where it compiles, strlen()-scanning a
  `char` array that is not NUL-terminated within its bounds reads past
  the end of the array (confirmed with AddressSanitizer). Trimming only
  a single trailing byte, without scanning, avoids both problems.
- Documented via docs/mkdocs/docs/api/macros/json_strict_nul_handling.md,
  linked from the macros index/nav/features page, the FAQ entry, and
  the parse/accept/operator>> reference pages.
- Tested in unit-class_parser.cpp and unit-deserialization.cpp, default
  state unguarded and opt-in state guarded. Since the library itself
  #undefs the macro at the end of json.hpp (as JSON_BRACE_INIT_COPY_SEMANTICS
  already does), a plain `#if defined(JSON_STRICT_NUL_HANDLING)` guard
  after the include never actually triggers; the tests instead capture
  the command-line value into a test-local macro before including the
  header. A few pre-existing fixtures elsewhere (std::array<uint8_t, N>
  sized one larger than their literal, relying on value-initialization
  to silently add a trailing zero byte) needed the same one-byte
  adjustment to keep passing under the opt-in behavior.

Unlike the precedent, this adds a proper `JSON_StrictNulHandling` CMake
option (rather than a raw -DCMAKE_CXX_FLAGS injection) and wires its
ci_test_strict_nul_handling target into the ci_cmake_options job matrix
in .github/workflows/ubuntu.yml, so the opt-in build is actually
exercised in CI -- closing the one gap in the precedent's own CI setup
(ci_test_brace_init_copy_semantics is defined but never referenced by
any workflow, so it has never actually run).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Clarify where JSON_STRICT_NUL_HANDLING does not reject NUL bytes

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-24 06:55:21 +02:00

318 lines
14 KiB
Markdown

# Frequently Asked Questions (FAQ)
## Known bugs
### Brace initialization yields arrays
!!! question
Why does
```cpp
json j{true};
```
and
```cpp
json j(true);
```
yield different results (`#!json [true]` vs. `#!json true`)?
This is a known issue, and -- even worse -- the behavior differs between GCC and Clang. The "culprit" for this is the library's constructor overloads for initializer lists to allow syntax like
```cpp
json array = {1, 2, 3, 4};
```
for arrays and
```cpp
json object = {{"one", 1}, {"two", 2}};
```
for objects.
!!! tip
To avoid any confusion and ensure portable code, **do not** use brace initialization with the types `basic_json`, `json`, or `ordered_json` unless you want to create an object or array as shown in the examples above.
To explicitly create a single-element array, use `json::array({value})`:
```cpp
json j = json::array({true}); // [true]
```
**Opt-in copy semantics (since version 3.12.0)**
If you define `JSON_BRACE_INIT_COPY_SEMANTICS` to `1` before including the library, single-element brace initialization is treated as copy/move instead of creating a single-element array:
```cpp
#define JSON_BRACE_INIT_COPY_SEMANTICS 1
#include <nlohmann/json.hpp>
json obj = {{"key", "value"}};
json j{obj}; // -> {"key":"value"} (copy, not array)
```
Without the macro (default behavior), `json j{obj}` creates `[{"key":"value"}]`. This opt-in macro fixes issue #5074 while preserving backwards compatibility for existing code.
## Limitations
### Relaxed parsing
!!! question
Can you add an option to ignore trailing commas?
This library does not support any feature that would jeopardize interoperability.
### Parse errors reading non-ASCII characters
!!! question "Questions"
- Why is the parser complaining about a Chinese character?
- Does the library support Unicode?
- I get an exception `[json.exception.parse_error.101] parse error at line 1, column 53: syntax error while parsing value - invalid string: ill-formed UTF-8 byte; last read: '"Testé$')"`
The library supports **Unicode input** as follows:
- Only **UTF-8** encoded input is supported, which is the default encoding for JSON, according to [RFC 8259](https://tools.ietf.org/html/rfc8259.html#section-8.1).
- `std::u16string` and `std::u32string` can be parsed, assuming UTF-16 and UTF-32 encoding, respectively. These encodings are not supported when reading from files or other input containers.
- Other encodings such as Latin-1 or ISO 8859-1 are **not** supported and will yield parse or serialization errors.
- The library will not replace [Unicode noncharacters](http://www.unicode.org/faq/private_use.html#nonchar1).
- Invalid surrogates (e.g., incomplete pairs such as `\uDEAD`) will yield parse errors.
- The strings stored in the library are UTF-8 encoded. When using the default string type (`std::string`), note that its length/size functions return the number of stored bytes rather than the number of characters or glyphs.
- When you store strings with different encodings in the library, calling [`dump()`](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_a50ec80b02d0f3f51130d4abb5d1cfdc5.html#a50ec80b02d0f3f51130d4abb5d1cfdc5) may throw an exception unless `json::error_handler_t::replace` or `json::error_handler_t::ignore` are used as error handlers.
In most cases, the parser is right to complain, because the input is not UTF-8 encoded. This is especially true for Microsoft Windows, where Latin-1 or ISO 8859-1 is often the standard encoding.
### NUL bytes in the input
!!! question "Questions"
- Why does `json::parse()` silently ignore part of my input?
- Why does a `std::string`/buffer with extra data after the JSON text parse without error, while a similar-looking string with extra text does not?
A `'\0'` (NUL) byte anywhere in the input is treated the same as the real end of the input, rather than as an ordinary (and, outside of a string, invalid) byte. Everything from that byte onward is silently ignored, without a parse error — including further, otherwise well-formed JSON:
```cpp
json::parse(std::string("123") + '\0'); // == 123, no error
json::parse(std::string("123") + '\0' + "true"); // == 123, the "true" is silently ignored too
```
This is different from any other unexpected trailing byte, which *does* raise [`parse_error.101`](../home/exceptions.md#jsonexceptionparse_error101):
```cpp
json::parse("123x"); // throws parse_error.101: unexpected additional data
```
This falls out of the same convention used when no explicit input length is given at all: `json::parse(const char*)` already stops at the first NUL byte via `strlen()`, since a bare pointer has no length of its own. The library applies that same NUL-terminated-C-string convention uniformly, rather than only when a length is genuinely unavailable — so a `std::string`, iterator range, or container whose content happens to include a NUL byte is affected the same way a raw `const char*` would be.
If your input may contain a trailing or embedded NUL that is **not** meant to signal the end of the JSON text — for instance, a fixed-size, zero-padded buffer — trim it yourself before calling `parse()`, since the library will otherwise silently stop there instead of raising an error:
```cpp
s.resize(s.find('\0')); // drop everything from the first NUL onward, if any
json::parse(s);
```
**Opt-in strict handling (since version 3.13.0)**
Manually trimming every input is easy to forget. If you define [`JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md) to `1` before including the library, a `'\0'` byte is instead rejected like any other unexpected byte and raises `parse_error.101`, instead of being treated as end of input:
```cpp
#define JSON_STRICT_NUL_HANDLING 1
#include <nlohmann/json.hpp>
json::parse(std::string("123") + '\0'); // throws parse_error.101 instead of silently returning 123
```
This macro defaults to `0` (disabled, preserving the behavior described above) to avoid breaking existing code that may depend on it, even unknowingly; it is planned to become the default in version 4.0.0. See [its documentation](../api/macros/json_strict_nul_handling.md) for details, including how it also affects `char` arrays such as string literals.
Note that this is unrelated to an *unescaped* NUL byte occurring **inside** a quoted JSON string, which is a different, already-invalid case and is correctly rejected either way:
```cpp
json::parse(std::string("\"") + '\0' + "\""); // throws parse_error.101: control character U+0000 (NUL) must be escaped to \u0000
```
### Wide string handling
!!! question
Why are wide strings (e.g., `std::wstring`) dumped as arrays of numbers?
As described [above](#parse-errors-reading-non-ascii-characters), the library assumes UTF-8 as encoding. To store a wide string, you need to change the encoding.
!!! example
```cpp
#include <codecvt> // codecvt_utf8
#include <locale> // wstring_convert
// encoding function
std::string to_utf8(std::wstring& wide_string)
{
static std::wstring_convert<std::codecvt_utf8<wchar_t>> utf8_conv;
return utf8_conv.to_bytes(wide_string);
}
json j;
std::wstring ws = L"車B1234 こんにちは";
j["original"] = ws;
j["encoded"] = to_utf8(ws);
std::cout << j << std::endl;
```
The result is:
```json
{
"encoded": "車B1234 こんにちは",
"original": [36554, 66, 49, 50, 51, 52, 32, 12371, 12435, 12395, 12385, 12399]
}
```
## Usage
### Thread safety
!!! question
Is `basic_json` thread-safe?
No. `basic_json` provides no built-in synchronization, the same as `std::map` or `std::vector`. Concurrent reads of
the same value from multiple threads are safe, as are concurrent (non-overlapping) accesses to independent `json`
objects. However, any concurrent write to a `json` object -- or a concurrent read while another thread writes to the
same object -- is a data race and requires external synchronization (e.g., a `std::mutex`) by the caller.
### Schema validation
!!! question
Does this library support JSON Schema validation?
Not directly, but the companion project [json-schema-validator](https://github.com/pboettch/json-schema-validator)
builds JSON Schema (draft 4, 6, 7, and 2019-09) validation on top of this library and is a common recommendation
for this use case.
## Exceptions
### Parsing without exceptions
!!! question
Is it possible to indicate a parse error without throwing an exception?
Yes, see [Parsing and exceptions](../features/parsing/parse_exceptions.md).
### Key name in exceptions
!!! question
Can I get the key of the object item that caused an exception?
Yes, you can. Please define the symbol [`JSON_DIAGNOSTICS`](../api/macros/json_diagnostics.md) to get [extended diagnostics messages](exceptions.md#extended-diagnostic-messages).
## Serialization issues
### Number precision
!!! question
- It seems that precision is lost when serializing a double.
- Can I change the precision for floating-point serialization?
The library uses `std::numeric_limits<number_float_t>::digits10` (15 for IEEE `double`s) digits for serialization. This value is sufficient to guarantee roundtripping. If one uses more than this number of digits of precision, then string -> value -> string is not guaranteed to round-trip.
!!! quote "[cppreference.com](https://en.cppreference.com/w/cpp/types/numeric_limits/digits10)"
The value of `std::numeric_limits<T>::digits10` is the number of base-10 digits that can be represented by the type T without change, that is, any number with this many significant decimal digits can be converted to a value of type T and back to decimal form, without change due to rounding or overflow.
!!! tip
The website https://float.exposed gives a good insight into the internal storage of floating-point numbers.
See [this section](../features/types/number_handling.md#number-serialization) on the library's number handling for more information.
### Serializing untrusted or invalid UTF-8
!!! question "Questions"
- Why does `dump()` throw when I serialize data that came from the network?
- Is CVE-2024-34363 a vulnerability in this library?
Crashes reported against this library that stem from an uncaught
[`type_error.316`](exceptions.md#jsonexceptiontype_error316) while serializing unvalidated input (e.g.,
CVE-2024-34363) are a usage issue, not a library vulnerability:
[`dump()`](../api/basic_json/dump.md) throws in its default `strict` mode because
[RFC 8259](https://datatracker.ietf.org/doc/html/rfc8259#section-8.1) requires JSON text to be valid UTF-8.
The recommended pattern is to pass a non-strict [`error_handler`](../api/basic_json/error_handler_t.md) or to handle the
exception:
```cpp
// replace invalid sequences with U+FFFD instead of throwing
const auto s = j.dump(-1, ' ', false, json::error_handler_t::replace);
```
### Using JSON values with `std::format` or `fmt`
!!! question
- Can I use `std::format("{}", j)` on a JSON value?
- Can I use `fmt::format("{}", j)` or `fmt::print("{}", j)` (the [{fmt}](https://github.com/fmtlib/fmt) library) on a JSON value?
`std::format` works out of the box since version 3.13.0, as long as the standard library provides
`<format>` (see [`JSON_HAS_STD_FORMAT`](../api/macros/json_has_std_format.md)); see
[`std::formatter<basic_json>`](../api/basic_json/std_formatter.md) for details, including the `#!cpp "{:#}"`
pretty-print spec, indent widths (`#!cpp "{:2}"`), and custom indent characters (`#!cpp "{:.>#}"`).
For `fmt`, the library ships [`format_as`](../api/basic_json/format_as.md), a small customization point
`fmt` looks for via argument-dependent lookup. It only has an effect on fmt 10.0.0 through 11.0.2 — from
fmt 11.1.0 onwards, `fmt` no longer picks up a `format_as` overload that returns a `std::string`. On such
versions (or any version, if you also want the same `#!cpp "{:#}"`/width/fill-and-align spec support that
`std::formatter<basic_json>` has), define your own `fmt::formatter` specialization; see
[`format_as`](../api/basic_json/format_as.md) for a recipe that mirrors it.
If you get ambiguous-overload errors when passing a JSON value to `fmt::format`/`fmt::print` without any
`fmt::formatter<json>` specialization in scope, that's `fmt` picking up `basic_json`'s implicit
`operator ValueType()` conversion operator (see [#964](https://github.com/nlohmann/json/issues/964) and
[#958](https://github.com/nlohmann/json/issues/958)); disabling it via
[`JSON_USE_IMPLICIT_CONVERSIONS 0`](../api/macros/json_use_implicit_conversions.md) avoids the ambiguity.
## Compilation issues
### Android SDK
!!! question
Why does the code not compile with Android SDK?
Android defaults to using very old compilers and C++ libraries. To fix this, add the following to your `Application.mk`. This will switch to the LLVM C++ library, the Clang compiler, and enable C++11 and other features disabled by default.
```ini
APP_STL := c++_shared
NDK_TOOLCHAIN_VERSION := clang3.6
APP_CPPFLAGS += -frtti -fexceptions
```
The code compiles successfully with [Android NDK](https://developer.android.com/ndk/index.html?hl=ml), Revision 9 - 11 (and possibly later) and [CrystaX's Android NDK](https://www.crystax.net/en/android/ndk) version 10.
### Missing STL function
!!! question "Questions"
- Why do I get a compilation error `'to_string' is not a member of 'std'` (or similarly, for `strtod` or `strtof`)?
- Why does the code not compile with MinGW or Android SDK?
This is not an issue with the code, but rather with the compiler itself. On Android, see above to build with a newer environment. For MinGW, please refer to [this site](http://tehsausage.com/mingw-to-string) and [this discussion](https://github.com/nlohmann/json/issues/136) for information on how to fix this bug. For Android NDK using `APP_STL := gnustl_static`, please refer to [this discussion](https://github.com/nlohmann/json/issues/219).