starred/json
1
0
Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-01 14:15:50 +01:00
Code Issues Packages Projects Releases 10 Wiki Activity
Files
abbe52d6de81323fa0b159da2c36f0589756af16
json/tests/src/unit-deserialization.cpp
T
Niels Lohmann abbe52d6de Add JSON_PRECISE_STREAM_POSITION to leave the character that terminates a number in the stream (#5344) 
* docs: qualify the operator>> stream positioning guarantee

operator>>'s notes state that it leaves the stream positioned right
after the parsed value, so that concatenated JSON values can be read
back to back. That does not hold when the value is a number: a number
is only terminated by the character that follows it, and the lexer's
unget() is simulated (it rewinds only the lexer's own bookkeeping),
so that character stays consumed from the stream.

Document the actual behaviour: the guarantee holds for all value types
except numbers, which must be followed by whitespace. Also qualify the
cross-reference on the JSON Lines page, which repeated the unqualified
claim.

Documentation only; the behaviour itself is tracked in #5340.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* fix: restore the character that terminates a number (#5340)

operator>> is documented to leave the stream positioned right after the
parsed value, so that concatenated JSON values can be read back to back.
That did not hold for numbers: a number is only terminated by the
character following it, and lexer::scan_number() reads that character
and calls unget() -- which is simulated and rewinds only the lexer's own
bookkeeping. input_stream_adapter consumes via sbumpc() with no matching
sungetc(), so the terminating character stayed consumed and the next
extraction started one byte too late ('1true' left the stream at 'rue').

Propagating unget() to the adapter directly does not work: next_unget
makes the following get() replay the cached character, so the terminator
would be delivered twice. Instead, restore the still-pending character
once at the end of a non-strict parse, where the input is handed back to
the caller:

- input_stream_adapter gains unget_character() (sungetc()) and advertises
  it via supports_unget, detected the same way as supports_seek.
- lexer::restore_pending_unget() turns a pending simulated unget of a
  real (non-EOF) character into a real one and clears next_unget so the
  character is not also replayed. It is a no-op for adapters that cannot
  unget, and reports failure when sungetc() fails, in which case the
  input is left as it was before.
- parser calls it on the three non-strict paths, i.e. for operator>> and
  sax_parse(strict = false).

Strict parse()/accept() are unaffected: they require the input to end
after the value, so the character is consumed by the end-of-input check
anyway. Parse error messages and reported positions are unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* tests: fix CI failures in the #5340 test helpers

Four CI failures, all in the new test code:

- GCC (-Werror=useless-cast): drop the `json(...)` wrapper around
  `json::parse(...)`, which already returns a `json`.
- GCC (-Werror=unused-result): assign the discarded `json::parse()`
  result to a dummy, the idiom used elsewhere in the test suite, and
  catch `json::parse_error&` for consistency.
- clang-tidy (google-default-arguments): remove the default argument
  from the `pbackfail()` override; `sungetc()` supplies the base
  declaration's default.
- MSVC (bad allocation): `no_putback_streambuf::underflow()` set a
  one-character get area without advancing `m_pos`, so an implementation
  whose `istream::get` peeks before it bumps re-read the same character
  forever. Keep no get area at all: `underflow()` peeks, `uflow()`
  consumes, and `sungetc()` still always lands in `pbackfail()`, which
  is what the test needs.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* fix: leave the character that terminates a number in the input

Read the character following a number without consuming it, instead of
consuming it and putting it back. input_stream_adapter now peeks with
sgetc() and only steps over the character when the next one is requested
or when the adapter is destroyed, so releasing it cannot fail - no
putback position is required from the streambuf.

Suggested by gregmarr in #5344.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: match the version history wording to the peek-based fix

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: drop the whitespace-separator caveat from the parsing pages

The caveat added in #5343 describes the behavior this branch fixes: a
number no longer consumes the character that terminates it, so
concatenated values need no separator.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* refactor: split the strict and non-strict paths in parser

Folding the release_lookahead() call into the existing strict check left
the "in strict mode" comment on an else-if branch, and made the strict
condition in sax_parse() redundant with the branch it followed.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Put the stream position fix behind JSON_PRECISE_STREAM_POSITION

Leaving the character that terminates a number in the stream is observable:
reading "1,2,3" with repeated operator>> works today only because the comma
after each number is swallowed, and std::getline after a number skips the
line break. Both break with the fix, so make it opt-in for 3.x, as suggested
by @gregmarr in the review.

- JSON_PRECISE_STREAM_POSITION (default 0) selects the peek-based
  input_stream_adapter. Without it, the adapter is the consuming one from
  develop and has no supports_lookahead, so lexer::release_lookahead() and
  the parser's calls to it compile to nothing.
- The macro changes input_stream_adapter's layout and member functions, so
  it gets the ABI tag _psp, after _bics. The ABI config tests, the natvis
  generator, and nlohmann_json.natvis (regenerated) know the tag.
- The tests for the fix move to unit-precise-stream-position.cpp, which
  defines the macro itself and runs in every build, and gain the two cases
  above. unit-deserialization.cpp pins the default behavior instead.
- The docs describe the default behavior again and point to the new macro
  page; version history says "added in 3.13.0, planned default in 4.0.0".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-25 17:56:18 +02:00

1370 lines
52 KiB
C++
Raw Blame History

// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
// capture whether JSON_STRICT_NUL_HANDLING was enabled on the command line
// (e.g. -DJSON_STRICT_NUL_HANDLING=1) *before* including json.hpp, since the
// library #undefs JSON_STRICT_NUL_HANDLING itself once the header has been
// fully processed (see include/nlohmann/detail/macro_unscope.hpp)
#if defined(JSON_STRICT_NUL_HANDLING) && (JSON_STRICT_NUL_HANDLING == 1)
#define JSON_TEST_STRICT_NUL_HANDLING_ENABLED 1
#endif
#include <nlohmann/json.hpp>
using nlohmann::json;
#ifdef JSON_TEST_NO_GLOBAL_UDLS
using namespace nlohmann::literals; // NOLINT(google-build-using-namespace)
#endif
#include <iostream>
#include <iterator>
#include <sstream>
#include <string>
#include <valarray>
#if defined(_WIN32)
#define NOMINMAX
#include <windows.h> // for GetACP()
#endif
namespace
{
struct SaxEventLogger : public nlohmann::json_sax<json>
{
bool null() override
{
events.emplace_back("null()");
return true;
}
bool boolean(bool val) override
{
events.emplace_back(val ? "boolean(true)" : "boolean(false)");
return true;
}
bool number_integer(json::number_integer_t val) override
{
events.push_back("number_integer(" + std::to_string(val) + ")");
return true;
}
bool number_unsigned(json::number_unsigned_t val) override
{
events.push_back("number_unsigned(" + std::to_string(val) + ")");
return true;
}
bool number_float(json::number_float_t /*val*/, const std::string& s) override
{
events.push_back("number_float(" + s + ")");
return true;
}
bool string(std::string& val) override
{
events.push_back("string(" + val + ")");
return true;
}
bool binary(json::binary_t& val) override
{
std::string binary_contents = "binary(";
std::string comma_space;
for (auto b : val)
{
binary_contents.append(comma_space);
binary_contents.append(std::to_string(static_cast<int>(b)));
comma_space = ", ";
}
binary_contents.append(")");
events.push_back(binary_contents);
return true;
}
bool start_object(std::size_t elements) override
{
if (elements == (std::numeric_limits<std::size_t>::max)())
{
events.emplace_back("start_object()");
}
else
{
events.push_back("start_object(" + std::to_string(elements) + ")");
}
return true;
}
bool key(std::string& val) override
{
events.push_back("key(" + val + ")");
return true;
}
bool end_object() override
{
events.emplace_back("end_object()");
return true;
}
bool start_array(std::size_t elements) override
{
if (elements == (std::numeric_limits<std::size_t>::max)())
{
events.emplace_back("start_array()");
}
else
{
events.push_back("start_array(" + std::to_string(elements) + ")");
}
return true;
}
bool end_array() override
{
events.emplace_back("end_array()");
return true;
}
bool parse_error(std::size_t position, const std::string& /*last_token*/, const json::exception& /*ex*/) override
{
events.push_back("parse_error(" + std::to_string(position) + ")");
return false;
}
std::vector<std::string> events {}; // NOLINT(readability-redundant-member-init)
};
struct SaxEventLoggerExitAfterStartObject : public SaxEventLogger
{
bool start_object(std::size_t elements) override
{
if (elements == (std::numeric_limits<std::size_t>::max)())
{
events.emplace_back("start_object()");
}
else
{
events.push_back("start_object(" + std::to_string(elements) + ")");
}
return false;
}
};
struct SaxEventLoggerExitAfterKey : public SaxEventLogger
{
bool key(std::string& val) override
{
events.push_back("key(" + val + ")");
return false;
}
};
struct SaxEventLoggerExitAfterStartArray : public SaxEventLogger
{
bool start_array(std::size_t elements) override
{
if (elements == (std::numeric_limits<std::size_t>::max)())
{
events.emplace_back("start_array()");
}
else
{
events.push_back("start_array(" + std::to_string(elements) + ")");
}
return false;
}
};
template <typename T>
class proxy_iterator
{
public:
using iterator = typename T::iterator;
using value_type = typename std::iterator_traits<iterator>::value_type;
using reference = typename std::iterator_traits<iterator>::reference;
using pointer = typename std::iterator_traits<iterator>::pointer;
using difference_type =
typename std::iterator_traits<iterator>::difference_type;
using iterator_category = std::input_iterator_tag;
proxy_iterator() = default;
explicit proxy_iterator(iterator& it) : m_it(std::addressof(it)) {}
proxy_iterator& operator++()
{
++*m_it;
return *this;
}
proxy_iterator& operator--()
{
--*m_it;
return *this;
}
bool operator==(const proxy_iterator& rhs) const
{
return (m_it && rhs.m_it) ? (*m_it == *rhs.m_it) : (m_it == rhs.m_it);
}
bool operator!=(const proxy_iterator& rhs) const
{
return !(*this == rhs);
}
reference operator*() const
{
return **m_it;
}
private:
iterator* m_it = nullptr;
};
// JSON_HAS_CPP_20
#if defined(__cpp_char8_t)
bool check_utf8()
{
#if defined(_WIN32)
// Runtime check of the active ANSI code page
// 65001 == UTF-8
return GetACP() == 65001;
#elif defined(__ICC) || defined(__INTEL_COMPILER)
// classic Intel ICC does not encode narrow string literals containing
// non-ASCII source characters as UTF-8, so comparing a decoded u8 literal
// against a narrow string literal containing the same characters fails
return false;
#else
return true;
#endif
}
#endif
} // namespace
TEST_CASE("deserialization")
{
SECTION("successful deserialization")
{
SECTION("stream")
{
std::stringstream ss1;
std::stringstream ss2;
std::stringstream ss3;
ss1 << R"(["foo",1,2,3,false,{"one":1}])";
ss2 << R"(["foo",1,2,3,false,{"one":1}])";
ss3 << R"(["foo",1,2,3,false,{"one":1}])";
const json j = json::parse(ss1);
CHECK(json::accept(ss2));
CHECK(j == json({"foo", 1, 2, 3, false, {{"one", 1}}}));
SaxEventLogger l;
CHECK(json::sax_parse(ss3, &l));
CHECK(l.events.size() == 11);
CHECK(l.events == std::vector<std::string>(
{
"start_array()", "string(foo)", "number_unsigned(1)",
"number_unsigned(2)", "number_unsigned(3)", "boolean(false)",
"start_object()", "key(one)", "number_unsigned(1)",
"end_object()", "end_array()"
}));
}
SECTION("string literal")
{
const auto* s = R"(["foo",1,2,3,false,{"one":1}])";
const json j = json::parse(s);
CHECK(json::accept(s));
CHECK(j == json({"foo", 1, 2, 3, false, {{"one", 1}}}));
SaxEventLogger l;
CHECK(json::sax_parse(s, &l));
CHECK(l.events.size() == 11);
CHECK(l.events == std::vector<std::string>(
{
"start_array()", "string(foo)", "number_unsigned(1)",
"number_unsigned(2)", "number_unsigned(3)", "boolean(false)",
"start_object()", "key(one)", "number_unsigned(1)",
"end_object()", "end_array()"
}));
}
SECTION("string_t")
{
json::string_t const s = R"(["foo",1,2,3,false,{"one":1}])";
const json j = json::parse(s);
CHECK(json::accept(s));
CHECK(j == json({"foo", 1, 2, 3, false, {{"one", 1}}}));
SaxEventLogger l;
CHECK(json::sax_parse(s, &l));
CHECK(l.events.size() == 11);
CHECK(l.events == std::vector<std::string>(
{
"start_array()", "string(foo)", "number_unsigned(1)",
"number_unsigned(2)", "number_unsigned(3)", "boolean(false)",
"start_object()", "key(one)", "number_unsigned(1)",
"end_object()", "end_array()"
}));
}
SECTION("operator<<")
{
std::stringstream ss;
ss << R"(["foo",1,2,3,false,{"one":1}])";
json j;
j << ss;
CHECK(j == json({"foo", 1, 2, 3, false, {{"one", 1}}}));
}
SECTION("operator>>")
{
std::stringstream ss;
ss << R"(["foo",1,2,3,false,{"one":1}])";
json j;
ss >> j;
CHECK(j == json({"foo", 1, 2, 3, false, {{"one", 1}}}));
}
SECTION("operator>> with a NUL byte after the value (issue #5530)")
{
// operator>> parses non-strictly (it does not require the whole
// stream to be consumed), so a NUL byte following a complete
// value is simply left unread on the stream and never reaches
// the "expected end of input" check that JSON_STRICT_NUL_HANDLING
// affects; this holds regardless of the macro (verified below for
// the opt-in state as well)
std::string data = "123";
data.push_back('\0');
std::istringstream ss(data);
json j;
ss >> j;
CHECK(j == json(123));
CHECK(ss.good());
}
SECTION("user-defined string literal")
{
CHECK("[\"foo\",1,2,3,false,{\"one\":1}]"_json == json({"foo", 1, 2, 3, false, {{"one", 1}}}));
}
}
SECTION("unsuccessful deserialization")
{
SECTION("stream")
{
std::stringstream ss1;
std::stringstream ss3;
std::stringstream ss4;
std::stringstream ss5;
ss1 << R"(["foo",1,2,3,false,{"one":1})";
ss3 << R"(["foo",1,2,3,false,{"one":1})";
ss4 << R"(["foo",1,2,3,false,{"one":1})";
ss5 << R"(["foo",1,2,3,false,{"one":1})";
json _;
CHECK_THROWS_WITH_AS(_ = json::parse(ss1), "[json.exception.parse_error.101] parse error at line 1, column 29: syntax error while parsing array - unexpected end of input; expected ']'", json::parse_error&);
CHECK(!json::accept(ss3));
json j_error;
CHECK_NOTHROW(j_error = json::parse(ss4, nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(ss5, &l));
CHECK(l.events.size() == 11);
CHECK(l.events == std::vector<std::string>(
{
"start_array()", "string(foo)", "number_unsigned(1)",
"number_unsigned(2)", "number_unsigned(3)", "boolean(false)",
"start_object()", "key(one)", "number_unsigned(1)",
"end_object()", "parse_error(29)"
}));
}
SECTION("string")
{
json::string_t const s = R"(["foo",1,2,3,false,{"one":1})";
json _;
CHECK_THROWS_WITH_AS(_ = json::parse(s), "[json.exception.parse_error.101] parse error at line 1, column 29: syntax error while parsing array - unexpected end of input; expected ']'", json::parse_error&);
CHECK(!json::accept(s));
json j_error;
CHECK_NOTHROW(j_error = json::parse(s, nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(s, &l));
CHECK(l.events.size() == 11);
CHECK(l.events == std::vector<std::string>(
{
"start_array()", "string(foo)", "number_unsigned(1)",
"number_unsigned(2)", "number_unsigned(3)", "boolean(false)",
"start_object()", "key(one)", "number_unsigned(1)",
"end_object()", "parse_error(29)"
}));
const char* string = nullptr;
CHECK_THROWS_WITH_AS(_ = json::parse(string), "[json.exception.parse_error.101] parse error: attempting to parse an empty input; check that your input string or stream contains the expected JSON", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::parse(nullptr), "[json.exception.parse_error.101] parse error: attempting to parse an empty input; check that your input string or stream contains the expected JSON", json::parse_error&);
}
SECTION("operator<<")
{
std::stringstream ss;
ss << R"(["foo",1,2,3,false,{"one":1})";
json j;
CHECK_THROWS_WITH_AS(j << ss, "[json.exception.parse_error.101] parse error at line 1, column 29: syntax error while parsing array - unexpected end of input; expected ']'", json::parse_error&);
}
SECTION("operator>>")
{
std::stringstream ss;
ss << R"(["foo",1,2,3,false,{"one":1})";
json j;
CHECK_THROWS_WITH_AS(ss >> j, "[json.exception.parse_error.101] parse error at line 1, column 29: syntax error while parsing array - unexpected end of input; expected ']'", json::parse_error&);
}
#if defined(JSON_TEST_STRICT_NUL_HANDLING_ENABLED)
SECTION("operator>> with a NUL byte where a value is expected (JSON_STRICT_NUL_HANDLING == 1, issue #5530)")
{
// a trailing NUL byte *after* a complete value is unaffected by the
// macro (see the successful-deserialization "operator>> with a NUL
// byte after the value" section above): operator>> parses
// non-strictly and never reaches the "expected end of input" check
// that the macro changes. A NUL byte where a *value* is expected,
// however, goes through the same token dispatch as any other input
// and is affected: with the macro enabled it now raises
// parse_error.101 (like any other unrecognized byte) instead of
// being silently treated the same as an empty stream.
std::string const data(1, '\0');
std::istringstream ss(data);
json j;
CHECK_THROWS_WITH_AS(ss >> j,
"[json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - invalid literal; last read: '<U+0000>'",
json::parse_error&);
}
#endif
SECTION("user-defined string literal")
{
CHECK_THROWS_WITH_AS("[\"foo\",1,2,3,false,{\"one\":1}"_json, "[json.exception.parse_error.101] parse error at line 1, column 29: syntax error while parsing array - unexpected end of input; expected ']'", json::parse_error&);
}
}
SECTION("contiguous containers")
{
SECTION("directly")
{
SECTION("from std::vector")
{
std::vector<uint8_t> const v = {'t', 'r', 'u', 'e'};
CHECK(json::parse(v) == json(true));
CHECK(json::accept(v));
SaxEventLogger l;
CHECK(json::sax_parse(v, &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
}
SECTION("from std::vector<signed char>")
{
std::vector<signed char> const v = {'t', 'r', 'u', 'e'};
CHECK(json::parse(v) == json(true));
CHECK(json::accept(v));
SaxEventLogger l;
CHECK(json::sax_parse(v, &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
// bytes outside ASCII are negative here and must not be sign-extended;
// 0xC3 and 0xA9 do not fit in signed char (MSVC C4309), so spell them as negative values
std::vector<signed char> const umlaut = {'"', static_cast<signed char>(0xC3 - 0x100), static_cast<signed char>(0xA9 - 0x100), '"'};
CHECK(json::parse(umlaut) == json("\xC3\xA9"));
CHECK(json::accept(umlaut));
// 0xFF (spelled as -1 to stay in range) must not be reported as end of input
std::vector<signed char> const trailing = {'t', 'r', 'u', 'e', static_cast<signed char>(0xFF - 0x100)};
json _;
CHECK_THROWS_WITH_AS(_ = json::parse(trailing), "[json.exception.parse_error.101] parse error at line 1, column 5: syntax error while parsing value - invalid literal; last read: 'true\xFF'; expected end of input", json::parse_error&);
CHECK(!json::accept(trailing));
}
SECTION("from std::array")
{
// sized to exactly the length of "true": a size of 5 would leave
// a value-initialized trailing 0x00 element that is only
// silently accepted as end-of-input by default and would fail
// under JSON_STRICT_NUL_HANDLING
std::array<uint8_t, 4> const v { {'t', 'r', 'u', 'e'} };
CHECK(json::parse(v) == json(true));
CHECK(json::accept(v));
SaxEventLogger l;
CHECK(json::sax_parse(v, &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
}
SECTION("from array")
{
uint8_t v[] = {'t', 'r', 'u', 'e'}; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
CHECK(json::parse(v) == json(true));
CHECK(json::accept(v));
SaxEventLogger l;
CHECK(json::sax_parse(v, &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
}
SECTION("from chars")
{
auto* v = new uint8_t[5]; // NOLINT(cppcoreguidelines-owning-memory)
v[0] = 't';
v[1] = 'r';
v[2] = 'u';
v[3] = 'e';
v[4] = '\0';
CHECK(json::parse(v) == json(true));
CHECK(json::accept(v));
SaxEventLogger l;
CHECK(json::sax_parse(v, &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
delete[] v; // NOLINT(cppcoreguidelines-owning-memory)
}
SECTION("from std::string")
{
std::string const v = {'t', 'r', 'u', 'e'};
CHECK(json::parse(v) == json(true));
CHECK(json::accept(v));
SaxEventLogger l;
CHECK(json::sax_parse(v, &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
}
SECTION("from std::initializer_list")
{
std::initializer_list<uint8_t> const v = {'t', 'r', 'u', 'e'};
CHECK(json::parse(v) == json(true));
CHECK(json::accept(v));
SaxEventLogger l;
CHECK(json::sax_parse(v, &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
}
SECTION("empty container")
{
std::vector<uint8_t> const v;
json _;
CHECK_THROWS_AS(_ = json::parse(v), json::parse_error&);
CHECK(!json::accept(v));
SaxEventLogger l;
CHECK(!json::sax_parse(v, &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(1)"}));
}
}
SECTION("via iterator range")
{
SECTION("from std::vector")
{
std::vector<uint8_t> v = {'t', 'r', 'u', 'e'};
CHECK(json::parse(std::begin(v), std::end(v)) == json(true));
CHECK(json::accept(std::begin(v), std::end(v)));
SaxEventLogger l;
CHECK(json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
}
SECTION("from std::array")
{
// sized to exactly the length of "true", see the analogous
// "from std::array" section above for why
std::array<uint8_t, 4> v { {'t', 'r', 'u', 'e'} };
CHECK(json::parse(std::begin(v), std::end(v)) == json(true));
CHECK(json::accept(std::begin(v), std::end(v)));
SaxEventLogger l;
CHECK(json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
}
SECTION("from array")
{
uint8_t v[] = {'t', 'r', 'u', 'e'}; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
CHECK(json::parse(std::begin(v), std::end(v)) == json(true));
CHECK(json::accept(std::begin(v), std::end(v)));
SaxEventLogger l;
CHECK(json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
}
SECTION("from std::string")
{
std::string v = {'t', 'r', 'u', 'e'};
CHECK(json::parse(std::begin(v), std::end(v)) == json(true));
CHECK(json::accept(std::begin(v), std::end(v)));
SaxEventLogger l;
CHECK(json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
}
SECTION("from std::initializer_list")
{
std::initializer_list<uint8_t> const v = {'t', 'r', 'u', 'e'};
CHECK(json::parse(std::begin(v), std::end(v)) == json(true));
CHECK(json::accept(std::begin(v), std::end(v)));
SaxEventLogger l;
CHECK(json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
}
SECTION("from std::valarray")
{
std::valarray<uint8_t> v = {'t', 'r', 'u', 'e'};
CHECK(json::parse(std::begin(v), std::end(v)) == json(true));
CHECK(json::accept(std::begin(v), std::end(v)));
SaxEventLogger l;
CHECK(json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
}
SECTION("with empty range")
{
std::vector<uint8_t> v;
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(1)"}));
}
SECTION("iterator_input_adapter advances iterators correctly")
{
using nlohmann::json;
using nlohmann::detail::input_format_t;
using nlohmann::detail::json_sax_dom_parser;
using proxy = proxy_iterator<std::string>;
std::string str1 = "[1]";
std::string str2 = "[2]";
std::string str = str1 + str2;
auto first = str.begin();
auto last = str.end();
json j;
json_sax_dom_parser<json, nlohmann::detail::string_input_adapter_type> sax(j, true);
CHECK(json::sax_parse(proxy(first), proxy(last), &sax,
input_format_t::json, false));
CHECK(j.dump() == str1);
CHECK(std::string(first, last) == str2);
}
}
// these cases are required for 100% line coverage
SECTION("error cases")
{
SECTION("case 1")
{
std::array<std::uint8_t, 9> v = {{'\"', 'a', 'a', 'a', 'a', 'a', 'a', '\\', 'u'}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(10)"}));
}
SECTION("case 2")
{
std::array<std::uint8_t, 10> v = {{'\"', 'a', 'a', 'a', 'a', 'a', 'a', '\\', 'u', '1'}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(11)"}));
}
SECTION("case 3")
{
std::array<std::uint8_t, 17> v = {{'\"', 'a', 'a', 'a', 'a', 'a', 'a', '\\', 'u', '1', '1', '1', '1', '1', '1', '1', '1'}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(18)"}));
}
SECTION("case 4")
{
std::array<std::uint8_t, 17> v = {{'\"', 'a', 'a', 'a', 'a', 'a', 'a', 'u', '1', '1', '1', '1', '1', '1', '1', '1', '\\'}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(18)"}));
}
SECTION("case 5")
{
std::array<std::uint8_t, 3> v = {{'\"', 0x7F, 0xC1}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(3)"}));
}
SECTION("case 6")
{
std::array<std::uint8_t, 4> v = {{'\"', 0x7F, 0xDF, 0x7F}};
json _;
CHECK_THROWS_WITH_AS(_ = json::parse(std::begin(v), std::end(v)), "[json.exception.parse_error.101] parse error at line 1, column 4: syntax error while parsing value - invalid string: ill-formed UTF-8 byte; last read: '\"\x7f\xdf\x7f'", json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(4)"}));
}
SECTION("case 7")
{
std::array<std::uint8_t, 4> v = {{'\"', 0x7F, 0xDF, 0xC0}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(4)"}));
}
SECTION("case 8")
{
std::array<std::uint8_t, 4> v = {{'\"', 0x7F, 0xE0, 0x9F}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(4)"}));
}
SECTION("case 9")
{
std::array<std::uint8_t, 4> v = {{'\"', 0x7F, 0xEF, 0xC0}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(4)"}));
}
SECTION("case 10")
{
std::array<std::uint8_t, 4> v = {{'\"', 0x7F, 0xED, 0x7F}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(4)"}));
}
SECTION("case 11")
{
std::array<std::uint8_t, 4> v = {{'\"', 0x7F, 0xF0, 0x8F}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(4)"}));
}
SECTION("case 12")
{
std::array<std::uint8_t, 4> v = {{'\"', 0x7F, 0xF0, 0xC0}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(4)"}));
}
SECTION("case 13")
{
std::array<std::uint8_t, 4> v = {{'\"', 0x7F, 0xF3, 0x7F}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(4)"}));
}
SECTION("case 14")
{
std::array<std::uint8_t, 4> v = {{'\"', 0x7F, 0xF3, 0xC0}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(4)"}));
}
SECTION("case 15")
{
std::array<std::uint8_t, 4> v = {{'\"', 0x7F, 0xF4, 0x7F}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"parse_error(4)"}));
}
SECTION("case 16")
{
std::array<std::uint8_t, 6> v = {{'{', '\"', '\"', ':', '1', '1'}};
json _;
CHECK_THROWS_AS(_ = json::parse(std::begin(v), std::end(v)), json::parse_error&);
CHECK(!json::accept(std::begin(v), std::end(v)));
json j_error;
CHECK_NOTHROW(j_error = json::parse(std::begin(v), std::end(v), nullptr, false));
CHECK(j_error.is_discarded());
SaxEventLogger l;
CHECK(!json::sax_parse(std::begin(v), std::end(v), &l));
CHECK(l.events.size() == 4);
CHECK(l.events == std::vector<std::string>(
{
"start_object()", "key()", "number_unsigned(11)",
"parse_error(7)"
}));
}
}
}
SECTION("ignoring byte-order marks")
{
std::string bom = "\xEF\xBB\xBF";
SECTION("BOM only")
{
json _;
CHECK_THROWS_WITH_AS(_ = json::parse(bom), "[json.exception.parse_error.101] parse error at line 1, column 4: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::parse(std::istringstream(bom)), "[json.exception.parse_error.101] parse error at line 1, column 4: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal", json::parse_error&);
SaxEventLogger l;
CHECK(!json::sax_parse(bom, &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>(
{
"parse_error(4)"
}));
}
SECTION("BOM and content")
{
CHECK(json::parse(bom + "1") == 1);
CHECK(json::parse(std::istringstream(bom + "1")) == 1);
SaxEventLogger l1;
SaxEventLogger l2;
CHECK(json::sax_parse(std::istringstream(bom + "1"), &l1));
CHECK(json::sax_parse(bom + "1", &l2));
CHECK(l1.events.size() == 1);
CHECK(l1.events == std::vector<std::string>(
{
"number_unsigned(1)"
}));
CHECK(l2.events.size() == 1);
CHECK(l2.events == std::vector<std::string>(
{
"number_unsigned(1)"
}));
}
SECTION("2 byte of BOM")
{
json _;
CHECK_THROWS_WITH_AS(_ = json::parse(bom.substr(0, 2)), "[json.exception.parse_error.101] parse error at line 1, column 3: syntax error while parsing value - invalid BOM; must be 0xEF 0xBB 0xBF if given; last read: '\xEF\xBB'", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::parse(std::istringstream(bom.substr(0, 2))), "[json.exception.parse_error.101] parse error at line 1, column 3: syntax error while parsing value - invalid BOM; must be 0xEF 0xBB 0xBF if given; last read: '\xEF\xBB'", json::parse_error&);
SaxEventLogger l1;
SaxEventLogger l2;
CHECK(!json::sax_parse(std::istringstream(bom.substr(0, 2)), &l1));
CHECK(!json::sax_parse(bom.substr(0, 2), &l2));
CHECK(l1.events.size() == 1);
CHECK(l1.events == std::vector<std::string>(
{
"parse_error(3)"
}));
CHECK(l2.events.size() == 1);
CHECK(l2.events == std::vector<std::string>(
{
"parse_error(3)"
}));
}
SECTION("1 byte of BOM")
{
json _;
CHECK_THROWS_WITH_AS(_ = json::parse(bom.substr(0, 1)), "[json.exception.parse_error.101] parse error at line 1, column 2: syntax error while parsing value - invalid BOM; must be 0xEF 0xBB 0xBF if given; last read: '\xEF'", json::parse_error&);
CHECK_THROWS_WITH_AS(_ = json::parse(std::istringstream(bom.substr(0, 1))), "[json.exception.parse_error.101] parse error at line 1, column 2: syntax error while parsing value - invalid BOM; must be 0xEF 0xBB 0xBF if given; last read: '\xEF'", json::parse_error&);
SaxEventLogger l1;
SaxEventLogger l2;
CHECK(!json::sax_parse(std::istringstream(bom.substr(0, 1)), &l1));
CHECK(!json::sax_parse(bom.substr(0, 1), &l2));
CHECK(l1.events.size() == 1);
CHECK(l1.events == std::vector<std::string>(
{
"parse_error(2)"
}));
CHECK(l2.events.size() == 1);
CHECK(l2.events == std::vector<std::string>(
{
"parse_error(2)"
}));
}
SECTION("variations")
{
// calculate variations of each byte of the BOM to make sure
// that the BOM and only the BOM is skipped
for (int i0 = -1; i0 < 2; ++i0)
{
for (int i1 = -1; i1 < 2; ++i1)
{
for (int i2 = -1; i2 < 2; ++i2)
{
// debug output for the variations
CAPTURE(i0)
CAPTURE(i1)
CAPTURE(i2)
std::string s;
s.push_back(static_cast<char>(bom[0] + i0));
s.push_back(static_cast<char>(bom[1] + i1));
s.push_back(static_cast<char>(bom[2] + i2));
if (i0 == 0 && i1 == 0 && i2 == 0)
{
// without any variation, we skip the BOM
CHECK(json::parse(s + "null") == json());
CHECK(json::parse(std::istringstream(s + "null")) == json());
SaxEventLogger l;
CHECK(json::sax_parse(s + "null", &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>(
{
"null()"
}));
}
else
{
// any variation is an error
json _;
CHECK_THROWS_AS(_ = json::parse(s + "null"), json::parse_error&);
CHECK_THROWS_AS(_ = json::parse(std::istringstream(s + "null")), json::parse_error&);
SaxEventLogger l;
CHECK(!json::sax_parse(s + "null", &l));
CHECK(l.events.size() == 1);
if (i0 != 0)
{
CHECK(l.events == std::vector<std::string>(
{
"parse_error(1)"
}));
}
else if (i1 != 0)
{
CHECK(l.events == std::vector<std::string>(
{
"parse_error(2)"
}));
}
else
{
CHECK(l.events == std::vector<std::string>(
{
"parse_error(3)"
}));
}
}
}
}
}
}
SECTION("preserve state after parsing")
{
std::istringstream s(bom + "123 456");
json j;
j << s;
CHECK(j == 123);
j << s;
CHECK(j == 456);
}
}
SECTION("SAX and early abort")
{
std::string const s = R"([1, ["string", 43.12], null, {"key1": true, "key2": false}])";
SaxEventLogger default_logger;
SaxEventLoggerExitAfterStartObject exit_after_start_object;
SaxEventLoggerExitAfterKey exit_after_key;
SaxEventLoggerExitAfterStartArray exit_after_start_array;
json::sax_parse(s, &default_logger);
CHECK(default_logger.events.size() == 14);
CHECK(default_logger.events == std::vector<std::string>(
{
"start_array()", "number_unsigned(1)", "start_array()",
"string(string)", "number_float(43.12)", "end_array()", "null()",
"start_object()", "key(key1)", "boolean(true)", "key(key2)",
"boolean(false)", "end_object()", "end_array()"
}));
json::sax_parse(s, &exit_after_start_object);
CHECK(exit_after_start_object.events.size() == 8);
CHECK(exit_after_start_object.events == std::vector<std::string>(
{
"start_array()", "number_unsigned(1)", "start_array()",
"string(string)", "number_float(43.12)", "end_array()", "null()",
"start_object()"
}));
json::sax_parse(s, &exit_after_key);
CHECK(exit_after_key.events.size() == 9);
CHECK(exit_after_key.events == std::vector<std::string>(
{
"start_array()", "number_unsigned(1)", "start_array()",
"string(string)", "number_float(43.12)", "end_array()", "null()",
"start_object()", "key(key1)"
}));
json::sax_parse(s, &exit_after_start_array);
CHECK(exit_after_start_array.events.size() == 1);
CHECK(exit_after_start_array.events == std::vector<std::string>(
{
"start_array()"
}));
}
SECTION("JSON Lines")
{
SECTION("Example file")
{
std::stringstream ss;
ss << R"({"name": "Gilbert", "wins": [["straight", "7♣"], ["one pair", "10♥"]]}
{"name": "Alexa", "wins": [["two pair", "4♠"], ["two pair", "9♠"]]}
{"name": "May", "wins": []}
{"name": "Deloise", "wins": [["three of a kind", "5♣"]]}
)";
std::string line;
int object_count = 0;
while (std::getline(ss, line))
{
++object_count;
CHECK(json::accept(line));
}
CHECK(object_count == 4);
}
SECTION("Example file without trailing newline")
{
std::stringstream ss;
ss << R"({"name": "Gilbert", "wins": [["straight", "7♣"], ["one pair", "10♥"]]}
{"name": "Alexa", "wins": [["two pair", "4♠"], ["two pair", "9♠"]]}
{"name": "May", "wins": []}
{"name": "Deloise", "wins": [["three of a kind", "5♣"]]})";
std::string line;
int object_count = 0;
while (std::getline(ss, line))
{
++object_count;
CHECK(json::accept(line));
}
CHECK(object_count == 4);
}
}
SECTION("stream position after extraction without JSON_PRECISE_STREAM_POSITION (#5340)")
{
// By default, the character that terminates a number is consumed, so
// the stream is left one byte too far after a number (and only after a
// number). JSON_PRECISE_STREAM_POSITION changes this; see
// unit-precise-stream-position.cpp. These checks pin the default.
const auto remaining = [](std::istream & is)
{
return std::string(std::istreambuf_iterator<char>(is), std::istreambuf_iterator<char>());
};
SECTION("the character after a number is consumed")
{
std::istringstream ss("1true");
json j;
ss >> j;
CHECK(j == 1);
CHECK(remaining(ss) == "rue");
}
SECTION("the character after other values is not consumed")
{
std::istringstream ss("[1]true");
json j;
ss >> j;
CHECK(j == json::parse("[1]"));
CHECK(remaining(ss) == "true");
}
SECTION("comma-separated numbers can be read one by one")
{
std::istringstream ss("1,2,3");
json j1, j2, j3;
ss >> j1 >> j2 >> j3;
CHECK(j1 == 1);
CHECK(j2 == 2);
CHECK(j3 == 3);
}
SECTION("std::getline after a number skips the line break")
{
std::istringstream ss("42\nfoo");
json j;
std::string line;
ss >> j;
std::getline(ss, line);
CHECK(j == 42);
CHECK(line == "foo");
}
}
// build with C++20
// JSON_HAS_CPP_20
#if defined(__cpp_char8_t)
SECTION("Using _json with char8_t literals #4945")
{
// Regular narrow string literal
const auto j1 = R"({"key": "value", "num": 42})"_json;
CHECK(j1["key"] == "value");
CHECK(j1["num"] == 42);
// UTF-8 prefixed literal (C++20 and later);
// MSVC may not set /utf-8, so we need to check
if (check_utf8())
{
const auto j2 = u8R"({"emoji": "😀", "msg": "hello"})"_json;
CHECK(j2["emoji"] == "😀");
CHECK(j2["msg"] == "hello");
}
const auto j3 = u8R"({"key": "value", "num": 42})"_json;
CHECK(j3["key"] == "value");
CHECK(j3["num"] == 42);
}
#endif
}
// select the types to test - char8_t is only available since C++20 if and only
// if __cpp_char8_t is defined.
#define TYPE_LIST(...) __VA_ARGS__
#if defined(__cpp_char8_t) && (__cpp_char8_t >= 201811L)
#define ASCII_TYPES TYPE_LIST(char, wchar_t, char16_t, char32_t, char8_t)
#else
#define ASCII_TYPES TYPE_LIST(char, wchar_t, char16_t, char32_t)
#endif
TEST_CASE_TEMPLATE("deserialization of different character types (ASCII)", T, ASCII_TYPES) // NOLINT(readability-math-missing-parentheses, bugprone-throwing-static-initialization)
{
std::vector<T> const v = {'t', 'r', 'u', 'e'};
CHECK(json::parse(v) == json(true));
CHECK(json::accept(v));
SaxEventLogger l;
CHECK(json::sax_parse(v, &l));
CHECK(l.events.size() == 1);
CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
}
TEST_CASE_TEMPLATE("deserialization of different character types (UTF-8)", T, char, unsigned char, std::uint8_t) // NOLINT(readability-math-missing-parentheses, bugprone-throwing-static-initialization)
{
// a star emoji
std::vector<T> const v = {'"', static_cast<T>(0xe2u), static_cast<T>(0xadu), static_cast<T>(0x90u), static_cast<T>(0xefu), static_cast<T>(0xb8u), static_cast<T>(0x8fu), '"'};
CHECK(json::parse(v).dump(-1, ' ', true) == "\"\\u2b50\\ufe0f\"");
CHECK(json::accept(v));
SaxEventLogger l;
CHECK(json::sax_parse(v, &l));
CHECK(l.events.size() == 1);
}
TEST_CASE_TEMPLATE("deserialization of different character types (UTF-16)", T, char16_t) // NOLINT(readability-math-missing-parentheses, bugprone-throwing-static-initialization)
{
// a star emoji
std::vector<T> const v = {static_cast<T>('"'), static_cast<T>(0x2b50), static_cast<T>(0xfe0f), static_cast<T>('"')};
CHECK(json::parse(v).dump(-1, ' ', true) == "\"\\u2b50\\ufe0f\"");
CHECK(json::accept(v));
SaxEventLogger l;
CHECK(json::sax_parse(v, &l));
CHECK(l.events.size() == 1);
}
TEST_CASE_TEMPLATE("deserialization of different character types (UTF-32)", T, char32_t) // NOLINT(readability-math-missing-parentheses, bugprone-throwing-static-initialization)
{
// a star emoji
std::vector<T> const v = {static_cast<T>('"'), static_cast<T>(0x2b50), static_cast<T>(0xfe0f), static_cast<T>('"')};
CHECK(json::parse(v).dump(-1, ' ', true) == "\"\\u2b50\\ufe0f\"");
CHECK(json::accept(v));
SaxEventLogger l;
CHECK(json::sax_parse(v, &l));
CHECK(l.events.size() == 1);
}
Reference in New Issue View Git Blame Copy Permalink