mirror of https://github.com/nlohmann/json.git synced 2026-09-30 14:24:59 +01:00

Files

T

Niels Lohmann e94e164b07 Add BON8 support

Add to_bon8/from_bon8 and input_format_t::bon8 for BON8, a binary format
that uses the byte values that cannot begin a UTF-8 character as type
markers, so strings need no length prefix. It is the most compact of the
supported binary formats on the benchmark files.

The reader is non-recursive like the other binary readers. A string ends
at the first byte that cannot continue it, so the reader hands the one or
two bytes it reads past a string back to the value that follows. The
writer produces the canonical representation of the specification, except
for NFC normalization; its output is identical to that of the reference
implementation (HikoGUI) on all files of the test data.

The round-trip tests need the .bon8 files of json_test_data 3.2.0.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

2026-09-24 22:53:08 +02:00

3.1 KiB

Raw Blame History

Fuzz testing

Each parser of the library (JSON, BJData, BON8, BSON, CBOR, MessagePack, and UBJSON) can be fuzz tested. Currently, libFuzzer and afl++ are supported.

Corpus creation

For most effective fuzzing, a corpus should be provided. A corpus is a directory with some simple input files that cover several features of the parser and is hence a good starting point for mutations.

TEST_DATA_VERSION=3.2.0
wget https://github.com/nlohmann/json_test_data/archive/refs/tags/v$TEST_DATA_VERSION.zip
unzip v$TEST_DATA_VERSION.zip
rm v$TEST_DATA_VERSION.zip
for FORMAT in json bjdata bon8 bson cbor msgpack ubjson
do
  rm -fr corpus_$FORMAT
  mkdir corpus_$FORMAT
  find json_test_data-$TEST_DATA_VERSION -size -5k -name "*.$FORMAT" -exec cp "{}" "corpus_$FORMAT" \;
done
rm -fr json_test_data-$TEST_DATA_VERSION

The generated corpus can be used with both libFuzzer and afl++. The remainder of this documentation assumes the corpus directories have been created in the tests directory.

libFuzzer

To use libFuzzer, you need to pass -fsanitize=fuzzer as FUZZER_ENGINE. In the tests directory, call

make fuzzers FUZZER_ENGINE="-fsanitize=fuzzer"

This creates a fuzz tester binary for each parser that supports these command line options.

In case your default compiler is not a Clang compiler that includes libFuzzer (Clang 6.0 or later), you need to set the CXX variable accordingly. Note the compiler provided by Xcode (AppleClang) does not contain libFuzzer. Please install Clang via Homebrew calling brew install llvm and add CXX=$(brew --prefix llvm)/bin/clang to the make call:

make fuzzers FUZZER_ENGINE="-fsanitize=fuzzer" CXX=$(brew --prefix llvm)/bin/clang

Then pass the corpus directory as command-line argument (assuming it is located in tests):

./parse_cbor_fuzzer corpus_cbor

The fuzzer should be able to run indefinitely without crashing. In case of a crash, the tested input is dumped into a file starting with crash-.

afl++

To use afl++, you need to pass -fsanitize=fuzzer as FUZZER_ENGINE. It will be replaced by a libAFLDriver.a to re-use the same code written for libFuzzer with afl++. Furthermore, set afl-clang-fast++ as compiler.

CXX=afl-clang-fast++ make fuzzers FUZZER_ENGINE="-fsanitize=fuzzer"

Then the fuzzer is called like this in the tests directory:

afl-fuzz -i corpus_cbor -o out  -- ./parse_cbor_fuzzer

The fuzzer should be able to run indefinitely without crashing. In case of a crash, the tested input is written to the directory out.

OSS-Fuzz

The library is further fuzz-tested 24/7 by Google's OSS-Fuzz project. It uses the same fuzzers target as above and also relies on the FUZZER_ENGINE variable. See the used build script for more information.

In case the build at OSS-Fuzz fails, an issue will be created automatically.

3.1 KiB Raw Blame History