* finished fuzzer and added fuzzverter util
* fixed typo
* renamed cmd line params
* fixed libproto_mutator download path
* small fixes
* small fixes
* small fixes
* renamed sample corpus proto
* small fixes
* try building on github with LIBCPP_DEBUG enabled
* temporarily marked proto fuzzer as a manual test
* code review
* use a dedicated proto-fuzzer feature to work around LIBCPP_DEBUG=1 crash in proto code
* code review comments, added README.md
* minor fixes to the text
* Update bazel/cc_toolchains/clang_cc_toolchain_config.bzl
Co-authored-by: Jon Meow <jperkins@google.com>
* use Carbon source representation for "empty Main()" instead of text format proto representation
* fixed typo
* made FuzzerUtil produce the full carbon source (proto converted + Main if needed) to decrease code duplication a bit
* typo
* switched to text proto format per code review
* Update executable_semantics/prelude.h
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/README.md
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/README.md
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/README.md
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/README.md
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/README.md
Co-authored-by: Jon Meow <jperkins@google.com>
* review comments
* removed unnecessary file mode variables
* Update executable_semantics/fuzzing/fuzzverter.cpp
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/README.md
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/README.md
Co-authored-by: Jon Meow <jperkins@google.com>
* code review comments
* Update executable_semantics/syntax/BUILD
Co-authored-by: Jon Meow <jperkins@google.com>
* buildifier
Co-authored-by: Jon Meow <jperkins@google.com>
* initial fuzzer proto
* visibility change
* use newer protocol buffer version which has the defs.bzl bug fixed
* Adjusted fix_cc_deps to work with protobuf external repo
* explicitly load rules_cc to avoid a frozenset bug in the version loaded by protobuf
* use explit deps, use llvm's zlib
* restored cxx settings
* deps change
* Cleaned up WORKSPACE and changed the test to read carbon sources from testdata
* updated comment
* adapted to new ErrorOr return value
* proto buffer 3.19.2 -> 3.19.4
* changed comment
* Apply suggestions from code review
Co-authored-by: Jon Meow <jperkins@google.com>
* Apply suggestions from code review
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/BUILD
Co-authored-by: Jon Meow <jperkins@google.com>
* addressed review comments
* updated Unimplemented error message
* Addressed review comments
* more review comments
* switched to loading protobuf via rules_proto()
* Ignore protobuf headers in fix_cc_deps.py until the script supports alias rules
* renamed repeated proto fields to be plural
* added @zlib to check_non_test_cc_deps
* Update common/fuzzing/BUILD
Co-authored-by: Jon Meow <jperkins@google.com>
* review comments
* set is_omitted_expression for return value
Co-authored-by: Jon Meow <jperkins@google.com>
* Initial implementation of block string literals following lexical_conventions/string_literals.md.
Enabled yyinput() in flex to implement parsing.
Added ParseBlockStringLiteral() helper to handler further transformations such as indenting.
Modified formar_grammar to support single-quoted strings to prevent a failure on lexer.lpp.
* Fixed _find_string_end quote parameter type int -> str.
* Update executable_semantics/syntax/BUILD
Co-authored-by: Geoff Romer <gromer@google.com>
* Addressed code review comments - split table-drived test into individual tests, renamed constants to match style guide.
* Addressed code review comments -- using EXPECT_THAT_EXPECTED() in tests, lexer comments and code cleanup.
Co-authored-by: Geoff Romer <gromer@google.com>
This avoids needing to have nearly as many rules here which should
reduce its churn.
I've tested that this reaches the exact same set of transitive
dependencies.
Note, only the last commit here is new.
This moves over to the vanilla upstream GoogleTest pulled in the more
expected manner with Bazel. It also adds Abseil and Google Benchmark
libraries in the same fashion (there are cross dependencies here).
As part of this, also introduce a dependency check test that can enforce
basic layering of dependencies. For example, this lets us ensure that
non-test Carbon code only depends on LLVM and Clang despite having other
libraries available. There remains some cleanup to improve the way these
dependency tests work, but this at least ensures we don't regress.
I've also provided workarounds to allow both Carbon code and LLVM code
to freely be used with GoogleTest (and other `std::ostream` based
output code). This is done by extending the code in
`//common/ostream.h`. One downside is that it requires opening the
`llvm` namespace and adding an ADL_found overload there. I think on
balance this is still a win and doesn't make me too nervous.
The new version of GoogleTest requires printing more often from matchers
and so I've also added several printing routines to types that
previously didn't require them. Otherwise, most of the updates are just
using the more conventional upstream style of including the headers and
adding `ostream.h` where it is needed.
I did consider moving code over to use `std::ostream` instead of LLVM's
`raw_ostream`, but the advantages of not doing virtual dispatch still
seem significant, and it also seems good to retain access to LLVM's
formatting utilities built around `raw_ostream` given that we can't pull
arbitrary dependencies into Carbon code outside of test code.
All of this was slightly motivated by requests for newer features in
GoogleTest, but much more-so by my desire to have access to Google
Benchmark and Abseil when writing benchmarks. For example, using
Abseil's random number generator seems extremely helpful when generating
inputs for benchmarks. The growing dependencies between these packages
further motivated me to just pull them all in and ensure they worked
well.
Arguably missed in #769
Note, this is reminding me we have more class members to rename for `_`, but I felt it's best to use the new naming instead of adding more to clean up.
Revives BisonWrap because this seems a reasonable use of it (avoiding the need to have an std::optional or pointer for Alternative, both of which I thought could be unclear about the intent).
I'm seeing if I can upstream thundergolfer/bazel-mypy-integration#43, but we can also point at my fork for the time being.
This should resolve conflicts with mypy treating imports as non-hermetic, creating inconsistent behavior if packages are/aren't installed locally.
Doesn't add much logic, only takes advantage of parser structure for the ordering enforcement.
Note import_nonexistent tests should probably fail, but writing import tests needs a chain of functionality, and I figured I'd just start adding some to validate the syntax (not adding existent imports because that'd require multi-file structure).
Along with #789 this addresses most of #769 although global_arena is still a TODO (that's widespread and overlaps with other changes so I wanted to do it after these are in).
Per https://github.com/bazelbuild/rules_cc this still isn't necessary. There's no build-time enforcement, so usage is inconsistent/incorrect. Rather than letting this linger, remove it pending Bazel tooling enforcing it.
It was in my mind to add String in order to support libraries in `package`. `print` is added in order to have a String go to stdout. I've tried to do `print` in a way that won't be too hard to add other printable types, but it's probably also somewhat optional here -- that is, if desired, I could remove it. But it was a lot easier to doublecheck `\n` behavior with it, and I suspect it'll be helpful in other tests if it supports more value types.
On the side, this also fixes dereferencing in Pattern/Expression Print() calls, which I was noticing printing pointers instead of values. This may be another argument for moving away from passing pointers, since this seems to be a difficult-to-catch error.
Co-authored-by: Geoff Romer <gromer@google.com>
`Pattern` is intended to pilot some changes I would like to apply to all our sum types:
- The alternatives are expressed as derived classes rather than members of a `std::variant`.
- The alternatives are classes in the [style guide sense](https://google.github.io/styleguide/cppguide.html#Structs_vs._Classes), meaning they can have invariants, but can't have public data members.
- Creating an object is expressed using a constructor rather than a factory function.
- Accessing an alternative is expressed as a cast (using LLVM's RTTI system) rather than `std::get` or a `Get` method.
Co-authored-by: Jon Meow <46229924+jonmeow@users.noreply.github.com>
- Switch code to llvm::raw_ostream as part of standardizing output forms.
- Preferring llvm::raw_ostream over std::ostream because other tooling code should be expected to rely on llvm more closely, and an overall preference towards library consistency.
- There are a couple spots in syntax/ that still use std streams, but I'd prefer to take a separate PR to see how best to address those.
- std::boolalpha doesn't work with llvm, so I've implemented equivalent in a couple places (not enough that it felt like worth making a helper function).
- Implement Print(ostream) as consistently as we can, as an instance member.
- This facilitates the use of the common/ostream.h template to provide operators.
- Preferring this approach so that Print is easily accessible via gdb, per suggestion on #executable-semantics.
- Switch code currently calling `type->Print(ostream)` to instead do `ostream << *type`.
- Remove the unused `PrintTypeEnv`, nothing used it and the declaration didn't match the definition.
- Move Heap and Frame/Scope to their own headers.
- Move some functions to more appropriate headers (e.g. CopyValue -> value.h).
- Define a separate Bazel rule for each header/cpp pair.
- Modify PrintValue to not print the frames of a ContinuationValue. This was necessary to break a dependency cycle between PrintValue, PrintFrame, and Action::Print.
Co-authored-by: Jon Meow <46229924+jonmeow@users.noreply.github.com>
In practice, this means that `executable_semantics` will be built and run using the configuration specified on the command line, rather than e.g. always using `-c opt`.
Also fix a bug exposed by this change.
Adds all the necessary machinery to our toolchain and Bazel
configuration to support ASan. This includes ensuring sufficient debug
information is available for backtraces, etc.
As part of ASan, it enables UBSan to catch more basic undefined behavior
in C++. It also enables more complete checking in ASan for lifetime
bugs.
These configs can be enabled in any build mode with `--config=asan`.
They are also enabled by default in `-c fastbuild` where asserts are
also enabled. The goal is to have a single build mode that catches the
overwhelming majority of correctness issues.
Leak checking is part of ASan and finds leaks in `executable_semantics`
code that probably aren't interesting to fix right now. I've disabled
leak checking in the `BUILD` file for the test that showed this --
everything else passed. If more things need this disabled, the same
`BUILD` change should be easily replicated.
If you see unsymbolized backtraces, you may need to either put
`llvm-symbolizer` on your path, or point the `ASAN_SYMBOLIZER_PATH`
environment variable at it. For example, in the project root you could
do something like the following to use the downloaded toolchain's
symbolizer:
```bash export
ASAN_SYMBOLIZER_PATH=$PWD/bazel-clang-toolchain/bin/llvm-symbolizer
```
I'll try to update documentation soon with this as well.
* Move nontrivial logic out of `parser.ypp` into `FieldList`, rename it to `ParenContents`, make it a class, and add tests
* Use a `FieldInitializer` struct instead of `std::pair<std::string, Expression*>` to represent the fields of a tuple
Slowly bringing this into line with Bison's C++ example parser
so we can use strong semantic values for symbols rather than
leaking pointers. First step is to thread a `ParseAndLexContext`
object through the whole syntactic analysis state, like the
example has. In the example, it's called `driver`.
Co-authored-by: Geoff Romer <gromer@google.com>
Distinguishes parts that come from the parser and lexer. It used to be that all
the files were called "syntax*", but lexing and parsing are distinct phases that
are easier to keep track of when distinguished. syntax.yy.cpp being the source
file generated by flex, containing the lexer was particularly confusing, because
the yy tends to indicate it is a yacc/Bison product, and the ".tab." substring,
indicating "tables" is not really useful to the developer.
These names also match up with what Bison's C++ example uses, which will make
the transition easier.
Jon Meow
pk19604014andJon Meow
Jon Meow
Chandler Carruth
Jeremy G. SiekandJon Meow
Dave Abrahams