Summary:
An `llvm::APInt` is always treated as a signed value by `operator<<`;
check [1]. This resulted in printing incorrect values for tokens that
have their MSB set to 1. For example, a value 9 would be printed as -7
since its `APInt` object would be 4-bits wide. However, integer literals
are always tokenized without the sign character so it is safe to treat
the values as unsigned for printing pruposes.
[1] https://llvm.org/doxygen/APInt_8h_source.html
Co-authored-by: ergawy <kareem.ergawy@guardsquare.com>
Summary:
Fixes a small compilation error where an llvm::Error variable was being
returned by copy rather than by move. The llvm::Error copy constructor
is deleted.
Co-authored-by: ergawy <kareem.ergawy@guardsquare.com>
MAP_POPULATE is a Linux mmap flag that optionally assists read-ahead on
the mapping. It is nonstandard and omission does not affect the
underlying mapping, so fall back to not including the flag, rather than
the reverse.
This is how I'm interpreting discussion:
- Basic elements are getting set to an ID.
- SetName exists to assign a name (which can then be referred to later with an identifier expression) to an ID.
- Expressions are broken down into a series of operations which operate on IDs.
So with something like the last test:
```
fn Main() { return 12 + 34; }
```
This becomes:
```
Function(%0,
{IntegerLiteral(%3, 12),
IntegerLiteral(%2, 34),
BinaryOperator(%1, +, %3, %2),
Return(%1),
})
SetName(`Main`, %0)
```
Note I'm treating blocks as fairly equal to the top of a file now, and basically eliminating boundaries between things. That's because we have discussed also supporting code like:
```
fn Foo() {
fn Bar() {}
Bar();
}
```
Here a declaration of a function is occurring inside a code block, so it felt like eliminating the difference was the best choice.
I know you'd commented on the separation of nodes to individual files before; I still think we're going to have a lot of different types of nodes, and so separating them out into individual files makes them easier to browse.
Working on toolchain semantics:
- SemanticsIR is set up as a container for the semantic tree.
- SemanticsIRFactory builds the tree, with separate transformations for each ParseNodeKind.
- ParseSubtreeConsumer is a helper for transforming a ParseTree::Node's children, managing size/nodes to prevent errors.
- The nodes subdirectory contains SemanticIR nodes.
- MetaNode is used to represent nodes which have "sub-classes": Statements, Declarations, and Expressions.
- MetaNodeBlock is used to represent nodes which exist together in a block with name lookup: Statements and Declarations (not Expressions).
This is traversing children first in order to address the RPO format of ParseTree. This means that when lists are formed, they're reversed to be in code-order (`FixReverseOrdering`).
This is still very much incomplete -- the main intent at present is to demonstrate structure.
My intent is to add CARBON_ bas a prefix, and this makes that easier by creating a tool for auto-fixing guards in general.
string_literal is a manual fix -- it had no guard and I didn't automate that (technically I think I could, especially by enforcing the file header/footer, but it didn't feel quite worth it to me).
I'm doing this to avoid macro name conflicts, following https://google.github.io/styleguide/cppguide.html#Preprocessor_Macros: "If you do export a macro from a header, it must have a globally unique name. To achieve this, it must be named with a prefix consisting of your project's namespace name (but upper case)."
Commands run:
```
sed -i 's/\(DCHECK\|CHECK\|FATAL\|MAKE_UNIQUE_NAME\|MAKE_UNIQUE_NAME_IMPL\|RAW_EXITING_STREAM\|RETURN_IF_ERROR\|RETURN_IF_ERROR_IMPL\|ASSIGN_OR_RETURN\|ASSIGN_OR_RETURN_IMPL\|DIAGNOSTIC_KIND\|RETURN_IF_STACK_LIMITED\)(/CARBON_\1(/g' $(git ls-files *.cpp *.h *.lpp *.ypp *.def ':!third_party')
sed -i 's/#undef DIAGNOSTIC_KIND/#undef CARBON_DIAGNOSTIC_KIND/' toolchain/diagnostics/diagnostic_registry.def
```
Note this isn't *quite* everything, but it's intended to be a large pass at everything:
```
╚╡git grep '#define ' *.cpp *.h *.lpp *.ypp *.def ':!third_party' | grep -v '#define CARBON' | grep -v _H_
explorer/syntax/lexer.lpp: #define YY_USER_ACTION \
explorer/syntax/lexer.lpp: #define SIMPLE_TOKEN(name) \
explorer/syntax/lexer.lpp: #define ARG_TOKEN(name, arg) \
explorer/syntax/parse_and_lex_context.h:#define YY_DECL \
migrate_cpp/cpp_refactoring/var_decl.cpp:#define ABSTRACT_TYPE(Class, Base)
migrate_cpp/cpp_refactoring/var_decl.cpp:#define TYPE(Class, Base) \
```
We may in particular want to do a pass to clean up #ifdef guards and make them be CARBON_ rooted.
Note the use of an enum allows for the operator to be defined for formatting. This reduces the custom formatting required, which is a simplification carrying over to error handling changes I'm working on too.
Also, I kind of feel the code's now clearer about what's supported; I'm dropping CHECKs that seemed redundant with the enum.
Note I've added a few TODOs, particularly that multi-line strings should only consume until the dedent.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
There's currently a bug with empty files, in that it initializes SourceBuffer with an invalid StringRef that results in a crash. That got me looking at the std::optional TODO, but the issue is that there are really three states:
- Buffered
- mmapped (not buffered)
- Moved out of (no longer initialized)
Technically an optional could work if we initialize the buffer on move out, indicating the mmap is gone. But the mode setup felt better to me.
And then this also adds the size check. Which is really how I started looking at this.
I was mainly looking at keywords trying to figure out what needs work and the amount to which it doesn't reflect the design confused me (including some things that we've decided not to include, and some things I'm not aware of discussion about). I figured this cleanup would at least make it somewhat clearer why things are in there.
I'm treating https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/lexical_conventions/words.md as canonical, with `_` and `xor` as presumably deliberate exceptions. Similarly avoiding symbol tokens because I assume you'll push proposals for the difference.
I dropped the `Keyword` qualifier because `is` makes `IsKeyword` a name conflict, and dropping the qualifier seemed like the more consistent solution (it doesn't do `AmpSymbol`, after all). If we need clarity I might lean towards a separate namespace to avoid naming conflicts.
This switches to generating per-file targets such as `//executable_semantics/testdata:tuple/equality_false.carbon.test`, instead of one test covering all files. Essentially this relies on bazel instead of lit to handle test parallelization.
Kareem Ergawyandergawy
3405691582
Josh Soref
prprprpony
cui fliter
pk19604014
Jon Meow