From the driver's perspective, `FindPreludeFiles` is closely tied to
`compile`. This makes it difficult to refactor commands without
affecting the test dependencies on `FindPreludeFiles`. `InstallPaths`
seems like a decent home since it is responsible for the install
structure.
I'm switching to an `Error` return to allow callers to choose how to
handle it (e.g., in file tests, we typically don't want the direct error
stream).
This teaches our source generation tool to create interesting type
references. This include both referencing a weighted distribution of
explicitly specified types, and referencing types that are being defined
in the generated file.
Generating more interesting explicit types will exercise more of
Carbon's prelude, but because C++ doesn't have an automatic prelude with
fundamental types like `int64_t` or tuples, we include some minimal
headers when generating the C++ analog. This likely makes the comparison
more fair rather than less fair as Carbon's toolchain isn't processing
just the generated source, but also its prelude.
The current set of fixed types is based primarily on the set of types
that the toolchain currently implements and a set that seems reasonably
interesting to exercise for compile time performance. We want to try to
cover things that should be optimized in the toolchain, even if a single
source file might not typically hit all of them.
The weights of everything are completely arbitrary, based on intuition
and some hand inspection of some random source files. There is also an
intentional bias towards non-zero coverage and so the tail is much
larger than it should be in reality. The result is that the weights more
reflect the _priority_ of optimizing compile time than the _observed_
distribution in practice. We can refine the weighting scheme in the
future though, potentially with multiple modes to separate coverage from
maximally representative weights, etc. The goal is just to have a
starting point.
The scheme for referencing the defined types requires some care and
complexity to avoid referencing types before they are defined while
still referencing all of the types defined and ensuring the number of
references is stable even as the order is randomized to avoid fixed
patterns in the source code.
All of this also triggered some minor refactoring of the state used to
generate class definitions in the source generator. There are probably
some good follow-on refactoring opportunities, but I'd prefer to leave
those to future work.
I don't have any tests here because most of how this is observable is
already tested -- the existing tests ensure the file sizes remain
consistent and that the generated code is compiled correctly. But if
folks have any ideas of useful tests here, happy to add them.
---------
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
Consolidates both main libraries into `//testing/base`, and factors out
the exe path handling for benchmarks and unit tests into a common
library to remove duplication. Refactors how that logic is managed to be
cleaner and avoid a confusing bool that came up in code review.
Updates all the tests and benchmarks that use these. I still need to
update other benchmarks to use the same main, but I wanted to keep this
PR somewhat minimal.
This also fixes a bug noticed in passing that the compilation benchmark
didn't have the required dependency on the benchmark library itself,
just the benchmark main library.
The big addition here is a very, very rough and very early skeleton of a
source code generator framework. This builds upon the lexers identifier
synthesis logic, improving on its framework and wiring it up with the
most rudimentary of source file generation. This is just enough to
roughly replicate my "big API file" source code benchmarks.
The source generation works *very* hard to both vary the structure and
content of the source as much as possible while ensuring the same
*total* amount of each construct is in use, from bytes in identifiers to
line breaks, parameters, etc. This lets us generate randomly structure
inputs that should consistently take the exact same amount of total work
to compile.
The complex identifier synthesis logic from the lexer's benchmark is
moved over here and the lexer uses APIs in the source generator for
identifiers. The other source synthesis in the lexer's benchmark isn't
yet moved over, but should likely be slowly absorbed here as it can be
refactored into a more principled and re-usable form. Some bits may stay
of course if they're just too lexer-specific.
Next, this adds a simple end-to-end compile benchmark for the driver
that directly and much more clearly reproduces all the measurements I've
done manually up until now. It should also be easy to extend to more
patterns over time as we add support to the source generator to produce
those patterns.
Last but not least, I've added a tiny CLI to the source generator so
that you can generate source code manually. This is especially nice for
generating demo source code to actually run through the driver or look
at in an editor. The CLI can also generate C++ source code which lets us
do some minimal comparative benchmarking between Carbon and C++/Clang.
There are huge number of TODOs in the source generation framework. This
is going to be a large ongoing effort I suspect.
There are also a bunch of rough edges I've left to try and get this out
for review sooner. I've left TODOs for refactorings that really need to
be done here, but hoping these can maybe be follow-ups. If not, please
flag and I'll try to layer them on here.
Sample compile benchmark output, nicely showing where we are w.r.t. our
goal speeds (2x behind on lex and check, 5x on parse) at least on a
recent AMD server CPU:
```
------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations Lines
------------------------------------------------------------------------------------------------------
BM_CompileAPIFileDenseDecls<Phase::Lex>/256 29420 ns 29419 ns 22860 6.62847M/s
BM_CompileAPIFileDenseDecls<Phase::Lex>/1024 146130 ns 146128 ns 4840 6.69959M/s
BM_CompileAPIFileDenseDecls<Phase::Lex>/4096 601584 ns 601577 ns 1020 6.69573M/s
BM_CompileAPIFileDenseDecls<Phase::Lex>/16384 2547578 ns 2547313 ns 280 6.404M/s
BM_CompileAPIFileDenseDecls<Phase::Lex>/65536 10816591 ns 10816389 ns 80 6.05193M/s
BM_CompileAPIFileDenseDecls<Phase::Lex>/262144 52191320 ns 52189828 ns 20 5.02261M/s
BM_CompileAPIFileDenseDecls<Phase::Parse>/256 101706 ns 101698 ns 6900 1.91745M/s
BM_CompileAPIFileDenseDecls<Phase::Parse>/1024 512161 ns 512162 ns 1380 1.9115M/s
BM_CompileAPIFileDenseDecls<Phase::Parse>/4096 2078426 ns 2078430 ns 340 1.938M/s
BM_CompileAPIFileDenseDecls<Phase::Parse>/16384 8795786 ns 8795583 ns 100 1.85468M/s
BM_CompileAPIFileDenseDecls<Phase::Parse>/65536 35073596 ns 35072973 ns 20 1.86639M/s
BM_CompileAPIFileDenseDecls<Phase::Parse>/262144 151100688 ns 151097370 ns 20 1.73483M/s
BM_CompileAPIFileDenseDecls<Phase::Check>/256 957059 ns 957049 ns 740 203.751k/s
BM_CompileAPIFileDenseDecls<Phase::Check>/1024 1956134 ns 1955985 ns 360 500.515k/s
BM_CompileAPIFileDenseDecls<Phase::Check>/4096 5797864 ns 5797417 ns 120 694.792k/s
BM_CompileAPIFileDenseDecls<Phase::Check>/16384 21219608 ns 21217584 ns 40 768.843k/s
BM_CompileAPIFileDenseDecls<Phase::Check>/65536 96311116 ns 96302334 ns 20 679.734k/s
BM_CompileAPIFileDenseDecls<Phase::Check>/262144 371637963 ns 371609964 ns 20 705.387k/s
```
Lest someone think this is *bad*, the fact that we're already within 2x
of our rather audacious goals makes me quite happy. =D
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This also factors out the code for doing this location from the driver
to a tiny helper library.
The motivation for this is letting tests locate data files like the
prelude or other files needed by the toolchain more easily. A subsequent
PR will use this heavily in the Clang runner and related logic for
example.
I'd kept these in to separate the bazel module update from the BUILD
file changes, then forgot about it. I think all of these can be cleanly
removed now. I think it's something we should clean up for consistency
with the bazel central repository names; I think it's best to reduce
that divergence.
llvm_zlib and llvm_zstd remain because of how llvm depends on the
particular names.
Per their documentation, the `llvm::Initialize*` functions are only
supposed to be called by the main program, not by a library like
toolchain/codegen. Fixes a hang due to a data race in multithreaded
autoupdate.
Add a utility class `Carbon::InitLLVM` to do the common LLVM
initialization shared by all Carbon tools, optionally including
initializing the LLVM targets. Because the LLVM targets add a lot of
binary size, only initialize them for binaries that opt in by depending
on a new target `//common:all_llvm_targets`.
Also fix `//explorer:file_test` and `//explorer:file_test.trace` to
share a binary rather than linking an identical binary twice.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Running `bazel test //...` reported:
```
Test execution time outside of range for MODERATE tests.
Consider setting timeout="short" or size="small".
```
This change adds size="small" to avoid such warnings being reported.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
The main motivation for this is to get python loads in using the
`native-py` lint fix. However, enabling that made me wonder, maybe we
should fix in general?
`native-cc` is delayed, but not wholly cancelled (and `native-py`
picking up might indicate `native-cc` won't be too far behind). There's
also some automated fixes for `.append` and dict sorting -- this felt
okay to me, maybe not something to eagerly add but probably not worth
stopping buildifier from fixing (I've noticed the warnings in the past
and had been ignoring them).
Running everything does mean that load orders are sorted automatically
now, which I think is a positive. Most generally, I think these fixes
aren't _harmful_, and having them done automatically seems beneficial:
my biggest concern about `native-py` and `native-cc` was actually that
regressions wouldn't be caught, but this addresses that issue
automatically.
Jon Ross-Perkins
Chandler Carruthandjosh11b
David Blaikie
Jonathan B. CoeandChandler Carruth