Commit Graph
42 Commits
Author SHA1 Message Date
Richard SmithandChristopher Di Bella 9bcee64b32 Add some encapsulation to CppDomain (#7579)
Start tracking the domain within `CppContext`s instead of having them
duplicate its fields. This allows us to remove the shared ownership of
the clang parser.

---------

Co-authored-by: Christopher Di Bella <cjdb.ns@gmail.com>
2026-07-31 18:08:13 +00:00
Richard Smith 7d89ac98c7 Add a flag to build a single ASTContext shared across all compilations (#7567)
Instead of building one Clang `ASTContext` per compilation, the
`--share-cpp-ast` flag causes us to build a single `ASTContext` and
share it across all contexts. One new abstraction is added: `CppDomain`
represents the Carbon-side view of a Clang AST that might be shared
across multiple `SemIR::File`s. This object owns the Clang instance and
the AST.

For now, we have no isolation between the C++ state exposed to different
Carbon compilations, and we have no multiplexing of generated LLVM IR
from C++ into different Carbon compilations, so the mode is not usable
yet. The plan is to keep it behind a flag until it's ready.

Assisted-by: Gemini via Antigravity
2026-07-28 00:19:40 +00:00
Richard Smith be70c092fa Add a text mode for fingerprinting. (#7231)
The intent is to add visibility into how the fingerprint is computed, so
that fingerprinting issues and mangling collisions can be more readily
understood and fixed.

Assisted-by: Gemini via Antigravity
2026-05-19 23:16:51 +00:00
Jon Ross-Perkins 45ca3d28f5 Drop "diagnostic" from some filenames in the "diagnostics" folder (#6686)
Mainly because "sorting_diagnostic_consumer" is legacy, since
`SortingDiagnosticConsumer` became `SortingConsumer`. Also better
reflecting contents of these files.

Where I'm not renaming, I'm less positive about dropping "diagnostics"
from "file_diagnostics" and "null_diagnostics" (which contain both a
consumer and emitter, and "null.h" seems like poor naming), so not doing
that here. Also "diagnostic.h" contains `struct Diagnostic`, so is a
decent fit.

Assisted-by: Google Antigravity with Gemini 3 Flash
2026-02-04 17:24:55 +00:00
David Blaikie f1f6005d4a Perform Clang IRGen during check (#6569)
Background:
https://docs.google.com/document/d/1wi85FRiWh4X9A-gCYMVGKR40-q5fM6-3JaSpePk-XCY/edit?usp=sharing
And specifically this work is essentially an alternative to #5543

Clang's code generation is implemented through an ASTListener
(clang::CodeGenerator) that is attached throughout Clang's
parsing/sema/code
generation phases and acts on Clang AST incrementally throughout that
process.

Prior to this patch, Carbon has only created the CodeGenerator during
Carbon's
`lower` phase, missing out on key callbacks that would be made by Clang
during
`check`. Some of these issues were addressed by #6237 and #6483 - but
there were
still remaining cases where the delayed processing lead to missing
functionality.

With #6483 much of the Clang code that made multithreaded complexity of
#5543 is
no longer present, and we have access to the point of ASTListener
registration
so we can register the CodeGenerator there and consume its resulting
llvm::Module during lower.

Examples of some of the bugs this addresses are seen in the linked doc,
and
checked in as tests in this change in
`clang_code_generator_callbacks.carbon`

An indicental bug that's also fixed, and caused all the other test case
churn,
is that the `CodeGenerator` created during `lower` wasn't getting passed
the
Clang `CodeGenOpts` and was creating its own default - so, most notably,
optimization flags were not respected. This meant that the LLVM IR from
Clang
was always -O0 style IR (optnone, no inlinehint, no TBAA, etc). With
this
change, now the Clang IRGen gets the real `CodeGenOpts` and respects
optimization/other flags specified there.

This is only meant to be a rough proof of concept - I'm totally open to
reworking this in any way (even quite substantially) if folks have ideas
about
how this should be implemented most generally/elegantly/etc.
2026-01-14 00:54:37 +00:00
Geoff Romer 2380be2ae1 Add flag to dump the raw SemIR in the event of a crash. (#6558) 2026-01-06 21:56:08 +00:00
Richard SmithandJon Ross-Perkins d208e950c7 Encapsulate clang::ASTUnit in SemIR::CppFile. (#6459)
This intends to avoid proliferation of dependencies on the exact API of
`clang::ASTUnit`, and would enable us to more easily switch to a
different approach that gives us more control over the construction of
the Clang AST.

Also remove some unnecessary tracking of the `CppFile` and instead
always retrieve it from the `SemIR::File`.

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
2025-12-04 21:05:40 +00:00
Jon Ross-Perkins 49ba8cf3e1 Switch class to use a blanket impl for Destroy (#6125)
Right now, the class destroy impl is incorrectly generated (first
discussed [in
Discord](https://discord.com/channels/655572317891461132/941071822756143115/1418614787449032826)).
If we want it to be correct, deferred definition logic would need to be
added, and the declaration would need to be moved inside the `class`
scope (along with whatever generic logic that needs).

This instead switches to a blanket impl, to avoid creating latent bugs
with generating the `impl` and function body in the wrong scope. This
approach uses the same blanket impl as aggregate destruction that was
added by #6098.

The intent here is to allow progress on other parts of `Destroy`. For
example, under this model the implementation of the function body could
be done as part of lowering the specific.
2025-09-29 16:05:06 +00:00
Dana Jansens 64139e5d65 Stop using Map for the cache in InstFingerprinter (#6019)
This takes the debug runtime of
`toolchain/check/testdata/interop/cpp/function/arithmetic_types_bridged.carbon`
from 4.7s down to about 4s (so 15% faster overall).

There's still lots of room to improve this test which seems to be
hitting lots of pathological behaviour, but InstNamer is 30% of the
runtime, with fingerprinting's `InstFingerprinter::GetOrCompute`
consuming 10% of cycles. We reduce its impact by using a vector of
vectors instead of a Map for the cache of fingerprints. After this
change InstNamer drops below 24% of the runtime.

Also move the instruction name when giving it to `AllocateName` since it
receives std::string by value, though this doesn't show up in the
profile for the test.
2025-09-08 16:15:10 +00:00
Boaz Brickner 9f108bad6e Rename cpp_ast to clang_ast_unit (#5926)
Followup of #5924.
2025-08-08 08:11:49 +00:00
Boaz Brickner 52ed26235d Add a flag to dump the C++ AST (#5918)
Use it to dump the AST that includes a generated C++ thunk.
Based on #5917.

Also added printing of the full actual text when check fails to make
debugging easier.
Changed line replacement to allow removing complete lines.

Part of #5514.
2025-08-07 17:09:30 +00:00
Jon Ross-PerkinsandGeoff Romer 7209ad7c9f Generate Destroy impls for classes (#5873)
Although this focused on `Destroy` support, some choices here around
`implicit_type_impls` are because copy/move will likely follow a similar
approach. I'm trying not to predict too much about how we'll structure
those, but I'm putting `Destroy` impl logic in a file that could perhaps
be shared with those. They'd likely be interested in similar things,
e.g. traversing members of types (particularly class, struct literal,
tuple literal).

At present this sets the destroy function as `no_op` which is consistent
with current logic, but has a TODO to correctly define.

Constant importing for functions changes slightly due to some issues I
was having with `GetFunctionType`. zygoloid suggested this approach to
avoid `EvalInst` logic.

Adds a flag for controlling whether to generating these impls. While
this does generation for `class`, as noted above this'll also need to be
done for tuples and struct literals, which would leave the `none.carbon`
min_prelude unable to use any types. Note if destruction *would* occur,
it'll still look up `Core.Destroy` for that and fail, but that's already
true of any test using `none.carbon`. I'm trying to use the flag to see
if we can keep `none.carbon` working mostly-consistently.

I'd tried separating out the flag to #5852, but that got a lot of
pushback over whether the behavior was appropriate. I'm hoping that the
interactions here make it clearer why the particular approach -- the
goal is not to enable advanced testing, or create some new end-user
behavior that we really support, it's just to keep no-prelude tests
functional. The main question raised there was why not just keep
generating `impl T as Core.Destroy` if `fn destroy` is present -- but I
think here it should be apparent that would require additional
complexity, as the generation of `impl T as Core.Destroy` is not
currently conditioned based on the implementation of `fn destroy`. I'd
rather add complexity to this flag only if it's enabling interesting
test functionality.

---------

Co-authored-by: Geoff Romer <gromer@google.com>
2025-08-04 19:39:22 +00:00
Jon Ross-Perkins bd4fbb4393 Expand use of CheckIRId stores (#5820)
This is trying to make it clearer when vectors are being indexed with
`CheckIRId`.

The only one that I still kind of want to change is the
`SmallVector<std::unique_ptr<CompilationUnit>>`, but because it's a
`unique_ptr` that's a little more complex. I may not bother.

Note, some of the changes around nuanced `SmallVector` interactions were
based on trying to copy the way `SmallVector` itself takes arguments,
like with range passing.
2025-07-21 20:02:27 +00:00
Richard SmithandChandler Carruth 553dd6e531 Build the clang::CompilerInvocation in the driver. (#5784)
Add driver flags to specify clang driver arguments.

---------

Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
2025-07-18 16:30:47 +00:00
Richard SmithandChandler Carruth 3776e464e0 Properly set up C++ include paths and similar environment settings when parsing imported C++. (#5767)
Stop using the clang tooling library to build an ASTUnit; that library
is set up to process clang frontend arguments, assuming that something
has already built frontend arguments from the compiler arguments. It is
also too encapsulated and doesn't let us inspect and modify the compiler
invocation before it's executed.

Instead, build the AST unit directly in two phases:

* FIrst, take a list of clang driver arguments and convert them into a
list of compiler arguments, using `clang::createInvocation`. Internally,
this uses the clang driver to build a frontend invocation, including
building system-specific include paths as needed.
* Then, directly build an ASTUnit from this compiler invocation.

I've factored this so that we can split out the `createInvocation` step,
with the intention that we may want to move it out of check and into the
carbon driver with the rest of the driver-level argument handling, and
we may want to customize some of the clang options before we invoke the
clang frontend with that set of options.

In order to make the invocation reusable, it no longer depends on the
name of the carbon file importing the C++ code. In place of synthesizing
a header file name as `<foo.carbon>.generated.cpp_imports.h`, we now
insert line marker directives into the generated header so that errors
in that header cause Clang to point a diagnostic back at the Carbon
source file itself. This results in a minor improvement in the
diagnostic output: we no longer refer to a nonexistent generated file.
But the snippet still contains text that doesn't match the source code,
so it remains imperfect.

---------

Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
2025-07-09 03:09:43 +00:00
Jon Ross-Perkins 57ef976802 Move dumping into the phase factory functions (#5747)
By moving dumping, we can have dumping occur before verification that
might CHECK-fail (e.g. parse tree and llvm IR verification).

I'm dropping vlogging of raw semir. It was only done when dumping, so
`-v` would print zero copies and `-v --dump-raw-sem-ir` would print two
copies. The lack of complaints about this suggests it's not needed.

I'm making a small change to drop newlines between textual and raw
semir. This is an edge case so I don't expect people to really notice in
general, but it seemed unusually aware of what's on a stream, and it
made it harder to do the dump_stream/raw_dump_stream approach, which I
felt would be decent in general, since check is the only phase that can
emit two different things (which I could also just drop -- we don't
really use raw semir anymore, it doesn't seem like a big need to be able
to print it with textual semir, but I'm assuming to just maintain
existing behavior).

In parse, we were previously dumping the tree on verification errors.
I'm removing that because now `--dump-parse-tree` should work fine,
where previously it wouldn't.
2025-07-01 15:51:41 +00:00
Jon Ross-Perkins 2de746e83c Switch compile functions to use options structs (#5742)
I've been mulling this mainly for the parameter complexity of
check/lower, but doing lex/parse for symmetry.

I'm motivated by the plan to move dumping for all of them into the
respective functions, because of discussion about llvm-verifier. That
basically would add another bool parameter (or more) to each of these.
My instinct is we're going to probably accrue a little more over time,
so I'm suggesting this as maybe adding the boundary a little simpler
and/or easier to read.

Note it may make sense to refactor a little further, e.g. maybe
Lower::Context could receive the full set of options and pick out what
it wants, but I figured creating the struct itself would be a decent
start.

I'm trying to put things into options when we can produce a reasonable
default if the user doesn't assign a value. I'm using an explicit
constructor so that values can be added without affecting every caller.

A different factoring would be to pass in everything through the param
struct, but that just felt weird when I was trying it out.

Removing `inst_namer` and `module_name` from `LowerToLLVM` params --
both of these can be inferred from `sem_ir`, and I'm not seeing a
particular reason to maintain them at the call site.
2025-06-27 22:08:47 +00:00
Jon Ross-Perkins 20c20595ba Fix language-server crash with cpp_ast (#5604)
Removes the nullptr default for safety.

Note, I think cpp support doesn't allow things like `<version>` or
inline code yet, and language-server support doesn't allow non-hermetic
files, so the best I can test is an error.
2025-06-03 22:09:51 +00:00
Boaz Brickner 852d0191a9 Add support for importing C++ inline functions (#5427)
This requires:
* Making `FunctionDecl` mutable since generating code
(`HandleTopLevelDecl()`) requires a mutable declaration and since we
manually add `used` attribute to force code generation.
* Passing the file system to `Lower` since it's needed by Clang code
generation.
* Creating an internal Clang LLVM module and link it against the Carbon
LLVM module.

Demo:

```c++
// hello_world.h

extern int puts;
inline void hello_world() {
  ((int (*)(const char*))&puts)("hello world");
}
```

```carbon
// main.carbon

library "Main";

import Cpp library "hello_world.h";

fn Run() -> i32 {
  Cpp.hello_world();
  return 0;
}
```

```shell
$ bazel-bin/toolchain/carbon compile main.carbon
$ bazel-bin/toolchain/carbon link main.o --output=demo
$ ./demo
hello world
```

Based on https://github.com/carbon-language/carbon-lang/pull/5406.

Part of #5405.
2025-05-21 07:02:05 +00:00
Jon Ross-Perkins 0683742f19 Cache multi-IR info, particularly include_in_dumps (#5408)
Right now we construct `tree_and_subtrees_getters` a couple different
ways, it's just not obvious because one's abstracted in `check`. But
also, when formatting IR, we'll repeatedly do the `IncludeInDumps`
string check, which felt odd to me since it only needs to be calculated
once per IR.

This also shifts `CheckIRId` selection a little earlier, and in doing so
makes `CheckParseTrees` accept a sparse `units` argument. I actually
think this is a positive: it makes `CheckIRId` a little more stable
across possible command lines, when file loading fails (which is the
only time that a file will have a `CompilationUnit` but not a
`Check::Unit`).

Trying to build on the shared issue between these, I'm adding a
`MultiUnitCache` to store the calculated arrays. For the subtree
getters, this is very minor and avoids at most one incremental array
construction (moving logic out of `CompileSubcommand::Run` might be the
bigger benefit). For `include_in_dumps`, when dumping SemIR, this is
changing a calculation run once per entity (in each IR) to be calculated
once per IR (globally), i.e. O(M*N) -> O(N).

Note this seems to be marginal for performance of file_test:

- Before: Stats over 10 runs: max = 5.3s, min = 4.7s, avg = 4.9s, dev =
0.2s
- After: Stats over 10 runs: max = 4.9s, min = 4.7s, avg = 4.8s, dev =
0.1s

I was mainly thinking about this in the context of dumping SemIR ranges.
There, the impact may actually decrease because a range won't do any
cross-IR printing. But, I'm expecting to add another layer for whether
we're printing IR for a file, and that made the `should_format_entity`
callback stick out for me.
2025-05-02 22:53:46 +00:00
Jon Ross-Perkins 0a3efb76ed Use DiagnosticEmitter for phase-specific types (#5188)
Given the namespacing of `Diagnostics` in #5173, now we can use
`DiagnosticEmitter` for phase-specific emitters. This is consistent with
how we do `Context`, and also check had started this with
`DiagnosticBuilder` in anticipation of the namespacing.

Also renames `Emitter::DiagnosticBuilder` to `Emitter::Builder` for
consistency with other `Diagnostics` entities.

In check, I'm still splitting `DiagnosticEmitterBase` and
`DiagnosticEmitter` just to keep the emitter definition separate from
the context.

Also cleans up some incorrect check diagnostic emitter dependencies in
lower.
2025-03-27 00:41:30 +00:00
Jon Ross-Perkins acbe6530c3 Move diagnostics into a namespace (#5173)
What this really does is avoids shadowing names, so that we can
comfortable have things like `Check::DiagnosticEmitter` or
`Check::DiagnosticLoc` without shadowing being a concern.

Note, down this path I'm also thinking about:

- Renaming misc DiagnosticConsumer/DiagnosticEmitter classes, possibly
just to DiagnosticConsumer/DiagnosticEmitter (so
`Check::DiagnosticEmitter` instead of `SemIRLocDiagnosticEmitter`).
- Dropping `Diagnostic` from `Emitter::DiagnosticBuilder`.
- But not for `Check::DiagnosticBuilder`, because `Check::Builder` would
be ambiguous.
- Renaming diagnostics/diagnostic_* to drop "diagnostic".

[Discussion about SemIRLoc ->
DiagnosticLoc](https://discord.com/channels/655572317891461132/655578254970716160/1353771570463768698)
reminded me of this (in particular the older [Check::DiagnosticBuilder
discussion](https://discord.com/channels/655572317891461132/655578254970716160/1344363562608627763)),
but I'd only do that rename if there's matching consensus about a path
forward where we keep SemIRLoc, and in a way that it's only ever used
for diagnostics (the divergence from which is at the root of current
LocId discussion).

I'm trying to keep that separate from a namespace addition for clarity.
2025-03-26 19:12:10 +00:00
Boaz Brickner 87b9cab7b1 Add support for importing a trivial global C++ function (#5033)
ASTUnit is owned by `CompileSubcommand`, passed through `Unit` to be
populated in `ImportCppFiles()` and used via `SemIR::File`.
When generating the AST, pass `-x c++` args to compile C++ (temporary
until we pass args properly).
`Cpp` namespace is marked as a special namespace and has dedicated logic
in `LookupNameInExactScope()`.
The logic for importing declarations from C++ to Carbon is in
`import_cpp.cpp`, but we're likely to want to refactor this
signfiicantly over time as it grows (perhaps a dedicated directory?).

Part of #4666.
2025-03-03 10:38:19 +00:00
Jon Ross-Perkins e79d3be5bd Combine DiagnosticConverter into DiagnosticEmitter (#4878)
At present, we typically define a DiagnosticConverter, then store an
instance of it and a DiagnosticEmitter that wraps it. This is relatively
minor in general, but I've been trying to create more self-contained
DiagnosticEmitter classes (which hold their own DiagnosticConverter,
similar to NullDiagnosticEmitter), and there it just gets in the way.

Since we don't reuse DiagnosticConverter instances, this combines the
definition into DiagnosticEmitter. Mainly this means we don't have a
separate object in play, and less to carry around.

The most impact is probably to SemIRDiagnosticConverter, which was also
the most complex. Now `SemIRLocDiagnosticEmitter`, this gets some
different construction flow. Note in the PR I've split the file rename
to its own commit, to try to help delta views. However, the most
substantial parts of the refactoring are split into #4876, which this
depends upon.
2025-02-06 20:27:57 +00:00
Jon Ross-PerkinsandChandler Carruth 7eee9a3489 Refactor resolving a location into a SemIR library (#4876)
At present, lower depends on `Check::SemIRDiagnosticConverter` for debug
info. That was to support a quick implementation of debug info, but
isn't great because it's both an unusual dependency on check's
implementation, and relying on diagnostic structures for debug info.

This cleans that up by splitting relevant logic out to a library in
sem_ir, and having lowering use sem_ir's library instead of check's.
Additionally, a small refactoring of `Parse::TreeAndSubtrees` to allow
getting locations in lowering without going through a `DiagnosticLoc`.
I'm adding `Parse::GetTreeAndSubtreesFn` in because it's a complex
signature to have in so many spots.

I chose to have `ResolveNodeId` return a `SmallVector` because it seemed
likely to be fairly compact, but that could also be using an optional
callback to handle resolved node IDs, possibly just returning the last
entry. This could be switched if preferred.

Note this change shouldn't affect behavior, it's just moving code
around.

---------

Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
2025-02-06 00:55:20 +00:00
Jon Ross-Perkins 133717cd7e Eliminate NodeLocConverter (#4870)
I'm looking at eliminating `DiagnosticConverter`. This change removes
`NodeLocConverter` (albeit adding `UnitAndImportsDiagnosticConverter`),
and in doing so, refactors lex conversion functions to extract them out
from the `DiagnosticConverter` functions.

I'll be following up with changes that collapse `DiagnosticConverter`
logic into `DiagnosticEmitter` locations. The intent is that we
shouldn't need separate ownership of both types.
2025-01-30 22:30:33 +00:00
Boaz Brickner 5b70a3ea91 Generate AST when importing a cpp file (#4790)
Ignore the AST and support a single Cpp import, for now.
Report cpp compilation errors and warnings.
Part of #4666
2025-01-14 20:45:50 +00:00
Jon Ross-Perkins 87b3671330 Refactor single-unit checking out of check.cpp (#4649)
This is primarily moving code around, to try to create a logical split
of the code in check.cpp, makingthe API boundaries clearer.

There's one small, deliberate logic change around false returns from
`HandleParseNode`, where before there was a `CARBON_CHECK` instantiated
by the `#define` (per `NodeKind`), and now it's outside the `#define`
(done mainly because the message didn't keep up with the `Handle##Name`
-> `HandleParseNode` rename).
2024-12-10 21:00:51 +00:00
Jon Ross-Perkins e7a86b03c6 Remove offsets from InstId formatting, trying to name more (#4645)
The offsets were originally added to deal with churn from builtins in
the raw semir. In textual semir, we mostly see instruction IDs for
imports, and builtins have also settled down more.

On imports, where possible, use the `EntityNameId` for an import instead
of printing an instruction. Next, show the source location if we have a
node. Only show the instruction if there's no location.

This also exposes `Parse::Tree` and `TokenizedBuffer`, so that we can
pass a `SemIR::File` without the component parts. In particular this
allows us to get the `TokenizedBuffer` for import IRs without
substantial structural modifications. We may want to make these optional
for serialized `SemIR` later, but the nodes/tokens contain source
location, which we'd need for debug information -- so it's not clear how
much we can really make them optional without substantial information
loss.

Reduce arguments to just `File` in a few spots, as a result of the
accompanying `TokenizedBuffer` and `Parse::Tree`. Also updates style to
pass around `const File*` where the reference is maintained, instead of
`const File&`.

I was considering keeping a direct reference to the tree and tokens on
`Context`, but initially my thought was it wouldn't make much
difference. I can re-add those if desired, just as direct caching of the
`File` fields.
2024-12-06 21:17:24 +00:00
Dana Jansens b705be9527 Use a Timings* in place of optional<Timings>* (#4607)
The current representation has two nested presence indicatators (the
optional bool, the null pointer). Currently the pointer is never null,
but the style guide suggests that T* should be used for parameters that
may or may not be present, so we do not need the optional here.

> When passing an object's address as an argument, use a reference
> unless one of the following cases applies:
>
> - If the parameter is optional, use a pointer and document that it
>   may be null.

Once the parameter is just a pointer, the ScopedTiming field does not
need an optional either, and can just store the pointer.

It would be more preferable to have an optional representation of a
sometimes-null pointer like optional<T&> to describe a sometimes-null
pointer, as this would allow clearer runtime diagnostics when used
incorrectly (a check failure in unwrapping) and would be better
self-documenting through syntax instead of a comment. But we do not
currently have such a primitive.
2024-12-03 19:01:12 +00:00
Jon Ross-Perkins 17272cf93c Change how CheckParseTrees receives NodeLocConverters (#4563)
This is really a set of closely related changes:

1) We really shouldn't be creating a Check::Unit for _Lower_. This fixes
that by storing the diagnostic converter for reuse.
2) Rather than passing in node converters to Check as their own array,
pass diagnostic converters as part of Check::Unit.
3) To support creating the converters early, pass SemIR::File
pre-constructed.
4) Since SemIR::File construction was used to track "checked", add
`is_checked` for that.
5) Clarifies a subtle edge case around `input_filename_` use with `-`.

Note the key consequence of this change, where I actually started, is
that `Check` only has one array of `Check::Unit` instead of receiving
`NodeLocConverter` as a separate array.
2024-11-22 16:51:10 +00:00
Sam EstepandJon Ross-Perkins e0e305536e Collect timing data per unit for each phase (#4512)
This PR adds a `--dump-timings` flag to the `compile` subcommand
(similar to the existing `--dump-mem-usage` flag), which collects timing
data per compilation unit for each compilation phase. For example, on my
2020 M1 MacBook:

```
$ bazel build -c opt //toolchain
$ bazel-bin/toolchain/install/run_carbon compile --phase=lower --dump-timings examples/sieve.carbon | tail
...
---
filename:        'examples/sieve.carbon'
nanoseconds:
  lex:             30792
  parse:           25458
  check:           226625
  lower:           1136958
  Total:           1419833
...
```

Most of the changes are pretty straightforward. There were a couple I
wasn't sure about though; let me know if I should change:

- new `Timings` class in its own file, pretty similar to the existing
`MemUsage` class
- added a `timings_` field to the `CompilationUnit` class
- added a `timings` field to the `Check::Unit` struct
- renamed `CheckParseTree` function to `CheckParseTreeInner` for ease of
timing with early `return`

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
2024-11-12 20:47:08 +00:00
Chandler Carruth 4148161e24 Refactor value store code to use separate files. (#4477)
This is in anticipation of making the integer value store be customized
heavily. I'd like to extract it from the common code when doing that, so
first disentangling them here without any intended change in
functionality or behavior to enable that.

I've tried to update `#include`s to be as minimal as I can and added a
few missing includes spotted in the process.

I've split the test for value store to include what was easy focused on
just the value store templates rather than the unified shared value
stores.

This might surface some opportunities for adding more tests, but for
this PR, just doing the minimal restructuring.
2024-11-04 04:00:17 +00:00
David BlaikieandJon Ross-Perkins c5ada29ba9 Add filename and line number to function debug info metadata (#4243)
Refactors a bunch of the SemIRDiagnosticConverter to be able to use that
from Lower to access source locations there to use in debug info.

I assume some of this is a bit jank/would need to be fixed/improved in
the future - like the context functor that's passed into ConvertLoc?
(not totally clear what that's for/what the debug info will be missing
out on in its absence, I could throw a FIXME in there if you like)

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
2024-08-23 00:05:26 +00:00
Jon Ross-Perkins f67791cfee Separate subtree size information from parse nodes. (#4174)
Move subtree sizes over to TreeAndSubtrees, using the different
structure to represent the additional parse work that occurs, as well as
making it clear which functions require the extra information. My intent
is to make it hard to use this by accident.

The subtree size is still tracked during Parse::Tree construction. I
think a lot of that can be cleaned up, although we use it during
placeholder assignment so it may take some work. I wanted to see what
people thought about this before taking action on such a change.

I'm using a 1m line source file generated by #4124 for testing. Command
is `time bazel-bin/toolchain/install/prefix_root/bin/carbon compile
--phase=check --dump-mem-usage ~/tmp/data.carbon`

At head, what I'm seeing is:

```
...
parse_tree_.node_impls_:
  used_bytes:      61516116
  reserved_bytes:  61516116
...
Total:
  used_bytes:      447814230
  reserved_bytes:  551663894
...
1.43s user 0.14s system 99% cpu 1.565 total
```

With `Tree::Verify` disabled completely, it looks like:
```
parse_tree_.node_impls_:
  used_bytes:      41010744
  reserved_bytes:  41010744
...
Total:
  used_bytes:      427308858
  reserved_bytes:  531158522
...
1.20s user 0.13s system 99% cpu 1.332 total
```

Re-enabling just the basic verification (what is now `Tree::Verify`),
I'm seeing maybe 0.05s slower, but that's within noise for my system. I
do see variability in my timing results, and overall I think this is a
0.2s +/- 0.1s improvement versus the earlier (always testing `Extract`
code) implementation. That's opt; debug builds will be unaffected,
because the same checking occurs as before.

Note, the subtree size is a third of the node representation, which is
why I'm showing the decrease in memory usage here.
2024-07-31 19:39:45 +00:00
Richard SmithandJon Ross-Perkins 62fe0cd385 Remove the builtin IR, and instead define builtin types locally. (#3910)
We don't need it any more, and removing it simplifies a few things:

- One fewer predefined `File` and reserved ID.
- We now have simply `Builtin` instructions for builtins, instead of
having an `ImportRef` that indirectly references a `Builtin`.
- `ConstantId`s now always refer directly to a local constant, instead
of sometimes referring to an `ImportRef` for a constant in the builtins
IR.

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
2024-04-24 18:32:59 +00:00
Jon Ross-Perkins 895e90e791 Start including the prelude for testing. (#3861)
- Adds an empty prelude.carbon file
- Imports that file in any non-Core package file
  - Adds --disable-prelude-import to avoid that
- Adds --exclude-dump-file-prefix to be able to hide files from dumping
- Used to hide core files (we can't do this by package name due to lex
dumps, for example)
- Restructures some tests to not rely on `i32`, particularly `alias`
tests (which rely on a name ref) and tests with no prelude.

I'm adding the framework for switching i32 to calling Int32 in the
prelude, but I'm running into a separate error actually switching over.
So that *mostly* works, but isn't quite ready for prime time. However,
maybe the current state of this PR is still useful to review since it
does a lot of the infrastructure work and adds the %Core everywhere?
2024-04-07 17:11:42 +00:00
Jon Ross-PerkinsandRichard Smith d024403dc4 Refactor checking flow to allow for ordering based on import/package. (#3379)
As I was working on this, I noticed `import` and `library` syntax needs
to be fixed for how it imports the current package, and for `Main`
libraries. This mostly reflects the current state in its testing.

Otherwise, this should handle most of the errors I could think of:
dependency cycles, redundant imports, etc.

It does not actually deal with the nuances of cross-IR references.

---------

Co-authored-by: Richard Smith <richard@metafoo.co.uk>
2023-11-14 22:24:03 +00:00
Jon Ross-Perkins 1b55ad86dd Extend SharedValueStores to SemIR (#3313)
Building on #3311, change SemIR to use the SharedValueStore. Since this
removes hermeticity, raw output no longer prints ints, reals, and
strings. TokenizedBuffer accessors are modified to return IDs because
values are often passed through in semantics without needing to read
them.

I would've put SharedValueStores on Context, except for the
GetArrayBoundValue convenience method. I felt awkward removing that, so
it's on File, at least for now. That's then used by the formatter and
Lower too. The flipside of this is that TokenizedBuffer has a
SharedValueStores only for printing, so maybe that's similar enough to
what File is doing.

This doesn't start shifting other SemIR members to ValueStore, but that
seems like a next step.
2023-10-20 17:53:00 +00:00
Jon Ross-PerkinsandChandler Carruth ec182fb00d Rename lexer dir to lex (#3179)
Continuing with #3070. Just a dir and file rename (only prefix change is
lexer_file_test). Everything in the lex dir should be marked as a move.

Note, I think this closes #3070. There may still be further cleanup
later, but the organizational changes suggested there are being
completed.

---------

Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
2023-09-01 02:39:04 +00:00
Jon Ross-Perkins c555b39a2c Rename parser dir to parse (#3178)
Continuing with #3070. Just a dir and file rename (mostly removing
prefixes, although for parse_tree_fuzzer and parse_tree_file_test I'm
dropping "tree" instead of "parse"). Everything in the parse dir should
be marked as a move.
2023-09-01 01:35:45 +00:00
Jon Ross-Perkins 1c748c0f14 Split semantics into check and sem_ir directories (#3176)
Continuing along with #3070. Note this is just a file rename, with BUILD
edits; every file previously in semantics/ should show as moved (except
maybe BUILDs, which split).
2023-08-31 19:54:32 +00:00