Carbon is accelerating and adjusting its safety strategy, specifically
to flesh out its memory safety strategy and reflect simplifying
developments in the safety space.
This proposal replaces the previous directional safety strategy with a
new concrete and updated framework for the safety design. It includes a
specific framework for memory safety, simplified build modes, specific
"safety modes", and terminology.
This proposal also provides a _directional_ suggestion for temporal and
data-race safety specifically.
In addition to fully building out the above directional component, there
are several other aspects of our safety design that will follow in
subsequent proposals. The hope is to establish the initial framework
here.
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
Co-authored-by: Mike Forster <michael@forster.pro>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Add a content keyword to file_test, `[[@0xAB]]`, that expands to the
code unit 0xAB, and use that instead of putting raw malformed code units
in test files.
Instead of printing the raw input bytes in snippets in diagnostics,
replace non-printable characters with <AB> in the output, being careful
to still compute the location of the caret and underscore properly.
Fix the algorithm for importing declarations in dependency order to
properly walk the dependency graph. Add the parent declaration of a
declaration to the dependency set so that we have a parent declaration
context to import a declaration into.
Fixes a crash when attempting to import a class whose parent is not
imported.
Specifically this adds `WriteStream` to get an LLVM-style
`raw_fd_ostream` for an open file, and `Rename` corresponding to
`rename` and `renameat` Unix-like system calls.
Some basic testing for both is added as well.
This was split out of work to switch the runtimes building to use the
new filesystem library.
Add a `Core.String` class to the prelude representing a string view, and
rename the `String` keyword to `str` and make it evaluate to
`Core.String`.
`Core.String` is represented as a pair of a pointer to a character
(actually, to the first character of a string, but we don't have a way
of modeling that yet) and a size (which should be pointer-width, but is
currently always a `u64` as we don't have a `usize` equivalent yet).
`Core.String` values are generated directly by the toolchain for string
literal expressions.
This follows the direction established at the recent summit, but the
design implemented here has not been through the proposal process yet.
Trying to figure out an easy way to debug semir in the prelude, #5703
removed an option to set `--exclude-dump-file-prefix` to empty. But,
this is probably an improvement over that flow... With this change, it's
possible to add `//@dump-sem-ir-file` to a specific prelude file, and
its full IR will be printed. Additionally, it becomes an option with the
default `--dump-sem-ir-ranges=only` to add `//@dump-sem-ir-file` and get
the full file's IR.
This addresses/avoids the duplicate import of vtables.
I went through a few iterations/etc along the way and left them in the
commit
history for the PR in case any of them are useful to illustrate how I
got here,
or worth revisiting.
Essentially I ended up with a circularity in importing - importing the
class
imported the vtable_decl which imported the virtual functions - and then
pending
specifics of the virtual functions needed the self specific of the
enclosing
class which wasn't ready yet.
Adding ImportRef to the vtable_decl to break the cycle caused me trouble
when
naming the vtable_decl instructions - so I tried making the functions in
the
vtable unloaded ImportRefs instead. That worked, but meant that
importing a
class still was doing O(number of vtable entries) even if the vtable
wasn't
used.
So I revisited the lazy vtable_decl - figured out how to make the naming
work
(when building the vtable_ptr, even though the vtable_decl doesn't have
to be
loaded for the vtable_ptr, I force it to be loaded anyway, to load the
vtable so
it's usable by lowering, etc). And then I could go back to the old
non-lazy
loaded vtable entries (using some loaded ImportRefs in the cases where
we needed
them/had already adopted them).
Then thinking about the VtablePtr instruction, went back/forth on
exactly what
it needed - went from VtablePtr's member being a VtableDecl InstId, to a
ClassId, then back to a VtableId as it was before this patch.
Naming the instructions has one oddity, that the VtableDecl and
VtablePtr
instructions seem to need to add the pending name for the VtableId -
despite not
using the VtableId in their own name - should the inst namer be doing
this work
for parameters of instructions rather than requiring the inst to do it
deliberately? (or am I holding it wrong in some way?)
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
* `\x` escapes are not permitted in character literals
* ASCII control characters (U+0000 .. U+001F) are not permitted in
character literals unless specified with escape sequences.
Generalize the f64 support to support other sizes. Also provide interop
support for `float`, `_Float16`, and `__float128`.
Also lay some groundwork for non-standard floating-point types, though
we don't have any syntax to name them yet.
I noticed while trying to set up an associated constant in the prelude
that we weren't supporting bool value imports; this goes through and
addresses support for simple builtin types.
Array initialization fails on declaration, which seems like a bug but
I'm only documenting it here.
Also fix missing export of `Core.FloatLiteral`
I checked and this doesn't seem to affect #5952, which is doing more
float changes.
The main change here is that a bad type appearing somewhere within a
field or base class of a class shouldn't cause an import of that class
to fail. Instead, only that field or base class becomes inaccessible
from Carbon.
Also improve the way that type importing errors are diagnosed. While we
lose the precision of a diagnostic saying why a type is not supported,
we gain a useful source location for where the type was mentioned in C++
code.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Don't convert to f64 until we know that's the type that we actually
want. Also reimplement the conversion from RealId to FloatId to perform
an exact conversion with a real check for overflow, rather than
performing an approximate conversion via the host `double` type.
Unfortunately, LLVM doesn't expose its integer mantissa and exponent to
APFloat conversion, so we convert the RealId back to a string for now.
The LLVM conversion also detects overflow only if the literal would
round to having an out-of-range exponent, not if the literal is outside
the range of values of the type as the Carbon design expects. It's not
clear to me which rule we actually want here, so for simplicitly I'm
using the LLVM rule for now.
In preparation for adding other floating-point types beyond f64.
Tidies up extraneous move, unnecessary function style type cast, and
simplifies the temporary directory string construction. These were
noticed during another PR review.
Also corrects support for older glibc versions, including the
GNU-specific quirks of `strerror_r`. Restricts the fancier formatting
with the name of the error number to when a recent glibc is available.
Lastly, filters the benchmarks in the benchmark test down to smaller
ones to avoid test timeout flakiness.
* Rename the type.
* Change lowering to lower FloatLiteralType values as the placeholder
`{}` value we use for literals instead of as an LLVM f64.
* Change eval to convert the type as part of a floating point
conversion, so that lowering can lower converted constants properly.
For now we still represent a value of FloatLiteralType as a
double-precision APFloat. (That will need to change so that we can
losslessly convert literals to f80 / f128 values, and so that we can
convert literals to f32 values without double-rounding.)
We add a virtual node (`CompileTimeBindingPatternStart`) as the first
child of `CompileTimeBindingPattern` which holds the identifier
underneath it, so that it is checked just before the type expression of
the `CompileTimeBindingPattern`. When we reach this virtual node during
check, we add `.Self` as a name in the current scope, and when we reach
`CompileTimeBindingPattern` we remove it from scope, which ensures it's
present during only the checking of the type expression for the compile
time pattern.
At the moment the `.Self` has a different type (it's a `TypeType`) than
other `.Self` in the facet type (which are a single `FacetType`), but
the intention is to immediately substitute it out of the facet type
entirely, replacing it with a reference to the compile time binding (a
`BindSymbolicName`) itself. A TODO has been added for this.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This removes a bunch of manual filesystem helpers and complexity that
are directly provided by the new library.
It also moves all of the install paths detection to use
`std::filesystem::path` instead of the LLVM path library. The goal is to
consolidate all our logic onto a single stack, and the standard one
seems the best for that purpose. This does give up some of the
optimizations of this code to avoid memory allocation, but in practice
that likely isn't a critical issue. And with the new filesystem library
we can likely do more to avoid that by using directory-object-relative
filesystem access. However, that will have to wait for moving more parts
of the toolchain over to use this set of filesystem abstractions. There
is a related TODO left in the manifest handling code.
The standard filesystem API lacks significant functionality, ranging
from correct and secure creation of directories and files within them by
using `openat` and avoiding [TOCTOU] issues, to support for filesystem
locking.
[TOCTOU]: https://en.wikipedia.org/wiki/Time-of-check_to_time-of-use
The LLVM filesystem library has more functionality, but uses an API that
is increasingly diverging from the standard, and also fails to defend
against TOCTOU.
This library is designed to carefully model the Unix or POSIX filesystem
concepts of `openat` to avoid TOCTOU. However, it also tries to limit
itself to an API subset that LLVM's filesystem library has also
implemneted and so we have a strong reason to expect to be possible to
port to Windows reasonably.
This PR included several benchmarks that show that this implementation
is also faster for the majority of operations than the C++ standard
library. The only places where there is a consistent regression is in
recursively creating directories, and this is directly connected to the
approach of using `openat` as the basis. Even there, while the wall time
regresses, the cycles and instructions are significantly improved.
There are a number of operations not yet included here, I've focused on
a core set of opening, closing, creating, and removing, and then adding
those that I saw the current toolchain code using actively. I'll plan to
expand the operations as needed going forward.
A follow-up PR that I'll finish polishing and send next ports
`//toolchain/install` to consistently use this library and
`std::filesystem::path` to both exercise the library and showcase its
use. I'll be working systematically across the toolchain to converge all
the code, extending this library as needed.
For reference, benchmark results on my macOS laptop:
https://gist.github.com/chandlerc/29d1f4d465a835b8be5174a48dad2e8f
Benchmark results on a Asahi Linux M1 Mac Mini:
https://gist.github.com/chandlerc/c42d43dd6b9b91746ab314b2afa152f7
Benchmark results on a Linux server with weirdly slow FS operations:
https://gist.github.com/chandlerc/48301a7383eb3972d53351b7e35e0561
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
The self access is important; for the test `generic_class.carbon` being
added to `toolchain/check/testdata/class/destroy_calls.carbon`, it was
using `%T.as.Destroy` instead of `%D.as.Destroy`, indicating the default
blank impl was being used instead of the type-specific version. That
test is trying to focus on the issue, but the delta is visible in a
couple other files in this PR, for example
`toolchain/check/testdata/class/generic/init.carbon`.
I'm separately working on getting rid of the default impl, which is how
I noticed this.
Add missing builtins for float compound assignment, for building a
FloatType, and for converting a float literal to FloatType. Switch
`Core.Float` to being a class and add impls for the various
floating-point operators.
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: Carbon Infra Bot <carbon-external-infra@google.com>
When an error diagnostic has an unattached location, for example because
the diagnostic points into a file that's in the prelude, use the next
attached location to position the error diagnostic's CHECK line. In
particular, if the error is followed by a note, use the position of the
note to determine where to place the error.
This exposes a general mechanism to do final fixups of the CHECK lines
to individual file_test binaries, which the toolchain's binary uses to
special-case error / warning CHECK lines.
With help from Richard Smith debugging/identifying this.
Hmm - looks like maybe the Self type import ref may have the same
problem? (or at least it seems to have the same quirk in the semir dump,
where the inst id is mentioned in the `import_ref` insts, but is not
defined elsewhere, has no name, and says `[no loc]`. I'll look into that
separately. (hmm, maybe this is just an unloaded ImportRef, actually)
When initializing a C++ thunk parameter:
* If we have an initializing expression, materialize a temporary and
pass its address.
* If we have a reference expression, pass its address directly.
* If we have a value expression with a pointer value representation,
pass the pointer.
* Otherwise, create a new temporary and initialize it with a copy of the
argument, and pass its address.
Instead of a bespoke structure based on the `EntityName` in the
`ImplWitnessAccess`' `.Self` type, use the constant value of the
`ImplWitnessAccess` as the map key in `AccessRewriteValues`. This is
okay after #5883 makes the `ImplWitnessAccess` to `.Self` canonically
the same regardless of how it's constructed with nested `where`
expressions.
Introduce `KnownInstId` which tracks in the type system that an `InstId`
is known to refer to a specific typed inst structure. This avoids
writing CHECKs and comments and allows compiler enforcement.
This makes all `.Self` references in a facet type canonically the same
(which will remain true iff they refer to the same `Self` type in the
future), removing the need to do more complex comparisons between them
using the EntityName, interface, and index. This allows the comparison
of types containing `.Self` references to be done correctly regardless
of where the `.Self` appears, as such type expressions will all be
canonically equal if they otherwise equal now, regardless of whether
they are written in the context where `.Self` could have seen different
`Self` facet types.
In order to retain access to constraints on a base `.Self` facet type,
in the case of applying `where` to an existing facet type, we:
- Give the base facet type as a `RequirementBaseFacetType` constraint so
that eval of `WhereExpr` can find and copy all the constraints off of
it.
- Introduce eager/early rewrite constraint resolution, which allows a
constraint to eagerly resolve access to earlier rewrite constraints
(`where .A = () and .B = .A` is eagerly transformed into `where .A = ()
and .B = ()`) before the full constraint resolution step. This allows
use of rewrite constraints in larger type expressions, such as `where .A
= () and .B = C(.A)` and `C` will know that the argument is `()`.
Use it to dump the AST that includes a generated C++ thunk.
Based on #5917.
Also added printing of the full actual text when check fails to make
debugging easier.
Changed line replacement to allow removing complete lines.
Part of #5514.
I was thinking about this for destruction, which I may not be able to
use it for, but still think this may be a good change to keep features
consistent.
toolchain/check/testdata/builtins/char/basics.carbon and
toolchain/lower/testdata/builtins/char.carbon are probably the most
interesting tests here. The parse tests is required because this adds a
new node kind, and we need coverage of it; but the attached info is
minor. There's a fair amount of test churn here because I'm adding the
Core.Char and Core.CharLiteral types as new singletons.
My intent here is that `CharId` is always a unicode code point, even
when the type is a `Char` and thus must be a single UTF-8 code unit
(single byte). This mainly means the stored value of a `CharValue` can
be printed internally without knowing the type.
---------
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Similar to 26ec78ec00 - vtable entries
can't be unattached symbolic constants, because if they are they can't
be `GetValueInSpecific`d because they lack the context for their
generic/specific.
Importing mostly wants to make/import things as unattached constants -
but `ImportRef` supports attached constants, so use those - but we don't
need the laziness, so use `LoadedImportRef`.
When the C++ function has a parameter that is not a pointer and not a
signed integer of 32 or 64 bits, generate a thunk.
Terminology:
* Callee function: The C++ function we actually want to call.
* Thunk function: The C++ function we generated that calls the callee
function.
* A simple ABI type, for now, is one of:
* A pointer
* signed integer with 32 bits
* signed integer with 64 bits
The thunk function is marked `always_inline` and uses the `asm`
attribute to set its mangled to the callee function mangled name
suffixed with `".carbon_thunk"`.
When importing a C++ function, we decide whether calling it requires a
thunk and if so we generate it and import it as well, which is currently
a recursive call.
When calling the thunk function, we initialize a temporary storage for
each non simple ABI parameter type and take its address. This can be
optimized when the variable is already in storage.
Not supported yet:
* Functions with non void return values.
* Member methods.
Moved unsigned int param test from `arithmetic_types_direct.carbon` to
`arithmetic_types_bridged.carbon`, since only signed integers aren't
bridged using a thunk.
C++ Interop Demo:
```c++
// hello_world.h
struct S {
S() {}
S(const S&) { x = 1; }
int x;
};
void hello_world(S s);
```
```c++
// hello_world.cpp
#include "hello_world.h"
#include <cstdio>
void hello_world2(S s) { printf("hello_world2: %d\n", s.x); }
void hello_world(S s) {
printf("hello_world: %d\n", s.x);
hello_world2(s);
}
```
```carbon
// main.carbon
library "Main";
import Cpp library "hello_world.h";
fn Run() -> i32 {
var s : Cpp.S;
Cpp.hello_world(s);
return 0;
}
```
```shell
$ clang -c hello_world.cpp
$ bazel-bin/toolchain/carbon compile main.carbon
$ bazel-bin/toolchain/carbon link hello_world.o main.o --output=demo
$ ./demo
hello_world: 1
hello_world2: 1
```
Before this change (no thunk - copy constructor not called when calling
`hello_world()`):
```shell
$ ./demo
hello_world: -1219172304
hello_world2: 1
```
This adds support for importing C++ code directly from source rather
than via a `#include`.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
In preparation for `FloatValue` being used more generally, and not only
for literals.
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Jon Ross-Perkins
David BlaikieandRichard Smith
Boaz Brickner
dependabot[bot]