carbon-language/carbon-lang
1
0
Fork 0
mirror of https://github.com/carbon-language/carbon-lang.git synced 2026-09-24 19:20:13 +01:00
Code Issues Packages Projects Releases 10 Wiki Activity

Updates to generics design details, part 1 (#3231)

Browse Source 
First step in updating `docs/design/generics/details.md`. It
incorporates changes from proposals: #989 #2138 #2173 #2200 #2360 #2964
#3162 , but there are still more changes from those proposals to be
made.

It also switches away from suggesting static-dispatch witness tables,
and creates an appendix to describe that decision.

---------

Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This commit is contained in:
josh11b
2023-09-19 19:06:47 +00:00
committed by GitHub
co-authored by Richard Smith
parent e3d3122f1d
commit a8ca499450
9 changed files with 1118 additions and 843 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/design/README.md
+13 -19
View File
@@ -2569,8 +2569,7 @@ class ContactInfo {
>
> - [Aliases](aliases.md)
> - ["Aliasing" in "Code and name organization"](code_and_name_organization/README.md#aliasing)
> - <!-- [`alias` a name from an interface impl](generics/details.md#avoiding-name-collisions) -->
> [`alias` a name from an interface impl](generics/details.md#external-impl)
> - [`alias` a name from an interface impl](generics/details.md#avoiding-name-collisions)
> - [`alias` a name in a named constraint](generics/details.md#named-constraints)
> - Proposal
> [#107: Code and name organization](https://github.com/carbon-language/carbon-lang/pull/107)
@@ -2803,8 +2802,7 @@ In addition to function requirements, interfaces can contain:
- [requirements that other interfaces be implemented](generics/details.md#interface-requiring-other-interfaces)
or
[interfaces that this interface extends](generics/details.md#interface-extension)
- <!-- [associated facets](generics/details.md#associated-facets) -->
[associated facets](generics/details.md#associated-types) and other
- [associated facets](generics/details.md#associated-facets) and other
[associated constants](generics/details.md#associated-constants)
- [interface defaults](generics/details.md#interface-defaults)
- [`final` interface members](generics/details.md#final-members)
@@ -2857,16 +2855,14 @@ In this case, `Print` is not a direct member of `Circle`, but:
}
```
<!-- [`extend`](generics/details.md#extend-impl) keyword... -->
To include the members of the interface as direct members of the type, use the
`extend` keyword, as in `extend impl as Printable`. This is only permitted on
`impl` declarations in the body of a class definition.
[`extend`](generics/details.md#extend-impl) keyword, as in
`extend impl as Printable`. This is only permitted on `impl` declarations in the
body of a class definition.
Without `extend`, implementations don't have to be in the same library as the
type definition, subject to the orphan rule
([1](generics/details.md#impl-lookup), [2](generics/details.md#orphan-rule)) for
[coherence](generics/terminology.md#coherence).
type definition, subject to the [orphan rule](generics/details.md#orphan-rule)
for [coherence](generics/terminology.md#coherence).
Interfaces and implementations may be
[forward declared](generics/details.md#forward-declarations-and-cyclic-references)
@@ -2928,8 +2924,7 @@ fn DrawTies[T:! Renderable & GameResult](x: T) {
> References:
>
> - <!-- [Combining interfaces by anding facet types](generics/details.md#combining-interfaces-by-anding-facet-types) -->
> [Combining interfaces by anding type-of-types](generics/details.md#combining-interfaces-by-anding-type-of-types)
> - [Combining interfaces by anding facet types](generics/details.md#combining-interfaces-by-anding-facet-types)
> - Question-for-leads issue
> [#531: Combine interfaces with `+` or `&`](https://github.com/carbon-language/carbon-lang/issues/531)
> - Proposal
@@ -3559,12 +3554,11 @@ function.
Carbon interfaces with no C++ equivalent, such as
[`CommonTypeWith(U)`](#common-type), may be implemented for C++ types
out-of-line in Carbon code. To satisfy the orphan rule
([1](generics/details.md#impl-lookup), [2](generics/details.md#orphan-rule)),
each C++ library will have a corresponding Carbon wrapper library that must be
imported instead of the C++ library if the Carbon wrapper exists. **TODO:**
Perhaps it will automatically be imported, so a wrapper may be added without
requiring changes to importers?
out-of-line in Carbon code. To satisfy the
[orphan rule](generics/details.md#orphan-rule), each C++ library will have a
corresponding Carbon wrapper library that must be imported instead of the C++
library if the Carbon wrapper exists. **TODO:** Perhaps it will automatically be
imported, so a wrapper may be added without requiring changes to importers?
### Templates
docs/design/classes.md
+4 -4
View File
@@ -1174,11 +1174,11 @@ methods whose implementation may be overridden in a derived class.
Only methods defined in the scope of the class definition may be virtual, not
any defined in
[external interface `impl` declarations](/docs/design/generics/details.md#external-impl).
[out-of-line interface `impl` declarations](/docs/design/generics/details.md#out-of-line-impl).
Interface methods may be implemented using virtual methods when the
[impl is internal](/docs/design/generics/details.md#implementing-interfaces),
and calls to those methods by way of the interface will do virtual dispatch just
like a direct call to the method does.
[impl is inline](/docs/design/generics/details.md#inline-impl), and calls to
those methods by way of the interface will do virtual dispatch just like a
direct call to the method does.
[Class functions](#class-functions) may not be declared virtual.
docs/design/expressions/implicit_conversions.md
+5 -5
View File
@@ -19,7 +19,7 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
- [Data types](#data-types)
- [Same type](#same-type)
- [Pointer conversions](#pointer-conversions)
- [Type-of-types](#type-of-types)
- [Facet types](#facet-types)
- [Consistency with `as`](#consistency-with-as)
- [Extensibility](#extensibility)
- [Alternatives considered](#alternatives-considered)
@@ -182,11 +182,11 @@ var r: Base** = &p;
*r = q;
```
### Type-of-types
### Facet types
A type `T` with [type-of-type](../generics/terminology.md#facet-type) `TT1` can
be implicitly converted to the type-of-type `TT2` if `T`
[satisfies the requirements](../generics/details.md#subtyping-between-type-of-types)
A type `T` with [facet type](../generics/terminology.md#facet-type) `TT1` can be
implicitly converted to the facet type `TT2` if `T`
[satisfies the requirements](../generics/details.md#subtyping-between-facet-types)
of `TT2`.
## Consistency with `as`
docs/design/expressions/member_access.md
+1 -1
View File
@@ -385,7 +385,7 @@ resolution.
Multiple lookups can be performed when resolving a member access expression with
a [template binding](#compile-time-bindings). We resolve this the same way as
when looking in multiple interfaces that are
[combined with `&`](/docs/design/generics/details.md#combining-interfaces-by-anding-type-of-types):
[combined with `&`](/docs/design/generics/details.md#combining-interfaces-by-anding-facet-types):
- If more than one distinct member is found, after performing
[`impl` lookup](#impl-lookup) if necessary, the lookup is ambiguous, and the
docs/design/generics/appendix-coherence.md
+5 -5
View File
@@ -36,8 +36,8 @@ implements interfaces. There are a few main problematic use cases to consider:
`Song` type to support "by title", "by artist", and "by album" orderings.
- Implementing an interface for a type when there is no relationship between
the libraries defining the interface and the type.
- When the implementation of an interface for a type uses an associated type
that can't be referenced from the file or files where the implementation is
- When the implementation of an interface for a type relies on something that
can't be referenced from the file or files where the implementation is
allowed to be defined.
These last two cases are highlighted as concerns in Rust in
@@ -208,9 +208,9 @@ This has some downsides:
varies instead of being known statically.
- It is slower to execute from dynamic dispatch and the inability to inline.
- In some cases it may not be feasible to use dynamic dispatch. For example,
if an interface method returns an associated type, we might not know the
calling convention of the function without knowing some details about the
type.
if the return type of an interface method involves an associated constant,
we might not know the calling convention of the function without knowing
some details about the value of that constant.
As a result, this doesn't make sense as the default behavior for Carbon based on
its [goals](/docs/project/goals.md). That being said, this could be a feature
docs/design/generics/appendix-witness.md
+271
View File
@@ -0,0 +1,271 @@
# Generics appendix: Witness tables
<!--
Part of the Carbon Language project, under the Apache License v2.0 with LLVM
Exceptions. See /LICENSE for license information.
SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-->
<!-- toc -->
## Table of contents
- [Overview](#overview)
- [Terminology](#terminology)
- [Witness tables](#witness-tables)
- [Dynamic-dispatch witness table](#dynamic-dispatch-witness-table)
- [Static-dispatch witness table](#static-dispatch-witness-table)
- [Limitations of witness tables](#limitations-of-witness-tables)
- [Associated constants](#associated-constants)
- [Blanket implementations](#blanket-implementations)
- [Specialization](#specialization)
- [Calling templated functions](#calling-templated-functions)
- [Implementing some Carbon generic features with witness tables](#implementing-some-carbon-generic-features-with-witness-tables)
- [Overview](#overview-1)
- [Example](#example)
- [Associated facets example](#associated-facets-example)
<!-- tocstop -->
## Overview
Witness tables are a strategy for implementing generics, specifically for
allowing the behavior of a generic function to vary with the values of generic
parameters. They have some nice properties:
- They can be used both for runtime and compile-time dispatch.
- They can support separate compilation even with compile-time dispatch.
However, it can be a challenge to implement some features of a generic system
with witness tables. This leads to limitations on the generic system, additional
runtime overhead, or both.
Swift uses witness tables for both static and dynamic dispatch, accepting both
limitations and overhead. Carbon and Rust only use witness tables for dynamic
dispatch, and apply limitations to control the runtime overhead when using that
feature. As an implementation detail, Carbon compilers might also use witness
tables for static dispatch, for example when the code conforms to the
limitations of what witness tables support. However, part of the point of this
document is to state the limitations and obstacles of doing that.
## Terminology
### Witness tables
[Witness tables](https://forums.swift.org/t/where-does-the-term-witness-table-come-from/54334/4)
are an implementation strategy where values passed to a compile-time type
binding are compiled into a table of required functionality. That table is then
filled in for a given passed-in type with references to the implementation on
the original type. The generic is implemented using calls into entries in the
witness table, which turn into calls to the original type. This doesn't
necessarily imply a runtime indirection: it may be a purely compile-time
separation of concerns. However, it insists on a full abstraction boundary
between the generic user of a type and the concrete implementation.
A simple way to imagine a witness table is as a struct of function pointers, one
per method in the interface. However, in practice, it's more complex because it
must model things like associated facets and interfaces.
Witness tables are called "dictionary passing" in Haskell. Outside of generics,
a [vtable](https://en.wikipedia.org/wiki/Virtual_method_table) is very similar
to a witness table, "witnessing" the specific descendant of a base class.
Vtables, however, are passed as part of the object instead of separately.
### Dynamic-dispatch witness table
For dynamic-dispatch witness tables, actual function pointers are formed and
used as a dynamic, runtime indirection. As a result, the generic code **will
not** be duplicated for different witness tables.
### Static-dispatch witness table
For static-dispatch witness tables, the implementation is required to collapse
the table indirections at compile time. As a result, the generic code **will**
be duplicated for different witness tables.
Static-dispatch may be implemented as a performance optimization for
dynamic-dispatch that increases generated code size. The final compiled output
may not retain the witness table.
## Limitations of witness tables
### Associated constants
An interface with associated constants can use that to allow the signature of a
function to vary. A similar issue arises with argument and return values
involving `Self`. This adds to the cost of calling such functions, for example
if they are not passed by pointer, then the generated code must support
arguments and return values with a size only known at runtime.
For this reason, Rust's dynamic trait dispatch system, trait objects, only works
with traits that are
["object safe,"](https://doc.rust-lang.org/reference/items/traits.html#object-safety)
which includes a requirement that
[all the associated types have specified values](https://github.com/rust-lang/rfcs/blob/master/text/0195-associated-items.md#trait-objects).
This reduces the expressivity of Rust traits to the subset that could be
supported by a C++ abstract base class.
Swift instead supports types with size only known at runtime for its
[ABI stability and dynamic linking features](https://faultlore.com/blah/swift-abi/#what-is-abi-stability-and-dynamic-linking),
and can use that to
[support more generic features with dynamic dispatch](https://faultlore.com/blah/swift-abi/#polymorphic-generics).
This comes with runtime overhead.
### Blanket implementations
[Blanket implementations](details.md#blanket-impl-declarations) allow you define
an implementation of interface `Y` for any type implementing interface `X`. This
allows a function to use the functionality of `Y` while only having a
requirement that `X` be implemented. This creates the problem of how to go from
a witness table for `X` to a witness table for `Y`.
Rust supports blanket implementations using monomorphization, but this only
works with static dispatch. Swift does not support blanket implementations. This
is possibly a result of the limitations of using witness tables to implement
generics.
### Specialization
Specialization compounds the difficulty of the previous two issues.
An interface with an associated facet might be implemented using witness tables
by including a reference to the associated facet's witness table in the witness
table for the interface. This doesn't, though, give you a witness table for
parameterized types using the associated facet as an argument. Synthesizing
those witness tables is particularly tricky if the implementation is different
for specific types due to specialization.
Similarly, a blanket implementation can guarantee that some implementation of an
interface exists. Specialization means that actual implementation of that
interface for specific types is not the one given by the blanket implementation.
Furthermore, that specialized implementation may be in an unrelated library.
They may be found anywhere in the program, not necessarily in the dependencies
of the code that needs to use a particular witness table.
As a result, specialization is not supported by Swift, which uses witness
tables. Specialization is being considered for Rust, and is compatible with its
monomorphization model used for static dispatch.
### Calling templated functions
Carbon's planned approach to support calling a templated function from a
checked-generic function, decided in
[issue #2153](https://github.com/carbon-language/carbon-lang/issues/2153),
relies on monomorphization. Trying to rely on witness tables would result in
different semantics for calling the same function with the same types, depending
on which witness tables were available at the callsite.
## Implementing some Carbon generic features with witness tables
### Overview
A possible model for generating code for a generic function is to use a
[witness table](#witness-tables) to represent how a type implements an
interface:
- [Interfaces](details.md#interfaces) are types of witness tables.
- An [impl](details.md#implementing-interfaces) is a witness table value.
We can think of the interface as defining a struct type with a field for every
interface member. An implementation of that interface for a type is a value of
that struct type, which we call a witness or witness table. For example, the
function and method members of an interface correspond to function pointer
fields. An implementation will have function pointer values pointing to the
functions defining the implementation of that interface for a given type. This
is like a [vtable](https://en.wikipedia.org/wiki/Virtual_method_table), except
stored separately from the object.
A witness might
[have references to other witness tables](#associated-facets-example), in order
to support these interface features and members:
- [associated facets](details.md#associated-facets)
- [type parameters](details.md#parameterized-interfaces)
- [interface requirements](details.md#interface-requiring-other-interfaces)
It also could contain constants, to store the values of
[associated constants](details.md#associated-constants), or the type's size.
### Example
For example, this `Vector` interface:
```carbon
interface Vector {
fn Add[self: Self](b: Self) -> Self;
fn Scale[self: Self](v: f64) -> Self;
}
```
from [the generic details design](details.md#interfaces) could be thought of
defining a witness table type like:
```
class Vector {
// `Self` is the representation type, which is only
// known at compile time.
var Self:! type;
// `fnty` is placeholder syntax for a "function type",
// so `Add` is a function that takes two `Self` parameters
// and returns a value of type `Self`.
var Add: fnty(a: Self, b: Self) -> Self;
var Scale: fnty(a: Self, v: f64) -> Self;
}
```
The [impl of `Vector` for `Point_Inline`](details.md#inline-impl) would be a
value of this type:
```
var VectorForPoint_Inline: Vector = {
.Self = Point_Inline,
// `lambda` is placeholder syntax for defining a
// function value.
.Add = lambda(a: Point_Inline, b: Point_Inline) -> Point_Inline {
return {.x = a.x + b.x, .y = a.y + b.y};
},
.Scale = lambda(a: Point_Inline, v: f64) -> Point_Inline {
return {.x = a.x * v, .y = a.y * v};
},
};
```
Since generic arguments (where the parameter is declared using `:!`) are passed
at compile time, the actual value of `VectorForPoint_Inline` can be used to
generate the code for functions using that impl.
### Associated facets example
The associated facet can be modeled by a witness table field in the interface's
witness table.
```
interface Iterator {
fn Advance[addr self: Self*]();
}
interface Container {
let IteratorType:! Iterator;
fn Begin[addr self: Self*]() -> IteratorType;
}
```
could be represented by:
```
class Iterator {
var Self:! type;
var Advance: fnty(this: Self*);
...
}
class Container {
var Self:! type;
// Witness that IteratorType implements Iterator.
var IteratorType:! Iterator*;
// Method
var Begin: fnty (this: Self*) -> IteratorType->Self;
...
}
```
docs/design/generics/details.md
+790 -763
View File
File diff suppressed because it is too large Load Diff
docs/design/generics/terminology.md
+28 -45
View File
@@ -45,9 +45,7 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
- [Type erasure](#type-erasure)
- [Archetype](#archetype)
- [Extending an interface](#extending-an-interface)
- [Witness tables](#witness-tables)
- [Dynamic-dispatch witness table](#dynamic-dispatch-witness-table)
- [Static-dispatch witness table](#static-dispatch-witness-table)
- [Dynamic-dispatch witness table](#dynamic-dispatch-witness-table)
- [Instantiation](#instantiation)
- [Specialization](#specialization)
- [Template specialization](#template-specialization)
@@ -69,7 +67,7 @@ called a _generic parameter_, to it. So:
- a _generic function_ is a function with at least one compile-time parameter,
which could be an explicit argument to the function or
[deduced](#deduced-parameter);
- a _generic type_ is a function with a compile-time parameter, for example a
- a _generic type_ is a type with a compile-time parameter, for example a
container type parameterized by the type of the contained elements;
- a _generic interface_ is an [interface](#interface) with
[a compile-time parameter](#interface-parameters-and-associated-constants).
@@ -132,7 +130,7 @@ Expected difference between checked and template parameters:
</td>
</tr>
<tr>
<td>supports separate type checking; may also support separate compilation, for example when implemented using dynamic witness tables
<td>supports separate type checking; may also support separate compilation
</td>
<td>separate compilation only to the extent that C++ supports it
</td>
@@ -651,42 +649,34 @@ of another interface, plus some additional API. Types implementing the extended
interface should automatically be considered to have implemented the narrower
interface.
## Witness tables
## Dynamic-dispatch witness table
[Witness tables](https://forums.swift.org/t/where-does-the-term-witness-table-come-from/54334/4)
are an implementation strategy where values passed to a generic type parameter
are compiled into a table of required functionality. That table is then filled
in for a given passed-in type with references to the implementation on the
original type. The generic is implemented using calls into entries in the
witness table, which turn into calls to the original type. This doesn't
necessarily imply a runtime indirection: it may be a purely compile-time
separation of concerns. However, it insists on a full abstraction boundary
between the generic user of a type and the concrete implementation.
Dynamic-dispatch
[witness tables](https://forums.swift.org/t/where-does-the-term-witness-table-come-from/54334/4)
are an implementation strategy that uses a table accessed at runtime to allow
behavior of a function to vary. This allows a function to work with any type
implementing a facet type (such as an interface). For example, the witness table
might contain pointers to the implementations of the functions of the interface.
This can be done to reduce the size of generated code, at the expense of
additional indirection at runtime.
A simple way to imagine a witness table is as a struct of function pointers, one
per method in the interface. However, in practice, it's more complex because it
must model things like associated facets and interfaces.
It can also allow a function to dynamically dispatch when the runtime type of a
value is not known. This is the implementation strategy for
[boxed protocol types in Swift](https://docs.swift.org/swift-book/documentation/the-swift-programming-language/opaquetypes/#Boxed-Protocol-Types)
and
[trait objects in Rust](https://doc.rust-lang.org/book/ch17-02-trait-objects.html).
Note that this often comes with limitations, since for example it is much more
difficult to support when the associated constants of the interface are not
known.
Witness tables are called "dictionary passing" in Haskell. Outside of generics,
a [vtable](https://en.wikipedia.org/wiki/Virtual_method_table) is a witness
table that witnesses that a class is a descendant of an abstract base class, and
is passed as part of the object instead of separately.
Typically a reference to the witness table will be passed separately from the
object, unlike a
[virtual method table](https://en.wikipedia.org/wiki/Virtual_method_table),
which otherwise is very similar to a witness table, "witnessing" the specific
descendant of a base class.
### Dynamic-dispatch witness table
For dynamic-dispatch witness tables, actual function pointers are formed and
used as a dynamic, runtime indirection. As a result, the generic code **will
not** be duplicated for different witness tables.
### Static-dispatch witness table
For static-dispatch witness tables, the implementation is required to collapse
the table indirections at compile time. As a result, the generic code **will**
be duplicated for different witness tables.
Static-dispatch may be implemented as a performance optimization for
dynamic-dispatch that increases generated code size. The final compiled output
may not retain the witness table.
Carbon's approach to using witness tables is detailed in an
[appendix](appendix-witness.md).
## Instantiation
@@ -696,7 +686,7 @@ replaces the template components with the concrete type and its implementation
operations. It allows duck typing and lazy binding. Instantiation implies
template code **will** be duplicated.
Unlike [static-dispatch witness tables](#static-dispatch-witness-table) and
Unlike static-dispatch witness tables (as in Swift) and
[monomorphization (as in Rust)](https://doc.rust-lang.org/book/ch10-01-syntax.html#performance-of-code-using-generics),
this is done **before** type checking completes. Only when the template is used
with a concrete type is the template fully type checked, and it type checks
@@ -728,13 +718,6 @@ This restriction is needed to preserve the ability to perform type checking of
generic definitions that reference a type that can be specialized, without
statically knowing which specialization will be used.
While there is nothing fundamentally incompatible about specialization with
checked generics, even when implemented using witness tables, the result may be
surprising because the selection of the specialized generic happens outside of
the witness-table-based indirection between the generic code and the concrete
implementation. Provided all selection relies exclusively on interfaces, this
still satisfies the fundamental constraints of generics.
## Conditional conformance
Conditional conformance is when you have a parameterized type that has one API
proposals/p0731.md
+1 -1
View File
@@ -248,7 +248,7 @@ kinds of interface parameters. "Multi" parameters would work as described in the
"Deducible" type parameters would only allow one implementation of an interface,
not one per interface & type parameter combination. These deducible type
parameters could be inferred like
[associated types](/docs/design/generics/details.md#associated-types) are. For
[associated types](/docs/design/generics/details.md#associated-facets) are. For
example, we could make a `Stack` interface that took a deducible `ElementType`
parameter. You would only be able to implement that interface once for a type,
which would allow you to infer the `ElementType` parameter like so: