carbon-language/carbon-lang
1
0
Fork 0
mirror of https://github.com/carbon-language/carbon-lang.git synced 2026-10-05 22:02:55 +01:00
Code Issues Packages Projects Releases 10 Wiki Activity

Bitwise operators (#1191)

Browse Source 
Add bitwise and bit-shift operators `&`, `|`, `^`, `<<`, `>>`. Replace C++ `~` with unary prefix `^`.

Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This commit is contained in:
Richard Smith
2022-05-06 14:26:55 -07:00
committed by GitHub
co-authored by Chandler Carruth
parent 0a93289706
commit c4fecf720f
5 changed files with 753 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/design/expressions/README.md
+37 -5
View File
@@ -65,6 +65,12 @@ graph BT
click memberAccess "https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/expressions/member_access.md"
negation["-x"]
click negation "https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/expressions/arithmetic.md"
complement["^x"]
click complement "https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/expressions/bitwise.md"
unary((" "))
as["x as T"]
click as "https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/expressions/implicit_conversions.md"
@@ -80,6 +86,17 @@ graph BT
modulo["x % y"]
click modulo "https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/expressions/arithmetic.md"
bitwise_and>"x & y"]
bitwise_or>"x | y"]
bitwise_xor>"x ^ y"]
click bitwise_and "https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/expressions/bitwise.md"
click bitwise_or "https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/expressions/bitwise.md"
click bitwise_xor "https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/expressions/bitwise.md"
shift["x << y<br>
x >> y"]
click shift "https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/expressions/bitwise.md"
comparison["x == y<br>
x != y<br>
x < y<br>
@@ -91,12 +108,16 @@ graph BT
not["not x"]
click not "https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/expressions/logical_operators.md"
logicalOperand((" "))
and>"x and y"]
click and "https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/expressions/logical_operators.md"
or>"x or y"]
click or "https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/expressions/logical_operators.md"
logicalExpression((" "))
if>"if x then y else z"]
click if "https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/expressions/if.md"
@@ -104,18 +125,23 @@ graph BT
memberAccess --> parens & braces & unqualifiedName
negation --> memberAccess
complement --> memberAccess
unary --> negation & complement
%% Use a longer arrow here to put `not` next to `and` and `or`.
not -----> memberAccess
multiplication & modulo & as --> negation
multiplication & modulo & as & bitwise_and & bitwise_or & bitwise_xor & shift --> unary
addition --> multiplication
comparison --> modulo & addition & as
and & or --> comparison & not
if & expressionEnd --> and & or
comparison --> modulo & addition & as & bitwise_and & bitwise_or & bitwise_xor & shift
logicalOperand --> comparison & not
and & or --> logicalOperand
logicalExpression --> and & or
if & expressionEnd --> logicalExpression
```
The diagram's attributes are:
- Each node represents a precedence group.
- Each non-empty node represents a precedence group. Empty circles are used to
simplify the graph, and do not represent a precedence group.
- When an expression is composed from different precedence groups, the
interpretation is determined by the precedence edges:
@@ -221,11 +247,17 @@ Most expressions are modeled as operators:
| Category | Operator | Syntax | Function |
| ---------- | ------------------------------- | --------- | --------------------------------------------------------------------- |
| Arithmetic | [`-`](arithmetic.md) (unary) | `-x` | The negation of `x`. |
| Bitwise | [`^`](bitwise.md) (unary) | `^x` | The bitwise complement of `x`. |
| Arithmetic | [`+`](arithmetic.md) | `x + y` | The sum of `x` and `y`. |
| Arithmetic | [`-`](arithmetic.md) (binary) | `x - y` | The difference of `x` and `y`. |
| Arithmetic | [`*`](arithmetic.md) | `x * y` | The product of `x` and `y`. |
| Arithmetic | [`/`](arithmetic.md) | `x / y` | `x` divided by `y`, or the quotient thereof. |
| Arithmetic | [`%`](arithmetic.md) | `x % y` | `x` modulo `y`. |
| Bitwise | [`&`](bitwise.md) | `x & y` | The bitwise AND of `x` and `y`. |
| Bitwise | [`\|`](bitwise.md) | `x \| y` | The bitwise OR of `x` and `y`. |
| Bitwise | [`^`](bitwise.md) (binary) | `x ^ y` | The bitwise XOR of `x` and `y`. |
| Bitwise | [`<<`](bitwise.md) | `x << y` | `x` bit-shifted left `y` places. |
| Bitwise | [`>>`](bitwise.md) | `x >> y` | `x` bit-shifted right `y` places. |
| Conversion | [`as`](as_expressions.md) | `x as T` | Converts the value `x` to the type `T`. |
| Comparison | [`==`](comparison_operators.md) | `x == y` | Equality: `true` if `x` is equal to `y`. |
| Comparison | [`!=`](comparison_operators.md) | `x != y` | Inequality: `true` if `x` is not equal to `y`. |
docs/design/expressions/arithmetic.md
+16 -6
View File
@@ -55,11 +55,17 @@ standard library.
## Precedence and associativity
```mermaid
graph TD
negation["-x"] --> multiplicative & modulo
multiplicative>"x * y<br> x / y"] --> additive
additive>"x + y<br> x - y"]
%%{init: {'themeVariables': {'fontFamily': 'monospace'}}}%%
graph BT
negation["-x"]
multiplication>"x * y<br>
x / y"]
addition>"x + y<br>
x - y"]
modulo["x % y"]
multiplication & modulo --> negation
addition --> multiplication
```
<small>[Instructions for reading this diagram.](README.md#precedence)</small>
@@ -153,8 +159,12 @@ programming errors:
will be aborted, or the arithmetic will evaluate to a mathematically
incorrect result, such as a two's complement result or zero. The program
might not in all cases be aborted immediately -- for example, multiple
overflow checks might be combined into one, and if the result of an
arithmetic operation is never observed, the abort may not happen at all.
overflow checks might be combined into one -- but no control flow or memory
access that depends on the value will be performed.
**TODO:** Unify the description of these programming errors with those of
bit-shift domain errors, document the behavior in a common place and link to it
from here.
**TODO:** In a hardened build, should we prefer to trap on overflow, give a
two's complement result, or produce zero? Using zero may defeat some classes of
docs/design/expressions/bitwise.md
+283
View File
@@ -0,0 +1,283 @@
# Bitwise and shift operators
<!--
Part of the Carbon Language project, under the Apache License v2.0 with LLVM
Exceptions. See /LICENSE for license information.
SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-->
<!-- toc -->
## Table of contents
- [Overview](#overview)
- [Precedence and associativity](#precedence-and-associativity)
- [Integer types](#integer-types)
- [Integer constants](#integer-constants)
- [Extensibility](#extensibility)
- [Alternatives considered](#alternatives-considered)
- [References](#references)
<!-- tocstop -->
## Overview
Carbon provides a conventional set of operators for operating on bits:
```
var a: u8 = 5;
var b: u8 = 3;
var c: i8 = -5;
// 250
var complement: u8 = ^a;
// 1
var bitwise_and: u8 = a & b;
// 7
var bitwise_or: u8 = a | b;
// 6
var bitwise_xor: u8 = a ^ b;
// 40
var left_shift: u8 = a << b;
// 2
var logical_right_shift: u8 = a >> 1;
// -3
var arithmetic_right_shift: i8 = c >> 1;
```
These operators have [predefined meanings](#integer-types) for Carbon's integer
types. User-defined types can define the meaning of these operations by
[implementing an interface](#extensibility) provided as part of the Carbon
standard library.
## Precedence and associativity
```mermaid
%%{init: {'themeVariables': {'fontFamily': 'monospace'}}}%%
graph BT
complement["^x"]
bitwise_and>"x & y"]
bitwise_or>"x | y"]
bitwise_xor>"x ^ y"]
shift["x << y<br>
x >> y"]
bitwise_and & bitwise_or & bitwise_xor & shift --> complement
```
<small>[Instructions for reading this diagram.](README.md#precedence)</small>
Parentheses are required when mixing different bitwise and bit-shift operators.
Binary `&`, `|`, and `^` are left-associative. The bit-shift operators `<<` and
`>>` are non-associative.
```
// ✅ Same as (1 | 2) | 4, evaluates to 7.
var a: i32 = 1 | 2 | 4;
// ❌ Error, parentheses are required to distinguish between
// (3 | 5) & 6, which evaluates to 6, and
// 3 | (5 & 6), which evaluates to 7.
var b: i32 = 3 | 5 & 6;
// ❌ Error, parentheses are required to distinguish between
// (1 << 2) << 3, which evaluates to 4 << 3 == 32, and
// 1 << (2 << 3), which evaluates to 1 << 16 == 65536.
var c: i32 = 1 << 2 << 3;
// ❌ Error, can't repeat the `^` operator. Use `^(^4)` or simply `4`.
var d: i32 = ^^4;
```
## Integer types
Bitwise and bit-shift operators are supported for Carbon's built-in integer
types, and, unless that behavior is [overridden](#extensibility), for types that
can be implicitly converted to integer types, as follows:
For binary bitwise operators, if one operand has an integer type and the other
operand can be implicitly converted to that type, then it is. If both operands
are of integer type, this results in the following conversions:
- If the types are `uN` and `uM`, or they are `iN` and `iM`, the operands are
converted to the larger type.
- If one type is `iN` and the other type is `uM`, and `M` < `N`, the `uM`
operand is converted to `iN`.
A built-in binary bitwise `&`, `|`, or `^` operation is performed if, after the
above conversion step, the operands have the same integer type. The result type
is that type, and the result value is produced by applying the relevant
operation -- AND, OR, or XOR -- to each pair of corresponding bits in the input,
including the sign bit for a signed integer type.
A built-in complement operation is performed if the operand can be implicitly
converted to an integer type. The result type is that type, and the result value
is produced by flipping all bits in the input, including the sign bit for a
signed integer type. `^a` is equivalent to `a ^ x`, where `x` is the
all-one-bits value of the same type as `a`.
A built-in bit-shift operation is performed if both operands are, or can be
implicitly converted to, integer types. The result type is the converted type of
the first operand. The result value is produced by shifting the first operand
left for `<<` or right for `>>` a number of positions equal to the second
operand. Vacant positions are filled with `0` bits, except for a right shift
where the first operand is of a signed type and has a negative value, in which
case they are filled with `1` bits.
For the purposes of bit-shifts, bits are ordered by significance, with the most
significant bit being the leftmost bit and the least significant bit being the
rightmost bit. As a consequence, in the absence of overflow a left shift is
equivalent to a multiplication by a power of 2 and a right shift is equivalent
to a division by a power of two, rounding downwards.
The second operand of a bit-shift is required to be between zero (inclusive) and
the bit-width of the first operand (exclusive); it is a programming error if the
second operand is not within that range.
- In a development build, they will be caught immediately when they happen at
runtime.
- In a performance build, the optimizer may assume that this programming error
does not occur.
- In a hardened build, the result will have well the defined behavior of
either aborting the program or performing a shift of an unspecified number
of bits, which if wider than the first operand will result in `0` or `-1`.
In the case where the program is aborted, the program might not in all cases
be aborted immediately -- for example, multiple checks might be combined
into one -- but no control flow or memory access that depends on the value
will be performed.
**TODO:** Unify the description of these programming errors with those of
arithmetic overflow, document the behavior in a common place and link to it from
here.
## Integer constants
These operations can also be applied to a pair of integer constants, or to an
integer constant and a value of integer type, as follows:
- If any binary bitwise or bit-shift operator is applied to two integer
constants, or the unary `^` operator is applied to an integer constant, the
result is an integer constant. Integer constants are treated as having
infinitely many high-order bits, where all but finitely many of those bits
are sign bits. For example, `-1` comprises infinitely many `1` bits. Note
that there is no difference between an arithmetic and a logical right shift
on an integer constant, because every bit always has a higher-order bit to
shift from.
- It is easy to produce extremely large numbers by left-shifting an
integer constant. For example, the binary representation of
`1 << (1 << 1000)` is thought to be substantially larger than the total
entropy in the observable universe. In practice, Carbon implementations
will set a much lower limit on the largest integer constant that they
support.
- If a binary bitwise `&`, `|`, or `^` operation is applied to an integer
constant and a value of an integer type to which the constant can be
implicitly converted, the operand that is an integer constant is implicitly
converted to the integer type and the computation is performed as described
[above](#integer-types).
- If the second operand of a bit-shift operator is an integer constant and the
first opreand is not, and the second operand is between 0 (inclusive) and
the bit-width of the first operand (exclusive), the integer constant is
converted to an integer type that can hold its value and the computation is
performed as described above.
Other operations involving integer constants are invalid. For example, a bitwise
`&` between a `u8` and an integer constant `500` is invalid because `500`
doesn't fit into `u8`, and `1 << n` is invalid if `n` is an integer variable
because we don't know what type to use to compute the result.
Note that the unary `^` operator applied to a non-negative integer constant
results in a negative integer constant, and the binary `^` operator gives a
negative result if exactly one of the input operands was negative. For example,
`^0 == -1` evaluates to `true`.
## Extensibility
Bitwise and shift operators can be provided for user-defined types by
implementing the following family of interfaces:
```
// Unary `^`.
interface BitComplement {
let Result:! Type = Self;
fn Op[me: Self]() -> Result;
}
```
```
// Binary `&`.
interface BitAndWith(U:! Type) {
let Result:! Type = Self;
fn Op[me: Self](other: U) -> Result;
}
constraint BitAnd {
extends BitAndWith(Self) where .Result = Self;
}
```
```
// Binary `|`.
interface BitOrWith(U:! Type) {
let Result:! Type = Self;
fn Op[me: Self](other: U) -> Result;
}
constraint BitOr {
extends BitOrWith(Self) where .Result = Self;
}
```
```
// Binary `^`.
interface BitXorWith(U:! Type) {
let Result:! Type = Self;
fn Op[me: Self](other: U) -> Result;
}
constraint BitXor {
extends BitXorWith(Self) where .Result = Self;
}
```
```
// Binary `<<`.
interface LeftShiftWith(U:! Type) {
let Result:! Type = Self;
fn Op[me: Self](other: U) -> Result;
}
constraint LeftShift {
extends LeftShiftWith(Self) where .Result = Self;
}
```
```
// Binary `>>`.
interface RightShiftWith(U:! Type) {
let Result:! Type = Self;
fn Op[me: Self](other: U) -> Result;
}
constraint RightShift {
extends RightShiftWith(Self) where .Result = Self;
}
```
Given `x: T` and `y: U`:
- The expression `^x` is rewritten to `x.(BitComplement.Op)()`.
- The expression `x & y` is rewritten to `x.(BitAndWith(U).Op)(y)`.
- The expression `x | y` is rewritten to `x.(BitOrWith(U).Op)(y)`.
- The expression `x ^ y` is rewritten to `x.(BitXorWith(U).Op)(y)`.
- The expression `x << y` is rewritten to `x.(LeftShiftWith(U).Op)(y)`.
- The expression `x >> y` is rewritten to `x.(RightShiftWith(U).Op)(y)`.
Implementations of these interfaces are provided for built-in types as necessary
to give the semantics described above.
## Alternatives considered
- [Use different symbols for bitwise operators](/proposals/p1191.md#use-different-symbols-for-bitwise-operators)
- [Provide different operators for arithmetic and logical shifts](/proposals/p1191.md#provide-different-operators-for-arithmetic-and-logical-shifts)
- [Provide rotate operators](/proposals/p1191.md#provide-rotate-operators)
- [Guarantee the behavior of large shifts](/proposals/p1191.md#guarantee-behavior-of-large-shifts)
- [Support shifting a constant by a variable](/proposals/p1191.md#support-shifting-a-constant-by-a-variable)
## References
- Proposal
[#1191: bitwise and shift operators](https://github.com/carbon-language/carbon-lang/pull/1191).
docs/design/expressions/implicit_conversions.md
+16
View File
@@ -140,6 +140,21 @@ except:
have singleton types; see issue
[#508](https://github.com/carbon-language/carbon-lang/issues/508).
In addition to the above rules, a negative integer constant `k` can be
implicitly converted to the type `uN` if the value `k` + 2<sup>N</sup> can be
exactly represented, and converts to that value. Note that this conversion
violates the "semantics-preserving" test. For example, `(-1 as u8) as i32`
produces the value `255` whereas `-1 as i32` produces the value `-1`. However,
this conversion is important in order to allow bitwise operations with masks, so
we allow it:
```
// We allow ^0 == -1 to convert to `u32` to represent an all-ones value.
var a: u32 = ^0;
// ^4 == -5 is negative, but we want to allow it to convert to u32 here.
var b: u32 = a & ^4;
```
### Same type
The following conversion is available for every type `T`:
@@ -214,6 +229,7 @@ types.
- [Provide no implicit conversions](/proposals/p0820.md#no-conversions)
- [Provide no extensibility](/proposals/p0820.md#no-extensibility)
- [Apply implicit conversions transitively](/proposals/p0820.md#transitivity)
- [Do not allow negative constants to convert to unsigned types](/proposals/p1191.md#converting-complements-to-unsigned-types)
## References
proposals/p1191.md
+401
View File
@@ -0,0 +1,401 @@
# Bitwise and shift operators
<!--
Part of the Carbon Language project, under the Apache License v2.0 with LLVM
Exceptions. See /LICENSE for license information.
SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-->
[Pull request](https://github.com/carbon-language/carbon-lang/pull/1191)
<!-- toc -->
## Table of contents
- [Problem](#problem)
- [Background](#background)
- [Overflow in shift operators](#overflow-in-shift-operators)
- [Proposal](#proposal)
- [Details](#details)
- [Rationale](#rationale)
- [Alternatives considered](#alternatives-considered)
- [Use different symbols for bitwise operators](#use-different-symbols-for-bitwise-operators)
- [Use a multi-character spelling](#use-a-multi-character-spelling)
- [Don't provide an xor operator](#dont-provide-an-xor-operator)
- [Use `~`, or some other symbol, for complement](#use--or-some-other-symbol-for-complement)
- [Provide different operators for arithmetic and logical shifts](#provide-different-operators-for-arithmetic-and-logical-shifts)
- [Provide rotate operators](#provide-rotate-operators)
- [Guarantee behavior of large shifts](#guarantee-behavior-of-large-shifts)
- [Support shifting a constant by a variable](#support-shifting-a-constant-by-a-variable)
- [Converting complements to unsigned types](#converting-complements-to-unsigned-types)
<!-- tocstop -->
## Problem
Carbon needs operations for working with bit representations of values.
## Background
C++ provides four bitwise operations for Boolean algebra: complement (`~`), and
(`&`), or (`|`), and xor (`^`). These are all useful in bit-maipulation code
(although `^` is used substantially less than the others). In addition, C++
provides two bit-shift operators `<<` and `>>` that can perform three different
operations: left shift, arithmetic right shift, and logical right shift. The
meaning of `>>` is determined by the signedness of the first operand.
C and Swift use the same set of operators as C++. Rust uses most of the same
operators, but uses `!` instead of `~` for complement, unifying it with the
logical not operator, which is spelled `not` in Carbon and as `!` in Rust and
C++. Go uses most of the same operators as C++, but uses unary prefix `^`
instead of `~` for complement, mirroring binary `^` for xor.
In addition to the operators provided by C and C++, bit-rotate operators are
present in some languages, and a short notation for them may be convenient.
Finally, there are other non-degenerate binary Boolean operations with no common
operator symbol:
- The "implies" operator (equivalent to `~a | b`).
- The "implied by" operator (equivalent to `a | ~b`).
- The complement of each of the other operators (NAND, NOR, XNOR, "does not
imply", "is not implied by").
### Overflow in shift operators
The behavior of shift operators in C++ had a turbulent past. The behavior of
shift operators has always been undefined if the right operand is out of range
-- not between zero inclusive and the bit-width of the left operator exclusive
-- but other restrictions have varied:
- Unsigned left shift has never had any restrictions on the first operand.
- For signed left shift:
- In C++98, the result was fully unspecified.
- In C++11, the result was specified only if the first operand was
non-negative and the result fit into the destination type -- that is, if
no 1 bit is shifted into the sign bit.
- In C++14, the result was specified only if the first operand was
non-negative and the result fit into the unsigned type corresponding to
the destination type -- that is, if no 1 bit is shifted out of the sign
bit.
- In C++20 onwards, there are no restrictions beyond a range restriction
on the right operand, and the result is otherwise always specified, even
if the left operand is negative.
- Unsigned right shift has never had any restrictions on the first operand.
- For signed right shift:
- In C++17 and earlier, if the left operand is negative, the result is
implementation-defined.
- In C++20 onwards, the result is always specified, even if the left
operand is negative.
There is a clear trend towards defining more cases, following two's complement
rules.
## Proposal
Use the same operator set as C++, but replace `~` with unary prefix `^`.
Define the behavior for all cases of `<<` and `>>` except where the right
operand is either negative or is greater than or equal to the bit-width of the
left operand.
## Details
See changes to the design, and in particular
[the new section on bitwise and shift operators](/docs/design/expressions/bitwise.md).
## Rationale
- [Performance-critical software](/docs/project/goals.md#performance-critical-software)
- Bit operations are important low-level primitives. Providing operators
for them is important in order to allow low-level high-performance code
to be written elegantly in Carbon.
- By not defining semantics for `<<` and `>>` when the right-hand operand
is out of range, we can directly use hardware instructions for these
operations whose behavior in these cases vary between architectures.
- [Code that is easy to read, understand, and write](/docs/project/goals.md#code-that-is-easy-to-read-understand-and-write)
- Using operator notation rather than function call notation for bitwise
operators improves readability in code making heavy use of these
operations.
- [Practical safety and testing mechanisms](/docs/project/goals.md#practical-safety-and-testing-mechanisms)
- Carbon follows C++ in treating `<<` and `>>` as programming errors when
the right hand operand is out of range, but Carbon guarantees that such
errors will not directly result in unbounded misbehavior in hardened
builds.
- [Modern OS platforms, hardware architectures, and environments](/docs/project/goals.md#modern-os-platforms-hardware-architectures-and-environments)
- All hardware architectures we care to support are natively two's
complement architectures, and that assumption allows us to define the
semantics of shift operators in the way that makes the most sense for
such architectures.
- Our bitwise operations make no assumptions about the endianness of the
hardware architecture, although the shift operators make the most sense
on a little-endian or big-endian architecture, which are the only
endiannesses we expect to see in modern hardware platforms.
- [Interoperability with and migration from existing C++ code](/docs/project/goals.md#interoperability-with-and-migration-from-existing-c-code)
- The same set of operators is provided as in C++, making it easy for
programmers and programs to migrate, with the exception that the `~`
operator is mechanically replaced with `^`. This change is expected to
be easy for programmers to accommodate, especially given that Rust's
choice to replace `~` with `!` does not seem to be a source of sustained
complaints.
- The extensiblity support reflects the full scope of operator overloading
in C++, permitting separate overloading of each of the bitwise operators
with custom return types. This should allow smooth interoperability with
C++ overloaded operators.
## Alternatives considered
### Use different symbols for bitwise operators
The operator syntax for bitwise operators was decided in
[#545](https://github.com/carbon-language/carbon-lang/issues/545). Some of the
specific alternatives considered are discussed below.
#### Use a multi-character spelling
We considered using various multi-character spellings for the bitwise and, or,
xor, and complement operators:
- `&:`, `|:`, `^:`, `~:`
- `.&.`, `.|.`, `.^.`, `.~.`
- `.*.`, `.+.`, `.!=.`, `.!.`
- `/\`, `\/`, `(+)`, `-|`
- `bitand`, `bitor`, `bitxor`, `compl`
The advantage of switching to such a set of operators is that this would free up
the single-character `&`, `|`, `^`, and `~` tokens for other uses that may occur
more frequently in Carbon programs. We have some candidate uses for these
operators already:
- `&` is used for combining interfaces and as a unary address-of operator.
- `|` may be useful for sum types, for alternatives in patterns, or as another
kind of bracket as in
[Ruby's lambda notation](https://ruby-doc.org/docs/ruby-doc-bundle/Manual/man-1.4/syntax.html#iter).
- `~` may be useful as a destructive move notation.
- `^` may be useful as a postfix pointer dereference operator.
Other motivations for switching to a different set of spellings include:
- There are some concerns that `<` and `<<` are visually similar, analogous to
`&` and `&&`.
- Carbon has moved away from `&&` and other punctuation based _logical_
operators and towards keywords like `and`. Bitwise operators could similarly
switch to keywords like `bitand`.
However, moving substantially away from the C++ operator set comes with a set of
concerns:
- There are strong established expectations and intuitions about these
operators and their spellings among C++ practitioners.
- In some of the code that uses these operators, they are used a lot, and a
more cumbersome operator may consequently cause an outsized detriment on
readability.
- These operations are used particularly in the area of low-level,
high-performance code, which is an area for which we want Carbon to be
especially appealing. Using short operators for these operations
demonstrates our commitment to providing good support for such code.
- Even if we didn't use these operators as bit operators, we would still want
to exercise caution when using them for some other purpose to avoid
surprising C++ developers.
- While some visual similarity exists such as between `<` and `<<`, the
contexts in which these operators are used are sufficiently different to
avoid any serious concerns.
- The primary motivation of using `and` instead of `&&` doesn't apply for
bitwise operators: the _logical_ operator represents _control flow_.
Separating logical and bitwise "and" operations more visibly seems
especially important because of this control flow semantic difference.
Without any control flow and with the keywords being significantly longer
for bitwise operations, the above considerations were the dominant ones that
led us to stick with familiar `&` and `|` spellings.
#### Don't provide an xor operator
We considered omitting the `^` operator, providing this functionality in some
other way, such as a named function or an `xor` keyword, while keeping the `&`
and `|` symbols for bitwise operations. We could take a similar approach for the
complement operation, such as by using a `compl` keyword. The primary motivation
was to avoid spending two precious operator characters on relatively uncommon
operations. However, we did not want to apply the same change to `&` and `|`,
and it seemed important to maintain consistency between the three binary bitwise
operators from C++.
Using `^` for both operations provides some of the benefits here, allowing us to
reclaim `~`, without introducing the inconsistency that would result from using
keywords.
#### Use `~`, or some other symbol, for complement
We could follow C++ and use `~` as the complement operator. However, using `~`
for this purpose spends a precious operator character on a relatively uncommon
operation, and `~` is often visually confusible with `-`, creating the potential
for readability problems. Also, in C++, `~` is overloaded to also refer to
destruction, and we may want to use the same notation for destruction or
destructive move semantics in Carbon.
We found `^` to be a satisfying alternative with a good rationale and mnemonic:
`^` is a bit-flipping operator -- `a ^ b` flips the bits in `a` that are
specified in `b` -- and complement is an operator that flips _all_ the bits.
`^a` is equivalent to `a ^ n`, where `n` is the all-one-bits value in the type
of `a`.
We also considered using `!` for complement, like Rust does. Unlike in Rust,
this would not be a generalization of `!` on `bool`, because we use `not` for
`bool` negation, and repurposing `!` in this way compared to C++ seemed
confusing.
### Provide different operators for arithmetic and logical shifts
We could provide different operators for arithmetic right shift and logical
right shift. This might allow programmers to better express their intent.
However, it seems unnecessary, as using the type of the left operand is a
strategy that doesn't appear to have caused significant problems in practice in
the languages that have followed it.
Basing the kind of shift on the signedness of the left operand also follows from
viewing a negative number as having an infinite number of leading 1 bits, which
is the underlying mathematical model behind the two's complement representation.
### Provide rotate operators
We could provide bitwise rotation operators. However, there doesn't seem to be a
sufficient need to justify adding another operator symbol for this purpose.
### Guarantee behavior of large shifts
Logically, the behavior of shifts is meaningful for all values of the second
operand:
- A shift by an amount greater than or equal to the bit-width of the first
operand will shift out all of the original bits, producing a result where
all value bits are the same.
- A shift in one direction by a negative amount is treated as a shift in the
opposite direction by the negation of that amount.
Put another way, we can view the bits of the first operand as an N-bit window
into an infinite sequence of bits, with infinitely many leading sign bits (all
zeroes for an unsigned value) and infinitely many trailing zero bits after a
notional binary point, and a shift moves that window around. Or equivalently, a
shift is always a multiplication by 2<sup>N</sup> followed by rounding and
wrapping.
We could provide the correct result for all shifts, regardless of the magnitude
of the second operand. This is the approach taken by Python, except that Python
rejects negative shift counts. The primary reason we do not do this is lack of
hardware support. For example, x86 does not have an instruction to perform this
operation. Rather, x86 provides shift instructions that mask off all bits of the
second operand except for the bottom 5 or 6, meaning that a left shift of a
64-bit operand by 64 will return the operand unchanged rather than producing
zero.
We could instead provide x86-like behavior, guaranteeing to consider only the
lowest `N` bits of the second operand when the first operand is an `iN` or `uN`.
This is the approach taken by Java for its 32-bit `int` type and 64-bit `long`
type, where the second operand is taken modulo 32 or 64, respectively, and in
JavaScript, where operands of bitwise and bit-shift operators are treated as
32-bit integers and the second operand of a shift is taken modulo 32. This
approach would provide an operation that can be implemented by a single
instruction on x86 platforms when `N` is 32 or 64, and for all smaller types and
for all other platforms the operation can be implemented with two instructions:
a mask and a shift. For larger types, single-instruction support may not be
available, but nonetheless the performance will be close to optimal, requiring
at most one additional mask. There is still some performance cost in some cases,
but the primary reason we do not do this is the same reason we choose to not
define signed integer overflow: this masked result is unlikely to be the value
that the developer actually wanted.
Instead of the above options, Carbon treats a second operand that is not in the
interval [0, N) as a programming error, just like signed integer overflow:
- Debugging builds can detect and report this error without the risk of false
positives.
- Performance builds can optimize on the basis that this situation will not
occur, and can in particular use the dedicated x86 instructions that ignore
the high order bits of the second operand.
- Optimized builds guarantee that either the programming error results in
program termination or that _some_ value is produced, and moreover that said
value is the result of applying _some_ mathematical shift to the input. For
example, it's valid for an `i32` shift to be implemented by an x86 64-bit
shift that will produce 0 if the second operand is in [32, 63) but that will
treat a second operand of, say, 64 or -64 the same as 0.
### Support shifting a constant by a variable
We considered various ways to support
```
var a: i32 = ...;
var b: i32 = 1 << a;
var c: i32 = 1234 >> a;
```
with no explicit type specified for the first operand of a bit-shift operator.
We considered the following options:
- Use the type of the second operand as the result type. This would be
surprising, because the type of the second operand doesn't otherwise
influence the result type of a built-in bit-shift operator.
- Use some suitable integer type that can fit the first operand. However, this
is unlikely to do the right thing for a left-shift, and will frequently pick
either a type that's too large, resulting in the program being rejected due
to narrowing, or a type that's too small, resulting in a program that has
undefined behavior due to the second operand being too large. We could apply
this approach only for right shifts, but it was deemed too inconsistent to
use different rules for left and right shifts.
- We could find a way to defer picking the type in which the operation is
performed until later. For example, we could treat `1 << a` as a value of a
new type that carries its left-hand operand as a type parameter and its
right-hand operand as runtime state, and allow that type to be converted in
the same way as its integer constant. However, this would introduce
substantial complexity: reasonable and expected uses such as
```
var mask: u32 = (1 << a) - 1;
```
would require a second new type for a shifted value plus an offset, and
general support would require a facility analogous to
[expression templates](https://en.wikipedia.org/wiki/Expression_templates).
Further, this facility would allow implicit conversions that notionally
overflow, such as would happen in the above example when `a` is greater
than 32.
In the absence of a good approach, we disallow such conversions for now. The
above example can be written as:
```
var a: i32 = ...;
var b: i32 = (1 as i32) << a;
var c: i32 = (1234 as i32) >> a;
```
### Converting complements to unsigned types
We view an integer constant has having infinitely many high-order sign bits
followed by some number of lower-order value bits. As a consequence, the
complement of a positive integer constant is negative. As a result, some
important forms of initialization use a negative integer constant initializer
for an unsigned type:
```
// Initializer here is the integer value -8.
var mask: u32 = ^7;
```
We considered some options for handling this:
- We could allow negative integer constants to convert to unsigned types if
doing so only discards sign bits. This violates the "semantics-preserving"
rule for implicit conversions.
- We could change our model of integer constants to distinguish between
"masks" -- numbers with infinitely many 1 bits preceding the value bits that
are nonetheless not considered to be negative. This was considered to
introduce too much complexity.
- We could allow conversions to unsigned types from signed types and negative
constants in general, or at least in cases where the signed operand is no
wider than the unsigned type, and perform wrapping. The latter option seems
plausible, but we don't have sufficient motivation for it, and were worried
about a risk of bugs from allowing an implicit conversion at runtime that
converts a negative value to an unsigned type.
- We could reject such initializations, with an explicit conversion required
to convert such values to unsigned types. This seems to present unacceptable
ergonomics for code performing bit-manipulation.
On balance, our preferred option was to permit implicit conversions from
negative literals to unsigned types so long as we only discard sign bits.