From e257051612e4217295e206fd3274fc75e22d206a Mon Sep 17 00:00:00 2001 From: Richard Smith Date: Fri, 31 Jan 2025 19:03:30 -0800 Subject: [PATCH] No predeclared identifiers, `Core` is a keyword (#4864) Introduce a principle that the Carbon language should not encroach on the developer's namespace. Satisfy this principle by making `Core` a keyword. --------- Co-authored-by: Chandler Carruth --- .../code_and_name_organization/README.md | 5 +- docs/design/lexical_conventions/words.md | 4 +- .../principles/namespace_cleanliness.md | 122 ++++++++++++ proposals/p4864.md | 177 ++++++++++++++++++ .../textmate/Syntaxes/carbon.tmLanguage.json | 2 +- utils/tree_sitter/queries/highlights.scm | 2 + 6 files changed, 309 insertions(+), 3 deletions(-) create mode 100644 docs/project/principles/namespace_cleanliness.md create mode 100644 proposals/p4864.md diff --git a/docs/design/code_and_name_organization/README.md b/docs/design/code_and_name_organization/README.md index 2c527c0c3d5e..58d3ddc9bac8 100644 --- a/docs/design/code_and_name_organization/README.md +++ b/docs/design/code_and_name_organization/README.md @@ -546,6 +546,7 @@ the caller. ```regex import IDENTIFIER (library NAME_PATH)?; +import Core (library NAME_PATH)?; import library NAME_PATH; import library default; ``` @@ -554,7 +555,9 @@ An import with a package name `IDENTIFIER` declares a package entity named after the imported package, and makes API entities from the imported library available through it. `Main` cannot be imported from other packages; in other words, only `import library NAME_PATH` syntax can be used to import from `Main`. Imports of -`Main//default` are invalid. +`Main//default` are invalid. The keyword `Core` can be used as a package name in +an import in order to import portions of the standard library that are not part +of the prelude. The full name path is a concatenation of the names of the package entity, any namespace entities applied, and the final entity addressed. Child namespaces or diff --git a/docs/design/lexical_conventions/words.md b/docs/design/lexical_conventions/words.md index 2b5dae3a0831..43f0aff81ee6 100644 --- a/docs/design/lexical_conventions/words.md +++ b/docs/design/lexical_conventions/words.md @@ -39,7 +39,7 @@ in Unicode Normalization Form C (NFC). @@ -54,6 +54,7 @@ The following words are interpreted as keywords: - `auto` - `base` - `break` +- `Core` - `case` - `choice` - `class` @@ -92,6 +93,7 @@ The following words are interpreted as keywords: - `return` - `returned` - `Self` +- `self` - `template` - `then` - `type` diff --git a/docs/project/principles/namespace_cleanliness.md b/docs/project/principles/namespace_cleanliness.md new file mode 100644 index 000000000000..bc9a6ba11007 --- /dev/null +++ b/docs/project/principles/namespace_cleanliness.md @@ -0,0 +1,122 @@ +# Principle: Namespace cleanliness + + + + + +## Table of contents + +- [Background](#background) +- [Principle](#principle) +- [Applications of this principle](#applications-of-this-principle) +- [Exceptions](#exceptions) +- [Alternatives considered](#alternatives-considered) + + + +## Background + +Names and entities in a program can come from multiple sources -- from a local +declaration, from an import, from the standard library, or from the prelude. +Names can be imported from Carbon code or imported or derived from code written +in another language such as C++ or an interface description language such as +that of Protobuf, MIDL, or CORBA. Names can be selected for use in a program +that language designers later decide they want to use as keywords. And in order +to use a library, it is sometimes necessary to redeclare the same name that that +library chose. + +This puts a lot of pressure on the language to support a free choice of naming +for entities. Different languages make different choices in this space: + +- Many languages have a set of keywords that are not usable as identifiers, + with no workaround. If this set collides with a name needed by user code, + the user is left to solve this problem, often by rewriting the identifier in + some way (`klass` or `class_`), which sometimes conflicts with the general + naming convention used by the code. And conversely, suboptimal choices are + made for new language keywords to avoid causing problems for existing code. +- C and C++ reserve a family of identifiers, such as those beginning with an + underscore and a capital letter. However, it's not clear which audiences the + reserved identifiers are for, and this leads to collisions between standard + library vendors and compiler authors, as well as between implementation + extensions and language extensions. + - MSVC provides a `__identifier(keyword)` extension that allows using a + keyword as an identifier. This extension is also implemented by Clang in + `-fms-extensions` mode. + - GCC provides an `__asm__(symbol)` extension that allows a specific + symbol to be assigned to an object or function, which provides ABI + compatibility but not source compatibility with code that uses a keyword + as a symbol name. This extension is also implemented by Clang. +- Python reserves some identifiers but still allows them to be freely + overwritten (such as `bool`) and reserves some identifiers but rejects + assignment to them (such as `True`). +- Rust provides a raw identifier syntax to allow most identifiers with + reserved meaning to be used by a program, but + [not all](https://internals.rust-lang.org/t/raw-identifiers-dont-work-for-all-identifiers/9094): + `self`, `Self`, `super`, `extern`, and `crate` cannot be used as raw + identifiers. Rust also predeclares a large number of library names in every + file, but allows them to be shadowed by user declarations with the same + name. +- Swift provides a raw identifier syntax using backticks: `` `class` ``, and + is + [considering](https://github.com/swiftlang/swift-evolution/blob/main/proposals/0451-escaped-identifiers.md) + extending this to allow arbitrary non-word-shaped character sequences + between the `` ` ``s. + +Carbon provides +[raw identifier syntax](/docs/design/lexical_conventions/words.md#raw-identifiers), +for example `r#for`, to allow using keywords as identifiers. Carbon also intends +to have strict shadowing rules that may make predeclared identifiers that are +_not_ keywords difficult or impossible to redeclare and use in inner scopes. + +## Principle + +In Carbon, the language does not encroach on the developer's namespace. There +are no predeclared or reserved identifiers. In cases where the language gives +special meaning to a word or to word-shaped syntax such as `i32`, that special +meaning can always be undone with raw identifier syntax, `r#`. + +Conversely, when adding language keywords, we will not select an inferior +keyword merely to avoid the risk of breaking existing programs. We will still +take into account how often it is desirable to use the word as an identifier, +including in domain-specific contexts, because that is a factor in whether it +would make a good keyword, and will manage the rollout of new keywords to make +it straightforward to migrate existing uses to `r#` or a different name. + +## Applications of this principle + +- Words like `final` and `base` that only have special meaning in a few + contexts, and could otherwise be made available as identifiers, are keywords + in Carbon. `{.base = ...}` and `{.r#base = ...}` specify different member + names. +- Words like `self` that are declared by the developer but nonetheless have + special language-recognized meaning are keywords in Carbon. `[self:! Self]` + introduces a self parameter; `[r#self:! Self]` introduces a deduced + parameter. +- Words like `Self` that are implicitly declared by the language in some + contexts are keywords, even though we could treat them as user-declared + identifiers that are merely implicitly declared in some cases. +- Words like `i32` that are treated as type literals rather than keywords can + be used as identifiers with raw identifier syntax `r#i32`. +- There are no predeclared identifiers imported from the prelude. If an entity + is important enough to be available by default, we should add a keyword, and + allow the name of the entity to be used for other purposes with `r#`. +- The predeclared package name `Core` is a keyword. A package named `r#Core` + is an unrelated package, and `Core.foo` always refers to members of the + predeclared `Core` package. + +## Exceptions + +For now, we reserve the package names `Main` and `Cpp`. These names aren't +predeclared in any scope, and the name `Main` is not even usable from within +source files to refer to the main package. However, there is currently no way to +avoid collisions between the package name `Cpp` and a top-level entity named +`Cpp` if they are both used in the same source file. + +## Alternatives considered + +- [Have both predeclared identifiers and keywords](/proposals/p4864.md#have-both-predeclared-identifiers-and-keywords) +- [Reserve words with a certain spelling](/proposals/p4864.md#reserve-words-with-a-certain-spelling) diff --git a/proposals/p4864.md b/proposals/p4864.md new file mode 100644 index 000000000000..eaf9e1bcfef4 --- /dev/null +++ b/proposals/p4864.md @@ -0,0 +1,177 @@ +# No predeclared identifiers, `Core` is a keyword + + + +[Pull request](https://github.com/carbon-language/carbon-lang/pull/4864) + + + +## Table of contents + +- [Abstract](#abstract) +- [Problem](#problem) +- [Background](#background) +- [Proposal](#proposal) +- [Details](#details) +- [Rationale](#rationale) +- [Future work](#future-work) + - [Package name `Cpp`](#package-name-cpp) + - [Package name `Main`](#package-name-main) +- [Alternatives considered](#alternatives-considered) + - [Have both predeclared identifiers and keywords](#have-both-predeclared-identifiers-and-keywords) + - [Reserve words with a certain spelling](#reserve-words-with-a-certain-spelling) + + + +## Abstract + +Introduce a principle that the Carbon language should not encroach on the +developer's namespace. Satisfy this principle by making `Core` a keyword. + +## Problem + +Ongoing design work needs rules for how to expose types such as a primitive +array type to Carbon code, and in particular, if we choose to make it available +by default, whether that should be accomplished by a keyword or a predeclared +identifier. + +## Background + +See the +[Background section of the added principle](/docs/project/principles/namespace_cleanliness.md#background). + +## Proposal + +We choose to not have any predeclared identifiers in Carbon. If a word has +special meaning to the language, then that word is a keyword, and a plain +identifier with no special meaning is always available using raw identifier +syntax. + +## Details + +See [the principle document](/docs/project/principles/namespace_cleanliness.md) +for details of the added principle. In addition, we make one change and one +clarification: + +- `Core` is changed from being an identifier that happens to be the name of + the Carbon standard library, and happens to be predeclared in every source + file as naming that library, to being a keyword. The keyword can only be + used: + + - When importing the `Core` package. + - When implementing the `Core` package as part of the language + implementation. + - As a keyword naming the `Core` package, much like the `package` keyword. + + The identifier `r#Core` can be used freely and does not conflict with the + keyword. This includes use of `r#Core` as the name of a package. Language + constructs that are defined in terms of entities in the `Core` package refer + specifically to the package named with the _keyword_ `Core`, not to any + other entity named `Core`. + +- The `self` keyword is now included in the list of keywords. It is already + treated as a keyword by the toolchain. + +## Rationale + +- [Language tools and ecosystem](/docs/project/goals.md#language-tools-and-ecosystem) + - Code generation tools can have a uniform handling for all words with + special meaning, with no need to alter the spelling of names from other + languages. + - Language tools can determine the meaning of `Core.` without + needing to do any name lookup or sophisticated analysis. +- [Software and language evolution](/docs/project/goals.md#software-and-language-evolution) + - Migration between versions of Carbon with a changed set of reserved + words can be done uniformly. + - Adding names to the prelude remains a non-breaking change. Adding new + predeclared names requires adding a keyword, with the same cost and + value tradeoffs regardless of whether the keyword names a library + declaration or introduces new language syntax. +- [Code that is easy to read, understand, and write](/docs/project/goals.md#code-that-is-easy-to-read-understand-and-write) + - Syntax highlighting tools can easily distinguish between words with + special meaning and words with program-defined meaning. + - The meaning of core language constructs can be defined as a rewrite in + terms of `Core.` without concern that `Core` may have some + different local interpretation. +- [Interoperability with and migration from existing C++ code](/docs/project/goals.md#interoperability-with-and-migration-from-existing-c-code) + - All C++ identifiers are nameable from Carbon code without conflicts. + Virtual functions introduced in Carbon can be overridden in Carbon + regardless of their name. C++ code can be migrated to Carbon even if its + name in C++ has special meaning in Carbon. +- [Principle: Prefer providing only one way to do a given thing](/docs/project/principles/one_way.md) + - This proposal specifies that there is only one way to give words special + meaning in Carbon, and one way to resolve issues if that special meaning + conflicts with another desired meaning. + +## Future work + +### Package name `Cpp` + +The special package name `Cpp` that refers to code written in C++ is not made a +keyword by this proposal, but this proposal is also not deciding that it should +_not_ be a keyword. While this name has special meaning to the language, it's +not predeclared in any context, so it's considered to be out of scope. A future +proposal that describes the details of C++ import should determine whether this +name becomes a keyword. Notably, making `Cpp` a keyword would also allow an +`import Cpp` declaration to have custom syntax, which may be useful. + +### Package name `Main` + +The special package name `Main` that is currently reserved in all package name +contexts is not made a keyword in this proposal either. There would be no +meaning in making it a keyword, as it is never used as a special package name in +Carbon source files. However, we could consider using an empty package name as +the name of the main package, and unreserving the package name `Main`, if it +becomes a concern that we reserve this name. + +## Alternatives considered + +### Have both predeclared identifiers and keywords + +We could provide both predeclared identifiers and keywords. Many languages +follow this path. However, predeclared identifiers have some problems compared +to keywords: + +- In order to locally declare a name matching a predeclared identifier, the + name would need to be shadowed. + - Such shadowing may be invalid, depending on how the name is used. + - Readability is harmed by using a name used as basic vocabulary with a + different, local meaning. + - Shadowing a predeclared identifier typically makes the original name + hard to access -- an alias or similar must be established in advance. +- There need to be two different stories for how to deal with adding a new + word with special meaning to the language, depending on whether it is a + keyword. +- For each word with special meaning, we must make an arbitrary decision as to + which kind it is, resulting in a largely meaningless distinction that + nonetheless is visible and would need to be known by developers in some + contexts. + +### Reserve words with a certain spelling + +We could reserve words with certain spellings for future use as keywords or as +vendor extensions. Some languages do this: + +- C reserves words starting with an underscore followed by a capital letter or + an underscore. +- C++ additionally reserves words containing a double underscore anywhere. +- Python uses the `__name__` namespace for certain special names, and by + convention these names are reserved for that purpose. + +In Carbon we could accomplish this by saying that all words of the reserved +forms are keywords, with no meaning ascribed to them yet. + +However, we do not have a clear need for such reserved words at this time, and +we would not want to use such spellings when we do add language keywords later. +Moreover, C++ programs frequently declare reserved words in practice, and we +should expect the same in Carbon. Without enforcement, the names are not +effectively reserved. + +If we find a need at a later time to introduce vendor-specific language +extension keywords, we can revisit this, but should also consider alternatives +such as a `k#foo` spelling to turn what is normally an identifier into a +(potentially vendor-specific) keyword. diff --git a/utils/textmate/Syntaxes/carbon.tmLanguage.json b/utils/textmate/Syntaxes/carbon.tmLanguage.json index f3fe9304dc56..3e1260423abe 100644 --- a/utils/textmate/Syntaxes/carbon.tmLanguage.json +++ b/utils/textmate/Syntaxes/carbon.tmLanguage.json @@ -223,7 +223,7 @@ "patterns": [ { "name": "support.class.carbon", - "match": "(?<=\\bpackage\\s)\\w+" + "match": "(?<=\\b(package|Core)\\s)\\w+" }, { "name": "support.variable.carbon", diff --git a/utils/tree_sitter/queries/highlights.scm b/utils/tree_sitter/queries/highlights.scm index 18085e7db84c..c3b811900bb5 100644 --- a/utils/tree_sitter/queries/highlights.scm +++ b/utils/tree_sitter/queries/highlights.scm @@ -89,6 +89,7 @@ "auto" "base" "break" + ; "Core" "case" "choice" "class" @@ -126,6 +127,7 @@ "return" "returned" "Self" + ; "self" "template" "then" "type"