Improve diagnostics and error recovery for invalid identifiers. (#7249)

If a keyword or a sized type literal (eg, `f2`) is used in a context
where we are confident that we are expecting an identifier -- either
before a `:` in a binding pattern or after a `.` in a member access or
designator -- then recover as if a raw identifier was used.

This appears to be a particular stumbling block for coding agents, so
seems worth paying special attention to.

Add a mechanism to the tokenized buffer to track additional tokens
synthesized for error recovery so that we can keep the lexed token
sequence immutable and still satisfy the invariants throughout the rest
of the toolchain for recovery tokens. Thanks to chandlerc for suggesting
this approach!

toolchain/lex/tokenized_buffer.cpp

@@ -204,6 +204,32 @@ auto TokenizedBuffer::IsRecoveryToken(TokenIndex token) const -> bool {
   return recovery_tokens_[token.index];
 }
 
+auto TokenizedBuffer::AddPostLexingRecoveryTokenAsIdentifier(TokenIndex token)
+    -> TokenIndex {
+  auto kind = GetKind(token);
+  CARBON_CHECK(kind.is_word(),
+               "Invalid token kind {0} for recovery as identifier", kind);
+  CARBON_CHECK(kind != TokenKind::Identifier, "Recovery not required");
+
+  auto identifier_id = value_stores_->identifiers().Add(GetTokenText(token));
+  auto info = token_infos_.Get(token);
+  info.ResetAsErrorRecoveryIdentifier(identifier_id);
+  return AddPostLexingRecoveryToken(info);
+}
+
+auto TokenizedBuffer::AddPostLexingRecoveryToken(TokenInfo info) -> TokenIndex {
+  // Only resize once to avoid quadratic behavior if lots of recovery tokens are
+  // added.
+  if (recovery_tokens_.empty()) {
+    recovery_tokens_.resize(token_infos_.size());
+  }
+
+  auto token = token_infos_.Add(info);
+  recovery_tokens_.push_back(true);
+  ++post_lexing_recovery_tokens_;
+  return token;
+}
+
 auto TokenizedBuffer::GetIndentColumnNumber(LineIndex line) const -> int {
   return line_infos_.Get(line).indent + 1;
 }