[8.8.0] Accept and ignore Bazel 10 Starlark type syntax (#30373)

### Description

Backports the Bazel 10 Starlark type syntax parser to the 8.x line in
"accept but ignore" mode: `.bzl` files using type annotations, type
aliases, generic `def` parameters, `...`, `cast()`, and `isinstance()`
now load successfully, with all type information ignored. This matches
master's default behavior (`--experimental_starlark_type_syntax`
defaults to true, type checking off ⇒ annotations are parsed tolerantly
as arbitrary expressions and never resolved).

Without this PR, 8.8.0 fails to load even the most basic annotated file
with a syntax error:

```starlark
def f(x: list[str]):  # syntax error at ':': type annotations are disallowed
    pass
```

The following commits are cherry-picked, in order:

* 299e903710 Extend syntax with parameter type annotations
* 2dbda274cd Use type specific productions
* d1c3727795 Extend syntax with return types
* ab034aea73 Implement --experimental_starlark_types* flags
* fb7ca12439 Implement TypeApplication and fix NodePrinter
* aa61fc8a8d Allow empty type arguments for type applications (parser
support only)
* 0b615ff55e Introduce Expression.parseTypeExpression() (parser support
only)
* f60a22e8ef Allow parsing arbitrary uninterpreted Starlark type
expressions
* f3653ca8ae Allow parsing type alias statements
* 88678c00c4 Allow parsing generic type parameters in `def` statements
* 96d2f60fb9 Allow parsing `cast` expressions (and evaluating them as
the value argument)
* 664e87b594 Evaluate type alias statements as a no-op
* 5c8655ca1f Add variable type annotation syntax and resolver behavior
* 2390ce6760 Add Ellipsis token and node
* 515083c24a Add `isinstance` keyword, and allow isinstance(x,t) to be
parsed - but not resolved
* 44665ecd71 Properly gate dynamic type checking with flag (syntax
tolerance only)
* f38f0e448a Disallow type syntax in .scl files
* f1b3273f9f, 2004bad12f, f0011d8f29 (the three parser-only slices from
#30092: `tuple[T, ...]`, `tuple[()]`, struct type)

### How to review

Every commit keeps its original `Change-Id`, so the deviation of each
cherry-pick from its master original can be computed mechanically with
`git range-diff` (checking out this PR as `<pr-head>`):

```sh
git log --reverse --format=%H origin/release-8.8.0..<pr-head> | while read pick; do
  cid=$(git log -1 --format=%B $pick | sed -n 's/^Change-Id: //p' | head -1)
  orig=$(git log --format=%H --grep="Change-Id: $cid" origin/master | tail -1)
  git range-diff --creation-factor=100 $orig^! $pick^!
done
```

Only the lines range-diff marks as changed between the two patches need
human review; everything else is upstream code verbatim. Running this
today gives the following review-effort map (changed diff lines per
commit): all commits are at or below 22 except 0b615ff55e (99),
f60a22e8ef (51), 664e87b594 (44), 5c8655ca1f (267), 44665ecd71 (133),
and f38f0e448a (91) — which are exactly the sliced/adapted commits
explained below. For the last three commits, substitute the
already-sliced commits from #30092 (1681dfb115, b172b104c3, 745ec85067)
as the range-diff baseline instead of the master originals; against that
baseline they deviate by 2, 12, and 2 lines respectively.

Each commit is best reviewed against its master original (referenced by
`PiperOrigin-RevId`/`Change-Id` in the message). Everything not listed
below is verbatim upstream code; in particular, the type grammar in
`Parser.java` (all `parseType*` productions, type aliases, generics,
`cast`/`isinstance`) and the new AST node files `TypeAliasStatement`,
`TypeApplication`, `Ellipsis`, and `IsInstanceExpression` are
byte-identical to current master.

Non-trivial deviations from master, grouped by cause:

**1. The type checking machinery is intentionally not backported**
(~4,800 lines on master: `StarlarkType`, `Types`, `TypeChecker`,
`TypeTagger`, `TypeResolver`, and the dynamic checks in
`Eval`/`StarlarkFunction`):

* 0b615ff55e is taken without the static `Resolver.resolveType()` (it
needs the types package).
* 44665ecd71 is reduced to its
`FileOptions`/`Parser`/flag/`BzlCompileFunction` hunks; the
`Eval`/`StarlarkFunction` dynamic-check gating hunks are dropped. The
`StarlarkSemantics.EXPERIMENTAL_STARLARK_TYPE_CHECKING` key constant is
added by hand since `BzlCompileFunction` still needs it.
* `CastExpression` lacks master's `starlarkType` field/getter/setter
(the type-tagging slot, only written by `TypeTagger`).
* Hunks touching checker test files
(`TypeCheckTest`/`DynamicTypeCheckTest`, the syntax-package
`StarlarkTypesTest`) are dropped wherever a commit touched them.
* Net effect: `--experimental_starlark_type_checking` (default off) on
8.x only switches annotations from tolerant to structured parsing; it
performs no type checking.

**2. The doc-comments feature (03921eded1) is not backported**, so its
threading is stripped from 5c8655ca1f's hunks:

* `AssignmentStatement` and `VarStatement` have no `docComments`
field/getter; their constructors take one fewer parameter.
* `Parser#parseAssignment` does not call `maybeParseTrailingDocComment`
(method not introduced), and `Expression#parseExpression` does not skip
`DOC_COMMENT_*` tokens.
* `Resolver#bind` is `bind(id, isLoad, hasType)` instead of master's
four-parameter form; `createBindingsForLHS` takes no doc-comments
parameter.

**3. `Binding.isSyntactic` does not exist on 8.x.** In the
one-annotation-per-declaration check inside `Resolver#bind`, master's
`bind.isSyntactic` is replaced by `bind.first != null` (on 8.x,
`Binding#first` is documented as "first binding use, if syntactic", so
the two are equivalent).

**4. Older test scaffolding on 8.x:**

* `EvaluationTestCase` gains `setFileOptions`/`getFileOptions` (ported
from master, folded into the 664e87b594 pick) because the cherry-picked
`EvaluationTest` cases need them.
* In `ParserTest`/`NodePrinterTest`/`ResolverTest`, only each commit's
own payload tests are taken where the surrounding context consists of
master-only tests that don't exist on 8.x; two `NodePrinterTest`
assertions stay in 8.x's `join(...)` style.

**5. The `parseExpression` → `parseExpr` rename** from f60a22e8ef is
applied in full so the parser stays textually close to master.

### Verification

`//src/test/java/net/starlark/java/...` and
`//src/test/java/com/google/devtools/build/lib/starlark:StarlarkTypesTest`
pass. Additionally verified end-to-end: a workspace exercising unknown
type names, providers as types, `list[str]`, `tuple[int, ...]`,
`tuple[()]`, `struct[{"a": int}]`, parameterized type aliases, variable
annotations, string-literal types, and runtime `cast()` passthrough
loads identically (default flags) with a Bazel built from this branch
and one built from master, and annotated module-level assignments
execute correctly.

### Motivation

Forward compatibility for the 8.x LTS line: once rulesets start adopting
Bazel 10 type annotations, users on Bazel 8 must still be able to load
those rulesets. This is the 8.x counterpart of the tolerance work that
already landed for 9.0.0 (#27760, #27838, #28069) and is pending for
9.3.0 (#30092).

### Build API Changes

Yes: this extends the accepted `.bzl` grammar and adds the experimental
flags `--experimental_starlark_type_syntax` (default true),
`--experimental_starlark_types_allowed_paths`, and
`--experimental_starlark_type_checking` (default false) to 8.x. Part of
the Starlark types effort (#27370). The change is purely additive and
backward compatible: previously invalid syntax now parses and is
ignored; `.scl` files continue to reject type syntax.

### Checklist

- [x] I have added tests for the new use cases (if any).
- [ ] I have updated the documentation (if applicable).

### Release Notes

RELNOTES: Bazel now parses and ignores the Starlark type annotation
syntax introduced by newer Bazel versions (gated by
`--experimental_starlark_type_syntax`, enabled by default).

---------

Co-authored-by: Googler <ilist@google.com>
Co-authored-by: arostovtsev <arostovtsev@google.com>
Co-authored-by: brandjon <brandjon@google.com>
Co-authored-by: Ian (Hee) Cha <heec@google.com>
33 files changed
tree: 10fa3be3bc46b42420e7bef1a8e1e5495fa870ac
  1. .bazelci/
  2. .github/
  3. examples/
  4. scripts/
  5. site/
  6. src/
  7. third_party/
  8. tools/
  9. .bazelrc
  10. .bazelversion
  11. .gitattributes
  12. .gitignore
  13. AUTHORS
  14. bazel_downloader.cfg
  15. BUILD
  16. CHANGELOG.md
  17. CODE_OF_CONDUCT.md
  18. CODEOWNERS
  19. combine_distfiles.py
  20. combine_distfiles_to_tar.sh
  21. compile.sh
  22. CONTRIBUTING.md
  23. CONTRIBUTORS
  24. distdir.bzl
  25. extensions.bzl
  26. LICENSE
  27. maven_install.json
  28. MODULE.bazel
  29. MODULE.bazel.lock
  30. README.md
  31. repositories.bzl
  32. requirements.txt
  33. SECURITY.md
  34. workspace_deps.bzl
README.md

Bazel

{Fast, Correct} - Choose two

Build and test software of any size, quickly and reliably.

  • Speed up your builds and tests: Bazel rebuilds only what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.

  • One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.

  • Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.

  • Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.

Getting Started

Documentation

Reporting a Vulnerability

To report a security issue, please email security@bazel.build with a description of the issue, the steps you took to create the issue, affected versions, and, if known, mitigations for the issue. Our vulnerability management team will respond within 3 working days of your email. If the issue is confirmed as a vulnerability, we will open a Security Advisory. This project follows a 90 day disclosure timeline.

Contributing to Bazel

See CONTRIBUTING.md

Build status