bazel skyframe: use CompiledBuildFile for results of static BUILD processing

This change introduces a new type, PackageFunction.CompiledBuildFile,
to hold all the results of static processing of the syntax tree of a BUILD
file, so that the syntax tree can be discarded. The results include:
- the compiled Program,
- any errors from parsing, name resolution, or BUILD dialect checks,
- the set of literal strings passed to glob(), and
- the mapping of top-level function calls to generator_name strings.

The goal of this change is to cleanly separate the static processing
of BUILD files from the dynamic processing (execution), to unblock work
on various Starlark optimizations such as "flat frames" and a compiler.
These optimizations require that the syntax tree can be decorated with
information during name resolution, which is impossible in the current
code because the cache holds unresolved syntax trees, and trees are
resolved as a side effect of execution.

However, this change also opens up opportunities for Skyframe
optimizations, such as separating BUILD compilation and execution
so that we needn't re-read/parse/resolve/compile a BUILD file just
because one of its .bzl loads has changed; and prefetching loads
as soon as the BUILD file has been parsed.

Errors from checkBuildSyntax are now reported before Starlark name
resolution, and resolution is now performed even if a BUILD file
is not executed due to others (hence test changes), but there
should otherwise be no observable behavior change.

Once this lands, we can remove the resolveScope(false) hack from Starlark
and (at long last) get to work on major optimizations.

Sorry for the messy diff. A number of things have crossed the boundary
between lib.packages and the loading-phase parts of lib.skyframe,
which should one day become part of lib.packages.

PiperOrigin-RevId: 343386400
15 files changed
tree: 02dfee819fe5adcc5000ff17e466235d4aedb430
  1. .bazelci/
  2. examples/
  3. scripts/
  4. site/
  5. src/
  6. third_party/
  7. tools/
  8. .bazelrc
  9. .gitattributes
  10. .gitignore
  11. AUTHORS
  12. BUILD
  13. CHANGELOG.md
  14. CODEBASE.md
  15. CODEOWNERS
  16. combine_distfiles.py
  17. combine_distfiles_to_tar.sh
  18. compile.sh
  19. CONTRIBUTING.md
  20. CONTRIBUTORS
  21. distdir.bzl
  22. ISSUE_TEMPLATE.md
  23. LICENSE
  24. README.md
  25. WORKSPACE
README.md

Bazel

{Fast, Correct} - Choose two

Build and test software of any size, quickly and reliably.

  • Speed up your builds and tests: Bazel rebuilds only what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.

  • One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.

  • Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.

  • Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.

Getting Started

Documentation

Contributing to Bazel

See CONTRIBUTING.md

Build status