Treat runfiles middlemen as aggregating middlemen.

The goal is to, for an action execution, make only a single skyframe
edge to every required runfiles tree rather than an edge to every
runfiles artifact directly. We accomplish this by pulling the runfiles
artifacts' metadata for a particular runfiles tree through the
appropriate runfiles middlemen artifact in one batch. This CL fixes
https://github.com/bazelbuild/bazel/issues/3217.

This change makes runfiles middlemen more similar to aggregating
middlemen. We however continue to treat runfiles middlemen slightly
differently than true aggregating middlemen. Namely, we do not expand
runfiles middlemen into the final action inputs. The reasons for this
are:

1. It can make latent bugs like
https://github.com/bazelbuild/bazel/issues/4033 more severe.

2. The runfiles tree builder action's output MANIFEST contains
absolute paths, which interferes with remote execution if its metadata
is added to a cache hash.

3. In the sandbox, it causes the runfiles artifacts to be mounted at
their exec path locations in addition to within the runfiles
trees. This makes the sandbox less strict. (Perhaps tolerably?)

4. It saves a modicum of (transient) memory and time to not expand.

If the first two issues are fixed, it could be a nice simplification
to completely replace runfiles middlemen with aggregating middlemen.

Change-Id: I2d2148297f897af4c4c6dc055d87f8a6fad0061e
PiperOrigin-RevId: 198307109
3 files changed
tree: 356e0ddc647426520eb3eb13f1bb795e4602118b
  1. .bazelci/
  2. examples/
  3. scripts/
  4. site/
  5. src/
  6. third_party/
  7. tools/
  8. .gitattributes
  9. .gitignore
  10. AUTHORS
  11. BUILD
  12. CHANGELOG.md
  13. combine_distfiles.py
  14. combine_distfiles_to_tar.sh
  15. compile.sh
  16. CONTRIBUTING.md
  17. CONTRIBUTORS
  18. distdir.bzl
  19. ISSUE_TEMPLATE.md
  20. LICENSE
  21. README.md
  22. WORKSPACE
README.md

Bazel

{Fast, Correct} - Choose two

Build and test software of any size, quickly and reliably.

  • Speed up your builds and tests: Bazel only rebuilds what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.

  • One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.

  • Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.

  • Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.

Getting Started

Documentation

Contributing to Bazel

See CONTRIBUTING.md

Build status

Bazel is released in ‘Beta’. See the product roadmap to learn about the path toward a stable 1.0 release.