Remove on-demand runfile tree creation from test strategy

Bazel builds runfiles trees when building an executable target. This
allows a binary to be run directly (without bazel test/run). The flag
--nobuild_runfile_trees disables this behavior. In order to still be
able to run tests that require a runfiles tree the
Local/StandaloneTestStrategy has code to create / update it before
executing the test.

However, most execution strategies in Bazel don't use the runfiles tree:
  * Actions being executed using sandboxing don't have access to the
    runfiles tree as their working directory is an ephemeral directory
    created by the sandbox. Instead, the sandbox treats runfiles as normal
    inputs and creates a new tree for every action.
  * Actions being executed with the remote strategy don't have access to
    the runfiles tree either as they run on a different machine. Just like
    the sandbox the remote strategy treats runfiles as normal inputs and
    uploads them to the remote worker.

The runfiles tree is only used by actions executed with the standalone/local
or worker (with --worker_sandboxing=false) strategy as they use the exec
root as their working directory.

This change makes it so that if --nobuild_runfile_links is specified a
runfiles tree is only created if the execution strategy actually uses
it.

We found that this can lead to significant performance improvements in
particular for very large runfile trees. Also, before this change in
remote execution we would create a runfiles tree even for remote cache
hits.

As a side effect this change also removes an imparity between binaries
and tests, in that binaries had no such logic to on-demand create a
runfiles tree if --nobuild_runfile_links is set.

I will send out a follow up change with a similar optimization the
input / output manifest.

Note to Reviewers:
 * RunfilesTreeUpdater replaces TestStrategy#getLocalRunfilesDirectory.
 * The locking logic in RunfilesTreeUpdater implements one monitor per
   runfiles directory. The monitor ensures that a runfiles directory
   can't be modified concurrently. It replaces synchronized(inputManifest)
   from getLocalRunfilesDirectory. We can't use synchronized(inputManifest)
   for two reasons:
     1) The execution strategies don't have access to it.
     2) As noted above, in a follow up change I want to enable this logic
     to also work when --nobuild_runfile_manifests is set in which case no
     inputManifest object to use as monitor will exist.

Closes #9244.

PiperOrigin-RevId: 266376390
14 files changed
tree: 22738b4a57457908e4b3c9e12ed7f19fd13a73de
  1. .bazelci/
  2. examples/
  3. scripts/
  4. site/
  5. src/
  6. third_party/
  7. tools/
  8. .bazelrc
  9. .gitattributes
  10. .gitignore
  11. AUTHORS
  12. BUILD
  13. CHANGELOG.md
  14. CODEOWNERS
  15. combine_distfiles.py
  16. combine_distfiles_to_tar.sh
  17. compile.sh
  18. CONTRIBUTING.md
  19. CONTRIBUTORS
  20. distdir.bzl
  21. ISSUE_TEMPLATE.md
  22. LICENSE
  23. README.md
  24. WORKSPACE
README.md

Bazel

{Fast, Correct} - Choose two

Build and test software of any size, quickly and reliably.

  • Speed up your builds and tests: Bazel only rebuilds what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.

  • One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.

  • Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.

  • Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.

Getting Started

Documentation

Contributing to Bazel

See CONTRIBUTING.md

Build status

Bazel is released in ‘Beta’. See the product roadmap to learn about the path toward a stable 1.0 release.