Remove on-demand runfile tree creation from test strategy
Bazel builds runfiles trees when building an executable target. This
allows a binary to be run directly (without bazel test/run). The flag
--nobuild_runfile_trees disables this behavior. In order to still be
able to run tests that require a runfiles tree the
Local/StandaloneTestStrategy has code to create / update it before
executing the test.
However, most execution strategies in Bazel don't use the runfiles tree:
* Actions being executed using sandboxing don't have access to the
runfiles tree as their working directory is an ephemeral directory
created by the sandbox. Instead, the sandbox treats runfiles as normal
inputs and creates a new tree for every action.
* Actions being executed with the remote strategy don't have access to
the runfiles tree either as they run on a different machine. Just like
the sandbox the remote strategy treats runfiles as normal inputs and
uploads them to the remote worker.
The runfiles tree is only used by actions executed with the standalone/local
or worker (with --worker_sandboxing=false) strategy as they use the exec
root as their working directory.
This change makes it so that if --nobuild_runfile_links is specified a
runfiles tree is only created if the execution strategy actually uses
it.
We found that this can lead to significant performance improvements in
particular for very large runfile trees. Also, before this change in
remote execution we would create a runfiles tree even for remote cache
hits.
As a side effect this change also removes an imparity between binaries
and tests, in that binaries had no such logic to on-demand create a
runfiles tree if --nobuild_runfile_links is set.
I will send out a follow up change with a similar optimization the
input / output manifest.
Note to Reviewers:
* RunfilesTreeUpdater replaces TestStrategy#getLocalRunfilesDirectory.
* The locking logic in RunfilesTreeUpdater implements one monitor per
runfiles directory. The monitor ensures that a runfiles directory
can't be modified concurrently. It replaces synchronized(inputManifest)
from getLocalRunfilesDirectory. We can't use synchronized(inputManifest)
for two reasons:
1) The execution strategies don't have access to it.
2) As noted above, in a follow up change I want to enable this logic
to also work when --nobuild_runfile_manifests is set in which case no
inputManifest object to use as monitor will exist.
Closes #9244.
PiperOrigin-RevId: 266376390
{Fast, Correct} - Choose two
Build and test software of any size, quickly and reliably.
Speed up your builds and tests: Bazel only rebuilds what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.
One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.
Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.
Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.
Follow our tutorials:
See CONTRIBUTING.md
Bazel is released in ‘Beta’. See the product roadmap to learn about the path toward a stable 1.0 release.