Add Skyframe support for external computations

Add a SkyFunction.Environment.dependOnFuture() method to register a
dependency on the completion of a ListenableFuture. Once all such
futures are completed, the current node is re-enqueued.

We combine all futures that are registered in the same pass, and treat
them as a single logical dependency. To that end, we add another counter
to NodeEntry that does just that, but otherwise use the same signalDep
method.

This is in preparation for supporting async action execution, i.e.,
allowing actions to run in parallel without blocking a Skyframe thread.
We have reason to believe that this will be a significant performance
win for large builds: if there are more actions that could be run than
available jobs, but cranking up jobs is prohibitively costly.

In addition, async action execution also improves Bazel's command-line
output; since we no longer block Skyframe threads on action execution,
these threads are free to continue exploring the action graph, which
means we'll see the number of outstanding actions more quickly go to the
expected maximum for that build (excluding flaky test retries and
exclusive tests, which are not modeled in the action graph).

Progress on #6394.

PiperOrigin-RevId: 234450510
18 files changed
tree: 84c481fb214fc413c1ffdca8e2bc80e020524b7b
  1. .bazelci/
  2. examples/
  3. scripts/
  4. site/
  5. src/
  6. third_party/
  7. tools/
  8. .gitattributes
  9. .gitignore
  10. AUTHORS
  11. BUILD
  12. CHANGELOG.md
  13. CODEOWNERS
  14. combine_distfiles.py
  15. combine_distfiles_to_tar.sh
  16. compile.sh
  17. CONTRIBUTING.md
  18. CONTRIBUTORS
  19. distdir.bzl
  20. ISSUE_TEMPLATE.md
  21. LICENSE
  22. README.md
  23. WORKSPACE
README.md

Bazel

{Fast, Correct} - Choose two

Build and test software of any size, quickly and reliably.

  • Speed up your builds and tests: Bazel only rebuilds what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.

  • One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.

  • Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.

  • Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.

Getting Started

Documentation

Contributing to Bazel

See CONTRIBUTING.md

Build status

Bazel is released in ‘Beta’. See the product roadmap to learn about the path toward a stable 1.0 release.