Add support for fetching repos using a worker thread

Added flag `--experimental_worker_for_repo_fetching`, whose value can be `off` (no change), `platform` (uses a platform/OS thread to do repo fetching), `virtual` (uses a virtual thread feat. Project Loom).

When a worker thread is used, the Skyframe thread does not perform any fetching itself. Instead, it gives its Environment object to the worker thread, and let it do the bulk of the work. When a new Skyframe dependency is registered and is found missing, the worker thread doesn't restart; instead, it signals to the host thread that it should restart, and waits for the host thread to give back a fresh Environment object.

This weird dance can work with both platform threads and virtual threads. Right now, passing `--experimental_worker_for_repo_fetching=virtual` results in a fatal error, but that should be easily fixable when JDK20+ lands.

Work towards https://github.com/bazelbuild/bazel/issues/10515

PiperOrigin-RevId: 538909616
Change-Id: Iacc8b84ca2f90821fba5add171a4778ed2b48ab9
11 files changed
tree: ced01ad77262d5ad99262d6c9023eaf628e7b5c3
  1. .bazelci/
  2. .github/
  3. examples/
  4. scripts/
  5. site/
  6. src/
  7. third_party/
  8. tools/
  9. .bazelrc
  10. .bazelversion
  11. .gitattributes
  12. .gitignore
  13. AUTHORS
  14. BUILD
  15. CHANGELOG.md
  16. CODE_OF_CONDUCT.md
  17. CODEOWNERS
  18. combine_distfiles.py
  19. combine_distfiles_to_tar.sh
  20. compile.sh
  21. CONTRIBUTING.md
  22. CONTRIBUTORS
  23. distdir.bzl
  24. distdir_deps.bzl
  25. extensions.bzl
  26. LICENSE
  27. maven_install.json
  28. MODULE.bazel
  29. README.md
  30. repositories.bzl
  31. SECURITY.md
  32. WORKSPACE
  33. WORKSPACE.bzlmod
README.md

Bazel

{Fast, Correct} - Choose two

Build and test software of any size, quickly and reliably.

  • Speed up your builds and tests: Bazel rebuilds only what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.

  • One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.

  • Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.

  • Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.

Getting Started

Documentation

Reporting a Vulnerability

To report a security issue, please email security@bazel.build with a description of the issue, the steps you took to create the issue, affected versions, and, if known, mitigations for the issue. Our vulnerability management team will respond within 3 working days of your email. If the issue is confirmed as a vulnerability, we will open a Security Advisory. This project follows a 90 day disclosure timeline.

Contributing to Bazel

See CONTRIBUTING.md

Build status