Windows,Bazel client: multithreaded file writing

The Bazel client on Windows now writes extracted
binaries to disk in parallel. On all other systems
it writes them serially (as before).

This change makes blaze.cc:ActuallyExtractData()
about 3x faster when using a HDD. (In previous
experiments I saw no speedup with multi-threaded
writing on machines with an SSD.)

The Windows-specific code uses the native
Threadpool API of Windows, creating a pool of at
least 8 and at most 16 threads. (This seems to be
a good balance between speed and thread count.)

The OS manages everything about the pool; Bazel
submits callbacks and the pool executes them
asynchronously.

blaze.cc:ActuallyExtractData() speed, before:
- Windows: 6.48s (avg) / 6.38s (median)
- Linux (Debian): 4.78s (avg) / 4.79s (median)

blaze.cc:ActuallyExtractData() speed, after:
- Windows (8-16 threads): 2.05s (avg) / 2.01s (md)
- Windows (1 thread): 5.77s (avg) / 5.74s (median)

See https://github.com/bazelbuild/bazel/issues/5444

Change-Id: I7211f3d28eb8b9837352c16ff8df0411d5a9ebe1

Closes #5600.

Change-Id: I7a74d62a563c92948a4dfa8ad5ac83eae018db10
PiperOrigin-RevId: 204891217
4 files changed
tree: 29a52a443382e9222538205751ff58fa0673ad07
  1. .bazelci/
  2. examples/
  3. scripts/
  4. site/
  5. src/
  6. third_party/
  7. tools/
  8. .gitattributes
  9. .gitignore
  10. AUTHORS
  11. BUILD
  12. CHANGELOG.md
  13. combine_distfiles.py
  14. combine_distfiles_to_tar.sh
  15. compile.sh
  16. CONTRIBUTING.md
  17. CONTRIBUTORS
  18. distdir.bzl
  19. ISSUE_TEMPLATE.md
  20. LICENSE
  21. README.md
  22. WORKSPACE
README.md

Bazel

{Fast, Correct} - Choose two

Build and test software of any size, quickly and reliably.

  • Speed up your builds and tests: Bazel only rebuilds what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.

  • One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.

  • Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.

  • Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.

Getting Started

Documentation

Contributing to Bazel

See CONTRIBUTING.md

Build status

Bazel is released in ‘Beta’. See the product roadmap to learn about the path toward a stable 1.0 release.