Prevent broken cache entries on concurrent file changes

Local execution has an inherent race condition: if a user modifies a file while an action is executed, then it is impossible for Bazel to tell which version of the file was actually read during action execution. The file may have been modified before or after the tool has read it, or, in the worst case, the tool may have read both the original and the modified version. In addition, the file may be changed back to the original state before Bazel can check the file, so computing the digest before / after may not be sufficient.

This is a concern for both local and remote caches, although the cost of poisoning a shared remote cache is significantly higher, and is what has triggered this work.

Fixes #3360.

We solve this by keeping a reference to the FileContentsProxy, and using that to check for modificaitons before storing the cache entry. We output a warning if this check fails.

This change does not increase memory consumption; Java objects are always allocated in multiples of 8 bytes, we use compressed oops, and the FileArtifactValue currently has 12 bytes worth of fields (excl. object overhead), so adding another pointer is effectively free.

As a possible performance optimization on purely local builds, we could also consider not computing digests at all, and only use the FileContentsProxy for caching.

PiperOrigin-RevId: 182510358
8 files changed
tree: 05b8d52d6a254800377f1e9d479acae5584a348a
  1. examples/
  2. scripts/
  3. site/
  4. src/
  5. third_party/
  6. tools/
  7. .gitattributes
  8. .gitignore
  9. AUTHORS
  10. BUILD
  11. CHANGELOG.md
  12. combine_distfiles.py
  13. combine_distfiles_to_tar.sh
  14. compile.sh
  15. CONTRIBUTING.md
  16. CONTRIBUTORS
  17. ISSUE_TEMPLATE.md
  18. LICENSE
  19. README.md
  20. WORKSPACE
README.md

Bazel

{Fast, Correct} - Choose two

Build and test software of any size, quickly and reliably.

  • Speed up your builds and tests: Bazel only rebuilds what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.

  • One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.

  • Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.

  • Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.

Getting Started

Documentation

Contributing to Bazel

See CONTRIBUTING.md

Build Status

Bazel is released in ‘Beta’. See the product roadmap to learn about the path toward a stable 1.0 release.