Google Git
Sign in
bazel / bazel / 1ebb04bb5b7773837f6d5653b65378c4bb8df7f6
commit1ebb04bb5b7773837f6d5653b65378c4bb8df7f6[log] [tgz]
authorChi Wang <chiwang@google.com>Fri Mar 03 07:13:57 2023 -0800
committerCopybara-Service <copybara-worker@google.com>Fri Mar 03 07:15:21 2023 -0800
tree1d1a520acaee86851fabea8ab1f24912f57434d1
parent424e73937c2dc71ca8a9915170bbd09c8d9ba5c5 [diff]
Store TTL in RemoteFileArtifactValue

When building without the bytes, Bazel stores `RemoteFileArtifactValue` in skyframe (inmemory) and in local action cache which represents a file that is stored remotely. Bazel assumes that the remote file will never expire which is wrong. In practice, remote cache often evict files due to space constraint, and when it happens, the builds could fail.

This PR introduces flag `--experimental_remote_cache_ttl` which tells Bazel at least how long the remote cache could store a file after returning a reference of it to Bazel. Bazel calculates the TTL of the file and store it in the `RemoteFileArtifactValue`. In an incremental build, Bazel will discard the `RemoteFileArtifactValue` and rerun the generating actions if it finds out that the `RemoteFileArtifactValue` is expired. The new field `expireAtEpochMilli` replaces `actionId` (deleted by f62a8b9f6a7e840c648f44f094390c3b85b236df), so there shouldn't be memory regression.

There are two places Bazel checks the TTL:
1. If the skyframe has in-memory state about previous builds (e.g. incremental builds), the `SkyValue`s are marked as dirty if the `RemoteFileArtifactValue` is expired.
2. When checking local action cache, if the `RemoteFileArtifactValue` is expired, the cache entry is ignored.

So that the generating actions can be re-executed.

Part of #16660.

Closes #17639.

RELNOTES: Add flag `--experimental_remote_cache_ttl` and set the default value to 3 hours.
PiperOrigin-RevId: 513819724
Change-Id: I9c9813621d04d5b1b94312be39384962feae2f7b
23 files changed
tree: 1d1a520acaee86851fabea8ab1f24912f57434d1
  1. .bazelci/
  2. .github/
  3. examples/
  4. scripts/
  5. site/
  6. src/
  7. third_party/
  8. tools/
  9. .bazelrc
  10. .gitattributes
  11. .gitignore
  12. AUTHORS
  13. BUILD
  14. CHANGELOG.md
  15. CODE_OF_CONDUCT.md
  16. CODEOWNERS
  17. combine_distfiles.py
  18. combine_distfiles_to_tar.sh
  19. compile.sh
  20. CONTRIBUTING.md
  21. CONTRIBUTORS
  22. distdir.bzl
  23. distdir_deps.bzl
  24. LICENSE
  25. maven_install.json
  26. MODULE.bazel
  27. README.md
  28. SECURITY.md
  29. WORKSPACE
  30. WORKSPACE.bzlmod
README.md

Bazel

{Fast, Correct} - Choose two

Build and test software of any size, quickly and reliably.

  • Speed up your builds and tests: Bazel rebuilds only what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.

  • One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.

  • Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.

  • Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.

Getting Started

Documentation

Reporting a Vulnerability

To report a security issue, please email security@bazel.build with a description of the issue, the steps you took to create the issue, affected versions, and, if known, mitigations for the issue. Our vulnerability management team will respond within 3 working days of your email. If the issue is confirmed as a vulnerability, we will open a Security Advisory. This project follows a 90 day disclosure timeline.

Contributing to Bazel

See CONTRIBUTING.md

Build status