Do not flatten tree artifacts in `ActionInputMap`.

When constructing an `ActionInputMap`, we will add all of the action inputs to it, hashed on the exec path. For tree artifacts, we would add both the tree and all of the children from its expansion. That poses a memory usage problem in presence of large tree artifacts -- for one such tree artifact, we would create `O(actions depending on it)` references to each of the files in the tree artifacts.

Reduce the memory usage of `ActionInputMap` by storing a composite entry for the tree children instead of an entry for each individual child. This entry is a reference to the map of children which already is stored in `TreeArtifactValue`. Adjust the lookup logic to work with composite entries in the map. Add a trie representation of the tree artifact locations to allow for faster response when searching for paths which do not exist in the map.

Change `ActionInputMap::getMetadata(String)` to `ActionInputMap::getMetadata(PathString)` so that we can pass the value from the callers (all of them already have it) and prevent creating garbage `PathFragment` when walking the trie.

PiperOrigin-RevId: 384297684
15 files changed
tree: c2bb2c52ac328e6364ccec7d7c0451bee9c45c8f
  1. .bazelci/
  2. examples/
  3. scripts/
  4. site/
  5. src/
  6. third_party/
  7. tools/
  8. .bazelrc
  9. .gitattributes
  10. .gitignore
  11. AUTHORS
  12. BUILD
  13. CHANGELOG.md
  14. CODEBASE.md
  15. CODEOWNERS
  16. combine_distfiles.py
  17. combine_distfiles_to_tar.sh
  18. compile.sh
  19. CONTRIBUTING.md
  20. CONTRIBUTORS
  21. distdir.bzl
  22. distdir_deps.bzl
  23. ISSUE_TEMPLATE.md
  24. LICENSE
  25. README.md
  26. WORKSPACE
README.md

Bazel

{Fast, Correct} - Choose two

Build and test software of any size, quickly and reliably.

  • Speed up your builds and tests: Bazel rebuilds only what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.

  • One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.

  • Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.

  • Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.

Getting Started

Documentation

Contributing to Bazel

See CONTRIBUTING.md

Build status