Query performance: Support length-delimited protos

Add support for --output=streamed_proto, which outputs the targets as
a stream of length-delimited plain protos rather than a single
monolithic proto.

For queries with a lot of results, this is significantly faster (up to
~30% in my tests, though note that my baseline test contained a series
of as-yet unsubmitted improvements).

The primary driver of the performance improvement on the Bazel side
appears to be the decreased memory pressure resulting in lower gc
times.

In any case, the requirement to retain the full result on the Bazel
side as well as the receiver side is a showstopper for large queries.

We might want to consider wrapping the resulting targets in another
proto that allows us to add some per-package information in the future.
Another performance issue is the duplicate location field on every
single target, which constitutes up to ~30% of the output size. Even
with relative paths, we see around ~14% of the output being location
data.

PiperOrigin-RevId: 251636100
2 files changed
tree: 903d21622b3d7666800f85b629c4dd0a3ef606c2
  1. .bazelci/
  2. examples/
  3. scripts/
  4. site/
  5. src/
  6. third_party/
  7. tools/
  8. .bazelrc
  9. .gitattributes
  10. .gitignore
  11. AUTHORS
  12. BUILD
  13. CHANGELOG.md
  14. CODEOWNERS
  15. combine_distfiles.py
  16. combine_distfiles_to_tar.sh
  17. compile.sh
  18. CONTRIBUTING.md
  19. CONTRIBUTORS
  20. distdir.bzl
  21. ISSUE_TEMPLATE.md
  22. LICENSE
  23. README.md
  24. WORKSPACE
README.md

Bazel

{Fast, Correct} - Choose two

Build and test software of any size, quickly and reliably.

  • Speed up your builds and tests: Bazel only rebuilds what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.

  • One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.

  • Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.

  • Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.

Getting Started

Documentation

Contributing to Bazel

See CONTRIBUTING.md

Build status

Bazel is released in ‘Beta’. See the product roadmap to learn about the path toward a stable 1.0 release.