commit | 9beabe09bab5c371d353cca3c77c4e57de555ac0 | [log] [tgz] |
---|---|---|
author | ulfjack <ulfjack@google.com> | Wed Feb 13 23:41:59 2019 -0800 |
committer | Copybara-Service <copybara-piper@google.com> | Wed Feb 13 23:43:35 2019 -0800 |
tree | 7d1c57d39ae26dc14a271679a03e5eeb77165d89 | |
parent | 23e244bddfd8b9939f8fe9dbc5e760ce8ec0618e [diff] |
Move signaledDeps to DirtyBuildingState This requires using DirtyBuildingState also for the initial evaluation of a node. As a side effect, this reduces the set of possible states, which might make it easier to reason about the state. States: previously, NodeEntry distinguished between initial evaluation and incremental update, with 2 different states (just created / marked and evaluating) for each case. With this change, the case distinction goes away. This subtly changes the semantics of isDirty(): previously, isDirty returned false for new nodes; now it returns true for new nodes after their computation is triggered. This change should reduce memory consumption of InMemoryNodeEntry objects per instance for nodes that are done, but at the expense of additional temporary DirtyBuildingState instances for nodes that are not done. I am making this change in preparation for adding another field to the dirty state, which would otherwise increase memory consumption of all done nodes. = Memory consumption While this decreases the memory consumption of Skyframe nodes in the done state, it also increases the memory consumption of Skyframe nodes that have to be recomputed, especially for new nodes, but also for incrementally updated nodes. New nodes now have to allocate a DirtyBuildingState instance while they are in-flight, and incrementally updated nodes now have an additional field in the DirtyBuildingState. For incrementally updated nodes, the difference is a wash - we save some in InMemoryNodeEntry and lose the same amount in DirtyBuildingState. For clean builds, whether or not this is a reduction in memory use depends on the ratio of in-flight to done nodes. The higher the ratio, the higher the memory consumption. Note that the ratio goes to zero at the end of a build, so this is a strict win for memory consumption at rest (i.e., between builds), and only a concern while Bazel is active. For some representative target, I saw ratios of less than 25% at max during the loading+analysis phase, and dropping off sharply as packages were completed. During the execution phase, the ratio only barely got above 10% and usually stayed below 5%. Assuming a memory difference of 8 bytes per done node (since the Jvm performs 8-byte alignment), and an increase of 24 bytes per in-flight node, any ratio below 1/3 is a net win for post-gc memory consumption, although at the cost of slightly higher gc pressure. This is clearly the case from the numbers I collected. For incremental builds, the number of invalidated and recomputed nodes is usually much, much lower (I saw numbers as low as 0.02%), which virtually guarantees that this change is a net win. PiperOrigin-RevId: 233899818
{Fast, Correct} - Choose two
Build and test software of any size, quickly and reliably.
Speed up your builds and tests: Bazel only rebuilds what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.
One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.
Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.
Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.
Follow our tutorials:
See CONTRIBUTING.md
Bazel is released in ‘Beta’. See the product roadmap to learn about the path toward a stable 1.0 release.