tree 84c481fb214fc413c1ffdca8e2bc80e020524b7b
parent 45e68f2f4344afa32f91b835888123cda3a77dc6
author ulfjack <ulfjack@google.com> 1550487115 -0800
committer Copybara-Service <copybara-worker@google.com> 1550487267 -0800

Add Skyframe support for external computations

Add a SkyFunction.Environment.dependOnFuture() method to register a
dependency on the completion of a ListenableFuture. Once all such
futures are completed, the current node is re-enqueued.

We combine all futures that are registered in the same pass, and treat
them as a single logical dependency. To that end, we add another counter
to NodeEntry that does just that, but otherwise use the same signalDep
method.

This is in preparation for supporting async action execution, i.e.,
allowing actions to run in parallel without blocking a Skyframe thread.
We have reason to believe that this will be a significant performance
win for large builds: if there are more actions that could be run than
available jobs, but cranking up jobs is prohibitively costly.

In addition, async action execution also improves Bazel's command-line
output; since we no longer block Skyframe threads on action execution,
these threads are free to continue exploring the action graph, which
means we'll see the number of outstanding actions more quickly go to the
expected maximum for that build (excluding flaky test retries and
exclusive tests, which are not modeled in the action graph).

Progress on #6394.

PiperOrigin-RevId: 234450510