Add Skyframe support for external computations
Add a SkyFunction.Environment.dependOnFuture() method to register a
dependency on the completion of a ListenableFuture. Once all such
futures are completed, the current node is re-enqueued.
We combine all futures that are registered in the same pass, and treat
them as a single logical dependency. To that end, we add another counter
to NodeEntry that does just that, but otherwise use the same signalDep
method.
This is in preparation for supporting async action execution, i.e.,
allowing actions to run in parallel without blocking a Skyframe thread.
We have reason to believe that this will be a significant performance
win for large builds: if there are more actions that could be run than
available jobs, but cranking up jobs is prohibitively costly.
In addition, async action execution also improves Bazel's command-line
output; since we no longer block Skyframe threads on action execution,
these threads are free to continue exploring the action graph, which
means we'll see the number of outstanding actions more quickly go to the
expected maximum for that build (excluding flaky test retries and
exclusive tests, which are not modeled in the action graph).
Progress on #6394.
PiperOrigin-RevId: 234450510
diff --git a/src/main/java/com/google/devtools/build/skyframe/InMemoryNodeEntry.java b/src/main/java/com/google/devtools/build/skyframe/InMemoryNodeEntry.java
index 058a3dc..233127d 100644
--- a/src/main/java/com/google/devtools/build/skyframe/InMemoryNodeEntry.java
+++ b/src/main/java/com/google/devtools/build/skyframe/InMemoryNodeEntry.java
@@ -244,6 +244,12 @@
dirtyBuildingState = null;
}
+ @Override
+ public synchronized void addExternalDep() {
+ Preconditions.checkNotNull(dirtyBuildingState, this);
+ dirtyBuildingState.addExternalDep();
+ }
+
protected final synchronized Set<SkyKey> setStateFinishedAndReturnReverseDepsToSignal() {
Set<SkyKey> reverseDepsToSignal =
ReverseDepsUtility.consolidateDataAndReturnNewElements(this, getOpToStoreBare());
