commit 3740768a2f625e7b3310f69bcec47b5dd0898cf1
author ulfjack <ulfjack@google.com> Tue Sep 17 06:30:49 2019 -0700
committer Copybara-Service <copybara-worker@google.com> Tue Sep 17 06:32:57 2019 -0700
tree 565ff310192248afaaaec80c1f28ae3eee556520
parent 365a26bd89587d46e5cd4375dd4b1b7a8a6e878d

Avoid all configured targets depending on all build infos

Instead, throw a MissingDepException from CachingAnalysisEnvironment if
the build info is missing, and restart the analysis of that configured
target in the ConfiguredTargetFunction if so.

This removes several Skyframe edges per configured target, since most of
them never attempt to access the build info. For builds with a few
hundred thousand targets, that's several million edges.

The build info Skyframe nodes are the currently the nodes with the
largest number of reverse deps, which creates a bottleneck in
InMemoryNodeEntry.addReverseDepAndCheckIfDone during analysis, causing
significant lock contention. In the builds we have looked at, this was
the second most common source of lock contention.

However, we have not measured any significant reduction in end-to-end
build times.

There is already an injected key for the build info factories, but it is
never actually used, which looks like an incomplete refactoring. This
change finishes that refactoring with the nice benefit of removing the
build info factories from the SkyframeExecutor (doing so is a
prerequisite for allowing multiple concurrent executions in the same
SkyframeExecutor).

PiperOrigin-RevId: 269555110

src/main/java/com/google/devtools/build/lib/skyframe/AspectFunction.java

diff --git a/src/main/java/com/google/devtools/build/lib/skyframe/AspectFunction.java b/src/main/java/com/google/devtools/build/lib/skyframe/AspectFunction.java
index d1f2d63..f6f3554 100644
--- a/src/main/java/com/google/devtools/build/lib/skyframe/AspectFunction.java
+++ b/src/main/java/com/google/devtools/build/lib/skyframe/AspectFunction.java
@@ -23,6 +23,7 @@
 import com.google.devtools.build.lib.analysis.AliasProvider;
 import com.google.devtools.build.lib.analysis.AspectResolver;
 import com.google.devtools.build.lib.analysis.CachingAnalysisEnvironment;
+import com.google.devtools.build.lib.analysis.CachingAnalysisEnvironment.MissingDepException;
 import com.google.devtools.build.lib.analysis.ConfiguredAspect;
 import com.google.devtools.build.lib.analysis.ConfiguredAspectFactory;
 import com.google.devtools.build.lib.analysis.ConfiguredTarget;
@@ -650,6 +651,9 @@
                     aspectConfiguration,
                     view.getHostConfiguration(aspectConfiguration),
                     key);
+      } catch (MissingDepException e) {
+        Preconditions.checkState(env.valuesMissing());
+        return null;
       } finally {
         CurrentRuleTracker.endConfiguredAspect();
       }