First iteration of NestedSet-on-Skyframe
Main Idea:
When an Action is executed with ActionExecutionFunction, the actions's input
NestedSet<Artifact> could be evaluated as an ArtifactNestedSetKey [1].
ArtifactNestedSetFunction then evaluates the ArtifactNestedSetKey:
- Evaluates the directs elements as Artifacts. Commit the result into artifactToSkyValueMap.
- Evaluates the transitive elements as ArtifactNestedSetKey.
ActionExecutionFunction can then access this map to get the Artifacts' values.
[1] Heuristic: If the size of the NestedSet exceeds a certain threshold, we evaluate it as an ArtifactNestedSetKey.
Notes:
- The new feature is hidden behind a flag --experimental_nested_set_as_skykey_threshold. The default code path is not affected. We'd like to get this checked into the codebase, then from there improve it incrementally.
- It currently only works without the Action Cache.
Next Steps:
- Figure out the main source of additional pressure on GC & address it
- Add tests
RELNOTES: Introduce --experimental_nested_set_as_skykey_threshold
PiperOrigin-RevId: 273938957
diff --git a/src/main/java/com/google/devtools/build/lib/skyframe/ArtifactNestedSetValue.java b/src/main/java/com/google/devtools/build/lib/skyframe/ArtifactNestedSetValue.java
new file mode 100644
index 0000000..4343fc7
--- /dev/null
+++ b/src/main/java/com/google/devtools/build/lib/skyframe/ArtifactNestedSetValue.java
@@ -0,0 +1,45 @@
+// Copyright 2019 The Bazel Authors. All rights reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+package com.google.devtools.build.lib.skyframe;
+
+import com.google.devtools.build.lib.actions.ActionExecutionException;
+import com.google.devtools.build.lib.concurrent.ThreadSafety.Immutable;
+import com.google.devtools.build.lib.concurrent.ThreadSafety.ThreadSafe;
+import com.google.devtools.build.skyframe.SkyKey;
+import com.google.devtools.build.skyframe.SkyValue;
+import com.google.devtools.build.skyframe.ValueOrException2;
+import java.io.IOException;
+import java.util.Map;
+
+/**
+ * Represent a "promise" that the Artifacts under a NestedSet is evaluated by Skyframe and the
+ * ValueOrException is available in {@link ArtifactNestedSetFunction#artifactToSkyValueMap}
+ */
+@Immutable
+@ThreadSafe
+public class ArtifactNestedSetValue implements SkyValue {
+
+ private static ArtifactNestedSetValue singleton = null;
+
+ static ArtifactNestedSetValue createOrGetInstance() {
+ if (singleton == null) {
+ singleton = new ArtifactNestedSetValue();
+ }
+ return singleton;
+ }
+
+ Map<SkyKey, ValueOrException2<IOException, ActionExecutionException>> getArtifactToSkyValueMap() {
+ return ArtifactNestedSetFunction.getInstance().getArtifactToSkyValueMap();
+ }
+}
