Minimize the cost of Skyframe restarts during CC autoconfiguration.
Repository functions restart whenever they have to resolve a label due
to the way Skyframe currently works. This is problematic because such
functions are often costly: they try to detect features of the host
system by building and running tools, and they touch the file system.
To minimize the cost of these restarts, which are unavoidable, resolve
the paths to all necessary labels upfront. This way, while the restarts
still happen, they have no side-effects and thus are cheap. As a
side-effect of this work, remove the strange and undocumented "tpl"
function, which only made things more cryptic and harder to refactor.
This cuts down Bazel's own analysis time on macOS on my Mac Pro 2013
from 14 seconds to 6 seconds by trimming about 15 unnecessary rebuilds
and executions of the xcode-locator-bin tool to just 1. In other
words: one fewer awkward pause during analysis time.
Additionally, there is the hope that this will fix the problems users
observe where Bazel writes persistent garbage during autoconfiguration
(needing "clean --expunge" to recover) when Visual Studio Code is
running and Santa is enabled on macOS; see
https://github.com/bazelbuild/bazel/issues/4603. This is most likely
a bug in Santa (see https://github.com/google/santa/issues/260) but
we were tickling it due to our apparently-abusive behavior of building,
executing, and then removing the same thing over and over again.
Fixes https://github.com/bazelbuild/bazel/issues/5196 and should also
fix https://github.com/bazelbuild/bazel/issues/4603.
RELNOTES: None.
PiperOrigin-RevId: 198434395
diff --git a/tools/cpp/lib_cc_configure.bzl b/tools/cpp/lib_cc_configure.bzl
index 6e43342..8999806 100644
--- a/tools/cpp/lib_cc_configure.bzl
+++ b/tools/cpp/lib_cc_configure.bzl
@@ -15,6 +15,30 @@
"""Base library for configuring the C++ toolchain."""
+def resolve_labels(repository_ctx, labels):
+ """Resolves a collection of labels to their paths.
+
+ Label resolution can cause the evaluation of Skylark functions to restart.
+ For functions with side-effects (like the auto-configuration functions, which
+ inspect the system and touch the file system), such restarts are costly.
+ We cannot avoid the restarts, but we can minimize their penalty by resolving
+ all labels upfront.
+
+ Among other things, doing less work on restarts can cut analysis times by
+ several seconds and may also prevent tickling kernel conditions that cause
+ build failures. See https://github.com/bazelbuild/bazel/issues/5196 for
+ more details.
+
+ Args:
+ repository_ctx: The context with which to resolve the labels.
+ labels: Labels to be resolved expressed as a list of strings.
+
+ Returns:
+ A dictionary with the labels as keys and their paths as values.
+ """
+ return dict([(label, repository_ctx.path(Label(label))) for label in labels])
+
+
def escape_string(arg):
"""Escape percent sign (%) in the string so it can appear in the Crosstool."""
if arg != None:
@@ -22,6 +46,7 @@
else:
return None
+
def split_escaped(string, delimiter):
"""Split string on the delimiter unless %-escaped.
@@ -39,11 +64,12 @@
split_escaped("a::b", ":") -> [ "a", "", "", "b" ]
Args:
- string: a string to be splitted
- delimiter: non-empty string not containing %-sign to be used as a delimiter
+ string: The string to be split.
+ delimiter: Non-empty string not containing %-sign to be used as a
+ delimiter.
Returns:
- a list of substrings
+ A list of substrings.
"""
if delimiter == "": fail("Delimiter cannot be empty")
if delimiter.find("%") != -1: fail("Delimiter cannot contain %-sign")
@@ -162,15 +188,6 @@
return "k8" if result.stdout.strip() in ["amd64", "x86_64", "x64"] else "piii"
-def tpl(repository_ctx, template, substitutions={}, out=None):
- if not out:
- out = template
- repository_ctx.template(
- out,
- Label("@bazel_tools//tools/cpp:%s.tpl" % template),
- substitutions)
-
-
def is_cc_configure_debug(repository_ctx):
"""Returns True if CC_CONFIGURE_DEBUG is set to 1."""
env = repository_ctx.os.environ
