---

layout: posts title: Using Skylark remote repositories to auto-detect the C++ toolchain.

Skylark remote repositories let you create custom external repositories using Skylark. This not only enables creating rules for custom package systems such as PyPi but also generating a repository to reflect the toolchain installed on the workstation Bazel is running on. We explain here how we implemented auto-configuration for the C++ toolchain.

Principles

Crosstool: a compiler capable of building for a certain architecture, which can be different from the host architecture (e.g., gcc running on Linux and building for Raspberry Pi).

C++ toolchains are configured in Bazel using a crosstool target and a CROSSTOOL file.

This crosstool target (:default_toolchain) is the first step in moving the contents of the CROSSTOOL file entirely into BUILD file rules. The CROSSTOOL file defines where to find the C++ compiler, its include directories and also the various flag to use at each compilation step.

When your C++ compiler is not in the standard location, then this static CROSSTOOL file cannot find it. To cope with the variety of installation out there, we created a cc_configure Skylark repository rule that will generates a @local_config_cc//tools/cpp package containing a generated CROSSTOOL file based on the information we gathered from the operating system.

Implementation

The cc_configure rule is actually a macro wrapping the cc_autoconf enforcing the local_config_cc name for the repository. The implementation of the cc_autoconf rule does the following step:

• Detect the cpu_value using the repository_ctx.os.name attribute.
• Generates a static package if we do not support the target platform.
• Detect the C++ compiler path using repository_ctx.which and the CC environment variable with repository_ctx.os.environ.
• Detect some more tool paths, still using repository_ctx.which.
• Generates the various flag for the CROSSTOOL file, testing flags against the detected compiler using repository_ctx.execute. We also detect the include directories with repository_ctx.execute.
• With the gathered information, generate the C++ tools package: its BUILD file, wrapper script for Darwin and CROSSTOOL file using repository_ctx.template.

So using the function provided by repository_ctx, we can discover the binaries on the system, what version they are, and which options they support, then generate a configuration to match the local C++ toolchain.

Creating your own repository rules

When creating a Skylark remote repository, a few things should be taken in considerations:

• The Skylark implementation of a remote repository is run during the loading phase of the repository, which means that unless the rule definition is changed in the WORKSPACE file or the implementation fails, it will not be re-run unless the user does a bazel clean --expunge. We are thinking of further command to force re-run that loading phase for a specific remote repository (#974).
• Skylark remote repository can do a lot of non hermetic operation, it is recommended to check as many things as possible to ensure hermeticity (and overall, we recommend using a vendored toolchain instead of using auto-detected one if reproducibility is important). For example, it is recommended to use the sha256 argument of the repository_ctx.download method.
• Naming a rule can be complex and we recommend to not use standard suffix of classical rules for remote repositories (e.g. *_library or *_binary). If you create a package rule, a good name would probably be xxx_package (e.g., pypi_package). If you create an autoconfiguration rule, xxx_configure is probably the best name (e.g. cc_configure).