layout: documentation title: Extensions - Overview

Overview

Loading an extension

Extensions are files with the .bzl extension. Use the load statement to import a symbol from an extension.

load("//build_tools/rules:maprule.bzl", "maprule")

This code will load the file build_tools/rules/maprule.bzl and add the maprule symbol to the environment. This can be used to load new rules, functions or constants (e.g. a string, a list, etc.). Multiple symbols can be imported by using additional arguments to the call to load. Arguments must be string literals (no variable) and load statements must appear at top-level, i.e. they cannot be in a function body.

load also supports aliases, i.e. you can assign different names to the imported symbols.

load("//build_tools/rules:maprule.bzl", maprule_alias = "maprule")

You can define multiple aliases within one load statement. Moreover, the argument list can contain both aliases and regular symbol names. The following example is perfectly legal (please note when to use quotation marks).

load(":my_rules.bzl", "some_rule", nice_alias = "some_other_rule")

In a .bzl file, symbols starting with _ are private and cannot be loaded from another file. Visibility doesn‘t affect loading (yet): you don’t need to use exports_files to make a .bzl file visible.

Macros and rules

A macro is a function that instantiates rules. It is useful when a BUILD file is getting too repetitive or too complex, as it allows you to reuse some code. The function is evaluated as soon as the BUILD file is read. After the evaluation of the BUILD file, Bazel has little information about macros: if your macro generates a genrule, Bazel will behave as if you wrote the genrule. As a result, bazel query will only list the generated genrule.

A rule is more powerful than a macro. It can access Bazel internals and have full control over what is going on. It may for example pass information to other rules.

If you want to reuse simple logic, start with a macro. If a macro becomes complex, it is often a good idea to make it a rule. Support for a new language is typically done with a rule. Rules are for advanced users: we expect that most people will never have to write one, they will only load and call existing rules.

Evaluation model

A build consists of three phases.

  • Loading phase. First, we load and evaluate all extensions and all BUILD files that are needed for the build. The execution of the BUILD files simply instantiates rules (each time a rule is called, it gets added to a graph). This is where macros are evaluated.

  • Analysis phase. The code of the rules is executed (their implementation function), and actions are instantiated. An action describes how to generate a set of outputs from a set of inputs, e.g. “run gcc on hello.c and get hello.o”. It is important to note that we have to list explicitly which files will be generated before executing the actual commands. In other words, the analysis phase takes the graph generated by the loading phase and generates an action graph.

  • Execution phase. Actions are executed, when at least one of their outputs is required. If a file is missing or if a command fails to generate one output, the build fails. Tests are also run during this phase.

Bazel uses parallelism to read, parse and evaluate the .bzl files and BUILD files. A file is read at most once per build and the result of the evaluation is cached and reused. A file is evaluated only once all its dependencies (load() statements) have been resolved. By design, loading a .bzl file has no visible side-effect, it only defines values and functions.

Bazel tries to be clever: it uses dependency analysis to know which files must be loaded, which rules must be analyzed, and which actions must be executed. For example, if a rule generates actions that we don't need for the current build, they will not be executed.

Backward-incompatible changes

As we make changes and polish the extension mechanism, old features may be removed and new features that are not backwards-compatible may be added.

Each release, new incompatible changes will be behind a flag with its default value set to false. In later releases, the flag will be enabled by default, or the flag will be removed entirely.

To check if your code will be compatible with future releases:

  • build your code with the flag --all_incompatible_changes, or
  • use boolean flags to enable/disable specific incompatible changes.

This following are the planned incompatible changes that are implemented and guarded behind flags.

Set constructor

We are removing the set constructor. Use depset instead. set and depset are equivalent, you just need to do search and replace to update the old code.

We are doing this to reduce confusion between the specialized depset data structure and Python's set datatype.

  • Flag: --incompatible_disallow_set_constructor
  • Default: false

Keyword-only arguments

Keyword-only parameters are parameters that can be called only using their name.

def foo(arg1, *, arg2): pass

foo(3, arg2=3)

def bar(arg1, *rest, arg2): pass

bar(3, arg2=3)

In both examples, arg2 must be named at the call site. To preserve syntactic compatibility with Python 2, we are removing this feature (which we have never documented).

  • Flag: --incompatible_disallow_keyword_only_args
  • Default: false

Mutating +=

We are changing left += right when left is a list. The old behavior is equivalent to left = left + right, which creates a new list and assigns it to left. The new behavior does not rebind left, but instead just mutates the list in-place.

def fct():
  li = [1]
  alias = li
  li += [2]
  # Old behavior: alias == [1]
  # New behavior: alias == [1, 2]

This change makes Skylark more compatible with Python and avoids performance issues. The += operator for tuples is unaffected.

  • Flag: --incompatible_list_plus_equals_inplace
  • Default: false

Dictionary concatenation

We are removing the + operator on dictionaries. This includes the += form where the left-hand side is a dictionary. This is done to improve compatibility with Python. A possible workaround is to use the .update method instead.

  • Flag: --incompatible_disallow_dict_plus
  • Default: false

Load argument is a label

Historically, the first argument of load could be a path with an implicit .bzl suffix. We are going to require that all load statements use the label syntax.

load("/path/foo", "var")  # deprecated
load("//path:foo.bzl", "var")  # recommended
  • Flag: --incompatible_load_argument_is_label
  • Default: false

Top level if statements

This change forbids if statements at the top level of .bzl files (they are already forbidden in BUILD files). This change ensures that every global value has a single declaration. This restriction is consistent with the idea that global values cannot be redefined.

  • Flag: --incompatible_disallow_toplevel_if_statement
  • Default: false

Comprehensions variables

This change makes list and dict comprehensions follow Python 3‘s semantics instead of Python 2’s. That is, comprehensions have their own local scopes, and variables bound by comprehensions are not accessible in the outer scope.

As a temporary measure to help detect breakage, this change also causes variables defined in the immediate outer scope to become inaccessible if they are shadowed by any variables in a comprehension. This disallows any uses of the variable's name where its meaning would differ under the Python 2 and Python 3 semantics. Variables above the immediate outer scope are not affected.

def fct():
  x = 10
  y = [x for x in range(3)]
  return x

The meaning of this program depends on the flag:

  • Under Skylark without this flag: x is 10 before the comprehension and 2 afterwards. (2 is the last value assigned to x while evaluating the comprehension.)

  • Under Skylark with this flag: x becomes inaccessible after the comprehension, so that return x is an error. If we moved the x = 10 to above the function, so that x became a global variable, then no error would be raised, and the returned number would be 10.

In other words, please do not refer to a loop variable outside the list or dict comprehension.

  • Flag: --incompatible_comprehension_variables_do_not_leak
  • Default: false

Depset is no longer iterable

When the flag is set to true, depset objects are not treated as iterable. If you need an iterable, call the .to_list() method. This affects for loops and many functions, e.g. list, tuple, min, max, sorted, all, and any. The goal of this change is to avoid accidental iteration on depset, which can be expensive.

deps = depset()
[x.path for x in deps]  # deprecated
[x.path for x in deps.to_list()]  # recommended

sorted(deps)  # deprecated
sorted(deps.to_list())  # recommended
  • Flag: --incompatible_depset_is_not_iterable
  • Default: false

String is no longer iterable

When the flag is set to true, string objects are not treated as iterable. This affects for loops and many functions, e.g. list, tuple, min, max, sorted, all, and any. String iteration has been a source of errors and confusion, such as this error:

def my_macro(name, srcs):
  for src in srcs:
    # do something with src

my_macro("foo")  # equivalent to: my_macro(["f", "o", "o"])

String indexing and len are still allowed. If you need to iterate over a string, you may explicitly use:

my_string="hello world"
for i in range(len(my_string)):
  char = my_string[i]
  # do something with char
  • Flag: --incompatible_string_is_not_iterable
  • Default: false

Dictionary literal has no duplicates

When the flag is set to true, duplicated keys are not allowed in the dictionary literal syntax.

{"a": 2, "b": 3, "a": 4}  # error

When the flag is false, the last value overrides the previous value (so the example above is equivalent to {"a": 4, "b": 3}. This behavior has been a source of bugs, which is why we are going to forbid it.

If you really want to override a value, use a separate statement: mydict["a"] = 4.

  • Flag: --incompatible_dict_literal_has_no_duplicates
  • Default: false

Checked arithmetic

When set, arithmetic operations (+, -, *) will fail in case of overflow. All integers are stored using signed 32 bits.

  • Flag: --incompatible_incompatible_checked_arithmetic
  • Default: false

Profiling the code

To profile your code and analyze the performance, use the --profile flag:

$ bazel build --nobuild --profile=/tmp/prof //path/to:target
$ bazel analyze-profile /tmp/prof --html --html_details

Then, open the generated HTML file (/tmp/prof.html in the example).