blob: ab5cf44daf21900d912dbc53d57687dbcb293174 [file] [log] [blame]
---
layout: documentation
title: User's Guide
---
<h1>A User's Guide to Bazel</h1>
<p>
To run Bazel, go to
your base <a href="build-ref.html#workspace">workspace</a> directory
or any of its subdirectories and type <code>bazel</code>.
</p>
<pre>
% bazel help
[Bazel release bazel-&lt;<i>version</i>&gt;]
Usage: bazel &lt;command&gt; &lt;options&gt; ...
Available commands:
<a href='user-manual.html#analyze-profile'>analyze-profile</a> Analyzes build profile data.
<a href='user-manual.html#aquery'>aquery</a> Executes a query on the <a href='#analysis-phase'>post-analysis</a> action graph.
<a href='#build'>build</a> Builds the specified targets.
<a href='user-manual.html#canonicalize'>canonicalize-flags</a> Canonicalize Bazel flags.
<a href='user-manual.html#clean'>clean</a> Removes output files and optionally stops the server.
<a href='user-manual.html#query'>cquery</a> Executes a <a href='#analysis-phase'>post-analysis</a> dependency graph query.
<a href='user-manual.html#dump'>dump</a> Dumps the internal state of the Bazel server process.
<a href='user-manual.html#help'>help</a> Prints help for commands, or the index.
<a href='user-manual.html#info'>info</a> Displays runtime info about the bazel server.
<a href='#fetch'>fetch</a> Fetches all external dependencies of a target.
<a href='user-manual.html#mobile-install'>mobile-install</a> Installs apps on mobile devices.
<a href='user-manual.html#query'>query</a> Executes a dependency graph query.
<a href='user-manual.html#run'>run</a> Runs the specified target.
<a href='user-manual.html#shutdown'>shutdown</a> Stops the Bazel server.
<a href='user-manual.html#test'>test</a> Builds and runs the specified test targets.
<a href='user-manual.html#version'>version</a> Prints version information for Bazel.
Getting more help:
bazel help &lt;command&gt;
Prints help and options for &lt;command&gt;.
bazel help <a href='user-manual.html#startup_options'>startup_options</a>
Options for the JVM hosting Bazel.
bazel help <a href='#target-patterns'>target-syntax</a>
Explains the syntax for specifying targets.
bazel help info-keys
Displays a list of keys used by the info command.
</pre>
<p>
The <code>bazel</code> tool performs many functions, called
commands. The most commonly used ones are <code>bazel build</code>
and <code>bazel test</code>. You can browse the online help
messages using <code>bazel help</code>.
</p>
<h2 id='build'>Building programs with Bazel</h2>
<h3>The <code>build</code> command</h3>
<p>
Type <code>bazel build</code> followed by the name of the
<a href="#target-patterns">target</a> you wish to build. Here's a typical
session:
</p>
<pre>
% bazel build //foo
____Loading package: foo
____Loading package: bar
____Loading package: baz
____Loading complete. Analyzing...
____Building 1 target...
____[0 / 3] Executing Genrule //bar:helper_rule
____[1 / 3] Executing Genrule //baz:another_helper_rule
____[2 / 3] Building foo/foo.bin
Target //foo:foo up-to-date:
bazel-bin/foo/foo.bin
bazel-bin/foo/foo
____Elapsed time: 9.905s
</pre>
<p>
Bazel prints the progress messages as it loads all the
packages in the transitive closure of dependencies of the requested
target, then analyzes them for correctness and to create the build actions,
finally executing the compilers and other tools of the build.
</p>
<p>
Bazel prints progress messages during
the <a href='#execution-phase'>execution phase</a> of the build, showing the
current build step (compiler, linker, etc.) that is being started,
and the number completed over the total number of build actions. As the
build starts the number of total actions will often increase as Bazel
discovers the entire action graph, but the number will usually stabilize
within a few seconds.
</p>
<p>
At the end of the build Bazel
prints which targets were requested, whether or not they were
successfully built, and if so, where the output files can be found.
Scripts that run builds can reliably parse this output; see <a
href='#flag--show_result'><code class='flag'>--show_result</code></a> for more
details.
</p>
<p>
Typing the same command again:
</p>
<pre>
% bazel build //foo
____Loading...
____Found 1 target...
____Building complete.
Target //foo:foo up-to-date:
bazel-bin/foo/foo.bin
bazel-bin/foo/foo
____Elapsed time: 0.280s
</pre>
<p>
we see a "null" build: in this case, there are no packages to
re-load, since nothing has changed, and no build steps to execute.
(If something had changed in "foo" or some of its dependencies, resulting in the
reexecution of some build actions, we would call it an "incremental" build, not a
"null" build.)
</p>
<p>
Before you can start a build, you will need a Bazel workspace. This is
simply a directory tree that contains all the source files needed to build
your application.
Bazel allows you to perform a build from a completely read-only volume.
</p>
<h3 id='target-patterns'>Specifying targets to build</h3>
<p>
Bazel allows a number of ways to specify the targets to be built.
Collectively, these are known as <i>target patterns</i>. This syntax is
used in commands like <code>build</code>, <code>test</code>,
or <code>query</code>.
</p>
<p>
Whereas <a href="build-ref.html#labels">labels</a> are used
to specify individual targets, e.g. for declaring dependencies in
BUILD files, Bazel's target patterns are a syntax for specifying
multiple targets: they are a generalization of the label syntax
for <i>sets</i> of targets, using wildcards. In the simplest case,
any valid label is also a valid target pattern, identifying a set of
exactly one target.
</p>
<p>
All target patterns starting with <code>//</code> are resolved relative to the current
workspace.
<table>
<tr>
<td><code>//foo/bar:wiz</code></td>
<td>Just the single target <code>//foo/bar:wiz</code>.</td>
</tr>
<tr>
<td><code>//foo/bar</code></td>
<td>Equivalent to <code>//foo/bar:bar</code>.</td>
</tr>
<tr>
<td><code>//foo/bar:all</code></td>
<td>All rules in the package <code>foo/bar</code>.</td>
</tr>
<tr>
<td><code>//foo/...</code></td>
<td>All rules in all packages beneath the directory <code>foo</code>.</td>
</tr>
<tr>
<td><code>//foo/...:all</code></td>
<td>All rules in all packages beneath the directory <code>foo</code>.</td>
</tr>
<tr>
<td><code>//foo/...:*</code></td>
<td>All targets (rules and files) in all packages beneath the directory <code>foo</code>.</td>
</tr>
<tr>
<td><code>//foo/...:all-targets</code></td>
<td>All targets (rules and files) in all packages beneath the directory <code>foo</code>.</td>
</tr>
</table>
</p>
<p>
Target patterns which do not begin with <code>//</code> are resolved relative to the
current <em>working directory</em>. These examples assume a working directory of
<code>foo</code>:
<table>
<tr>
<td><code>:foo</code></td>
<td>Equivalent to <code>//foo:foo</code>.</td>
</tr>
<tr>
<td><code>bar:wiz</code></td>
<td>Equivalent to <code>//foo/bar:wiz</code>.</td>
</tr>
<tr>
<td><code>bar/wiz</code></td>
<td>Equivalent to:
<code>//foo/bar/wiz:wiz</code> if <code>foo/bar/wiz</code> is a package,
<code>//foo/bar:wiz</code> if <code>foo/bar</code> is a package,
<code>//foo:bar/wiz</code> otherwise.
</td>
</tr>
<tr>
<td><code>bar:all</code></td>
<td>Equivalent to <code>//foo/bar:all</code>.</td>
</tr>
<tr>
<td><code>:all</code></td>
<td>Equivalent to <code>//foo:all</code>.</td>
</tr>
<tr>
<td><code>...:all</code></td>
<td>Equivalent to <code>//foo/...:all</code>.</td>
</tr>
<tr>
<td><code>...</code></td>
<td>Equivalent to <code>//foo/...:all</code>.</td>
</tr>
<tr>
<td><code>bar/...:all</code></td>
<td>Equivalent to <code>//foo/bar/...:all</code>.</td>
</tr>
</table>
</p>
<p>
By default, directory symlinks are followed for recursive target patterns,
except those that point to under the output base, such as the convenience
symlinks that are created in the root directory of the workspace.
</p>
<p>
In addition, Bazel does not follow symlinks when evaluating recursive target
patterns in any directory that contains a file named as follows:
<pre>
DONT_FOLLOW_SYMLINKS_WHEN_TRAVERSING_THIS_DIRECTORY_VIA_A_RECURSIVE_TARGET_PATTERN
</pre>
</p>
<p>
<code>foo/...</code> is a wildcard over <em>packages</em>,
indicating all packages recursively beneath
directory <code>foo</code> (for all roots of the package
path). <code>:all</code> is a wildcard
over <em>targets</em>, matching all rules within a package. These two may be
combined, as in <code>foo/...:all</code>, and when both wildcards
are used, this may be abbreviated to <code>foo/...</code>.
</p>
<p>
In addition, <code>:*</code> (or <code>:all-targets</code>) is a
wildcard that matches <em>every target</em> in the matched packages,
including files that aren't normally built by any rule, such
as <code>_deploy.jar</code> files associated
with <code>java_binary</code> rules.
</p>
<p>
This implies that <code>:*</code> denotes a <em>superset</em>
of <code>:all</code>; while potentially confusing, this syntax does
allow the familiar <code>:all</code> wildcard to be used for
typical builds, where building targets like the <code>_deploy.jar</code>
is not desired.
</p>
<p>
In addition, Bazel allows a slash to be used instead of the colon
required by the label syntax; this is often convenient when using
Bash filename expansion. For example, <code>foo/bar/wiz</code> is
equivalent to <code>//foo/bar:wiz</code> (if there is a
package <code>foo/bar</code>) or to <code>//foo:bar/wiz</code> (if
there is a package <code>foo</code>).
</p>
<p>
Many Bazel commands accept a list of target patterns as arguments,
and they all honor the prefix negation operator <code>-</code>.
This can be used to subtract a set of targets from the set specified
by the preceding arguments. Note that this means order matters.
For example,
</p>
<pre>
bazel build foo/... bar/...
</pre>
<p>
means "build all
targets beneath <code>foo</code> <i>and</i> all targets
beneath <code>bar</code>", whereas
</p>
<pre>
bazel build -- foo/... -foo/bar/...
</pre>
<p>
means "build all targets beneath <code>foo</code> <i>except</i>
those beneath <code>foo/bar</code>".
(The <code>--</code> argument is required to prevent the subsequent
arguments starting with <code>-</code> from being interpreted as
additional options.)
</p>
<p>
It's important to point out though that subtracting targets this way will not
guarantee that they are not built, since they may be dependencies of targets
that weren't subtracted. For example, if there were a target
<code>//foo:all-apis</code> that among others depended on
<code>//foo/bar:api</code>, then the latter would be built as part of
building the former.
</p>
<p>
Targets with <code>tags = ["manual"]</code> will not be included in wildcard target patterns (
<code>...</code>, <code>:*</code>, <code>:all</code>, etc.).
You should specify such test targets with explicit target patterns on the command
line if you want Bazel to build/test them.
</p>
<h3 id='fetch'>Fetching external dependencies</h3>
<p>
By default, Bazel will download and symlink external dependencies during the
build. However, this can be undesirable, either because you'd like to know
when new external dependendencies are added or because you'd like to
"prefetch" dependencies (say, before a flight where you'll be offline). If you
would like to prevent new dependencies from being added during builds, you
can specify the <code>--fetch=false</code> flag. Note that this flag only
applies to repository rules that do not point to a directory in the local
file system. Changes, for example, to <code>local_repository</code>,
<code>new_local_repository</code> and Android SDK and NDK repository rules
will always take effect regardless of the value <code>--fetch</code> .
</p>
<p>
If you disallow fetching during builds and Bazel finds new external
dependencies, your build will fail.
</p>
<p>
You can manually fetch dependencies by running <code>bazel fetch</code>. If
you disallow during-build fetching, you'll need to run <code>bazel
fetch</code>:
<ol>
<li>Before you build for the first time.
<li>After you add a new external dependency.
</ol>
Once it has been run, you should not need to run it again until the WORKSPACE
file changes.
</p>
<p>
<code>fetch</code> takes a list of targets to fetch dependencies for. For
example, this would fetch dependencies needed to build <code>//foo:bar</code>
and <code>//bar:baz</code>:
<pre>
$ bazel fetch //foo:bar //bar:baz
</pre>
</p>
<p>
To fetch all external dependencies for a workspace, run:
<pre>
$ bazel fetch //...
</pre>
</p>
<p>
You do not need to run bazel fetch at all if you have all of the tools you are
using (from library jars to the JDK itself) under your workspace root.
However, if you're using anything outside of the workspace directory then Bazel
will automatically run <code>bazel fetch</code> before running
<code>bazel build</code>.
</p>
<h4 id='repository-cache'>The repository cache</h4>
<p>
Bazel tries to avoid fetching the same file several times, even if the same
file is needed in different workspaces, or if the definition of an external
repository changed but it still needs the same file to download. To do so,
bazel caches all files downloaded in the repository cache which, by default,
is located at <code>~/.cache/bazel/_bazel_$USER/cache/repos/v1/</code>. The
location can be changed by the <code>--repository_cache</code> option. The
cache is shared between all workspaces and installed versions of bazel.
An entry is taken from the cache if
Bazel knows for sure that it has a copy of the correct file, that is, if the
download request has a SHA256 sum of the file specified and a file with that
hash is in the cache. So specifying a hash for each external file is
not only a good idea from a security perspective; it also helps avoiding
unnecessary downloads.
</p>
<p>
Upon each cache hit, the modification time of the file in the cache is
updated. In this way, the last use of a file in the cache directory can easily
be determined, for example to manually clean up the cache. The cache is never
cleaned up automatically, as it might contain a copy of a file that is no
longer available upstream.
</p>
<h4 id='distdir'>Distribution files directories</h4>
<p>
Besides the repository cache, Bazel has a second mechanism to avoid
unnecessary downloads. Using the <code>--distdir</code> option, you can
specify additional read-only directories to look for files instead of fetching
them. A file is taken from such a directory if the file name is equal to
the base name of the URL and additionally the hash of the file is equal to the
one specified in the download request (again, no file is taken from a distdir,
if no hash is specified). While the condition on the file name is not
necessary for correctness, it reduces the number of candidate files to one per
specified directory. In this way, specifying distribution files directories
remains efficient, even if the number of files in such a directory grows
large.
</p>
<h3 id='configurations'>Build configurations and cross-compilation</h3>
<p>
All the inputs that specify the behavior and result of a given
build can be divided into two distinct categories.
The first kind is the intrinsic information stored in the BUILD
files of your project: the build rule, the values of its attributes,
and the complete set of its transitive dependencies.
The second kind is the external or environmental data, supplied by
the user or by the build tool: the choice of target architecture,
compilation and linking options, and other toolchain configuration
options. We refer to a complete set of environmental data as
a <b>configuration</b>.
</p>
<p>
In any given build, there may be more than one configuration.
Consider a cross-compile, in which you build
a <code>//foo:bin</code> executable for a 64-bit architecture,
but your workstation is a 32-bit machine. Clearly, the build
will require building <code>//foo:bin</code> using a toolchain
capable of creating 64-bit executables, but the build system must
also build various tools used during the build itself&mdash;for example
tools that are built from source, then subsequently used in, say, a
genrule&mdash;and these must be built to run on your workstation.
Thus we can identify two configurations: the <b>host
configuration</b>, which is used for building tools that run during
the build, and the <b>target configuration</b> (or <i>request
configuration</i>, but we say "target configuration" more often even
though that word already has many meanings), which is
used for building the binary you ultimately requested.
</p>
<p>
Typically, there are many libraries that are prerequisites of both
the requested build target (<code>//foo:bin</code>) and one or more of
the host tools, for example some base libraries. Such libraries must be built
twice, once for the host configuration, and once for the target
configuration.<br/>
Bazel takes care of ensuring that both variants are built, and that
the derived files are kept separate to avoid interference; usually
such targets can be built concurrently, since they are independent
of each other. If you see progress messages indicating that a given
target is being built twice, this is most likely the explanation.
</p>
<p>
Bazel uses one of two ways to select the host configuration, based
on the <code class='flag'>--distinct_host_configuration</code> option. This
boolean option is somewhat subtle, and the setting may improve (or
worsen) the speed of your builds.
</p>
<h4><code class='flag'>--distinct_host_configuration=false</code></h4>
<p>
When this option is false, the host and
request configurations are identical: all tools required during the
build will be built in exactly the same way as target programs.
This setting means that no libraries need to be built twice during a
single build, so it keeps builds short.
However, it does mean that any change to your request configuration
also affects your host configuration, causing all the tools to be
rebuilt, and then anything that depends on the tool output to be
rebuilt too. Thus, for example, simply changing a linker option
between builds might cause all tools to be re-linked, and then all
actions using them reexecuted, and so on, resulting in a very large rebuild.
Also, please note: if your host architecture is not capable of
running your target binaries, your build will not work.
</p>
<p>
If you frequently make changes to your request configuration, such
as alternating between <code>-c opt</code> and <code>-c dbg</code>
builds, or between simple- and cross-compilation, we do not
recommend this option, as you will typically rebuild the majority of
your codebase each time you switch.
</p>
<h4><code class='flag'>--distinct_host_configuration=true</code> <i>(default)</i></h4>
<p>
If this option is true, then instead of using the same configuration
for the host and request, a completely distinct host configuration
is used. The host configuration is derived from the target
configuration as follows:
</p>
<ul>
<li>Use the same version of Crosstool
(<code class='flag'>--crosstool_top</code>) as specified in the request
configuration, unless <code class='flag'>--host_crosstool_top</code> is
specified.
</li>
<li>
Use the value of <code class="flag">--host_cpu</code> for
<code class='flag'>--cpu</code>
(default: <code>k8</code>).
</li>
<li>Use the same values of these options as specified in the request
configuration:
<code class='flag'>--compiler</code>,
<code class='flag'>--use_ijars</code>,
If <code class='flag'>--host_crosstool_top</code> is used, then the value of
<code class='flag'>--host_cpu</code> is used to look up a
<code>default_toolchain</code> in the Crosstool
(ignoring <code class='flag'>--compiler</code>) for the host configuration.
</li>
<li>
Use the value of <code class="flag">--host_javabase</code> for
<code class='flag'>--javabase</code>
</li>
<li>
Use the value of <code class="flag">--host_java_toolchain</code> for
<code class='flag'>--java_toolchain</code>
</li>
<li>Use optimized builds for C++ code (<code>-c opt</code>).
</li>
<li>Generate no debugging information (<code class='flag'>--copt=-g0</code>).
</li>
<li>Strip debug information from executables and shared libraries
(<code class='flag'>--strip=always</code>).
</li>
<li>Place all derived files in a special location, distinct from
that used by any possible request configuration.
</li>
<li>Suppress stamping of binaries with build data
(see <code class='flag'>--embed_*</code> options).
</li>
<li>All other values remain at their defaults.
</li>
</ul>
<p>
There are many reasons why it might be preferable to select a
distinct host configuration from the request configuration.
Some are too esoteric to mention here, but two of them are worth
pointing out.
</p>
<p>
Firstly, by using stripped, optimized binaries, you reduce the time
spent linking and executing the tools, the disk space occupied by
the tools, and the network I/O time in distributed builds.
</p>
<p>
Secondly, by decoupling the host and request configurations in all
builds, you avoid very expensive rebuilds that would result from
minor changes to the request configuration (such as changing a linker options
does), as described earlier.
</p>
<p>
That said, for certain builds, this option may be a hindrance. In
particular, builds in which changes of configuration are infrequent
(especially certain Java builds), and builds where the amount of code that
must be built in both host and target configurations is large, may
not benefit.
</p>
<h3 id='correctness'>Correct incremental rebuilds</h3>
<p>
One of the primary goals of the Bazel project is to ensure correct
incremental rebuilds. Previous build tools, especially those based
on Make, make several unsound assumptions in their implementation of
incremental builds.
</p>
<p>
Firstly, that timestamps of files increase monotonically. While
this is the typical case, it is very easy to fall afoul of this
assumption; syncing to an earlier revision of a file causes that file's
modification time to decrease; Make-based systems will not rebuild.
</p>
<p>
More generally, while Make detects changes to files, it does
not detect changes to commands. If you alter the options passed to
the compiler in a given build step, Make will not re-run the
compiler, and it is necessary to manually discard the invalid
outputs of the previous build using <code>make clean</code>.
</p>
<p>
Also, Make is not robust against the unsuccessful termination of one
of its subprocesses after that subprocess has started writing to
its output file. While the current execution of Make will fail, the
subsequent invocation of Make will blindly assume that the truncated
output file is valid (because it is newer than its inputs), and it
will not be rebuilt. Similarly, if the Make process is killed, a
similar situation can occur.
</p>
<p>
Bazel avoids these assumptions, and others. Bazel maintains a database
of all work previously done, and will only omit a build step if it
finds that the set of input files (and their timestamps) to that
build step, and the compilation command for that build step, exactly
match one in the database, and, that the set of output files (and
their timestamps) for the database entry exactly match the
timestamps of the files on disk. Any change to the input files or
output files, or to the command itself, will cause re-execution of
the build step.
</p>
<p>
The benefit to users of correct incremental builds is: less time
wasted due to confusion. (Also, less time spent waiting for
rebuilds caused by use of <code>make clean</code>, whether necessary
or pre-emptive.)
</p>
<h4>Build consistency and incremental builds</h4>
<p>
Formally, we define the state of a build as <i>consistent</i> when
all the expected output files exist, and their contents are correct,
as specified by the steps or rules required to create them. When
you edit a source file, the state of the build is said to
be <i>inconsistent</i>, and remains inconsistent until you next run
the build tool to successful completion. We describe this situation
as <i>unstable inconsistency</i>, because it is only temporary, and
consistency is restored by running the build tool.
</p>
<p>
There is another kind of inconsistency that is pernicious: <i>stable
inconsistency</i>. If the build reaches a stable inconsistent
state, then repeated successful invocation of the build tool does
not restore consistency: the build has gotten "stuck", and the
outputs remain incorrect. Stable inconsistent states are the main
reason why users of Make (and other build tools) type <code>make
clean</code>. Discovering that the build tool has failed in this
manner (and then recovering from it) can be time consuming and very
frustrating.
</p>
<p>
Conceptually, the simplest way to achieve a consistent build is to
throw away all the previous build outputs and start again: make
every build a clean build. This approach is obviously too
time-consuming to be practical (except perhaps for release
engineers), and therefore to be useful, the build tool must be able
to perform incremental builds without compromising consistency.
</p>
<p>
Correct incremental dependency analysis is hard, and as described
above, many other build tools do a poor job of avoiding stable
inconsistent states during incremental builds. In contrast, Bazel
offers the following guarantee: after a successful invocation of the
build tool during which you made no edits, the build will be in a
consistent state. (If you edit your source files during a build,
Bazel makes no guarantee about the consistency of the result of the
current build. But it does guarantee that the results of
the <i>next</i> build will restore consistency.)
</p>
<p>
As with all guarantees, there comes some fine print: there are some
known ways of getting into a stable inconsistent state with Bazel.
We won't guarantee to investigate such problems arising from deliberate
attempts to find bugs in the incremental dependency analysis, but we
will investigate and do our best to fix all stable inconsistent
states arising from normal or "reasonable" use of the build tool.
</p>
<p>
If you ever detect a stable inconsistent state with Bazel, please report a bug.
</p>
<h4 id='sandboxing'>Sandboxed execution</h4>
<p>
Bazel uses sandboxes to guarantee that actions run hermetically<sup>1</sup> and correctly.
Bazel runs <i>Spawn</i>s (loosely speaking: actions) in sandboxes that only contain the minimal
set of files the tool requires to do its job. Currently sandboxing works on Linux 3.12 or newer
with the <code>CONFIG_USER_NS</code> option enabled, and also on macOS 10.11 or newer.
</p>
<p>
Bazel will print a warning if your system does not support sandboxing to alert you to the fact
that builds are not guaranteed to be hermetic and might affect the host system in unknown ways.
To disable this warning you can pass the <code>--ignore_unsupported_sandboxing</code> flag to
Bazel.
</p>
<p>
On some platforms such as <a href="https://cloud.google.com/kubernetes-engine/">Google Kubernetes
Engine</a> cluster nodes or Debian, user namespaces are deactivated by default due to security
concerns. This can be checked by looking at the file
<code>/proc/sys/kernel/unprivileged_userns_clone</code>: if it exists and contains a 0, then
user namespaces can be activated with <code>sudo sysctl kernel.unprivileged_userns_clone=1</code>.
</p>
<p>
In some cases, the Bazel sandbox fails to execute rules because of the system setup. The symptom
is generally a failure that output a message similar to
<code>namespace-sandbox.c:633: execvp(argv[0], argv): No such file or directory</code>. In that
case, try to deactivate the sandbox for genrules with <code>--strategy=Genrule=standalone</code>
and for other rules with <code>--spawn_strategy=standalone</code>. Also please report a bug on our
issue tracker and mention which Linux distribution you're using so that we can investigate and
provide a fix in a subsequent release.
</p>
<p>
<sup>1</sup>: Hermeticity means that the action only uses its declared input files and no other
files in the filesystem, and it only produces its declared output files.
</p>
<h3 id='phases'>Phases of a build</h3>
<p>
In Bazel, a build occurs in three distinct phases; as a user,
understanding the difference between them provides insight into the
options which control a build (see below).
</p>
<h4 id='loading-phase'>Loading phase</h4>
<p>
The first is <b>loading</b> during which all the necessary BUILD
files for the initial targets, and their transitive closure of
dependencies, are loaded, parsed, evaluated and cached.
</p>
<p>
For the first build after a Bazel server is started, the loading
phase typically takes many seconds as many BUILD files are loaded
from the file system. In subsequent builds, especially if no BUILD
files have changed, loading occurs very quickly.
</p>
<p>
Errors reported during this phase include: package not found, target
not found, lexical and grammatical errors in a BUILD file,
and evaluation errors.
</p>
<h4 id='analysis-phase'>Analysis phase</h4>
<p>
The second phase, <b>analysis</b>, involves the semantic analysis
and validation of each build rule, the construction of a build
dependency graph, and the determination of exactly what work is to
be done in each step of the build.
</p>
<p>
Like loading, analysis also takes several seconds when computed in
its entirety. However, Bazel caches the dependency graph from
one build to the next and only reanalyzes what it has to, which can
make incremental builds extremely fast in the case where the
packages haven't changed since the previous build.
</p>
<p>
Errors reported at this stage include: inappropriate dependencies,
invalid inputs to a rule, and all rule-specific error messages.
</p>
<p>
The loading and analysis phases are fast because
Bazel avoids unnecessary file I/O at this stage, reading only BUILD
files in order to determine the work to be done. This is by design,
and makes Bazel a good foundation for analysis tools, such as
Bazel's <a href='#query'>query</a> command, which is implemented
atop the loading phase.
</p>
<h4 id='execution-phase'>Execution phase</h4>
<p>
The third and final phase of the build is <b>execution</b>. This
phase ensures that the outputs of each step in the build are
consistent with its inputs, re-running compilation/linking/etc. tools as
necessary. This step is where the build spends the majority of
its time, ranging from a few seconds to over an hour for a large
build. Errors reported during this phase include: missing source
files, errors in a tool executed by some build action, or failure of a tool to
produce the expected set of outputs.
</p>
<h2 id='client/server'>Client/server implementation</h2>
<p>
The Bazel system is implemented as a long-lived server process.
This allows it to perform many optimizations not possible with a
batch-oriented implementation, such as caching of BUILD files,
dependency graphs, and other metadata from one build to the
next. This improves the speed of incremental builds, and allows
different commands, such as <code>build</code>
and <code>query</code> to share the same cache of loaded packages,
making queries very fast.
</p>
<p>
When you run <code>bazel</code>, you're running the client. The
client finds the server based on the output base, which by default is
determined by the path of the base workspace directory and your
userid, so if you build in multiple workspaces, you'll have multiple
output bases and thus multiple Bazel server processes. Multiple
users on the same workstation can build concurrently in the same
workspace because their output bases will differ (different userids).
If the client cannot find a running server instance, it starts a new
one. The server process will stop after a period of inactivity (3 hours,
by default, which can be modified using the startup option <code>--max_idle_secs</code>).
</p>
<p>
For the most part, the fact that there is a server running is
invisible to the user, but sometimes it helps to bear this in mind.
For example, if you're running scripts that perform a lot of
automated builds in different directories, it's important to ensure
that you don't accumulate a lot of idle servers; you can do this by
explicitly shutting them down when you're finished with them, or by
specifying a short timeout period.
</p>
<p>
The name of a Bazel server process appears in the output of <code>ps
x</code> or <code>ps -e f</code> as
<code>bazel(<i>dirname</i>)</code>, where <i>dirname</i> is the
basename of the directory enclosing the root of your workspace directory.
For example:
</p>
<pre>
% ps -e f
16143 ? Sl 3:00 bazel(src-johndoe2) -server -Djava.library.path=...
</pre>
<p>
This makes it easier to find out which server process belongs to a
given workspace. (Beware that with certain other options
to <code>ps</code>, Bazel server processes may be named just
<code>java</code>.) Bazel servers can be stopped using
the <a href='#shutdown'>shutdown</a> command.
</p>
<p>
When running <code>bazel</code>, the client first checks that the
server is the appropriate version; if not, the server is stopped and
a new one started. This ensures that the use of a long-running
server process doesn't interfere with proper versioning.
</p>
<h2 id='bazelrc'>
<code>.bazelrc</code>, the Bazel configuration file,
the <code class='flag'>--bazelrc=<var>file</var></code> option, and
the <code class='flag'>--config=<var>value</var></code> option
</h2>
<p>
Bazel accepts many options. Typically, some of these are varied
frequently (for example, <code class='flag'>--subcommands</code>) while others stay the
same across several builds (e.g. <code class='flag'>--package_path</code>).
To avoid having to specify these unchanged options for every build (and other commands)
Bazel allows you to specify options in a configuration file.
</p>
<h4>Where are the <code>.bazelrc</code> files?</h4>
<p>
Bazel looks for an optional configuration file in the following locations,
in the order shown below. The options are interpreted in this order, so
options in later files can override a value from an earlier file if a
conflict arises. All options to control which of these files are loaded are
startup options, which means they must occur after <code>bazel</code> and
before the command (<code>build</code>, etc).
</p>
<ol>
<li>
Unless the <code class='flag'>--nosystem_rc</code> is present, Bazel looks for
the system .bazelrc file: on Unix, it lives at <code>/etc/bazel.bazelrc</code>,
and on Windows at <code>%ProgramData%/bazel.bazelrc</code>.
If another system-specified location is required, this value can be
changed by setting <code>BAZEL_SYSTEM_BAZELRC_PATH</code> in
<code>src/main/cpp:option_processor</code> and using this custom Bazel binary.
The system-specified location may contain environment variable
substitutions, as <code>%var_name%</code> on Windows
or <code>${var_name}</code> on Unix.
</li>
<li>
Unless the <code class='flag'>--noworkspace_rc</code> is present, Bazel looks
for the <code>.bazelrc</code> file in your workspace directory.
</li>
<li>
Unless the <code class='flag'>--nohome_rc</code> is present, Bazel looks for
the home, or user, bazelrc: the file <code>.bazelrc</code> in your home
directory.
</li>
<li>
An additional .rc file can be specified by the
<code class='flag'>--bazelrc=<var>file</var></code> startup option. If this
option is not present, no additional file is loaded. Unlike in the three
default locations specified above, an incorrect path or non-existent file
will fail if passed explicitly.
</li>
</ol>
<h4><code>.bazelrc</code> syntax and semantics</h4>
<p>
Like all UNIX "rc" files, the <code>.bazelrc</code> file is a text file with
a line-based grammar. Lines starting <code>#</code> are considered comments
and are ignored, as are blank lines. Each line contains a sequence of words,
which are tokenized according to the same rules as the Bourne shell.
</p>
<h5>Imports</h5>
<p>
Lines that start with <code>import</code> are special: if Bazel encounters such
a line in a <code>.bazelrc</code> file, it parses the contents of the file
referenced by the import statement, too. Options specified in an imported file
take precedence over options specified before the import statement. Options
specified after the import statement take precedence over the options in the
imported file. Options in files imported later take precedence over files
imported earlier. To specify a path that is relative to the workspace root,
write <code>import %workspace%/path/to/bazelrc</code>.
</p>
<h5>Option defaults</h5>
<p>
Most lines of a bazelrc define default option values. The first word on each
line specifies when these defaults are applied:
</p>
<ol>
<li>
<code>startup</code>: startup options, which go before the command, and
are described in <code>bazel help startup_options</code>.
</li>
<li>
<code>common</code>: options that apply to all Bazel commands.
</li>
<li>
<code><i>command</i></code>: Bazel command, such as <code>build</code>
or <code>query</code> to which the options apply. These options also apply
to all commands that inherit from the specified command. (For example,
<code>test</code> inherits from <code>build</code>.)
</li>
</ol>
<p>
Each of these lines may be used more than once and the arguments that follow
the first word are combined as if they had appeared on a single line.
(Users of CVS, another tool with a "Swiss army knife" command-line interface,
will find the syntax similar to that of <code>.cvsrc</code>.)
</p>
<p>
Options specified in the command line always take precedence over those from
a configuration file. Within the configuration file, precedence is
given by specificity. This means that lines for a more specific command take
precedence over lines for a less specific command, with <code>common</code>
getting lowest precedence (for example, the <code>test</code> command inherits
all the options from the <code>build</code> command, so the line
<code>test --foo=bar</code> takes precedence over the line
<code>build --foo=baz</code>, regardless of which rc file or what order
these two lines are in). Two lines specifying options for the same command at
equal specificity are parsed in the order in which they appear within the file.
The user-specific configuration file takes precedence over the master file.
</p>
<p>
Because this precedence rule does not match the file order, we recommend
that the file follows the same order, with <code>common</code> options at the
top, and most-specific commands near the bottom. This way, the order in which
the options are read is the same as the order in which they are applied,
which is more intuitive.
</p>
<p>
The arguments specified on a line of an rc file may include arguments that are
not options, such as the names of build targets, and so on. These, like the
options specified in the same files, have lower precedence than their siblings
on the command line, and are always prepended to the explicit list of non-
option arguments.
</p>
<h5><code>--config</code></h5>
<p>
In addition to setting option defaults, the rc file can be used to group
options and provide a shorthand for common groupings. This is done by adding
a <code>:name</code> suffix to the command. These options are ignored by
default, but will be included when the option
<code>--config=<var>name</var></code> is present, either on the command line
or in a <code>.bazelrc</code> file, recursively, even inside of another
config definition. The options specified by <code>command:name</code> will
only be expanded for applicable commands, in the precedence order described
above.
</p>
<p>
Note that configs can be defined in any <code>.bazelrc</code> file, and that
all lines of the form <code>command:name</code> (for applicable commands)
will be expanded, across the different rc files. In order to avoid name
conflicts, we suggest that configs defined in personal rc files start
with an underscore ('_') to avoid unintentional name sharing.
</p>
<p>
<code>--config=foo</code> expands to the options defined in the rc files
"in-place" so that the options specified for the config have the same
precedence that the <code>--config=foo</code> option had.
</p>
<h5>Example</h5>
<p>
Here's an example <code>~/.bazelrc</code> file:
</p>
<pre>
# Bob's Bazel option defaults
startup --host_jvm_args=-XX:-UseParallelGC
import /home/bobs_project/bazelrc
build --show_timestamps --keep_going --jobs 600
build --color=yes
query --keep_going
# Definition of --config=memcheck
build:memcheck --strip=never --test_timeout=3600
</pre>
<h3 id='startup files'>
Other files governing Bazel's behavior
</h3>
<h4 id='.bazelignore'><code>.bazelignore</code></h4>
<p>
You can specify directories within the workspace
that you want Bazel to ignore, such as related projects
that use other build systems. Place a file called
<code>.bazelignore</code> at the root of the workspace
and add the directories you want Bazel to ignore, one per
line. Entries are relative to the workspace root.
</p>
<h2 id='scripting'>Calling Bazel from scripts</h2>
<p>
Bazel can be called from scripts in order to perform a build, run
tests or query the dependency graph. Bazel has been designed to
enable effective scripting, but this section lists some details to
bear in mind to make your scripts more robust.
</p>
<h3>Choosing the output base</h3>
<p>
The <code class='flag'>--output_base</code> option controls where the Bazel process should
write the outputs of a build to, as well as various working files used
internally by Bazel, one of which is a lock that guards against
concurrent mutation of the output base by multiple Bazel processes.
</p>
<p>
Choosing the correct output base directory for your script depends
on several factors. If you need to put the build outputs in a
specific location, this will dictate the output base you need to
use. If you are making a "read only" call to Bazel
(e.g. <code>bazel query</code>), the locking factors will be more important.
In particular, if you need to run multiple instances of your script
concurrently, you will need to give each one a different (or random) output
base.
</p>
<p>
If you use the default output base value, you will be contending for
the same lock used by the user's interactive Bazel commands. If the
user issues long-running commands such as builds, your script will
have to wait for those commands to complete before it can continue.
</p>
<h3>Notes about Server Mode</h3>
<p>
By default, Bazel uses a long-running <a
href='#client/server'>server process</a> as an optimization. When running Bazel
in a script, don't forget to call <code>shutdown</code> when you're finished
with the server, or, specify <code class='flag'>--max_idle_secs=5</code> so
that idle servers shut themselves down promptly.
</p>
<h3>What exit code will I get?</h3>
<p>
Bazel attempts to differentiate failures due to the source code under
consideration from external errors that prevent Bazel from executing properly.
Bazel execution can result in following exit codes:
</p>
<b>Exit Codes common to all commands:</b>
<ul>
<li><code>0</code> - Success</li>
<li><code>2</code> - Command Line Problem, Bad or Illegal flags or command
combination, or Bad Environment Variables. Your command line must be
modified.</li>
<li><code>8</code> - Build Interrupted but we terminated with an orderly shutdown.</li>
<li><code>32</code> - External Environment Failure not on this machine.</li>
<li><code>33</code> - OOM failure. You need to modify your command line.</li>
<li><code>34</code> - Reserved for Google-internal use.</li>
<li><code>35</code> - Reserved for Google-internal use.</li>
<li><code>36</code> - Local Environmental Issue, suspected permanent.</li>
<li><code>37</code> - Unhandled Exception / Internal Bazel Error.</li>
<li><code>38</code> - Reserved for Google-internal use.</li>
<li><code>40-44</code> - Reserved for errors in Bazel's command line launcher,
<code>bazel.cc</code> that are not command line
related. Typically these are related to bazel server
being unable to launch itself.</li>
</ul>
<b>Return codes for commands <code>bazel build</code>, <code>bazel test</code>.</b>
<ul>
<li><code>1</code> - Build failed.</li>
<li><code>3</code> - Build OK, but some tests failed or timed out.</li>
<li><code>4</code> - Build successful but no tests were found even though
testing was requested.</li>
</ul>
<b>For <code>bazel run</code>:</b>
<ul>
<li><code>1</code> - Build failed.</li>
<li>If the build succeeds but the executed subprocess returns a non-zero exit code it will be the
exit code of the command as well.</li>
</ul>
<b>For <code>bazel query</code>:</b>
<ul>
<li><code>3</code> - Partial success, but the query encountered 1 or more
errors in the input BUILD file set and therefore the
results of the operation are not 100% reliable.
This is likely due to a <code class='flag'>--keep_going</code> option
on the command line.</li>
<li><code>7</code> - Command failure.</li>
</ul>
<p>
Future Bazel versions may add additional exit codes, replacing generic failure
exit code <code>1</code> with a different non-zero value with a particular
meaning. However, all non-zero exit values will always constitute an error.
</p>
<h3>Reading the .bazelrc file</h3>
<p>
By default, Bazel will read the <a
href='#bazelrc'><code>.bazelrc</code> file</a> from the base workspace
directory or the user's home directory. Whether or not this is
desirable is a choice for your script; if your script needs to be
perfectly hermetic (e.g. when doing release builds), you should
disable reading the .bazelrc file by using the option
<code class='flag'>--bazelrc=/dev/null</code>. If you want to perform a build
using the user's preferred settings, the default behavior is better.
</p>
<h3>Command log</h3>
<p>
The Bazel output is also available in a command log file which you can
find with the following command:
</p>
<pre>
% bazel info command_log
</pre>
<p>
The command log file contains the interleaved stdout and stderr streams
of the most recent Bazel command. Note that running <code>bazel info</code>
will overwrite the contents of this file, since it then becomes the most
recent Bazel command. However, the location of the command log file will
not change unless you change the setting of the <code class='flag'>--output_base</code>
or <code class='flag'>--output_user_root</code> options.
</p>
<h3>Parsing output</h3>
<p>
The Bazel output is quite easy to parse for many purposes. Two
options that may be helpful for your script are
<code class='flag'>--noshow_progress</code> which suppresses progress messages,
and <code class='flag'>--show_result <var>n</var></code>, which controls whether
or not "build up-to-date" messages are printed; these messages may
be parsed to discover which targets were successfully built, and the
location of the output files they created. Be sure to specify a
very large value of <i>n</i> if you rely on these messages.
</p>
<h2 id='profiling'>Troubleshooting performance by profiling</h2>
<p>
The first step in analyzing the performance of your build is to profile your build with the
<a href='#flag--profile'><code class='flag'>--profile</code></a> option.
</p>
<p>
The file generated by the <a href='#flag--profile'><code class='flag'>--profile</code></a>
command is a binary file. Once you have generated this binary profile, you can analyze it using
Bazel's <a href='#analyze-profile'><code>analyze-profile</code></a> command. By default, it will
print out summary analysis information for each of the specified profile datafiles. This includes
cumulative statistics for different task types for each build phase and an analysis of the
critical execution path.
</p>
<p>
The first section of the default output describes an overview of the time spent on the different
build phases:
</p>
<pre>
=== PHASE SUMMARY INFORMATION ===
Total launch phase time 6.00 ms 0.01%
Total init phase time 864 ms 1.11%
Total loading phase time 21.841 s 28.05%
Total analysis phase time 5.444 s 6.99%
Total preparation phase time 155 ms 0.20%
Total execution phase time 49.473 s 63.54%
Total finish phase time 83.9 ms 0.11%
Total run time 77.866 s 100.00%
</pre>
<p>
The following sections show the execution time of different tasks happening during a particular
phase:
</p>
<pre>
=== INIT PHASE INFORMATION ===
Total init phase time 864 ms
Total time (across all threads) spent on:
Type Total Count Average
VFS_STAT 2.72% 1 23.5 ms
VFS_READLINK 32.19% 1 278 ms
=== LOADING PHASE INFORMATION ===
Total loading phase time 21.841 s
Total time (across all threads) spent on:
Type Total Count Average
SPAWN 3.26% 154 475 ms
VFS_STAT 10.81% 65416 3.71 ms
[...]
SKYLARK_BUILTIN_FN 13.12% 45138 6.52 ms
=== ANALYSIS PHASE INFORMATION ===
Total analysis phase time 5.444 s
Total time (across all threads) spent on:
Type Total Count Average
SKYFRAME_EVAL 9.35% 1 4.782 s
SKYFUNCTION 89.36% 43332 1.06 ms
=== EXECUTION PHASE INFORMATION ===
Total preparation time 155 ms
Total execution phase time 49.473 s
Total time finalizing build 83.9 ms
Action dependency map creation 0.00 ms
Actual execution time 49.473 s
Total time (across all threads) spent on:
Type Total Count Average
ACTION 2.25% 12229 10.2 ms
[...]
SKYFUNCTION 1.87% 236131 0.44 ms
</pre>
<p>
The last section shows the critical path:
</p>
<pre>
Critical path (32.078 s):
Id Time Percentage Description
1109746 5.171 s 16.12% Building [...]
1109745 164 ms 0.51% Extracting interface [...]
1109744 4.615 s 14.39% Building [...]
[...]
1109639 2.202 s 6.86% Executing genrule [...]
1109637 2.00 ms 0.01% Symlinking [...]
1109636 163 ms 0.51% Executing genrule [...]
4.00 ms 0.01% [3 middleman actions]
</pre>
<p>
You can use the following options to display more detailed information:
</p>
<ul>
<li id='dump-text-format'><a href='#flag--dump'><code>--dump=text</code></a>
<p>
This option prints all recorded tasks in the order they occurred. Nested tasks are indented
relative to the parent. For each task, output includes the following information:
</p>
<pre>
[task type] [task description]
Thread: [thread id] Id: [task id] Parent: [parent task id or 0 for top-level tasks]
Start time: [time elapsed from the profiling session start] Duration: [task duration]
[aggregated statistic for nested tasks, including count and total duration for each nested task]
</pre>
</li>
<li id='dump-raw-format'><a href='#flag--dump'><code>--dump=raw</code></a>
<p>
This option is most useful for automated analysis with scripts. It outputs each task record on
a single line using '|' delimiter between fields. Fields are printed in the following order:
</p>
<ol>
<li>thread id - integer positive number, identifies owner thread for the task</li>
<li>task id - integer positive number, identifies specific task</li>
<li>parent task id for nested tasks or 0 for root tasks</li>
<li>task start time in ns, relative to the start of the profiling session</li>
<li>task duration in ns. Please note that this will include duration of all subtasks.</li>
<li>aggregated statistic for immediate subtasks per type. This will include type name (lower
case), number of subtasks for that type and their cumulative duration. Types are
space-delimited and information for single type is comma-delimited.</li>
<li>task type (upper case)</li>
<li>task description</li>
</ol>
Example:
<pre>
1|1|0|0|0||PHASE|Launch Bazel
1|2|0|6000000|0||PHASE|Initialize command
1|3|0|168963053|278111411||VFS_READLINK|/[...]
1|4|0|571055781|23495512||VFS_STAT|/[...]
1|5|0|869955040|0||PHASE|Load packages
[...]
</pre>
</li>
<li id='dump-html-format'><a href='#flag--html'><code>--html</code></a>
<p>
This option writes a file called <code>&lt;profile-file&gt;.html</code> in the directory of the
profile file. Open it in your browser to see the visualization of the actions in your build.
Note that the file can be quite large and may push the capabilities of your browser &ndash;
please wait for the file to load.
</p>
<p>
In most cases, the HTML output from <a href='#flag--html'><code>--html</code></a> is easier to
read than the <a href='#flag--dump'><code>--dump</code></a> output.
It includes a Gantt chart that displays time on the horizontal axis and
threads of execution along the vertical axis. If you click on the Statistics link in the top
right corner of the page, you will jump to a section that lists summary analysis information
from your build.
</p>
<ul>
<li><a href='#flag--html_details'><code>--html_details</code></a>
<p>
Additionally passing this option will render a more detailed execution chart and additional
tables on the performance of built-in and user-defined Skylark functions. Beware that this
increases the file size and the load on the browser considerably.
</p>
</li>
</ul></li>
</ul>
<p>If Bazel appears to be hung, you can hit <kbd><kbd>ctrl</kbd> + <kbd>\</kbd></kbd> or send
Bazel a <code>SIGQUIT</code> signal (<code>kill -3 $(bazel info server_pid)</code>) to get a
thread dump in the file <code>$(bazel info output_base)/server/jvm.out</code>.
</p>
<p>
Since you may not be able to run <code>bazel info</code> if bazel is hung, the
<code>output_base</code> directory is usually the parent of the <code>bazel-&lt;workspace&gt;</code>
symlink in your workspace directory.
</p>