| --- |
| layout: documentation |
| title: User's guide |
| --- |
| <h1>A user's guide to Bazel</h1> |
| |
| |
| <p> |
| To run Bazel, go to |
| |
| your base <a href="build-ref.html#workspace">workspace</a> directory |
| or any of its subdirectories and type <code>bazel</code>. |
| </p> |
| |
| <pre> |
| % bazel help |
| [Bazel release bazel-<<i>version</i>>] |
| Usage: bazel <command> <options> ... |
| |
| Available commands: |
| <a href='user-manual.html#analyze-profile'>analyze-profile</a> Analyzes build profile data. |
| <a href='user-manual.html#aquery'>aquery</a> Executes a query on the <a href='#analysis-phase'>post-analysis</a> action graph. |
| <a href='#build'>build</a> Builds the specified targets. |
| <a href='user-manual.html#canonicalize'>canonicalize-flags</a> Canonicalize Bazel flags. |
| <a href='user-manual.html#clean'>clean</a> Removes output files and optionally stops the server. |
| |
| <a href='user-manual.html#query'>cquery</a> Executes a <a href='#analysis-phase'>post-analysis</a> dependency graph query. |
| |
| <a href='user-manual.html#dump'>dump</a> Dumps the internal state of the Bazel server process. |
| |
| <a href='user-manual.html#help'>help</a> Prints help for commands, or the index. |
| <a href='user-manual.html#info'>info</a> Displays runtime info about the bazel server. |
| |
| <a href='#fetch'>fetch</a> Fetches all external dependencies of a target. |
| <a href='user-manual.html#mobile-install'>mobile-install</a> Installs apps on mobile devices. |
| |
| <a href='user-manual.html#query'>query</a> Executes a dependency graph query. |
| |
| <a href='user-manual.html#run'>run</a> Runs the specified target. |
| <a href='user-manual.html#shutdown'>shutdown</a> Stops the Bazel server. |
| <a href='user-manual.html#test'>test</a> Builds and runs the specified test targets. |
| <a href='user-manual.html#version'>version</a> Prints version information for Bazel. |
| |
| Getting more help: |
| bazel help <command> |
| Prints help and options for <command>. |
| bazel help <a href='user-manual.html#startup_options'>startup_options</a> |
| Options for the JVM hosting Bazel. |
| bazel help <a href='#target-patterns'>target-syntax</a> |
| Explains the syntax for specifying targets. |
| bazel help info-keys |
| Displays a list of keys used by the info command. |
| |
| </pre> |
| <p> |
| The <code>bazel</code> tool performs many functions, called |
| commands. The most commonly used ones are <code>bazel build</code> |
| and <code>bazel test</code>. You can browse the online help |
| messages using <code>bazel help</code>. |
| </p> |
| |
| |
| <h2 id='build'>Building programs with Bazel</h2> |
| <h3>The <code>build</code> command</h3> |
| |
| <p> |
| Type <code>bazel build</code> followed by the name of the |
| <a href="#target-patterns">target</a> you wish to build. Here's a typical |
| session: |
| </p> |
| <pre> |
| % bazel build //foo |
| ____Loading package: foo |
| ____Loading package: bar |
| ____Loading package: baz |
| ____Loading complete. Analyzing... |
| ____Building 1 target... |
| ____[0 / 3] Executing Genrule //bar:helper_rule |
| ____[1 / 3] Executing Genrule //baz:another_helper_rule |
| ____[2 / 3] Building foo/foo.bin |
| Target //foo:foo up-to-date: |
| bazel-bin/foo/foo.bin |
| bazel-bin/foo/foo |
| ____Elapsed time: 9.905s |
| </pre> |
| <p> |
| Bazel prints the progress messages as it loads all the |
| packages in the transitive closure of dependencies of the requested |
| target, then analyzes them for correctness and to create the build actions, |
| finally executing the compilers and other tools of the build. |
| </p> |
| <p> |
| Bazel prints progress messages during |
| the <a href='#execution-phase'>execution phase</a> of the build, showing the |
| current build step (compiler, linker, etc.) that is being started, |
| and the number completed over the total number of build actions. As the |
| build starts the number of total actions will often increase as Bazel |
| discovers the entire action graph, but the number will usually stabilize |
| within a few seconds. |
| </p> |
| <p> |
| At the end of the build Bazel |
| prints which targets were requested, whether or not they were |
| successfully built, and if so, where the output files can be found. |
| Scripts that run builds can reliably parse this output; see <a |
| href='user-manual.html#flag--show_result'><code class='flag'>--show_result</code></a> for more |
| details. |
| </p> |
| <p> |
| Typing the same command again: |
| </p> |
| <pre> |
| % bazel build //foo |
| ____Loading... |
| ____Found 1 target... |
| ____Building complete. |
| Target //foo:foo up-to-date: |
| bazel-bin/foo/foo.bin |
| bazel-bin/foo/foo |
| ____Elapsed time: 0.280s |
| </pre> |
| <p> |
| we see a "null" build: in this case, there are no packages to |
| re-load, since nothing has changed, and no build steps to execute. |
| (If something had changed in "foo" or some of its dependencies, resulting in the |
| re-execution of some build actions, we would call it an "incremental" build, not a |
| "null" build.) |
| </p> |
| |
| <p> |
| Before you can start a build, you will need a Bazel workspace. This is |
| simply a directory tree that contains all the source files needed to build |
| your application. |
| Bazel allows you to perform a build from a completely read-only volume. |
| </p> |
| <h3 id='target-patterns'>Specifying targets to build</h3> |
| <p> |
| Bazel allows a number of ways to specify the targets to be built. |
| Collectively, these are known as <i>target patterns</i>. This syntax is |
| used in commands like <code>build</code>, <code>test</code>, |
| or <code>query</code>. |
| </p> |
| |
| <p> |
| Whereas <a href="build-ref.html#labels">labels</a> are used |
| to specify individual targets, e.g. for declaring dependencies in |
| BUILD files, Bazel's target patterns are a syntax for specifying |
| multiple targets: they are a generalization of the label syntax |
| for <i>sets</i> of targets, using wildcards. In the simplest case, |
| any valid label is also a valid target pattern, identifying a set of |
| exactly one target. |
| </p> |
| |
| <p> |
| All target patterns starting with <code>//</code> are resolved relative to the current |
| workspace. |
| <table> |
| <tr> |
| <td><code>//foo/bar:wiz</code></td> |
| <td>Just the single target <code>//foo/bar:wiz</code>.</td> |
| </tr> |
| <tr> |
| <td><code>//foo/bar</code></td> |
| <td>Equivalent to <code>//foo/bar:bar</code>.</td> |
| </tr> |
| <tr> |
| <td><code>//foo/bar:all</code></td> |
| <td>All rules in the package <code>foo/bar</code>.</td> |
| </tr> |
| <tr> |
| <td><code>//foo/...</code></td> |
| <td>All rules in all packages beneath the directory <code>foo</code>.</td> |
| </tr> |
| <tr> |
| <td><code>//foo/...:all</code></td> |
| <td>All rules in all packages beneath the directory <code>foo</code>.</td> |
| </tr> |
| <tr> |
| <td><code>//foo/...:*</code></td> |
| <td>All targets (rules and files) in all packages beneath the directory <code>foo</code>.</td> |
| </tr> |
| <tr> |
| <td><code>//foo/...:all-targets</code></td> |
| <td>All targets (rules and files) in all packages beneath the directory <code>foo</code>.</td> |
| </tr> |
| </table> |
| </p> |
| |
| <p> |
| Target patterns which do not begin with <code>//</code> are resolved relative to the |
| current <em>working directory</em>. These examples assume a working directory of |
| <code>foo</code>: |
| <table> |
| <tr> |
| <td><code>:foo</code></td> |
| <td>Equivalent to <code>//foo:foo</code>.</td> |
| </tr> |
| <tr> |
| <td><code>bar:wiz</code></td> |
| <td>Equivalent to <code>//foo/bar:wiz</code>.</td> |
| </tr> |
| <tr> |
| <td><code>bar/wiz</code></td> |
| <td>Equivalent to: |
| <code>//foo/bar/wiz:wiz</code> if <code>foo/bar/wiz</code> is a package, |
| <code>//foo/bar:wiz</code> if <code>foo/bar</code> is a package, |
| <code>//foo:bar/wiz</code> otherwise. |
| </td> |
| </tr> |
| <tr> |
| <td><code>bar:all</code></td> |
| <td>Equivalent to <code>//foo/bar:all</code>.</td> |
| </tr> |
| <tr> |
| <td><code>:all</code></td> |
| <td>Equivalent to <code>//foo:all</code>.</td> |
| </tr> |
| <tr> |
| <td><code>...:all</code></td> |
| <td>Equivalent to <code>//foo/...:all</code>.</td> |
| </tr> |
| <tr> |
| <td><code>...</code></td> |
| <td>Equivalent to <code>//foo/...:all</code>.</td> |
| </tr> |
| <tr> |
| <td><code>bar/...:all</code></td> |
| <td>Equivalent to <code>//foo/bar/...:all</code>.</td> |
| </tr> |
| </table> |
| </p> |
| |
| <p> |
| By default, directory symlinks are followed for recursive target patterns, |
| except those that point to under the output base, such as the convenience |
| symlinks that are created in the root directory of the workspace. |
| </p> |
| |
| <p> |
| In addition, Bazel does not follow symlinks when evaluating recursive target |
| patterns in any directory that contains a file named as follows: |
| <pre> |
| DONT_FOLLOW_SYMLINKS_WHEN_TRAVERSING_THIS_DIRECTORY_VIA_A_RECURSIVE_TARGET_PATTERN |
| </pre> |
| </p> |
| |
| <p> |
| <code>foo/...</code> is a wildcard over <em>packages</em>, |
| indicating all packages recursively beneath |
| directory <code>foo</code> (for all roots of the package |
| path). <code>:all</code> is a wildcard |
| over <em>targets</em>, matching all rules within a package. These two may be |
| combined, as in <code>foo/...:all</code>, and when both wildcards |
| are used, this may be abbreviated to <code>foo/...</code>. |
| </p> |
| <p> |
| In addition, <code>:*</code> (or <code>:all-targets</code>) is a |
| wildcard that matches <em>every target</em> in the matched packages, |
| including files that aren't normally built by any rule, such |
| as <code>_deploy.jar</code> files associated |
| with <code>java_binary</code> rules. |
| </p> |
| <p> |
| This implies that <code>:*</code> denotes a <em>superset</em> |
| of <code>:all</code>; while potentially confusing, this syntax does |
| allow the familiar <code>:all</code> wildcard to be used for |
| typical builds, where building targets like the <code>_deploy.jar</code> |
| is not desired. |
| </p> |
| <p> |
| In addition, Bazel allows a slash to be used instead of the colon |
| required by the label syntax; this is often convenient when using |
| Bash filename expansion. For example, <code>foo/bar/wiz</code> is |
| equivalent to <code>//foo/bar:wiz</code> (if there is a |
| package <code>foo/bar</code>) or to <code>//foo:bar/wiz</code> (if |
| there is a package <code>foo</code>). |
| </p> |
| <p> |
| Many Bazel commands accept a list of target patterns as arguments, |
| and they all honor the prefix negation operator <code>-</code>. |
| This can be used to subtract a set of targets from the set specified |
| by the preceding arguments. Note that this means order matters. |
| For example, |
| </p> |
| <pre> |
| bazel build foo/... bar/... |
| </pre> |
| <p> |
| means "build all |
| targets beneath <code>foo</code> <i>and</i> all targets |
| beneath <code>bar</code>", whereas |
| </p> |
| <pre> |
| bazel build -- foo/... -foo/bar/... |
| </pre> |
| <p> |
| means "build all targets beneath <code>foo</code> <i>except</i> |
| those beneath <code>foo/bar</code>". |
| |
| (The <code>--</code> argument is required to prevent the subsequent |
| arguments starting with <code>-</code> from being interpreted as |
| additional options.) |
| </p> |
| <p> |
| It's important to point out though that subtracting targets this way will not |
| guarantee that they are not built, since they may be dependencies of targets |
| that weren't subtracted. For example, if there were a target |
| <code>//foo:all-apis</code> that among others depended on |
| <code>//foo/bar:api</code>, then the latter would be built as part of |
| building the former. |
| </p> |
| <p> |
| Targets with <code>tags = ["manual"]</code> will not be included in wildcard target patterns ( |
| <code>...</code>, <code>:*</code>, <code>:all</code>, etc.). |
| You should specify such test targets with explicit target patterns on the command |
| line if you want Bazel to build/test them. |
| </p> |
| |
| <h3 id='fetch'>Fetching external dependencies</h3> |
| |
| <p> |
| By default, Bazel will download and symlink external dependencies during the |
| build. However, this can be undesirable, either because you'd like to know |
| when new external dependendencies are added or because you'd like to |
| "prefetch" dependencies (say, before a flight where you'll be offline). If you |
| would like to prevent new dependencies from being added during builds, you |
| can specify the <code>--fetch=false</code> flag. Note that this flag only |
| applies to repository rules that do not point to a directory in the local |
| file system. Changes, for example, to <code>local_repository</code>, |
| <code>new_local_repository</code> and Android SDK and NDK repository rules |
| will always take effect regardless of the value <code>--fetch</code> . |
| </p> |
| |
| <p> |
| If you disallow fetching during builds and Bazel finds new external |
| dependencies, your build will fail. |
| </p> |
| |
| <p> |
| You can manually fetch dependencies by running <code>bazel fetch</code>. If |
| you disallow during-build fetching, you'll need to run <code>bazel |
| fetch</code>: |
| <ol> |
| <li>Before you build for the first time. |
| <li>After you add a new external dependency. |
| </ol> |
| Once it has been run, you should not need to run it again until the WORKSPACE |
| file changes. |
| </p> |
| |
| <p> |
| <code>fetch</code> takes a list of targets to fetch dependencies for. For |
| example, this would fetch dependencies needed to build <code>//foo:bar</code> |
| and <code>//bar:baz</code>: |
| <pre> |
| $ bazel fetch //foo:bar //bar:baz |
| </pre> |
| </p> |
| |
| <p> |
| To fetch all external dependencies for a workspace, run: |
| <pre> |
| $ bazel fetch //... |
| </pre> |
| </p> |
| |
| <p> |
| You do not need to run bazel fetch at all if you have all of the tools you are |
| using (from library jars to the JDK itself) under your workspace root. |
| However, if you're using anything outside of the workspace directory then Bazel |
| will automatically run <code>bazel fetch</code> before running |
| <code>bazel build</code>. |
| </p> |
| |
| <h4 id='repository-cache'>The repository cache</h4> |
| |
| <p> |
| Bazel tries to avoid fetching the same file several times, even if the same |
| file is needed in different workspaces, or if the definition of an external |
| repository changed but it still needs the same file to download. To do so, |
| bazel caches all files downloaded in the repository cache which, by default, |
| is located at <code>~/.cache/bazel/_bazel_$USER/cache/repos/v1/</code>. The |
| location can be changed by the <code>--repository_cache</code> option. The |
| cache is shared between all workspaces and installed versions of bazel. |
| An entry is taken from the cache if |
| Bazel knows for sure that it has a copy of the correct file, that is, if the |
| download request has a SHA256 sum of the file specified and a file with that |
| hash is in the cache. So specifying a hash for each external file is |
| not only a good idea from a security perspective; it also helps avoiding |
| unnecessary downloads. |
| </p> |
| |
| <p> |
| Upon each cache hit, the modification time of the file in the cache is |
| updated. In this way, the last use of a file in the cache directory can easily |
| be determined, for example to manually clean up the cache. The cache is never |
| cleaned up automatically, as it might contain a copy of a file that is no |
| longer available upstream. |
| </p> |
| |
| <h4 id='distdir'>Distribution files directories</h4> |
| |
| <p> |
| Besides the repository cache, Bazel has a second mechanism to avoid |
| unnecessary downloads. Using the <code>--distdir</code> option, you can |
| specify additional read-only directories to look for files instead of fetching |
| them. A file is taken from such a directory if the file name is equal to |
| the base name of the URL and additionally the hash of the file is equal to the |
| one specified in the download request (again, no file is taken from a distdir, |
| if no hash is specified). While the condition on the file name is not |
| necessary for correctness, it reduces the number of candidate files to one per |
| specified directory. In this way, specifying distribution files directories |
| remains efficient, even if the number of files in such a directory grows |
| large. |
| </p> |
| |
| <h3 id='configurations'>Build configurations and cross-compilation</h3> |
| |
| <p> |
| All the inputs that specify the behavior and result of a given |
| build can be divided into two distinct categories. |
| The first kind is the intrinsic information stored in the BUILD |
| files of your project: the build rule, the values of its attributes, |
| and the complete set of its transitive dependencies. |
| The second kind is the external or environmental data, supplied by |
| the user or by the build tool: the choice of target architecture, |
| compilation and linking options, and other toolchain configuration |
| options. We refer to a complete set of environmental data as |
| a <b>configuration</b>. |
| </p> |
| <p> |
| In any given build, there may be more than one configuration. |
| Consider a cross-compile, in which you build |
| a <code>//foo:bin</code> executable for a 64-bit architecture, |
| but your workstation is a 32-bit machine. Clearly, the build |
| will require building <code>//foo:bin</code> using a toolchain |
| capable of creating 64-bit executables, but the build system must |
| also build various tools used during the build itself—for example |
| tools that are built from source, then subsequently used in, say, a |
| genrule—and these must be built to run on your workstation. |
| Thus we can identify two configurations: the <b>host |
| configuration</b>, which is used for building tools that run during |
| the build, and the <b>target configuration</b> (or <i>request |
| configuration</i>, but we say "target configuration" more often even |
| though that word already has many meanings), which is |
| used for building the binary you ultimately requested. |
| </p> |
| <p> |
| Typically, there are many libraries that are prerequisites of both |
| the requested build target (<code>//foo:bin</code>) and one or more of |
| the host tools, for example some base libraries. Such libraries must be built |
| twice, once for the host configuration, and once for the target |
| configuration.<br/> |
| Bazel takes care of ensuring that both variants are built, and that |
| the derived files are kept separate to avoid interference; usually |
| such targets can be built concurrently, since they are independent |
| of each other. If you see progress messages indicating that a given |
| target is being built twice, this is most likely the explanation. |
| </p> |
| <p> |
| Bazel uses one of two ways to select the host configuration, based |
| on the <code class='flag'>--distinct_host_configuration</code> option. This |
| boolean option is somewhat subtle, and the setting may improve (or |
| worsen) the speed of your builds. |
| </p> |
| |
| <h4><code class='flag'>--distinct_host_configuration=false</code></h4> |
| <p> |
| We do not recommend this option. |
| </p> |
| <ul> |
| <li>Building google3 Java code is only supported with distinct host |
| configurations. |
| </li> |
| <li>If you frequently make changes to your request configuration, such |
| as alternating between <code>-c opt</code> and <code>-c dbg</code> |
| builds, or between simple- and cross-compilation, you will typically |
| rebuild the majority of your codebase each time you switch. |
| </li> |
| </ul> |
| <p> |
| When this option is false, the host and |
| request configurations are identical: all tools required during the |
| build will be built in exactly the same way as target programs. |
| This setting means that no libraries need to be built twice during a |
| single build. |
| However, it does mean that any change to your request configuration |
| also affects your host configuration, causing all the tools to be |
| rebuilt, and then anything that depends on the tool output to be |
| rebuilt too. Thus, for example, simply changing a linker option |
| between builds might cause all tools to be re-linked, and then all |
| actions using them re-executed, and so on, resulting in a very large rebuild. |
| Also, please note: if your host architecture is not capable of |
| running your target binaries, your build will not work. |
| </p> |
| |
| <h4><code class='flag'>--distinct_host_configuration=true</code> <i>(default)</i></h4> |
| <p> |
| If this option is true, then instead of using the same configuration |
| for the host and request, a completely distinct host configuration |
| is used. The host configuration is derived from the target |
| configuration as follows: |
| </p> |
| <ul> |
| <li>Use the same version of Crosstool |
| (<code class='flag'>--crosstool_top</code>) as specified in the request |
| configuration, unless <code class='flag'>--host_crosstool_top</code> is |
| specified. |
| </li> |
| <li> |
| Use the value of <code class="flag">--host_cpu</code> for |
| <code class='flag'>--cpu</code> |
| |
| (default: <code>k8</code>). |
| </li> |
| <li>Use the same values of these options as specified in the request |
| configuration: |
| <code class='flag'>--compiler</code>, |
| <code class='flag'>--use_ijars</code>, |
| If <code class='flag'>--host_crosstool_top</code> is used, then the value of |
| <code class='flag'>--host_cpu</code> is used to look up a |
| <code>default_toolchain</code> in the Crosstool |
| (ignoring <code class='flag'>--compiler</code>) for the host configuration. |
| </li> |
| <li> |
| Use the value of <code class="flag">--host_javabase</code> for |
| <code class='flag'>--javabase</code> |
| </li> |
| <li> |
| Use the value of <code class="flag">--host_java_toolchain</code> for |
| <code class='flag'>--java_toolchain</code> |
| </li> |
| <li>Use optimized builds for C++ code (<code>-c opt</code>). |
| </li> |
| <li>Generate no debugging information (<code class='flag'>--copt=-g0</code>). |
| </li> |
| <li>Strip debug information from executables and shared libraries |
| (<code class='flag'>--strip=always</code>). |
| </li> |
| <li>Place all derived files in a special location, distinct from |
| that used by any possible request configuration. |
| </li> |
| <li>Suppress stamping of binaries with build data |
| (see <code class='flag'>--embed_*</code> options). |
| </li> |
| <li>All other values remain at their defaults. |
| </li> |
| </ul> |
| <p> |
| There are many reasons why it might be preferable to select a |
| distinct host configuration from the request configuration. |
| Some are too esoteric to mention here, but two of them are worth |
| pointing out. |
| </p> |
| <p> |
| Firstly, by using stripped, optimized binaries, you reduce the time |
| spent linking and executing the tools, the disk space occupied by |
| the tools, and the network I/O time in distributed builds. |
| </p> |
| <p> |
| Secondly, by decoupling the host and request configurations in all |
| builds, you avoid very expensive rebuilds that would result from |
| minor changes to the request configuration (such as changing a linker options |
| does), as described earlier. |
| </p> |
| <p> |
| That said, for certain builds, this option may be a hindrance. In |
| particular, builds in which changes of configuration are infrequent |
| (especially certain Java builds), and builds where the amount of code that |
| must be built in both host and target configurations is large, may |
| not benefit. |
| </p> |
| |
| <h3 id='correctness'>Correct incremental rebuilds</h3> |
| |
| <p> |
| One of the primary goals of the Bazel project is to ensure correct |
| incremental rebuilds. Previous build tools, especially those based |
| on Make, make several unsound assumptions in their implementation of |
| incremental builds. |
| </p> |
| <p> |
| Firstly, that timestamps of files increase monotonically. While |
| this is the typical case, it is very easy to fall afoul of this |
| assumption; syncing to an earlier revision of a file causes that file's |
| modification time to decrease; Make-based systems will not rebuild. |
| </p> |
| <p> |
| More generally, while Make detects changes to files, it does |
| not detect changes to commands. If you alter the options passed to |
| the compiler in a given build step, Make will not re-run the |
| compiler, and it is necessary to manually discard the invalid |
| outputs of the previous build using <code>make clean</code>. |
| </p> |
| <p> |
| Also, Make is not robust against the unsuccessful termination of one |
| of its subprocesses after that subprocess has started writing to |
| its output file. While the current execution of Make will fail, the |
| subsequent invocation of Make will blindly assume that the truncated |
| output file is valid (because it is newer than its inputs), and it |
| will not be rebuilt. Similarly, if the Make process is killed, a |
| similar situation can occur. |
| </p> |
| <p> |
| Bazel avoids these assumptions, and others. Bazel maintains a database |
| of all work previously done, and will only omit a build step if it |
| finds that the set of input files (and their timestamps) to that |
| build step, and the compilation command for that build step, exactly |
| match one in the database, and, that the set of output files (and |
| their timestamps) for the database entry exactly match the |
| timestamps of the files on disk. Any change to the input files or |
| output files, or to the command itself, will cause re-execution of |
| the build step. |
| </p> |
| <p> |
| The benefit to users of correct incremental builds is: less time |
| wasted due to confusion. (Also, less time spent waiting for |
| rebuilds caused by use of <code>make clean</code>, whether necessary |
| or pre-emptive.) |
| </p> |
| |
| <h4>Build consistency and incremental builds</h4> |
| <p> |
| Formally, we define the state of a build as <i>consistent</i> when |
| all the expected output files exist, and their contents are correct, |
| as specified by the steps or rules required to create them. When |
| you edit a source file, the state of the build is said to |
| be <i>inconsistent</i>, and remains inconsistent until you next run |
| the build tool to successful completion. We describe this situation |
| as <i>unstable inconsistency</i>, because it is only temporary, and |
| consistency is restored by running the build tool. |
| </p> |
| <p> |
| There is another kind of inconsistency that is pernicious: <i>stable |
| inconsistency</i>. If the build reaches a stable inconsistent |
| state, then repeated successful invocation of the build tool does |
| not restore consistency: the build has gotten "stuck", and the |
| outputs remain incorrect. Stable inconsistent states are the main |
| reason why users of Make (and other build tools) type <code>make |
| clean</code>. Discovering that the build tool has failed in this |
| manner (and then recovering from it) can be time consuming and very |
| frustrating. |
| </p> |
| <p> |
| Conceptually, the simplest way to achieve a consistent build is to |
| throw away all the previous build outputs and start again: make |
| every build a clean build. This approach is obviously too |
| time-consuming to be practical (except perhaps for release |
| engineers), and therefore to be useful, the build tool must be able |
| to perform incremental builds without compromising consistency. |
| </p> |
| <p> |
| Correct incremental dependency analysis is hard, and as described |
| above, many other build tools do a poor job of avoiding stable |
| inconsistent states during incremental builds. In contrast, Bazel |
| offers the following guarantee: after a successful invocation of the |
| build tool during which you made no edits, the build will be in a |
| consistent state. (If you edit your source files during a build, |
| Bazel makes no guarantee about the consistency of the result of the |
| current build. But it does guarantee that the results of |
| the <i>next</i> build will restore consistency.) |
| </p> |
| <p> |
| As with all guarantees, there comes some fine print: there are some |
| known ways of getting into a stable inconsistent state with Bazel. |
| We won't guarantee to investigate such problems arising from deliberate |
| attempts to find bugs in the incremental dependency analysis, but we |
| will investigate and do our best to fix all stable inconsistent |
| states arising from normal or "reasonable" use of the build tool. |
| </p> |
| <p> |
| If you ever detect a stable inconsistent state with Bazel, please report a bug. |
| |
| </p> |
| |
| <h4 id='sandboxing'>Sandboxed execution</h4> |
| <p> |
| Bazel uses sandboxes to guarantee that actions run hermetically<sup>1</sup> and correctly. |
| Bazel runs <i>Spawn</i>s (loosely speaking: actions) in sandboxes that only contain the minimal |
| set of files the tool requires to do its job. Currently sandboxing works on Linux 3.12 or newer |
| with the <code>CONFIG_USER_NS</code> option enabled, and also on macOS 10.11 or newer. |
| </p> |
| <p> |
| Bazel will print a warning if your system does not support sandboxing to alert you to the fact |
| that builds are not guaranteed to be hermetic and might affect the host system in unknown ways. |
| To disable this warning you can pass the <code>--ignore_unsupported_sandboxing</code> flag to |
| Bazel. |
| </p> |
| |
| <p> |
| On some platforms such as <a href="https://cloud.google.com/kubernetes-engine/">Google Kubernetes |
| Engine</a> cluster nodes or Debian, user namespaces are deactivated by default due to security |
| concerns. This can be checked by looking at the file |
| <code>/proc/sys/kernel/unprivileged_userns_clone</code>: if it exists and contains a 0, then |
| user namespaces can be activated with <code>sudo sysctl kernel.unprivileged_userns_clone=1</code>. |
| </p> |
| <p> |
| In some cases, the Bazel sandbox fails to execute rules because of the system setup. The symptom |
| is generally a failure that output a message similar to |
| <code>namespace-sandbox.c:633: execvp(argv[0], argv): No such file or directory</code>. In that |
| case, try to deactivate the sandbox for genrules with <code>--strategy=Genrule=standalone</code> |
| and for other rules with <code>--spawn_strategy=standalone</code>. Also please report a bug on our |
| issue tracker and mention which Linux distribution you're using so that we can investigate and |
| provide a fix in a subsequent release. |
| </p> |
| |
| <p> |
| <sup>1</sup>: Hermeticity means that the action only uses its declared input files and no other |
| files in the filesystem, and it only produces its declared output files. |
| </p> |
| |
| |
| <h3 id='phases'>Phases of a build</h3> |
| |
| <p> |
| In Bazel, a build occurs in three distinct phases; as a user, |
| understanding the difference between them provides insight into the |
| options which control a build (see below). |
| </p> |
| |
| <h4 id='loading-phase'>Loading phase</h4> |
| <p> |
| The first is <b>loading</b> during which all the necessary BUILD |
| files for the initial targets, and their transitive closure of |
| dependencies, are loaded, parsed, evaluated and cached. |
| </p> |
| <p> |
| For the first build after a Bazel server is started, the loading |
| phase typically takes many seconds as many BUILD files are loaded |
| from the file system. In subsequent builds, especially if no BUILD |
| files have changed, loading occurs very quickly. |
| </p> |
| <p> |
| Errors reported during this phase include: package not found, target |
| not found, lexical and grammatical errors in a BUILD file, |
| and evaluation errors. |
| </p> |
| |
| <h4 id='analysis-phase'>Analysis phase</h4> |
| <p> |
| The second phase, <b>analysis</b>, involves the semantic analysis |
| and validation of each build rule, the construction of a build |
| dependency graph, and the determination of exactly what work is to |
| be done in each step of the build. |
| </p> |
| <p> |
| Like loading, analysis also takes several seconds when computed in |
| its entirety. However, Bazel caches the dependency graph from |
| one build to the next and only reanalyzes what it has to, which can |
| make incremental builds extremely fast in the case where the |
| packages haven't changed since the previous build. |
| </p> |
| <p> |
| Errors reported at this stage include: inappropriate dependencies, |
| invalid inputs to a rule, and all rule-specific error messages. |
| </p> |
| <p> |
| The loading and analysis phases are fast because |
| Bazel avoids unnecessary file I/O at this stage, reading only BUILD |
| files in order to determine the work to be done. This is by design, |
| and makes Bazel a good foundation for analysis tools, such as |
| Bazel's <a href='#query'>query</a> command, which is implemented |
| atop the loading phase. |
| </p> |
| |
| <h4 id='execution-phase'>Execution phase</h4> |
| <p> |
| The third and final phase of the build is <b>execution</b>. This |
| phase ensures that the outputs of each step in the build are |
| consistent with its inputs, re-running compilation/linking/etc. tools as |
| necessary. This step is where the build spends the majority of |
| its time, ranging from a few seconds to over an hour for a large |
| build. Errors reported during this phase include: missing source |
| files, errors in a tool executed by some build action, or failure of a tool to |
| produce the expected set of outputs. |
| </p> |
| |
| |
| |
| <h2 id='client/server'>Client/server implementation</h2> |
| |
| <p> |
| The Bazel system is implemented as a long-lived server process. |
| This allows it to perform many optimizations not possible with a |
| batch-oriented implementation, such as caching of BUILD files, |
| dependency graphs, and other metadata from one build to the |
| next. This improves the speed of incremental builds, and allows |
| different commands, such as <code>build</code> |
| and <code>query</code> to share the same cache of loaded packages, |
| making queries very fast. |
| </p> |
| <p> |
| When you run <code>bazel</code>, you're running the client. The |
| client finds the server based on the output base, which by default is |
| determined by the path of the base workspace directory and your |
| userid, so if you build in multiple workspaces, you'll have multiple |
| output bases and thus multiple Bazel server processes. Multiple |
| users on the same workstation can build concurrently in the same |
| workspace because their output bases will differ (different userids). |
| If the client cannot find a running server instance, it starts a new |
| one. The server process will stop after a period of inactivity (3 hours, |
| by default, which can be modified using the startup option <code>--max_idle_secs</code>). |
| </p> |
| <p> |
| For the most part, the fact that there is a server running is |
| invisible to the user, but sometimes it helps to bear this in mind. |
| For example, if you're running scripts that perform a lot of |
| automated builds in different directories, it's important to ensure |
| that you don't accumulate a lot of idle servers; you can do this by |
| explicitly shutting them down when you're finished with them, or by |
| specifying a short timeout period. |
| </p> |
| <p> |
| The name of a Bazel server process appears in the output of <code>ps |
| x</code> or <code>ps -e f</code> as |
| <code>bazel(<i>dirname</i>)</code>, where <i>dirname</i> is the |
| basename of the directory enclosing the root of your workspace directory. |
| For example: |
| </p> |
| <pre> |
| % ps -e f |
| 16143 ? Sl 3:00 bazel(src-johndoe2) -server -Djava.library.path=... |
| </pre> |
| <p> |
| This makes it easier to find out which server process belongs to a |
| given workspace. (Beware that with certain other options |
| to <code>ps</code>, Bazel server processes may be named just |
| <code>java</code>.) Bazel servers can be stopped using |
| the <a href='#shutdown'>shutdown</a> command. |
| </p> |
| |
| <p> |
| When running <code>bazel</code>, the client first checks that the |
| server is the appropriate version; if not, the server is stopped and |
| a new one started. This ensures that the use of a long-running |
| server process doesn't interfere with proper versioning. |
| </p> |
| |
| <h2 id='bazelrc'> |
| <code>.bazelrc</code>, the Bazel configuration file, |
| the <code class='flag'>--bazelrc=<var>file</var></code> option, and |
| the <code class='flag'>--config=<var>value</var></code> option |
| </h2> |
| |
| <p> |
| Bazel accepts many options. Typically, some of these are varied |
| frequently (for example, <code class='flag'>--subcommands</code>) while others stay the |
| same across several builds (e.g. <code class='flag'>--package_path</code>). |
| To avoid having to specify these unchanged options for every build (and other commands) |
| Bazel allows you to specify options in a configuration file. |
| </p> |
| |
| |
| <h4>Where are the <code>.bazelrc</code> files?</h4> |
| <p> |
| Bazel looks for optional configuration files in the following locations, |
| in the order shown below. The options are interpreted in this order, so |
| options in later files can override a value from an earlier file if a |
| conflict arises. All options that control which of these files are loaded are |
| startup options, which means they must occur after <code>bazel</code> and |
| before the command (<code>build</code>, <code>test</code>, etc). |
| </p> |
| <ol> |
| <li> |
| <p><b>The system RC file</b>, unless <code class='flag'>--nosystem_rc</code> is present.</p> |
| |
| <p>Path:</p> |
| <ul> |
| <li>On Linux/macOS/Unixes: <code>/etc/bazel.bazelrc</code></li> |
| <li>On Windows: <code>%ProgramData%\bazel.bazelrc</code></li> |
| </ul> |
| |
| <p>It is not an error if this file does not exist.</p> |
| |
| <p>If another system-specified location is required, you must build a custom Bazel binary, |
| overriding the <code>BAZEL_SYSTEM_BAZELRC_PATH</code> value in |
| <a href="https://github.com/bazelbuild/bazel/blob/0.28.0/src/main/cpp/BUILD#L141"><code>//src/main/cpp:option_processor</code></a>. |
| The system-specified location may contain environment variable references, such as |
| <code>${VAR_NAME}</code> on Unix or <code>%VAR_NAME%</code> on Windows.</p> |
| </li> |
| <li> |
| <p><b>The workspace RC file</b>, unless <code class='flag'>--noworkspace_rc</code> is |
| present.</p> |
| |
| <p>Path: <code>.bazelrc</code> in your workspace directory (next to the main |
| <code>WORKSPACE</code> file).</p> |
| |
| <p>It is not an error if this file does not exist.</p> |
| </li> |
| <li> |
| <p><b>The home RC file</b>, unless <code class='flag'>--nohome_rc</code> is present.</p> |
| |
| <p>Path:</p> |
| <ul> |
| <li>On Linux/macOS/Unixes: <code>$HOME/.bazelrc</code></li> |
| <li>On Windows: <code>%USERPROFILE%\.bazelrc</code> if exists, or |
| <code>%HOME%/.bazelrc</code></li> |
| </ul> |
| |
| <p>It is not an error if this file does not exist.</p> |
| </li> |
| <li> |
| <p><b>The user-specified RC file</b>, if specified with |
| <code class='flag'>-bazelrc=<var>file</var></code><p> |
| |
| <p>This flag is optional. However, if the flag is specified, then the file must exist.</p> |
| </li> |
| </ol> |
| <h4><code>.bazelrc</code> syntax and semantics</h4> |
| <p> |
| Like all UNIX "rc" files, the <code>.bazelrc</code> file is a text file with |
| a line-based grammar. Empty lines and lines starting with <code>#</code> (comments) |
| are ignored. Each line contains a sequence of words, |
| which are tokenized according to the same rules as the Bourne shell. |
| </p> |
| |
| <h5>Imports</h5> |
| <p> |
| Lines that start with <code>import</code> or <code>try-import</code> are special: use these to |
| load other "rc" files. To specify a path that is relative to the workspace root, write |
| <code>import %workspace%/path/to/bazelrc</code>. |
| </p> |
| <p> |
| The difference between <code>import</code> and <code>try-import</code> is that Bazel fails if |
| the <code>import</code>'ed file is missing (or can't be read), but not so for a |
| <code>try-import</code>'ed file. |
| </p> |
| <p>Import precedence:</p> |
| <ul> |
| <li>Options in the imported file take precedence over options specified before |
| the import statement.</li> |
| <li>Options specified after the import statement take precedence over the |
| options in the imported file.</li> |
| <li>Options in files imported later take precedence over files |
| imported earlier.</li> |
| </ul> |
| |
| <h5>Option defaults</h5> |
| <p> |
| Most lines of a bazelrc define default option values. The first word on each |
| line specifies when these defaults are applied: |
| </p> |
| <ul> |
| <li> |
| <code>startup</code>: startup options, which go before the command, and |
| are described in <code>bazel help startup_options</code>. |
| </li> |
| <li> |
| <code>common</code>: options that apply to all Bazel commands. |
| </li> |
| <li> |
| <code><i>command</i></code>: Bazel command, such as <code>build</code> |
| or <code>query</code> to which the options apply. These options also apply |
| to all commands that inherit from the specified command. (For example, |
| <code>test</code> inherits from <code>build</code>.) |
| </li> |
| </ul> |
| <p> |
| Each of these lines may be used more than once and the arguments that follow |
| the first word are combined as if they had appeared on a single line. |
| (Users of CVS, another tool with a "Swiss army knife" command-line interface, |
| will find the syntax similar to that of <code>.cvsrc</code>.) For example, the |
| lines: |
| |
| <pre>build --test_tmpdir=/tmp/foo --verbose_failures |
| build --test_tmpdir=/tmp/bar</pre> |
| |
| are combined as: |
| |
| <pre>build --test_tmpdir=/tmp/foo --verbose_failures --test_tmpdir=/tmp/bar</pre> |
| |
| so the effective flags are <code>--verbose_failures</code> and |
| <code>--test_tmpdir=/tmp/bar</code>. |
| </p> |
| |
| <p>Option precedence:</p> |
| <ul> |
| <li> |
| Options on the command line always take precedence over those in rc files. For example, if a rc |
| file says <code>build -c opt</code> but the command line flag is <code>-c dbg</code>, the |
| command line flag takes precedence. |
| </li> |
| <li> |
| <p> |
| Within the rc file, precedence is governed by specificity: lines for a more specific command |
| take precedence over lines for a less specific command. |
| </p> |
| <p> |
| Specificity is defined by inheritance. Some commands inherit options from other commands, |
| making the inheriting command more specific than the base command. For example |
| <code>test</code> inherits from the <code>build</code> command, so all |
| <code>bazel build</code> flags are valid for <code>bazel test</code>, and all |
| <code>build</code> lines apply also to <code>bazel test</code> unless there's a |
| <code>test</code> line for the same option. If the rc file says: |
| |
| <pre>test -c dbg --test_env=PATH |
| build -c opt --verbose_failures</pre> |
| |
| then <code>bazel build //foo</code> will use <code>-c opt --verbose_failures</code>, and |
| <code>bazel test //foo</code> will use <code>--verbose_failures -c dbg --test_env=PATH</code>. |
| </p> |
| <p> |
| The inheritance (specificity) graph is: |
| </p> |
| <ul> |
| <li>Every command inherits from <code>common</code></li> |
| <li>The following commands inherit from (and are more specific than) <code>build</code>: |
| <code>test</code>, <code>run</code>, <code>clean</code>, <code>mobile-install</code>, |
| <code>info</code>, <code>print_action</code>, <code>config</code>, <code>cquery</code>, and |
| <code>aquery</code> |
| </li> |
| <li><code>coverage</code> inherits from <code>test</code></li> |
| </ul> |
| </li> |
| <li> |
| Two lines specifying options for the same command at equal specificity are parsed in the |
| order in which they appear within the file. |
| </li> |
| <li> |
| Because this precedence rule does not match the file order, it helps readability if you follow |
| the precedence order within rc files: start with <code>common</code> options at the top, and |
| end with the most-specific commands at the bottom of the file. This way, the order in which |
| the options are read is the same as the order in which they are applied, which is more |
| intuitive. |
| </li> |
| </ul> |
| |
| <p> |
| The arguments specified on a line of an rc file may include arguments that are |
| not options, such as the names of build targets, and so on. These, like the |
| options specified in the same files, have lower precedence than their siblings |
| on the command line, and are always prepended to the explicit list of non- |
| option arguments. |
| </p> |
| |
| <h5><code>--config</code></h5> |
| <p> |
| In addition to setting option defaults, the rc file can be used to group |
| options and provide a shorthand for common groupings. This is done by adding |
| a <code>:name</code> suffix to the command. These options are ignored by |
| default, but will be included when the option |
| <code>--config=<var>name</var></code> is present, either on the command line |
| or in a <code>.bazelrc</code> file, recursively, even inside of another |
| config definition. The options specified by <code>command:name</code> will |
| only be expanded for applicable commands, in the precedence order described |
| above. |
| </p> |
| <p> |
| Note that configs can be defined in any <code>.bazelrc</code> file, and that |
| all lines of the form <code>command:name</code> (for applicable commands) |
| will be expanded, across the different rc files. In order to avoid name |
| conflicts, we suggest that configs defined in personal rc files start |
| with an underscore ('_') to avoid unintentional name sharing. |
| </p> |
| <p> |
| <code>--config=foo</code> expands to the options defined in the rc files |
| "in-place" so that the options specified for the config have the same |
| precedence that the <code>--config=foo</code> option had. |
| </p> |
| |
| <h5>Example</h5> |
| <p> |
| Here's an example <code>~/.bazelrc</code> file: |
| </p> |
| <pre> |
| # Bob's Bazel option defaults |
| |
| startup --host_jvm_args=-XX:-UseParallelGC |
| import /home/bobs_project/bazelrc |
| build --show_timestamps --keep_going --jobs 600 |
| build --color=yes |
| query --keep_going |
| |
| # Definition of --config=memcheck |
| build:memcheck --strip=never --test_timeout=3600 |
| </pre> |
| |
| |
| <h3 id='startup files'> |
| Other files governing Bazel's behavior |
| </h3> |
| |
| <h4 id='.bazelignore'><code>.bazelignore</code></h4> |
| <p> |
| You can specify directories within the workspace |
| that you want Bazel to ignore, such as related projects |
| that use other build systems. Place a file called |
| <code>.bazelignore</code> at the root of the workspace |
| and add the directories you want Bazel to ignore, one per |
| line. Entries are relative to the workspace root. |
| </p> |
| |
| <h2 id='scripting'>Calling Bazel from scripts</h2> |
| |
| <p> |
| Bazel can be called from scripts in order to perform a build, run |
| tests or query the dependency graph. Bazel has been designed to |
| enable effective scripting, but this section lists some details to |
| bear in mind to make your scripts more robust. |
| </p> |
| |
| <h3>Choosing the output base</h3> |
| |
| <p> |
| The <code class='flag'>--output_base</code> option controls where the Bazel process should |
| write the outputs of a build to, as well as various working files used |
| internally by Bazel, one of which is a lock that guards against |
| concurrent mutation of the output base by multiple Bazel processes. |
| </p> |
| <p> |
| Choosing the correct output base directory for your script depends |
| on several factors. If you need to put the build outputs in a |
| specific location, this will dictate the output base you need to |
| use. If you are making a "read only" call to Bazel |
| (e.g. <code>bazel query</code>), the locking factors will be more important. |
| In particular, if you need to run multiple instances of your script |
| concurrently, you will need to give each one a different (or random) output |
| base. |
| </p> |
| <p> |
| If you use the default output base value, you will be contending for |
| the same lock used by the user's interactive Bazel commands. If the |
| user issues long-running commands such as builds, your script will |
| have to wait for those commands to complete before it can continue. |
| </p> |
| |
| <h3>Notes about Server Mode</h3> |
| |
| <p> |
| By default, Bazel uses a long-running <a |
| href='#client/server'>server process</a> as an optimization. When running Bazel |
| in a script, don't forget to call <code>shutdown</code> when you're finished |
| with the server, or, specify <code class='flag'>--max_idle_secs=5</code> so |
| that idle servers shut themselves down promptly. |
| </p> |
| |
| <h3>What exit code will I get?</h3> |
| |
| <p> |
| Bazel attempts to differentiate failures due to the source code under |
| consideration from external errors that prevent Bazel from executing properly. |
| Bazel execution can result in following exit codes: |
| </p> |
| |
| <b>Exit Codes common to all commands:</b> |
| <ul> |
| <li><code>0</code> - Success</li> |
| <li><code>2</code> - Command Line Problem, Bad or Illegal flags or command |
| combination, or Bad Environment Variables. Your command line must be |
| modified.</li> |
| <li><code>8</code> - Build Interrupted but we terminated with an orderly shutdown.</li> |
| <li><code>32</code> - External Environment Failure not on this machine.</li> |
| <li><code>33</code> - OOM failure. You need to modify your command line.</li> |
| |
| <li><code>34</code> - Reserved for Google-internal use.</li> |
| <li><code>35</code> - Reserved for Google-internal use.</li> |
| <li><code>36</code> - Local Environmental Issue, suspected permanent.</li> |
| <li><code>37</code> - Unhandled Exception / Internal Bazel Error.</li> |
| <li><code>38</code> - Reserved for Google-internal use.</li> |
| <li><code>40-44</code> - Reserved for errors in Bazel's command line launcher, |
| <code>bazel.cc</code> that are not command line |
| related. Typically these are related to bazel server |
| being unable to launch itself.</li> |
| <li><code>45</code> - Error publishing results to the Build Event Service.</li> |
| |
| </ul> |
| |
| <b>Return codes for commands <code>bazel build</code>, <code>bazel test</code>.</b> |
| <ul> |
| <li><code>1</code> - Build failed.</li> |
| <li><code>3</code> - Build OK, but some tests failed or timed out.</li> |
| <li><code>4</code> - Build successful but no tests were found even though |
| testing was requested.</li> |
| </ul> |
| |
| <b>For <code>bazel run</code>:</b> |
| <ul> |
| <li><code>1</code> - Build failed.</li> |
| <li>If the build succeeds but the executed subprocess returns a non-zero exit code it will be the |
| exit code of the command as well.</li> |
| </ul> |
| |
| |
| <b>For <code>bazel query</code>:</b> |
| <ul> |
| <li><code>3</code> - Partial success, but the query encountered 1 or more |
| errors in the input BUILD file set and therefore the |
| results of the operation are not 100% reliable. |
| This is likely due to a <code class='flag'>--keep_going</code> option |
| on the command line.</li> |
| <li><code>7</code> - Command failure.</li> |
| </ul> |
| |
| <p> |
| Future Bazel versions may add additional exit codes, replacing generic failure |
| exit code <code>1</code> with a different non-zero value with a particular |
| meaning. However, all non-zero exit values will always constitute an error. |
| </p> |
| |
| <h3>Reading the .bazelrc file</h3> |
| |
| <p> |
| By default, Bazel will read the <a |
| href='#bazelrc'><code>.bazelrc</code> file</a> from the base workspace |
| directory or the user's home directory. Whether or not this is |
| desirable is a choice for your script; if your script needs to be |
| perfectly hermetic (e.g. when doing release builds), you should |
| disable reading the .bazelrc file by using the option |
| <code class='flag'>--bazelrc=/dev/null</code>. If you want to perform a build |
| using the user's preferred settings, the default behavior is better. |
| </p> |
| |
| <h3>Command log</h3> |
| |
| <p> |
| The Bazel output is also available in a command log file which you can |
| find with the following command: |
| </p> |
| |
| <pre> |
| % bazel info command_log |
| </pre> |
| |
| <p> |
| The command log file contains the interleaved stdout and stderr streams |
| of the most recent Bazel command. Note that running <code>bazel info</code> |
| will overwrite the contents of this file, since it then becomes the most |
| recent Bazel command. However, the location of the command log file will |
| not change unless you change the setting of the <code class='flag'>--output_base</code> |
| or <code class='flag'>--output_user_root</code> options. |
| </p> |
| |
| <h3>Parsing output</h3> |
| |
| <p> |
| The Bazel output is quite easy to parse for many purposes. Two |
| options that may be helpful for your script are |
| <code class='flag'>--noshow_progress</code> which suppresses progress messages, |
| and <code class='flag'>--show_result <var>n</var></code>, which controls whether |
| or not "build up-to-date" messages are printed; these messages may |
| be parsed to discover which targets were successfully built, and the |
| location of the output files they created. Be sure to specify a |
| very large value of <i>n</i> if you rely on these messages. |
| </p> |
| |
| <h2 id='profiling'>Troubleshooting performance by profiling</h2> |
| |
| <p> |
| The first step in analyzing the performance of your build is to profile your build with the |
| <a href='user-manual.html#flag--profile'><code class='flag'>--profile</code></a> option. |
| </p> |
| |
| <p> |
| The file generated by the <a href='user-manual.html#flag--profile'><code class='flag'>--profile</code></a> |
| command is a binary file. Once you have generated this binary profile, you can analyze it using |
| Bazel's <a href='user-manual.html#analyze-profile'><code>analyze-profile</code></a> command. By default, it will |
| print out summary analysis information for each of the specified profile datafiles. This includes |
| cumulative statistics for different task types for each build phase and an analysis of the |
| critical execution path. |
| </p> |
| |
| <p> |
| The first section of the default output describes an overview of the time spent on the different |
| build phases: |
| </p> |
| <pre> |
| === PHASE SUMMARY INFORMATION === |
| |
| Total launch phase time 6.00 ms 0.01% |
| Total init phase time 864 ms 1.11% |
| Total loading phase time 21.841 s 28.05% |
| Total analysis phase time 5.444 s 6.99% |
| Total preparation phase time 155 ms 0.20% |
| Total execution phase time 49.473 s 63.54% |
| Total finish phase time 83.9 ms 0.11% |
| Total run time 77.866 s 100.00% |
| </pre> |
| |
| <p> |
| The following sections show the execution time of different tasks happening during a particular |
| phase: |
| </p> |
| <pre> |
| === INIT PHASE INFORMATION === |
| |
| Total init phase time 864 ms |
| |
| Total time (across all threads) spent on: |
| Type Total Count Average |
| VFS_STAT 2.72% 1 23.5 ms |
| VFS_READLINK 32.19% 1 278 ms |
| |
| === LOADING PHASE INFORMATION === |
| |
| Total loading phase time 21.841 s |
| |
| Total time (across all threads) spent on: |
| Type Total Count Average |
| SPAWN 3.26% 154 475 ms |
| VFS_STAT 10.81% 65416 3.71 ms |
| [...] |
| SKYLARK_BUILTIN_FN 13.12% 45138 6.52 ms |
| |
| === ANALYSIS PHASE INFORMATION === |
| |
| Total analysis phase time 5.444 s |
| |
| Total time (across all threads) spent on: |
| Type Total Count Average |
| SKYFRAME_EVAL 9.35% 1 4.782 s |
| SKYFUNCTION 89.36% 43332 1.06 ms |
| |
| === EXECUTION PHASE INFORMATION === |
| |
| Total preparation time 155 ms |
| Total execution phase time 49.473 s |
| Total time finalizing build 83.9 ms |
| |
| Action dependency map creation 0.00 ms |
| Actual execution time 49.473 s |
| |
| Total time (across all threads) spent on: |
| Type Total Count Average |
| ACTION 2.25% 12229 10.2 ms |
| [...] |
| SKYFUNCTION 1.87% 236131 0.44 ms |
| </pre> |
| |
| <p> |
| The last section shows the critical path: |
| </p> |
| <pre> |
| Critical path (32.078 s): |
| Id Time Percentage Description |
| 1109746 5.171 s 16.12% Building [...] |
| 1109745 164 ms 0.51% Extracting interface [...] |
| 1109744 4.615 s 14.39% Building [...] |
| [...] |
| 1109639 2.202 s 6.86% Executing genrule [...] |
| 1109637 2.00 ms 0.01% Symlinking [...] |
| 1109636 163 ms 0.51% Executing genrule [...] |
| 4.00 ms 0.01% [3 middleman actions] |
| </pre> |
| |
| <p> |
| You can use the following options to display more detailed information: |
| </p> |
| |
| <ul> |
| <li id='dump-text-format'><a href='user-manual.html#flag--dump'><code>--dump=text</code></a> |
| <p> |
| This option prints all recorded tasks in the order they occurred. Nested tasks are indented |
| relative to the parent. For each task, output includes the following information: |
| </p> |
| <pre> |
| [task type] [task description] |
| Thread: [thread id] Id: [task id] Parent: [parent task id or 0 for top-level tasks] |
| Start time: [time elapsed from the profiling session start] Duration: [task duration] |
| [aggregated statistic for nested tasks, including count and total duration for each nested task] |
| </pre> |
| </li> |
| <li id='dump-raw-format'><a href='user-manual.html#flag--dump'><code>--dump=raw</code></a> |
| <p> |
| This option is most useful for automated analysis with scripts. It outputs each task record on |
| a single line using '|' delimiter between fields. Fields are printed in the following order: |
| </p> |
| <ol> |
| <li>thread id - integer positive number, identifies owner thread for the task</li> |
| <li>task id - integer positive number, identifies specific task</li> |
| <li>parent task id for nested tasks or 0 for root tasks</li> |
| <li>task start time in ns, relative to the start of the profiling session</li> |
| <li>task duration in ns. Please note that this will include duration of all subtasks.</li> |
| <li>aggregated statistic for immediate subtasks per type. This will include type name (lower |
| case), number of subtasks for that type and their cumulative duration. Types are |
| space-delimited and information for single type is comma-delimited.</li> |
| <li>task type (upper case)</li> |
| <li>task description</li> |
| </ol> |
| |
| Example: |
| <pre> |
| 1|1|0|0|0||PHASE|Launch Bazel |
| 1|2|0|6000000|0||PHASE|Initialize command |
| 1|3|0|168963053|278111411||VFS_READLINK|/[...] |
| 1|4|0|571055781|23495512||VFS_STAT|/[...] |
| 1|5|0|869955040|0||PHASE|Load packages |
| [...] |
| </pre> |
| </li> |
| <li id='dump-html-format'><a href='user-manual.html#flag--html'><code>--html</code></a> |
| <p> |
| This option writes a file called <code><profile-file>.html</code> in the directory of the |
| profile file. Open it in your browser to see the visualization of the actions in your build. |
| Note that the file can be quite large and may push the capabilities of your browser – |
| please wait for the file to load. |
| </p> |
| <p> |
| In most cases, the HTML output from <a href='user-manual.html#flag--html'><code>--html</code></a> is easier to |
| read than the <a href='user-manual.html#flag--dump'><code>--dump</code></a> output. |
| It includes a Gantt chart that displays time on the horizontal axis and |
| threads of execution along the vertical axis. If you click on the Statistics link in the top |
| right corner of the page, you will jump to a section that lists summary analysis information |
| from your build. |
| </p> |
| <ul> |
| <li><a href='user-manual.html#flag--html_details'><code>--html_details</code></a> |
| <p> |
| Additionally passing this option will render a more detailed execution chart and additional |
| tables on the performance of built-in and user-defined Skylark functions. Beware that this |
| increases the file size and the load on the browser considerably. |
| </p> |
| </li> |
| </ul></li> |
| </ul> |
| |
| <p>If Bazel appears to be hung, you can hit <kbd><kbd>ctrl</kbd> + <kbd>\</kbd></kbd> or send |
| Bazel a <code>SIGQUIT</code> signal (<code>kill -3 $(bazel info server_pid)</code>) to get a |
| thread dump in the file <code>$(bazel info output_base)/server/jvm.out</code>. |
| </p> |
| |
| <p> |
| Since you may not be able to run <code>bazel info</code> if bazel is hung, the |
| <code>output_base</code> directory is usually the parent of the <code>bazel-<workspace></code> |
| symlink in your workspace directory. |
| </p> |