blob: b7cab2687278617dd1f5bab75b38b415aac0f096 [file] [log] [blame]
David Chen8fe82a32016-08-24 10:55:41 +00001---
2layout: documentation
Greg Estrenc0816752020-02-20 13:04:29 -08003title: Concepts and terminology
David Chen8fe82a32016-08-24 10:55:41 +00004---
Greg Estrenc0816752020-02-20 13:04:29 -08005<h1>Concepts and terminology</h1>
David Chen8fe82a32016-08-24 10:55:41 +00006<p>
7 This document provides an overview of the source tree layout and the
8 terminology used in Bazel.
9</p>
David Chen8fe82a32016-08-24 10:55:41 +000010<h2 id="intro">Introduction</h2>
11
12<p>Bazel builds software from source code organized in a directory called
13 a workspace. Source files in the workspace are organized in a nested
14 hierarchy of packages, where each package is a directory that contains a set
15 of related source files and one BUILD file. The BUILD file specifies what
16 software outputs can be built from the source.
17</p>
Greg Estrenc0816752020-02-20 13:04:29 -080018<h2 id="packages_targets">Workspace, packages and targets</h2>
David Chen8fe82a32016-08-24 10:55:41 +000019<h3 id="workspace">Workspace</h3>
20
21<p>A <em>workspace</em> is a directory on your filesystem that contains the
22 source files for the software you want to build, as well as symbolic links
23 to directories that contain the build outputs. Each workspace directory has
aehlig7bdfb572019-08-08 05:08:37 -070024 a text file named <code>WORKSPACE</code> which may be empty, or may contain
dzc205125b2017-06-26 11:01:47 +020025 references to <a href="external.html">external dependencies</a>
aehlig7bdfb572019-08-08 05:08:37 -070026 required to build the outputs.</p>
27
28<p>Directories containing a file called
29 <code>WORKSPACE</code> are considered the root of a workspace.
30 Therefore, Bazel ignores any directory trees in a workspace rooted
31 at a subdirectory containing a <code>WORKSPACE</code> file (as they form
32 another workspace).</p>
33
Yun Peng0af987b2019-11-11 07:19:54 -080034<p>Bazel also supports <code>WORKSPACE.bazel</code> file as an alias of <code>WORKSPACE</code> file.
35 If both files exist, <code>WORKSPACE.bazel</code> will take the priority.</p>
36
aehlig7bdfb572019-08-08 05:08:37 -070037<h3 id="repositories">Repositories</h3>
38<p>Code is organized in <em>repositories</em>. The directory containing
39 the <code>WORKSPACE</code> file is the root of the main repository, also
40 called <code>@</code>. Other, (external) repositories
41 are defined in the <code>WORKSPACE</code> file using workspace rules.
42
43<p>The workspace rules bundled with Bazel are documented in the
44 <a href="be/workspace.html">Workspace Rules</a> section in the Build
45 Encyclopedia and the documentation on
46 <a href="repo/index.html">embeded Starlark repository rules</a>.</p>
47
panzhongxian28fe62e2019-08-21 07:36:32 -070048<p>As external repositories are repositories themselves, they often contain
aehlig7bdfb572019-08-08 05:08:37 -070049 a <code>WORKSPACE</code> file as well. However, these additional
50 <code>WORKSPACE</code> files are ignored by Bazel. In particular,
51 repositories depended upon transitively are not added automatically.</p>
52
David Chen8fe82a32016-08-24 10:55:41 +000053<h3 id="packages">Packages</h3>
54<p>
aehlig7bdfb572019-08-08 05:08:37 -070055 The primary unit of code organization in a repository is
Alex Beggscd91d1c2018-07-26 12:39:50 -070056 the <i>package</i>. A package is a collection of related files and a
David Chen8fe82a32016-08-24 10:55:41 +000057 specification of the dependencies among them.
58</p>
laurentlb5ec875a2018-10-18 11:22:28 -070059
David Chen8fe82a32016-08-24 10:55:41 +000060<p>
61 A package is defined as a directory containing a file
laurentlb49939052018-10-18 13:02:55 -070062 named <code>BUILD</code> or <code>BUILD.bazel</code>,
laurentlb5ec875a2018-10-18 11:22:28 -070063 residing beneath the top-level directory in the
David Chen8fe82a32016-08-24 10:55:41 +000064 workspace. A package includes all files in its directory, plus all
65 subdirectories beneath it, except those which themselves contain a BUILD
66 file.
67</p>
laurentlb5ec875a2018-10-18 11:22:28 -070068
David Chen8fe82a32016-08-24 10:55:41 +000069<p>
70 For example, in the following directory tree
71 there are two packages, <code>my/app</code>,
72 and the subpackage <code>my/app/tests</code>.
73 Note that <code>my/app/data</code> is not a package, but a directory
74 belonging to package <code>my/app</code>.
75</p>
76
77<pre>
78src/my/app/BUILD
79src/my/app/app.cc
80src/my/app/data/input.txt
81src/my/app/tests/BUILD
82src/my/app/tests/test.cc
83</pre>
84<h3 id="targets">Targets</h3>
85
86<p>
87 A package is a container. The elements of a package are called
88 <i>targets</i>. Most targets are one of two principal kinds, <i>files</i>
89 and <i>rules</i>. Additionally, there is another kind of target,
90 <a href="be/functions.html#package_group">package groups</a>,
91 but they are far less numerous.
92</p>
93
David Chen8fe82a32016-08-24 10:55:41 +000094<p>
95 Files are further divided into two kinds.
96 <i>Source files</i> are usually written by the efforts of people,
97 and checked in to the repository.
98 <i>Generated files</i>, sometimes called derived files,
99 are not checked in, but are generated by the build tool from source
100 files according to specific rules.
101</p>
102
103<p>
104 The second kind of target is the <i>rule</i>. A rule specifies the
105 relationship between a set of input and a set of output files,
106 including the necessary steps to derive the outputs from the inputs.
107 The outputs of a rule are always generated files. The inputs to a
108 rule may be source files, but they may be generated files also;
109 consequently, outputs of one rule may be the inputs to another,
110 allowing long chains of rules to be constructed.
111</p>
112
113<p>
114 Whether the input to a rule is a source file or a generated file is
115 in most cases immaterial; what matters is only the contents of that
116 file. This fact makes it easy to replace a complex source file with
117 a generated file produced by a rule, such as happens when the burden
118 of manually maintaining a highly structured file becomes too
119 tiresome, and someone writes a program to derive it. No change is
120 required to the consumers of that file. Conversely, a generated
121 file may easily be replaced by a source file with only local
122 changes.
123</p>
124
125<p>
126 The inputs to a rule may also include <i>other rules</i>. The
127 precise meaning of such relationships is often quite complex and
128 language- or rule-dependent, but intuitively it is simple: a C++
129 library rule A might have another C++ library rule B for an input.
Googler793b18d2019-05-09 08:52:01 -0700130 The effect of this dependency is that B's header files are
David Chen8fe82a32016-08-24 10:55:41 +0000131 available to A during compilation, B's symbols are available to A
132 during linking, and B's runtime data is available to A during
133 execution.
134</p>
135
136<p>
137 An invariant of all rules is that the files generated by a rule
138 always belong to the same package as the rule itself; it is not
139 possible to generate files into another package. It is not uncommon
140 for a rule's inputs to come from another package, though.
141</p>
142
143<p>
144 Package groups are sets of packages whose purpose is to limit accessibility
145 of certain rules. Package groups are defined by the
146 <code>package_group</code> function. They have two properties: the list of
147 packages they contain and their name. The only allowed ways to refer to them
148 are from the <code>visibility</code> attribute of rules or from the
149 <code>default_visibility</code> attribute of the <code>package</code>
150 function; they do not generate or consume files. For more information, refer
151 to the appropriate section of the <a
152 href='be/functions.html#package_group'>Build Encyclopedia</a>.
153</p>
154
155
156<h3 id="labels">Labels</h3>
157
158<p>
159 All targets belong to exactly one package. The name of a target is
160 called its <em>label</em>, and a typical label in canonical form
161 looks like this:
162</p>
163
aehlig7bdfb572019-08-08 05:08:37 -0700164
David Chen8fe82a32016-08-24 10:55:41 +0000165<pre>
aehlig7bdfb572019-08-08 05:08:37 -0700166@myrepo//my/app/main:app_binary
David Chen8fe82a32016-08-24 10:55:41 +0000167</pre>
168
169<p>
aehlig7bdfb572019-08-08 05:08:37 -0700170 In the typical case that a label refers to the same repository it occurs
171 in, the repository name may be left out. So, inside <code>@myrepo</code>
172 this label is usually written as
173</p>
David Chen8fe82a32016-08-24 10:55:41 +0000174
aehligc2efa6f2019-08-08 08:18:44 -0700175<pre>
176//my/app/main:app_binary
177</pre>
178
179<p>
180
David Chen8fe82a32016-08-24 10:55:41 +0000181 Each label has two parts, a package name (<code>my/app/main</code>)
182 and a target name (<code>app_binary</code>). Every label uniquely
183 identifies a target. Labels sometimes appear in other forms; when
184 the colon is omitted, the target name is assumed to be the same as
185 the last component of the package name, so these two labels are
186 equivalent:
187</p>
188
189<pre>
190//my/app
191//my/app:app
192</pre>
193
194<p>
195 Short-form labels such as <code>//my/app</code> are not to
196 be confused with package names. Labels start with <code>//</code>,
197 but package names never do, thus <code>my/app</code> is the
198 package containing <code>//my/app</code>.
199
200 (A common misconception is that <code>//my/app</code> refers
201 to a package, or to <em>all</em> the targets in a package; neither
202 is true.)
203</p>
204
205<p>
206 Within a BUILD file, the package-name part of label may be omitted,
207 and optionally the colon too. So within the BUILD file for package
208 <code>my/app</code> (i.e. <code>//my/app:BUILD</code>),
209 the following "relative" labels are all equivalent:
210</p>
211
212<pre>
213//my/app:app
214//my/app
215:app
216app
217</pre>
218
219<p>
220 (It is a matter of convention that the colon is omitted for files,
221 but retained for rules, but it is not otherwise significant.)
222</p>
223
224<p>
225 Similarly, within a BUILD file, files belonging to the package may
226 be referenced by their unadorned name relative to the package
227 directory:
228</p>
229
230
231<pre>
232generate.cc
233testdata/input.txt
234</pre>
235
236<p>
237 But from other packages, or from the command-line, these file
238 targets must always be referred to by their complete label, e.g.
239 <code>//my/app:generate.cc</code>.
240</p>
241
242<p>
243 Relative labels cannot be used to refer to targets in other
244 packages; the complete package name must always be specified in this
245 case. For example, if the source tree contains both the package
246 <code>my/app</code> and the package
247 <code>my/app/testdata</code> (i.e., each of these two
248 packages has its own BUILD file). The latter package contains a
249 file named <code>testdepot.zip</code>. Here are two ways (one
250 wrong, one correct) to refer to this file within
251 <code>//my/app:BUILD</code>:
252</p>
253
254<pre>
255<span class="discouraged">testdata/testdepot.zip</span> # Wrong: testdata is a different package.
256//my/app/testdata:testdepot.zip # Right.
257</pre>
258
259<p>
260 If, by mistake, you refer to <code>testdepot.zip</code> by the wrong
261 label, such as <code>//my/app:testdata/testdepot.zip</code>
262 or <code>//my:app/testdata/testdepot.zip</code>, you will get an
263 error from the build tool saying that the label "crosses a package
264 boundary". You should correct the label by putting the colon after
265 the directory containing the innermost enclosing BUILD file, i.e.,
266 <code>//my/app/testdata:testdepot.zip</code>.
267</p>
268
Michael McLoughlin896391b2020-02-03 15:44:51 -0800269<p>
270 Labels starting with <code>@//</code> are references to the main
271 repository, which will still work even from external repositories.
272 Therefore <code>@//a/b/c</code> is different from
273 <code>//a/b/c</code> when referenced from an external repository.
274 The former refers back to the main repository, while the latter
275 looks for <code>//a/b/c</code> in the external repository itself.
276 This is especially relevant when writing rules in the main
277 repository that refer to targets in the main repository, and will be
278 used from external repositories.
279</p>
280
David Chen8fe82a32016-08-24 10:55:41 +0000281<h3 id="lexi">Lexical specification of a label</h3>
282
283<p>
284 The syntax of labels is intentionally strict, so as to
285 forbid metacharacters that have special meaning to the shell. This
286 helps to avoid inadvertent quoting problems, and makes it easier to
287 construct tools and scripts that manipulate labels, such as the
288
dmartingdffa6362017-08-30 19:23:32 +0200289 <a href='query.html'>Bazel Query Language</a>.
290
twiggd0aac3d2019-10-25 09:42:55 -0700291 The precise details of allowed target names are below.
David Chen8fe82a32016-08-24 10:55:41 +0000292</p>
293
294<h4 id="name">Target names, <code>//...:<b>target-name</b></code></h4>
295
296<p><code>target-name</code> is the name of the target within the package.
297 The name of a rule is the value of the <code>name</code>
Googler98384492019-05-14 11:26:47 -0700298 attribute in the rule's declaration in a BUILD file; the name
David Chen8fe82a32016-08-24 10:55:41 +0000299 of a file is its pathname relative to the directory containing
300 the BUILD file.
301 Target names must be composed entirely of
302 characters drawn from the set <code>a</code>–<code>z</code>,
303 <code>A</code>–<code>Z</code>, <code>0</code>–<code>9</code>,
twiggd0aac3d2019-10-25 09:42:55 -0700304 and the punctuation symbols <code>!%-@^_` "#$&'()*-+,;<=>?[]{|}~/.</code>.
David Chen8fe82a32016-08-24 10:55:41 +0000305 Do not use <code>..</code> to refer to files in other packages; use
306 <code>//<var>packagename</var>:<var>filename</var></code> instead.
307 Filenames must be relative pathnames in normal form, which means
308 they must neither start nor end with a slash
309 (e.g. <code>/foo</code> and <code>foo/</code> are forbidden) nor
310 contain multiple consecutive slashes as path separators
311 (e.g. <code>foo//bar</code>). Similarly, up-level references
312 (<code>..</code>) and current-directory references
313 (<code>./</code>) are forbidden. The sole exception to this
314 rule is that a target name may consist of exactly
315 '<code>.</code>'.
316</p>
317
318<p>While it is common to use <code>/</code> in the name of a file
319 target, we recommend that you avoid the use of <code>/</code> in the
320 names of rules. Especially when the shorthand form of a label is
321 used, it may confuse the reader. The
322 label <code>//foo/bar/wiz</code> is always a shorthand
323 for <code>//foo/bar/wiz:wiz</code>, even if there is no such package
324 <code>foo/bar/wiz</code>; it never refers to <code>//foo:bar/wiz</code>,
325 even if that target exists.</p>
326
327<p>However, there are some situations where use of a slash is
328 convenient, or sometimes even necessary. For example, the name of
329 certain rules must match their principal source file, which may
330 reside in a subdirectory of the package.</p>
331
332<h4>Package names, <code>//<b>package-name</b>:...</code></h4>
333<p>
334 The name of a package is the name of the directory containing its
335
336 BUILD file, relative to the top-level directory of the source tree.
337 For example: <code>my/app</code>.
David Chen8fe82a32016-08-24 10:55:41 +0000338
Googler0bcc9842016-09-15 14:06:13 +0000339 Package names must be composed entirely of characters drawn from
340 the set <code>A</code>-<code>Z</code>, <code>a</code>–<code>z</code>,
341 <code>0</code>–<code>9</code>, '<code>/</code>', '<code>-</code>',
342 '<code>.</code>', and '<code>_</code>', and cannot start with
343 a slash.
David Chen8fe82a32016-08-24 10:55:41 +0000344<p>
345 For a language with a directory structure that is significant
346 to its module system (e.g. Java), it is important to choose directory names
347 that are valid identifiers in the language.
348</p>
349
350<p>
351 Although Bazel allows a package at the build root (e.g. <code>//:foo</code>), this
352 is not advised and projects should attempt to use more descriptively named
353 packages.
354</p>
355<p>
356 Package names may not contain the substring <code>//</code>, nor
357 end with a slash.
358</p>
359
360<h3 id="rules">Rules</h3>
361
362<p>
363 A rule specifies the relationship between inputs and outputs, and the
364 steps to build the outputs. Rules can be of one of many different
365 kinds or <i>classes</i>, which produce compiled
366 executables and libraries, test executables and other supported
367 outputs as described in the
368 <a href="be/overview.html">Build Encyclopedia</a>.
369</p>
370
371<p>
372 Every rule has a name, specified by the <code>name</code> attribute,
373 of type string. The name must be a syntactically valid target name,
374 as specified <a href='#name'>above</a>. In some cases, the name is
375 somewhat arbitrary, and more interesting are the names of the files
376 generated by the rule; this is true of genrules. In other
377 cases, the name is significant: for <code>*_binary</code>
378 and <code>*_test</code> rules, for example, the rule name determines
379 the name of the executable produced by the build.
380</p>
381
laurentlb5ec875a2018-10-18 11:22:28 -0700382<pre>
383cc_binary(
384 name = "my_app",
385 srcs = ["my_app.cc"],
386 deps = [
387 "//absl/base",
388 "//absl/strings",
389 ],
390)
391</pre>
392
David Chen8fe82a32016-08-24 10:55:41 +0000393<p>
394 Every rule has a set of <i>attributes</i>; the applicable attributes
395 for a given rule, and the significance and semantics of each
396 attribute are a function of the rule's class; see
397 the <a href='be/overview.html'>Build
laurentlb5ec875a2018-10-18 11:22:28 -0700398 Encyclopedia</a> for a list of rules and their
David Chen8fe82a32016-08-24 10:55:41 +0000399 corresponding attributes. Each attribute has a name and a
Laszlo Csomor6abc7492018-06-12 01:58:02 -0700400 type. Some of the common types an attribute can have are integer,
David Chen8fe82a32016-08-24 10:55:41 +0000401 label, list of labels, string, list of strings, output label,
402 list of output labels. Not all attributes need to be specified in
403 every rule. Attributes thus form a dictionary from keys (names) to
404 optional, typed values.
405</p>
406
407<p>
408 The <code>srcs</code> attribute present in many rules has type "list
Googlerf772a692019-07-09 11:12:25 -0700409 of labels"; its value, if present, is a list of labels, each being
David Chen8fe82a32016-08-24 10:55:41 +0000410 the name of a target that is an input to this rule.
411</p>
412
413<p>
414 The <code>outs</code> attribute present in many rules has type "list
415 of output labels"; this is similar to the type of
416 the <code>srcs</code> attribute, but differs in two significant
417 ways. Firstly, due to the invariant that the outputs of a rule
418 belong to the same package as the rule itself, output labels cannot
419 include a package component; they must be in one of the "relative"
420 forms shown above. Secondly, the relationship implied by an
421 (ordinary) label attribute is inverse to that implied by an output
422 label: a rule <i>depends on</i> its <code>srcs</code>, whereas a rule <i>is
423 depended on by</i> its <code>outs</code>. The two types of label attributes
424 thus assign direction to the edges between targets, giving rise to a
425 dependency graph.
426</p>
427
428<p>
David Chen8fe82a32016-08-24 10:55:41 +0000429 This directed acyclic graph over targets is called the
430 "target graph" or "build dependency graph", and is the domain over
dmartingdffa6362017-08-30 19:23:32 +0200431 which the <a href='query.html'>Bazel Query tool</a> operates.
David Chen8fe82a32016-08-24 10:55:41 +0000432</p>
433
434
Greg Estrenc0816752020-02-20 13:04:29 -0800435<h2 id="BUILD_files">BUILD files</h2>
David Chen8fe82a32016-08-24 10:55:41 +0000436
437<p>
438 The previous section described packages, targets and labels, and the
439 build dependency graph abstractly. In this section, we'll look at
440 the concrete syntax used to define a package.
441</p>
442
443<p>
444 By definition, every package contains a BUILD file, which is a short
laurentlb5ec875a2018-10-18 11:22:28 -0700445 program.
446 BUILD files are evaluated using an imperative language,
laurentlb353dd742018-10-03 15:25:02 -0700447 <a href="https://github.com/bazelbuild/starlark/">Starlark</a>.
David Chen8fe82a32016-08-24 10:55:41 +0000448
laurentlb5ec875a2018-10-18 11:22:28 -0700449 They are interpreted as a sequential list of statements.
laurentlb353dd742018-10-03 15:25:02 -0700450</p>
451
452<p>
laurentlb5ec875a2018-10-18 11:22:28 -0700453 In general, order does matter: variables must be defined before they are used, for
454 example. However, most BUILD files consist only of declarations of
455 build rules, and the relative order of these statements is
456 immaterial; all that matters is <em>which</em> rules were declared,
457 and with what values, by the time package evaluation completes.
458
459 When a build rule function, such as <code>cc_library</code>, is
460 executed, it creates a new target in the graph. This target can later be
461 referred using a label.
462
463 So, in simple BUILD files, rule declarations can be re-ordered
464 freely without changing the behavior.
465</p>
466
467
468<p>
469 To encourage a clean separation between code and data, BUILD files cannot
laurentlb353dd742018-10-03 15:25:02 -0700470 contain function definitions, <code>for</code> statements or
471 <code>if</code> statements (but list comprehensions and <code>if</code>
472 expressions are allowed). Functions should be declared in <code>.bzl</code>
laurentlb4b6f3622019-02-11 09:42:49 -0800473 files instead. Additionally, <code>*args</code> and <code>**kwargs</code>
474 arguments are not allowed in BUILD files; instead list all the arguments
475 explicitly.
laurentlb353dd742018-10-03 15:25:02 -0700476</p>
477
478<p>
laurentlb5ec875a2018-10-18 11:22:28 -0700479 Crucially, programs in Starlark are unable to perform
laurentlb353dd742018-10-03 15:25:02 -0700480 arbitrary I/O. This invariant makes the
481 interpretation of BUILD files hermetic, i.e. dependent only on a
482 known set of inputs, which is essential for ensuring that builds are
483 reproducible.
484</p>
485
486<p>
487 BUILD files should be written using only ASCII characters,
488 although technically they are interpreted using the Latin-1
489 character set.
490</p>
491
David Chen8fe82a32016-08-24 10:55:41 +0000492<p>
laurentlb5ec875a2018-10-18 11:22:28 -0700493 Since BUILD files need to be updated whenever the dependencies of the
494 underlying code change, they are typically maintained by multiple
495 people on a team. BUILD file authors are encouraged to use comments
496 liberally to document the role of each build target, whether or not it
497 is intended for public use, and to document the role of the package
498 itself.
David Chen8fe82a32016-08-24 10:55:41 +0000499</p>
500
laurentlb78f245d2018-10-23 10:58:16 -0700501<h3 id="load">Loading an extension</h3>
502
503Bazel extensions are files ending in <code>.bzl</code>. Use
504the <code>load</code> statement to import a symbol from an extension.
505
506<pre>
507load("//foo/bar:file.bzl", "some_library")
laurentlb51482d72018-10-23 15:33:01 -0700508</pre>
laurentlb78f245d2018-10-23 10:58:16 -0700509
510This code will load the file <code>foo/bar/file.bzl</code> and add the
511<code>some_library</code> symbol to the environment. This can be used to load
512new rules, functions or constants (e.g. a string, a list, etc.). Multiple
513symbols can be imported by using additional arguments to the call
514to <code>load</code>. Arguments must be string literals (no variable)
515and <code>load</code> statements must appear at top-level, i.e. they cannot be
516in a function body.
517
518The first argument of <code>load</code> is a <a href="#labels">label</a>
519identifying a <code>.bzl</code> file. If it is a relative label, it is resolved
520with respect to the package (not directory) containing the current
521<code>bzl</code> file. Relative labels in <code>load</code> statements should
522use a leading <code>:</code>.
523
524<code>load</code> also supports aliases, i.e. you can assign different names to
525the imported symbols.
526
527<pre>
528load("//foo/bar:file.bzl", library_alias = "some_library")
529</pre>
530
531You can define multiple aliases within one <code>load</code> statement.
532Moreover, the argument list can contain both aliases and regular symbol names.
533The following example is perfectly legal (please note when to use quotation
534marks).
535
536<pre>
537load(":my_rules.bzl", "some_rule", nice_alias = "some_other_rule")
538</pre>
539
540In a <code>.bzl</code> file, symbols starting with <code>_</code> are not
541exported and cannot be loaded from another file. Visibility doesn't affect
542loading (yet): you don't need to use <code>exports_files</code> to make
543a <code>.bzl</code> file visible.
544
David Chen8fe82a32016-08-24 10:55:41 +0000545<h2 id="funcs">Types of build rule</h2>
546
547<p>
548 The majority of build rules come in families, grouped together by
549 language. For
550 example, <code>cc_binary</code>, <code>cc_library</code>
551 and <code>cc_test</code> are the build rules for C++ binaries,
552 libraries, and tests, respectively. Other languages use the same
553 naming scheme, with a different prefix, e.g. <code>java_*</code> for
laurentlb5ec875a2018-10-18 11:22:28 -0700554 Java. Some of these functions are documented in the
555 <a href="be/overview.html">Build Encyclopedia</a>, but it is possible
556 for anyone to create new rules.
David Chen8fe82a32016-08-24 10:55:41 +0000557</p>
558
559<ul>
560 <li><p><code>*_binary</code>
561 rules build executable programs in a given language. After a
562 build, the executable will reside in the build tool's binary
563 output tree at the corresponding name for the rule's label,
564 so <code>//my:program</code> would appear at
565 (e.g.) <code>$(BINDIR)/my/program</code>. </p>
566
567 <p>Such rules also create a runfiles directory
568
569 containing all the files mentioned in a <code>data</code>
570 attribute belonging to the rule, or any rule in its transitive
571 closure of dependencies; this set of files is gathered together in
572 one place for ease of deployment to production.</p>
573 </li>
574
575 <li><p><code>*_test</code>
576 rules are a specialization of a <code>*_binary</code> rule, used for automated
577 testing. Tests are simply programs that return zero on success.
578
579 </p>
580
581 <p>
582 Like binaries, tests also have runfiles trees, and the files
583 beneath it are the only files that a test may legitimately open
584 at runtime. For example, a program <code>cc_test(name='x',
585 data=['//foo:bar'])</code> may open and
586
587 read <code>$TEST_SRCDIR/workspace/foo/bar</code> during execution.
588 (Each programming language has its own utility function for
589 accessing the value of <code>$TEST_SRCDIR</code>, but they are all
590 equivalent to using the environment variable directly.)
591 Failure to observe the rule will cause the test to fail when it is
592 executed on a remote testing host.
593
594 </p>
595 </li>
596
597 <li><code>*_library</code>
598 rules specify separately-compiled modules in the given
599 programming language. Libraries can depend on other libraries,
600 and binaries and tests can depend on libraries, with the expected
601 separate-compilation behavior.
602 </li>
603</ul>
604
605<h2 id="dependencies">Dependencies</h2>
606
607<p>
608 A target <code>A</code> <i>depends upon</i> a target
609 <code>B</code> if <code>B</code> is needed by <code>A</code> at
610 build or execution time. The <i>depends upon</i> relation induces a
laurentlb5ec875a2018-10-18 11:22:28 -0700611 <a href="https://en.wikipedia.org/wiki/Directed_acyclic_graph">Directed
612 Acyclic Graph</a> (DAG) over targets, and we call this a
David Chen8fe82a32016-08-24 10:55:41 +0000613 <em>dependency graph</em>.
614
615 A target's <em>direct</em> dependencies are those other targets
616 reachable by a path of length 1 in the dependency graph. A target's
617 <em>transitive</em> dependencies are those targets upon which it
618 depends via a path of any length through the graph.
619</p>
620
621<p>
622 In fact, in the context of builds, there are two dependency graphs,
623 the graph of <em>actual dependencies</em> and the graph of
624 <em>declared dependencies</em>. Most of the time, the two graphs
625 are so similar that this distinction need not be made, but it is
626 useful for the discussion below.
627</p>
628
629<h3 id="actual_and_declared_dependencies">Actual and declared dependencies</h3>
630
631<p>
632 A target <code>X</code> is <i>actually dependent</i> on target
Googler11b0b1f2019-10-08 02:03:02 -0700633 <code>Y</code> if and only if <code>Y</code> must be present, built
634 and up-to-date in order for <code>X</code> to be built correctly.
David Chen8fe82a32016-08-24 10:55:41 +0000635 "Built" could mean generated, processed, compiled, linked,
636 archived, compressed, executed, or any of the other kinds of tasks
637 that routinely occur during a build.
638</p>
639
640<p>
641 A target <code>X</code> has a <i>declared dependency</i> on target
Googler11b0b1f2019-10-08 02:03:02 -0700642 <code>Y</code> if and only if there is a dependency edge from <code>X</code>
643 to <code>Y</code> in the package of <code>X</code>.
David Chen8fe82a32016-08-24 10:55:41 +0000644</p>
645
646<p>
647 For correct builds, the graph of actual dependencies <i>A</i> must
648 be a subgraph of the graph of declared dependencies <i>D</i>. That
649 is, every pair of directly-connected nodes <code>x --&gt; y</code>
650 in <i>A</i> must also be directly connected in <i>D</i>. We say
651 <i>D</i> is an <em>overapproximation</em> of <i>A</i>.
652</p>
653
654<p>
655 It is important that it not be too much of an overapproximation,
656 though, since redundant declared dependencies can make builds slower and
657 binaries larger.
658</p>
659
660<p>
661 What this means for BUILD file writers is that every rule must
662 explicitly declare all of its actual direct dependencies to the
663 build system, and no more.
664
665 Failure to observe this principle causes undefined behavior: the
666 build may fail, but worse, the build may depend on some prior
667 operations, or upon which transitive declared dependencies the target
668 happens to have. The build tool attempts aggressively to check for
669 missing dependencies and report errors, but it is not possible for
670 this checking to be complete in all cases.
671</p>
672
673<p>
674
675 You need not (and should not) attempt to list everything indirectly imported,
676 even if it is "needed" by A at execution time.
677</p>
678
679<p>
680 During a build of target <code>X</code>, the build tool inspects the
681 entire transitive closure of dependencies of <code>X</code> to ensure that
682 any changes in those targets are reflected in the final result,
683 rebuilding intermediates as needed.
684</p>
685
686<p>
687 The transitive nature of dependencies leads to a common mistake.
688 Through careless programming, code in one file may use code provided
689 by an <em>indirect</em> dependency, i.e. a transitive but not direct
690 edge in the declared dependency graph. Indirect dependencies do not
691 appear in the BUILD file. Since the rule doesn't
692 directly depend on the provider, there is no way to track changes,
693 as shown in the following example timeline:
694</p>
695
696<div class="greenbox">
697<p><b>1. At first, everything works</b></p>
698
699<p>The code in package <code>a</code> uses code in package <code>b</code>.
700The code in package <code>b</code> uses code in package <code>c</code>,
701and thus <code>a</code> transitively depends on <code>c</code>.</p>
702
703<div style="float:left; width: 49%; margin-top: -20px;">
704<p><code>a/BUILD</code></p>
705<pre class="code">
706<b>rule(
707 name = "a",
708 srcs = "a.in",
709 deps = "//b:b",
710)</b>
711</pre>
712<p><code>a/a.in</code></p>
713<pre class="code">
714<b>import b;
715b.foo();</b>
716</pre>
717</div>
718<div style="float:right; width: 49%; margin-top: -20px; ">
719<p><code>b/BUILD</code></p>
720<pre class="code">
721<b>rule(
722 name = "b",
723 srcs = "b.in",
724 deps = "//c:c",
725)</b>
726</pre>
727<p><code>b/b.in</code></p>
728<pre class="code">
729<b>import c;
730function foo() {
731 c.bar();
732}</b>
733</pre>
734</div>
Googler11b0b1f2019-10-08 02:03:02 -0700735<table style='margin: auto; width: 100%'><tr>
736<td style='padding: 10px; text-align: center'>
737<!-- digraph G {
738 graph [size="4,4"];
739 node [shape=circle];
740 rankdir="LR";
741 a -> b -> c;
742} -->
743<img src="images/a_b_c.svg" alt="a_b_c.svg" style="margin-left=10;" />
744<p><i>Declared dependency graph</i></p>
745</td>
746<td style='padding: 10px; text-align: center'>
747<!-- digraph G {
748 graph [size="4,4"];
749 node [shape=circle];
750 rankdir="LR";
751 a -> b -> c;
752} -->
753<img src="images/a_b_c.svg" alt="a_b_c.svg" style="margin-left=10;" />
754<p><i>Actual dependency graph</i></p>
755</td>
756</tr></table>
David Chen8fe82a32016-08-24 10:55:41 +0000757The declared dependencies overapproximate the actual dependencies.
758All is well.
759</div>
760
761<div class="greenbox">
762<p><b>2. A latent hazard is introduced.</b></p>
763<p>
764 Someone carelessly adds code to <code>a</code> that creates a direct
765 actual dependency on <code>c</code>, but forgets to declare it.
766</p>
767<div style="float:left; width: 49%; margin-top: -20px; ">
768<p><code>a/a.in</code></p>
769<pre class="code">
770import b;
771<b>import c;</b>
772b.foo();
773<b>c.garply();</b>
774</pre>
775</div>
776
Googler11b0b1f2019-10-08 02:03:02 -0700777<table style='margin: auto; width: 100%'><tr>
778<td style='padding: 10px; text-align: center'>
779<!-- digraph G {
780 graph [size="4,4"];
781 node [shape=circle];
782 rankdir="LR";
783 a -> b -> c;
784} -->
785<img src="images/a_b_c.svg" alt="a_b_c.svg" style="margin-left=10;" />
786<p><i>Declared dependency graph</i></p>
787</td>
788<td style='padding: 10; text-align: center'>
789<!-- digraph G {
790 graph [size="4,4"];
791 node [shape=circle];
792 rankdir="LR";
793 a -> b -> c;
794 a -> c [constraint=false];
795} -->
796<img src="images/a_b_c_ac.svg" alt="a_b_c_ac.svg" style="margin-left=10;" />
797<p><i>Actual dependency graph</i></p>
798</td>
799</tr></table>
David Chen8fe82a32016-08-24 10:55:41 +0000800The declared dependencies no longer overapproximate the actual
801dependencies. This may build ok, because the transitive closures of
802the two graphs are equal, but masks a problem: <code>a</code> has an
803actual but undeclared dependency on <code>c</code>.
804</div>
805
806<div class="greenbox">
807<p><b>3. The hazard is revealed</b> </p>
808<p>
809 Someone refactors <code>b</code> so that it no longer depends on
810 <code>c</code>, inadvertently breaking <code>a</code> through no
811 fault of their own.
812</p>
813<div style="float:right; width: 49%; margin-top: -20px; ">
814<p><code>b/BUILD</code></p>
815<pre class="code">
816rule(
817 name = "b",
818 srcs = "b.in",
819 <b>deps = "//d:d"</b>,
820)
821</pre>
822<p><code>b/b.in</code></p>
823<pre class="code">
824<b>import d;</b>
825function foo() {
826 <b>d.baz();</b>
827}
828</pre>
829</div>
Googler11b0b1f2019-10-08 02:03:02 -0700830<table style='margin: auto; width: 100%'><tr>
831<td style='padding: 10px; text-align: center'>
832<!-- digraph G {
833 graph [size="4,4"];
834 node [shape=circle];
835 rankdir="LR";
836 a -> b;
837 b -> c [style=invis];
838} -->
839<img src="images/ab_c.svg" alt="ab_c.svg" style="margin-left=10;" />
840<p><i>Declared dependency graph</i></p>
841</td>
842<td style='padding: 10; text-align: center'>
843<!-- digraph G {
844 graph [size="4,4"];
845 node [shape=circle];
846 rankdir="LR";
847 a -> b;
848 b -> c [style=invis];
849 a -> c [constraint=false];
850} -->
851<img src="images/a_b_a_c.svg" alt="a_b_a_c.svg" style="margin-left=10;" />
852<p><i>Actual dependency graph</i></p>
853</td>
854</tr></table>
David Chen8fe82a32016-08-24 10:55:41 +0000855<p>
856 The declared dependency graph is now an underapproximation of the
857 actual dependencies, even when transitively closed; the build is
858 likely to fail.
859
860 The problem could have been averted by ensuring that the actual
861 dependency from <code>a</code> to <code>c</code> introduced in Step
862 2 was properly declared in the BUILD file.
863</div>
864
865<h3 id="types_of_dependencies">Types of dependencies</h3>
866
867<p>
868 Most build rules have three attributes for specifying different kinds
869 of generic dependencies: <code>srcs</code>, <code>deps</code> and
870 <code>data</code>. These are explained below. See also
871 <a href='be/common-definitions.html'>Attributes common
dannark27486a62017-06-26 06:36:31 +0200872 to all rules</a> in the Build Encyclopedia.
David Chen8fe82a32016-08-24 10:55:41 +0000873</p>
874
875<p>
876 Many rules also have additional attributes for rule-specific kinds
877 of dependency, e.g. <code>compiler</code>, <code>resources</code>,
878 etc. These are detailed in the Build Encyclopedia.
879</p>
880
881<h4 id="srcs"><code>srcs</code> dependencies</h4>
882<p>
883 Files consumed directly by the rule or rules that output source files.
884</p>
885
886<h4 id="deps"><code>deps</code> dependencies</h4>
887<p>
888 Rule pointing to separately-compiled modules providing header files,
889 symbols, libraries, data, etc.
890</p>
891
892<h4 id="data"><code>data</code> dependencies</h4>
893<p>A build target might need some data files to run correctly. These
894 data files aren't source code: they don't affect how the target is
895 built. For example, a unit test might compare a function's output
896 to the contents of a file. When we build the unit test, we
897 don't need the file; but we do need it when we run the test. The
898 same applies to tools that are launched during execution.
899
900<p>The build system runs tests in an isolated directory where only files
901 listed as "data" are available. Thus, if a binary/library/test
902 needs some files to run, specify them (or a build rule containing
903 them) in data. For example:
904</p>
905
906<pre>
907# I need a config file from a directory named env:
908java_binary(
909 name = "setenv",
910 ...
911 data = [":env/default_env.txt"],
912)
913
914# I need test data from another directory
915sh_test(
916 name = "regtest",
917 srcs = ["regtest.sh"],
918 data = [
919 "//data:file1.txt",
920 "//data:file2.txt",
921 ...
922 ],
923)
924</pre>
925
926<p>These files are available using the relative path
927<code>path/to/data/file</code>. In tests, it is also possible to refer to
928them by joining the paths of the test's source directory and the workspace-relative
929path, e.g.
930
931<code>${TEST_SRCDIR}/workspace/path/to/data/file</code>.
Greg Estrenc0816752020-02-20 13:04:29 -0800932 <h3 id="label_directory">Using labels to reference directories</h3>
David Chen8fe82a32016-08-24 10:55:41 +0000933
934 <p>As you look over our <code>BUILD</code> files, you might notice
935 that some <code>data</code> labels refer to directories.
936 These labels end with <code>/.</code> or <code>/</code> like so:
937
938<pre>
939<span style="text-decoration: line-through">data = ["//data/regression:unittest/."]</span> # don't use this
940</pre>
941<p>
942or like so:
943</p>
944<pre>
945<span style="text-decoration: line-through">data = ["testdata/."]</span> # don't use this
946</pre>
947
948<p>
949or like so:
950</p>
951
952<pre>
953<span style="text-decoration: line-through">data = ["testdata/"]</span> # don't use this
954</pre>
955 <p>This seems convenient, particularly for tests (since it allows a test to
956 use all the data files in the directory).
957 </p>
958
959 <p>But try not to do this. In order to ensure correct incremental rebuilds (and
960 re-execution of tests) after a change, the build system must be
961 aware of the complete set of files that are inputs to the build (or
962 test). When you specify a directory, the build system will perform
963 a rebuild only when the directory itself changes (due to addition or
964 deletion of files), but won't be able to detect edits to individual
965 files as those changes do not affect the enclosing directory.
966 Rather than specifying directories as inputs to the build system,
967 you should enumerate the set of files contained within them, either
968 explicitly or using the
969 <a href='be/functions.html#glob'><code>glob()</code></a> function.
970 (Use <code>**</code> to force the <a href='be/functions.html#glob'>
971 <code>glob()</code></a> to be recursive.)
972 </p>
973
974<pre>
975data = glob(["testdata/**"]) # use this instead
976</pre>
977
978 <p>Unfortunately, there are some scenarios where directory labels must be used.
979 For example, if the <code>testdata</code> directory contains files whose
980 names do not conform to the strict <a href='#lexi'>label syntax</a>
981 (e.g. they contain certain punctuation symbols), then explicit
982 enumeration of files, or use of the
983 <a href='be/functions.html#glob'><code>glob()</code></a> function will
984 produce an invalid labels error. You must use directory labels in this case,
985 but beware of the concomitant risk of incorrect rebuilds described above.
986 </p>
987
988 <p>If you must use directory labels, keep in mind that you can't refer to the parent
989 package with a relative "<code>../</code>" path; instead, use an absolute path like
990 "<code>//data/regression:unittest/.</code>".
991 </p>
992
993 <p>Note that directory labels are only valid for data dependencies. If you try to use
994 a directory as a label in an argument other than <code>data</code>, it
995 will fail and you will get a (probably cryptic) error message.
996 </p>
997