blob: f0ac3122097b96eaece16e55ee08a7224b2ed4ed [file] [log] [blame] [view]
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +01001
2What is Bazel?
3-------------
4
5Bazel is a build system, i.e. a system that will run compilers and
6tests to assemble your software, similar to Make, Ant, Gradle, Buck, Pants,
7and Maven.
8
9
10What is special about Bazel?
11-----------------------
12
13Bazel was designed to fit the way software is developed at Google. It
14has the following features:
15
16* Multi-language support: Bazel supports Java and C++ out of the box,
17 and can be extended to support arbitrary programming languages.
18
19* High-level build language: Projects are described in the BUILD
20 language, a concise text format that describes a project as sets of
Han-Wen Nienhuys03549332015-02-20 13:21:24 +000021 small interconnected libraries, binaries and tests. By contrast, in
22 systems like Make you have to describe individual files and compiler
23 invocations.
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +010024
25* Multi-platform support: The same tool and the same BUILD files can
26 be used to build software for different architectures, and even
27 different platforms. At Google, we use Bazel to build both server
Han-Wen Nienhuys03549332015-02-20 13:21:24 +000028 applications running on systems in our data centers and client apps
29 running on mobile phones.
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +010030
31* Reproducibility: In BUILD files, each library, test, and binary must
Han-Wen Nienhuys03549332015-02-20 13:21:24 +000032 specify its direct dependencies completely. Bazel uses this
33 dependency information to know what must be rebuilt when you make
34 changes to a source file, and which tasks can run in parallel. This
35 means that all builds are incremental and will always produce the
36 same result.
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +010037
38* Scalable: Bazel can handle large builds; at Google, it is common for
39 a server binary to have 100k source files, and builds where no files
40 were changed take about ~200ms.
41
42
Han-Wen Nienhuys03549332015-02-20 13:21:24 +000043Why doesn't't Google use ...?
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +010044-----------------------
45
46* Make, Ninja: These systems give very exact control over what commands
47 get invoked to build files, but it's up to the user to write rules
48 that are correct.
49
50 Users interact with Bazel on a higher level. For example, it has
51 built-in rules for "Java test", "C++ binary", and notions such as
52 "target platform" and "host platform". The rules have been battle
53 tested to be foolproof.
54
55* Ant and Maven: Ant and Maven are primarily geared toward Java, while
56 Bazel handles multiple languages. Bazel encourages subdividing
57 codebases in smaller reusable units, and can rebuild only ones that
58 need rebuilding. This speeds up development when working with larger
59 codebases.
60
61* Gradle: Bazel configuration files are much more structured than
62 Gradle's, letting Bazel understand exactly what each action does.
63 This allows for more parallelism and better reproducibility.
64
65* Buck, Pants: Both systems were created ex-Googlers at Twitter and
66 Facebook. They have been modeled on Bazel, but their feature set
67 has not caught up to Bazel, so it's not an alternative for us.
68
69
70What is Bazel's origin?
71-----------------------
72
73Bazel is a flavor of the tool that Google uses to build its server
74software internally. It has expanded to also build the client apps
75(iOS, Android) that connect to our servers.
76
77
78Did you rewrite your internal tool as open-source? Is it a fork?
79----------------------------------------------------------------
80
Han-Wen Nienhuys03549332015-02-20 13:21:24 +000081Bazel shares most of its code with the internal tool, and its rules
82are used for millions of builds every day.
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +010083
84
85Why did Google build Bazel?
86---------------------------
87
88A long time ago, Google built its software using large, generated
89Makefiles. These led to slow and unreliable builds, which began to
90interfere with our developers' productivity and the company's
91agility. Hence, we built Bazel.
92
93
94Does Bazel require a build cluster?
95-----------------------------------
96
97Google's in-house flavor of Bazel does use [build
98clusters](http://google-engtools.blogspot.com/2011/09/build-in-cloud-distributing-build-steps.html),
Han-Wen Nienhuys03549332015-02-20 13:21:24 +000099so Bazel does have hooks in the code base to plug in a remote build
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100100cache or a remote execution system.
101
102The code base we are opening up runs tasks locally. We are confident
103that this is fast enough for most of our users.
104
105
106How does the Google development process work?
107----------------------------------------------
108
109For our server code, we use the following development workflow:
110
111* All of our server code base is in a single, gigantic version control
112 system.
113
114* Everybody builds their software with Bazel.
115
116* Different teams own different parts of the source tree, and make
117 their components available as BUILD targets.
118
119* Branching is primarily used for managing releases, so everybody
120 develops their software at head.
121
122Bazel is a cornerstones of this philosophy: since Bazel requires all
123dependencies to be fully specified, we can predict which programs and
124tests are affected by a change, and vet them before submission.
125
126More background on the development process at Google can be found on
127the [eng tools blog](http://google-engtools.blogspot.com/).
128
129
130Why are you opening up Bazel?
131-----------------------------
132
133Building software should be fun and easy, and slow and unpredictable
134builds take the fun out of programming.
135
136
137Why would I want to use Bazel?
138------------------------------
139
140* Bazel may give you faster build times because it can recompile only
141 the files that need to be recompiled. Similarly, it can skip
142 re-running tests it knows haven't changed.
143
144* Bazel produces deterministic results. This eliminates skews
145 between incremental vs clean builds, laptop vs CI system, etc.
146
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000147* Bazel can build different client and server apps with the same tool
148 from the same workspace. For example, you can change a client/server
149 protocol in a single commit, and test that the updated mobile app
150 works with the updated server, building both with the same tool,
151 reaping all the aforementioned benefits of Bazel.
152
153
154Can I see examples?
155-------------------
156
157Yes, for a simple example, see
158
159 https://github.com/google/bazel/blob/master/base_workspace/examples/cpp/BUILD
160
161The bazel source code itself provides more complex examples, eg.
162
163 https://github.com/google/bazel/blob/master/src/main/java/BUILD
164 https://github.com/google/bazel/blob/master/src/test/java/BUILD
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100165
166
167What is Bazel best at?
168----------------------
169
170Bazel shines at building and testing projects with the following properties:
171
172* Projects with a large codebase
173* Projects written in (multiple) compiled languages
174* Projects that deploy on multiple platforms
175* Projects that have extensive tests
176
177
178On what platforms does Bazel run?
179---------------------------------
180
181Currently, Linux and MacOS. Porting to other Unix platforms should be
182straightforward, provided a JDK is available for the platform.
183
184
185What about Windows?
186-------------------
187
188We have experimented with a Windows port using MinGW/MSYS (see
189README.windows), but have no plans to invest in this port right
190now. Due to its Unix heritage, porting Bazel is significant work. For
191example, Bazel uses symlinks extensively, which has varying levels of
192support across Windows versions.
193
194
195What should I not use Bazel for?
196--------------------------------
197
198* Bazel tries to be smart about caching. This means it is a bad match
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000199 for build steps that should not be cached. For example, the following
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100200 steps should not be controlled from Bazel:
201
202 * A compilation step that fetches data from the internet.
203 * A test step that connects to the QA instance of your site.
204 * A deployment step that changes your site's cloud configuration.
205
206* Bazel tries to minimize expensive compilation steps. If you are only
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000207 using interpreted languages directly, such as JavaScript or Python,
208 Bazel will likely not interest you.
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100209
210
211
212How stable is Bazel's feature set?
213--------------------
214
215The core features (C++, Java, and shell rules) have extensive use
216inside Google, so they are thoroughly tested and have very little
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000217churn. Similarly, we test new versions of Bazel across hundreds of
218thousands of targets every day to find regressions, and we release new
219versions multiple times every month.
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100220
221In short, except for features marked as experimental, at any point in
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000222time, Bazel should Just Work. Changes to non-experimental rules will
223be backward compatible.
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100224
225You can recognize the different experimental features as follows:
226
227* Options: Experimental features start with --experimental_.
228
229* Rules: Experimental rules are undocumented in the
230 build encyclopedia. TODO(bazel-team): add link.
231
232* Attributes: Experimental rule attributes are undocumented in the
233 build encyclopedia. TODO(bazel-team): add link.
234
235* TODO(bazel-team): document skylark status more precisely.
236
237
238How stable is Bazel as a binary?
239--------------------
240
241Inside Google, we make sure that Bazel crashes are very rare. This
242should also hold for our open-source codebase.
243
244
245How can I start using Bazel?
246----------------------------
247
248See our [getting started
249doc](https://github.com/google/bazel/blob/master/docs/getting-started.md)
250
251TODO(bazel-team): more doc links
252
253
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000254Why do I need to have a tools/ directory in my source tree?
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100255----------------------------------------------------
256
257Your project never works in isolation. Typically, it builds with a
258certain version of the JDK/C++ compiler, with a certain test driver
259framework, on a certain version of your operating system.
260
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000261To guarantee builds are reproducible even when we upgrade our
262workstations, we at Google check most of these tools into version
263control, including the toolchains and Bazel itself. By convention, we
264do this in a directory called "tools".
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100265
266Bazel allows tools such as the JDK to live outside your workspace, but
267the configuration data for this (where is the JDK, where is the C++
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000268compiler?) still needs to be somewhere, and that place is also the
269tools/ directory.
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100270
271Bazel comes with a base_workspace/ directory, containing a minimal set
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000272of configuration files, suitable for running toolchains from standard
273system directories, e.g., /usr/bin/.
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100274
275
276Doesn't Docker solve the reproducibility problems?
277--------------------------------------------------
278
279With Docker you can easily create sandboxes with fixed OS releases,
280eg. Ubuntu 12.04, Fedora 21. This solves the problem of
281reproducibility for the system environment (i.e. "which version of
282/usr/bin/c++ do I need?").
283
284It does not address reproducibility with regard to changes in the
Daniel Wagner-Hall50a11942015-02-06 18:25:18 +0000285source code. Running Make with an imperfectly written Makefile inside a
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100286Docker container can still yield unpredictable results.
287
288Inside Google, we check in tools for reproducibility. In this way, we
289can vet changes to tools ("upgrade GCC to 4.6.1") with the same
290mechanism as changes to base libraries ("fix bounds check in OpenSSL").
291
292
293Will Bazel make my builds reproducible automatically?
294-----------------------------------------------------
295
296For Java and C++ binaries, yes, assuming you do not change the
297toolchain. If you have build steps that involve custom recipes
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000298(eg. executing binaries through a shell script inside a rule), you
299will need to take some extra care:
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100300
301 * Do not use dependencies that were not declared. Sandboxed
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000302 execution (--spawn_strategy=sandboxed, only on Linux) can
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100303 help find undeclared dependencies.
304
Laurent Le Brun40776632015-02-06 18:17:04 +0000305 * Avoid storing timestamps in generated files. ZIP files and other
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100306 archives are especially prone to this.
307
308 * Avoid connecting to the network. Sandboxed execution can help here
309 too.
310
311 * Avoid processes that use random numbers, in particular, dictionary
312 traversal is randomized in many programming languages.
313
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000314TODO(bazel-team): maybe this should be part of the skylark docs
315instead.
316
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100317
318Do you have binary releases?
319----------------------------
320
321No, but we should. Stay tuned.
322
323
324I use Eclipse/IntelliJ. How does Bazel interoperate with IDEs?
325--------------------------------------------------------------
326
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000327We currently have no IDE integration API, although the "bazel query"
328command could be used as a building block for for basic IDE support?
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100329
330
331I use Jenkins/CircleCI/TravisCI. How does Bazel interoperate with CI systems?
332-----------------------------------------------------------------------------
333
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000334Bazel returns a non-zero exit code if the build or test invocation
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100335fails, and this should be enough for basic CI integration. Since
336Bazel does not need clean builds for correctness, the CI system can
337be configured to not clean before starting a build/test run.
338
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000339TODO(bazel-team): link to exit codes.
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100340
341What future features can we expect in Bazel?
342--------------------------------------------
343
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000344Our initial goal is to work on Google's internal use-cases. This
345includes Google's principal languages (C++, Java, Go) and major
346platforms (Linux, Android, iOS). For practical reasons, not all of
347these are currently open-sourced.
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100348
349TODO(bazel-team): add link to public roadmap.
350
351
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100352What about Python?
353------------------
354
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000355It is possible to write Python rules as extensions (see below). See
356the following files for an example of generating self-contained zip
357files for python:
Han-Wen Nienhuys89ab4a12015-02-06 18:49:56 +0000358
359 https://github.com/google/bazel/blob/master/base_workspace/tools/build_rules/py_rules.bzl
360 https://github.com/google/bazel/tree/master/base_workspace/examples/py
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100361
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000362Our internal Python rules support internal use-cases which rely heavily
363on native extensions, and its supporting infrastructure is specific to
364us, so we have no plans to open-source it.
365
366
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100367
368Can I use Bazel for my LISP/Python/Haskell/Scala/Rust project?
369-----------------------------------------------
370
371We have an extension mechanism that allows you to add new rules
372without recompiling Bazel.
373
374For documentation: see TODO(bazel-team).
375
376At present, the extension mechanism is experimental though.
377
378
379I need more functionality; can I add rules that are compiled into Bazel?
380---------------------------------------------
381
382If our extension mechanism is insufficient for your use case, email
383the mailing list for advice: bazel-discuss@googlegroups.com.
384
385
386
387Can I contribute to the Bazel code base?
388----------------------------------------
389
390Not yet, but we are working on a process.
391
392Please consider the following:
393
394* Before starting to code, discuss your plan to make sure we agree on
395 it. We can be reached at bazel-discuss@googlegroups.com.
396
397* All contributors must sign a Contributor License Agreement at
398
399 https://cla.developers.google.com/
400
401* All contributions will have to go through pre-commit code review.
402
403* Contributions need a Google team member as sponsor.
404
405TODO(bazel-team): set up a process
406
Han-Wen Nienhuys03549332015-02-20 13:21:24 +0000407TODO(bazel-team): link to detailed patch acceptance policy.
408
Han-Wen Nienhuysd08b27f2015-02-25 16:45:20 +0100409
410Why isn't all development done in the open?
411-------------------------------------------
412
413We still have to refactor the interfaces between the public code in
414Bazel and our internal extensions frequently. This makes it hard to do
415much development in the open.
416
417TODO(bazel-team): link to roadmap.
418
419
420How do I contact the team?
421--------------------------
422
423We are reachable at bazel-discuss@googlegroups.com.
424
425
426Where do I report bugs?
427-----------------------
428
429Send us e-mail at bazel-discuss@googlegroups.com.
430
431
432
433What's up with the word "Blaze" in the codebase?
434------------------------------------------------
435
436This is an internal name for the system. Please refer to Bazel as
437Bazel.
438
439
440Why do other Google projects (Android, Chrome) use other build tools?
441---------------------------------------------------------------------
442
443Until now, Bazel was not available externally, so open source projects
444such as Chromium, Android, etc. could not use it. In addition, lack of
445Windows support is a problem for building Windows applications, such
446as Chrome.
447
448
449How do you pronounce "Bazel"?
450-----------------------------
451
452The same way as "basil" (the herb) in US English: "BAY-zel". It rhymes with
453"hazel". IPA: /ˈbeɪzˌəl/