David Chen | 8fe82a3 | 2016-08-24 10:55:41 +0000 | [diff] [blame] | 1 | --- |
| 2 | layout: documentation |
| 3 | title: Output Directory Layout |
| 4 | --- |
| 5 | |
| 6 | # Output Directory Layout |
| 7 | |
| 8 | ## Requirements |
| 9 | |
| 10 | Requirements for an output directory layout: |
| 11 | |
| 12 | * Don't collide if multiple users are building on the same box. |
| 13 | * Support building in multiple workspaces at the same time. |
| 14 | * Support building for multiple target configurations in the same workspace. |
| 15 | * Don't collide with any other tools. |
| 16 | * Be easy to access. |
| 17 | * Be easy to clean, even selectively. |
| 18 | * Is unambiguous, even if the user relies on symbolic links when changing into |
| 19 | his/her client directory. |
| 20 | * All the build state per user should be underneath one directory ("I'd like to |
| 21 | clean all the .o files from all my clients.") |
| 22 | |
| 23 | ## Documentation of the current Bazel output directory layout |
| 24 | |
| 25 | The solution that's currently implemented: |
| 26 | |
| 27 | * Bazel must be invoked from a directory containing a WORKSPACE file. It reports |
| 28 | an error if it is not. We call this the _workspace directory_. |
| 29 | * The _outputRoot_ directory is ~/.cache/bazel. (Unless `$TEST_TMPDIR` is |
| 30 | set, as in a test of bazel itself, in which case this directory is used |
| 31 | instead.) |
| 32 | * We stick the Bazel user's build state beneath `outputRoot/_bazel_$USER`. This |
| 33 | is called the _outputUserRoot_ directory. |
| 34 | * Beneath the `outputUserRoot` directory, we create an `installBase` directory |
| 35 | whose name is "install" plus the MD5 hash of the Bazel installation manifest. |
| 36 | * Beneath the `outputUserRoot` directory, we also create an `outputBase` |
| 37 | directory whose name is the MD5 hash of the path name of the workspace |
| 38 | directory. So, for example, if Bazel is running in the workspace directory |
| 39 | `/home/user/src/my-project` (or in a directory symlinked to that one), then we |
| 40 | create an output base directory called: |
| 41 | `/home/.cache/bazel/_bazel_user/7ffd56a6e4cb724ea575aba15733d113`. |
| 42 | * Users can use Bazel's `--output_base` startup option to override the default |
| 43 | output base directory. For example, |
| 44 | `bazel --output_base=/tmp/bazel/output build x/y:z`. |
| 45 | * Users can also use Bazel's `--output_user_root` startup option to override the |
| 46 | default install base and output base directories. For example: |
| 47 | `bazel --output_user_root=/tmp/bazel build x/y:z`. |
| 48 | |
| 49 | We put symlinks "bazel-<workspace-name>" and "bazel-out", as well as |
| 50 | "bazel-bin", "bazel-genfiles", and "bazel-includes" in the workspace directory; |
| 51 | these symlinks points to some directories inside a target-specific directory |
| 52 | inside the output directory. These symlinks are only for the user's convenience, |
| 53 | as Bazel itself does not use them. Also, we only do this if the workspace |
| 54 | directory is writable. The names of the "bazel-bin", "bazel-genfiles", and |
| 55 | "bazel-include" symlinks are affected by the `--symlink_prefix` option to bazel, |
| 56 | but "bazel-<workspace-name>" and "bazel-out" are not. |
| 57 | |
| 58 | ## Bazel internals: Directory layout |
| 59 | |
| 60 | The directories are laid out as follows: |
| 61 | |
| 62 | <pre> |
| 63 | <workspace-name>/ <== The workspace directory |
| 64 | bazel-my-project => <...my-project> <== Symlink to execRoot |
| 65 | bazel-out => <...bin> <== Convenience symlink to outputPath |
| 66 | bazel-bin => <...bin> <== Convenience symlink to most recent written bin dir $(BINDIR) |
| 67 | bazel-genfiles => <...genfiles> <== Convenience symlink to most recent written genfiles dir $(GENDIR) |
| 68 | |
| 69 | /home/user/.cache/bazel/ <== Root for all Bazel output on a machine: outputRoot |
| 70 | _bazel_$USER/ <== Top level directory for a given user depends on the user name: |
| 71 | outputUserRoot |
| 72 | install/ |
| 73 | fba9a2c87ee9589d72889caf082f1029/ <== Hash of the Bazel install manifest: installBase |
| 74 | _embedded_binaries/ <== Contains binaries and scripts unpacked from the data section of |
| 75 | the bazel executable on first run (e.g. helper scripts and the |
| 76 | main Java file BazelServer_deploy.jar) |
| 77 | 7ffd56a6e4cb724ea575aba15733d113/ <== Hash of the client's workspace directory (e.g. |
| 78 | /home/some-user/src/my-project): outputBase |
| 79 | action_cache/ <== Action cache directory hierarchy |
| 80 | This contains the persistent record of the file metadata |
| 81 | (timestamps, and perhaps eventually also MD5 sums) used by the |
| 82 | FilesystemValueChecker. |
| 83 | action_outs/ <== Action output directory. This contains a file with the |
| 84 | stdout/stderr for every action from the most recent bazel run |
| 85 | that produced output. |
| 86 | command.log <== A copy of the stdout/stderr output from the most recent bazel |
| 87 | command. |
| 88 | external/ <== The directory that remote repositories are downloaded/symlinked |
| 89 | into. |
| 90 | server/ <== The Bazel server puts all server-related files (such as socket |
| 91 | file, logs, etc) here. |
David Chen | 8fe82a3 | 2016-08-24 10:55:41 +0000 | [diff] [blame] | 92 | server.log <== Server logs. |
| 93 | <workspace-name>/ <== Working tree for the Bazel build & root of symlink forest: execRoot |
| 94 | _bin/ <== Helper tools are linked from or copied to here. |
| 95 | |
| 96 | bazel-out/ <== All actual output of the build is under here: outputPath |
| 97 | local_linux-fastbuild/ <== one subdirectory per unique target BuildConfiguration instance; |
| 98 | this is currently encoded |
| 99 | bin/ <== Bazel outputs binaries for target configuration here: $(BINDIR) |
| 100 | foo/bar/_objs/baz/ <== Object files for a cc_* rule named //foo/bar:baz |
| 101 | foo/bar/baz1.o <== Object files from source //foo/bar:baz1.cc |
| 102 | other_package/other.o <== Object files from source //other_package:other.cc |
| 103 | foo/bar/baz <== foo/bar/baz might be the artifact generated by a cc_binary named |
| 104 | //foo/bar:baz |
| 105 | foo/bar/baz.runfiles/ <== The runfiles symlink farm for the //foo/bar:baz executable. |
| 106 | MANIFEST |
| 107 | <workspace-name>/ |
| 108 | ... |
| 109 | genfiles/ <== Bazel puts generated source for the target configuration here: |
| 110 | $(GENDIR) |
| 111 | foo/bar.h e.g. foo/bar.h might be a headerfile generated by //foo:bargen |
| 112 | testlogs/ <== Bazel internal test runner puts test log files here |
| 113 | foo/bartest.log e.g. foo/bar.log might be an output of the //foo:bartest test with |
| 114 | foo/bartest.status foo/bartest.status containing exit status of the test (e.g. |
| 115 | PASSED or FAILED (Exit 1), etc) |
| 116 | include/ <== a tree with include symlinks, generated as needed. The |
| 117 | bazel-include symlinks point to here. This is used for |
| 118 | linkstamp stuff, etc. |
| 119 | host/ <== BuildConfiguration for build host (user's workstation), for |
| 120 | building prerequisite tools, that will be used in later stages |
| 121 | of the build (ex: Protocol Compiler) |
| 122 | <packages>/ <== Packages referenced in the build appear as if under a regular workspace |
| 123 | </pre> |
| 124 | |
| 125 | The layout of the *.runfiles directories is documented in more detail in the places pointed to by RunfilesSupport. |
| 126 | |
| 127 | ## `bazel clean` |
| 128 | |
| 129 | `bazel clean` does an `rm -rf` on the `outputPath` and the `action_cache` |
Laszlo Csomor | 836bda0 | 2016-09-19 16:42:39 +0000 | [diff] [blame] | 130 | directory. It also removes the workspace symlinks. The `--expunge` option |
| 131 | will clean the entire outputBase. |