juliexxia | 548c395 | 2018-03-15 09:23:34 -0700 | [diff] [blame] | 1 | --- |
| 2 | layout: documentation |
| 3 | title: Cquery (configurable query) |
| 4 | --- |
| 5 | |
| 6 | <h1>Cquery (Configurable Query)</h1> |
| 7 | |
| 8 | <ul class="toc"> |
| 9 | <li><a href="#overview">Overview</a></li> |
| 10 | <li><a href="#basic-syntax">Basic Syntax</a></li> |
| 11 | <li><a href="#functions">Functions</a></li> |
| 12 | <li><a href="#options">Options</a></li> |
| 13 | <li><a href="#known-issues">Known Issues</a></li> |
| 14 | <li><a href="#updates">Updates</a></li> |
| 15 | </ul> |
| 16 | |
| 17 | <h2 id='overview'>Overview</h2> |
| 18 | |
| 19 | <p> |
| 20 | <code>cquery</code> is a command complementary to <code>query</code> that properly handles |
| 21 | configurations. While <code>cquery</code> returns answers that are more correct, it does not |
| 22 | replace <code>query</code> as it does not support all of <code>query</code>'s functions.</p> |
| 23 | <p> |
| 24 | <code>cquery</code> achieves configuration awareness by running after the |
| 25 | <a href="https://docs.bazel.build/versions/master/skylark/concepts.html#evaluation-model">analysis phase</a> |
| 26 | of a build, unlike traditional <code>query</code>, which runs after the loading phase.</p> |
| 27 | <p> |
| 28 | The most common use for <code>cquery</code> is obtaining answers that correctly evaluate |
| 29 | <a href="https://docs.bazel.build/versions/master/be/common-definitions.html#configurable-attributes"><code>select()</code></a> |
| 30 | statements in configurable attributes. For example: |
| 31 | </p> |
| 32 | |
| 33 | <pre> |
| 34 | $ cat > tree/BUILD <<EOF |
| 35 | sh_library( |
| 36 | name = "ash", |
| 37 | deps = select({ |
| 38 | ":excelsior": [":manna-ash"], |
| 39 | ":americana": [":white-ash"], |
| 40 | "//conditions:default": [":common-ash"], |
| 41 | }), |
| 42 | ) |
| 43 | sh_library(name = "manna-ash") |
| 44 | sh_library(name = "white-ash") |
| 45 | sh_library(name = "common-ash") |
| 46 | config_setting( |
| 47 | name = "excelsior", |
| 48 | values = {"define": "species=excelsior"}, |
| 49 | ) |
| 50 | config_setting( |
| 51 | name = "americana", |
| 52 | values = {"define": "species=americana"}, |
| 53 | ) |
| 54 | EOF |
| 55 | |
| 56 | #Traditional query |
| 57 | $ bazel query "deps(//tree:ash)" --define species=excelsior --noimplicit_deps |
| 58 | //tree:ash |
| 59 | //tree:white-ash |
| 60 | //tree:manna-ash |
| 61 | //tree:common-ash |
| 62 | |
| 63 | #cquery |
| 64 | $ bazel cquery "deps(//tree:ash)" --define species=excelsior --noimplicit_deps |
| 65 | //tree:ash (hash-of-config) |
| 66 | //tree:manna-ash (hash-of-config) |
| 67 | </pre> |
| 68 | |
| 69 | |
| 70 | <p> |
| 71 | Keep in mind that <code>cquery</code> works only on the configured target graph. Because of this, |
| 72 | it does not have insight into artifacts like build actions nor access to |
| 73 | <code><a href="https://docs.bazel.build/versions/master/be/general.html#test_suite">test_suite</a></code> |
| 74 | rules as they are not configured targets. |
| 75 | </p> |
| 76 | |
| 77 | <h2 id='basic-syntax'>Basic Syntax</h2> |
| 78 | |
| 79 | <p>A simple example of the syntax for <code>cquery</code> is as follows:</p> |
| 80 | |
| 81 | <p><code>bazel cquery "function(//target)"</code></p> |
| 82 | |
| 83 | <p>The query expression (in quotes) consists of the following: |
| 84 | |
| 85 | <ul> |
| 86 | <li> |
| 87 | <b><code>function(...)</code></b> is the function to run on the target. <code>cquery</code> |
| 88 | supports most of the same <a href="https://docs.bazel.build/versions/master/query.html#functions">functions</a> |
| 89 | as traditional <code>query</code>. |
| 90 | </li> |
| 91 | <li><b><code>//target</code></b> is the expression fed to the function. In this example, the |
| 92 | expression is a simple target, but the query language also allows nesting of functions. |
| 93 | See the <a href="query-how-to.md">Query How-To</a> for examples. |
| 94 | </li> |
| 95 | </ul> |
| 96 | |
| 97 | <p> |
| 98 | Note that <code>cquery</code> requires a target on which to run the |
| 99 | loading and analysis phases. Unless otherwise specified, <code>cquery</code> parses |
| 100 | the target(s) listed in the query expression. See the <code>--universe_scope</code> |
| 101 | option below for querying targets built under other targets. |
| 102 | </p> |
| 103 | |
| 104 | <h2 id='functions'>Functions</h2> |
| 105 | |
| 106 | <p> |
| 107 | Of the <a href="https://docs.bazel.build/versions/master/query.html#functions" title="list of query functions">set of functions</a> |
| 108 | supported by the traiditional <code>query</code>, <code>cquery</code> supports all but siblings, buildfiles, and tests. The |
| 109 | functions listed below are <code>cquery</code>-specific. |
| 110 | </p> |
| 111 | |
| 112 | <h3>config</h3> |
| 113 | |
| 114 | <p><code>expr ::= config(expr, word)</code></p> |
| 115 | |
| 116 | <p> |
| 117 | The <code>config</code> operator attempts to return the result of the first argument, configured |
| 118 | in the configuration specified by the second argument. Today, the second argument can only take in |
| 119 | three options <code>target</code>, <code>host</code>, or <code>null</code> but we hope to expand |
| 120 | this functionality to be able to input custom configurations (or custom configuration diffs from |
| 121 | the default target configuration). |
| 122 | </p> |
| 123 | |
| 124 | <p><code>$ bazel cquery config(//foo, host) --universe_scope=//bar</code></p> |
| 125 | |
| 126 | <p> |
| 127 | Note that the above example query will return <code>//foo</code> configured in the host configuration |
| 128 | <em>if and only if</em> <code>//foo</code> exists in the host configuration in the universe of |
| 129 | this query (which we've set to the transitive closure of <code>//bar</code>). |
| 130 | </p> |
| 131 | |
| 132 | <p> |
| 133 | If not all results of the first argument can be found in the specified |
| 134 | configuration, then only those that can be found are returned. If no results of the |
| 135 | first argument can be found in the specified configuration, an error is thrown. |
| 136 | </p> |
| 137 | |
| 138 | <h2 id='options'>Options</h2> |
| 139 | |
| 140 | <h3>Build Options</h3> |
| 141 | |
| 142 | <p> |
| 143 | <code>cquery</code> runs on top of a regular Bazel build and thus inherits the set of |
| 144 | <a href="https://docs.bazel.build/versions/master/command-line-reference.html#build-options">options</a> |
| 145 | available during a build. |
| 146 | </p> |
| 147 | |
| 148 | <h3>Cquery options</h3> |
| 149 | |
| 150 | <h4><code>--universe_scope</code> (comma-separated list)</h4> |
| 151 | |
| 152 | <p> |
| 153 | Often, the dependencies of configured targets go through |
| 154 | <a href="https://docs.bazel.build/versions/master/skylark/rules.html#configurations">transitions</a>, |
| 155 | which causes their configuration to differ from their dependent. This flag allows you to query |
| 156 | a target as if it were built as a dependency or a transitive dependency of another target. For |
| 157 | example: |
| 158 | </p> |
| 159 | |
| 160 | <pre> |
| 161 | # x/BUILD |
| 162 | genrule( |
| 163 | name = "my_gen", |
| 164 | srcs = ["x.in"], |
| 165 | outs = ["x.cc"], |
| 166 | cmd = "$(locations :tool) $< >$@", |
| 167 | tools = [":tool"], |
| 168 | ) |
| 169 | cc_library( |
| 170 | name = "tool", |
| 171 | ) |
| 172 | </pre> |
| 173 | |
| 174 | <p> |
| 175 | Genrules configure their tools in the |
| 176 | <a href="https://docs.bazel.build/versions/master/skylark/rules.html#configurations">host configuration</a> |
| 177 | so the following queries would produce the following outputs: |
| 178 | </p> |
| 179 | |
| 180 | <table class="table table-condensed table-bordered table-params"> |
| 181 | <thead> |
| 182 | <tr> |
| 183 | <th>Query</th> |
| 184 | <th>Target Built</th> |
| 185 | <th>Output</th> |
| 186 | </tr> |
| 187 | </thead> |
| 188 | <tbody> |
| 189 | <tr> |
| 190 | <td>bazel cquery "//x:tool"</td> |
| 191 | <td>//x:tool</td> |
| 192 | <td>//x:tool(targetconfig)</td> |
| 193 | </tr> |
| 194 | <tr> |
| 195 | <td>bazel cquery "//x:tool" --universe_scope="//x:my_gen"</td> |
| 196 | <td>//x:my_gen</td> |
| 197 | <td>//x:tool(hostconfig)</td> |
| 198 | </tr> |
| 199 | </tbody> |
| 200 | </table> |
| 201 | |
| 202 | <p> |
| 203 | If this flag is set, its contents are built. <em>If it's not set, all targets |
| 204 | mentioned in the query expression are built</em> instead. The transitive closure of the |
| 205 | built targets are used as the universe of the query. Either way, the targets to |
| 206 | be built must be buildable at the top level (that is, compatible with top-level |
| 207 | options). <code>cquery</code> returns results in the transitive closure of these |
| 208 | top-level targets. |
| 209 | </p> |
| 210 | |
| 211 | <p> |
| 212 | Even if it's possible to build all targets in a query expression at the top |
| 213 | level, it may be beneficial to not do so. For example, explicitly setting |
| 214 | <code>--universe_scope</code> could prevent building targets multiple times in |
| 215 | configurations you don't care about. It could also help specify which configuration version of a |
| 216 | target you're looking for (since it's not currently possible |
| 217 | to fully specify this any other way). We recommend that you set this |
| 218 | flag if your query expression is more complex than <code>deps(//foo)</code>. |
| 219 | </p> |
| 220 | |
| 221 | <h4><code>--implicit_deps</code> (boolean, default=True)</h4> |
| 222 | |
| 223 | <p> |
| 224 | Setting this flag to false filters out all results that aren't explicitly set in |
| 225 | the BUILD file and instead set elsewhere by Bazel. |
| 226 | </p> |
| 227 | |
| 228 | <h4><code>--host_deps</code> (boolean, default=True)</h4> |
| 229 | |
| 230 | <p> |
| 231 | Setting this flag to false filters out all configured targets for which the |
| 232 | path from the queried target to them crosses a transition between the target |
| 233 | configuration and the |
| 234 | <a href="https://docs.bazel.build/versions/master/skylark/rules.html#configurations">host configuration</a>. |
| 235 | If the queried target is in a non-host configuration, setting <code>--nohost_deps</code> will |
| 236 | only return targets that also are in non-host configurations. If the queried |
| 237 | target is in a host configuration, setting <code>--nohost_deps</code> will only return |
| 238 | targets also in the host configuration. |
| 239 | </p> |
| 240 | |
juliexxia | 931c182 | 2018-03-27 14:24:25 -0700 | [diff] [blame] | 241 | <h2 id='output-formats'>Output Formats</h2> |
| 242 | |
| 243 | <p> By default, cquery outputs results in a dependency-ordered list of label and configuration pairs. |
| 244 | There are other options for exposing the results as well. </p> |
| 245 | |
| 246 | <h3> Transitions </h3> |
| 247 | |
| 248 | <p> |
| 249 | Configuration <a href="https://docs.bazel.build/versions/master/skylark/rules.html#configurations">transitions</a> |
| 250 | are used to build targets underneath the top level targets in different configurations than the top |
| 251 | level targets. For example, a target might impose a transition to the host transition on all |
| 252 | dependencies in its <code>tools</code> attribute. These are known as attribute transitions. |
| 253 | Rules can also impose transitions on their own configurations, known as rule class transitions. |
| 254 | This output format outputs information about these transitions such as what type they are and the |
| 255 | effect they have on build options. |
| 256 | </p> |
| 257 | |
| 258 | <p>This output format is triggered by the <code>--transitions</code> flag which by default is set to |
| 259 | <code>NONE</code>. It can be set to <code>FULL</code> or <code>LITE</code> mode. <code>FULL</code> |
| 260 | mode outputs information about rule class transitions and attribute transitions including a detailed |
| 261 | diff of the options before and after the transition. <code>LITE</code> mode outputs the same information |
| 262 | without the options diff. |
| 263 | </p> |
| 264 | |
juliexxia | 548c395 | 2018-03-15 09:23:34 -0700 | [diff] [blame] | 265 | <h2 id='known-issues'>Known Issues</h2> |
| 266 | |
| 267 | <ul> |
| 268 | <li> |
| 269 | <strong>Inaccessible configurations.</strong> With the exception of the host configuration being |
| 270 | printed as <code>HOST</code>, Configurations are currently output as |
| 271 | hashes and there is no way for the user to input them (and thus to directly |
| 272 | specify the configuration to query a target in).</li> |
| 273 | <li> |
| 274 | <strong>No support for <a href="https://docs.bazel.build/versions/master/skylark/aspects.html">aspects</a>, |
| 275 | <a href="https://docs.bazel.build/versions/master/query.html#output-formats" title="list of query output formats">output |
| 276 | options</a>, or recursive target patterns (/...).</strong></li> |
| 277 | <li> |
| 278 | <strong>Non-deterministic output.</strong> <code>Cquery</code> does not automatically wipe the build |
| 279 | graph from previous commands and is therefore prone to picking up results |
| 280 | from past queries. For example, <code>genquery</code> exerts a host transition on its <code>tools</code> |
| 281 | attribute - that is, it configures its tools in the |
| 282 | <a href="https://docs.bazel.build/versions/master/skylark/rules.html#configurations">host configuration</a>. |
| 283 | We can see the lingering effects of that transition below.</li> |
| 284 | </ul> |
| 285 | |
| 286 | <pre> |
| 287 | $ cat > foo/BUILD <<<EOF |
| 288 | genrule( |
| 289 | name = "my_gen", |
| 290 | srcs = ["x.in"], |
| 291 | outs = ["x.cc"], |
| 292 | cmd = "$(locations :tool) $< >$@", |
| 293 | tools = [":tool"], |
| 294 | ) |
| 295 | cc_library( |
| 296 | name = "tool", |
| 297 | ) |
| 298 | EOF |
| 299 | |
| 300 | $ bazel cquery "//foo:tool" |
| 301 | tool(target_config) |
| 302 | |
| 303 | $ bazel cquery "deps(//foo:my_gen)" |
| 304 | my_gen (target_config) |
| 305 | tool (host_config) |
| 306 | ... |
| 307 | |
| 308 | $ bazel cquery "//foo:tool" |
| 309 | tool(host_config) |
| 310 | </pre> |
| 311 | |
| 312 | <p> |
| 313 | Workaround: change any startup option to force re-analysis of configured targets. For example, |
| 314 | add <code>--test_arg=<whatever></code> to your build command. |
| 315 | </p> |
| 316 | |
| 317 | <h2 id='updates'>Updates</h2> |
| 318 | |
| 319 | <p> |
| 320 | The Bazel configurability team is continously improving <code>cquery</code>. If you want to |
| 321 | ask questions, stay updated, or get involved, contact juliexxia@google.com |
| 322 | </p> |