blob: 21c5b9caf3905bf588392ab61518a7a9699156a5 [file] [log] [blame] [view]
Project: /_project.yaml
Book: /_book.yaml
# Action Graph Query (aquery)
The `aquery` command allows you to query for actions in your build graph.
It operates on the post-analysis Configured Target Graph and exposes
information about **Actions, Artifacts and their relationships.**
`aquery` is useful when you are interested in the properties of the Actions/Artifacts
generated from the Configured Target Graph. For example, the actual commands run
and their inputs/outputs/mnemonics.
The tool accepts several command-line [options](#command-options).
Notably, the aquery command runs on top of a regular Bazel build and inherits
the set of options available during a build.
It supports the same set of functions that is also available to traditional
`query` but `siblings`, `buildfiles` and
`tests`.
An example `aquery` output (without specific details):
<pre>
$ bazel aquery 'deps(//some:label)'
action 'Writing file some_file_name'
Mnemonic: ...
Target: ...
Configuration: ...
ActionKey: ...
Inputs: [...]
Outputs: [...]
</pre>
## Basic syntax {:#basic-syntax}
A simple example of the syntax for `aquery` is as follows:
`bazel aquery "aquery_function(function(//target))"`
The query expression (in quotes) consists of the following:
* `aquery_function(...)`: functions specific to `aquery`.
More details [below](#using-aquery-functions).
* `function(...)`: the standard [functions](/reference/query#functions)
as traditional `query`.
* `//target` is the label to the interested target.
<pre>
# aquery examples:
# Get the action graph generated while building //src/target_a
$ bazel aquery '//src/target_a'
# Get the action graph generated while building all dependencies of //src/target_a
$ bazel aquery 'deps(//src/target_a)'
# Get the action graph generated while building all dependencies of //src/target_a
# whose inputs filenames match the regex ".*cpp".
$ bazel aquery 'inputs(".*cpp", deps(//src/target_a))'
</pre>
## Using aquery functions {:#using-aquery-functions}
There are three `aquery` functions:
* `inputs`: filter actions by inputs.
* `outputs`: filter actions by outputs
* `mnemonic`: filter actions by mnemonic
`expr ::= inputs(word, expr)`
The `inputs` operator returns the actions generated from building `expr`,
whose input filenames match the regex provided by `word`.
`$ bazel aquery 'inputs(".*cpp", deps(//src/target_a))'`
`outputs` and `mnemonic` functions share a similar syntax.
You can also combine functions to achieve the AND operation. For example:
<pre>
$ bazel aquery 'mnemonic("Cpp.*", (inputs(".*cpp", inputs("foo.*", //src/target_a))))'
</pre>
The above command would find all actions involved in building `//src/target_a`,
whose mnemonics match `"Cpp.*"` and inputs match the patterns
`".*cpp"` and `"foo.*"`.
Important: aquery functions can't be nested inside non-aquery functions.
Conceptually, this makes sense since the output of aquery functions is Actions,
not Configured Targets.
An example of the syntax error produced:
<pre>
$ bazel aquery 'deps(inputs(".*cpp", //src/target_a))'
ERROR: aquery filter functions (inputs, outputs, mnemonic) produce actions,
and therefore can't be the input of other function types: deps
deps(inputs(".*cpp", //src/target_a))
</pre>
## Options {:#options}
### Build options {:#build-options}
`aquery` runs on top of a regular Bazel build and thus inherits the set of
[options](/reference/command-line-reference#build-options)
available during a build.
### Aquery options {:#aquery-options}
#### `--output=(text|summary|proto|jsonproto|textproto), default=text` {:#output}
The default output format (`text`) is human-readable,
use `proto`, `textproto`, or `jsonproto` for machine-readable format.
The proto message is `analysis.ActionGraphContainer`.
#### `--include_commandline, default=true` {:#include-commandline}
Includes the content of the action command lines in the output (potentially large).
#### `--include_artifacts, default=true` {:#include-artifacts}
Includes names of the action inputs and outputs in the output (potentially large).
#### `--include_aspects, default=true` {:#include-aspects}
Whether to include Aspect-generated actions in the output.
#### `--include_param_files, default=false` {:#include-param-files}
Include the content of the param files used in the command (potentially large).
Warning: Enabling this flag will automatically enable the `--include_commandline` flag.
#### `--skyframe_state, default=false` {:#skyframe-state}
Without performing extra analysis, dump the Action Graph from Skyframe.
Note: Specifying a target with `--skyframe_state` is currently not supported.
This flag is only available with `--output=proto` or `--output=textproto`.
## Other tools and features {:#other-tools-features}
### Querying against the state of Skyframe {:#querying-against-skyframe}
[Skyframe](/reference/skyframe) is the evaluation and
incrementality model of Bazel. On each instance of Bazel server, Skyframe stores the dependency graph
constructed from the previous runs of the [Analysis phase](/docs/build#analysis).
In some cases, it is useful to query the Action Graph on Skyframe.
An example use case would be:
1. Run `bazel build //target_a`
2. Run `bazel build //target_b`
3. File `foo.out` was generated.
_As a Bazel user, I want to determine if `foo.out` was generated from building
`//target_a` or `//target_b`_.
One could run `bazel aquery 'outputs("foo.out", //target_a)'` and
`bazel aquery 'outputs("foo.out", //target_b)'` to figure out the action responsible
for creating `foo.out`, and in turn the target. However, the number of different
targets previously built can be larger than 2, which makes running multiple `aquery`
commands a hassle.
As an alternative, the `--skyframe_state` flag can be used:
<pre>
# List all actions on Skyframe's action graph
$ bazel aquery --output=proto --skyframe_state
# or
# List all actions on Skyframe's action graph, whose output matches "foo.out"
$ bazel aquery --output=proto --skyframe_state 'outputs("foo.out")'
</pre>
With `--skyframe_state` mode, `aquery` takes the content of the Action Graph
that Skyframe keeps on the instance of Bazel, (optionally) performs filtering on it and
outputs the content, without re-running the analysis phase.
#### Special considerations {:#special-considerations}
##### Output format {:#output-format}
`--skyframe_state` is currently only available for `--output=proto`
and `--output=textproto`
##### Non-inclusion of target labels in the query expression {:#target-labels-non-inclusion}
Currently, `--skyframe_state` queries the whole action graph that exists on Skyframe,
regardless of the targets. Having the target label specified in the query together with
`--skyframe_state` is considered a syntax error:
<pre>
# WRONG: Target Included
$ bazel aquery --output=proto --skyframe_state **//target_a**
ERROR: Error while parsing '//target_a)': Specifying build target(s) [//target_a] with --skyframe_state is currently not supported.
# WRONG: Target Included
$ bazel aquery --output=proto --skyframe_state 'inputs(".*.java", **//target_a**)'
ERROR: Error while parsing '//target_a)': Specifying build target(s) [//target_a] with --skyframe_state is currently not supported.
# CORRECT: Without Target
$ bazel aquery --output=proto --skyframe_state
$ bazel aquery --output=proto --skyframe_state 'inputs(".*.java")'
</pre>
### Comparing aquery outputs {:#comparing-aquery-outputs}
You can compare the outputs of two different aquery invocations using the `aquery_differ` tool.
For instance: when you make some changes to your rule definition and want to verify that the
command lines being run did not change. `aquery_differ` is the tool for that.
The tool is available in the [bazelbuild/bazel](https://github.com/bazelbuild/bazel/tree/master/tools/aquery_differ){: .external} repository.
To use it, clone the repository to your local machine. An example usage:
<pre>
$ bazel run //tools/aquery_differ -- \
--before=/path/to/before.proto \
--after=/path/to/after.proto \
--input_type=proto \
--attrs=cmdline \
--attrs=inputs
</pre>
The above command returns the difference between the `before` and `after` aquery outputs:
which actions were present in one but not the other, which actions have different
command line/inputs in each aquery output, ...). The result of running the above command would be:
<pre>
Aquery output 'after' change contains an action that generates the following outputs that aquery output 'before' change doesn't:
...
/list of output files/
...
[cmdline]
Difference in the action that generates the following output(s):
/path/to/abc.out
--- /path/to/before.proto
+++ /path/to/after.proto
@@ -1,3 +1,3 @@
...
/cmdline diff, in unified diff format/
...
</pre>
#### Command options {:#command-options}
`--before, --after`: The aquery output files to be compared
`--input_type=(proto|text_proto), default=proto`: the format of the input
files. Support is provided for `proto` and `textproto` aquery output.
`--attrs=(cmdline|inputs), default=cmdline`: the attributes of actions
to be compared.
### Aspect-on-aspect {:#aspect-on-aspect}
It is possible for [Aspects](/rules/aspects)
to be applied on top of each other. The aquery output of the action generated by
these Aspects would then include the _Aspect path_, which is the sequence of
Aspects applied to the target which generated the action.
An example of Aspect-on-Aspect:
<pre>
t0
^
| <- a1
t1
^
| <- a2
t2
</pre>
Let t<sub>i</sub> be a target of rule r<sub>i</sub>, which applies an Aspect a<sub>i</sub>
to its dependencies.
Assume that a2 generates an action X when applied to target t0. The text output of
`bazel aquery --include_aspects 'deps(//t2)'` for action X would be:
<pre>
action ...
Mnemonic: ...
Target: //my_pkg:t0
Configuration: ...
AspectDescriptors: [//my_pkg:rule.bzl%**a2**(foo=...)
-> //my_pkg:rule.bzl%**a1**(bar=...)]
...
</pre>
This means that action `X` was generated by Aspect `a2` applied onto
`a1(t0)`, where `a1(t0)` is the result of Aspect `a1` applied
onto target `t0`.
Each `AspectDescriptor` has the following format:
<pre>
AspectClass([param=value,...])
</pre>
`AspectClass` could be the name of the Aspect class (for native Aspects) or
`bzl_file%aspect_name` (for Starlark Aspects). `AspectDescriptor` are
sorted in topological order of the
[dependency graph](rules/aspects#aspect_basics).
### Linking with the JSON profile {:#linking-with-json-profile}
While aquery provides information about the actions being run in a build (why they're being run,
their inputs/outputs), the [JSON profile](/rules/performance#performance-profiling)
tells us the timing and duration of their execution.
It is possible to combine these 2 sets of information via a common denominator: an action's primary output.
To include actions' outputs in the JSON profile, generate the profile with
`--experimental_include_primary_output --noexperimental_slim_json_profile`.
Slim profiles are incompatible with the inclusion of primary outputs. An action's primary output
is included by default by aquery.
We don't currently provide a canonical tool to combine these 2 data sources, but you should be
able to build your own script with the above information.
## Known issues {:#known-issues}
### Handling shared actions {:#handling-shared-actions}
Sometimes actions are
[shared](https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/actions/Actions.java;l=59;drc=146d51aa1ec9dcb721a7483479ef0b1ac21d39f1){: .external}
between configured targets.
In the execution phase, those shared actions are
[simply considered as one](https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/actions/Actions.java;l=241;drc=003b8734036a07b496012730964ac220f486b61f){: .external} and only executed once.
However, aquery operates on the pre-execution, post-analysis action graph, and hence treats these
like separate actions whose output Artifacts have the exact same `execPath`. As a result,
equivalent Artifacts appear duplicated.
The list of aquery issues/planned features can be found on
[GitHub](https://github.com/bazelbuild/bazel/labels/team-Performance){: .external}.
## FAQs {:#faqs}
### The ActionKey remains the same even though the content of an input file changed. {:#actionkey-same}
In the context of aquery, the `ActionKey` refers to the `String` gotten from
`[ActionAnalysisMetadata#getKey](https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/actions/ActionAnalysisMetadata.java;l=89;drc=8b856f5484f0117b2aebc302f849c2a15f273310){: .external}`:
<pre>
Returns a string encoding all of the significant behaviour of this Action that might affect the
output. The general contract of `getKey` is this: if the work to be performed by the
execution of this action changes, the key must change.
...
Examples of changes that should affect the key are:
- Changes to the BUILD file that materially affect the rule which gave rise to this Action.
- Changes to the command-line options, environment, or other global configuration resources
which affect the behaviour of this kind of Action (other than changes to the names of the
input/output files, which are handled externally).
- An upgrade to the build tools which changes the program logic of this kind of Action
(typically this is achieved by incorporating a UUID into the key, which is changed each
time the program logic of this action changes).
Note the following exception: for actions that discover inputs, the key must change if any
input names change or else action validation may falsely validate.
</pre>
This excludes the changes to the content of the input files, and is not to be confused with
`[RemoteCacheClient#ActionKey](https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/remote/common/RemoteCacheClient.java;l=38;drc=21577f202eb90ce94a337ebd2ede824d609537b6){: .external}`.
## Updates {:#updates}
For any issues/feature requests, please file an issue [here](https://github.com/bazelbuild/bazel/issues/new).