blob: c4665ca6072079496365e925782eb1581987ac23 [file] [log] [blame]
---
layout: documentation
title: Aquery (action graph query)
---
<h1>Aquery (action graph query)</h1>
<h2 id='overview'>Overview</h2>
<p>
The <code>aquery</code> command allows you to query for actions in your build graph.
It operates on the post-analysis Configured Target Graph and exposes
information about <b>Actions, Artifacts and their relationships.</b>
</p>
<p>
<code>aquery</code> is useful when we are interested in the properties of the Actions/Artifacts
generated from the Configured Target Graph. For example, the actual commands run
and their inputs/outputs/mnemonics.
</p>
<p>
The tool accepts several command-line <a href="#aquery-options">options</a>.
Notably, the aquery command runs on top of a regular Bazel build and inherits
the set of options available during a build.
</p>
<p>
It supports the same set of functions that is also available to traditional
<code>query</code> but <code>siblings</code>, <code>buildfiles</code> and
<code>tests</code>.
</p>
<p>An example <code>aquery</code> output (without specific details):</p>
<pre>
$ bazel aquery 'deps(//some:label)'
action 'Writing file some_file_name'
Mnemonic: ...
Target: ...
Configuration: ...
ActionKey: ...
Inputs: [...]
Outputs: [...]
</pre>
<h2 id='basic-syntax'>Basic syntax</h2>
<p>A simple example of the syntax for <code>aquery</code> is as follows:</p>
<p><code>bazel aquery "aquery_function(function(//target))"</code></p>
<p>The query expression (in quotes) consists of the following:
<ul>
<li>
<code>aquery_function(...)</code>: functions specific to <code>aquery</code>.
More details <a href="#functions">below</a>.
</li>
<li>
<code>function(...)</code>: the standard <a href="query.html#functions">functions</a>
as traditional <code>query</code>.
</li>
<li>
<code>//target</code> is the label to the interested target.
</li>
</ul>
<pre>
# aquery examples:
# Get the action graph generated while building //src/target_a
$ bazel aquery '//src/target_a'
# Get the action graph generated while building all dependencies of //src/target_a
$ bazel aquery 'deps(//src/target_a)'
# Get the action graph generated while building all dependencies of //src/target_a
# whose inputs filenames match the regex ".*cpp".
$ bazel aquery 'inputs(".*cpp", deps(//src/target_a))'
</pre>
<h2 id='functions'>Aquery functions</h2>
<p>There are currently 3 <code>aquery</code> functions:</p>
<ul>
<li><code>inputs</code>: filter actions by inputs.</li>
<li><code>outputs</code>: filter actions by outputs</li>
<li><code>mnemonic</code>: filter actions by mnemonic</li>
</ul>
<p><code>expr ::= inputs(word, expr)</code></p>
<p>
The <code>inputs</code> operator returns the actions generated from building <code>expr</code>,
whose input filenames match the regex provided by <code>word</code>.
</p>
<p><code>$ bazel aquery 'inputs(".*cpp", deps(//src/target_a))'</code></p>
<p><code>outputs</code> and <code>mnemonic</code> functions share a similar syntax.</p>
<p>You can also combine functions to achieve the AND operation. For example:</p>
<pre>
$ bazel aquery 'mnemonic("Cpp.*", (inputs(".*cpp", inputs("foo.*", //src/target_a))))'
</pre>
<p>
The above command would find all actions involved in building <code>//src/target_a</code>,
whose mnemonics match <code>"Cpp.*"</code> and inputs match the patterns
<code>".*cpp"</code> and <code>"foo.*"</code>.
</p>
<h3>Important: aquery functions can't be nested inside non-aquery functions</h3>
<ul>
<li>Conceptually this makes sense since the output of aquery functions is Actions,
not Configured Targets.</li>
<li>
An example of the syntax error produced:
<pre>
$ bazel aquery 'deps(inputs(".*cpp", //src/target_a))'
ERROR: aquery filter functions (inputs, outputs, mnemonic) produce actions,
and therefore can't be the input of other function types: deps
deps(inputs(".*cpp", //src/target_a))
</pre>
</li>
</ul>
<h2 id='options'>Options</h2>
<h3>Build options</h3>
<p>
<code>aquery</code> runs on top of a regular Bazel build and thus inherits the set of
<a href="command-line-reference.html#build-options">options</a>
available during a build.
</p>
<h3 id="aquery-options">Aquery options</h3>
<h4><code class='flag'>--output=(text|proto|textproto|jsonproto), default=text</code></h4>
<p>
The default output format (<code>text</code>) is human-readable,
use <code>proto</code>, <code>textproto</code>, or <code>jsonproto</code> for machine-readable format.
</p>
<h4><code class='flag'>--include_commandline, default=true</code></h4>
<p>
Includes the content of the action command lines in the output (potentially large).
</p>
<h4><code class='flag'>--include_artifacts, default=true</code></h4>
<p>
Includes names of the action inputs and outputs in the output (potentially large).
</p>
<h4><code class='flag'>--include_aspects, default=false</code></h4>
<p>
Whether to include Aspect-generated actions in the output.
</p>
<h4><code class='flag'>--include_param_files, default=false</code></h4>
<p>
Include the content of the param files used in the command (potentially large).
Warning: Enabling this flag will automatically enable the <code>--include_commandline</code> flag.
</p>
<h4><code class='flag'>--skyframe_state, default=false</code></h4>
<p>
Without performing extra analysis, dump the current Action Graph from Skyframe.
Note: Specifying a target with <code>--skyframe_state</code> is currently not supported.
This flag is only available with <code>--output=proto</code> or <code>--output=textproto</code>.
</p>
<h2 id='misc'>Other tools and features</h2>
<h3 id="skyframe-state">Querying against the state of Skyframe</h3>
<p>
<a href="https://bazel.build/designs/skyframe.html">Skyframe</a> is the evaluation and
incrementality model of Bazel. On each instance of Bazel server, Skyframe stores the dependency graph
constructed from the previous runs of the <a href="guide.html#analysis-phase">Analysis phase</a>.
</p>
<p>
In some cases, it is useful to query the Action Graph currently on Skyframe.
An example use case would be:
</p>
<ol>
<li>Run <code>bazel build //target_a</code></li>
<li>Run <code>bazel build //target_b</code></li>
<li>File <code>foo.out</code> was generated.</li>
</ol>
<p>
<i>As a Bazel user, I want to determine if <code>foo.out</code> was generated from building
<code>//target_a</code> or <code>//target_b</code></i>.
</p>
<p>
One could run <code>bazel aquery 'outputs("foo.out", //target_a)'</code> and
<code>bazel aquery 'outputs("foo.out", //target_b)'</code> to figure out the action responsible
for creating <code>foo.out</code>, and in turn the target. However, the number of different
targets previously built can be larger than 2, which makes running multiple <code>aquery</code>
commands a hassle.
</p>
<p>As an alternative, the <code>--skyframe_state</code> flag can be used:</p>
<pre>
# List all actions on Skyframe's action graph
$ bazel aquery --output=proto --skyframe_state
# or
# List all actions on Skyframe's action graph, whose output matches "foo.out"
$ bazel aquery --output=proto --skyframe_state 'outputs("foo.out")'
</pre>
<p>
With <code>--skyframe_state</code> mode, <code>aquery</code> takes the content of the Action Graph
that Skyframe keeps on the current instance of Bazel, (optionally) performs filtering on it and
outputs the content, without re-running the analysis phase.
</p>
<h4>Special considerations</h4>
<h5>Output format</h5>
<p><code>--skyframe_state</code> is currently only available for <code>--output=proto</code>
and <code>--output=textproto</code></p>
<h5>Non-inclusion of target labels in the query expression</h5>
<p>
Currently, <code>--skyframe_state</code> queries the whole action graph that exists on Skyframe,
regardless of the targets. Having the target label specified in the query together with
<code>--skyframe_state</code> is considered a syntax error:
</p>
<pre>
# WRONG: Target Included
$ bazel aquery --output=proto --skyframe_state <b>//target_a</b>
ERROR: Error while parsing '//target_a)': Specifying build target(s) [//target_a] with --skyframe_state is currently not supported.
# WRONG: Target Included
$ bazel aquery --output=proto --skyframe_state 'inputs(".*.java", <b>//target_a</b>)'
ERROR: Error while parsing '//target_a)': Specifying build target(s) [//target_a] with --skyframe_state is currently not supported.
# CORRECT: Without Target
$ bazel aquery --output=proto --skyframe_state
$ bazel aquery --output=proto --skyframe_state 'inputs(".*.java")'
</pre>
<h3 id="diff-tool">Comparing aquery outputs</h3>
<p>
You can compare the outputs of two different aquery invocations using the <code>aquery_differ</code> tool.
For instance: when you make some changes to your rule definition and want to verify that the
command lines being run did not change. <code>aquery_differ</code> is the tool for that.
</p>
<p>
The tool is available in the <a href="https://github.com/bazelbuild/bazel/tree/master/tools/aquery_differ">bazelbuild/bazel</a> repository.
To use it, clone the repository to your local machine. An example usage:
</p>
<pre>
$ bazel run //tools/aquery_differ -- \
--before=/path/to/before.proto \
--after=/path/to/after.proto \
--input_type=proto \
--attrs=cmdline \
--attrs=inputs
</pre>
<p>
The above command returns the difference between the <code>before</code> and <code>after</code> aquery outputs:
which actions were present in one but not the other, which actions have different
command line/inputs in each aquery output, ...). The result of running the above command would be:
</p>
<pre>
Aquery output 'after' change contains an action that generates the following outputs that aquery output 'before' change doesn't:
...
/list of output files/
...
[cmdline]
Difference in the action that generates the following output(s):
/path/to/abc.out
--- /path/to/before.proto
+++ /path/to/after.proto
@@ -1,3 +1,3 @@
...
/cmdline diff, in unified diff format/
...
</pre>
<h4>Command options</h4>
<p><code class='flag'>--before, --after</code>: The aquery output files to be compared</p>
<p>
<code class='flag'>--input_type=(proto|text_proto), default=proto</code>: the format of the input
files. We currently support <code>proto</code> and <code>textproto</code> aquery output.
</p>
<p>
<code class='flag'>--attrs=(cmdline|inputs), default=cmdline</code>: the attributes of actions
to be compared.
</p>
<h3 id="aspect-on-aspect">Aspect-on-aspect</h3>
<p>
It is possible for <a href="https://docs.bazel.build/versions/master/skylark/aspects.html">Aspects</a>
to be applied on top of each other. The aquery output of the action generated by these Aspects would
then include the <i>Aspect path</i>, which is the sequence of Aspects applied to the target which generated the action.
</p>
<p>An example of Aspect-on-Aspect:</p>
<pre>
t0
^
| <- a1
t1
^
| <- a2
t2
</pre>
<p>
Let t<sub>i</sub> be a target of rule r<sub>i</sub>, which applies an Aspect a<sub>i</sub>
to its dependencies.
</p>
<p>Assume that a2 generates an action X when applied to target t0. The text output of
<code>bazel aquery --include_aspects 'deps(//t2)'</code> for action X would be:</p>
<pre>
action ...
Mnemonic: ...
Target: //my_pkg:t0
Configuration: ...
AspectDescriptors: [//my_pkg:rule.bzl%<b>a2</b>(foo=...)
-> //my_pkg:rule.bzl%<b>a1</b>(bar=...)]
...
</pre>
<p>
This means that action <code>X</code> was generated by Aspect <code>a2</code> applied onto
<code>a1(t0)</code>, where <code>a1(t0)</code> is the result of Aspect <code>a1</code> applied
onto target <code>t0</code>.
</p>
<p>Each <code>AspectDescriptor</code> has the following format:</p>
<pre>
AspectClass([param=value,...])
</pre>
<p>
<code>AspectClass</code> could be the name of the Aspect class (for native Aspects) or
<code>bzl_file%aspect_name</code> (for Starlark Aspects). <code>AspectDescriptor</code> are
sorted in topological order of the
<a href="https://docs.bazel.build/versions/master/skylark/aspects.html#aspect-basics">dependency graph</a>.
</p>
<h2 id='known-issues'>Known issues</h2>
<h3 id='shared-actions'>Handling Shared Actions</h3>
<p>
Sometimes actions are
<a href="https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/actions/Actions.java;l=59;drc=146d51aa1ec9dcb721a7483479ef0b1ac21d39f1">shared</a>
between configured targets.
In the execution phase, those shared actions are
<a href="https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/actions/Actions.java;l=241;drc=003b8734036a07b496012730964ac220f486b61f">simply considered as one</a> and only executed once.
However, aquery operates on the pre-execution, post-analysis action graph, and hence treats these
like separate actions whose output Artifacts have the exact same <code>execPath</code>. As a result,
we'll have equivalent Artifacts appearing duplicated.
</p>
<p>
The list of aquery issues/planned features can be found on
<a href="https://github.com/bazelbuild/bazel/labels/team-Performance">GitHub</a>.
</p>
<h2 id='faqs'>FAQs</h2>
<h3 id='action-key'>The ActionKey remains the same even though the content of an input file changed.</h3>
<p>
In the context of aquery, the <code>ActionKey</code> refers to the <code>String</code> gotten from
<code><a href="https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/actions/ActionAnalysisMetadata.java;l=89;drc=8b856f5484f0117b2aebc302f849c2a15f273310">ActionAnalysisMetadata#getKey</a></code>:
</p>
<pre>
Returns a string encoding all of the significant behaviour of this Action that might affect the
output. The general contract of <code>getKey</code> is this: if the work to be performed by the
execution of this action changes, the key must change.
...
Examples of changes that should affect the key are:
- Changes to the BUILD file that materially affect the rule which gave rise to this Action.
- Changes to the command-line options, environment, or other global configuration resources
which affect the behaviour of this kind of Action (other than changes to the names of the
input/output files, which are handled externally).
- An upgrade to the build tools which changes the program logic of this kind of Action
(typically this is achieved by incorporating a UUID into the key, which is changed each
time the program logic of this action changes).
Note the following exception: for actions that discover inputs, the key must change if any
input names change or else action validation may falsely validate.
</pre>
<p>
This excludes the changes to the content of the input files, and is not to be confused with
<code><a href="https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/remote/common/RemoteCacheClient.java;l=38;drc=21577f202eb90ce94a337ebd2ede824d609537b6">RemoteCacheClient#ActionKey</a></code>.
</p>
<h2 id='updates'>Updates</h2>
<p>
Please contact twerth@google.com and leba@google.com for any issue/feature request.
</p>