olaola | a5e36ee | 2018-06-15 08:16:44 -0700 | [diff] [blame] | 1 | # Execution Log Parser |
| 2 | |
Googler | 2d60e01 | 2024-10-01 01:52:32 -0700 | [diff] [blame] | 3 | This tool is used to inspect and parse the Bazel execution logs. Currently |
| 4 | supported formats are `binary`, `json`, and `compact`. |
| 5 | |
olaola | a5e36ee | 2018-06-15 08:16:44 -0700 | [diff] [blame] | 6 | To generate the execution log, run e.g.: |
| 7 | |
| 8 | bazel build \ |
Googler | 2d60e01 | 2024-10-01 01:52:32 -0700 | [diff] [blame] | 9 | --execution_log_compact_file=/tmp/exec.log :hello_world |
olaola | a5e36ee | 2018-06-15 08:16:44 -0700 | [diff] [blame] | 10 | |
Googler | 2d60e01 | 2024-10-01 01:52:32 -0700 | [diff] [blame] | 11 | Then build the parser and run it: |
olaola | a5e36ee | 2018-06-15 08:16:44 -0700 | [diff] [blame] | 12 | |
Googler | b93f4b8 | 2024-02-02 07:40:04 -0800 | [diff] [blame] | 13 | bazel build src/tools/execlog:parser |
olaola | a5e36ee | 2018-06-15 08:16:44 -0700 | [diff] [blame] | 14 | bazel-bin/src/tools/execlog/parser --log_path=/tmp/exec.log |
| 15 | |
| 16 | This will simply print the log contents to stdout in text form. |
Googler | 943c17b | 2018-08-20 09:45:53 -0700 | [diff] [blame] | 17 | |
| 18 | To output results to a file, use `--output_path`: |
| 19 | |
| 20 | bazel-bin/src/tools/execlog/parser --log_path=/tmp/exec.log \ |
| 21 | --output_path=/tmp/exec.log.txt |
| 22 | |
| 23 | To limit the output to a certain runner, use `--restrict_to_runner` option. |
| 24 | For example, |
| 25 | |
| 26 | bazel-bin/src/tools/execlog/parser --log_path=/tmp/exec.log \ |
| 27 | --restrict_to_runner="linux-sandbox" |
| 28 | |
| 29 | Will limit the output to those actions that were ran in the linux sandbox. |
| 30 | |
| 31 | |
| 32 | Note that because Bazel is nondeterministic, different runs of the same build |
| 33 | may produce logs where actions are in a different order. To achieve a more |
| 34 | meaningful textual diff, use the parser to convert both files at the same time: |
| 35 | |
| 36 | bazel-bin/src/tools/execlog/parser --log_path=/tmp/exec1.log \ |
| 37 | --log_path=/tmp/exec2.log \ |
| 38 | --output_path=/tmp/exec1.log.txt \ |
| 39 | --output_path=/tmp/exec2.log.txt |
| 40 | |
| 41 | This will convert `/tmp/exec1.log` to text as-is, but will reorder `/tmp/exec2.log` |
| 42 | to match the order of actions found in `/tmp/exec1.log`. Actions are matched if |
| 43 | their first output is the same. Actions that are not found in `/tmp/exec1.log` |
| 44 | are put at the end of `/tmp/exec2.log.txt`. |
| 45 | |
| 46 | Note that this reordering makes it easier to see differences using text-based |
| 47 | diffing tools, but may break the logical sequence of actions in |
| 48 | `/tmp/exec2.log.txt`. |
Googler | b93f4b8 | 2024-02-02 07:40:04 -0800 | [diff] [blame] | 49 | |
| 50 | # Execution Log Converter |
| 51 | |
| 52 | This tool is used to convert between Bazel execution log formats. |
Googler | d8c606c | 2024-06-21 06:29:42 -0700 | [diff] [blame] | 53 | Currently supported formats are `binary`, `json`, and `compact`. |
Googler | b93f4b8 | 2024-02-02 07:40:04 -0800 | [diff] [blame] | 54 | |
| 55 | For example, to convert from the binary format to the JSON format: |
| 56 | |
| 57 | bazel build src/tools/execlog:converter |
| 58 | bazel-bin/src/tools/execlog/converter \ |
| 59 | --input binary:/tmp/binary.log --output json:/tmp/json.log |
| 60 | |
| 61 | By default, the output will be in the same order as the input. To sort in a |
Googler | d8c606c | 2024-06-21 06:29:42 -0700 | [diff] [blame] | 62 | deterministic order, use `--sort`. |
| 63 | |
| 64 | For large log files, you might need to increase the JVM heap size. To do so, |
| 65 | pass a corresponding `--jvm_flag`, for example `--jvm_flag=-Xmx4g` for 4GB of heap. |
| 66 | |