commit f20b33c968ff7b5bdfbb5676aa730a5f9c5e4012
author ulfjack <ulfjack@google.com> Thu Jan 30 07:36:30 2020 -0800
committer Copybara-Service <copybara-worker@google.com> Thu Jan 30 07:37:44 2020 -0800
tree faaac08c381728f3529ae99e94c1fc01294cfc2c
parent 9f7534025cbe5de6c78f203c8d416ccf1a15ad95

Add flag to replay stdout/stderr across builds

This adds an experimental flag, --experimental_replay_action_out_err,
which changes Bazel's behavior to replay action stdout/stderr on
subsequent builds, even if the action is cached in Skyframe or in the
action cache.

I made 4 changes for this to work:

1. Add a new directory, bazel-out/_action/ which contains persistent
   stdout/stderr files, which use the action key as the name (so the names
   are also deterministic across server restarts).

2. Report the action output as a warning rather than an info message
   (info messages are not cached by Skyframe but discarded after being
   reported once).

3. Report the warning message to the Skyframe event listener, rather
   than the global reporter.

4. Replay the warning message on a cache hit.

There are a few shortcomings of this approach:

1. It switches the entire ActionExecutionContext to the Skyframe event
   listener if the new flag is set, which could also cause replay for
   other events that are posted to this listener as long as the action
   is a Skyframe hit. When changing configuration options or restarting
   the server, these events are dropped as we currently have no way of
   persisting them in the action cache.

2. Its interaction with action rewinding is not ideal. Action rewinding
   can rerun actions after their stdout/stderr have already been posted
   to the CLI.

   When an action is re-executed after rewinding, all events from that
   action are suppressed, and the empty set is recorded in Skyframe,
   replacing the previous (possibly non-empty) set of events.

   At this time, action rewinding is primarily used in contexts where
   this is not a concern.

3. If the stdout/stderr files are locally modified, then Bazel replays
   the modified files. It might be nicer to have Bazel rerun the
   corresponding actions in this case by recording the stdout/stderr
   hashes in the action cache and checking them on incremental builds.

4. Flipping the flag on/off invalidates neither the Skyframe nor the
   on-disk action cache, and can therefore cause replay of stale or no
   events.

   As a workaround, the flag should be set in the bazelrc. Note that
   some tools may invoke bazel without the bazelrc, which can cause this
   problem to occur even so.

This was highly requested for Apple/iOS development to make tool
warnings more visible to developers, and this implementation may be
sufficient for that use case.

Changes to the output filter are taken into account. However, some
actions opt-in to unconditionally report their output, which is not
supported if the flag is enabled. Currently, that only applies to test
actions.

Personally, I think action out/err replay should be enabled by default.
I find the current behavior highly unintuitive because the visibility of
warning messages depends on the state of the various caches which are
intentionally opaque. That would also address (4) above.

PiperOrigin-RevId: 292344740