tree 3818f6d3e89ffff5981ac05e0d06097aac635e24
parent a617a901265b62bfdd41f762e589ce12b67a3b08
author buchgr <buchgr@google.com> 1574244072 -0800
committer Copybara-Service <copybara-worker@google.com> 1574244136 -0800

Support remote execution in repository_rule

Motivation:

This change adds support for running a command on a remote machine from a
repository_rule and getting its stdout, stderr and exit code back. The
initial use case of this feature is to allow a repository_rule to inspect
a remote machine, discover the installed tools, libraries and environment
in order to generate Bazel platforms and toolchains for the remote
environment.

The upload of files as part of the command and download of its outputs is
not supported. The use cases looked at so far don't have a need for it.
For example, Bazel's own cc_configure rule only uses one file as part of
ctx.execute(). The contents of the file are "int main() {}". There the
lack of file inputs can simply be worked around with
"echo 'int main() {}' > empty.cc && gcc empty.cc". I am not aware of a
use case that would require downloading output files.

How to Use:

This feature is not on by default. It can be enabled with
--experimental_repository_remote_exec.

repository_rule gains an additional attribute remotable. If true any
repository_ctx.execute() call may be executed on a remote worker. That
is, if Bazel is configured build/test with remote execution so is
repository_ctx.execute():

def _cc_configure_impl(ctx):
  result = ctx.execute(["/bin/bash", "-c", "export"])
  print("stdout: " + result.stdout)
  print("stderr: " + result.stderr)
  print("exit code: " + result.return_code)

cc_configure = repository_rule(
  implementation = _cc_configure_impl,
  remotable = True,
)

Note: The 'remotable' attribute is set by rule author as any such rule needs
to be carefully crafted to work with both local and remote execution.

The remote platform of repository_ctx.execute() is specified via the newly
introduced exec_properties attribute on the repository_rule target:

load(..., "cc_configure")

cc_configure(
  name = "remote_cc_configure",
  exec_properties = {
    "OSFamily" : "Linux",
    "ISA" : "x86-64"
  }
)

The intended usage is for the repository_rule to generate the Bazel
platform and toolchains for the remote platform that the build/test
actions will be run on:

bazel build \
  --experimental_repository_remote_exec \
  --platforms=@remote_cc_configure//:platform \
  --remote_executor=remote.example.com \
  ...

If --remote_executor is not specified then repository_ctx.execute()
is executed on the local/host machine (just like before this change).

RELNOTES: None
PiperOrigin-RevId: 281476515