tree 39a5c30e1b44919930d4f7fe981276babe5377a7
parent 6e54699884cfad49d4e8f6dd59a4050bc95c4edf
author Philipp Schrader <philipp.schrader@gmail.com> 1652973977 -0700
committer GitHub <noreply@github.com> 1652973977 -0500
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsBcBAABCAAQBQJihmGZCRBK7hj4Ov3rIwAAlpEIADwImgVohttd5urKO99xlCpz
 QLI8jv+98eOLTR5aes52mxoIMFzxYglniu3LQwjIpdbFMhHWAPj+XcfFdYnKCi7n
 xxUreSyOtAmdqp5btLiR1cGPfgTy8i1xTL4Yw/R5AkUh0t3IHHm1nl+9kTQz8b03
 0qyxstgN2wlHvZi6Sy9ccDvk9hYbp3h/yDLdihgH2b11FZxnwSGNNqnce9TgqbMe
 PqVB15Ogkb1DpX5yP5Yj4Am9wAK6qM8iGiqHGfmvnPZx81ADLZILlrqrNkZw8fom
 bD/DjkMq+PV1L+H1bX7oNWBFxP3nonJIjxM2ISaKkVOQ+IKqAcPke9HBooX7UfI=
 =eVFC
 -----END PGP SIGNATURE-----
 

Update PythonZipper action to use CommandLineItem.CapturingMapFn (#15472)

### Summary

This PR attempts to address https://github.com/bazelbuild/bazel/issues/14890.

This updates the PythonZipper action to use a `CommandLineItem.CapturingMapFn` to defer expanding arguments until after analysis. I used `CapturingMapFn`, as it seemed like the only way to go about implementing the arg filtering that is done by the current code to exclude the `executable` and `zipFile` artifacts.

### Test Plan

Use the example from https://github.com/bazelbuild/bazel/issues/14890 of looking at memory usage in tensorflow before/after this patch is applied on top of version `5.0.0`

Initial Setup
```
git clone git@github.com:tensorflow/tensorflow.git
cd tensorflow
python3 -m venv venv
source venv/bin/activate
pip install numpy
```

View memory usage at 5.0.0:
```bash
# STARTUP_FLAGS from https://bazel.build/rules/performance#enabling-memory-tracking
$ bazel ${STARTUP_FLAGS[@]} build --nobuild //tensorflow/python/...
$ bazel ${STARTUP_FLAGS[@]} dump --rules | egrep '(RULE|py_)'
RULE                                 COUNT     ACTIONS          BYTES         EACH
py_test                              1,714      15,390    413,338,632      241,154
py_library                           1,102           0      2,097,152        1,903
py_binary                               33         198      8,914,840      270,146
py_runtime                               6           0              0            0
py_runtime_pair                          3           0              0            0
_concat_flatbuffer_py_srcs               2           2              0            0

$ bazel ${STARTUP_FLAGS[@]} info used-heap-size-after-gc
635MB
```

View memory usage at 5.0.0 with this patch applied:
```bash
$ ~/code/bazel/bazel-bin/src/bazel ${STARTUP_FLAGS[@]} dump --rules | egrep '(RULE|py_)'
RULE                                 COUNT     ACTIONS          BYTES         EACH
py_test                              1,714      15,390     65,323,312       38,111
py_library                           1,102           0      2,359,576        2,141
py_binary                               33         198        524,400       15,890
py_runtime                               6           0              0            0
py_runtime_pair                          3           0              0            0
_concat_flatbuffer_py_srcs               2           2              0            0

$ ~/code/bazel/bazel-bin/src/bazel ${STARTUP_FLAGS[@]} info used-heap-size-after-gc
283MB
```

Ensure that the generated actions have not changed:
```bash
$ bazel aquery --build_python_zip 'mnemonic("PythonZipper", //tensorflow/python/...)' > /tmp/bazel_5_out
$ ~/code/bazel/bazel-bin/src/bazel aquery --build_python_zip 'mnemonic("PythonZipper", //tensorflow/python/...)' > /tmp/bazel_5_patched_out
$ diff /tmp/bazel_5_out /tmp/bazel_5_patched_out
# note: no diff output
```

Closes #15037.

PiperOrigin-RevId: 441257695
(cherry picked from commit 9610ae889e6fd45280c5beb7fe8f5bef2d736878)

Co-authored-by: Alex Torok <alex.torok@aurora.tech>