commit eb349967c8f11bad10db7fe2526bd3c60bca7ee7
author arostovtsev <arostovtsev@google.com> Mon Sep 20 11:49:59 2021 -0700
committer Copybara-Service <copybara-worker@google.com> Mon Sep 20 11:51:14 2021 -0700
tree 3475bc7fedbd2ebae97e54c6bbaa7b050b5e10ef
parent e49f557ef0c700297fddd9800f28ecffc99ed6b5

Allow native.existing_rule and native.existing_rules to return a lightweight view.

Experimental implementation of
https://github.com/bazelbuild/proposals/blob/main/designs/2021-06-15-improving-native.existing_rules.md

Gated behind the --experimental_existing_rules_immutable_view flag. See
https://github.com/bazelbuild/bazel/issues/13907

To check the performance impact, I queried a toy package that in a loop, creates
a genrule and then on each loop iteration performs one of the following tasks:

* checks if rule with a given name existed in `existing_rules()` ("in")
* for each rule in `existing_rules()`, checks if an attribute with a given name exists ("shallow iteration")
* for 50% of rules in `existing_rules()`, checks if a particular attribute value exists under any name ("50% deep iteration");
* iterates every rule name, attribute name, and attribute value ("deep iteration");

Query time in seconds without --experimental_existing_rules_immutable_view:

```
n	in	shallow 50%deep	deep
1000	2.73	2.84	3.36	3.61
2000	8.42	9.07	10.97	11.62
4000	30.95	33.81	40.88	44.17
```

With --experimental_existing_rules_immutable_view:

```
n	in	shallow 50%deep	deep
1000	0.40	0.64	2.58	4.19
2000	0.43	1.08	7.96	13.64
4000	0.51	2.59	28.83	52.99
```

In other words, in the unusual case where we examine every attribute value
of every existing rule, the overhead of managing immutable views would cause
a 15-20% slowdown. In any other situation, where we don't need to materialize
every rule's attribute values, the immutable view approach yields a
substantial performance improvement.

We'd want to see if we can reduce the 15-20% worst-case penalty in followup
work.

RELNOTES: Adds --experimental_existing_rules_immutable_view flag to make the
native.existing_rule and native.existing_rules functions more efficient by
returning immutable, lightweight dict-like view objects instead of mutable
dicts.
PiperOrigin-RevId: 397805096