commit f3e99412bbc7e0eff5e3e37734f255e4fd9054cf
author nharmata <nharmata@google.com> Mon Jan 28 10:06:29 2019 -0800
committer Copybara-Service <copybara-piper@google.com> Mon Jan 28 10:08:23 2019 -0800
tree c2c0c8af493fd5f71f23f25eaaec652031d2a908
parent a4b2d053661f263d0d3ca6c6ecb2ef114659d8ee

Dedupe ImmutableLists created for attribute values at the Starlark -> Blaze barrier during BUILD file evaluation.

We dedupe across all lists across all attributes of all rule instances in a single package. It's very common for multiple rules to have attributes that happen to have the same value. Blaze currently unconditionally creates a shallow copy of each list-typed attribute value, which means there's not even a way for users to write optimized BUILD/bzl code. Assume we have

  def f(x):
    ...
    # assume 'f' has no side effects

and then consider the following BUILD/bzl code

  sh_library(name = 't1', srcs = ['f1.sh'], deps = f(42))
  sh_library(name = 't2', srcs = ['f2.sh'], deps = f(42))

A thoughtful user might hope/assume Blaze optimizes this and only evaluates 'f(42)' once and only has one copy of the result of 'f(42)' in memory. Both of these thoughts are false.

So the thoughtful user might then write this "optimized" code instead

  L = f(42)
  sh_library(name = 't1', srcs = ['f1.sh'], deps = L)
  sh_library(name = 't2', srcs = ['f2.sh'], deps = L)

here, 'f(42)' will be evaluated once. but Blaze internally still has two copies of L!

There are some good reasons that Blaze unconditionally creates copies of lists. One of them is that Starlark lists are mutable. Consider

  L = f(42)
  g(L) # assume 'g' has side-effects, e.g. rule instantiation
  L.append('blah')
  g(L)

Here, the first call to 'g' definitely needs to use the initial value of L.

But, not all BUILD/bzl code is like this and it's possibly to statically determine we're in the situation where L is *not* mutated [1], and so therefore it's possible to automatically share L in an optimizing manner.

[1] To be clear, I mean "it's possible to statically determine which of the 2 following situations we're in: (i) L is *definitely not* mutated or (ii) L *might be* mutated". This entails things like alias analysis too.

RELNOTES: None
PiperOrigin-RevId: 231239991