| // Copyright 2015 The Bazel Authors. All rights reserved. |
| // |
| // Licensed under the Apache License, Version 2.0 (the "License"); |
| // you may not use this file except in compliance with the License. |
| // You may obtain a copy of the License at |
| // |
| // http://www.apache.org/licenses/LICENSE-2.0 |
| // |
| // Unless required by applicable law or agreed to in writing, software |
| // distributed under the License is distributed on an "AS IS" BASIS, |
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| // See the License for the specific language governing permissions and |
| // limitations under the License. |
| package com.google.devtools.build.lib.syntax; |
| |
| import com.google.common.base.Preconditions; |
| import com.google.common.collect.Iterables; |
| import com.google.devtools.build.lib.events.Location; |
| import java.io.Serializable; |
| import java.util.ArrayList; |
| import java.util.Formattable; |
| import java.util.IdentityHashMap; |
| import java.util.List; |
| |
| /** |
| * An object that manages the capability to mutate Skylark objects and their {@link Environment}s. |
| * Collectively, the managed objects are called {@link Freezable}s. |
| * |
| * <p>Each {@code Environment}, and each of the mutable Skylark values (i.e., {@link |
| * SkylarkMutable}s) that are created in that {@code Environment}, holds a pointer to the same |
| * {@code Mutability} instance. Once the {@code Environment} is done evaluating, its {@code |
| * Mutability} is irreversibly closed ("frozen"). At that point, it is no longer possible to change |
| * either the bindings in that {@code Environment} or the state of its objects. This protects each |
| * {@code Environment} from unintentional and unsafe modification. |
| * |
| * <p>{@code Mutability}s enforce isolation between {@code Environment}s; it is illegal for an |
| * evaluation in one {@code Environment} to affect the bindings or values of another. In particular, |
| * the {@code Environment} for any Skylark module is frozen before its symbols can be imported for |
| * use by another module. Each individual {@code Environment}'s evaluation is single-threaded, so |
| * this isolation also translates to thread safety. Any number of threads may simultaneously access |
| * frozen data. (The {@code Mutability} itself is also thread-safe if and only if it is frozen.} |
| * |
| * <p>Although the mutability pointer of a {@code Freezable} contains some debugging information |
| * about its context, this should not affect the {@code Freezable}'s semantics. From a behavioral |
| * point of view, the only thing that matters is whether the {@code Mutability} is frozen, not what |
| * particular {@code Mutability} object is pointed to. |
| * |
| * <p>A {@code Mutability} also tracks which {@code Freezable} objects in its {@code Environment} |
| * are temporarily locked from mutation. This is used to prevent modification of iterables during |
| * loops. A {@code Freezable} may be locked multiple times (e.g., nested loops over the same |
| * iterable). Locking an object does not prohibit mutating its deeply contained values, such as in |
| * the case of a list of lists. |
| * |
| * <p>We follow two disciplines to ensure safety. First, all mutation methods of a {@code Freezable} |
| * must take in a {@code Mutability} as a parameter, and confirm that |
| * <ol> |
| * <li>the {@code Freezable} is not yet frozen, |
| * <li>the given {@code Mutability} matches the one referred to by the {@code Freezable}, and |
| * <li>the {@code Freezable} is not locked. |
| * </ol> |
| * It is a high-level error ({@link MutabilityException}, which gets translated to {@link |
| * EvalException}) to attempt to modify a frozen or locked value. But it is a low-level error |
| * ({@link IllegalArgumentException}) to attempt to modify a value using the wrong {@link |
| * Mutability} instance, since the user shouldn't be able to trigger this situation under normal |
| * circumstances. |
| * |
| * <p>Second, {@code Mutability}s are created using the try-with-resource style: |
| * <pre>{@code |
| * try (Mutability mutability = Mutability.create(fmt, ...)) { ... } |
| * }</pre> |
| * The general pattern is to create a {@code Mutability}, build an {@code Environment}, mutate that |
| * {@code Environment} and its objects, and possibly return the result from within the {@code try} |
| * block, relying on the try-with-resource construct to ensure that everything gets frozen before |
| * the result is used. The only code that should create a {@code Mutability} without using |
| * try-with-resource is test code that is not part of the Bazel jar. |
| * |
| * <p>We keep some (unchecked) invariants regarding where {@code Mutability} objects may appear |
| * within a compound value. |
| * <ol> |
| * <li>A compound value can never contain an unfrozen {@code Mutability} for any {@code |
| * Environment} except the one currently being evaluated. |
| * <li>If a value has the special {@link #IMMUTABLE} {@code Mutability}, all of its contents are |
| * themselves deeply immutable too (i.e. have frozen {@code Mutability}s). |
| * <li>If a value has the special {@link #SHALLOW_IMMUTABLE} {@code Mutability}, its contents may |
| * or may not be mutable. |
| * </ol> |
| * It follows that, if these invariants hold, an unfrozen value cannot appear as the child of a |
| * value whose {@code Mutability} is already frozen, unless this {@code Mutability} is the special |
| * {@code #SHALLOW_IMMUTABLE} instance. This knowledge is used by {@link SkylarkMutable#isImmutable} |
| * to prune traversals of a compound value. |
| * |
| * <p>There is a special API for freezing individual values rather than whole {@code Environment}s. |
| * Because this API makes it easier to violate the above invariants, you should avoid using it if at |
| * all possible; at the moment it is only used for serialization. Under this API, you may call |
| * {@link Freezable#unsafeShallowFreeze} to reset a value's {@code Mutability} pointer to be {@link |
| * #IMMUTABLE}. This operation has no effect on the {@code Mutability} itself. It is up to the |
| * caller to preserve or restore the above invariants by ensuring that any deeply contained values |
| * are also frozen. For safety and explicitness, this operation is disallowed unless the {@code |
| * Mutability}'s {@link #allowsUnsafeShallowFreeze} method returns true. |
| */ |
| public final class Mutability implements AutoCloseable, Serializable { |
| |
| /** |
| * If true, mutation of any {@link Freezable} associated with this {@code Mutability} is |
| * disallowed. |
| */ |
| private boolean isFrozen; |
| |
| /** |
| * For each locked {@link Freezable}, stores all {@link Location}s where it is locked. |
| * |
| * This field is set null once the {@code Mutability} is closed. This saves some space, and avoids |
| * a concurrency bug from multiple Skylark modules accessing the same {@code Mutability} at once. |
| */ |
| private IdentityHashMap<Freezable, List<Location>> lockedItems; |
| |
| /** For error reporting; a name for the context in which this {@code Mutability} is used. */ |
| private final Formattable annotation; |
| |
| /** Controls access to {@link Freezable#unsafeShallowFreeze}. */ |
| private final boolean allowsUnsafeShallowFreeze; |
| |
| private Mutability(Formattable annotation, boolean allowsUnsafeShallowFreeze) { |
| this.isFrozen = false; |
| // Seems unlikely that we'll often lock more than 10 things at once. |
| this.lockedItems = new IdentityHashMap<>(10); |
| this.annotation = Preconditions.checkNotNull(annotation); |
| this.allowsUnsafeShallowFreeze = allowsUnsafeShallowFreeze; |
| } |
| |
| /** |
| * Creates a {@code Mutability}. |
| * |
| * @param pattern is a {@link Printer#format} pattern used to lazily produce a string name |
| * for error reporting |
| * @param arguments are the optional {@link Printer#format} arguments to produce that string |
| */ |
| public static Mutability create(String pattern, Object... arguments) { |
| return new Mutability( |
| Printer.formattable(pattern, arguments), |
| /*allowsUnsafeShallowFreeze=*/ false); |
| } |
| |
| /** |
| * Creates a {@code Mutability} whose objects can be individually frozen; see docstrings for |
| * {@link Mutability} and {@link Freezable#unsafeShallowFreeze}. |
| */ |
| public static Mutability createAllowingShallowFreeze(String pattern, Object... arguments) { |
| return new Mutability( |
| Printer.formattable(pattern, arguments), |
| /*allowsUnsafeShallowFreeze=*/ true); |
| } |
| |
| public String getAnnotation() { |
| return annotation.toString(); |
| } |
| |
| @Override |
| public String toString() { |
| return String.format(isFrozen ? "(%s)" : "[%s]", annotation); |
| } |
| |
| public boolean isFrozen() { |
| return isFrozen; |
| } |
| |
| /** |
| * Return whether a {@link Freezable} belonging to this {@code Mutability} is currently locked. |
| * Frozen objects are not considered locked, though they are of course immutable nonetheless. |
| * |
| * @throws IllegalArgumentException if the {@code Freezable} does not belong to this {@code |
| * Mutability} |
| */ |
| public boolean isLocked(Freezable object) { |
| Preconditions.checkArgument(object.mutability().equals(this), |
| "trying to check the lock of an object from a different context"); |
| if (isFrozen) { |
| return false; |
| } |
| return lockedItems.containsKey(object); |
| } |
| |
| /** |
| * For a locked {@link Freezable} that belongs to this {@code Mutability}, return a List of the |
| * {@link Location}s corresponding to its current locks. |
| * |
| * @throws IllegalArgumentException if the {@code Freezable} does not belong to this {@code |
| * Mutability} |
| */ |
| public List<Location> getLockLocations(Freezable object) { |
| Preconditions.checkArgument(isLocked(object), |
| "trying to get lock locations for an object that is not locked"); |
| return lockedItems.get(object); |
| } |
| |
| /** |
| * Add a lock on a {@link Freezable} belonging to this {@code Mutability}. The object cannot be |
| * mutated until all locks on it are gone. For error reporting purposes each lock is |
| * associated with its originating {@link Location}. |
| * |
| * @throws IllegalArgumentException if the {@code Freezable} does not belong to this {@code |
| * Mutability} |
| */ |
| public void lock(Freezable object, Location loc) { |
| Preconditions.checkArgument(object.mutability().equals(this), |
| "trying to lock an object from a different context"); |
| if (isFrozen) { |
| return; |
| } |
| List<Location> locList; |
| if (!lockedItems.containsKey(object)) { |
| locList = new ArrayList<>(); |
| lockedItems.put(object, locList); |
| } else { |
| locList = lockedItems.get(object); |
| } |
| locList.add(loc); |
| } |
| |
| /** |
| * Remove the lock for a given {@link Freezable} that is associated with the given {@link |
| * Location}. |
| * |
| * @throws IllegalArgumentException if the object does not belong to this {@code Mutability}, or |
| * if the object has no lock corresponding to {@code loc} |
| */ |
| public void unlock(Freezable object, Location loc) { |
| Preconditions.checkArgument(object.mutability().equals(this), |
| "trying to unlock an object from a different context"); |
| if (isFrozen) { |
| // It's okay if we somehow got frozen while there were still locked objects. |
| return; |
| } |
| Preconditions.checkArgument(lockedItems.containsKey(object), |
| "trying to unlock an object that is not locked"); |
| |
| List<Location> locList = lockedItems.get(object); |
| if (!locList.remove(loc)) { |
| throw new IllegalArgumentException( |
| Printer.format( |
| "trying to unlock an object for a location at which it was not locked (%r)", loc)); |
| } |
| if (locList.isEmpty()) { |
| lockedItems.remove(object); |
| } |
| } |
| |
| /** |
| * Freezes this {@code Mutability}, rendering all {@link Freezable} objects that refer to it |
| * immutable. |
| * |
| * Note that freezing does not directly touch all the {@code Freezables}, so this operation is |
| * constant-time. |
| * |
| * @return this object, in the fluent style |
| */ |
| public Mutability freeze() { |
| // No need to track per-Freezable info since everything is immutable now. |
| lockedItems = null; |
| isFrozen = true; |
| return this; |
| } |
| |
| @Override |
| public void close() { |
| freeze(); |
| } |
| |
| /** |
| * Returns whether {@link Freezable}s having this {@code Mutability} allow the {@link |
| * #unsafeShallowFreeze} operation. |
| */ |
| public boolean allowsUnsafeShallowFreeze() { |
| return allowsUnsafeShallowFreeze; |
| } |
| |
| /** Indicates an illegal attempt to mutate a frozen or locked {@link Freezable}. */ |
| static class MutabilityException extends Exception { |
| MutabilityException(String message) { |
| super(message); |
| } |
| } |
| |
| /** |
| * An object that refers to a {@link Mutability} to decide whether to allow mutation. All |
| * {@link Freezable} Skylark objects created within a given {@link Environment} will share the |
| * same {@code Mutability} as that {@code Environment}. |
| */ |
| public interface Freezable { |
| /** |
| * Returns the {@link Mutability} associated with this {@code Freezable}. This should not change |
| * over the lifetime of the object, except by calling {@link #shallowFreeze} if applicable. |
| */ |
| Mutability mutability(); |
| |
| /** |
| * Freezes this object (and not its contents). Use with care. |
| * |
| * <p>This method is optional (i.e. may throw {@link NotImplementedException}). |
| * |
| * <p>If this object's {@link Mutability} is 1) not frozen, and 2) has {@link |
| * #allowUnsafeShallowFreeze} return true, then the object's {@code Mutability} reference is |
| * updated to point to {@link #IMMUTABLE}. Otherwise, this method throws {@link |
| * IllegalArgumentException}. |
| * |
| * <p>It is up to the caller to ensure that any contents of this {@code Freezable} are also |
| * frozen in order to preserve/restore the invariant that an immutable value cannot contain a |
| * mutable one unless the immutable value's {@code Mutability} is {@link #SHALLOW_IMMUTABLE}. |
| * Note that {@link SkylarkMutable#isImmutable} correctness and thread-safety are not guaranteed |
| * otherwise. |
| */ |
| default void unsafeShallowFreeze() { |
| throw new UnsupportedOperationException(); |
| } |
| |
| /** |
| * Throws {@link IllegalArgumentException} if the precondition for {@link #unsafeShallowFreeze} |
| * is violated. To be used by implementors of {@link #unsafeShallowFreeze}. |
| */ |
| static void checkUnsafeShallowFreezePrecondition( |
| Freezable freezable) { |
| Mutability mutability = freezable.mutability(); |
| if (mutability.isFrozen()) { |
| // It's not safe to rewrite the Mutability pointer if this is already frozen, because we |
| // could be accessed by multiple threads. |
| throw new IllegalArgumentException( |
| "cannot call unsafeShallowFreeze() on an object whose Mutability is already frozen"); |
| } |
| if (!mutability.allowsUnsafeShallowFreeze()) { |
| throw new IllegalArgumentException( |
| "cannot call unsafeShallowFreeze() on a mutable object whose Mutability's " |
| + "allowsUnsafeShallowFreeze() == false"); |
| } |
| } |
| } |
| |
| /** |
| * Checks that the given {@code Freezable} can be mutated using the given {@code Mutability}, and |
| * throws an exception if it cannot. |
| * |
| * @throws MutabilityException if the object is either frozen or locked |
| * @throws IllegalArgumentException if the given {@code Mutability} is not the same as the one |
| * the {@code Freezable} is associated with |
| */ |
| public static void checkMutable(Freezable object, Mutability mutability) |
| throws MutabilityException { |
| if (object.mutability().isFrozen()) { |
| // Throw MutabilityException, not IllegalArgumentException, even if the object was from |
| // another context. |
| throw new MutabilityException("trying to mutate a frozen object"); |
| } |
| |
| // Consider an {@link Environment} e1, in which is created {@link UserDefinedFunction} f1, that |
| // closes over some variable v1 bound to list l1. If somehow, via the magic of callbacks, f1 or |
| // l1 is passed as an argument to some function f2 evaluated in {@link Environment} e2 while e1 |
| // is still mutable, then e2, being a different {@link Environment}, should not be allowed to |
| // mutate objects from e1. It's a bug, that shouldn't happen in our current code base, so we |
| // throw an IllegalArgumentException. If in the future such situations are allowed to happen, |
| // then we should throw a MutabilityException instead. |
| if (!object.mutability().equals(mutability)) { |
| throw new IllegalArgumentException("trying to mutate an object from a different context"); |
| } |
| |
| if (mutability.isLocked(object)) { |
| Iterable<String> locs = |
| Iterables.transform(mutability.getLockLocations(object), Location::print); |
| throw new MutabilityException( |
| "trying to mutate a locked object (is it currently being iterated over by a for loop " |
| + "or comprehension?)\n" |
| + "Object locked at the following location(s): " |
| + String.join(", ", locs)); |
| } |
| } |
| |
| /** |
| * A {@code Mutability} indicating that a value is deeply immutable. |
| * |
| * <p>It is not associated with any particular {@link Environment}. |
| */ |
| public static final Mutability IMMUTABLE = create("IMMUTABLE").freeze(); |
| |
| /** |
| * A {@code Mutability} indicating that a value is shallowly immutable. |
| * |
| * <p>Under the invariants for this class, this is the only frozen {@code Mutability} whose values |
| * are permitted to directly or indirectly contain mutable values. |
| * |
| * <p>In practice, this instance is used as the {@code Mutability} for tuples. |
| */ |
| // TODO(bazel-team): We might be able to remove this instance, and instead have tuples and other |
| // immutable types store the same Mutability as other values in that environment. Then we can |
| // simplify the Mutability invariant, and implement deep-immutability checking in constant time |
| // for values whose Environments have been frozen. |
| // |
| // This would also affect structs (SkylarkInfo). Maybe they would implement an interface similar |
| // to SkylarkMutable, or the relevant methods could be worked into SkylarkValue. |
| public static final Mutability SHALLOW_IMMUTABLE = create("SHALLOW_IMMUTABLE").freeze(); |
| } |