site/en/external/extension.md

Project: /_project.yaml
Book: /_book.yaml

# Module extensions

{% include "_buttons.html" %}

Module extensions allow users to extend the module system by reading input data
from modules across the dependency graph, performing necessary logic to resolve
dependencies, and finally creating repos by calling repo rules. These extensions
have capabilities similar to repo rules, which enables them to perform file I/O,
send network requests, and so on. Among other things, they allow Bazel to
interact with other package management systems while also respecting the
dependency graph built out of Bazel modules.

You can define module extensions in `.bzl` files, just like repo rules. They're
not invoked directly; rather, each module specifies pieces of data called *tags*
for extensions to read. Bazel runs module resolution before evaluating any
extensions. The extension reads all the tags belonging to it across the entire
dependency graph.

## Extension usage

Extensions are hosted in Bazel modules themselves. To use an extension in a
module, first add a `bazel_dep` on the module hosting the extension, and then
call the [`use_extension`](/rules/lib/globals/module#use_extension) built-in function
to bring it into scope. Consider the following example — a snippet from a
`MODULE.bazel` file to use the "maven" extension defined in the
[`rules_jvm_external`](https://github.com/bazelbuild/rules_jvm_external){:.external}
module:

```python
bazel_dep(name = "rules_jvm_external", version = "4.5")
maven = use_extension("@rules_jvm_external//:extensions.bzl", "maven")
```

This binds the return value of `use_extension` to a variable, which allows the
user to use dot-syntax to specify tags for the extension. The tags must follow
the schema defined by the corresponding *tag classes* specified in the
[extension definition](#extension_definition). For an example specifying some
`maven.install` and `maven.artifact` tags:

```python
maven.install(artifacts = ["org.junit:junit:4.13.2"])
maven.artifact(group = "com.google.guava",
               artifact = "guava",
               version = "27.0-jre",
               exclusions = ["com.google.j2objc:j2objc-annotations"])
```

Use the [`use_repo`](/rules/lib/globals/module#use_repo) directive to bring repos
generated by the extension into the scope of the current module.

```python
use_repo(maven, "maven")
```

Repos generated by an extension are part of its API. In this example, the
"maven" module extension promises to generate a repo called `maven`. With the
declaration above, the extension properly resolves labels such as
`@maven//:org_junit_junit` to point to the repo generated by the "maven"
extension.

Note: Module extensions are evaluated lazily. This means that an extension will
typically not be evaluated unless some module brings one of its repositories
into scope using `use_repo` and that repository is referenced in a build. While
testing a module extension, `bazel mod deps` can be useful as it
unconditionally evaluates all module extensions.

## Extension definition

You can define module extensions similarly to repo rules, using the
[`module_extension`](/rules/lib/globals/bzl#module_extension) function. However,
while repo rules have a number of attributes, module extensions have
[`tag_class`es](/rules/lib/globals/bzl#tag_class), each of which has a number of
attributes. The tag classes define schemas for tags used by this extension. For
example, the "maven" extension above might be defined like this:

```python
# @rules_jvm_external//:extensions.bzl

_install = tag_class(attrs = {"artifacts": attr.string_list(), ...})
_artifact = tag_class(attrs = {"group": attr.string(), "artifact": attr.string(), ...})
maven = module_extension(
  implementation = _maven_impl,
  tag_classes = {"install": _install, "artifact": _artifact},
)
```

These declarations show that `maven.install` and `maven.artifact` tags can be
specified using the specified attribute schema.

The implementation function of module extensions are similar to those of repo
rules, except that they get a [`module_ctx`](/rules/lib/builtins/module_ctx) object,
which grants access to all modules using the extension and all pertinent tags.
The implementation function then calls repo rules to generate repos.

```python
# @rules_jvm_external//:extensions.bzl

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file")  # a repo rule
def _maven_impl(ctx):
  # This is a fake implementation for demonstration purposes only

  # collect artifacts from across the dependency graph
  artifacts = []
  for mod in ctx.modules:
    for install in mod.tags.install:
      artifacts += install.artifacts
    artifacts += [_to_artifact(artifact) for artifact in mod.tags.artifact]

  # call out to the coursier CLI tool to resolve dependencies
  output = ctx.execute(["coursier", "resolve", artifacts])
  repo_attrs = _process_coursier_output(output)

  # call repo rules to generate repos
  for attrs in repo_attrs:
    http_file(**attrs)
  _generate_hub_repo(name = "maven", repo_attrs)
```

### Extension identity

Module extensions are identified by the name and the `.bzl` file that appears
in the call to `use_extension`. In the following example, the extension `maven`
is identified by the `.bzl` file `@rules_jvm_external//:extension.bzl` and the
name `maven`:

```python
maven = use_extension("@rules_jvm_external//:extensions.bzl", "maven")
```

Re-exporting an extension from a different `.bzl` file gives it a new identity
and if both versions of the extension are used in the transitive module graph,
then they will be evaluated separately and will only see the tags associated
with that particular identity.

As an extension author you should make sure that users will only use your
module extension from one single `.bzl` file.

## Repository names and visibility

Repos generated by extensions have canonical names in the form of `{{ "<var>"
}}module_repo_canonical_name{{ "</var>" }}~{{ "<var>" }}extension_name{{
"</var>" }}~{{ "<var>" }}repo_name{{ "</var>" }}`. For extensions hosted in the
root module, the `{{ "<var>" }}module_repo_canonical_name{{ "</var>" }}` part is
replaced with the string `_main`. Note that the canonical name format is not an
API you should depend on — it's subject to change at any time.

This naming policy means that each extension has its own "repo namespace"; two
distinct extensions can each define a repo with the same name without risking
any clashes. It also means that `repository_ctx.name` reports the canonical name
of the repo, which is *not* the same as the name specified in the repo rule
call.

Taking repos generated by module extensions into consideration, there are
several repo visibility rules:

*   A Bazel module repo can see all repos introduced in its `MODULE.bazel` file
    via [`bazel_dep`](/rules/lib/globals/module#bazel_dep) and
    [`use_repo`](/rules/lib/globals/module#use_repo).
*   A repo generated by a module extension can see all repos visible to the
    module that hosts the extension, *plus* all other repos generated by the
    same module extension (using the names specified in the repo rule calls as
    their apparent names).
    *   This might result in a conflict. If the module repo can see a repo with
        the apparent name `foo`, and the extension generates a repo with the
        specified name `foo`, then for all repos generated by that extension
        `foo` refers to the former.
*   Similarly, in a module extension's implementation function, repos created
    by the extension can refer to each other by their apparent names in
    attributes, regardless of the order in which they are created.
    *   In case of a conflict with a repository visible to the module, labels
        passed to repository rule attributes can be wrapped in a call to
        [`Label`](/rules/lib/toplevel/attr#label) to ensure that they refer to
        the repo visible to the module instead of the extension-generated repo
        of the same name.

### Overriding and injecting module extension repos

The root module can use
[`override_repo`](/rules/lib/globals/module#override_repo) and
[`inject_repo`](/rules/lib/globals/module#inject_repo) to override or inject
module extension repos.

#### Example: Replacing `rules_java`'s `java_tools` with a vendored copy

```python
# MODULE.bazel
local_repository = use_repo_rule("@bazel_tools//tools/build_defs/repo:local.bzl", "local_repository")
local_repository(
  name = "my_java_tools",
  path = "vendor/java_tools",
)

bazel_dep(name = "rules_java", version = "7.11.1")
java_toolchains = use_extension("@rules_java//java:extension.bzl", "toolchains")

override_repo(java_toolchains, remote_java_tools = "my_java_tools")
```

#### Example: Patch a Go dependency to depend on `@zlib` instead of the system zlib

```python
# MODULE.bazel
bazel_dep(name = "gazelle", version = "0.38.0")
bazel_dep(name = "zlib", version = "1.3.1.bcr.3")

go_deps = use_extension("@gazelle//:extensions.bzl", "go_deps")
go_deps.from_file(go_mod = "//:go.mod")
go_deps.module_override(
  patches = [
    "//patches:my_module_zlib.patch",
  ],
  path = "example.com/my_module",
)
use_repo(go_deps, ...)

inject_repo(go_deps, "zlib")
```

```diff
# patches/my_module_zlib.patch
--- a/BUILD.bazel
+++ b/BUILD.bazel
@@ -1,6 +1,6 @@
 go_binary(
     name = "my_module",
     importpath = "example.com/my_module",
     srcs = ["my_module.go"],
-    copts = ["-lz"],
+    cdeps = ["@zlib"],
 )
```

## Best practices

This section describes best practices when writing extensions so they are
straightforward to use, maintainable, and adapt well to changes over time.

### Put each extension in a separate file

When extensions are in a different files, it allows one extension to load
repositories generated by another extension. Even if you don't use this
functionality, it's best to put them in separate files in case you need it
later. This is because the extension's identify is based on its file, so moving
the extension into another file later changes your public API and is a backwards
incompatible change for your users.

### Specify reproducibility

If your extension always defines the same repositories given the same inputs
(extension tags, files it reads, etc.) and in particular doesn't rely on
any [downloads](/rules/lib/builtins/module_ctx#download) that aren't guarded by
a checksum, consider returning
[`extension_metadata`](/rules/lib/builtins/module_ctx#extension_metadata) with
`reproducible = True`. This allows Bazel to skip this extension when writing to
the lockfile.

### Specify the operating system and architecture

If your extension relies on the operating system or its architecture type,
ensure to indicate this in the extension definition using the `os_dependent`
and `arch_dependent` boolean attributes. This ensures that Bazel recognizes the
need for re-evaluation if there are changes to either of them.

Since this kind of dependence on the host makes it more difficult to maintain
the lockfile entry for this extension, consider
[marking the extension reproducible](#specify_reproducibility) if possible.

### Only the root module should directly affect repository names

Remember that when an extension creates repositories, they are created within
the namespace of the extension. This means collisions can occur if different
modules use the same extension and end up creating a repository with the same
name. This often manifests as a module extension's `tag_class` having a `name`
argument that is passed as a repository rule's `name` value.

For example, say the root module, `A`, depends on module `B`. Both modules
depend on module `mylang`. If both `A` and `B` call
`mylang.toolchain(name="foo")`, they will both try to create a repository named
`foo` within the `mylang` module and an error will occur.

To avoid this, either remove the ability to set the repository name directly,
or only allow the root module to do so. It's OK to allow the root module this
ability because nothing will depend on it, so it doesn't have to worry about
another module creating a conflicting name.