Common problems of writing portable rules, and some solutions.
This document focuses on writing Windows-compatible rules.
Problems:
Length limit: maximum path length is 259 characters.
Though Windows also supports longer paths (up to 32767 characters), many programs are built with the lower limit.
Be aware of this about programs you run in the actions.
Working directory: is also limited to 259 characters.
Processes cannot cd into a directory longer than 259 characters.
Case-sensitivity: Windows paths are case-insensitive, Unix paths are case-sensitive.
Be aware of this when creating command lines for actions.
Path separators: are backslash (\), not forward slash (/).
Bazel stores paths Unix-style, i.e. with / separators. Though some Windows programs support Unix-style paths, others don‘t. Some built-in commands in cmd.exe support them, some don’t.
It's best to always use \ separators on Windows: replace / with \ when you create command lines and environment variables for actions.
Absolute paths: don't start with slash (/).
Absolute paths on Windows start with a drive letter, e.g. C:\foo\bar.txt. There's no single filesystem root.
Be aware of this if your rule checks if a path is absolute. (Absolute paths should be avoided though, they are often non-portable.)
Solutions:
Keep paths short.
Avoid long directory names, deeply nested directory structures, long file names, long workspace names, long target names.
All of these may become path components of actions' input files, and may exhaust the path length limit.
Use a short output root.
Use the --output_user_root=<path> flag to specify a short path for Bazel outputs. A good idea is to have a drive (or virtual drive) just for Bazel outputs (e.g. D:\), and adding this line to your .bazelrc file:
build --output_user_root=D:/
or
build --output_user_root=C:/_bzl
Use junctions.
Junctions are, loosely speaking[1], directory symlinks. Junctions are easy to create and can point to directories (on the same computer) with long paths. If a build action creates a junction whose path is short but whose target is long, then tools with short path limit can access the files in the junction'ed directory.
In .bat files or in cmd.exe you can create junctions like so:
mklink /J c:\path\to\junction c:\path\to\very\long\target\path
[1]: Strictly speaking Junctions are not Symbolic Links, but for sake of build actions we may regard Junctions as Directory Symlinks.
Replace / with \ in paths in actions / envvars.
When you create the command line or environment variables for an action, make the paths Windows-style. Example:
def as_path(p, is_windows): if is_windows: return p.replace("/", "\\") else: return p
Problems:
Case-sensitivity: Windows environment variable names are case-insensitive.
For example, in Java System.getenv("SystemRoot") and System.getenv("SYSTEMROOT") yields the same result. (This applies to other languages too.)
Hermeticity: actions should use as few custom environment variables as possible.
Environment variables are part of the action's cache key. If an action uses environment variables that change often, or are custom to users, that makes the rule less cache-able.
Solutions:
Only use upper-case environment variable names.
This works on Windows, macOS, and Linux.
Minimize action environments.
When using ctx.actions.run, set the environment to ctx.configuration.default_shell_env. If the action needs more environment variables, put them all in a dictionary and pass that to the action. Example:
load("@bazel_skylib//lib:dicts.bzl", "dicts") def _make_env(ctx, output_file, is_windows): out_path = output_file.path if is_windows: out_path = out_path.replace("/", "\\") return dicts.add(ctx.configuration.default_shell_env, {"MY_OUTPUT": out_path})
Problems:
Executable outputs: Every executable file must have an executable extension.
The most common extensions are .exe (binary files) and .bat (Batch scripts).
Be aware that shell scripts (.sh) are NOT executable on Windows, i.e. you cannot specify them as ctx.actions.run‘s executable. There’s also no +x permission that files can have, so you can't execute arbitrary files like on Linux.
Bash commands: For sake of portability, avoid running Bash commands directly in actions.
Bash is widespread on Unix-like systems, but it's often unavailable on Windows. Bazel itself is relying less and less on Bash (MSYS2), so in the future users would be less likely to have MSYS2 installed along with Bazel. To make rules easier to use on Windows, avoid running Bash commands in actions.
Line endings: Windows uses CRLF (\r\n), Unix-like systems uses LF (\n).
Be aware of this when comparing text files. Be mindful of your Git settings, especially of line endings when checking out or committing. (See Git's core.autocrlf setting.)
Solutions:
Use a Bash-less purpose-made rule.
native.genrule() is a wrapper for Bash commands, and it's often used to solve simple problems like copying a file or writing a text file. You can avoid relying on Bash (and reinventing the wheel): see if bazel-skylib has a purpose-made rule for your needs. None of them depends on Bash when built/tested on Windows.
Build rule examples:
copy_file() (source, documentation): copies a file somewhere else, optionally making it executable
write_file() (source, documentation): writes a text file, with the desired line endings (auto, unix, or windows), optionally making it executable (if it's a script)
run_binary() (source, documentation): runs a binary (or *_binary rule) with given inputs and expected outputs as a build action (this is a build rule wrapper for ctx.actions.run)
native_binary() (source, documentation): wraps a native binary in a *_binary rule, which you can bazel run or use in run_binary()'s tool attribute or native.genrule()'s tools attribute
Test rule examples:
diff_test() (source, documentation): test that compares contents of two files
native_test() (source, documentation): wraps a native binary in a *_test rule, which you can bazel test
On Windows, consider using .bat scripts for trivial things.
Instead of .sh scripts, you can solve trivial tasks with .bat scripts.
For example, if you need a script that does nothing, or prints a message, or exits with a fixed error code, then a simple .bat file will suffice. If your rule returns a DefaultInfo() provider, the executable field may refer to that .bat file on Windows.
And since file extensions don't matter on macOS and Linux, you can always use .bat as the extension, even for shell scripts.
Be aware that empty .bat files cannot be executed. If you need an empty script, write one space in it.
Use Bash in a principled way.
In Starlark build and test rules, use ctx.actions.run_shell to run Bash scripts and Bash commands as actions.
In Starlark macros, wrap Bash scripts and commands in a native.sh_binary() or native.genrule(). Bazel will check if Bash is available and run the script or command through Bash.
In Starlark repository rules, try avoiding Bash altogether. Bazel currently offers no way to run Bash commands in a principled way in repository rules.
Problems:
Files cannot be deleted while open.
Open files cannot be deleted (by default), attempts result in “Access Denied” errors. If you cannot delete a file, maybe a running process still holds it open.
Working directory of a running process cannot be deleted.
Processes have an open handle to their working directory, and the directory cannot be deleted until the process terminates.
Solutions:
In your code, try to close files eagerly.
In Java, use try-with-resources. In Python, use with open(...) as f:. In principle, try closing handles as soon as possible.