commit	0ba8307ff46746864e357d0b7834b3c4247421d0	[log] [tgz]
author	kabakaev <kabakaev@gmail.com>	Mon Jan 08 05:15:33 2018 -0800
committer	Copybara-Service <copybara-piper@google.com>	Mon Jan 08 05:17:33 2018 -0800
tree	6154ed3009a7c9f489d542258ceb0f0a05025e4b
parent	fa46927fb24de61a987f28d2c64b01cfbe0a6fcc [diff]

fix git.bzl error when outputRoot is in a git repo

`_clone_or_update()` should `git clone` external repositories only if repo is absent. Unfortunately, it may happen that the bazel outputRoot (~/.cache/bazel by default) is a subdirectory of some other git working copy. For example, user may track his whole home directory in git and add `.config` to .gitignore. In that case, it is not enough to check if the cache dir is a git repo. One must check that the cache dir holds a root of a git working copy. In other words, the `.git` folder must be in the repo dir, not on any level above.

* Steps to reproduce
    ```bash
    cd ~/.cache
    git init
    cd /some/project/that/uses/bazel-git_repository
    mv WORKSPACE WORKSPACE.orig
    (echo "load('@bazel_tools//tools/build_defs/repo:git.bzl', 'git_repository')" ; cat WORKSPACE.orig) > WORKSPACE
    bazel --batch build :all || echo "Ah, there is a bug"
    ```

Read on for definitive info.

* clone any project that uses `git_repository`
    ```bash
    [arch@archlinux bazelbuild]$ git clone https://github.com/bazelbuild/bazel-watcher.git
    [arch@archlinux bazelbuild]$ cd bazel-watcher
    ```

* enable Skylark implementation of `git_repository` as documented in https://github.com/bazelbuild/bazel/issues/1408#issuecomment-276815467
    ```bash
    [arch@archlinux bazel-watcher]$ mv WORKSPACE WORKSPACE.orig
    [arch@archlinux bazel-watcher]$ (echo "load('@bazel_tools//tools/build_defs/repo:git.bzl', 'git_repository')" ; cat WORKSPACE.orig) > WORKSPACE
    [arch@archlinux bazel-watcher]$ grep -v ^# WORKSPACE |head -n4
    load('@bazel_tools//tools/build_defs/repo:git.bzl', 'git_repository')

    git_repository(
        name = "com_github_bazelbuild_bazel_integration_testing",
    ```

* (optionally) define custom Bazel `outputRoot` directory (default is ~/.cache/bazel)
    ```bash
    [arch@archlinux bazel-watcher]$ rm -rf /tmp/.cache/ ; mkdir /tmp/.cache/
    [arch@archlinux bazel-watcher]$ export TEST_TMPDIR=/tmp/.cache/bazel
    ```

* try building the project to make sure everything works as expected
    ```bash
    [arch@archlinux bazel-watcher]$ bazel --batch build :all
    INFO: $TEST_TMPDIR defined: output root default is '/tmp/.cache/bazel' and max_idle_secs default is '15'.
    Extracting Bazel installation...
    WARNING: /tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/com_github_bazelbuild_bazel_integration_testing/WORKSPACE:1: Workspace name in /tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/com_github_bazelbuild_bazel_integration_testing/WORKSPACE (@build_bazel_integration_testing) does not match the name given in the repository's definition (@com_github_bazelbuild_bazel_integration_testing); this will cause a build error in future versions
    INFO: Analysed target //:go_prefix (6 packages loaded).
    INFO: Found 1 target...
    Target //:go_prefix up-to-date (nothing to build)
    INFO: Elapsed time: 15.088s, Critical Path: 0.05s
    INFO: Build completed successfully, 1 total action
    [arch@archlinux bazel-watcher]$ echo $?
    0
    ```

* reproduce the issue by placing the `outputRoot` under a git working copy
    ```bash
    [arch@archlinux bazel-watcher]$ rm -rf /tmp/.cache/ ; mkdir /tmp/.cache/
    [arch@archlinux bazel-watcher]$ cd /tmp/.cache/ ; git init
    Initialized empty Git repository in /tmp/.cache/.git/
    [arch@archlinux .cache]$ cd -
    [arch@archlinux bazel-watcher]$ bazel --batch build :all
    INFO: $TEST_TMPDIR defined: output root default is '/tmp/.cache/bazel' and max_idle_secs default is '15'.
    Extracting Bazel installation...
    ERROR: error loading package '': Encountered error while reading extension file 'tools/repositories.bzl': no such package '@com_github_bazelbuild_bazel_integration_testing//tools': Traceback (most recent call last):
        File "/tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/bazel_tools/tools/build_defs/repo/git.bzl", line 69
            _clone_or_update(ctx)
        File "/tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/bazel_tools/tools/build_defs/repo/git.bzl", line 44, in _clone_or_update
            fail(("error cloning %s:\n%s" % (ctx....)))
    error cloning com_github_bazelbuild_bazel_integration_testing:
    + cd /tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external
    + cd /tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/com_github_bazelbuild_bazel_integration_testing
    + git reset --hard 55a6a70dbcc2cc7699ee715746fb1452788f8d3c
    fatal: Could not parse object '55a6a70dbcc2cc7699ee715746fb1452788f8d3c'.
    + git fetch origin 55a6a70dbcc2cc7699ee715746fb1452788f8d3c:55a6a70dbcc2cc7699ee715746fb1452788f8d3c
    fatal: 'origin' does not appear to be a git repository
    fatal: Could not read from remote repository.

    Please make sure you have the correct access rights
    and the repository exists.
    ERROR: error loading package '': Encountered error while reading extension file 'tools/repositories.bzl': no such package '@com_github_bazelbuild_bazel_integration_testing//tools': Traceback (most recent call last):
        File "/tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/bazel_tools/tools/build_defs/repo/git.bzl", line 69
            _clone_or_update(ctx)
        File "/tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/bazel_tools/tools/build_defs/repo/git.bzl", line 44, in _clone_or_update
            fail(("error cloning %s:\n%s" % (ctx....)))
    error cloning com_github_bazelbuild_bazel_integration_testing:
    + cd /tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external
    + cd /tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/com_github_bazelbuild_bazel_integration_testing
    + git reset --hard 55a6a70dbcc2cc7699ee715746fb1452788f8d3c
    fatal: Could not parse object '55a6a70dbcc2cc7699ee715746fb1452788f8d3c'.
    + git fetch origin 55a6a70dbcc2cc7699ee715746fb1452788f8d3c:55a6a70dbcc2cc7699ee715746fb1452788f8d3c
    fatal: 'origin' does not appear to be a git repository
    fatal: Could not read from remote repository.

    Please make sure you have the correct access rights
    and the repository exists.
    INFO: Elapsed time: 4.974s
    FAILED: Build did NOT complete successfully (0 packages loaded)
    ```

* let's find out why it failed
    ```bash
    [arch@archlinux bazel-watcher]$ grep rev-parse /tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/bazel_tools/tools/build_defs/repo/git.bzl
        if ! ( cd '{dir}' && git rev-parse --git-dir ) >/dev/null 2>&1; then
    [arch@archlinux bazel-watcher]$ cd /tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/com_github_bazelbuild_bazel_integration_testing
    [arch@archlinux com_github_bazelbuild_bazel_integration_testing]$ git rev-parse --git-dir
    /tmp/.cache/.git
    [arch@archlinux com_github_bazelbuild_bazel_integration_testing]$ cd -
    ```

* let's fix git.bzl
    ```bash
    [arch@archlinux bazel-watcher]$ sed -i -E 's/git rev-parse --git-dir/[[ "$(git rev-parse --git-dir)" == '.git' ]]/' /tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/bazel_tools/tools/build_defs/repo/git.bzl
    [arch@archlinux bazel-watcher]$ grep rev-parse /tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/bazel_tools/tools/build_defs/repo/git.bzl
    if ! ( cd '{dir}' && [[ "$(git rev-parse --git-dir)" == .git ]] ) >/dev/null 2>&1; then
    ```

* make sure that Bazel works again
    ```bash
    [arch@archlinux bazel-watcher]$ ls -a /tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/com_github_bazelbuild_bazel_integration_testing
    .  ..
    [arch@archlinux bazel-watcher]$ bazel --batch build :all
    INFO: $TEST_TMPDIR defined: output root default is '/tmp/.cache/bazel' and max_idle_secs default is '15'.
    Error: corrupt installation: file '/tmp/.cache/bazel/_bazel_arch/install/f20169627a8110e2cc3d005319e97c94/_embedded_binaries/embedded_tools/tools/build_defs/repo/git.bzl' modified.  Please remove '/tmp/.cache/bazel/_bazel_arch/install/f20169627a8110e2cc3d005319e97c94' and try again.
    [arch@archlinux bazel-watcher]$ touch -m -t 202712120101 /tmp/.cache/bazel/_bazel_arch/install/f20169627a8110e2cc3d005319e97c94/_embedded_binaries/embedded_tools/tools/build_defs/repo/git.bzl
    [arch@archlinux bazel-watcher]$ bazel --batch build :all
    INFO: $TEST_TMPDIR defined: output root default is '/tmp/.cache/bazel' and max_idle_secs default is '15'.
    WARNING: /tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/com_github_bazelbuild_bazel_integration_testing/WORKSPACE:1: Workspace name in /tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/com_github_bazelbuild_bazel_integration_testing/WORKSPACE (@build_bazel_integration_testing) does not match the name given in the repository's definition (@com_github_bazelbuild_bazel_integration_testing); this will cause a build error in future versions
    INFO: Analysed target //:go_prefix (6 packages loaded).
    INFO: Found 1 target...
    Target //:go_prefix up-to-date (nothing to build)
    INFO: Elapsed time: 13.318s, Critical Path: 0.07s
    INFO: Build completed successfully, 1 total action
    [arch@archlinux bazel-watcher]$ ls -a /tmp/.cache/bazel/_bazel_arch/f2207b308c89ea5d32323052637210b1/external/com_github_bazelbuild_bazel_integration_testing
    .  ..  AUTHORS  bazel_integration_test  bazel_integration_test.bzl  bazel_version.bzl  BUILD  .ci  CODEOWNERS  CONTRIBUTING.md  .git  .gitignore  go  java  javatests  LICENSE  README.md  tools  WORKSPACE
    ```

Closes #4358.

PiperOrigin-RevId: 181151078

tools/build_defs/repo/git.bzl[diff]

1 file changed

tree: 6154ed3009a7c9f489d542258ceb0f0a05025e4b

README.md

Bazel

{Fast, Correct} - Choose two

Build and test software of any size, quickly and reliably.

Speed up your builds and tests: Bazel only rebuilds what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.
One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.
Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.
Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.

Getting Started

Documentation

Contributing to Bazel

See CONTRIBUTING.md

Bazel is released in ‘Beta’. See the product roadmap to learn about the path toward a stable 1.0 release.