tree a6c05d9694e21bf8299aefaec6f36c04604a423b
parent 3b6e35ed0c387705bbe68d0d787a01f719c75302
author djasper <djasper@google.com> 1550652813 -0800
committer Copybara-Service <copybara-worker@google.com> 1550652917 -0800

Implement asynchronous glob prefetching to specifically prefetch the globs() of
a file.

There is currently a mechanism in Bazel to asynchronously go off and prefetch
file system content, specifically by recursively reading subdirectories up to
maxDirectoriesToEagerlyVisitInGlobbing. This is started when the first glob in
a file is found. It is useful to warm up caches in networked filesystems, but
doesn't actually help evaluating the globs().  As it can create extra load on
the filesystem, it can actually make evaluation slower. Moreover, it implicitly
relies on the fact that in common packages, most files are actually read into
globs() in some way, which isn't always the case.

Instead, we can visit a build file's AST and eagerly try to explicitly extract
globs that should be visited. This change is doing that if
maxDirectoriesToEagerlyVisitInGlobbing is set to a sentinal value of -2. The
visitor cannot understand all globs, e.g. when some of the pattern strings are
constructed from complex expression, but the majority of globs is explicitly
spelled out.

RELNOTES: None.
PiperOrigin-RevId: 234751342