| commit | 1049fe89f12cfcae00902fbae6f6a5f1bc0b3a33 | [log] [tgz] |
|---|---|---|
| author | ThomasCJY <thomascjy93@gmail.com> | Thu Nov 05 23:35:14 2020 -0800 |
| committer | Copybara-Service <copybara-worker@google.com> | Thu Nov 05 23:36:14 2020 -0800 |
| tree | 6ee8010b6d28e6b0a43635bfa0e272ef21fa8dfd | |
| parent | f9f8ce77398470c2f0f22cfd704e15ec3ab024bd [diff] |
[Bazel] Improve Mobile-Install Incremental Manifest Generating by applying multi-thread **Background** While using `mobile-Install`, we noticed that it constantly takes more to run on incremental build. Take our app for example, the incremental build metrics for a single line kotlin code change looks like this: | Command | Time | |---------------------------|------| | bazel build + adb install | 63s | | mobile-install | 91s | After digging into it, I found that the bottleneck is the "Incremental Manifest Generating" action which takes a lot of time (35+ sec) for multidex build. The time is spent on the checksum calculation for all dex files. The SHA256 checksum for each dex file takes around 1-2 sec. Currently in our app we have 10 dex shards and each dex zip contains 2-3 dex files, processing them sequentially takes more than 30 seconds. **Change** In this PR, I added multithread support for this script so that the checksum calculation can be done concurrently and it improved the "Incremental Manifest Generating" to be done in 6 second (80%+ improvement). **Result & Test** After applying this change, the total incremental build time has been reduced to 43 seconds, with a 30s+ improvement from `Incremental Manifest Generating` step. Before: <img width=400 src="https://user-images.githubusercontent.com/6951238/92814439-f678ef80-f377-11ea-967f-92767a08587e.png"> After: <img width=400 src="https://user-images.githubusercontent.com/6951238/92814445-fb3da380-f377-11ea-8de6-ff8c6b77c3f8.png"> You can also easily verify this from command line: ``` jchen tmp % time python ../build_incremental_dexmanifest.py ../output/outmanifest.txt shard1.dex.zip shard10.dex.zip shard2.dex.zip shard3.dex.zip shard4.dex.zip shard5.dex.zip shard6.dex.zip shard7.dex.zip shard8.dex.zip shard9.dex.zip python ../build_incremental_dexmanifest.py ../output/outmanifest.txt 0.70s user 0.72s system 31% cpu 4.583 total jchen tmp % time python ../build_incremental_dexmanifest_before.py ../output/outmanifest.txt shard1.dex.zip shard10.dex.zip shard2.dex.zip shard3.dex.zip shard4.dex.zip shard5.dex.zip shard6.dex.zip shard7.dex.zip shard8.dex.zip shard9.dex.zip python ../build_incremental_dexmanifest_before.py ../output/outmanifest.txt 0.65s user 0.64s system 3% cpu 37.908 total ``` Closes #12085. PiperOrigin-RevId: 340996883
{Fast, Correct} - Choose two
Build and test software of any size, quickly and reliably.
Speed up your builds and tests: Bazel rebuilds only what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.
One tool, multiple languages: Build and test Java, C++, Android, iOS, Go, and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.
Scalable: Bazel helps you scale your organization, codebase, and continuous integration solution. It handles codebases of any size, in multiple repositories or a huge monorepo.
Extensible to your needs: Easily add support for new languages and platforms with Bazel's familiar extension language. Share and re-use language rules written by the growing Bazel community.
Follow our tutorials:
See CONTRIBUTING.md
