)]}'
{
  "commit": "bb28200de50dee08f7f7c4546ea2d320e8042580",
  "tree": "0d15089640aa8c220b66ac6c04ec2546f7398e3a",
  "parents": [
    "cca33299a58da23d1040d259ecd32be874ed174c"
  ],
  "author": {
    "name": "nharmata",
    "email": "nharmata@google.com",
    "time": "Tue Oct 27 15:01:52 2020 -0700"
  },
  "committer": {
    "name": "Copybara-Service",
    "email": "copybara-worker@google.com",
    "time": "Tue Oct 27 15:02:50 2020 -0700"
  },
  "message": "During change pruning, when a parent node notices all its children are done, finish processing that parent node intra-thread rather than enqueue it and have it be processed in the future by a different Skyframe thread.\n\nI discovered this room for improvement myself by running benchmarks with a Blaze binary instrumented with a ton of log statements in the Skyframe engine codebase. I was excited to go work on this commit. But when I started editing the code, I was surprised and happy to see that @ericfelly curiously had added a TODO for exactly this in passing in commit e97bba1 in 2016!\n\nFor highly parallel change pruning situations (e.g. N parent nodes each with a single common child node, such that N \u003e\u003e num_skyframe_threads \u003e\u003d num_cores), this avoids the contention of enqueueing each parent node and also is just plain faster since we process the parent node now rather than in the future. On a benchmark of a particularly extreme incremental build situation internally at Google, this increased the change pruning rate by ~175% and decreased overall wall time by ~60%.\n\nThe contention mentioned above is between busy Skyframe threads enqueueing more work units (PriorityBlockingQueue#offer) and idle Skyframe threads picking up a work unit (PriorityBlockingQueue#take). It\u0027s something that\u0027s been on my radar for ~years, and I even have a pending CL to completely remove it. I\u0027ve never prioritized pushing that CL through because I\u0027ve never been able to show the contention is obviously causing real problems, and my fix is to implement a low-contention priority queue data structure. Well, this CL here reduces the impact of that contention even more, so maybe I\u0027ll never get around to mailing out my new queue code ¯\\_(ツ)_/¯\n\nRELNOTES: None\nPiperOrigin-RevId: 339337198\n",
  "tree_diff": [
    {
      "type": "modify",
      "old_id": "2ebd793bffd6e1002543f65cab94faa6dfd1e61e",
      "old_mode": 33188,
      "old_path": "src/main/java/com/google/devtools/build/skyframe/AbstractParallelEvaluator.java",
      "new_id": "aa7a8f13de8731e42af0bb17841f71a19e007edd",
      "new_mode": 33188,
      "new_path": "src/main/java/com/google/devtools/build/skyframe/AbstractParallelEvaluator.java"
    },
    {
      "type": "modify",
      "old_id": "d0ecefe2bcbbaf31049ef9bc8a88ab68ac0beaca",
      "old_mode": 33188,
      "old_path": "src/main/java/com/google/devtools/build/skyframe/NodeEntryVisitor.java",
      "new_id": "109d4f1e3b0679186cfee32d825b957a1acca5c2",
      "new_mode": 33188,
      "new_path": "src/main/java/com/google/devtools/build/skyframe/NodeEntryVisitor.java"
    }
  ]
}