subcmds: reduce multiprocessing serialization overhead

Follow the same approach as 39ffd9977e to reduce serialization overhead.

Below benchmarks are tested with 2.7k projects on my workstation
(warm cache). git tracing is disabled for benchmark.

(seconds)              | v2.48 | v2.48 | this CL | this CL
	               |       |  -j32 |         |    -j32
-----------------------------------------------------------
with clean tree state:
branches (none)        |   5.6 |   5.9 |    1.0  |    0.9
status (clean)         |  21.3 |   9.4 |   19.4  |    4.7
diff (none)            |   7.6 |   7.2 |    5.7  |    2.2
prune (none)           |   5.7 |   6.1 |    1.3  |    1.2
abandon (none)         |  19.4 |  18.6 |    0.9  |    0.8
upload (none)          |  19.7 |  18.7 |    0.9  |    0.8
forall -c true         |   7.5 |   7.6 |    0.6  |    0.6
forall -c "git log -1" |  11.3 |  11.1 |    0.6  |    0.6

with branches:
start BRANCH --all     |  21.9 |  20.3 |   13.6  |    2.6
checkout BRANCH        |  29.1 |  27.8 |    1.1  |    1.0
branches (2)           |  28.0 |  28.6 |    1.5  |    1.3
abandon BRANCH         |  29.2 |  27.5 |    9.7  |    2.2

Bug: b/371638995
Change-Id: I53989a3d1e43063587b3f52f852b1c2c56b49412
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440221
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Kuang-che Wu <kcwu@google.com>
Commit-Queue: Kuang-che Wu <kcwu@google.com>

1 files changed, 7 insertions, 3 deletions

diff --git a/command.py b/command.py
index 2a2ce138..22115ac2 100644
--- a/command.py
+++ b/command.py
@@ -268,8 +268,10 @@ class Command:
             cls._parallel_context = None
 
     @classmethod
-    def _SetParallelContext(cls, context):
+    def _InitParallelWorker(cls, context, initializer):
         cls._parallel_context = context
+        if initializer:
+            initializer()
 
     @classmethod
     def ExecuteInParallel(
@@ -281,6 +283,7 @@ class Command:
         output=None,
         ordered=False,
         chunksize=WORKER_BATCH_SIZE,
+        initializer=None,
     ):
         """Helper for managing parallel execution boiler plate.
 
@@ -307,6 +310,7 @@ class Command:
             ordered: Whether the jobs should be processed in order.
             chunksize: The number of jobs processed in batch by parallel
                 workers.
+            initializer: Worker initializer.
 
         Returns:
             The |callback| function's results are returned.
@@ -318,8 +322,8 @@ class Command:
             else:
                 with multiprocessing.Pool(
                     jobs,
-                    initializer=cls._SetParallelContext,
-                    initargs=(cls._parallel_context,),
+                    initializer=cls._InitParallelWorker,
+                    initargs=(cls._parallel_context, initializer),
                 ) as pool:
                     submit = pool.imap if ordered else pool.imap_unordered
                     return callback(