sync: Always use WORKER_BATCH_SIZE

With 551285fa35, the comment about number
of workers no longer stands - dict is shared among multiprocesses and
real time information is available.

Using 2.7k projects as the baseline, using chunk size of 4 takes close
to 5 minutes. A chunk size of 32 takes this down to 40s - a reduction of
rougly 8 times which matches the increase.

R=gavinmak@google.com

Bug: b/371638995
Change-Id: Ida5fd8f7abc44b3b82c02aa0f7f7ae01dff5eb07
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438523
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This commit is contained in:
Josip Sokcevic
2024-10-07 17:33:38 +00:00
committed by LUCI
parent f7f9dd4deb
commit 454fdaf119
3 changed files with 43 additions and 27 deletions

View File

@ -2396,26 +2396,25 @@ class Project:
try:
# if revision (sha or tag) is not present then following function
# throws an error.
revs = [f"{self.revisionExpr}^0"]
upstream_rev = None
if self.upstream:
upstream_rev = self.GetRemote().ToLocal(self.upstream)
revs.append(upstream_rev)
self.bare_git.rev_list(
"-1",
"--missing=allow-any",
"%s^0" % self.revisionExpr,
*revs,
"--",
log_as_error=False,
)
if self.upstream:
rev = self.GetRemote().ToLocal(self.upstream)
self.bare_git.rev_list(
"-1",
"--missing=allow-any",
"%s^0" % rev,
"--",
log_as_error=False,
)
self.bare_git.merge_base(
"--is-ancestor",
self.revisionExpr,
rev,
upstream_rev,
log_as_error=False,
)
return True