We've had a limited version of this in CrOS for a long time. There's
nothing CrOS specific about it, so lets move it to the repo project so
everyone can utilize it.
Change-Id: I04cd94610c1100f3afcd2baf8c8e7ab13e589490
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299202
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Re-allow checking out projects to the top of the repo client checkout.
We add checks to prevent checking out files under .repo/ as that path
is only managed by us, and projects cannot inject content or settings
into it.
Bug: https://crbug.com/gerrit/14156
Bug: https://crbug.com/gerrit/14200
Change-Id: Id6bf9e882f5be748442b2c35bbeaee3549410b25
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299623
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
If the current project doesn't have any local branches, then there's
nothing to prune, so return right away. This avoids running a few
git commands when we aren't actually going to use the results, and
it avoids checking repository validity. Since we aren't going to do
anything in here, no need to check it.
Change-Id: Ie9d5c75a954e42807477299f3e5a63a92fac138b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299742
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Added the following methods to XmlManifest class.
+ GetDefaultGroupsStr() - return 'default,platform-' + platform.system().lower()
+ GetGroupsStr() - Same as gitc_utils.py's _manifest_groups func.
+ Replaced gitc_utils.py's_manifest_groups calls with GetGroupsStr.
+ Used the above methods to get groups in command.py::GetProjects
and part of init.py.
TODO: clean up these funcs to take structured group data more instead
of passing strings around everywhere that need parsing.
Tested the code with the following commands.
$ ./run_tests -v
Tested the sync code by using repo_dev alias and pointing to this CL
and verified prebuilts/fullsdk-linux directory has all the folders.
Tested repo init and repo sync with --use-superproject and without
--use-superproject argument.
$ repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject
$ repo_dev sync -c -j32
Bug: [google internal] b/181804931
Bug: https://crbug.com/gerrit/13707
Change-Id: Ia98585cbfa3a1449710655af55d56241794242b6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299422
Reviewed-by: Jonathan Nieder <jrn@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
The recent commit 84230009ee ("project:
make diff tools synchronous") broke repo diff if it includes % formats.
Add an explicit format string to fix.
Bug: https://crbug.com/gerrit/14208
Change-Id: Ie255a43c5b767488616b2b3dd15abc18f93bfab2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299402
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
There's a few rough edges here still, but no known corruption ones,
so open it up a bit for people to experiment with.
Bug: https://crbug.com/gerrit/11486
Change-Id: I81e0122ab6d3e032c546c8239dd4f03740676e80
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299242
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Superproject objects accept the optional argument “quiet”.
The following progress messages are displayed if quiet is false.
Displayed the following message whenever we find we have to make a new
folder (aka new remote), because if you started with repo init android
and later do googleplex-android that is when it will be slow.
"<location>: Performing initial setup for superproject; this might take
several minutes.".
After fetch completion, added the following notification:
"<location>: Initial setup for superproject completed."
Tested the code with the following commands.
$ ./run_tests -v
Tested the sync code by using repo_dev alias and pointing to this CL.
$ repo_dev init -u persistent-https://googleplex-android.git.corp.google.com/platform/manifest -b rvc-dev --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject
Bug: [google internal] b/181178282
Bug: https://crbug.com/gerrit/13707
Change-Id: Ia7fb85c6fb934faaa90c48fc0c55e7f41055f48a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299122
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
Change-Id: Ib748c61b1e65aee6dff8b97a9753d14c470a827f
Reported-by: Daniel Kutik <daniel.kutik@lavawerk.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299002
Reviewed-by: Daniel Kutik <daniel.kutik@lavawerk.com>
Reviewed-by: Ian Kasprzak <iankaz@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Allow the user to specify relative or absolute or any other funky
path that they want when using `repo init` or `repo sync`. Our
goal is to restrict the paths in the remote manifest git repo we
cloned from the network, not protect the user from themselves.
Bug: https://crbug.com/gerrit/14156
Change-Id: I1ccfb2a6bd1dce2bd765e261bef0bbf0f8a9beb6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298823
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
With the recent commit 0501b29e7a
("status: Use multiprocessing for `repo status -j<num>` instead of
threading"), the limitation with project serialization no longer
applies. It turns out that ad-hoc logic is expensive. In the CrOS
checkout (~1000 projects w/8 jobs by default), it adds about ~7sec
overhead to all invocations. With a fast nop run:
time repo forall -j8 -c true
This goes from ~11sec to ~4sec -- more than 50% speedup.
Change-Id: Ie6bcccd21eef20440692751b7ebd36c890d5bbcc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298724
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
The ProjectArgs function can be inlined which simplifies it quite a
bit. We shouldn't need the custom exception handling here either.
This also makes the next commit easier to review.
Change-Id: If3be04f58c302c36a0f20b99de0f67e78beac141
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298723
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
This is in preparation for simplifying the jobs support. The nested
function is referenced in the options object which can't be pickled,
so pull it out into a static method instead.
Change-Id: I01d3c4eaabcb8b8775ddf22312a6e142c84cb77d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298722
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
If a manifest checksout a project multiple times, repo download isn't
able to accurately pick the right project. We were just picking the
first result which could be a bit random for the user. If we hit that
situation, check if the cwd is one of the projects, and if it isn't,
we emit an error and tell the user it's an ambiguous request.
Bug: https://crbug.com/gerrit/13070
Change-Id: Id1059b81330229126b48c7312569b37504808383
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298702
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
The clone bundle logic assumes there is a one-to-one mapping between the
projects/ and project-objects/ trees. When using shared projects (where
we checkout different branches from the same project), this would lead us
to fetching the same clone bundle multiple times. Automatically skip the
clone bundle logic if the project-objects/ dir already exists.
Bug: https://crbug.com/gerrit/10993
Change-Id: I82c6fa1faf8605fd56c104fcea2a43dd4eecbce4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298682
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
On macOS, the Finder app likes to poop .DS_Store files in every path
that the user browses. If the user pokes around the .git/ tree, it
could generate a .DS_Store file in there too. When repo goes to read
all the local refs, it tries to decode this binary file as UTF-8 and
subsequently crashes.
Since paths that begin with . are not valid refs, ignore them like we
already do with paths that end in .lock. Also bump the check up to
ignore dirs that match since that follows the git rules: they apply
to any component in its path, not just the final path (name).
We don't implement the full valid ref algorithm that git employs as
it's a bit complicated, and we only really need to focus on what will
practically show up locally.
Bug: https://crbug.com/gerrit/14162
Change-Id: I6519f990e33cc58a72fcb00c0f983ad3285ace3d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298662
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Michael Mortensen <mmortensen@google.com>
When using Git under Windows, it seems that Git doesn't always parse
GIT_DIR correctly when it uses the Windows \ form, but does when it
uses / only.
For example, when using worktrees:
$ GIT_DIR='C:\Users\vapier\Desktop\repo\breakpad\tools\test\.git' git worktree list
fatal: not a git repository: ..\..\.repo\worktrees\linux-syscall-support.git\worktrees\test
$ GIT_DIR='C:/Users/vapier/Desktop/repo/breakpad/tools/test/.git' git worktree list
C:/Users/vapier/Desktop/repo/breakpad/.repo/worktrees/linux-syscall-support.git fd00dbbd0c06 (detached HEAD)
..\..\..\..\..\src\src\third_party\lss\.git fd00dbbd0c06 (detached HEAD)
..\..\..\..\..\tools\test\.git fd00dbbd0c06 (detached HEAD)
Change-Id: I666c03ae845ecb55d7f9800731ea6987d3e7f401
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298622
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
These attribute values are used to construct local filesystem paths,
so apply the existing filesystem checks to them.
Bug: https://crbug.com/gerrit/14156
Change-Id: Ibcceecd60fa74f0eb97cd9ed1a9792e139534ed4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298443
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
This function is currently written with copyfile & linkfile in mind.
Generalize the logic & function arguments slightly so we can reuse
in more places that make sense.
This changes the validation logic slightly too in that we no longer
allow "." for the dest attribute with copyfile & linkfile, nor for
the src attribute with copyfile. We already rejected those later on
when checking against the active filesystem, but now we reject them
a little sooner when parsing.
The empty path check isn't a new requirement exactly -- repo used to
crash on it, so it was effectively blocked, but now we diagnosis it.
Bug: https://crbug.com/gerrit/14156
Change-Id: I0fdb42a3da60ed149ff1997c5dd4b85da70eec3d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298442
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Use multiprocessing to run in parallel. When operating on multiple
projects, this can speed things up. Across 1000 repos, it goes from
~9sec to ~5sec with the default -j8.
Change-Id: Ida6dd565db78ff7bac0ecb25d2805e8a1bf78048
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297982
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
This avoids GIL limitations with using threads for parallel processing.
In a CrOS checkout with ~1000 repos, the nop case goes from ~6 sec down
to ~4 sec with -j8. Not a big deal, but shows that this actually works
to speed things up unlike the threading model.
This reworks the checkout logic to return results for processing in the
main thread instead of leaving every thread to do its own processing.
Bug: https://crbug.com/gerrit/12389
Change-Id: I143e5e3f7158e83ea67e2d14e5552153a874248a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298063
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Use multiprocessing to run in parallel. When operating on multiple
projects, this can greatly speed things up. Across 1000 repos, it
goes from ~30sec to ~3sec with the default -j8.
Change-Id: I0dc62d704c022dd02cac0bd67fe79224f4e34095
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297484
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
We're already keeping tracking of the start time, so might as
well use it to display overall execution time for steps.
Change-Id: Ib4cf8b2b0dfcdf7b776a84295d59cc569971bdf5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298482
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
+ superproject will be fetched into a directory with the name
“<remote name>-superproject.git” instead of the current
“superproject.git” folder.
+ Deleted _Clone method and added _Init method.
+ _Init method will do “git init --bare <remote>-superproject.git”.
It will create the folder and set up a bare repository in
<remote>-superproject.git folder.
+ _Fetch method, will pass <remote url>, <branch> arguments.
Moved the --filter argument from “git clone” to “git fetch”.
_Fetch method will execute the following command to fetch
superproject. Added --no-tags argument.
master: git fetch <remote url> --force --no-tags --filter blob:none
branch: git fetch <remote url> --force --no-tags --filter blob:none \
<branch>:<branch>
+ Performance improvements for aosp-master
++ repo init performance improved from 35 seconds to 17 seconds.
++ repo init --use-superproject is around 5 to 7 secsonds slower.
++ repo sync --use-superproject is around 3 to 4 minutes faster.
Tested the code with the following commands.
$ ./run_tests -v
Tested the sync code by using repo_dev alias and pointing to this CL.
$ time repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b master --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject
...
real 0m20.648s
user 0m8.046s
sys 0m3.271s
+ Without superproject
$ time repo init -u sso://android.git.corp.google.com/platform/manifest -b master --partial-clone --clone-filter=blob:limit=10M --repo-rev=main
real 0m13.078s
user 0m9.783s
sys 0m2.528s
$ time repo_dev sync -c -j32 --use-superproject
...
real 15m7.072s
user 110m7.216s
sys 20m17.559s
+ Without superproject
$ time repo sync -c -j32
...
real 19m25.644s
user 91m56.331s
sys 20m59.170s
Bug: [google internal] b/180492484
Bug: [google internal] b/179470886
Bug: [google internal] b/180124069
Bug: https://crbug.com/gerrit/13709
Bug: https://crbug.com/gerrit/13707
Change-Id: Ib04bd7f1e25ceb75532643e58ad0129300ba3299
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297702
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
This is in preparation for adding jobs support. The nested function
is referenced in the options object which can't be pickled, so pull
it out into a static method instead.
Change-Id: I280ed2bf26390a0203925517a0d17c13053becaa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297983
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
The default sync output should show a progress bar only for successful
commands, and the error output for any commands that fail. Implement
that policy here.
Bug: https://crbug.com/gerrit/11293
Change-Id: I85716032201b6e2b45df876b07dd79cb2c1447a5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297905
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
The idea for skipping some progress updates was to avoid spending
too much time on the progress bar itself. Unfortunately, for large
projects (100s if not 1000s) of repos, we get into the situation
with large/slow checkouts that we skip showing updates when a repo
finishes, but not enough repos finished to increase the percent.
Since the progress bar should be relatively fast compared to the
actual network & local dick operations, have it show an update
whenever the caller requests it. A test with ~1000 repos shows
that the progress bar in total adds <100ms.
Bug: https://crbug.com/gerrit/11293
Change-Id: I708a0c4bd923c59c7691a5b48ae33eb6fca4cd14
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297903
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
When people switch to non-default branches, they sometimes want to
switch back to the default, but don't know the exact name for that
branch. Add a -b HEAD shortcut for that.
Change-Id: I090230da25f9f5a169608115d483f660f555624f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297843
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
When sync moved to consume clone output, it merged stdout & stderr,
but the retry logic in this function is based on stderr only. Move
it over to checking stdout.
Change-Id: I71bdc18ed25c978055952721e3a768289d7a3bd2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297902
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
My recent 401c6f0725 ("init: make
--manifest-url flag optional") commit broke the unittest.
Change-Id: I19ad0e8c8cbb84ab5474ebc370e00acfe957e136
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298223
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
The XmlManifestTests class is getting to be large and we're only
adding more to it. Factor out the core logic into a new TestCase
so we can reuse it to better group more tests.
Change-Id: I5113444a4649a70ecfa8d83d3305959a953693f7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298222
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
If --use-superproject is passed as argument to "repo init", then
--use-superproject need not be specified during "repo sync".
Tested the code with the following commands.
$ time repo_dev sync -c -j32
...
WARNING: --use-superproject is experimental and not for general use
Bug: https://crbug.com/gerrit/13709
Bug: https://crbug.com/gerrit/13707
Change-Id: Ibb33f3038a2515f74a6c4f7cb785d354b26ee680
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298102
Tested-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Ian Kasprzak <iankaz@google.com>
For people coming across these docs and thinking that repo's methods
are good to replicate, add a note warning them against doing so.
Change-Id: I443a783794313851a6e7ba1c39baebac988bff9a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298164
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Now that we've converted the few users of this over to subprocess APIs,
we don't need this anymore. It's been a bit hairy to maintain across
different operating systems, so there's no desire to bring it back.
Using multiprocessing Pool to batch things has been working better in
general anyways.
Change-Id: I10769e96f60ecf27a80d8cc2aa0d1b199085252e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297682
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Python 3 renamed this method from isSet to is_set.
Change-Id: I8f9bb0b302d55873bed3cb20f2d994fa2d082157
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297742
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Now that these code paths are all synchronous, there's no need to run
our own poll loop to read & pass thru/save output. Delete all of that
and just let the subprocess module take care of it all.
Change-Id: Ic27fe71b6f964905cf280ce2b183bb7ee46f4a0d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297422
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Use multiprocessing to run diff in parallel.
Change-Id: I61e973d9c2cde039d5eebe8d0fe8bb63171ef447
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297483
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
This fixes intermingling of parallel jobs and simplifies the code
by switching to subprocess.run. This also provides stable output
in the order of projects by returning the output as a string that
the main loop outputs.
This drops support for interactive commands, but it's unclear if
anyone was relying on that, and the default behavior (-j2) made
that unreliable. If it turns out someone still wants this, we can
look at readding it.
Change-Id: I7555b4e7a15aad336667292614f730fb7a90bd26
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297482
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Use a pool contextmanager to take care of the messy details like
properly cleaning it up when aborting.
Change-Id: I264ebb591c2e67c9a975b6dcc0f14b29cc66a874
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297243
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
The status command runs a bunch of jobs in parallel, and each one
is responsible for writing to stdout directly. When running many
noisy jobs in parallel, output can get intermingled. Pass down a
StringIO buffer for writing to so we can return the entire output
as a string so the main job can handle displaying it. This fixes
interleaved output as well as making the output stable: we always
display results in the same project order now. By switching from
map to imap, this ends up not really adding any overhead.
Bug: https://crbug.com/gerrit/12231
Change-Id: Ic18b07c8074c046ff36e306eb8d392fb34fb6eca
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297242
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Extend the Command class to support adding the --jobs option to the
parser if the command declares it supports running in parallel. Also
pull the default value used for the number of local jobs into the
command module so local commands can share it.
Change-Id: I22b0f8d2cf69875013cec657b8e6c4385549ccac
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297024
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Every use of GitCommand in the tree just calls Wait as soon as it's
instantiated. Move the bulk of the logic into the init path to make
the call synchronous to simplify. We'll cleanup the users of the
Wait API to follup commits -- having this split makes it easier to
track down regressions.
Change-Id: I1e8c519efa912da723749ff7663558c04c1f491c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297244
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Python 3 has a simpler super() style so switch to it to make the
code a little simpler and to stop pylint warnings.
Change-Id: I1b3ccf57ae968d56a9a0bcfc1258fbd8bfa3afee
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297383
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Make sure all our custom exceptions can be pickled so that if they
get thrown in a multiprocess subprocess, we don't crash & hang due
to multiprocessing being unable to pickle+unpickle the exception.
Details/examples can be seen in Python reports like:
https://bugs.python.org/issue13751
Change-Id: Iddf14d3952ad4e2867cfc71891d6b6559130df4b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297382
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Since the --manifest-url flag is always required when creating a new
checkout, allow the url to be specified via a positional argument.
This brings it a little closer to the `git clone` UI.
Change-Id: Iaf18e794ae2fa38b20579243d067205cae5fae2f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297322
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
This is the only code in the tree that uses GitCommand asynchronously.
Rewrite it to use multiprocessing.Pool as it makes the code a little
bit easier to understand and simpler.
Change-Id: I3ed3b037f24aa1e9dfe8eec9ec21815cdda7678a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297143
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Michael Mortensen <mmortensen@google.com>
These are the only users in the tree that process the output as it's
produced. All others capture all the output first and then process
the results. However, these functions still don't fully return until
it's finished processing, and these funcs are in turn used in other
synchronous code paths. So it's unclear whether anyone will notice
that it's slightly slower or less interactive. Let's try it out and
see if users report issues.
This will allow us to simplify our custom GitCommand code and move it
over to Python's subprocess.run, and will help fix interleaved output
when running multiple commands in parallel (e.g. `repo diff -j8`).
Change-Id: Ida16fafc47119d30a629a8783babeba890515de0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297144
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
