Current clone bundle fetches from Google storage results HTTP/404
and curl exiting 56. This is basically WAI, so stop emitting
verbose error output whenever that happens. Also add a few more
curl exit statuses based on chromite history, and document them.
Change-Id: I3109f8a8a19109ba9bbd62780b40bbcd4fce9b76
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/432197
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Let people specify the exact topic when uploading CLs. The existing
-t option only supports setting the topic to the current local branch.
Add a --topic-branch long option to the existing -t to align it a bit
better with --hashtag & --hashtag-branch.
Change-Id: I010abc4a7f3c685021cae776dd1e597c22b79135
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/431997
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
This will make "repo sync -d --force-checkout" more reliable
in CI automation, as there are fewer things in the way that may
need manual intervention.
Bug: b/40015382
Change-Id: I8a79971724a3d9a8e2d682b7a0c04deda9e34177
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/423317
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Erik Elmeke <erik@haleytek.corp-partner.google.com>
Commit-Queue: Erik Elmeke <erik@haleytek.corp-partner.google.com>
Debian 7 Wheezy went EOL in May 2018. We don't need to carry support
for that anymore as there have been 5 major releases since. Ubuntu
Precise went EOL in Apr 2019 (including the extended support phase).
That means we can bump the required git version from 1.7.9 to 1.9.1.
git-1.7.9 was released in 2012 while git-1.9.1 was released in 2014.
So that shouldn't be a problem either. And we've been warning people
using git versions older than 1.9.1 for 3 years now that they need to
upgrade.
Change-Id: Ifbbf72f51010b0a944c2785895d1b605333f9146
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415637
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
... since it's just a simple wrapper of os.path.realpath now.
Change-Id: I7433e5fe09c64b130f06e2541151dce1961772c9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/416637
Tested-by: Kaiyi Li <kaiyili@google.com>
Reviewed-by: Greg Edelston <gredelston@google.com>
Commit-Queue: Kaiyi Li <kaiyili@google.com>
os.path.exists returns false on a broken symlink. This is not what repo
needs when checking if a project is setup properly.
For example, if src/foo/.git can't be resolved, repo tries to create
symlink and that results in FileExistsError.
Use lexists which returns True even if symlink is broken. That will
force path where repo checks where symlink is pointing to and will fix
it to the correct location.
Bug: b/281746795
Change-Id: Id3f7dc3a3cb6499d02ce7335eca992ddc7deb645
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415197
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: George Engelbrecht <engeg@google.com>
Reviewed-by: Greg Edelston <gredelston@google.com>
If a project can't be removed from checkout due to uncommitted changes
present, error.project is type of Project and not a string (as it is in
some cases). Project is not JSON serializable, resulting in exception
within exception handler:
TypeError: Object of type Project is not JSON serializable
This change casts project to string as a defensive mechanism. It also
passes project name instead of project object.
Change-Id: Ie7b782d73dc3647975755d5a3774d16ea6cd5348
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/413877
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
In some cases (e.g. in a CI system), it's desirable to be able to
instruct repo to force checkout. This flag passes --force flag to `git
checkout` operations.
Bug: b/327624021
Change-Id: I579edda546fb8147c4e1a267e2605fcf6e597421
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/411518
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: George Engelbrecht <engeg@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
If a project contains files not owned by the current user, remove will
fail. In order to ensure repo sync continues to work, rename the
affected project instead, and let user know about it.
Bug: 321273512
Change-Id: I0779d61fc67042308a0226adea7d98167252a5d3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/404372
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
There are still some verbose messages (e.g. "remote: ...") when doing
repo sync after a couple days. Let's hide them behind verbose flag.
Bug: N/A
Test: repo sync
Change-Id: I1408472c95ed80d9555adfe8f92211245c03cf41
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/400855
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Tomasz Wasilczyk <twasilczyk@google.com>
Commit-Queue: Tomasz Wasilczyk <twasilczyk@google.com>
Symbolic references need to be checked each time sync is called, not
only for newly created repositories. For example, it is possible to
change a project name to the already existing name, and that will result
in a broken git setup without this patch: refs/ will still point to the
old repository, whereas all objects will point to the new repository.
Bug: 40013418
Change-Id: I596d29d182986804989f0562fb45090224549b0f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/395798
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Most times a repo sync after some time (week+) results in a bunch of
messages, which are not very useful for average user:
- discarding 1 commits
- Deleting obsolete checkout.
Bug: N/A
Test: repo sync
Change-Id: I881eab61f9f261e98f3656c09e73ddd159ce288c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/397038
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Tested-by: Tomasz Wasilczyk <twasilczyk@google.com>
- Bump minimum version to Python 3.6.
- Use f-strings in a lot of places.
Change-Id: I2aa70197230fcec2eff8e7c8eb754f20c08075bb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389034
Tested-by: Jason R. Coombs <jaraco@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason R. Coombs <jaraco@google.com>
When a new shared project is added to manifest, there's a short window
where objects can be deleted that are used by other projects.
To close that window, set preciousObjects during git init. For
non-shared projects, repo should correct the state in the same execution
instance.
Bug: 288102993
Change-Id: I366f524535ac58c820d51a88599ae2108df9ab48
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390234
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
In Python 3, these exceptions were merged into OSError, so switch
everything over to that.
Change-Id: If876a28b692de5aa5c62a3bdc8c000793ce52c63
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390376
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Bug: 302680231
Change-Id: Icd01dd2ce62d737a4acb114e729189cd31f6bde9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/388234
Tested-by: Jason R. Coombs <jaraco@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason R. Coombs <jaraco@google.com>
Sometimes it is expected that a GitCommand executed in repo fails. In
such cases indicate in trace logs that the error was expected.
Bug: b/293344017
Change-Id: If137fae9ef9769258246f5b4494e070345db4a71
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/387714
Commit-Queue: Jason Chang <jasonnc@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
When comparing 2 manifests, most of the time is
spent getting the relevant commit id as it relies
on _allrefs which ends up loading all git references.
However, the value from `revisionIs` (when it is valid)
could be used directly leading to a huge performance improvement
(from 180+ seconds to less than 0.01 sec which is more
than 25000 times faster for manifests with 700+ projects).
Bug: 295282548
Change-Id: I5881aa4b2326cc17bbb4ee91d23293111f76ad7e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/385834
Tested-by: Sylvain Desodt <sylvain.desodt@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Sylvain Desodt <sylvain.desodt@gmail.com>
A previous change captured stderr when uploading git projects. This
change ensures stderr is sent to stderr.
Bug: b/297097597
Change-Id: I8314e1017d2a42b7b655fe43ce3c312d397894ca
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/384134
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason Chang <jasonnc@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
There was a bug in error handeling code that caused an uncaught
exception to be raised.
Bug: b/296316540
Change-Id: I49c72f29c00f26ba60de552f958bc6eddf841162
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/383254
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason Chang <jasonnc@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Per discussion in go/repo-error-update updated aggregated and exit
errors for sync command.
Aggregated errors are errors that result in eventual command failure.
Exit errors are errors that result in immediate command failure.
Also updated main.py to log aggregated and exit errors to git sessions
log
Bug: b/293344017
Change-Id: I77a21f14da32fe2e68c16841feb22de72e86a251
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/379614
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Commit-Queue: Jason Chang <jasonnc@google.com>
An investigation go/git-repo-shallow shows a number of problems
when doing a shallow git fetch/clone. This change introduces an
environment variable REPO_ALLOW_SHALLOW. When this environment variable
is set to 1 during a repo init or repo sync all shallow git fetch
commands are replaced with partial fetch commands. Any shallow
repository needing update is unshallowed. This behavior continues until
a subsequent repo sync command is run with REPO_ALLOW_SHALLOW set to 1.
Bug: b/274340522
Change-Id: I1c3188270629359e52449788897d9d4988ebf280
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/374754
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
If the original fetch attempt did not want tags, we should continue to
honor that when doing a retry fetch with depth set to None. This seems
to match the intent of the retry based on the inline comment and results
in a significant performance improvement when the original fetch-by-sha1
fails due to the server not allowing requests for unadvertised objects.
Change-Id: Ia26bb31ea9aecc4ba2d3e87fc0c5412472cd98c4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/374918
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Kaushik Lingarkar <kaushik.lingarkar@linaro.org>
Tested-by: Kaushik Lingarkar <kaushik.lingarkar@linaro.org>
Apply rules set by https://gerrit-review.googlesource.com/c/git-repo/+/362954/ across the codebase and fix any lingering errors caught
by flake8. Also check black formatting in run_tests (and CQ).
Bug: b/267675342
Change-Id: I972d77649dac351150dcfeb1cd1ad0ea2efc1956
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/363474
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Superproject update failures on single-manifest checkouts had an extra
space.
Bug: b/254523816
Change-Id: I6f71e42337e324a6975c5d6bba487f83abaf054f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/357056
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
If this is a project that is not using object sharing (there is only one
copy of the remote project) then clear preciousObjects.
To override this for a project, run:
git config --replace-all repo.preservePreciousObjects true
Change-Id: If3ea061c631c5ecd44ead84f68576012e2c7405c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350235
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
If a file that is copied using a <copyfile> tag is modified and not
committed or if it is committed to a detached head, then running `repo
sync` would update the target file as expected. However, if the
modified file is committed to a local branch, then running `repo sync'
would not update the target file as expected.
Change-Id: Ic98e37d1c2e51fd1bf15abf149c7d06190cfd6d2
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/344475
Reviewed-by: Mike Frysinger <vapier@google.com>
Replace tuple returns with namedtuples, to simplify adding new fields.
Extend the Sync_NetworkHalf return value to:
- success: True if successful (the former return value)
- remote_fetched: True if we called `git fetch`
Change-Id: If63c24c2f849523f77fa19c05bbf23a5e9a20ba9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/344534
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
This reverts commit 07d21e6bde.
Reason for revert: crbug.com/gerrit/16230, b/244467766 - breaks aosp-master-with-phones case
Change-Id: Id967d92f8622c2c13356b09e46ece9f20040aabc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/344314
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
If initializing the manifest fails for any reason, don't leave it in
a half complete state. This can cause problems if/when the user tries
to reinit because different codepaths will be taken. For example, if
we initialize manifests.git and don't finish probing the remote to see
what default branch it uses, we end up always using "master" even if
that isn't what the remote uses.
To avoid all of this, use .tmp dirs when initializing, and rename to
the final path only after we complete all the right steps.
We should roll this out to all projects we clone, but start with the
manifest project for now.
Bug: https://crbug.com/gerrit/13526
Bug: https://crbug.com/gerrit/15805
Change-Id: I0214338de69ee11e090285c6b0b211052804af06
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343539
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Normally git produces no output when creating or switching branches.
If there's a problem though, we want to show that to the user. So
switch from capturing all output to running in quiet mode.
Bug: https://crbug.com/gerrit/15819
Change-Id: I7873ecc7c3bacce591899cc9471cb0244eb74541
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343454
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
We almost always use self.remote.name when calling self.GetRemote,
so make that the default to simplify the code a bit.
Change-Id: Ifdf6e1370d6b8963b44e6d384b0fac8fa5c4f2ba
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343184
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
The `--wip` allow to bulk push changed as work-in-progress. This CL
intend to allow the opposite opperation by removing the wip mark on the
CL and set it to be ready to review
Change-Id: If0743c5b14829f77be2def5a8547060d06a5648c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/342214
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: William Escande <wescande@google.com>
