Most times a repo sync after some time (week+) results in a bunch of
messages, which are not very useful for average user:
- discarding 1 commits
- Deleting obsolete checkout.
Bug: N/A
Test: repo sync
Change-Id: I881eab61f9f261e98f3656c09e73ddd159ce288c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/397038
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Tested-by: Tomasz Wasilczyk <twasilczyk@google.com>
- Bump minimum version to Python 3.6.
- Use f-strings in a lot of places.
Change-Id: I2aa70197230fcec2eff8e7c8eb754f20c08075bb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389034
Tested-by: Jason R. Coombs <jaraco@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason R. Coombs <jaraco@google.com>
When a new shared project is added to manifest, there's a short window
where objects can be deleted that are used by other projects.
To close that window, set preciousObjects during git init. For
non-shared projects, repo should correct the state in the same execution
instance.
Bug: 288102993
Change-Id: I366f524535ac58c820d51a88599ae2108df9ab48
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390234
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
In Python 3, these exceptions were merged into OSError, so switch
everything over to that.
Change-Id: If876a28b692de5aa5c62a3bdc8c000793ce52c63
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390376
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Bug: 302680231
Change-Id: Icd01dd2ce62d737a4acb114e729189cd31f6bde9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/388234
Tested-by: Jason R. Coombs <jaraco@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason R. Coombs <jaraco@google.com>
Sometimes it is expected that a GitCommand executed in repo fails. In
such cases indicate in trace logs that the error was expected.
Bug: b/293344017
Change-Id: If137fae9ef9769258246f5b4494e070345db4a71
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/387714
Commit-Queue: Jason Chang <jasonnc@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
When comparing 2 manifests, most of the time is
spent getting the relevant commit id as it relies
on _allrefs which ends up loading all git references.
However, the value from `revisionIs` (when it is valid)
could be used directly leading to a huge performance improvement
(from 180+ seconds to less than 0.01 sec which is more
than 25000 times faster for manifests with 700+ projects).
Bug: 295282548
Change-Id: I5881aa4b2326cc17bbb4ee91d23293111f76ad7e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/385834
Tested-by: Sylvain Desodt <sylvain.desodt@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Sylvain Desodt <sylvain.desodt@gmail.com>
A previous change captured stderr when uploading git projects. This
change ensures stderr is sent to stderr.
Bug: b/297097597
Change-Id: I8314e1017d2a42b7b655fe43ce3c312d397894ca
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/384134
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason Chang <jasonnc@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
There was a bug in error handeling code that caused an uncaught
exception to be raised.
Bug: b/296316540
Change-Id: I49c72f29c00f26ba60de552f958bc6eddf841162
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/383254
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason Chang <jasonnc@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Per discussion in go/repo-error-update updated aggregated and exit
errors for sync command.
Aggregated errors are errors that result in eventual command failure.
Exit errors are errors that result in immediate command failure.
Also updated main.py to log aggregated and exit errors to git sessions
log
Bug: b/293344017
Change-Id: I77a21f14da32fe2e68c16841feb22de72e86a251
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/379614
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Commit-Queue: Jason Chang <jasonnc@google.com>
An investigation go/git-repo-shallow shows a number of problems
when doing a shallow git fetch/clone. This change introduces an
environment variable REPO_ALLOW_SHALLOW. When this environment variable
is set to 1 during a repo init or repo sync all shallow git fetch
commands are replaced with partial fetch commands. Any shallow
repository needing update is unshallowed. This behavior continues until
a subsequent repo sync command is run with REPO_ALLOW_SHALLOW set to 1.
Bug: b/274340522
Change-Id: I1c3188270629359e52449788897d9d4988ebf280
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/374754
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
If the original fetch attempt did not want tags, we should continue to
honor that when doing a retry fetch with depth set to None. This seems
to match the intent of the retry based on the inline comment and results
in a significant performance improvement when the original fetch-by-sha1
fails due to the server not allowing requests for unadvertised objects.
Change-Id: Ia26bb31ea9aecc4ba2d3e87fc0c5412472cd98c4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/374918
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Kaushik Lingarkar <kaushik.lingarkar@linaro.org>
Tested-by: Kaushik Lingarkar <kaushik.lingarkar@linaro.org>
Apply rules set by https://gerrit-review.googlesource.com/c/git-repo/+/362954/ across the codebase and fix any lingering errors caught
by flake8. Also check black formatting in run_tests (and CQ).
Bug: b/267675342
Change-Id: I972d77649dac351150dcfeb1cd1ad0ea2efc1956
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/363474
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Superproject update failures on single-manifest checkouts had an extra
space.
Bug: b/254523816
Change-Id: I6f71e42337e324a6975c5d6bba487f83abaf054f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/357056
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
If this is a project that is not using object sharing (there is only one
copy of the remote project) then clear preciousObjects.
To override this for a project, run:
git config --replace-all repo.preservePreciousObjects true
Change-Id: If3ea061c631c5ecd44ead84f68576012e2c7405c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350235
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
If a file that is copied using a <copyfile> tag is modified and not
committed or if it is committed to a detached head, then running `repo
sync` would update the target file as expected. However, if the
modified file is committed to a local branch, then running `repo sync'
would not update the target file as expected.
Change-Id: Ic98e37d1c2e51fd1bf15abf149c7d06190cfd6d2
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/344475
Reviewed-by: Mike Frysinger <vapier@google.com>
Replace tuple returns with namedtuples, to simplify adding new fields.
Extend the Sync_NetworkHalf return value to:
- success: True if successful (the former return value)
- remote_fetched: True if we called `git fetch`
Change-Id: If63c24c2f849523f77fa19c05bbf23a5e9a20ba9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/344534
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
This reverts commit 07d21e6bde.
Reason for revert: crbug.com/gerrit/16230, b/244467766 - breaks aosp-master-with-phones case
Change-Id: Id967d92f8622c2c13356b09e46ece9f20040aabc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/344314
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
If initializing the manifest fails for any reason, don't leave it in
a half complete state. This can cause problems if/when the user tries
to reinit because different codepaths will be taken. For example, if
we initialize manifests.git and don't finish probing the remote to see
what default branch it uses, we end up always using "master" even if
that isn't what the remote uses.
To avoid all of this, use .tmp dirs when initializing, and rename to
the final path only after we complete all the right steps.
We should roll this out to all projects we clone, but start with the
manifest project for now.
Bug: https://crbug.com/gerrit/13526
Bug: https://crbug.com/gerrit/15805
Change-Id: I0214338de69ee11e090285c6b0b211052804af06
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343539
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Normally git produces no output when creating or switching branches.
If there's a problem though, we want to show that to the user. So
switch from capturing all output to running in quiet mode.
Bug: https://crbug.com/gerrit/15819
Change-Id: I7873ecc7c3bacce591899cc9471cb0244eb74541
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343454
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
We almost always use self.remote.name when calling self.GetRemote,
so make that the default to simplify the code a bit.
Change-Id: Ifdf6e1370d6b8963b44e6d384b0fac8fa5c4f2ba
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343184
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
The `--wip` allow to bulk push changed as work-in-progress. This CL
intend to allow the opposite opperation by removing the wip mark on the
CL and set it to be ready to review
Change-Id: If0743c5b14829f77be2def5a8547060d06a5648c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/342214
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: William Escande <wescande@google.com>
Display the project path relative to the outermost manifest by default,
and relative to the sub manifest only when --this-manifest-only is
specified.
For project-related diagnostic messages, use the outermost manifest for
messages.
Change-Id: I4537d7dd412a2c182e77d6720e95c1b0ef70eb0e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/340754
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
This option will suppress the
Uncommitted changes in ... (did you forget to amend?)
prompt when there are untracked (unknown) files in the working copy.
The prompt is still shown if tracked files are modified.
Change-Id: Ia3fcc82989b7fad09b69214eda31e2d0dfc14600
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/340456
Tested-by: Martin Geisler <mgeisler@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
If the user says `--no-use-superproject`, then do not bother syncing the
superproject.
Also add/update docstrings and comments throughout.
Change-Id: I9cdad706130501bab9a22d3099a1dae605e9c194
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/338975
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
With this change, partial syncs (sync with a project list) are again
supported.
If the updated manifest includes new sub manifests, download them
inheriting options from the parent manifestProject.
Change-Id: Id952f85df2e26d34e38b251973be26434443ff56
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334819
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
For now, this is opt-in via environment variables:
- export REPO_USE_ALTERNATES=1
The shared project logic that shares the internal .git/objects/ dir
directly between multiple projects via the project-objects/ tree has
a lot of KI with random corruption. It all boils down to projects
sharing objects/ but not refs/. Git operations that use refs to see
what objects are reachable and discard the rest can easily discard
objects that are used by other projects.
Consider this project layout:
<show fs layout>
There are unique refs in each of these trees that are not visible in
the others. This means it's not safe to run basic operations like
git prune or git gc.
Since we can't share refs (each project needs to have unique refs
like HEAD in order to function), let's change how we share objects.
The old way involved symlinking .git/objects/ to the project-objects
tree. The new way shares objects using git's info/alternates.
This means project-objects/ will only contain objects that exist in
the remote project. Local per-project objects (like when creating
branches and making changes) will never be shared. When running a
prune or gc operation in the per-project state, it will only ever
repack or discard those per-project objects. The common shared
objects would only be cleaned up when running a common operation
(i.e. by repo itself).
One downside to this for users is if they try blending unrelated
upstream projects. For example, in CrOS we have multiple kernel
projects (for diff versions) checked out. If a dev fetched the
upstream Linus tree into one of them, the objects & tags would
not be shared with the others, so they would have to fetch the
upstream state for each project. Annoying, but better than the
current corruption situation we're in now.
Also if the dev runs a manual `git fetch` in the per-project to
sync it up to newer state than the last `repo sync` they ran,
the objects would get duplicated. However, git operations later
on should eventually dedupe this.
Bug: https://crbug.com/gerrit/15553
Change-Id: I313a9b8962f9d439ef98ac0ed37ecfb9e0b3864e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328101
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
This way we know we don't need to encode the labels.
Change-Id: Ib83ed8f4ed05f00b9d2d06a9dd3f304e4443430e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/337518
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
If init was run with --use-superproject, init failed.
If init was run without --{no,}use-superproject option then manifests
with <superproject/> elements were mishandled.
Bug: b/233226285
Test: manual
Change-Id: I737e71c89d2d7c324114f58bf2dc82b40e5beba7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/337534
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
When the user does not specify any manifest groups, this allows the
parent manifest to indicate which manifest groups should be used for
syncing the submanifest.
Change-Id: I88806ed35013d13dd2ab3cd245fcd4f9061112c4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/335474
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
When a manifest file is overridden, remember that and keep using the
override for the remainder of the process. If we need to revert it,
make the override name evaluate False.
Change-Id: I1eee05fec6988c1ee4a3c751c4b540d5b5d11797
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/335136
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
