Prior to this change there is no way to distinguish between git sessions logs
generated from repo source v.s. from git.
Bug: b/294446468
Change-Id: I309f59e146c30cb08a0637e8d0b9c5d9efd5cada
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/381794
Commit-Queue: Jason Chang <jasonnc@google.com>
Reviewed-by: Joanna Wang <jojwang@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Per discussion in go/repo-error-update updated aggregated and exit
errors for sync command.
Aggregated errors are errors that result in eventual command failure.
Exit errors are errors that result in immediate command failure.
Also updated main.py to log aggregated and exit errors to git sessions
log
Bug: b/293344017
Change-Id: I77a21f14da32fe2e68c16841feb22de72e86a251
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/379614
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Commit-Queue: Jason Chang <jasonnc@google.com>
In order to better analyze and track repo errors, repo command failures
need to be tied to specific errors in repo source code.
Additionally a new GitCommandError was added to differentiate between
general git related errors to failed git commands. Git commands that opt
into verification will raise a GitCommandError if the command failed.
The first step in this process is a general error refactoring
Bug: b/293344017
Change-Id: I46944b1825ce892757c8dd3f7e2fab7e460760c0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/380994
Commit-Queue: Jason Chang <jasonnc@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Reviewed-by: Joanna Wang <jojwang@google.com>
Save the latest time any project is fetched and checked out. This will
be used to detect partial checkouts.
Bug: b/286126621
Change-Id: I53b264dc70ba168d506076dbd693ef79a696b61d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/380514
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Joanna Wang <jojwang@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Fixed a couple of bugs in ExitEvent logging:
- log exitcode 130 on KeyboardInterrupt
- log exitcode 1 on unhandled Exception
- log errorevent with specific reason for exit
Before this CL an exitcode of 0 would be logged, and it would be
difficult to determine the cause of non-zero exit codes
Bug: b/287105597
Change-Id: I2d34f180581f9fbd77a1c78c966ebed065223af6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/377834
Tested-by: Jason Chang <jasonnc@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
New vs existing project may be a useful measure for analyzing
sync performance.
Bug: b/287105597
Change-Id: Ibea3e90c9fe3d16fd8b863bcae22b21963a6771a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/377574
Tested-by: Jason Chang <jasonnc@google.com>
Reviewed-by: Joanna Wang <jojwang@google.com>
A something.xml that gets included by two different
files, that both remove and add same shared project
to two different locations, would not work
prior to this change.
Reason is that remove killed all name keys, even
though reuse of same repo in different locations
is allowed.
Solve by adding optional attrib path to
<remove-project name="foo" path="only_this_path" />
and tweak remove-project.
Behaves as before without path, and deletes
more selectively when remove path is supplied.
As secondary feature, a project can now also be removed
by only using path, assuming a matching project name
can be found.
Change-Id: I502d9f949f5d858ddc1503846b170473f76dc8e2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/375694
Tested-by: Fredrik de Groot <fredrik.de.groot@aptiv.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
By chance, _sync_dict can be empty even though repo sync is still
working. In that case, the progress message shows incorrect info. Handle this case and fix a bug where "0 jobs" can show.
Bug: http://b/284465096
Change-Id: If915d953ba60e7cf84a6fb2d137fd6ed82abd3cc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/375494
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
It's possible that number of jobs is more than 0 when we
check length, but in the meantime number of jobs drops to
0. In that case, we are working with float(inf) which
causes other problems
Bug: 284383869
Change-Id: I5d070d1be428f8395df7fde8ca84866db46f2100
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/375134
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This matches the git behavior. From [1],
> When the `LESS` environment variable is unset, Git sets it to `FRX`
> (if `LESS` environment variable is set, Git does not change it at
> all).
The default $LESS is changed from FRSX to FRX since git 2.1.0 [2]. This
change also updates the default $LESS for repo.
[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-corepager
[2] b3275838d9
Bug: https://crbug.com/gerrit/16973
Change-Id: I64ccaa7b034fdb6a92c10025e47f5d07e85e6451
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/374894
Reviewed-by: Chih-Hsuan Yen <x5f4qvj3w3ge2tiq@chyen.cc>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Chih-Hsuan Yen <x5f4qvj3w3ge2tiq@chyen.cc>
An investigation go/git-repo-shallow shows a number of problems
when doing a shallow git fetch/clone. This change introduces an
environment variable REPO_ALLOW_SHALLOW. When this environment variable
is set to 1 during a repo init or repo sync all shallow git fetch
commands are replaced with partial fetch commands. Any shallow
repository needing update is unshallowed. This behavior continues until
a subsequent repo sync command is run with REPO_ALLOW_SHALLOW set to 1.
Bug: b/274340522
Change-Id: I1c3188270629359e52449788897d9d4988ebf280
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/374754
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Last of the recent `repo sync` UX changes. Show number of fetch jobs eg:
"Fetching: 3% (8/251) 0:03 | 8 jobs | 0:01 chromiumos/overlays/chrom.."
Bug: https://crbug.com/gerrit/11293
Change-Id: I1b3dcf3e56ae6731c6c6cb73cfce069b2f374b69
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/374920
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Joanna Wang <jojwang@google.com>
Change-Id: I91b99a7147c7c3cb5485d5406316c8ffd79f9272
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/374914
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Joanna Wang <jojwang@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
If the original fetch attempt did not want tags, we should continue to
honor that when doing a retry fetch with depth set to None. This seems
to match the intent of the retry based on the inline comment and results
in a significant performance improvement when the original fetch-by-sha1
fails due to the server not allowing requests for unadvertised objects.
Change-Id: Ia26bb31ea9aecc4ba2d3e87fc0c5412472cd98c4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/374918
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Kaushik Lingarkar <kaushik.lingarkar@linaro.org>
Tested-by: Kaushik Lingarkar <kaushik.lingarkar@linaro.org>
If the Debian banner is not used, then there won't be a space after the
version number: it'll be followed directly by a comma.
Bug: https://crbug.com/gerrit/16903
Change-Id: I12b873f32afc9424f42b772399c346f96ca95a96
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/372875
Tested-by: Saagar Jha <saagarjha@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
"Last synced: X" is printed only after a project finishes syncing.
Replace that with a message that shows the longest actively syncing
project.
Bug: https://crbug.com/gerrit/11293
Change-Id: I84c7873539d84999772cd554f426b44921521e85
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/372674
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Joanna Wang <jojwang@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Trace logs emitted from repo are not useful on error for many critical
commands. This change adds errors for critical commands to trace logs.
Change-Id: Ideb9358bee31e540bd84a94327a09ff9b0246a77
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/373814
Reviewed-by: Joanna Wang <jojwang@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Attribute groups can now be added to manifest include, thus
all projects in an included manifest file can easily modify
default branch without modifying all projects in that manifest file.
For example,
the main manifest.xml has an include node contain revision attribute,
```
<include name="include.xml" revision="r1" />
```
and the include.xml has some projects,
```
<project path="project1_path" name="project1_name" revision="r2" />
<project path="project2_path" name="project2_name" />
```
With this change, the final manifest will have revision="r1" for project2.
```
<project name="project1_name" path="project1_path" revision="r2" />
<project name="project2_name" path="project2_path" revision="r1" />
```
Test: added unit tests to cover the inheritance
Change-Id: I4b8547a7198610ec3a3c6aeb2136e0c0f3557df0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/369714
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Shuchuan Zeng <zengshuchuan@allwinnertech.com>
Tested-by: Shuchuan Zeng <zengshuchuan@allwinnertech.com>
Give users an indication that `repo sync` isn't stuck if taking a long
time to fetch.
Bug: https://crbug.com/gerrit/11293
Change-Id: Iccdaec918f86c9cc2db5dc12f9e3eef7ad0bcbda
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/371414
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Joanna Wang <jojwang@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
By default, datetime.isoformat() uses different format depending on
microseconds - if is equal to 0, microseconds are omitted, but otherwise
not.
Setting timespec = 'microseconds' ensures the format is the same
regardless of current time.
Change-Id: Icb1be31eb681247c7e46923cdeabb8f5469c20f0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/371694
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Joanna Wang <jojwang@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
We use dashes in --long-options, not underscores, so add a test to
make sure people don't accidentally add them.
Change-Id: Iffbce474d22cf1f6c2042f7882f215875c8df3cf
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/369734
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
https://gerrit-review.googlesource.com/c/git-repo/+/363474/24..25 meant
to improve run_tests UX by letting users rerun it quickly, but it also
removed CQ enforcement of formatting since CQ passes args to run_tests.
Run pytest first so devs don't have format first and always check black
formatting so it's enforced in CQ.
Bug: b/267675342
Change-Id: I09544f110a6eb71b0c6c640787e10b04991a804e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/365727
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Apply rules set by https://gerrit-review.googlesource.com/c/git-repo/+/362954/ across the codebase and fix any lingering errors caught
by flake8. Also check black formatting in run_tests (and CQ).
Bug: b/267675342
Change-Id: I972d77649dac351150dcfeb1cd1ad0ea2efc1956
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/363474
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Provide a consistent formatting style and tox commands to lint and
format.
Bug: b/267675342
Change-Id: I33ddfe07af8473f4334c347d156246bfb66d4cfe
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/362954
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
It ships with Python 3.8 by default, not 3.7.
Change-Id: I11401d1098b60285cfdccadb6a06bb33a5f95369
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/361634
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
dest_branch.
- This still prevents the case mentioned here:
https://gerrit-review.googlesource.com/c/50300
while also supporting dest_branch.
- Update _GetMergeBranch to get merge branches for any branch, not just
the one we happen to run `repo upload` in. (e.g. for uploading multiple
branches)
Bug: b/27955930
Change-Id: Ia8ee1d6a83a783c984bb2eb308bb11b3a721a95d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/360794
Commit-Queue: Joanna Wang <jojwang@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Joanna Wang <jojwang@google.com>
Remove logic to handle importing the right version of pytest.
'./run_tests' still works but this allows presubmit builders to test
using 'vpython3 ./run_tests'.
Google-Bug-Id: b/266734831
Change-Id: I6a543c1f4b5b4449e723095b4a70e5228b1ccd34
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/356717
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
If interrupt signal is sent to repo process while sync is running, repo
prints stack trace for each concurrent job that is currently running
with no useful information.
Instead, this change captures KeyboardInterrupt in each process and
prints one line about current project that is being processed.
Change-Id: Ieca760ed862341939396b8186ae04128d769cd56
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/357135
Reviewed-by: Joanna Wang <jojwang@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
repo sync progress bar is misleading. Many bug reports mentioned that
repo is stuck at the repo that is currently displayed in the progress
bar. Repo sync actually shows what repository is the last processed.
This change makes that obvious.
Change-Id: I962bf0bc65af7ac0ed98db86e9144f07d9e1f96f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/357134
Reviewed-by: Joanna Wang <jojwang@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Superproject update failures on single-manifest checkouts had an extra
space.
Bug: b/254523816
Change-Id: I6f71e42337e324a6975c5d6bba487f83abaf054f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/357056
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Update monorail component where actual git-repo bugs are.
Change-Id: I46c68053683d7aa93585bb5633a598f1578b1468
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/357057
Reviewed-by: Joanna Wang <jojwang@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
This can potentially show up when sync'ing projects with submodules
that are not declared in the manifest as well as the internal
'.repo/repo' project, which is likely not desirable from a user
standpoint.
Change-Id: I93d7fcd6e3fd1818357ea4537882a864dea9942c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/355920
Reviewed-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Michael Kelly <mkelly@arista.com>
Treat the values in the regex map as raw strings to fix
Invalid escape sequence 'g' (W605).
Change-Id: I53bf5d6bd1e1d6a1d1293e4f55640b6513bf3075
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/354698
Tested-by: Daniel Kutik <daniel.kutik@lavawerk.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Resolve TODO as we are now requiring Python 3.
Change-Id: I7821627bd5c606276741c98efedaf5b11aecbcc3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/354702
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Daniel Kutik <daniel.kutik@lavawerk.com>
