In order to better analyze and track repo errors, repo command failures
need to be tied to specific errors in repo source code.
Additionally a new GitCommandError was added to differentiate between
general git related errors to failed git commands. Git commands that opt
into verification will raise a GitCommandError if the command failed.
The first step in this process is a general error refactoring
Bug: b/293344017
Change-Id: I46944b1825ce892757c8dd3f7e2fab7e460760c0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/380994
Commit-Queue: Jason Chang <jasonnc@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Reviewed-by: Joanna Wang <jojwang@google.com>
Save the latest time any project is fetched and checked out. This will
be used to detect partial checkouts.
Bug: b/286126621
Change-Id: I53b264dc70ba168d506076dbd693ef79a696b61d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/380514
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Joanna Wang <jojwang@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
A something.xml that gets included by two different
files, that both remove and add same shared project
to two different locations, would not work
prior to this change.
Reason is that remove killed all name keys, even
though reuse of same repo in different locations
is allowed.
Solve by adding optional attrib path to
<remove-project name="foo" path="only_this_path" />
and tweak remove-project.
Behaves as before without path, and deletes
more selectively when remove path is supplied.
As secondary feature, a project can now also be removed
by only using path, assuming a matching project name
can be found.
Change-Id: I502d9f949f5d858ddc1503846b170473f76dc8e2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/375694
Tested-by: Fredrik de Groot <fredrik.de.groot@aptiv.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
If the Debian banner is not used, then there won't be a space after the
version number: it'll be followed directly by a comma.
Bug: https://crbug.com/gerrit/16903
Change-Id: I12b873f32afc9424f42b772399c346f96ca95a96
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/372875
Tested-by: Saagar Jha <saagarjha@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Attribute groups can now be added to manifest include, thus
all projects in an included manifest file can easily modify
default branch without modifying all projects in that manifest file.
For example,
the main manifest.xml has an include node contain revision attribute,
```
<include name="include.xml" revision="r1" />
```
and the include.xml has some projects,
```
<project path="project1_path" name="project1_name" revision="r2" />
<project path="project2_path" name="project2_name" />
```
With this change, the final manifest will have revision="r1" for project2.
```
<project name="project1_name" path="project1_path" revision="r2" />
<project name="project2_name" path="project2_path" revision="r1" />
```
Test: added unit tests to cover the inheritance
Change-Id: I4b8547a7198610ec3a3c6aeb2136e0c0f3557df0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/369714
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Shuchuan Zeng <zengshuchuan@allwinnertech.com>
Tested-by: Shuchuan Zeng <zengshuchuan@allwinnertech.com>
We use dashes in --long-options, not underscores, so add a test to
make sure people don't accidentally add them.
Change-Id: Iffbce474d22cf1f6c2042f7882f215875c8df3cf
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/369734
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Apply rules set by https://gerrit-review.googlesource.com/c/git-repo/+/362954/ across the codebase and fix any lingering errors caught
by flake8. Also check black formatting in run_tests (and CQ).
Bug: b/267675342
Change-Id: I972d77649dac351150dcfeb1cd1ad0ea2efc1956
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/363474
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
We don't want symlinks in the git tree as it causes pain for Windows
users. We also don't really need it as we can refactor the code we
want to import slightly.
Change-Id: I4537c07c50ee9449e9f53e0f132a386e8ffe16ec
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/354356
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
Patch this automatically for all tests rather than duplicating the
boilerplate in diff testcases.
Change-Id: I391d5c859974cda3d5680d34ede2ce6e9e925838
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/354358
Reviewed-by: Joanna Wang <jojwang@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
No need to implement our own caching logic with newer Python.
Change-Id: Idc3243b8e22ff020817b0a4f18c9b86b1222d631
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/354357
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
We use pytest now which doesn't need this boilerplate.
Change-Id: Ib71d90b3f1669897814ee768927b5b595ca8d789
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/354355
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Since --jobs defaults to 0, not None, we never pull the value out
of the manifest. Treat values of 0 and None the same to fix.
Bug: http://b/239712300
Bug: http://b/260908907
Change-Id: I9b1026682072366616825fd72f90bd90c10a252f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/354254
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
Multiple "Could not find reference" warnings in test_wrapper.py
and test_git_command.py resolved.
Test: tox
Signed-off-by: Daniel Kutik <daniel.kutik@lavawerk.com>
Change-Id: Ic254c378bbdae6bc3f8f29682ababb37db76adfe
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353235
Reviewed-by: Mike Frysinger <vapier@google.com>
On Linux/macOS we allow \n in the end of the line.
On Windows we allow both \r\n and \n. Here we also allow Unix line
seperators as tests might be excuted in for example git-shell.
Change-Id: I3975b563cf95407da92e5479980e670eb748b30e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353181
Tested-by: Daniel Kutik <daniel.kutik@lavawerk.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
With this change if a path ends with '/' on Linux/macOS
and ends with either '/' or '\' on Windows, the test will pass.
Signed-off-by: Daniel Kutik <daniel.kutik@lavawerk.com>
Change-Id: Id7d1b134f9c0bdf7ceaf149af304bbf90cbd7b21
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353180
Reviewed-by: Mike Frysinger <vapier@google.com>
Some tests were failing due to Windows not using utf-8
by default when executing the tests. Enforcing usage of utf-8
resolves these issues.
Signed-off-by: Daniel Kutik <daniel.kutik@lavawerk.com>
Change-Id: If42f6be2a2b688a6105ecf4fcdb541aade24519a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353179
Reviewed-by: Mike Frysinger <vapier@google.com>
Fixing multiple errors when running tests on Windows related
to path seperator being different ('\' instead of '/').
Signed-off-by: Daniel Kutik <daniel.kutik@lavawerk.com>
Change-Id: I26b44d092b925edecab46a4d88e77dd9dcb8df28
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353178
Reviewed-by: Mike Frysinger <vapier@google.com>
Before this commit, the test was hanging forever when
run on a Windows host. This should resolve that issue.
Signed-off-by: Daniel Kutik <daniel.kutik@lavawerk.com>
Change-Id: Id9ea6d54926b797db3d2978a2ae2930088201eec
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353125
Reviewed-by: Mike Frysinger <vapier@google.com>
Previously env dict building was untested and mixed with other mutative
actions. Extract the dict building into a dedicated function and author
tests to ensure the functionality is working as expected.
BUG: b/255376186
BUG: https://crbug.com/gerrit/16247
Change-Id: I0c88e53eb285c5c3fb27f8e6b3a903aedb8e02a8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/351874
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Sam Saccone <samccone@google.com>
On some systems, help2man produces color codes in the output. Remove
them to avoid manpage churn.
Also begin adding unit tests.
Change-Id: I3f0204b19d9cae524d3cb5fcfb61ee309b0931fc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/349655
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
If this is a project that is not using object sharing (there is only one
copy of the remote project) then clear preciousObjects.
To override this for a project, run:
git config --replace-all repo.preservePreciousObjects true
Change-Id: If3ea061c631c5ecd44ead84f68576012e2c7405c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350235
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
Before Python 3.8, xml.dom.minidom sorted the attributes of an element
when writing it to a file, while later versions output the attributes
in the order they were created. Avoid these differences by sorting the
attributes for each element before comparing the generated manifests
with the expected ones.
This corresponds to commit 5d58c18, but for new tests introduced since
it was integrated.
Change-Id: I5c360656a0968e6e8d57eb068c8e87da7dfa61c1
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/349917
Reviewed-by: LaMont Jones <lamontjones@google.com>
When we create superproject_override.xml, do not include projects that
are present from local_manifests/*. Such projects are fully under the
control of the local_manifests/ file.
Bug: b/238934278
Test: manual, ./run_tests
Change-Id: I40382ceb82d9cf7b8dc7b5f2abed3f6d4d80017e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/340877
Tested-by: Xin Li <delphij@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Reviewed-by: Sam Saccone 🐐 <samccone@google.com>
Now that we don't need to support Python 2, we can switch to this
API for better contextmanager logic.
Change-Id: I2d03e391121886547e7808a3b5c3b470c411533f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/337515
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
We need to iterate over multiple manifests, and generally use the
outer_client.manifest for multi-manifest support. This refactors the
use of self.manifest into a chosen manifest.
Change-Id: I992f21d610c929675e99555ece9c38df4b635839
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334699
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Always create Superproject when there is a <superproject> tag, and have
it hold the XML content, similar to how other manifest elements are
handled.
This also adds SetQuiet and SetPrintMessages to Superproject
consistent with manifest.SetUseLocalManifests.
Change-Id: I522bf3da542006575799f0640c67f7052704f266
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334641
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
The documentation states that a `sync-c` attribute in the manifest file
can set a default for whether only the current branch should be fetched
or all branches. This seems to have been broken for some time.
Commit 7356114 introduced the `--no-current-branch` CLI option and
relied on getting `None` via `optparse` if neither `--current-branch`
nor `--no-current-branch` was set to distinguish it from a boolean
value. If `None` was received, it would read the value from the manifest
option `sync-c`. The parsing went through the utility function
`_GetCurrentBranchOnly` which returned `True` if `--current-branch` had
been given on the command-line, or fell back on the "superproject"
setting, which would either return `True` or `None`. This would
incorrectly make `repo` fall back to the manifest setting even if the
user had given `--no-current-branch` if no superproject was requested --
the manifest became "too powerful":
Command-line Using superproject → `current_branch_only`
------------ ------------------ -----------------------
No From manifest
Yes True
--current-branch No True
--current-branch Yes True
--no-current-branch No From manifest ← wrong
--no-current-branch Yes True
In commit 0cb6e92 the superproject configuration value reading changed
from something that could return `None` to something that always
returned a boolean. If it returned `False`, this would then incorrectly
make `repo` ignore the manifest option even if neither
`--current-branch` nor `--no-current-branch` had been given. The
manifest default became useless:
Command-line Using superproject → `current_branch_only`
------------ ------------------ -----------------------
No False ← wrong
Yes True
--current-branch No True
--current-branch Yes True
--no-current-branch No False
--no-current-branch Yes True
By swapping the order in which the command-line option target and the
superproject setting is evaluated, things should work as documented:
Command-line Using superproject → `current_branch_only`
------------ ------------------ -----------------------
No From manifest
Yes True
--current-branch No True
--current-branch Yes True
--no-current-branch No False
--no-current-branch Yes True
Change-Id: I933c232d2fbecc6b9bdc364ebac181798bce9175
Tested-by: Daniel Andersson <daniel.r.andersson@volvocars.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334270
Reviewed-by: Mike Frysinger <vapier@google.com>
Git can write trace2 events to a Unix domain socket [1]. This can be
specified via Git's `trace2.eventTarget` config option, which we read to
determine where to log our own trace2 events. Currently, if the Git
config specifies a socket as the trace2 target, we fail to log any
traces.
Fix this by adding support for writing to a Unix domain socket,
following the same specification that Git supports.
[1]: https://git-scm.com/docs/api-trace2#_enabling_a_target
Change-Id: I928bc22ba04fba603a9132eb055141845fa48ab2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/332339
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Josh Steadmon <steadmon@google.com>
Path.readlink is only available on Python 3.9, breaking compatibility
with all python versions below. os.readlink is already used in other
places of this file, so use it here as well.
Change-Id: I5acf8f5334a3e7c8de9cea1939d7e2b9af5f30ae
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327844
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Sebastian Wagner <sebix@sebix.at>
Verify all the .git/ paths will be handled by the migration logic before
starting the migration. This way we still abort & log an error, but the
user gets to see it before we put the tree into a state that they have to
manually recover. Also add a few more known-safe-to-clobber paths.
Bug: https://crbug.com/gerrit/15273
Change-Id: If49d69b341bc960ddcafa30da333fb5ec7145b51
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327557
Reviewed-by: Colin Cross <ccross@android.com>
Tested-by: Mike Frysinger <vapier@google.com>
Historically we created a .git/ subdir in each source checkout and
symlinked individual files to the .repo/projects/ paths. This layer
of indirection isn't actually needed: the .repo/projects/ paths are
guaranteed to only ever have a 1-to-1 mapping with the actual git
checkout. So we don't need to worry about having files in .git/ be
isolated.
To that end, change how we manage the actual project checkouts from
a dir full of symlinks (and a few files) to a symlink to the internal
.repo/projects/ dir. This makes the code simpler & faster.
The directory structure we have today is:
.repo/
project-objects/chromiumos/third_party/kernel.git/
<paths omitted as not relevant to this change>
projects/src/third_party/kernel/
v3.8.git/
config
description -> …/project-objects/…/config
FETCH_HEAD
HEAD
hooks/ -> …/project-objects/…/hooks/
info/ -> …/project-objects/…/info/
logs/
objects/ -> …/project-objects/…/objects/
packed-refs
refs/
rr-cache/ -> …/project-objects/…/rr-cache/
src/third_party/kernel/
v3.8/
.git/
config -> …/projects/…/v3.8.git/config
description -> …/project-objects/…/v3.8.git/description
HEAD
hooks/ -> …/project-objects/…/v3.8.git/hooks/
index
info/ -> …/project-objects/…/v3.8.git/info/
logs/ -> …/projects/…/v3.8.git/logs/
objects/ -> …/project-objects/…/v3.8.git/objects/
packed-refs -> …/projects/…/v3.8.git/packed-refs
refs/ -> …/projects/…/v3.8.git/refs/
rr-cache/ -> …/project-objects/…/v3.8.git/rr-cache/
The directory structure we have after this commit:
.repo/
<nothing changes>
src/third_party/kernel/
v3.8/
.git -> …/projects/…/v3.8.git
Bug: https://crbug.com/gerrit/15273
Change-Id: I9dd8def23fbfb2f4cb209a93f8b1b2b24002a444
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323695
Reviewed-by: Mike Nichols <mikenichols@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
removeprefix is available i python 3.9. Mac and Linux are running in
a version below 3.9. Thus tests are failing with the following error:
"AttributeError: 'str' object has no attribute 'removeprefix' "
Replaced the removeprefix with custom function which we will delete
once Linux and macOS versions are updated.
Tested:
$ ./run_tests
Bug: [google internal] b/201453085
Change-Id: I9b4d564ff1176e1b4471805ef05472c1914cd9f9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319375
Tested-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
This allows us to move the repository to a new location in the source
tree without having to remove-project + add a new project tag.
Change-Id: I4dba6151842e57f6f2b8fe60cda260ecea68b7b4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310962
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Michael Kelly <mkelly@arista.com>
All the values of syncstate are strings, check the first byte and last
byte to see if it is an array. For syncstate data, there were no false
positives.
Tested:
$ repo_dev sync
Verified event logged for argv is "data-json".
$./run_tests
Bug: [google internal] b/201102002
Change-Id: Id56adb532b80267f08d09147ac663cdd5987ce87
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319075
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
Some of the file removal calls are subject to race conditions (if
something else deletes the file), so extend our remove API to have
an option to ignore ENOENT errors. Then update a bunch of random
call sites to use this new functionality.
Change-Id: I31a9090e135452033135337a202a4fc2dbf8b63c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319195
Reviewed-by: Sean McAllister <smcallis@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Fixed: "we need to make a special case for logging the argv; it
should probably be a "data-json" event so that we log this directly as
an array rather than an encoded string.
Tested:
$ repo_dev sync
Verified event logged for argv is "data-json".
$./run_tests
Bug: [google internal] b/201102002
Change-Id: I18ccec79c73c8dc931cb8afc472b2361db8aea4c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319055
Reviewed-by: Josh Steadmon <steadmon@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
Tested:
$ ./run_tests
Verified that a manifest that specified superproject revision would use
the specified revision, and superproject will use the default revision.
Note that this is a slight behavior change from earlier repo versions,
which would always use the branch name of the manifest itself. However,
the new behavior would be more consisitent with regular "project"
element and would allow superproject be used even if it is not enabled
for the particular manifest branch, so we have decided to make the
change as it would provide more flexibility and better matches what
other elements would do.
Bug: [google internal] b/187868160
Change-Id: I35255ee347aff6e65179f7879d52931f168b477e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/317643
Tested-by: Xin Li <delphij@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Currently, repo will fail to sync to a manifest if the definition
of the repo-hooks project comes after the repo-hooks element.
BUG=none
TEST=new test, run_tests
Change-Id: I0bf85625173492af6c6404d4b67543e96e670562
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/318520
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jack Neus <jackneus@google.com>
