Now that we don't need to support Python 2, we can switch to this
API for better contextmanager logic.
Change-Id: I2d03e391121886547e7808a3b5c3b470c411533f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/337515
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Always create Superproject when there is a <superproject> tag, and have
it hold the XML content, similar to how other manifest elements are
handled.
This also adds SetQuiet and SetPrintMessages to Superproject
consistent with manifest.SetUseLocalManifests.
Change-Id: I522bf3da542006575799f0640c67f7052704f266
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334641
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
This allows us to move the repository to a new location in the source
tree without having to remove-project + add a new project tag.
Change-Id: I4dba6151842e57f6f2b8fe60cda260ecea68b7b4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310962
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Michael Kelly <mkelly@arista.com>
Tested:
$ ./run_tests
Verified that a manifest that specified superproject revision would use
the specified revision, and superproject will use the default revision.
Note that this is a slight behavior change from earlier repo versions,
which would always use the branch name of the manifest itself. However,
the new behavior would be more consisitent with regular "project"
element and would allow superproject be used even if it is not enabled
for the particular manifest branch, so we have decided to make the
change as it would provide more flexibility and better matches what
other elements would do.
Bug: [google internal] b/187868160
Change-Id: I35255ee347aff6e65179f7879d52931f168b477e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/317643
Tested-by: Xin Li <delphij@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Currently, repo will fail to sync to a manifest if the definition
of the repo-hooks project comes after the repo-hooks element.
BUG=none
TEST=new test, run_tests
Change-Id: I0bf85625173492af6c6404d4b67543e96e670562
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/318520
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jack Neus <jackneus@google.com>
Sometimes, we don't care if the remove project is referring to a
non-existing project and we can just ignore it. This change allows us
to ignore remove-project entries if the project that they refer to
doesn't exist, making them effectively a no-op.
Because this change breaks existing configuration, we allow this to be
configuration controlled using the `optional` attribute in the
remove-project tag.
Change-Id: I6313a02983e81344eadcb4e47d7d6b037ee7420e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310964
Tested-by: Michael Kelly <mkelly@arista.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
It is possible that a clone bundle contained the object referenced by
the branch in the manifest and in the superproject, but not the branch
itself (for example, the branch may be newly created from an existing
branch, or is not vislble to the user downloading the clone bundle).
When --use-superproject is enabled, because we are overriding
revisionExpr with the SHA1 revision provided by the superproject, the
verification step would succeed, but because the expected branch do not
exist, it would confuse git-repo at a later time, as it is expecting the
remote branch to exist in the local clone.
In project.py, fix this by making SetRevisionId() to always remember
the actual branch name and verify it in _CheckForImmutableRevision()
so that we only skip the fetch step when both objects exists locally.
Bug: [google internal] b/191974277
Change-Id: I49d3ca0667f524c8c45f416492faf95b1dd822fb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310802
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Xin Li <delphij@google.com>
Before Python 3.8, xml.dom.minidom sorted the attributes of an element
when writing it to a file, while later versions output the attributes
in the order they were created. Avoid these differences by sorting the
attributes for each element before comparing the generated manifests
with the expected ones.
Bug: https://crbug.com/gerrit/14382
Change-Id: Ie2597727afcc48f9063a7261ad970e8a549f0587
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303326
Reviewed-by: Mike Frysinger <vapier@google.com>
+ In XmlManifest._Unload set 'bugurl' to Wrapper().BUG_URL.
+ contactinfo returns a namedtuple.
+ bug_url can be accessed as self._manifest.contactinfo.bugurl.
Tested the code with the following commands.
$ ./run_tests -v
Added contactinfo tag to default.xml and verified that bugurl is used.
Bug: [google internal] b/186220520.
Change-Id: Iaafd6465e072b2e47a0a0b548bf6cb608a0b0a04
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/306342
Tested-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
It will be used to let manifest authors self-register contact info.
This element can be repeated, and any later entries will clobber
earlier ones. This would allow manifest authors who extend
manifests to specify their own contact info.
It would have 1 required attribute: bugurl.
"bugurl" specifies the URL to file a bug against the manifest owner.
<contactinfo bugurl="bug-url"/>
TODO: This CL only implements the parsing logic and further work
will be in followup CLs.
Tested the code with the following commands.
$ ./run_tests tests/test_manifest_xml.py
$ ./run_tests -v
Bug: [google internal] b/186220520.
Change-Id: I47e765ba2dab5cdf850191129f4d4cd6b803f451
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305203
Tested-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Re-allow checking out projects to the top of the repo client checkout.
We add checks to prevent checking out files under .repo/ as that path
is only managed by us, and projects cannot inject content or settings
into it.
Bug: https://crbug.com/gerrit/14156
Bug: https://crbug.com/gerrit/14200
Change-Id: Id6bf9e882f5be748442b2c35bbeaee3549410b25
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299623
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Added the following methods to XmlManifest class.
+ GetDefaultGroupsStr() - return 'default,platform-' + platform.system().lower()
+ GetGroupsStr() - Same as gitc_utils.py's _manifest_groups func.
+ Replaced gitc_utils.py's_manifest_groups calls with GetGroupsStr.
+ Used the above methods to get groups in command.py::GetProjects
and part of init.py.
TODO: clean up these funcs to take structured group data more instead
of passing strings around everywhere that need parsing.
Tested the code with the following commands.
$ ./run_tests -v
Tested the sync code by using repo_dev alias and pointing to this CL
and verified prebuilts/fullsdk-linux directory has all the folders.
Tested repo init and repo sync with --use-superproject and without
--use-superproject argument.
$ repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject
$ repo_dev sync -c -j32
Bug: [google internal] b/181804931
Bug: https://crbug.com/gerrit/13707
Change-Id: Ia98585cbfa3a1449710655af55d56241794242b6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299422
Reviewed-by: Jonathan Nieder <jrn@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
Allow the user to specify relative or absolute or any other funky
path that they want when using `repo init` or `repo sync`. Our
goal is to restrict the paths in the remote manifest git repo we
cloned from the network, not protect the user from themselves.
Bug: https://crbug.com/gerrit/14156
Change-Id: I1ccfb2a6bd1dce2bd765e261bef0bbf0f8a9beb6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298823
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
These attribute values are used to construct local filesystem paths,
so apply the existing filesystem checks to them.
Bug: https://crbug.com/gerrit/14156
Change-Id: Ibcceecd60fa74f0eb97cd9ed1a9792e139534ed4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298443
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
This function is currently written with copyfile & linkfile in mind.
Generalize the logic & function arguments slightly so we can reuse
in more places that make sense.
This changes the validation logic slightly too in that we no longer
allow "." for the dest attribute with copyfile & linkfile, nor for
the src attribute with copyfile. We already rejected those later on
when checking against the active filesystem, but now we reject them
a little sooner when parsing.
The empty path check isn't a new requirement exactly -- repo used to
crash on it, so it was effectively blocked, but now we diagnosis it.
Bug: https://crbug.com/gerrit/14156
Change-Id: I0fdb42a3da60ed149ff1997c5dd4b85da70eec3d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298442
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
The XmlManifestTests class is getting to be large and we're only
adding more to it. Factor out the core logic into a new TestCase
so we can reuse it to better group more tests.
Change-Id: I5113444a4649a70ecfa8d83d3305959a953693f7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298222
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
As we were testing superproject setting revisionId attribute to SHA and
reloading the manifest, we found out revisionId attribute is not being
saved. Made the change to save the revisionId if it is not being saved.
Tested the code with the following commands.
$ ./run_tests -v
Bug: https://crbug.com/gerrit/13709
Bug: https://crbug.com/gerrit/13707
Tested-by: Raman Tenneti <rtenneti@google.com>
Change-Id: I95fdf655b19648ad3e9aba10b9bed8bb9439deb6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/296182
Reviewed-by: Ian Kasprzak <iankaz@google.com>
Added "--use-superporject" option to sync.py to fetch project SHAs from
superproject. If there are any missing projects in superprojects, it
prints the missing entries and exits. If there are no missing entries,
it will use SHAs from superproject to fetch the projects from git.
Tested the code with the following commands.
$ ./run_tests tests/test_manifest_xml.py
$ ./run_tests -v tests/test_git_superproject.py
$ ./run_tests -v
Tested the sync code by copying all the repo changes into my Android
AOSP checkout and adding <superporject> tag to default.xml. With
local modification to the code to print the status,
.../WORKING_DIRECTORY$ repo sync --use-superproject
repo: executing 'git clone' url: sso://android/platform/superproject
repo: executing 'git ls-tree'
Success: []
Bug: https://crbug.com/gerrit/13709
Tested-by: Raman Tenneti <rtenneti@google.com>
Change-Id: Id18665992428dd684c04b0e0b3a52f46316873a0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/293822
Reviewed-by: Mike Frysinger <vapier@google.com>
Added this test to verify that older versions of repo can handle
"<superproject" element. Tested by adding "<iankaz" unknown element.
Tested the code with the following commands.
$ ./run_tests tests/test_manifest_xml.py
$ ./run_tests -v
Bug: https://crbug.com/gerrit/13709
Tested-by: Raman Tenneti <rtenneti@google.com>
Change-Id: I858d56f38cefcfcd14474efdd631a5a940c3ce47
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/293482
Reviewed-by: Mike Frysinger <vapier@google.com>
At most one superproject may be specified. It will be used
to specify the URL of superproject.
It would have 3 attributes: remote, name, and default.
Only "name" is required while the others have reasonable defaults.
<remote name="superproject-url" review="<url>" />
<superproject remote="superproject-url" name="platform/superproject"/>
TODO: This CL only implements the parsing logic and further work
will be in followup CLs.
Tested the code with the following commands.
$ ./run_tests tests/test_manifest_xml.py
$ ./run_tests -v
Bug: https://crbug.com/gerrit/13709
Tested-by: Raman Tenneti <rtenneti@google.com>
Change-Id: I5b4bba02c8b59601c754cf6b5e4d07a1e16ce167
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292982
Reviewed-by: Mike Frysinger <vapier@google.com>
We allow project.groups to be whitespace or comma delimited, but
repo-hooks.enabled-list is only whitespace delimited. This hasn't
been a big deal as it's only ever had one valid value, but if we
want to add more, we should harmonize these a bit.
Refactor the groups method to be more generic, and run the enabled-
list attribute through it. Then add missing docs for it.
Change-Id: Iaa96a0faa9c4a68b313b49336751831b73bf855d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/290743
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Attrib groups can now be added to manifest include, thus
all projects in an included manifest file can easily be tagged
with a group without modifying all projects in that manifest file.
Include groups will add and recurse, meaning included manifest
projects will carry all parent includes. Intentionally, no support
added for group remove, to keep complexity down.
Group handling for projects is untouched, meaning a group set on
a project will still append to whatever was or was not inherited
in parent manifest includes, resulting in union of groups inherited
and set for the project itself.
Test: manual multi-level manifest include structure, in serial and parallel,
with different groups set on init
Test: added unit tests to cover the inheritance
Change-Id: Id2229aa6fd78d355ba598cc15c701b2ee71e5c6f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/283587
Tested-by: Fredrik de Groot <fredrik.de.groot@volvocars.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
We conflate the manifest & parsing logic with the management of the
repo client checkout in a single class. This makes testing just one
part (the manifest parsing) hard as it requires a full checkout too.
Start splitting the two apart into separate classes to make it easy
to reason about & test.
Change-Id: Iaf897c93db9c724baba6044bfe7a589c024523b2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/288682
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
We've been overly lenient with boolean parsing by ignoring invalid
values as "false" even if the user didn't intend that. Turn all
unknown values into warnings to avoid breaking existing manifests,
and unify the parsing logic in a helper to simplify.
We've been stricter about numbers, but still copying & pasting
inconsistent code. Add a helper for this too. For out of range
sync-j numbers (i.e. less than 1), throw a warning for now, but
mark it for future hard failures.
Change-Id: I924162b8036e6a5f1e31b6ebb24b6a26ed63712d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256457
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
We also need to check more things in the manifest/project handlers,
and use platform_utils in a few places to address Windows behavior.
Drop Python 2.7 from Windows testing as it definitely doesn't work
and we won't be fixing it.
Change-Id: I83d00ee9f1612312bb3f7147cb9535fc61268245
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256113
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Some Android/Nest manifests are using <linkfile> with src="." to
create stable paths to specific projects. Allow that specific
use case as it seems reasonable to support.
Bug: https://crbug.com/gerrit/11218
Change-Id: I5eadec257cd58ba0f8687c590ddc250a7a414a85
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254276
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Reject paths in <copyfile> & <linkfile> that point outside of their
respective scopes. This validates paths while parsing the manifest
as this should be quick & cheap: we don't access the filesystem as
this code runs before we've synced.
Bug: https://crbug.com/gerrit/11218
Change-Id: I8e17bb91f3f5b905a9d76391b29fbab4cb77aa58
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/232932
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Michael Mortensen <mmortensen@google.com>
