The superproject is initialized as a bare repo in Superproject:_Init().
That means that later operations must treat it as a bare repository,
specifying the gitdir and setting 'bare' appropriately when launching
GitCommand()s. It's also OK not to specify cwd here because GitCommand()
will drop cwd if bare == True anyways.
With this change, it's possible to run `repo init` and `repo sync` with the
Git config 'safe.bareRepository' set to 'explicit'. This config strengthens
Git's security posture against embedded bare repository attacks like
https://github.com/justinsteven/advisories/blob/main/2022_git_buried_bare_repos_and_fsmonitor_various_abuses.md.
Bug: b/227257481
Change-Id: I954a64c6883d2ca2af9c603e7076fd83b52584e9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389794
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jason R. Coombs <jaraco@google.com>
Tested-by: Emily Shaffer <emilyshaffer@google.com>
Reviewed-by: Emily Shaffer <emilyshaffer@google.com>
Commit-Queue: Jason R. Coombs <jaraco@google.com>
Currently we don't deal with schema-less URIs like
`git@github.com:foo` at all resulting in a scenario where we append
them to the manifest repo URL.
In order to deal with this, we munge both the manifest URL and the
fetch URL into a format we like and proceed with that.
Bug: https://g-issues.gerritcodereview.com/issues/40010331
Change-Id: I7b79fc4ed276630fdbeb235b94e327b172f0879b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/386954
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Michael Kelly <mkelly@arista.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Change-Id: I41371feb88c85e9da0656b9fab04057c22d1dcf4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/391514
Tested-by: Jason R. Coombs <jaraco@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason R. Coombs <jaraco@google.com>
- Bump minimum version to Python 3.6.
- Use f-strings in a lot of places.
Change-Id: I2aa70197230fcec2eff8e7c8eb754f20c08075bb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389034
Tested-by: Jason R. Coombs <jaraco@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason R. Coombs <jaraco@google.com>
Found via pylint:
W0231: __init__ method from base class 'Transport'
is not called (super-init-not-called)
Just fixed for code correctness and to avoid potential future bugs.
Change-Id: Ie1e723c2afe65d026d70ac01a16ee7a40c149834
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390676
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Daniel Kutik <daniel.kutik@lavawerk.com>
Commit-Queue: Daniel Kutik <daniel.kutik@lavawerk.com>
When a new shared project is added to manifest, there's a short window
where objects can be deleted that are used by other projects.
To close that window, set preciousObjects during git init. For
non-shared projects, repo should correct the state in the same execution
instance.
Bug: 288102993
Change-Id: I366f524535ac58c820d51a88599ae2108df9ab48
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390234
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
In Python 3, these exceptions were merged into OSError, so switch
everything over to that.
Change-Id: If876a28b692de5aa5c62a3bdc8c000793ce52c63
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390376
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
When parsing the manifest XML, the code looks for a top
level DOM node named "manifest". However, it doesn't check
that it's an element type node so if there is also an XML
document type declaration node present (which has the same
name as the root element) then it selects the wrong node
and hence you end up with no projects defined at all.
Change-Id: I8d101caffbbc2a06e56136ff21302e3f09cfc96b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390357
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Chris Allen <chris.allen@arm.com>
Commit-Queue: Chris Allen <chris.allen@arm.com>
When running the tests in my environment, tests that derived from `test_wrapper.GitCheckoutTestCase` would fail on commit or tag due to incomplete or incorrect gpg config. Ideally, the tests should not be dependent on the user's git config. This change ensures $HOME (or Windows equivalent) is replaced for the session.
Bug: 302797407
Change-Id: Ib42b712dd7b6602fee6e18329a8c6d52fb9458b9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/388235
Tested-by: Jason R. Coombs <jaraco@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason R. Coombs <jaraco@google.com>
adding the recently released python 3.12 to our
list of test environments.
Change-Id: I05ec0129ad29c16fff65ddfb389f251571f811a2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389754
Commit-Queue: Daniel Kutik <daniel.kutik@lavawerk.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Daniel Kutik <daniel.kutik@lavawerk.com>
added python 3.11 to the test matrix.
Change-Id: I0533205b5a10105b3144f770aa08c4c649aaf6be
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389675
Tested-by: Daniel Kutik <daniel.kutik@lavawerk.com>
Commit-Queue: Daniel Kutik <daniel.kutik@lavawerk.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
This doc was written back in 2019 when we were planning on the Python 3
migration. It isn't relevant anymore, and people are reading it thinking
we still support Python 2. Rewrite it to match current requirements and
to make it clear there is no support for older versions.
Bug: 302871152
Change-Id: I2acf3aee1816a03ee0a70774db8bf4a23713a03f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389455
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Stop running old repohooks via python2. Abort immediately with a
clear error for the user.
Bug: 302871152
Change-Id: I750c6cbbf3c7950e249512bb1bd023c32587eef5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389454
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Python 2 can't even parse this code anymore due to syntax changes,
so there's no point in checking for it explicitly.
Bug: 302871152
Change-Id: I9852ace5f5079d037c60fd3ac490d77e074e6875
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389434
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Python 3.6 and 3.7 do not have a default value for lru_cache maxsize.
Not setting it would cause:
TypeError: Expected maxsize to be an integer or None
Change-Id: I32d4fb6a0040a0c24da0b2f29f22f85a36c96531
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/388737
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Daniel Kutik <daniel.kutik@lavawerk.com>
Commit-Queue: Daniel Kutik <daniel.kutik@lavawerk.com>
Bug: 302680231
Change-Id: Icd01dd2ce62d737a4acb114e729189cd31f6bde9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/388234
Tested-by: Jason R. Coombs <jaraco@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason R. Coombs <jaraco@google.com>
Sometimes it is expected that a GitCommand executed in repo fails. In
such cases indicate in trace logs that the error was expected.
Bug: b/293344017
Change-Id: If137fae9ef9769258246f5b4494e070345db4a71
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/387714
Commit-Queue: Jason Chang <jasonnc@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
tox uses virtualenv under its hood for managing virtual environments.
Virtualenv 20.22.0 dropped support for Python <= 3.6.
Since we want to test against Python 3.6 we need to make sure we use
a version of virtualenv earlier than 20.22.0.
This error was not stopping any tests from passing but was printed
multiple times to stderr when executing the py36 target:
Error processing line 1 of [...]/.tox/py36/[...]/_virtualenv.pth:
Traceback (most recent call last):
File "/usr/lib/python3.6/site.py", line 168, in addpackage
exec(line)
File "<string>", line 1, in <module>
File "[...]/.tox/py36/[...]/_virtualenv.py", line 3
from __future__ import annotations
^
SyntaxError: future feature annotations is not defined
Source: https://tox.wiki/en/latest/faq.html#testing-end-of-life-python-versions
Change-Id: I27bd8200987ecf745108ee8c7561a365f542102a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/387694
Tested-by: Daniel Kutik <daniel.kutik@lavawerk.com>
Commit-Queue: Daniel Kutik <daniel.kutik@lavawerk.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
DeprecationWarning: datetime.datetime.utcnow() is deprecated and
scheduled for removal in a future version. Use timezone-aware objects to
represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
Change-Id: Ia2c46fb87c544d98cc2dd68a829f67d4770b479c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/386615
Tested-by: Łukasz Patron <priv.luk@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Łukasz Patron <priv.luk@gmail.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
We don't accept PRs via GH, so add a job to automatically close them
with an explanation for how to submit.
Change-Id: I5cc3176549a04ff23b04dae1110cd27a58ba1fd3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/386134
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
tests/test_subcmds_sync.py::LocalSyncState::test_prune_removed_projects
was failing in Python 3.6 and 3.7 due to topdir not being set with the
following error message:
TypeError: expected str, bytes or os.PathLike object, not MagicMock
topdir is accessed from within PruneRemovedProjects().
Test: tox with Python 3.6 to 3.11
Change-Id: I7ba5144df0a0126c01776384e2178136c3510091
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/382816
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Daniel Kutik <daniel.kutik@lavawerk.com>
Tested-by: Daniel Kutik <daniel.kutik@lavawerk.com>
tests/test_git_superproject.py::SuperprojectTestCase::test_Fetch was
failing in Python 3.6 and 3.7 due to attribute args only being
introduced in Python 3.8. Falling back on old way of accessing
the arguments.
Test: tox with Python 3.6 to 3.11
Change-Id: Iae1934a7bce8cbd6b4519e4dbc92d94e21b43435
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/382818
Tested-by: Daniel Kutik <daniel.kutik@lavawerk.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Daniel Kutik <daniel.kutik@lavawerk.com>
When comparing 2 manifests, most of the time is
spent getting the relevant commit id as it relies
on _allrefs which ends up loading all git references.
However, the value from `revisionIs` (when it is valid)
could be used directly leading to a huge performance improvement
(from 180+ seconds to less than 0.01 sec which is more
than 25000 times faster for manifests with 700+ projects).
Bug: 295282548
Change-Id: I5881aa4b2326cc17bbb4ee91d23293111f76ad7e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/385834
Tested-by: Sylvain Desodt <sylvain.desodt@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Sylvain Desodt <sylvain.desodt@gmail.com>
The logic in projectsDiff performs various operations which
suggest that a set is more appropriate than a list:
- membership lookup ("in")
- removal
Also, sorting can be performed on the the remaining elements at the
end (which will usually involve a much smaller number of elements).
(The performance gain is invisible in comparison to the time being
spent performing git operations).
Cosmetic chance:
- the definition of 'fromProj' is moved to be used in more places
- the values in diff["added"] are added with a single call to extend
Change-Id: I5ed22ba73b50650ca2d3a49a1ae81f02be3b3055
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/383434
Tested-by: Sylvain Desodt <sylvain.desodt@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Sylvain Desodt <sylvain.desodt@gmail.com>
If a KeyboardInterrupt is encountered before an error is aggregated then
the context surrounding the interrupt is lost. This change aggregates
errors as soon as possible for the sync command
Bug: b/293344017
Change-Id: Iac14f9d59723cc9dedbb960f14fdc1fa5b348ea3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/384974
Tested-by: Jason Chang <jasonnc@google.com>
Commit-Queue: Jason Chang <jasonnc@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Black will only check .py files when given a dir and --check, so list
our few standalone programs explicitly. This causes the repo launcher
to be reformatted since it was missed in the previous mass reformat.
Bug: b/267675342
Change-Id: Ic90a7f5d84fc02e9fccb05945310fd067e2ed764
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/385034
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
