We have a few files that we optionally symlink from the work tree
.git/ to the .repo/projects/ path. If they don't exist when we
first initialize, then we skip creating symlinks. If the files
are created later on under the work tree .git/, repo gets upset.
This can happen with the packed-refs file: if we don't have any
packed refs initially, we don't symlink it. But if git tries to
pack refs later on and creates the file, the project gets wedged.
We could create an empty file initially and then symlink it, but
for some files, it's not clear we want to always do that (e.g.
the .git/shallow setting). Instead, lets make handling of these
paths more dynamic. If they show up later on in the work tree
.git/ only, we'll take care of relocating & symlinking. This
also makes repo a little more robust and autorecovers incase a
path goes missing in one of the dirs.
Ideally we wouldn't monkey around at all here, but considering
the only option we give to users currently is to blow things
away with --force-sync, this seems a bit better.
Bug: https://crbug.com/gerrit/12324
Change-Id: Ia6960f1896ac6d890c762d7d053684a1c6ab2c87
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254632
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
flake8 reports:
E265 block comment should start with '# '
While we're at it, add a period at the end of the comment sentence.
Change-Id: Icb7119079a1d64e6defafc3f6d24e99dbf16139d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254596
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
When upload hooks fail, people are forced to use --no-verify to upload
CLs anyways. When projects have flaky hooks, this trains people to
always use that option. This is obviously bad: hooks might get fixed,
or some of the hooks are always good & people should review.
Lets add an --ignore-hooks option. This still runs the hooks, but any
failures will be ignored and allow the user to upload anyways.
Bug: https://crbug.com/gerrit/12230
Change-Id: Ide2ac8a40a656bfcd6aae20c3ce8118e06bf909b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254452
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
flake8 reports:
W391 blank line at end of file
Change-Id: I5498b2de2d1268d4f1f4b9e1760f9fa93a6da4cd
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254594
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
flake8 reports:
F841 local variable 'f' is assigned to but never used
Change-Id: If808eb381ee44c7da71e6281615a06a6723cf945
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254593
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
flake8 reports:
F811 redefinition of unused 'test_src_block_dir' from line 259
which is caused by having two methods with the same name. Rename
them both to better desribe their purpose.
Change-Id: If7612a42001776d71bb1a6a80fc631d3d262e6ce
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254449
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
This reverts commit 4abf8e6ef8.
The curl process for updating the cookie file is not atomic. When
fetching many bundles in parallel, we can sometimes corrupt the file
causing it to be cleared. Since users should manage gitcookies on
their own, leave it read-only.
Bug: https://crbug.com/gerrit/12300
Change-Id: Id472c99b197bc4cf8533c649f8881509f38643c1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254092
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
We were perhaps a bit too hasty to jump to git-2.10. Existing LTS
releases of Ubuntu are quite old still: Trusty has 1.9 while Xenial
has 2.5. While we plan on dropping support for those eventually as
we migrate to Python 3.6, we don't need to be so strict just yet on
the git versions.
We also want to disconnect the version the repo launcher requires
from the version the rest of the source tree requires. The repo
launcher doesn't need as many features, and being flexible there
allows us more freedom to upgrade & rollback as needed.
So we'll allow git-1.7 again, but start warning on any users older
than git-1.9. This aligns better with existing LTS releases, and
gives users a chance to start upgrading before we cut them off.
Change-Id: I140305dd8e42c9719c84e2aee0dc6a5c5b18da25
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254573
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
Going purely on upstream package release cycles doesn't tell the whole
story: a lot of people run LTS distros which will have older versions
of software we want to support.
Build out a table for us to quickly reference when making decisions as
to what versions of git/python we want to support, and when we can drop
them. This will also help to refer users to as why we made a specific
decision that might be affecting them.
Change-Id: I7aea24bbefd50e358aeacf11e8c15a346c8fb8a9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254572
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Some automatic git operations will prune objects on us, and not just
the gc step. Normally we don't care, but with shared projects, we
will have multiple git checkouts with refs that the others cannot
see, but with a shared object dir. Any pruning of objects based on
refs in just one repo can easily break the others.
git-2.7.0 introduced a preciousObjects setting which tells git to
never prune objects for this exact scenario: there might be refs in
some location that git is unable to see.
Change-Id: I781de27c5bbe1d4c70f0187566141c9cce088bd8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254392
Reviewed-by: Nasser Grainawi <nasser@codeaurora.org>
Reviewed-by: David Riley <davidriley@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Since tempfile.mkstemp() returns a file handle in binary mode,
make sure we turn our strings into bytes before writing.
Bug: https://crbug.com/gerrit/12043
Change-Id: I3e84d595e84b8bc12a1fbc7fd0bb3ea0ba2832b0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254393
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Some Android/Nest manifests are using <linkfile> with src="." to
create stable paths to specific projects. Allow that specific
use case as it seems reasonable to support.
Bug: https://crbug.com/gerrit/11218
Change-Id: I16dbe8d9fe42ea45440afcb61404c753bff1930d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254330
Reviewed-by: Chanho Park <parkch98@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
The code has an ad-hoc check in that it requires the launcher major
version to not be less than the source code version. We don't really
care about that requirement, and it doesn't fit with our other version
checks. Rework it so we explicitly declare the min launcher version
that is supported.
We'll start with requiring repo launcher 1.15 which was released back
in 2012. Hopefully no one has anything older than that, although it's
not clear we work with even newer versions than that :). But let's be
a little conservative with the first update to this logic.
Bug: https://crbug.com/gerrit/10418
Change-Id: I611d70c60324d313c76874e978b8499a491a5d00
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254278
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Some Android/Nest manifests are using <linkfile> with src="." to
create stable paths to specific projects. Allow that specific
use case as it seems reasonable to support.
Bug: https://crbug.com/gerrit/11218
Change-Id: I5eadec257cd58ba0f8687c590ddc250a7a414a85
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254276
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Execution of 'repo forall -p -c' doesn't work with Py3 and ends up
with an error:
Got an error, terminating the pool: TypeError: can only concatenate
str (not "bytes") to str
That's fixed by using the decode() method.
Change-Id: Ice01aaa1822dde8d957b5bf096021dd5a2b7dd51
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253659
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Jiri Tyr <jiri.tyr@gmail.com>
Add a helper function to unify the duplication of finding the full
path to the symbolic HEAD ref. This makes it easy to handle git
worktrees where .git is a file rather than a dir/symlink.
Bug: https://crbug.com/gerrit/11486
Change-Id: I9f794f1295ad0d98c7c13622f01ded51e4ba7846
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254074
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
These files are not directly executable, so drop the +x bits.
Change-Id: Iaf19a03a497686cc21103e7ddf08073173440dd1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254076
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
This allows these scripts to run through the active version of the
virtualenv python when invoked via tox.
Change-Id: Ib52f475b7b20c34d62cfd179a1341da1a08a8b5c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253974
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
Currently the only reference for these is the source which can be a
pita when needing to refer to something quickly.
Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Make sure we decode the bytes coming from the subprocess.Popen as
we're treating them as strings.
Change-Id: I44100ca5cd94f68a35d489936292eb641006edbe
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253973
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Instead of not using '-c' for '--current-branch' when using gitc, we
were only using '-c' when using gitc, so we still had the conflict with
the gitc option, and other users still couldn't use '-c'.
Test: repo init -u https://android.googlesource.com/platform/manifest; repo init -c
Test: repo gitc-init -u ... -b ... -c testing
Change-Id: I71e4950a49c281418249f0783c6a2ea34f0d3e2b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253795
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Dan Willemsen <dwillemsen@google.com>
On Python 3 several imports are to be imported from
different locations.
Signed-off-by: Remy Böhmer <linux@bohmer.net>
Change-Id: I4f243d145f65e38f74743a742583cfc5c5d76deb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/249610
Reviewed-by: Mike Frysinger <vapier@google.com>
Currently our default behavior is:
* Try to sync all repos
* If any errors seen, exit
* Try to garbage collect all repos
* If any errors seen, exit
* Try to update local project list
* If any errors seen, exit
* Try to checkout out all local repos
* If any errors seen, exit
Users find these incomplete syncs confusing, so lets try to complete
as much as possible by default and printing out summaries at the end.
Bug: https://crbug.com/gerrit/11293
Change-Id: Idd17cc9c3bbc574d8a0f08a30225dec7bfe414cb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/238554
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
We want to start warning about Python 2 usage, but we can't do it
simply because the shebang is /usr/bin/python which might be an old
version like python2.7.
We can't change the shebang because program name usage is spotty at
best: on some platforms (like macOS), it's not uncommon to not have
a `python3` wrapper, only a major.minor one like `python3.6`. Using
python3 wouldn't guarantee a new enough version of Python 3 anyways,
and we don't want to require Python 3.6 exactly, just that minimum.
So we check the current Python version. If it's older than the ver
of Python 3 we want, we search for a `python3.X` version to run. If
those don't work, we see if `python3` exists and is a new enough ver.
If it's not, we die if the current Python 3 is too old, and we start
issuing warnings if the current Python version is 2.7. This should
allow the user to take a bit more action by installing Python 3 on
their system without having to worry about changing /usr/bin/python.
Once we require Python 3 completely, we can simplify this logic a bit
by always bootstrapping up to Python 3 and failing with Python 2.
We have a few KI with Windows atm though, so keep it disabled there
until the fixes are merged.
Bug: https://crbug.com/gerrit/10418
Change-Id: I5e157defc788e31efb3e21e93f53fabdc7d75a3c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253136
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
The git-2.10 series was released in 2016. Since we're moving to
require Python 3.6 which was also released in 2016, bumping up the
git version seems reasonable. Also we don't really test any git
versions close to as old as 1.7.2 which was released in 2010.
Change-Id: Ib71b714de6cd0b7dd50d0b300b108a560ee27331
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253134
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
The use case is any situation where your manifest does
not exist on server, but where you still want to do
full sync for the projects, without having your
workspace manifest switched to other branch or
forwarded to latest or similar.
This allows syncing to a historical manifest in git log,
that does not have a branch, as well as when integrating
something together that has not been pushed upstream yet.
Changes can also exist locally on a manifest that is
behind head, meaning not requiring rebase to latest.
Tested using:
$ cd .repo/manifests/
$ git checkout <any hash 1>
$ <do local modifications>
$ repo sync --no-manifest-update
$ git checkout <any hash 2>
$ repo sync --no-manifest-update
Change-Id: I0c9773aa8bc5876813a2e7d7fec697abcb2d9e94
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/246445
Tested-by: Fredrik de Groot <fredrik.de.groot@volvocars.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
We keep getting requests for init to support -c. This conflicts with
gitc-init which allocates -c for its own use. Lets make this dynamic
so we keep it with "init" but omit it for "gitc-init".
Bug: https://crbug.com/gerrit/10200
Change-Id: Ibf69c2bbeff638e28e63cb08926fea0c622258db
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253252
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
* Mirrors may contain multiple projects, some of which may not
always contain the default revision.
* Only fetch the default revision explicitly if
'--current-branch' is set.
* Fixes breakage casued by
commit 6856f98467
"Fix repo mirror with --current-branch"
Bug: https://crbug.com/gerrit/12274
Change-Id: Iaafabe2992f76f3644b841f24245d3e19c9515a9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253093
Reviewed-by: Kuang-che Wu <kcwu@chromium.org>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Chirayu Desai <chirayudesai1@gmail.com>
This commit supports for the 'remote' attribute in
<extend-project>. This avoids the need to perform a <remove-project>
followed by a <project> in local manifests.
Change-Id: I9f9347913337ec9d159bc264d15ce97881ae5398
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253092
Tested-by: Kyunam Jo <kyunam.jo@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reject paths in <copyfile> & <linkfile> that try to use symlinks or
non-file or non-dirs.
We don't fully validate <linkfile> when src is a glob as it's a bit
complicated -- any component in the src could be the glob. We make
sure the destination is a directory, and that any paths in that dir
are created as symlinks. So while this can be used to read any path,
it can't be abused to write to any paths.
Bug: https://crbug.com/gerrit/11218
Change-Id: I68b6d789b5ca4e43f569e75e8b293b3e13d3224b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/233074
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reject paths in <copyfile> & <linkfile> that point outside of their
respective scopes. This validates paths while parsing the manifest
as this should be quick & cheap: we don't access the filesystem as
this code runs before we've synced.
Bug: https://crbug.com/gerrit/11218
Change-Id: I8e17bb91f3f5b905a9d76391b29fbab4cb77aa58
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/232932
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Michael Mortensen <mmortensen@google.com>
This hasn't been used in many years to sign a release, so drop it
from the keyring to avoid confusing people.
Bug: https://crbug.com/gerrit/12229
Change-Id: Ifca7eee713d167c11f32252975724e5858e4c007
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253133
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
This reflects the transition to the new 2.x series which will be
migrating to Python 3-only.
Bug: https://crbug.com/gerrit/10418
Change-Id: I6355ac955d26b930f8a3721d3526eec5bed92400
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253132
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
This reverts commit 1e01a74445.
Not all platforms support select.poll() currently it seems.
At least macOS's Python 2 doesn't (while macOS Python 3 does).
Lets back this out for the existing release series and once we
start repo-2 which is Python 3-only, we can put this back in.
Change-Id: I205206b0fa4fe2d755f4fbc6ec683ad125f27cc2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253072
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
In order to be able to use "--no-repo-verify" to work around an issue with
gpg-agent and long socket paths (see e.g. [1]), this change avoids GPG
being set up at all if that option is passed.
[1] https://github.com/elastic/elasticsearch/issues/17053
Change-Id: I1e5cbd8be2dc0084f12afe0ca33c789fdbc6fef9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/251108
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
This makes it consistent with the short option for current-branch in
repo sync.
Change-Id: I2848e87f45a66ef8d829576d0c0c4c0f7a8636a0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/241700
Tested-by: Diogo Ferreira <deovferreira@gmail.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
After Ib546f5ebbc8a23875fbd14bf166fbe95b7dd244e, repo info now displays
the current project revision in the 'Current revision' field.
While the output is more consistent, there are use cases for the
revision expression as shown in the manifest. This patch re-adds the
manifest revision as a new 'Manifest revision' field.
Change-Id: I50c1559dcb7ceb69af07352b956d78f85b8f592e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/240799
Tested-by: Diogo Ferreira <deovferreira@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
select() has a limit of FD_SETSIZE file descriptors. If you run repo
sync -j500 you'll pretty quickly hit this limit and get "file descriptor
out of range for select" errors. poll() has no such limit.
Change-Id: I21f350e472bda1db03dcbcc437645c23dbc7a901
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/248852
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Theodore Dubois <tbodt@google.com>
No functional changes, just unifying duplicate code paths.
Change-Id: I6afa797ca1e1eb90abdc0236325003ae070cbfb3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/247293
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
This makes it easy to run all the tests against multiple versions
of Python. We want to make sure Python 2.7 & 3.6+ work.
Change-Id: Ia7b16eb46a2aa7c240f03bb291987fa8cb215267
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/247174
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
This is needed to use tox, and tox lets us test multiple Python
versions easily.
Change-Id: I813c418a8f7109294a4adb9f6b21be459cbeca70
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/247173
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
