For large projects, clone bundle is useful because it provided a way to
efficiently transfer a large portion of git objects through CDN, without
needing to interact with git server. However, with partial clones, the
intention is to not download most of the objects, so the use of clone
bundles would defeat the space savings normally seen with partial
clones, as they are downloaded before the first fetch.
A new option, --clone-bundle is added to override this behavior.
Add a new repo.clonebundle variable which remembers the choice if
explicitly given from command line at repo init.
Change-Id: I03638474af303a82af34579e16cd4700690b5f43
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/268452
Tested-by: Xin Li <delphij@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
exit if no repo_main can be found right before executing the command.
This happens for instance when 'repo init' is run on root path
(for example in a container). Without this counter measure the tool
will crash at exec_command with
TypeError: sequence item 1: expected str instance, NoneType found
Change-Id: Ia8480cfe2151c3b35c9572789ad8cb619288cce1
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/263457
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@digital.ai>
When developing repo itself, it helps to run repo directly out of it
and to run bisection tools. The current _SetDefaultsTo logic fails
in that situation though as it wants a branch, but the source isn't
checked out to one. Now that we support tracking commits via the
--repo-rev setting, fall back to using the current HEAD commit.
Change-Id: I37d79fd9f7bea87d212421ebed6c8267ec95145f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/260192
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
This gets the unittests passing again for now.
Change-Id: Ibed430a305bc26b907ad0ea424c7eec7de37e942
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/259994
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
We respect this option when running the first `repo init`, but then
silently ignore it once the initial sync is done. Make sure users
are able to change things on the fly.
We refactor the wrapper API to allow reuse between the two init's.
Bug: https://crbug.com/gerrit/11045
Change-Id: Icb89a8cddca32f39a760a6283152457810b2392d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/260032
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
While the help/usage suggested that revisions would work, they never
actually did, and just throw confusing errors. Now that we warn if
the checkout isn't tracking a branch, allow people to specify commits
or tags explicitly. Hopefully our nags will be sufficient to keep
most people on the right path.
Bug: https://crbug.com/gerrit/11045
Change-Id: I6ea32c677912185f55ab20faaa23c6c0a4c483b3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/259492
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
We refer to this as "revision" in help text, and in REPO_REV envvar,
so rename to --repo-rev to be consistent. We keep --repo-branch for
backwards compatibility, but as a hidden option.
Bug: https://crbug.com/gerrit/11045
Change-Id: I1ecc282fba32917ed78a63850360c08469db849a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/259352
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
When the launcher handles the init subcommand, it takes care of
setting the repo url & branch itself when cloning. So we don't
need to pass them down to the checked out init subcommand.
Further, the init subcommand has never actually done anything
with those options, so there's no point in passing them.
We'll be changing the latter behavior so that init will reset
the url/branch when specified with an existing repo checkout
which means passing them through adds overhead: the launcher
will checkout to the right value, then chain to the sub-init
which will then reset the checkout to the same value.
Bug: https://crbug.com/gerrit/11045
Change-Id: Ia2a4ab9d86febc470aea4abd73d75bb10e848b56
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/259312
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
When we download git sources, we get a progress bar (good) and we get
a dump of all the refs we downloaded (bad) as it can easily be 100+ if
not 1000+ depending on the project (for each git repo!). Lets rework
the output behavior so that:
* quiet: Only errors.
* default: Progress bars (if on a tty).
* verbose: Full output (progress bars & downloaded refs).
Bug: https://crbug.com/gerrit/11293
Change-Id: I87a380075e79de6805f91095876dd1b37d32873a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256456
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Michael Mortensen <mmortensen@google.com>
We often ask users what OS/version they're running when debugging.
Include that in the version output to simplify the process.
Change-Id: Ie480b6d1c874e6f4c6f4996a03795077b844f858
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256732
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
We produce some simple "Get" messages that aren't super clear as to
what they're doing, especially for people not familiar with repo.
Rephrase these to explicitly state the thing we're doing so it's
clear why we're downloading a particular source.
Bug: https://crbug.com/gerrit/11293
Change-Id: I0749504f17c5385c6c65274a274e0ae25b117413
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256455
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
We don't use these for much yet, but init passes it down to the
project sync layers which already has support for verbose mode.
Change-Id: I651794f1b300be1688eeccf3941ba92c776812b5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256454
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
This copies the output style we use in git_command for showing output
and input redirections.
Change-Id: I449b27e7b262e1b30b24333109a1d91d9c7b1ce7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256453
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Switch the copyright holder to "The Android Open Source Project" to
match all the other source files in the tree, and move it to the top
of the file to match everything else we do.
Change-Id: Ie15d8e2bc004a626e45f715271deeaf3919dc44a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256235
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
This supports [alias] sections with repo subcommands just like git.
Change-Id: Ie9235b5d4449414e6a745814f0110bd6af74ea93
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255833
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
Hopefully enough issues should be resolved now that we can start
forcing Windows users into Python 3 too.
Change-Id: Ic4aad6a0b35ffec7d1372e3da6fca11a2b6fde0b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255353
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
This provides initial support for using git worktrees internally
instead of our own ad-hoc symlink tree. It's been lightly tested
which is why it's not currently exposed via --help.
When people opt-in to worktrees in an existing repo client checkout,
no projects are migrated. Instead, only new projects will use the
worktree method. This allows for limited testing/opting in without
having to completely blow things away or get a second checkout.
Bug: https://crbug.com/gerrit/11486
Change-Id: Ic3ff891b30940a6ba497b406b2a387e0a8517ed8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254075
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Trying to use booleans with names like "no_xxx" are hard to follow due
to the double negatives. Invert all of them so we only have positive
meanings to follow.
Change-Id: Ifd37d0368f97034d94aa2cf38db52c723ac0c6ed
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255493
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
If programs emit non-UTF-8 output, we currently throw a fatal error.
We largely only care about the exit status of programs, and even the
output we do parse is a bit minimal. Lets make it into a warning and
mangle the invalid bytes into U+FFFD. This should complain enough to
annoy but not to break when it's not necessary.
Bug: https://crbug.com/gerrit/12337#c2
Change-Id: Idbc94f19ff4d84d2e47e01960dd17d5b492d4a8a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255272
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
Some versions of gpg on Windows mishandle native paths with homedir.
It manifests itself like:
gpg: keybox 'C:\Users\.../.repoconfig\gnupg/pubring.kbx' created
gpg: C:\Users\.../.repoconfig\gnupg/trustdb.gpg: trustdb created
gpg: key 16530D5E920F5C65: public key "Repo Maintainer <repo@android.kernel.org>" imported
gpg: can't connect to the agent: Invalid value passed to IPC
gpg: Total number processed: 1
gpg: imported: 1
fatal: registering repo maintainer keys failed
It seems gpg (at least version 2.2.17) needs paths to be specified
in cygwin form (e.g. "/c/Users/.../.repoconfig/gnupg") otherwise
it fails to talk to its own processes. We can work around this
with a minor trick: we cd to the right path and then invoke gpg
with --homedir . and let gpg itself resolve . to whatever form it
really wants.
This is a bit hacky, but we don't control gpg, and this allows us
to avoid having to muck with the environment. Since --homedir has
been around since at least gpg-1.4.x from 2004, backwards compat
shouldn't be an issue.
While we're here, touch up the output a bit: there's no need to
dump all the chatty gpg output if things don't fail, so always
swallow the output. If things do fail, our exception handler
takes care of dumping the full stdout & stderr.
Change-Id: I74ab98e1e61e95318fda6faf57c6a8699f775935
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255120
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
We added support for `repo init -c` to main.py, but not to the
launcher, so the -c option only works after the first init has
run which kind of defeats its purpose. Rework the parser setup
so that we can tell it whether it's for "init" or "gitc-init"
and then add the -c option in the same way we do in main.py.
This has the benefit of getting the parser entirely out of the
module scope which makes it a lot easier to reason about, and
it means we can write some unittests.
Change-Id: Icbc2ec3aceb938d5a8f941d5fbce1548553dc5f7
Test: repo help init
Test: repo help gitc-init
Test: repo init -u https://android.googlesource.com/platform/manifest -c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255113
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
We've been using a DSA/1024 key to sign our tags. Time to update to
something a bit newer. We'll include RSA & ECC keys, but only use
RSA keys initially for backwards compatibility and see how it goes
with our user base.
Change-Id: I683c97b6fbd860f220ed4ddc7b21f07db279a916
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255112
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
repo:342:5: E306 expected 1 blank line before a nested definition, found 0
repo:617:5: F841 local variable 'ret' is assigned to but never used
Change-Id: I364fdb5dac8ebaff398b848935fe8356cb9ed2d3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255035
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
Now that we have a central run_command point, we can easily add
tracing support to the launcher script.
Change-Id: I9e0335c196cafd6263ff501925abfe835f036c5e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254755
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
We can't rely on subprocess.run yet as that requires Python 3.6,
but we can clean up the code we have with some ad-hoc replacement.
This unifies all the inconsistent subprocess.Popen usage we have.
Change-Id: I56af40a3df988ee47b299105d692ff419d07ad6b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254754
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
We can get version info when in a checkout, but it'd be helpful
to show that info at all times.
Change-Id: Ieeb44a503c9d7d8c487db4810bdcf3d5f6656c82
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254712
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
This helps with people tracing repo/git execution. We use a similar
format to git, but a little simpler since we always initialize the
env var setting, and we want to avoid too much overhead.
Bug: https://crbug.com/gerrit/12314
Change-Id: I75675b6cc4c6f7c4f5e09f54128eba9456364d04
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254331
Reviewed-by: Josh Steadmon <steadmon@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
We import the wrapper on the fly, so minimize how much code we run
in module scope. It's pointless/wasted when importing.
Change-Id: I4a71c2030325d0a639585671cd7ebe8f22687ecd
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254072
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
We were perhaps a bit too hasty to jump to git-2.10. Existing LTS
releases of Ubuntu are quite old still: Trusty has 1.9 while Xenial
has 2.5. While we plan on dropping support for those eventually as
we migrate to Python 3.6, we don't need to be so strict just yet on
the git versions.
We also want to disconnect the version the repo launcher requires
from the version the rest of the source tree requires. The repo
launcher doesn't need as many features, and being flexible there
allows us more freedom to upgrade & rollback as needed.
So we'll allow git-1.7 again, but start warning on any users older
than git-1.9. This aligns better with existing LTS releases, and
gives users a chance to start upgrading before we cut them off.
Change-Id: I140305dd8e42c9719c84e2aee0dc6a5c5b18da25
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254573
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
We want to start warning about Python 2 usage, but we can't do it
simply because the shebang is /usr/bin/python which might be an old
version like python2.7.
We can't change the shebang because program name usage is spotty at
best: on some platforms (like macOS), it's not uncommon to not have
a `python3` wrapper, only a major.minor one like `python3.6`. Using
python3 wouldn't guarantee a new enough version of Python 3 anyways,
and we don't want to require Python 3.6 exactly, just that minimum.
So we check the current Python version. If it's older than the ver
of Python 3 we want, we search for a `python3.X` version to run. If
those don't work, we see if `python3` exists and is a new enough ver.
If it's not, we die if the current Python 3 is too old, and we start
issuing warnings if the current Python version is 2.7. This should
allow the user to take a bit more action by installing Python 3 on
their system without having to worry about changing /usr/bin/python.
Once we require Python 3 completely, we can simplify this logic a bit
by always bootstrapping up to Python 3 and failing with Python 2.
We have a few KI with Windows atm though, so keep it disabled there
until the fixes are merged.
Bug: https://crbug.com/gerrit/10418
Change-Id: I5e157defc788e31efb3e21e93f53fabdc7d75a3c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253136
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
The git-2.10 series was released in 2016. Since we're moving to
require Python 3.6 which was also released in 2016, bumping up the
git version seems reasonable. Also we don't really test any git
versions close to as old as 1.7.2 which was released in 2010.
Change-Id: Ib71b714de6cd0b7dd50d0b300b108a560ee27331
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253134
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
This hasn't been used in many years to sign a release, so drop it
from the keyring to avoid confusing people.
Bug: https://crbug.com/gerrit/12229
Change-Id: Ifca7eee713d167c11f32252975724e5858e4c007
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253133
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
This reflects the transition to the new 2.x series which will be
migrating to Python 3-only.
Bug: https://crbug.com/gerrit/10418
Change-Id: I6355ac955d26b930f8a3721d3526eec5bed92400
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253132
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
In order to be able to use "--no-repo-verify" to work around an issue with
gpg-agent and long socket paths (see e.g. [1]), this change avoids GPG
being set up at all if that option is passed.
[1] https://github.com/elastic/elasticsearch/issues/17053
Change-Id: I1e5cbd8be2dc0084f12afe0ca33c789fdbc6fef9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/251108
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Use open() as a context manager to simplify the close logic and make
the code easier to read & understand. This is also more Pythonic.
Change-Id: I579d03cca86f99b2c6c6a1f557f6e5704e2515a7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244734
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
We've rolled quite a number of fixes since the last update, including
a lot of Python 3 improvements. Lets bump the wrapper version for it.
Change-Id: I6c6c04c3c8241bf8e8bcf26603549ae4595fede8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/237812
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
We use subprocess a lot in the wrapper, but we don't always read
or write the streams directly. When we do, make sure we convert
to/from bytes before trying to use the content.
Change-Id: I318bcc8e7427998348e359f60c3b49e151ffbdae
Reported-by: Michael Scott <mike@foundries.io>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/236612
Reviewed-by: Michael Scott <mike@foundries.io>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Michael Scott <mike@foundries.io>
A new option, --partial-clone is added to 'repo init' which tells repo
to utilize git's partial clone functionality, which reduces disk and
bandwidth usage when downloading by omitting blob downloads initially.
Different from restricting clone-depth, the user will have full access
to change history, etc., as the objects are downloaded on demand.
Change-Id: I60326744875eac16521a007bd7d5481112a98749
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/229532
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Xin Li <delphij@google.com>
This should help people get some bearings in the codebase.
Change-Id: I951238fe617a3ecb04a47ead3809ec72c8fbf5a1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/231232
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
