Python 3.11 introduces PYTHONSAFEPATH and the -P flag which, if enabled,
does not prepend the script's directory to sys.path by default.
This breaks repo because main.py expects its own directory to be part of
Python's import path.
This causes problems with tools that add PYTHONSAFEPATH to python
programs, most notably Bazel.
We will simply prepend main.py's path to PYTHONPATH instead.
Bug: 307767740
Change-Id: I94f3fda50213e450df0d1e2df6a0b8b597416973
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/391236
Tested-by: Duy Truong <duytruong@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Change-Id: I41371feb88c85e9da0656b9fab04057c22d1dcf4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/391514
Tested-by: Jason R. Coombs <jaraco@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason R. Coombs <jaraco@google.com>
- Bump minimum version to Python 3.6.
- Use f-strings in a lot of places.
Change-Id: I2aa70197230fcec2eff8e7c8eb754f20c08075bb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389034
Tested-by: Jason R. Coombs <jaraco@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason R. Coombs <jaraco@google.com>
In Python 3, these exceptions were merged into OSError, so switch
everything over to that.
Change-Id: If876a28b692de5aa5c62a3bdc8c000793ce52c63
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390376
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
DeprecationWarning: datetime.datetime.utcnow() is deprecated and
scheduled for removal in a future version. Use timezone-aware objects to
represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
Change-Id: Ia2c46fb87c544d98cc2dd68a829f67d4770b479c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/386615
Tested-by: Łukasz Patron <priv.luk@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Łukasz Patron <priv.luk@gmail.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Black will only check .py files when given a dir and --check, so list
our few standalone programs explicitly. This causes the repo launcher
to be reformatted since it was missed in the previous mass reformat.
Bug: b/267675342
Change-Id: Ic90a7f5d84fc02e9fccb05945310fd067e2ed764
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/385034
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Apply rules set by https://gerrit-review.googlesource.com/c/git-repo/+/362954/ across the codebase and fix any lingering errors caught
by flake8. Also check black formatting in run_tests (and CQ).
Bug: b/267675342
Change-Id: I972d77649dac351150dcfeb1cd1ad0ea2efc1956
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/363474
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Resolve TODO as we are now requiring Python 3.
Change-Id: I7821627bd5c606276741c98efedaf5b11aecbcc3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/354702
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Daniel Kutik <daniel.kutik@lavawerk.com>
People rarely care about the history of the manifest repo. Add a
parameter to specify depth for the manifest.
For now, make the default behavior the same as the current behavior. At
a future date, the default will be changed to 1. People who need the
full history should begin passing --manifest-depth=0 to preserve the
behavior when the default changes.
We can't reuse the existing --depth option because that applies to
all projects we clone, not just the manifest repo.
Bug: https://crbug.com/gerrit/16193, https://crbug.com/gerrit/16358
Change-Id: I9130fed3eaed656435c778a85cfe9d04e3a4a6a0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/349814
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
This reverts commit 076d54652e.
Reason for revert: crbug.com/gerrit/16358
Change-Id: I2970eb50677cca69786f71edffe4aa5271cf139f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/349834
Reviewed-by: Sam Saccone <samccone@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
If the specified repo dir doesn't actually have a `repo` program,
we only show a warning before continuing on, and then we fail in
weird ways. Since we really need the repo dir to contain repo,
have this be fatal and delete the results.
Bug: https://crbug.com/gerrit/13526
Change-Id: Icee4cba96136d470cbb459a81918c40205078f98
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343538
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
In case something goes wrong in the initial setup of the repo dir,
clone it into a temporary .repo/repo.tmp/ directory first, and then
rename it only when things have finished fully.
Bug: https://crbug.com/gerrit/13526
Change-Id: Ib0f5a975e4d436b0fb616fac70f5789c4e02a61a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343537
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
People rarely care about the history of the manifest repo. Change
the default to 1 to speed up initial setup. If people really want
the full history, they can pass --manifest-depth=0.
We can't reuse the existing --depth option because that applies to
all projects we clone, not just the manifest repo.
Bug: https://crbug.com/gerrit/16193
Change-Id: Ideab1712e9ffc743b9bae97903e074d7d429b134
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343435
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
People rarely care about having all manifest branches locally. Change
the default to only pull down the selected branch. If people want other
branches, the -b option will fetch it automatically, or people can use
--no-current-branch.
This only applies to the manifest project syncing, not the rest of the
projects that are in the checkout.
Bug: https://crbug.com/gerrit/16194
Change-Id: Ia9e7e2f23b8028d82772db72dcc7d6c32260be79
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343434
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
It was reported that git-lfs did not work with git-repo. Specifically,
`git read-tree -u` run by `repo sync` would fail git-lfs's smudge
filter. See https://github.com/github/git-lfs/issues/1422.
In fact, by the time `git read-tree -u` is run, the repository is not
bare. It is just that, the working directory is not the same as the
.git directory. git-lfs's filter should work. No one seems to have
delved into that issue.
Today, with newer versions of git-repo and git-lfs, that issue will
not reproduce. Tested with
- git 2.33, git-lfs 2.13 on macOS
- git 2.17, git-lfs 2.3 on ubuntu
So, it seems fine to add an option --enable-git-lfs-filter, default to
false, and stat that it may not work with older versions of git and
git-lfs in the help doc.
Bug: https://crbug.com/gerrit/14516
Change-Id: I8d21854eeeea541e072f63d6b10ad1253b1a9826
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328359
Tested-by: XD Trol <milestonejxd@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
If the user passes a bad --repo-rev setting in a new checkout, add a
tip to the error message that their option is probably bad instead of
just saying "unable to resolve".
If the user has already initialized a checkout, we'd display a raw
traceback which would confuse them. Swallow that and also include
the --repo-rev tip.
Bug: https://crbug.com/gerrit/15610
Change-Id: I5d72513c7b37bf9bb5d19862fcdfaf0d1f44e886
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328820
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Added --standalone_manifest to repo tool. If set, the
manifest is downloaded directly from the appropriate source
(currently, we only support GS) and used instead of creating
a manifest git checkout. The manifests.git repo is still created to
keep track of various config but is marked as being for a standalone
manifest so that the repo tool doesn't try to run networked git
commands in it.
BUG=b:192664812
TEST=existing tests (no coverage), manual runs
Change-Id: I84378cbc7f8e515eabeccdde9665efc8cd2a9d21
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312942
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Gitlab returns a 400 error when trying to fetch clone.bundle
from a repository containing the git-repo tool. The repo
launcher doesn't then fall back to not using a clone.bundle
file and the repo init fails.
Change-Id: Ia3390d0638ef9a39fb2fab84625b269d28caf1cc
Signed-off-by: Craig Northway <cnorthway@codeaurora.org>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305382
Reviewed-by: Mike Frysinger <vapier@google.com>
While this provides a way to undo earlier command line options (e.g.
`repo sync --tags --no-tags`) which can be helpful for scripting &
automation, this more importantly allows the user to override the
manifest settings for syncing tags from a project.
Bug: https://crbug.com/gerrit/12401
Change-Id: Id4c36cd82e6ca7cb073b5d63a09f6c7ccdebba83
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/304904
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
For most commands, this is more about providing a way to undo earlier
command line options (e.g. `repo info -c --no-current-branch`) which
can be helpful for scripting & automation. But for the sync command,
this is helpful to undo the setting that exists in the manifest itself.
With this in place, tweak the sync current_branch_only logic to only
apply the manifest settings when the user hasn't specified a command
line option.
Bug: https://crbug.com/gerrit/12401
Change-Id: I21e2384624680cc740d1b5d1e49c50589d2fe6a0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/304903
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Make it a bit easier for people to locate bug reporting info.
Change-Id: If9c8939c84ebd52eb96b353c1797afa25868bb85
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303943
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
The merge of the repo & init parser missed this default.
When running `repo init ...` in an existing checkout but w/out the -m
option, then repo would error out complaining that -m is required when
it didn't do this before.
Change-Id: I58035d48cc413b5d373702b9dc3b9ecd3fd1e900
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303945
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
Add new CommonOptions entry points to move the existing --jobs to,
and relocate all --verbose/--quiet options to that. This provides
both a consistent interface for users as well as for code.
Change-Id: Ifaf83b88872421f4749b073c472b4a67ca6c0437
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303224
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
partial-clone-exclude option excludes projects during
partial clone. This is a comma-delimited project names
(from manifest.xml). This option is persisted and it
is used by the sync command.
A project that has been unparital'ed will remain unpartial if
that project's name is specified in the --partial-clone-exclude
option. The project name should match exactly.
Added
$ ./run_tests -v
Bug: [google internal] b/175712967
"I can't "unpartial" my androidx-main checkout"
$ rm -rf androidx-main/
$ mkdir androidx-main/
$ cd androidx-main/
$ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M -m default.xml
$ repo_dev sync -c -j8
+ Verify a project is partial
$ cd frameworks/support/
$ git config -l | grep 'partial'
+ Unpartial a project.
$ /google/bin/releases/android/git_repack/git_unpartial
+ Verify project is unpartial
$ git config -l | grep 'partial'
$ cd ../..
+ Exclude the project from being unparial'ed after init and sync.
$ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --partial-clone-exclude="platform/frameworks/support,platform/frameworks/support-golden" -m default.xml
+ Verify project is unpartial
$ cd frameworks/support/
$ git config -l | grep 'partial'
$ cd ../..
$ repo_dev sync -c -j8
$ cd frameworks/support/
$ git config -l | grep 'partial'
$ cd ../..
+ Remove the project from exclude list and verify that project is partially cloned.
$ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --partial-clone-exclude= -m default.xml
$ repo_dev sync -c -j8
$ cd frameworks/support/
$ git config -l | grep 'partial'
Change-Id: Id5dba418eba1d3f54b54e826000406534c0ec196
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303162
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
We've grown a lot of options in here and it's hard to make sense of
them. Add more groups to try and make it easier to pick things out.
Change-Id: I6b9dc0e83f96137f974baf82d3fb86992b857bd2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/302803
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
These are manually kept in sync which is a pain. Have the init
subcmd reuse the wrapper code directly.
Change-Id: Ica73211422c64377bacc9bb3b1d1a8d9d5f7f4ca
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/302762
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
There's a few rough edges here still, but no known corruption ones,
so open it up a bit for people to experiment with.
Bug: https://crbug.com/gerrit/11486
Change-Id: I81e0122ab6d3e032c546c8239dd4f03740676e80
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299242
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
When people switch to non-default branches, they sometimes want to
switch back to the default, but don't know the exact name for that
branch. Add a -b HEAD shortcut for that.
Change-Id: I090230da25f9f5a169608115d483f660f555624f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297843
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Since the --manifest-url flag is always required when creating a new
checkout, allow the url to be specified via a positional argument.
This brings it a little closer to the `git clone` UI.
Change-Id: Iaf18e794ae2fa38b20579243d067205cae5fae2f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297322
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
Added --no-use-superproject to repo and init.py to disable use of
manifest superprojects.
Replaced the term "sha" with "commit id".
Added _GetBranch method to Superproject object.
Moved shared code between init and sync into SyncSuperproject function.
This function either does git clone or git fetch. If git fetch fails
it does git clone.
Changed Superproject constructor to accept manifest, repodir and branch
to avoid passing them to multiple functions as argument.
Changed functions that were raising exceptions to return either True
or False.
Saved the --use-superproject option in config as repo.superproject.
Updated internal-fs-layout.md document.
Updated the tests to work with the new API changes in Superproject.
Performance for the first time sync has improved from 20 minutes to
around 15 minutes.
Tested the code with the following commands.
$ ./run_tests -v
Tested the sync code by using repo_dev alias and pointing to this CL.
$ repo init took around 20 seconds longer because of cloning of superproject.
$ time repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b master --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject
...
real 0m35.919s
user 0m21.947s
sys 0m8.977s
First run
$ time repo sync --use-superproject
...
real 16m41.982s
user 100m6.916s
sys 19m18.753s
No difference in repo sync time after the first run.
Bug: [google internal] b/179090734
Bug: https://crbug.com/gerrit/13709
Bug: https://crbug.com/gerrit/13707
Change-Id: I12df92112f46e001dfbc6f12cd633c3a15cf924b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/296382
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
Currently we don't have a way for the checked out repo version to
declare the version of tools it needs before we start running it.
For somethings, like git, it's not a big deal as it can handle all
the asserts itself. But for things like Python, it's impossible
to reliably check before executing.
We're in this state now:
- we've been allowing Python 3.4, so the launcher accepts it
- the repo codebase starts using Python 3.6 features
- launcher tries to import us but hits syntax errors
- user is left confused and assuming new repo is broken because
they're seeing syntax errors
This scenario is playing out with old launchers that still accept
Python 2, and will continue to play out as time goes on and we want
to require newer versions of Python 3.
Lets create a JSON file to declare all these system requirements.
That file format is extremely stable, so loading & parsing from
even ancient versions of Python shouldn't be a problem. Then the
launcher can read these settings and check the system state before
attempting to execute any code. If the tools are too old, it can
clearly diagnose & display information to the user as to the real
problem (and not emit tracebacks or syntax errors).
We have a couple of different tool version checks already (git,
python, ssh) and can harmonize them in a single place.
This also allows us to assert a reverse dependency if the need
ever comes up: force the user to upgrade their `repo` launcher
before we'll let them run us. Even though the launcher warns
whenever a newer release is available, some users seem to ignore
that, or they don't use repo that often (on the scale of years),
and their upgrade jump is so dramatic that they fall back into
the syntax error pit.
Hopefully by the end of the year we can assume enough people
have upgraded their launcher such that we can delete all of the
duplicate version checks in the codebase. But until then, we'll
keep them to maintain coverage.
Change-Id: I5c12bbffdfd0a8ce978f39aa7f4674026fe9f4f8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/293003
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
We don't support Python 2 anymore, so stop allowing it to fallback.
If we try to run the latest version with Python 2, it just hits
syntax errors which confuses people. Dump a clear error message
that their system is too old and give up.
Bug: https://crbug.com/gerrit/13795
Change-Id: I38c243cf09502f670cddad72c2d0148f736515e0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292443
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
The codebase still supports Python 3.5, so allow use of that instead
of requiring Python 3.6+. Supporting this mode well is a bit tricky
as we want to first scan for newer versions before falling back to
older ones. And we have to avoid infinite loops in the process.
Bug: https://crbug.com/gerrit/13795
Change-Id: I47949a173899bfa9ab20d3fefa1a97bf002659f6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292442
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
We don't use javadoc in this project, so clean up the few places
that slipped in with the gitc code.
Change-Id: Ia365fb2d1e3188ad16b2f65b1a3b7e8466001946
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/291262
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
In the .repo discovery loop
while curdir != '/' and curdir != olddir:
... break if we found .repo ...
olddir = curdir
curdir = os.path.dirname(curdir)
the "while" condition is meant to avoid searching forever if we do not
find .repo before reaching the top-level directory of the filesystem.
For that purpose, the first half of the condition is redundant; once
we reach "/", the parent directory will be "/" again and the curdir !=
olddir check would suffice to terminate the search. Simplify by
removing the redundant first half of the check.
Noticed by code inspection. The first half of the check was retained
when introducing the second half in df14a70c ("Make path references OS
independent", 2011-01-09), in an excess of caution.
This also improves consistency a little: if I start with curdir =
'/home/me', then with the redundant check in place we search
/home/me
/home
before hitting / and giving up. On Windows, if I start with
'c:/users/me', then we search
c:/users/me
c:/users
c:/
before hitting a repetition and giving up. Fortunately it is not
common for people to set up repo clients at the top level of
filesystems, but consistently following the latter behavior should
make debugging a little easier in case it comes up.
Link: https://gerrit-review.googlesource.com/id/Ib9e830e3b9adfb1c4e56f3bcfba4746c401fb84f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/286002
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jonathan Nieder <jrn@google.com>
We've already released 2.7, and the next tag is 2.8, so this should
be pulled up to 2.8 so it'll stay in sync.
Change-Id: Id47bdbdb8050b29ea36442ac2149dd948648237f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/268572
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@digital.ai>
