- Bump minimum version to Python 3.6.
- Ran pyupgrade for Python 3.6, then again, then
- ran black
- fixed a lingering unused import, and
- removed a duplicate exception.
Change-Id: I2aa70197230fcec2eff8e7c8eb754f20c08075bb
Python 2 can't even parse this code anymore due to syntax changes,
so there's no point in checking for it explicitly.
Bug: 302871152
Change-Id: I9852ace5f5079d037c60fd3ac490d77e074e6875
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389434
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Not including it causes flaky behavior in some Chromium builders
because Chromium's custom Python build used by vpython relies on
argv[0] to find its own internal files.
Bug: https://crbug.com/1468522
Change-Id: I5c32ebe71c9b684d6ee50dbd8c3d6fcd51ca309b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/381974
Reviewed-by: Chenlin Fan <fancl@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Per discussion in go/repo-error-update updated aggregated and exit
errors for sync command.
Aggregated errors are errors that result in eventual command failure.
Exit errors are errors that result in immediate command failure.
Also updated main.py to log aggregated and exit errors to git sessions
log
Bug: b/293344017
Change-Id: I77a21f14da32fe2e68c16841feb22de72e86a251
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/379614
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Commit-Queue: Jason Chang <jasonnc@google.com>
Save the latest time any project is fetched and checked out. This will
be used to detect partial checkouts.
Bug: b/286126621
Change-Id: I53b264dc70ba168d506076dbd693ef79a696b61d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/380514
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Joanna Wang <jojwang@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Fixed a couple of bugs in ExitEvent logging:
- log exitcode 130 on KeyboardInterrupt
- log exitcode 1 on unhandled Exception
- log errorevent with specific reason for exit
Before this CL an exitcode of 0 would be logged, and it would be
difficult to determine the cause of non-zero exit codes
Bug: b/287105597
Change-Id: I2d34f180581f9fbd77a1c78c966ebed065223af6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/377834
Tested-by: Jason Chang <jasonnc@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Apply rules set by https://gerrit-review.googlesource.com/c/git-repo/+/362954/ across the codebase and fix any lingering errors caught
by flake8. Also check black formatting in run_tests (and CQ).
Bug: b/267675342
Change-Id: I972d77649dac351150dcfeb1cd1ad0ea2efc1956
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/363474
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Notes to the user need to go to stderr, and tracing should not be on for
fast exiting invocations (such as --help).
This makes it so that release/update-manpages works.
Change-Id: Ib183193c868a78c295a184c01c4532cd53d512eb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350794
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
With this change, partial syncs (sync with a project list) are again
supported.
If the updated manifest includes new sub manifests, download them
inheriting options from the parent manifestProject.
Change-Id: Id952f85df2e26d34e38b251973be26434443ff56
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334819
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
When a manifest file is overridden, remember that and keep using the
override for the remainder of the process. If we need to revert it,
make the override name evaluate False.
Change-Id: I1eee05fec6988c1ee4a3c751c4b540d5b5d11797
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/335136
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
To be addressed in another change:
- a partial `repo sync` (with a list of projects/paths to sync)
requires `--this-tree-only`.
Change-Id: I6c7400bf001540e9d7694fa70934f8f204cb5f57
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322657
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Simple API to make it easy to find the top of the repo client checkout
for users. This mirrors the `git rev-parse --show-toplevel` API.
Change-Id: I0c3f98def089d0fc9ebcfa50aa3dc02091c1c273
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312909
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Currently we have the behavior:
* `repo`: Equivalent to `repo help` -- only shows common subcommands
(with short description), and then exits 0.
* `repo --help`: Shows repo's core options, lists all commands (no
specific info), and then exits 0.
The first case is not behaving well:
* If you run `repo` without a specific subcommand, that's an error,
so we should be exiting 1 instead.
* Showing only subcommands and no actual option summary makes it seem
like repo itself doesn't take any options. This confuses users.
Let's rework things a bit. Now we have the behavior:
* `repo`: Shows repo's core options, lists all commands (no specific
info), and then exits 1.
* `repo --help`: Shows repo's core options, shows common subcommands
(with short description), and then exits 0.
* `repo --help-all`: Shows repo's core options, shows all subcommands
(with short description), and then exits 0.
Basically we swap the behavior of `repo` and `repo --help`, and fix
the exit status when the subcommand is missing.
The addition of --help-all is mostly for the man pages. We were
relying on `repo help --all` to generate the repo(1) man page, but
that too omitted the core repo options. Now the man page includes
all the core repo options and provides a summary of all commands.
Change-Id: I1f99b99d5b8af2591f96a078d0647a3d76d6b0fc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312908
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
We don't really care what the subcommand is set to when --version
output is requested, so stop enforcing it. This fixes some weird
behavior like `repo --version version` fails, but `repo --version
help` works.
The new logic skips subcommand validation, so `repo --version asdf`
will still display the version output. This matches git behavior,
and makes a bit of sense when we consider that the user really wants
to see the tool version, and probably doesn't care about anything
else on the command line.
Change-Id: I87454d473c2c8869344b3888a7affaa2e03f5b0f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312907
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
The _ParseArgs method parses the arguments and processes some of
the options, with the rest left to the _Run method. Simplify the
_ParseArgs method to only parse arguments and have _Run handle all
actual processing.
This will make it easier to add more terminal options (ones that
exit immediately without a subcommand), and makes it easier to
understand the overall code flow.
Change-Id: I47f7274c3f2b59378fd479e403e70fb24b681536
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312906
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Don't exit if there are missing commit ids in superproject.
This change implements the following suggestion from delphij@:
"we should note the event (so we know that --use-superproject but there
were some errors, e.g. manifest didn't specify commit id for some
reason, or if there is no superproject but --use-superproject is
used), print out a message telling the use that this is not support,
but continue as if --no-use-superproject was specified?"
Changes:
superproject:
+ Added git_trace2_event_log as an argument to the constructor.
+ Sync method returns SyncResult a NamedTuple of
++ success - True if sync of superproject is successful, or False.
++ fatal - True if caller should exit, Or False.
+ UpdateProjectsRevisionId returns UpdateProjectsResult a NamedTuple of
++ manifest_path - path name of the overriding manifest file instead
of None
++ fatal - True if caller should exit, Or False
+ _GetAllProjectsCommitIds returns CommitIdsResult a NamedTuple of
++ commit_ids - a dictionary with the projects/commit ids on success,
otherwise None
++ fatal - True if caller should exit, Or False
+ Added _SkipUpdatingProjectRevisionId a helper function to see if a
project's revision id needs to be updated or not. This function is
used to exclude projects from local manifest file.
+ Added the following error events into git_trace2_event_log
++ If superproject is missing in a manifest
++ If there are missing commit ids for projects.
command.py:
+ Deleted unused import - platform
+ Added git_trace2_event_log as a member so all subcmds can log error
events.
main.py:
+ Initialized git_trace2_event_log as a member of command object.
init.py:
+ Deleted unused import - optparse
init.py:
+ Called sys.exit only if Sync returns exit=True
sync.py:
+ Called sys.exit only if Superproject's UpdateProjectsRevisionId returns
exit=True
+ Reloaded the manifest only if manifest path is returned by
UpdateProjectsRevisionId. If not, fall back to the old way of doing
repo sync.
test_git_superproject:
+ Added code to verify error events are being logged.
+ Added a test for no superproject tag
+ Added test for UpdateProjectsRevisionId not updating the revision id
with the commit ids.
Tested the code with the following commands.
+ Positive test case with aosp-master.
$ repo_dev init -u persistent-https://android.git.corp.google.com/platform/manifest -b master --use-superproject
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
.../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed.
Your identity is: Raman Tenneti <rtenneti@google.com>
If you want to change this, please re-run 'repo init' with --config-name
repo has been initialized in .../android/aosp
$ repo_dev sync -j40 --use-superproject
remote: Total 12 (delta 4), reused 12 (delta 4)
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
.../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed.
...
repo sync has finished successfully.
+ Negative test case without superproject tag.
$ repo_dev sync -j40 --use-superproject
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
repo error: superproject tag is not defined in manifest: .../android/aosp/.repo/manifest.xml
error: Cannot get project commit ids from manifest
error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option
...
Checking out: 100% (1022/1022), done in 3.589s
repo sync has finished successfully.
+ Test for missing commit_id for a project.
$ repo_dev sync -j40 --use-superproject
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
.../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed.
error: please file a bug using go/repo-bug to report missing commit_ids for: ['build/blueprint']
error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option
...
Checking out: 100% (1022/1022), done in 3.364s
repo sync has finished successfully.
$ ./run_tests -v
...
...== 164 passed in 2.87s ==...
Bug: [google internal] b/189371541
Change-Id: I5ea49f87e8fa41be590fc0c914573e16c8cdfcfa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309162
Tested-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Instead of setting properties on the instantiated command, pass them
via the constructor like normal objects.
Change-Id: I8787499bd2be68565875ffe243c3cf2024b36ae7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309324
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Running repo with Python 3.5 fails due to the use of the encoding
parameter to subprocess.run(). There are also f-strings being used in
some of the tests.
This drops support for these systems:
* Ubuntu Xenial: released Apr 2016, EOS Apr 2021, EOL Apr 2024
* Debian Stretch: released Jun 2017, EOL Jun 2022
So the minimum required distros now are:
* Ubuntu Bionic: released Apr 2018 w/Python 3.6
* Debian Buster: released Jul 2019 w/Python 3.7
Change-Id: I1144f7ab6f882b10cac0131982df081fe4ac44f9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303363
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
The only time we really need ssh proxies is when we want to run many
connections and reuse them. That only happens when running sync.
Every other command makes at most two connections, and even then it's
only one or none. So the effort of setting up & tearing down ssh
proxies isn't worth it most of the time.
The big reason we want to move this logic to sync is that it's now
using multiprocessing for parallel work. The current ssh proxy code
is all based on threads, which means none of the logic is working
correctly. The current ssh design makes it hard to fix when all of
the state lives in the global/module scope.
So the first step to fixing this is top move the setup & teardown to
the one place that really needs it: sync. No other commands will use
proxies anymore, just direct connections.
Bug: https://crbug.com/gerrit/12389
Change-Id: Ibd351acdec39a87562b3013637c5df4ea34e03c6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305485
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
We had ssh logic sprinkled between two git modules, and neither was
quite the right home for it. This largely moves the logic as-is to
its new home. We'll leave major refactoring to followup commits.
Bug: https://crbug.com/gerrit/12389
Change-Id: I300a8f7dba74f2bd132232a5eb1e856a8490e0e9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305483
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Add new CommonOptions entry points to move the existing --jobs to,
and relocate all --verbose/--quiet options to that. This provides
both a consistent interface for users as well as for code.
Change-Id: Ifaf83b88872421f4749b073c472b4a67ca6c0437
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303224
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Add a new "event": "command", which is emitted at when all command
arguments have been processed.
Additional fields:
"name": Name of the primary command (ex: repo, git)
"subcommands"': List of the sub-commands once command-line arguments
are processed
Examples:
Command: repo --version
Event: {"event": "command", <common fields>,
"name": "repo",
"subcommands": ["version"]
}
Bug: [google internal] b/178507266
Testing:
- Unit tests
- Verified repo git trace2 logs had expected data
Change-Id: I825bd0ecedee45135382461a4ba10f987f09aef3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/300343
Reviewed-by: Ian Kasprzak <iankaz@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
We're committed to Python 3 at this point, so purge all the
is_python3 related dynamic checks.
Bug: https://crbug.com/gerrit/10418
Change-Id: I4c8b405d6de359b8b83223c9f4b9c8ffa18ea1a2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292383
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
We've been warning about Python 3.4 for almost a year. This drops
support for these systems:
* Ubuntu Trusty: released Apr 2014, EOL Apr 2022
* Debian Jessie: released Apr 2015, EOL Jun 2020
So the min required distros would now be:
* Ubuntu Xenial: released Sep 2015 w/Python 3.5
* Debian Stretch: released Jun 2017 w/Python 3.6
I don't think we're quite ready to drop Python 3.5 which would affect
Ubuntu Xenial -- we'd have to update to Ubuntu Bionic from Apr 2018.
Let's see how much the community reacts to loss of Python 3.4 first.
Bug: https://crbug.com/gerrit/10418
Change-Id: Ib24a57818fdca49e23db53e1bdd1f4c76b4963f7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/291502
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Ways to enable logging:
1) Set git's trace2.eventtarget config variable to desired logging path
2) Specify path via --git_trace2_event_log option
A unique logfile name is generated per repo execution (based on the repo session-id).
Testing:
1) Verified git config 'trace2.eventtarget' and flag enable logging.
2) Verified version/start/end events are expected format:
https://git-scm.com/docs/api-trace2#_event_format
3) Unit tests
Bug: https://crbug.com/gerrit/13706
Change-Id: I335eba68124055321c4149979bec36ac16ef81eb
Tested-by: Ian Kasprzak <iankaz@google.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292262
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
We conflate the manifest & parsing logic with the management of the
repo client checkout in a single class. This makes testing just one
part (the manifest parsing) hard as it requires a full checkout too.
Start splitting the two apart into separate classes to make it easy
to reason about & test.
Change-Id: Iaf897c93db9c724baba6044bfe7a589c024523b2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/288682
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
We've been warning about this for more than 6 months (with public
announcements even older). Lets make it a failure now to see who
hasn't upgraded yet.
Bug: https://crbug.com/gerrit/10418
Change-Id: Iec3e2cbf87de434021921616683d360bc4fef77a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/280796
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
The launcher already raised itself up to use Python 3 on the fly, and
the main.py script uses a plain `python` shebang. So make sure we use
the active interpreter when re-execing ourselves to avoid falling back
down to Python 2 (which then triggers warnings).
Change-Id: Ic53c07dead3bc9233e4089a0a422f83bb5ac2f91
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/263272
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@digital.ai>
The current subcmds design has singletons in all_commands. This isn't
exactly unusual, but the fact that our main & help subcommand will then
attach members to the classes before invoking them is. This makes it
hard to keep track of what members a command has access to, and the two
code paths (main & help) attach different members depending on what APIs
they then invoke.
Lets pull this back a step by storing classes in all_commands and leave
the instantiation step to when they're used. This doesn't fully clean
up the confusion, but gets us closer.
Change-Id: I6a768ff97fe541e6f3228358dba04ed66c4b070a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/259154
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
The branch->branches alias is setup in the main module when that
really belongs in the existing all_commands setup.
For help, rather than monkey patching all_commands to the class,
switch it to use the state directly from the module. This makes
it a bit more obvious where it's coming from rather than this one
subcommand having a |commands| member added externally to it.
Change-Id: I0200def09bf4774cad8012af0f4ae60ea3089dc0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/259153
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Some users get repo from their distro (e.g. /usr/bin/repo), so the
suggestion to copy over top of it makes people uneasy, if it's even
possible in the first place.
Bug: https://crbug.com/gerrit/12335
Change-Id: I9a0c83d6ba0f466fa8e6d61f674ee13396f9a968
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256893
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
This supports [alias] sections with repo subcommands just like git.
Change-Id: Ie9235b5d4449414e6a745814f0110bd6af74ea93
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255833
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
The pager setting is tristate (where None means "auto"),
so make sure we still handle that setting.
Change-Id: I89fe352572dd15922c61e3bb65ac33f847d01ee0
Test: `repo help upload` triggers the pager
Test: `repo -p help upload` triggers the pager
Test: `repo --no-pager help upload` doesn't trigger the pager
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255852
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Trying to use booleans with names like "no_xxx" are hard to follow due
to the double negatives. Invert all of them so we only have positive
meanings to follow.
Change-Id: Ifd37d0368f97034d94aa2cf38db52c723ac0c6ed
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255493
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
If an older launcher script is used with newer repo source tree, we
might be issuing python version warnings. Plus, we want to be able
to roll Python version requirements independently of the launcher.
Add some version checking here too.
Change-Id: Ia35fc821f93c429296bdf5fd578276fef796b649
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255592
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
All of the instances of this are related to Python 2 names that
don't exist in Python 3, and the warnings are raised when running
flake8 on Python 3.
All of these will go away once we completely remove support for
Python 2, so just suppress them inline. We don't globally suppress
the check so that we will still see legitimate errors if/when they
occur in new code.
Change-Id: Iccf955f50abfc9f83b371fc0af6cceb51037456f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255039
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
We increased the max line length to 100 columns which got rid of
the majority of these warnings, but there were still a few lines
that exceeded 100 columns.
Change-Id: Ib3372868ca2297f83073a14f91c8ae3df9d0d0e6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254699
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
