Commit Graph

134 Commits

Author SHA1 Message Date
Josip Sokcevic
a3a7372612 main: Stringify project name in error_info
If a project can't be removed from checkout due to uncommitted changes
present, error.project is type of Project and not a string (as it is in
some cases). Project is not JSON serializable, resulting in exception
within exception handler:

TypeError: Object of type Project is not JSON serializable

This change casts project to string as a defensive mechanism. It also
passes project name instead of project object.

Change-Id: Ie7b782d73dc3647975755d5a3774d16ea6cd5348
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/413877
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-03-15 19:26:10 +00:00
Josip Sokcevic
8896b68926 trace: Save trace2 sid in REPO_TRACE file
git-trace2 events contain additional information what git is doing under
the hood, and repo doesn't have visibility into.

Instead of relying on timestamp information to match REPO_TRACE with
git-trace2 events, add SID information into REPO_TRACE.

Change-Id: I37672a3face81858072c7a3ce34ca3379199dab5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/410280
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-02-22 20:55:09 +00:00
LuK1337
89f761cfef main: Log ManifestParseError exception messages
This lets us see manifest parsing error messages again.

Change-Id: I2d90b97cfb50e4520f79e75fa0d648c373b49e98
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/391477
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Łukasz Patron <priv.luk@gmail.com>
Tested-by: Łukasz Patron <priv.luk@gmail.com>
2023-11-06 19:39:24 +00:00
Jason R. Coombs
b32ccbb66b cleanup: Update codebase to expect Python 3.6
- Bump minimum version to Python 3.6.
- Use f-strings in a lot of places.

Change-Id: I2aa70197230fcec2eff8e7c8eb754f20c08075bb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389034
Tested-by: Jason R. Coombs <jaraco@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jason R. Coombs <jaraco@google.com>
2023-10-31 16:03:54 +00:00
Jason R. Coombs
ae824fb2fc cleanup: convert exceptions to OSError
In Python 3, these exceptions were merged into OSError, so switch
everything over to that.

Change-Id: If876a28b692de5aa5c62a3bdc8c000793ce52c63
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390376
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2023-10-21 00:56:10 +00:00
Mike Frysinger
d4aee6570b delete Python 2 (object) compat
Bug: 302871152
Change-Id: I39636d73a6e1d69efa8ade74f75c5381651e6dc8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390054
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2023-10-20 04:51:01 +00:00
Mike Frysinger
8f4f98582e main: drop Python 2 check
Python 2 can't even parse this code anymore due to syntax changes,
so there's no point in checking for it explicitly.

Bug: 302871152
Change-Id: I9852ace5f5079d037c60fd3ac490d77e074e6875
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389434
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2023-10-13 20:08:33 +00:00
Aravind Vasudevan
8bc5000423 Update logger.warn to logger.warning
Bug: 305035810
Change-Id: Ic2b35d5c3cbe92480c24da612f29382f5d26d4aa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389414
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
2023-10-13 19:34:26 +00:00
Aravind Vasudevan
b8fd19215f main: Use repo logger
Bug: b/292704435
Change-Id: Ica02e4c00994a2f64083bb36e8f4ee8aa45d76bd
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/386454
Reviewed-by: Jason Chang <jasonnc@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Tested-by: Aravind Vasudevan <aravindvasudev@google.com>
2023-09-18 20:06:30 +00:00
Mike Frysinger
6447733eb2 isort: format codebase
Change-Id: I6f11d123b68fd077f558d3c21349c55c5f251019
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/383715
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2023-08-22 18:32:22 +00:00
Jason Chang
8914b1f86d gitc: drop support
Bug: b/282775958
Change-Id: Ib6383d6fd82a017d0a6670d6558a905d41be321f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/375314
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Commit-Queue: Jason Chang <jasonnc@google.com>
2023-08-15 22:14:52 +00:00
Jason Chang
1a3612fe6d Raise RepoExitError in place of sys.exit
Bug: b/293344017
Change-Id: Icae4932b00e4068cba502a5ab4a0274fd7854d9d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/382214
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Jason Chang <jasonnc@google.com>
2023-08-10 23:46:31 +00:00
Gavin Mak
f1ddaaa553 main: Pass path to python binary as arg0 when restarting repo
Not including it causes flaky behavior in some Chromium builders
because Chromium's custom Python build used by vpython relies on
argv[0] to find its own internal files.

Bug: https://crbug.com/1468522
Change-Id: I5c32ebe71c9b684d6ee50dbd8c3d6fcd51ca309b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/381974
Reviewed-by: Chenlin Fan <fancl@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2023-08-08 05:50:07 +00:00
Jason Chang
32b59565b7 Refactor errors for sync command
Per discussion in go/repo-error-update updated aggregated and exit
errors for sync command.

Aggregated errors are errors that result in eventual command failure.
Exit errors are errors that result in immediate command failure.

Also updated main.py to log aggregated and exit errors to git sessions
log

Bug: b/293344017
Change-Id: I77a21f14da32fe2e68c16841feb22de72e86a251
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/379614
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Tested-by: Jason Chang <jasonnc@google.com>
Commit-Queue: Jason Chang <jasonnc@google.com>
2023-08-02 18:29:05 +00:00
Gavin Mak
1d2e99d028 sync: Track last completed fetch/checkout
Save the latest time any project is fetched and checked out. This will
be used to detect partial checkouts.

Bug: b/286126621
Change-Id: I53b264dc70ba168d506076dbd693ef79a696b61d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/380514
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Joanna Wang <jojwang@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2023-07-28 18:55:04 +00:00
Jason Chang
c657844efe main: Fix exitcode logging
Fixed a couple of bugs in ExitEvent logging:
- log exitcode 130 on KeyboardInterrupt
- log exitcode 1 on unhandled Exception
- log errorevent with specific reason for exit

Before this CL an exitcode of 0 would be logged, and it would be
difficult to determine the cause of non-zero exit codes

Bug: b/287105597
Change-Id: I2d34f180581f9fbd77a1c78c966ebed065223af6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/377834
Tested-by: Jason Chang <jasonnc@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
2023-06-26 15:38:30 +00:00
Gavin Mak
ea2e330e43 Format codebase with black and check formatting in CQ
Apply rules set by https://gerrit-review.googlesource.com/c/git-repo/+/362954/ across the codebase and fix any lingering errors caught
by flake8. Also check black formatting in run_tests (and CQ).

Bug: b/267675342
Change-Id: I972d77649dac351150dcfeb1cd1ad0ea2efc1956
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/363474
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2023-03-22 17:46:28 +00:00
Joanna Wang
24c6314fca Fix TRACE_FILE renaming.
Bug: b/258073923

Change-Id: I997961056388e1550711f73a6310788b5c7ad4d4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350934
Tested-by: Joanna Wang <jojwang@google.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
2022-11-09 01:24:49 +00:00
LaMont Jones
a3ff64cae5 Improve always-on-trace
Notes to the user need to go to stderr, and tracing should not be on for
fast exiting invocations (such as --help).

This makes it so that release/update-manpages works.

Change-Id: Ib183193c868a78c295a184c01c4532cd53d512eb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350794
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
2022-11-08 19:54:20 +00:00
Joanna Wang
a6c52f566a Set tracing to always on and save to .repo/TRACE_FILE.
- add `--trace_to_stderr` option so stderr will include trace outputs and any other errors that get sent to stderr
- while TRACE_FILE will only include trace outputs

piggy-backing on: https://gerrit-review.googlesource.com/c/git-repo/+/349154

Change-Id: I3895a84de4b2784f17fac4325521cd5e72e645e2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350114
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Joanna Wang <jojwang@google.com>
2022-11-03 21:07:07 +00:00
LaMont Jones
bdcba7dc36 sync: add multi-manifest support
With this change, partial syncs (sync with a project list) are again
supported.

If the updated manifest includes new sub manifests, download them
inheriting options from the parent manifestProject.

Change-Id: Id952f85df2e26d34e38b251973be26434443ff56
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334819
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
2022-05-26 00:03:37 +00:00
LaMont Jones
b90a422ab6 Override the manifest for the entire command
When a manifest file is overridden, remember that and keep using the
override for the remainder of the process.  If we need to revert it,
make the override name evaluate False.

Change-Id: I1eee05fec6988c1ee4a3c751c4b540d5b5d11797
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/335136
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-19 21:28:20 +00:00
LaMont Jones
cc879a97c3 Add multi-manifest support with <submanifest> element
To be addressed in another change:
 - a partial `repo sync` (with a list of projects/paths to sync)
   requires `--this-tree-only`.

Change-Id: I6c7400bf001540e9d7694fa70934f8f204cb5f57
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322657
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-02-17 21:57:55 +00:00
Mike Frysinger
73c43b839f repo: add --show-toplevel akin to git
Simple API to make it easy to find the top of the repo client checkout
for users.  This mirrors the `git rev-parse --show-toplevel` API.

Change-Id: I0c3f98def089d0fc9ebcfa50aa3dc02091c1c273
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312909
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-07-28 05:38:53 +00:00
Mike Frysinger
56345c345b repo: refactor help output handling
Currently we have the behavior:
* `repo`: Equivalent to `repo help` -- only shows common subcommands
  (with short description), and then exits 0.
* `repo --help`: Shows repo's core options, lists all commands (no
  specific info), and then exits 0.

The first case is not behaving well:
* If you run `repo` without a specific subcommand, that's an error,
  so we should be exiting 1 instead.
* Showing only subcommands and no actual option summary makes it seem
  like repo itself doesn't take any options.  This confuses users.

Let's rework things a bit.  Now we have the behavior:
* `repo`: Shows repo's core options, lists all commands (no specific
  info), and then exits 1.
* `repo --help`: Shows repo's core options, shows common subcommands
  (with short description), and then exits 0.
* `repo --help-all`: Shows repo's core options, shows all subcommands
  (with short description), and then exits 0.

Basically we swap the behavior of `repo` and `repo --help`, and fix
the exit status when the subcommand is missing.

The addition of --help-all is mostly for the man pages.  We were
relying on `repo help --all` to generate the repo(1) man page, but
that too omitted the core repo options.  Now the man page includes
all the core repo options and provides a summary of all commands.

Change-Id: I1f99b99d5b8af2591f96a078d0647a3d76d6b0fc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312908
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-07-28 05:38:34 +00:00
Mike Frysinger
a024bd33b8 repo: make --version always work
We don't really care what the subcommand is set to when --version
output is requested, so stop enforcing it.  This fixes some weird
behavior like `repo --version version` fails, but `repo --version
help` works.

The new logic skips subcommand validation, so `repo --version asdf`
will still display the version output.  This matches git behavior,
and makes a bit of sense when we consider that the user really wants
to see the tool version, and probably doesn't care about anything
else on the command line.

Change-Id: I87454d473c2c8869344b3888a7affaa2e03f5b0f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312907
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-07-28 05:37:52 +00:00
Mike Frysinger
968d646f04 repo: refactor internal --help/--version parsing
The _ParseArgs method parses the arguments and processes some of
the options, with the rest left to the _Run method.  Simplify the
_ParseArgs method to only parse arguments and have _Run handle all
actual processing.

This will make it easier to add more terminal options (ones that
exit immediately without a subcommand), and makes it easier to
understand the overall code flow.

Change-Id: I47f7274c3f2b59378fd479e403e70fb24b681536
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312906
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-07-28 05:37:27 +00:00
Raman Tenneti
784e16f3aa superproject: Don't exit if superproject tag doesn't exist in manifest.
Don't exit if there are missing commit ids in superproject.

This change implements the following suggestion from delphij@:

"we should note the event (so we know that --use-superproject but there
 were some errors, e.g. manifest didn't specify commit id for some
 reason, or if there is no superproject but --use-superproject is
 used), print out a message telling the use that this is not support,
 but continue as if --no-use-superproject was specified?"

Changes:

superproject:
+ Added git_trace2_event_log as an argument to the constructor.
+ Sync method returns SyncResult a NamedTuple of
  ++ success - True if sync of superproject is successful, or False.
  ++ fatal - True if caller should exit, Or False.
+ UpdateProjectsRevisionId returns UpdateProjectsResult a NamedTuple of
   ++ manifest_path - path name of the overriding manifest file instead
      of None
   ++ fatal - True if caller should exit, Or False
+ _GetAllProjectsCommitIds returns CommitIdsResult a NamedTuple of
  ++ commit_ids - a dictionary with the projects/commit ids on success,
     otherwise None
  ++ fatal - True if caller should exit, Or False
+ Added  _SkipUpdatingProjectRevisionId a helper function to see if a
  project's revision id needs to be updated or not. This function is
  used to exclude projects from local manifest file.
+ Added the following error events into git_trace2_event_log
  ++ If superproject is missing in a manifest
  ++ If there are missing commit ids for projects.

command.py:
+ Deleted unused import - platform
+ Added git_trace2_event_log as a member so all subcmds can log error
  events.

main.py:
+ Initialized git_trace2_event_log as a member of command object.

init.py:
+ Deleted unused import - optparse

init.py:
+ Called sys.exit only if Sync returns exit=True

sync.py:
+ Called sys.exit only if Superproject's UpdateProjectsRevisionId returns
  exit=True
+ Reloaded the manifest only if manifest path is returned by
  UpdateProjectsRevisionId. If not, fall back to the old way of doing
  repo sync.

test_git_superproject:
+ Added code to verify error events are being logged.
+ Added a test for no superproject tag
+ Added test for UpdateProjectsRevisionId not updating the revision id
  with the commit ids.

Tested the code with the following commands.

+ Positive test case with aosp-master.
  $ repo_dev init -u persistent-https://android.git.corp.google.com/platform/manifest -b master --use-superproject
  NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
  .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed.

  Your identity is: Raman Tenneti <rtenneti@google.com>
  If you want to change this, please re-run 'repo init' with --config-name

  repo has been initialized in .../android/aosp

  $ repo_dev sync -j40 --use-superproject
  remote: Total 12 (delta 4), reused 12 (delta 4)
  NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
  .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed.
  ...
  repo sync has finished successfully.

+ Negative test case without superproject tag.
  $ repo_dev sync -j40 --use-superproject
  NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
  repo error: superproject tag is not defined in manifest: .../android/aosp/.repo/manifest.xml
  error: Cannot get project commit ids from manifest
  error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option
  ...
  Checking out: 100% (1022/1022), done in 3.589s
  repo sync has finished successfully.

+ Test for missing commit_id for a project.
  $ repo_dev sync -j40 --use-superproject
  NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
  .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed.
  error: please file a bug using go/repo-bug to report missing commit_ids for: ['build/blueprint']
  error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option
  ...
  Checking out: 100% (1022/1022), done in 3.364s
  repo sync has finished successfully.

$ ./run_tests -v
  ...
  ...== 164 passed in 2.87s ==...

Bug: [google internal] b/189371541
Change-Id: I5ea49f87e8fa41be590fc0c914573e16c8cdfcfa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309162
Tested-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-06-16 04:48:35 +00:00
Mike Frysinger
d58d0dd3bf commands: pass settings via __init__
Instead of setting properties on the instantiated command, pass them
via the constructor like normal objects.

Change-Id: I8787499bd2be68565875ffe243c3cf2024b36ae7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309324
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-06-15 06:08:13 +00:00
Peter Kjellerstedt
a3b2edf1af Drop support for Python 3.5
Running repo with Python 3.5 fails due to the use of the encoding
parameter to subprocess.run(). There are also f-strings being used in
some of the tests.

This drops support for these systems:
* Ubuntu Xenial: released Apr 2016, EOS Apr 2021, EOL Apr 2024
* Debian Stretch: released Jun 2017, EOL Jun 2022

So the minimum required distros now are:
* Ubuntu Bionic: released Apr 2018 w/Python 3.6
* Debian Buster: released Jul 2019 w/Python 3.7

Change-Id: I1144f7ab6f882b10cac0131982df081fe4ac44f9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303363
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
2021-06-03 11:34:17 +00:00
Mike Frysinger
19e409c818 ssh: move proxy usage to the sync subcommand
The only time we really need ssh proxies is when we want to run many
connections and reuse them.  That only happens when running sync.
Every other command makes at most two connections, and even then it's
only one or none.  So the effort of setting up & tearing down ssh
proxies isn't worth it most of the time.

The big reason we want to move this logic to sync is that it's now
using multiprocessing for parallel work.  The current ssh proxy code
is all based on threads, which means none of the logic is working
correctly.  The current ssh design makes it hard to fix when all of
the state lives in the global/module scope.

So the first step to fixing this is top move the setup & teardown to
the one place that really needs it: sync.  No other commands will use
proxies anymore, just direct connections.

Bug: https://crbug.com/gerrit/12389
Change-Id: Ibd351acdec39a87562b3013637c5df4ea34e03c6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305485
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-05-10 21:10:29 +00:00
Mike Frysinger
5291eafa41 ssh: move all ssh logic to a common place
We had ssh logic sprinkled between two git modules, and neither was
quite the right home for it.  This largely moves the logic as-is to
its new home.  We'll leave major refactoring to followup commits.

Bug: https://crbug.com/gerrit/12389
Change-Id: I300a8f7dba74f2bd132232a5eb1e856a8490e0e9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305483
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-05-06 19:09:16 +00:00
Mike Frysinger
9180a07b8f command: make --verbose/--quiet available to all subcommands
Add new CommonOptions entry points to move the existing --jobs to,
and relocate all --verbose/--quiet options to that.  This provides
both a consistent interface for users as well as for code.

Change-Id: Ifaf83b88872421f4749b073c472b4a67ca6c0437
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303224
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-04-13 22:25:17 +00:00
Raman Tenneti
a5b40a2845 repo: Add a new "command" event type to git trace2 logging in repo.
Add a new "event": "command", which is emitted at when all command
arguments have been processed.

Additional fields:

"name": Name of the primary command (ex: repo, git)

"subcommands"': List of the sub-commands once command-line arguments
                are processed

Examples:

Command: repo --version

Event: {"event": "command", <common fields>,
        "name": "repo",
	"subcommands": ["version"]
	}

Bug: [google internal] b/178507266
Testing:
- Unit tests
- Verified repo git trace2 logs had expected data

Change-Id: I825bd0ecedee45135382461a4ba10f987f09aef3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/300343
Reviewed-by: Ian Kasprzak <iankaz@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-03-18 14:58:24 +00:00
Ian Kasprzak
835a34bdb9 Log repo.* config variables in git trace2 logger.
Bug: [google internal] b/181758736
Testing:
- Unit tests
- Verified repo git trace2 logs had expected data

Change-Id: I9af8a574377bd91115f085808c1271e9dee16a36
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299182
Tested-by: Ian Kasprzak <iankaz@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
2021-03-08 17:32:09 +00:00
Mike Frysinger
acf63b2892 drop pyversion & is_python3 checking
We're committed to Python 3 at this point, so purge all the
is_python3 related dynamic checks.

Bug: https://crbug.com/gerrit/10418
Change-Id: I4c8b405d6de359b8b83223c9f4b9c8ffa18ea1a2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292383
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-01-06 18:53:58 +00:00
Mike Frysinger
784ccfc040 strip python2-only coding:utf-8 & print_function settings
We're committed to Python 3 at this point, so clean up boilerplate.

Bug: https://crbug.com/gerrit/10418
Change-Id: Ib1719ba2eb65c53b94881a1a1bf203ddfcaaafed
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292382
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-01-06 18:53:05 +00:00
Mike Frysinger
128f34e874 main: require Python 3.5 now
We've been warning about Python 3.4 for almost a year.  This drops
support for these systems:
* Ubuntu Trusty: released Apr 2014, EOL Apr 2022
* Debian Jessie: released Apr 2015, EOL Jun 2020

So the min required distros would now be:
* Ubuntu Xenial: released Sep 2015 w/Python 3.5
* Debian Stretch: released Jun 2017 w/Python 3.6

I don't think we're quite ready to drop Python 3.5 which would affect
Ubuntu Xenial -- we'd have to update to Ubuntu Bionic from Apr 2018.
Let's see how much the community reacts to loss of Python 3.4 first.

Bug: https://crbug.com/gerrit/10418
Change-Id: Ib24a57818fdca49e23db53e1bdd1f4c76b4963f7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/291502
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-01-06 17:55:15 +00:00
Ian Kasprzak
30bc354e25 Enable git trace2 event format logging.
Ways to enable logging:
1) Set git's trace2.eventtarget config variable to desired logging path
2) Specify path via --git_trace2_event_log option

A unique logfile name is generated per repo execution (based on the repo session-id).

Testing:
1) Verified git config 'trace2.eventtarget' and flag enable logging.
2) Verified version/start/end events are expected format:
  https://git-scm.com/docs/api-trace2#_event_format
3) Unit tests

Bug: https://crbug.com/gerrit/13706
Change-Id: I335eba68124055321c4149979bec36ac16ef81eb
Tested-by: Ian Kasprzak <iankaz@google.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292262
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
2021-01-06 17:49:41 +00:00
Mike Frysinger
8c1e9cbef1 manifest_xml: refactor manifest parsing from client management
We conflate the manifest & parsing logic with the management of the
repo client checkout in a single class.  This makes testing just one
part (the manifest parsing) hard as it requires a full checkout too.

Start splitting the two apart into separate classes to make it easy
to reason about & test.

Change-Id: Iaf897c93db9c724baba6044bfe7a589c024523b2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/288682
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2020-11-18 19:10:57 +00:00
Mike Frysinger
a488af5ea5 main: require Python 3 now
We've been warning about this for more than 6 months (with public
announcements even older).  Lets make it a failure now to see who
hasn't upgraded yet.

Bug: https://crbug.com/gerrit/10418
Change-Id: Iec3e2cbf87de434021921616683d360bc4fef77a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/280796
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2020-11-17 15:04:20 +00:00
Mike Frysinger
dd37fb2222 main: re-exec self with the current interp
The launcher already raised itself up to use Python 3 on the fly, and
the main.py script uses a plain `python` shebang.  So make sure we use
the active interpreter when re-execing ourselves to avoid falling back
down to Python 2 (which then triggers warnings).

Change-Id: Ic53c07dead3bc9233e4089a0a422f83bb5ac2f91
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/263272
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@digital.ai>
2020-04-28 02:54:50 +00:00
Mike Frysinger
bb930461ce subcmds: stop instantiating at import time
The current subcmds design has singletons in all_commands.  This isn't
exactly unusual, but the fact that our main & help subcommand will then
attach members to the classes before invoking them is.  This makes it
hard to keep track of what members a command has access to, and the two
code paths (main & help) attach different members depending on what APIs
they then invoke.

Lets pull this back a step by storing classes in all_commands and leave
the instantiation step to when they're used.  This doesn't fully clean
up the confusion, but gets us closer.

Change-Id: I6a768ff97fe541e6f3228358dba04ed66c4b070a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/259154
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-03-17 00:08:52 +00:00
Mike Frysinger
d3639c53d5 subcmds: centralize all_commands logic
The branch->branches alias is setup in the main module when that
really belongs in the existing all_commands setup.

For help, rather than monkey patching all_commands to the class,
switch it to use the state directly from the module.  This makes
it a bit more obvious where it's coming from rather than this one
subcommand having a |commands| member added externally to it.

Change-Id: I0200def09bf4774cad8012af0f4ae60ea3089dc0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/259153
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-03-17 00:08:52 +00:00
Mike Frysinger
eea23b44a9 main: improve launcher update messaging wrt system installs
Some users get repo from their distro (e.g. /usr/bin/repo), so the
suggestion to copy over top of it makes people uneasy, if it's even
possible in the first place.

Bug: https://crbug.com/gerrit/12335
Change-Id: I9a0c83d6ba0f466fa8e6d61f674ee13396f9a968
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256893
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-26 23:20:10 +00:00
Mike Frysinger
949bc34267 main/repo: add support for subcommand aliases
This supports [alias] sections with repo subcommands just like git.

Change-Id: Ie9235b5d4449414e6a745814f0110bd6af74ea93
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255833
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2020-02-20 00:53:46 +00:00
Mike Frysinger
8a98efee5c main: fix pager logic after negation cleanup
The pager setting is tristate (where None means "auto"),
so make sure we still handle that setting.

Change-Id: I89fe352572dd15922c61e3bb65ac33f847d01ee0
Test: `repo help upload` triggers the pager
Test: `repo -p help upload` triggers the pager
Test: `repo --no-pager help upload` doesn't trigger the pager
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255852
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-19 08:00:11 +00:00
Mike Frysinger
c58ec4dba1 avoid negative variables
Trying to use booleans with names like "no_xxx" are hard to follow due
to the double negatives.  Invert all of them so we only have positive
meanings to follow.

Change-Id: Ifd37d0368f97034d94aa2cf38db52c723ac0c6ed
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255493
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-19 00:24:43 +00:00
Mike Frysinger
37f28f1b4e main: add python version checking
If an older launcher script is used with newer repo source tree, we
might be issuing python version warnings.  Plus, we want to be able
to roll Python version requirements independently of the launcher.
Add some version checking here too.

Change-Id: Ia35fc821f93c429296bdf5fd578276fef796b649
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255592
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-18 14:38:33 +00:00
David Pursehouse
a46bf7dc2a flake8: Suppress "F821 undefined name" inline for Python 2 names
All of the instances of this are related to Python 2 names that
don't exist in Python 3, and the warnings are raised when running
flake8 on Python 3.

All of these will go away once we completely remove support for
Python 2, so just suppress them inline. We don't globally suppress
the check so that we will still see legitimate errors if/when they
occur in new code.

Change-Id: Iccf955f50abfc9f83b371fc0af6cceb51037456f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255039
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
2020-02-15 04:45:16 +00:00