Commit Graph

117 Commits

Author SHA1 Message Date
Joanna Wang
24c6314fca Fix TRACE_FILE renaming.
Bug: b/258073923

Change-Id: I997961056388e1550711f73a6310788b5c7ad4d4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350934
Tested-by: Joanna Wang <jojwang@google.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
2022-11-09 01:24:49 +00:00
LaMont Jones
a3ff64cae5 Improve always-on-trace
Notes to the user need to go to stderr, and tracing should not be on for
fast exiting invocations (such as --help).

This makes it so that release/update-manpages works.

Change-Id: Ib183193c868a78c295a184c01c4532cd53d512eb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350794
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
2022-11-08 19:54:20 +00:00
Joanna Wang
a6c52f566a Set tracing to always on and save to .repo/TRACE_FILE.
- add `--trace_to_stderr` option so stderr will include trace outputs and any other errors that get sent to stderr
- while TRACE_FILE will only include trace outputs

piggy-backing on: https://gerrit-review.googlesource.com/c/git-repo/+/349154

Change-Id: I3895a84de4b2784f17fac4325521cd5e72e645e2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350114
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Joanna Wang <jojwang@google.com>
2022-11-03 21:07:07 +00:00
LaMont Jones
bdcba7dc36 sync: add multi-manifest support
With this change, partial syncs (sync with a project list) are again
supported.

If the updated manifest includes new sub manifests, download them
inheriting options from the parent manifestProject.

Change-Id: Id952f85df2e26d34e38b251973be26434443ff56
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334819
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
2022-05-26 00:03:37 +00:00
LaMont Jones
b90a422ab6 Override the manifest for the entire command
When a manifest file is overridden, remember that and keep using the
override for the remainder of the process.  If we need to revert it,
make the override name evaluate False.

Change-Id: I1eee05fec6988c1ee4a3c751c4b540d5b5d11797
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/335136
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-19 21:28:20 +00:00
LaMont Jones
cc879a97c3 Add multi-manifest support with <submanifest> element
To be addressed in another change:
 - a partial `repo sync` (with a list of projects/paths to sync)
   requires `--this-tree-only`.

Change-Id: I6c7400bf001540e9d7694fa70934f8f204cb5f57
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322657
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-02-17 21:57:55 +00:00
Mike Frysinger
73c43b839f repo: add --show-toplevel akin to git
Simple API to make it easy to find the top of the repo client checkout
for users.  This mirrors the `git rev-parse --show-toplevel` API.

Change-Id: I0c3f98def089d0fc9ebcfa50aa3dc02091c1c273
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312909
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-07-28 05:38:53 +00:00
Mike Frysinger
56345c345b repo: refactor help output handling
Currently we have the behavior:
* `repo`: Equivalent to `repo help` -- only shows common subcommands
  (with short description), and then exits 0.
* `repo --help`: Shows repo's core options, lists all commands (no
  specific info), and then exits 0.

The first case is not behaving well:
* If you run `repo` without a specific subcommand, that's an error,
  so we should be exiting 1 instead.
* Showing only subcommands and no actual option summary makes it seem
  like repo itself doesn't take any options.  This confuses users.

Let's rework things a bit.  Now we have the behavior:
* `repo`: Shows repo's core options, lists all commands (no specific
  info), and then exits 1.
* `repo --help`: Shows repo's core options, shows common subcommands
  (with short description), and then exits 0.
* `repo --help-all`: Shows repo's core options, shows all subcommands
  (with short description), and then exits 0.

Basically we swap the behavior of `repo` and `repo --help`, and fix
the exit status when the subcommand is missing.

The addition of --help-all is mostly for the man pages.  We were
relying on `repo help --all` to generate the repo(1) man page, but
that too omitted the core repo options.  Now the man page includes
all the core repo options and provides a summary of all commands.

Change-Id: I1f99b99d5b8af2591f96a078d0647a3d76d6b0fc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312908
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-07-28 05:38:34 +00:00
Mike Frysinger
a024bd33b8 repo: make --version always work
We don't really care what the subcommand is set to when --version
output is requested, so stop enforcing it.  This fixes some weird
behavior like `repo --version version` fails, but `repo --version
help` works.

The new logic skips subcommand validation, so `repo --version asdf`
will still display the version output.  This matches git behavior,
and makes a bit of sense when we consider that the user really wants
to see the tool version, and probably doesn't care about anything
else on the command line.

Change-Id: I87454d473c2c8869344b3888a7affaa2e03f5b0f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312907
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-07-28 05:37:52 +00:00
Mike Frysinger
968d646f04 repo: refactor internal --help/--version parsing
The _ParseArgs method parses the arguments and processes some of
the options, with the rest left to the _Run method.  Simplify the
_ParseArgs method to only parse arguments and have _Run handle all
actual processing.

This will make it easier to add more terminal options (ones that
exit immediately without a subcommand), and makes it easier to
understand the overall code flow.

Change-Id: I47f7274c3f2b59378fd479e403e70fb24b681536
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312906
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-07-28 05:37:27 +00:00
Raman Tenneti
784e16f3aa superproject: Don't exit if superproject tag doesn't exist in manifest.
Don't exit if there are missing commit ids in superproject.

This change implements the following suggestion from delphij@:

"we should note the event (so we know that --use-superproject but there
 were some errors, e.g. manifest didn't specify commit id for some
 reason, or if there is no superproject but --use-superproject is
 used), print out a message telling the use that this is not support,
 but continue as if --no-use-superproject was specified?"

Changes:

superproject:
+ Added git_trace2_event_log as an argument to the constructor.
+ Sync method returns SyncResult a NamedTuple of
  ++ success - True if sync of superproject is successful, or False.
  ++ fatal - True if caller should exit, Or False.
+ UpdateProjectsRevisionId returns UpdateProjectsResult a NamedTuple of
   ++ manifest_path - path name of the overriding manifest file instead
      of None
   ++ fatal - True if caller should exit, Or False
+ _GetAllProjectsCommitIds returns CommitIdsResult a NamedTuple of
  ++ commit_ids - a dictionary with the projects/commit ids on success,
     otherwise None
  ++ fatal - True if caller should exit, Or False
+ Added  _SkipUpdatingProjectRevisionId a helper function to see if a
  project's revision id needs to be updated or not. This function is
  used to exclude projects from local manifest file.
+ Added the following error events into git_trace2_event_log
  ++ If superproject is missing in a manifest
  ++ If there are missing commit ids for projects.

command.py:
+ Deleted unused import - platform
+ Added git_trace2_event_log as a member so all subcmds can log error
  events.

main.py:
+ Initialized git_trace2_event_log as a member of command object.

init.py:
+ Deleted unused import - optparse

init.py:
+ Called sys.exit only if Sync returns exit=True

sync.py:
+ Called sys.exit only if Superproject's UpdateProjectsRevisionId returns
  exit=True
+ Reloaded the manifest only if manifest path is returned by
  UpdateProjectsRevisionId. If not, fall back to the old way of doing
  repo sync.

test_git_superproject:
+ Added code to verify error events are being logged.
+ Added a test for no superproject tag
+ Added test for UpdateProjectsRevisionId not updating the revision id
  with the commit ids.

Tested the code with the following commands.

+ Positive test case with aosp-master.
  $ repo_dev init -u persistent-https://android.git.corp.google.com/platform/manifest -b master --use-superproject
  NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
  .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed.

  Your identity is: Raman Tenneti <rtenneti@google.com>
  If you want to change this, please re-run 'repo init' with --config-name

  repo has been initialized in .../android/aosp

  $ repo_dev sync -j40 --use-superproject
  remote: Total 12 (delta 4), reused 12 (delta 4)
  NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
  .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed.
  ...
  repo sync has finished successfully.

+ Negative test case without superproject tag.
  $ repo_dev sync -j40 --use-superproject
  NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
  repo error: superproject tag is not defined in manifest: .../android/aosp/.repo/manifest.xml
  error: Cannot get project commit ids from manifest
  error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option
  ...
  Checking out: 100% (1022/1022), done in 3.589s
  repo sync has finished successfully.

+ Test for missing commit_id for a project.
  $ repo_dev sync -j40 --use-superproject
  NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
  .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed.
  error: please file a bug using go/repo-bug to report missing commit_ids for: ['build/blueprint']
  error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option
  ...
  Checking out: 100% (1022/1022), done in 3.364s
  repo sync has finished successfully.

$ ./run_tests -v
  ...
  ...== 164 passed in 2.87s ==...

Bug: [google internal] b/189371541
Change-Id: I5ea49f87e8fa41be590fc0c914573e16c8cdfcfa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309162
Tested-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-06-16 04:48:35 +00:00
Mike Frysinger
d58d0dd3bf commands: pass settings via __init__
Instead of setting properties on the instantiated command, pass them
via the constructor like normal objects.

Change-Id: I8787499bd2be68565875ffe243c3cf2024b36ae7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309324
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-06-15 06:08:13 +00:00
Peter Kjellerstedt
a3b2edf1af Drop support for Python 3.5
Running repo with Python 3.5 fails due to the use of the encoding
parameter to subprocess.run(). There are also f-strings being used in
some of the tests.

This drops support for these systems:
* Ubuntu Xenial: released Apr 2016, EOS Apr 2021, EOL Apr 2024
* Debian Stretch: released Jun 2017, EOL Jun 2022

So the minimum required distros now are:
* Ubuntu Bionic: released Apr 2018 w/Python 3.6
* Debian Buster: released Jul 2019 w/Python 3.7

Change-Id: I1144f7ab6f882b10cac0131982df081fe4ac44f9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303363
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
2021-06-03 11:34:17 +00:00
Mike Frysinger
19e409c818 ssh: move proxy usage to the sync subcommand
The only time we really need ssh proxies is when we want to run many
connections and reuse them.  That only happens when running sync.
Every other command makes at most two connections, and even then it's
only one or none.  So the effort of setting up & tearing down ssh
proxies isn't worth it most of the time.

The big reason we want to move this logic to sync is that it's now
using multiprocessing for parallel work.  The current ssh proxy code
is all based on threads, which means none of the logic is working
correctly.  The current ssh design makes it hard to fix when all of
the state lives in the global/module scope.

So the first step to fixing this is top move the setup & teardown to
the one place that really needs it: sync.  No other commands will use
proxies anymore, just direct connections.

Bug: https://crbug.com/gerrit/12389
Change-Id: Ibd351acdec39a87562b3013637c5df4ea34e03c6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305485
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-05-10 21:10:29 +00:00
Mike Frysinger
5291eafa41 ssh: move all ssh logic to a common place
We had ssh logic sprinkled between two git modules, and neither was
quite the right home for it.  This largely moves the logic as-is to
its new home.  We'll leave major refactoring to followup commits.

Bug: https://crbug.com/gerrit/12389
Change-Id: I300a8f7dba74f2bd132232a5eb1e856a8490e0e9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305483
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-05-06 19:09:16 +00:00
Mike Frysinger
9180a07b8f command: make --verbose/--quiet available to all subcommands
Add new CommonOptions entry points to move the existing --jobs to,
and relocate all --verbose/--quiet options to that.  This provides
both a consistent interface for users as well as for code.

Change-Id: Ifaf83b88872421f4749b073c472b4a67ca6c0437
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303224
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-04-13 22:25:17 +00:00
Raman Tenneti
a5b40a2845 repo: Add a new "command" event type to git trace2 logging in repo.
Add a new "event": "command", which is emitted at when all command
arguments have been processed.

Additional fields:

"name": Name of the primary command (ex: repo, git)

"subcommands"': List of the sub-commands once command-line arguments
                are processed

Examples:

Command: repo --version

Event: {"event": "command", <common fields>,
        "name": "repo",
	"subcommands": ["version"]
	}

Bug: [google internal] b/178507266
Testing:
- Unit tests
- Verified repo git trace2 logs had expected data

Change-Id: I825bd0ecedee45135382461a4ba10f987f09aef3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/300343
Reviewed-by: Ian Kasprzak <iankaz@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-03-18 14:58:24 +00:00
Ian Kasprzak
835a34bdb9 Log repo.* config variables in git trace2 logger.
Bug: [google internal] b/181758736
Testing:
- Unit tests
- Verified repo git trace2 logs had expected data

Change-Id: I9af8a574377bd91115f085808c1271e9dee16a36
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299182
Tested-by: Ian Kasprzak <iankaz@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
2021-03-08 17:32:09 +00:00
Mike Frysinger
acf63b2892 drop pyversion & is_python3 checking
We're committed to Python 3 at this point, so purge all the
is_python3 related dynamic checks.

Bug: https://crbug.com/gerrit/10418
Change-Id: I4c8b405d6de359b8b83223c9f4b9c8ffa18ea1a2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292383
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-01-06 18:53:58 +00:00
Mike Frysinger
784ccfc040 strip python2-only coding:utf-8 & print_function settings
We're committed to Python 3 at this point, so clean up boilerplate.

Bug: https://crbug.com/gerrit/10418
Change-Id: Ib1719ba2eb65c53b94881a1a1bf203ddfcaaafed
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292382
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-01-06 18:53:05 +00:00
Mike Frysinger
128f34e874 main: require Python 3.5 now
We've been warning about Python 3.4 for almost a year.  This drops
support for these systems:
* Ubuntu Trusty: released Apr 2014, EOL Apr 2022
* Debian Jessie: released Apr 2015, EOL Jun 2020

So the min required distros would now be:
* Ubuntu Xenial: released Sep 2015 w/Python 3.5
* Debian Stretch: released Jun 2017 w/Python 3.6

I don't think we're quite ready to drop Python 3.5 which would affect
Ubuntu Xenial -- we'd have to update to Ubuntu Bionic from Apr 2018.
Let's see how much the community reacts to loss of Python 3.4 first.

Bug: https://crbug.com/gerrit/10418
Change-Id: Ib24a57818fdca49e23db53e1bdd1f4c76b4963f7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/291502
Reviewed-by: Chris Mcdonald <cjmcdonald@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-01-06 17:55:15 +00:00
Ian Kasprzak
30bc354e25 Enable git trace2 event format logging.
Ways to enable logging:
1) Set git's trace2.eventtarget config variable to desired logging path
2) Specify path via --git_trace2_event_log option

A unique logfile name is generated per repo execution (based on the repo session-id).

Testing:
1) Verified git config 'trace2.eventtarget' and flag enable logging.
2) Verified version/start/end events are expected format:
  https://git-scm.com/docs/api-trace2#_event_format
3) Unit tests

Bug: https://crbug.com/gerrit/13706
Change-Id: I335eba68124055321c4149979bec36ac16ef81eb
Tested-by: Ian Kasprzak <iankaz@google.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292262
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
2021-01-06 17:49:41 +00:00
Mike Frysinger
8c1e9cbef1 manifest_xml: refactor manifest parsing from client management
We conflate the manifest & parsing logic with the management of the
repo client checkout in a single class.  This makes testing just one
part (the manifest parsing) hard as it requires a full checkout too.

Start splitting the two apart into separate classes to make it easy
to reason about & test.

Change-Id: Iaf897c93db9c724baba6044bfe7a589c024523b2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/288682
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2020-11-18 19:10:57 +00:00
Mike Frysinger
a488af5ea5 main: require Python 3 now
We've been warning about this for more than 6 months (with public
announcements even older).  Lets make it a failure now to see who
hasn't upgraded yet.

Bug: https://crbug.com/gerrit/10418
Change-Id: Iec3e2cbf87de434021921616683d360bc4fef77a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/280796
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2020-11-17 15:04:20 +00:00
Mike Frysinger
dd37fb2222 main: re-exec self with the current interp
The launcher already raised itself up to use Python 3 on the fly, and
the main.py script uses a plain `python` shebang.  So make sure we use
the active interpreter when re-execing ourselves to avoid falling back
down to Python 2 (which then triggers warnings).

Change-Id: Ic53c07dead3bc9233e4089a0a422f83bb5ac2f91
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/263272
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@digital.ai>
2020-04-28 02:54:50 +00:00
Mike Frysinger
bb930461ce subcmds: stop instantiating at import time
The current subcmds design has singletons in all_commands.  This isn't
exactly unusual, but the fact that our main & help subcommand will then
attach members to the classes before invoking them is.  This makes it
hard to keep track of what members a command has access to, and the two
code paths (main & help) attach different members depending on what APIs
they then invoke.

Lets pull this back a step by storing classes in all_commands and leave
the instantiation step to when they're used.  This doesn't fully clean
up the confusion, but gets us closer.

Change-Id: I6a768ff97fe541e6f3228358dba04ed66c4b070a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/259154
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-03-17 00:08:52 +00:00
Mike Frysinger
d3639c53d5 subcmds: centralize all_commands logic
The branch->branches alias is setup in the main module when that
really belongs in the existing all_commands setup.

For help, rather than monkey patching all_commands to the class,
switch it to use the state directly from the module.  This makes
it a bit more obvious where it's coming from rather than this one
subcommand having a |commands| member added externally to it.

Change-Id: I0200def09bf4774cad8012af0f4ae60ea3089dc0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/259153
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-03-17 00:08:52 +00:00
Mike Frysinger
eea23b44a9 main: improve launcher update messaging wrt system installs
Some users get repo from their distro (e.g. /usr/bin/repo), so the
suggestion to copy over top of it makes people uneasy, if it's even
possible in the first place.

Bug: https://crbug.com/gerrit/12335
Change-Id: I9a0c83d6ba0f466fa8e6d61f674ee13396f9a968
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256893
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-26 23:20:10 +00:00
Mike Frysinger
949bc34267 main/repo: add support for subcommand aliases
This supports [alias] sections with repo subcommands just like git.

Change-Id: Ie9235b5d4449414e6a745814f0110bd6af74ea93
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255833
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2020-02-20 00:53:46 +00:00
Mike Frysinger
8a98efee5c main: fix pager logic after negation cleanup
The pager setting is tristate (where None means "auto"),
so make sure we still handle that setting.

Change-Id: I89fe352572dd15922c61e3bb65ac33f847d01ee0
Test: `repo help upload` triggers the pager
Test: `repo -p help upload` triggers the pager
Test: `repo --no-pager help upload` doesn't trigger the pager
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255852
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-19 08:00:11 +00:00
Mike Frysinger
c58ec4dba1 avoid negative variables
Trying to use booleans with names like "no_xxx" are hard to follow due
to the double negatives.  Invert all of them so we only have positive
meanings to follow.

Change-Id: Ifd37d0368f97034d94aa2cf38db52c723ac0c6ed
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255493
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-19 00:24:43 +00:00
Mike Frysinger
37f28f1b4e main: add python version checking
If an older launcher script is used with newer repo source tree, we
might be issuing python version warnings.  Plus, we want to be able
to roll Python version requirements independently of the launcher.
Add some version checking here too.

Change-Id: Ia35fc821f93c429296bdf5fd578276fef796b649
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255592
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-18 14:38:33 +00:00
David Pursehouse
a46bf7dc2a flake8: Suppress "F821 undefined name" inline for Python 2 names
All of the instances of this are related to Python 2 names that
don't exist in Python 3, and the warnings are raised when running
flake8 on Python 3.

All of these will go away once we completely remove support for
Python 2, so just suppress them inline. We don't globally suppress
the check so that we will still see legitimate errors if/when they
occur in new code.

Change-Id: Iccf955f50abfc9f83b371fc0af6cceb51037456f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255039
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
2020-02-15 04:45:16 +00:00
David Pursehouse
3cda50a41b pyflakes: Fix remaining "E501 line too long" warnings
We increased the max line length to 100 columns which got rid of
the majority of these warnings, but there were still a few lines
that exceeded 100 columns.

Change-Id: Ib3372868ca2297f83073a14f91c8ae3df9d0d0e6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254699
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
2020-02-13 04:54:10 +00:00
David Pursehouse
3c5114cd78 Don't format version to 5 characters in new version message
Change-Id: I6c734170173f77a6fef0678f189e198bdaeec425
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254668
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
2020-02-13 01:27:25 +00:00
David Pursehouse
7838e388ac Replace 'A new repo command' with 'A new version of repo'
Change-Id: I3288f5c963b69d05d113fc039e4b4f22721f1de9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254667
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
2020-02-13 01:27:25 +00:00
David Pursehouse
145e35b805 Fix usage of bare 'except'
flake8 reports:

  E722 do not use bare 'except'

Replace them with 'except Exception' per [1] which says:

  Bare except will catch exceptions you almost certainly don't want
  to catch, including KeyboardInterrupt (the user hitting Ctrl+C) and
  Python-raised errors like SystemExit

  If you don't have a specific exception you're expecting, at least
  except Exception, which is the base type for all "Regular" exceptions.

[1] https://stackoverflow.com/a/54948581

Change-Id: Ic555ea9482645899f5b04040ddb6b24eadbf9062
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254606
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
2020-02-12 06:49:25 +00:00
David Pursehouse
819827a42d Fix blank line issues reported by flake8
- E301 expected 1 blank line
- E302 expected 2 blank lines
- E303 too many blank lines
- E305 expected 2 blank lines after class or function definition
- E306 expected 1 blank line before a nested definition

Fixed automatically with autopep8:

  git ls-files | grep py$ | xargs autopep8 --in-place \
    --select E301,E302,E303,E305,E306

Manually fix issues in project.py caused by misuse of block comments.

Change-Id: Iee840fcaff48aae504ddac9c3e76d2acd484f6a9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254599
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
2020-02-12 06:36:40 +00:00
David Pursehouse
abdf750061 Fix indentation issues reported by flake8
- E121 continuation line under-indented for hanging indent
- E122 continuation line missing indentation or outdented
- E125 continuation line with same indent as next logical line
- E126 continuation line over-indented for hanging indent
- E127 continuation line over-indented for visual indent
- E128 continuation line under-indented for visual indent
- E129 visually indented line with same indent as next logical line
- E131 continuation line unaligned for hanging indent

Fixed automatically with autopep8:

  git ls-files | grep py$ | xargs autopep8 --in-place \
    --select E121,E122,E125,E126,E127,E128,E129,E131

Change-Id: Ifd95fb8e6a1a4d6e9de187b5787d64a6326dd249
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254605
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
2020-02-12 06:36:22 +00:00
David Pursehouse
54a4e6007a Fix various whitespace issues reported by pyflakes
- E201 whitespace after '['
- E202 whitespace before '}'
- E221 multiple spaces before operator
- E222 multiple spaces after operator
- E225 missing whitespace around operator
- E226 missing whitespace around arithmetic operator
- E231 missing whitespace after ','
- E261 at least two spaces before inline comment
- E271 multiple spaces after keyword

Fixed automatically with autopep8:

  git ls-files | grep py$ | xargs autopep8 --in-place \
    --select E201,E202,E221,E222,E225,E226,E231,E261,E271

Change-Id: I367113eb8c847eb460532c7c2f8643f33040308c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254601
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
2020-02-12 06:00:16 +00:00
David Pursehouse
9090e804ab Remove unused imports
flake8 reports:

  F401 'name' imported but unused

Change-Id: Id45d6efa87ddf53f2c4a0f0c4136ea361ab1b746
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254592
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
2020-02-12 05:40:10 +00:00
Mike Frysinger
3285e4b436 main: rework launcher version checking
The code has an ad-hoc check in that it requires the launcher major
version to not be less than the source code version.  We don't really
care about that requirement, and it doesn't fit with our other version
checks.  Rework it so we explicitly declare the min launcher version
that is supported.

We'll start with requiring repo launcher 1.15 which was released back
in 2012.  Hopefully no one has anything older than that, although it's
not clear we work with even newer versions than that :).  But let's be
a little conservative with the first update to this logic.

Bug: https://crbug.com/gerrit/10418
Change-Id: I611d70c60324d313c76874e978b8499a491a5d00
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254278
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2020-02-10 23:20:55 +00:00
Mike Frysinger
7c321f1bf6 repo: include subcommands in --help output
Also point people to `repo help` so it's easier to navigate the tool.

Bug: https://crbug.com/gerrit/12022
Change-Id: Ib3be331a2cef32caa193640bf8d54bd1443fce60
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/247292
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2019-12-05 05:00:21 +00:00
Rashed Abdel-Tawab
2058c63641 Only import imp on py2
imp is deprecatedon py3. It's also not used with py3, so just move it
to the py2 import block

Test: run `repo` command and verify warning is no longer present
Test: verify `repo sync` and `repo upload` function as expected
Change-Id: I9d59403d7819c4a478c9f54cbef114f8a96486a5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/239713
Tested-by: Rashed Abdel-Tawab <rashedabdeltawab@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2019-10-05 04:41:40 +00:00
Mike Frysinger
71b0f312b1 git_command: refactor User-Agent settings
Convert the RepoUserAgent function into a UserAgent class.  This
makes it cleaner to hold internal state, and will make it easier
to add a separate git User-Agent, although we don't do it here.

We make the RepoSourceVersion independent of GitCommand so that
it can be called by the class (later).

Bug: https://crbug.com/gerrit/11144
Change-Id: Iab4e1f974b8733a36b243b2d03f5085a96effa19
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/239232
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-10-01 05:40:28 +00:00
Mike Frysinger
369814b4a7 move UserAgent to git_command for wider user
We can't import the main module, so move the UserAgent helper out of
it and into the git_command module so it can be used in more places.

Bug: https://crbug.com/gerrit/11144
Change-Id: I8093c8a20bd1dc7d612d0e2a85180341817c0d86
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/231057
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2019-10-01 05:39:27 +00:00
Mike Frysinger
ae6cb08ae5 split out cli validation from execution
A common pattern in our subcommands is to verify the arguments &
options before executing things.  For some subcommands, that check
stage is quite long which makes the execution function even bigger.
Lets split that logic out of the execute phase so it's easier to
manage these.

This is most noticeable in the sync subcommand whose Execute func
is quite large, and the option checking makes up ~15% of it.

The manifest command's Execute can be simplified significantly as
the optparse configuration always sets output_file to a string.

Change-Id: I7097847ff040e831345e63de6b467ee17609990e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/234834
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-08-28 03:54:11 +00:00
Mike Frysinger
3fc157285c add a --trace-python option
This can help debug issues by tracing all the repo python code with
the standard trace module.

Change-Id: Ibb7f4496ab6c7f9e130238ddf3a07c831952697a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/234833
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2019-08-27 18:44:17 +00:00
Mike Frysinger
8a11f6f24c rename local trace module
There is a standard Python "trace" module, so having a local trace.py
prevents us being able to import that.  Rename the module to avoid.

Change-Id: I23e29ec95a2204bb168a641323d05e76968d9b57
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/234832
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-08-27 07:08:52 +00:00
Mike Frysinger
87fb5a1894 repo/main: add module docstrings
This should help people get some bearings in the codebase.

Change-Id: I951238fe617a3ecb04a47ead3809ec72c8fbf5a1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/231232
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-07-12 17:23:53 +00:00