Commit Graph

409 Commits

Author SHA1 Message Date
Mike Frysinger
d5c306b404 rebase: pull out project-independent settings from the for loop
This makes the code a bit easier to read by doing all the project
independent settings first instead of repeating it for every for
loop iteration.

Change-Id: I4ff21296e444627beba2f4b86561069f5e9a0d73
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/233554
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-08-08 17:41:40 +00:00
Mike Frysinger
a850ca2712 rebase/sync: use exit(1) for errors instead of exit(-1)
Callers don't actually see -1 (they'll usually see 255, but the exact
answer here is complicated).  Just switch to 1 as that's the standard
value tools use to indicate an error.

Change-Id: Ib712db1924bc3e5f7920bafd7bb5fb61f3bda44f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/233553
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-08-08 05:07:31 +00:00
Mike Frysinger
a34186e481 sync: fix handling of -f and local checkouts
The partial clone rework (commit 745be2ede1
"Add support for partial clone") changed the behavior when a single repo
hit a failure: it would always call sys.exit() immediately.  This isn't
even necessary as we already pass down an error event object which the
workers set and the parent checks.  Just delete the exit entirely.

Change-Id: Id72d8642aefa2bde24e1a438dbe102c3e3cabf48
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/233552
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-08-08 02:54:39 +00:00
Xin Li
745be2ede1 Add support for partial clone.
A new option, --partial-clone is added to 'repo init' which tells repo
to utilize git's partial clone functionality, which reduces disk and
bandwidth usage when downloading by omitting blob downloads initially.
Different from restricting clone-depth, the user will have full access
to change history, etc., as the objects are downloaded on demand.

Change-Id: I60326744875eac16521a007bd7d5481112a98749
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/229532
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Xin Li <delphij@google.com>
2019-07-16 00:23:16 +00:00
Mike Frysinger
ab85fe7c53 use print() instead of sys.stdout.write()
We're relying on sys.stdout.write() to flush its buffer which isn't
guaranteed, and is not the case in Python 3.  Change to use print()
everywhere to be standard, and utilize the end= keyword to get the
EOL semantics we need.

We can't use print's flush= keyword as that's only in Python 3.
Leave behind a TODO to clean it up when we can drop Python 2.

Bug: https://crbug.com/gerrit/10418
Change-Id: I562128c7f1e6d154f4a6ecdf33a70fa2811dc2af
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/230392
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2019-07-11 06:26:40 +00:00
Mike Frysinger
ca540aed19 git_command: drop custom version helper
Since ParseGitVersion can call `git --version` automatically, we don't
need this duplicate version() helper anymore.  The only other user is
the `repo version` code, so convert that to version_tuple().full.

Bug: https://crbug.com/gerrit/11144
Change-Id: I9d77822fc39f4ba28884d9183359169cabf5f17d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/231055
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-07-11 01:30:18 +00:00
Solomon Kinard
490e16385d Remove double forall from "repo help forall" output
%prog represents the full subcommand ("repo" + subcommand name), not a
Windows-style environment variable for "repo". The current help output
shows

 repo forall% forall ...

Correct the variable usage so it shows "repo forall ..." instead.

Change-Id: I1fea55572428cc922ddf24ace1168a3d8f82dad0
2019-07-08 22:42:38 +00:00
Mike Frysinger
0c0e934b69 sync: use integer division with job counts
Neither of the fields here expect floats so make sure we use integer
division when calculating things.

Bug: https://crbug.com/gerrit/10418
Change-Id: Ibda068b16a7bba7ff3efba442c4bbff4415caa6e
2019-06-14 14:47:01 +00:00
Mike Frysinger
9e71842fbf status: import print_function
This module uses print() so make sure we import the print function.
It doesn't really impact the current code due to the simple way it
is calling print, but we should be sane to avoid future issues.

Change-Id: I0b15344678c1dcb71207faa333c239b3fced1d62
2019-06-14 14:13:23 +00:00
Mike Frysinger
f601376e13 set default file encoding to utf-8
There's no reason to support any other encoding in these files.
This only affects the files themselves and not streams they open.

Bug: https://crbug.com/gerrit/10418
Change-Id: I053cb40cd3666ce5c8a0689b9dd938f24ca765bf
2019-06-13 14:30:52 +00:00
Mike Frysinger
31067c0ac5 tweak raise/dict syntax for Python 3 compat
Use the `raise` statement directly.

Switch to using .items() instead of .iteritems().  Python 3 doesn't
have .iteritems() as .items() is a generator, and these are small
enough that the Python 2 overhead should be negligible.

We have to run .keys() through list() in a few places as Python 3
uses a generator and we sometimes want to iterate more than once.
That's why we don't change all .keys() or .items() calls -- most
are in places where generators are fine.

Bug: https://crbug.com/gerrit/10418
Change-Id: I469899d9b77ffd77ccabb831bc4b217407fefe6f
2019-06-13 13:39:25 +00:00
Basil Gello
c745350ab9 diffmanifests: honor user-supplied manifest paths
The current implementation ignores the user-specified paths to
manifests.  if the "repo diffmanifests" is invoked with absolute
file paths for one or both manifests, the command fails with message:

fatal: duplicate path ... in /tmp/manifest-old.xml

Also the current implementation fails to expand the absolute path to
manifest files if "repo diffmanifests" is invoked with relative
paths, i.e "repo diffmanifests manifest-old.xml manifest-new.xml".

fatal: manifest manifest-old.xml not found

This commit fixes the first issue by disabling the local manifest
discovery for diffmanifests command, and the second issue by
expanding paths to manifests within "diffmanifests" sub-command.

Test: repo manifest --revision-as-HEAD -o /tmp/manifest-old.xml
      repo sync
      repo manifest --revision-as-HEAD -o /tmp/manifest-new.xml
      repo diffmanifests /tmp/manifest-old.xml /tmp/manifest-new.xml
Change-Id: Ia125d769bfbea75adb9aba81abbd8c636f2168d4
Signed-off-by: Vasyl Gello <vasek.gello@gmail.com>
2019-06-06 07:36:10 +00:00
Nasser Grainawi
d92464e8ef Honor --depth during repo init
If a user is asking for a shallow clone of the repos, they probably
expect a shallow clone of the manifest repo too. For very large
manifest repos, this can be a huge space and time savings. For one real-world
repo, a 'repo init --no-tags --current-branch' used 350MB of disk space and
took 7 minutes. Adding --depth 1 and this change reduced it to 10MB and 2.5
minutes.

Change-Id: I6fa662e174e623ede8861efc862ce26d65d4958d
2019-05-21 10:47:21 -06:00
Sebastian Schuberth
266f74c888 info: Use the non-formatting printer for headtext
If "repo init" was run in a path containing "%", "repo info" would fail
printing the path with

    File ".repo/repo/color.py", line 173, in f
      return fmt % args
    TypeError: not enough arguments for format string

as the "%" in the path name is interpreted as the start of a formatting
specifier. Avoid that by using the non-formatting printer for headtext
which does not require any formatting so there is no need to try to
expand "%" sequences.

Change-Id: Ie193b912191fe7cdabdce5c97bb100f0714f6e76
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
2019-04-17 19:46:05 +02:00
Kuang-che Wu
0d9b16d1d8 sync: deleted unused repos in reversed order (children before parent)
Bug: chromium:950002
Test: repo sync chromeos using release manifest file
Change-Id: I613df6a1973eb36acd806a703e72f5172554bcc7
2019-04-06 00:49:47 +08:00
Oleksii Okolielov
d3c0f5914f sync: Add option '--force-remove-dirty'
Forcefully remove dirty projects if option '--force-remove-dirty' is given.
The '--force-remove-dirty' option can be used to remove previously used
projects with uncommitted changes. WARNING: This may cause data to be lost
since uncommitted changes may be removed with projects that no longer exist
in the manifest.

Change-Id: I844a6e943ded522fdc7b1b942c0a1269768054bc
2019-03-11 13:06:49 -04:00
Ereth McKnight-MacNeil
12ee5446e9 init: Remove -c short option for --current-branch
This option conflicts with the gitc-init -c short option.

Bug: https://crbug.com/gerrit/10200
Change-Id: I06f37564429ca0bd4c0bbea6066daae4f663c838
2018-12-20 19:55:02 +00:00
Vadim Bendebury
bd8f658823 Add option for git-repo to support 'silent' uploads
When --ne/--no-emails is added to 'repo upload' command line, gerrit
server will not generate notification emails.

project.py:Project.UploadForReview method is modified to accept a
string recognizable by gerrit to indicate different sets of destination
email addressees, but the upload command line allows only one option -
disable sending emails completely.

Default repo upload behavior is not being changed.

TEST=tried in the Chrome OS repo, observed that patches uploaded with
     --ne or --no-emails indeed do not trigger any emails, while
     patches uploaded without these command line options still trigger
     email notifications.

Change-Id: I0301edec984907aedac277d883bd0e6d3099aedc
2018-11-05 14:01:05 -08:00
David Pursehouse
36391bf5ca Merge "init: --dissociate option to copy objects borrowed with --reference" 2018-10-28 23:30:50 +00:00
Renaud Paquay
bed8b62345 Add support for long paths
* Add more file i/o wrappers in platform_utils to allow using
  long paths (length > MAX_PATH) on Windows.

* Paths using the long path syntax ("\\?\" prefix) should never
  escape the platform_utils API surface area, so that this
  specific syntax is not visible to the rest of the repo code base.

* Forward many calls from os.xxx to platform_utils.xxx in various place
  to ensure long paths support, specifically when repo decides to delete
  obsolete directories.

* There are more places that need to be converted to support long paths,
  this commit is an initial effort to unblock a few common use cases.

* Also, fix remove function to handle directory symlinks

Change-Id: If82ccc408e516e96ff7260be25f8fd2fe3f9571a
2018-10-22 08:16:35 -07:00
Nikolai Merinov
09f0abb0ef init: --dissociate option to copy objects borrowed with --reference
"repo init --reference" has two purposes: to decrease bandwidth used
at clone time, and to decrease disk usage afterward, by using the
reference repositories as an alternate store of objects even after
the clone. The downside is that it makes the borrowing repositories
dependent on the reference repositories, so it is easy to end up
with missing objects by mistake after a cleanup operation like "git
gc".

To prevent that, v2.3.0-rc0~30^2 (clone: --dissociate option to mark
that reference is only temporary, 2014-10-14), "git clone" gained a
--dissociate option that makes --reference reuse objects from the
reference repository at clone time but copy them over instead of
using the reference as an alternate. This is more straightforward to
use than plain --reference, at the cost of higher disk usage.

Introduce a --dissociate to "repo init" that brings the same benefits
to repo. The option is simply passed on to "git clone".

Change-Id: Ib50a549eb71e0a2b3e234aea57537923962a80d4
2018-10-19 23:51:23 +05:00
Jonathan Nieder
b3133a3164 Merge "update markdown/help header format" 2018-10-10 05:54:59 +00:00
Mike Frysinger
3b24e7b557 update homepage URIs
Change-Id: I482a72bf296978625b1e82ef580b0e0d4d57ff25
2018-10-10 01:35:58 -04:00
Mike Frysinger
b8f7bb04d0 update markdown/help header format
Since gitiles recommends using # headers over ---/=== underlines,
change the manifest-format.md over and all our help texts.

Change-Id: I96391d41fba769e9f26870d497cf7cf01c8d8ab3
2018-10-10 01:28:43 -04:00
Mike Frysinger
3891b7519d manifest-format: convert to markdown
The gitiles system doesn't render .txt files, so convert this to .md
for better display online.

Change-Id: Ie12e46daf008dd8c97ae2ffd21fb68bd948fe625
2018-10-05 19:32:51 -04:00
David Pursehouse
65b0ba5aa0 Remove unused pylint suppressions
pylint is not used since bb5b1a0. The pyflakes cleanup mentioned in that
commit has not been done, but given that this project is no longer being
actively developed I don't think it's worth spending time doing it.

Leaving the pylint suppressions causes confusion because it leads people
to think that we are still using pylint.

Change-Id: If7d9f280a0f408c780f15915ffdb80579ae21f69
2018-07-24 22:20:08 +08:00
Akshay Verma
0f2e45a3a6 Pass refs to ls-remote
This will fix the issue of parsing large output locally

Change-Id: I9a5cf1238147a02c92a3fca53eab9bd57f9d16b4
2018-03-24 13:00:08 +05:30
Akshay Verma
cf7c0834cf Download latest patch when no patch is specified
When someone does "repo download -c <project> <change>"
without specifying a patch number, by default patch 1 is
downloaded. An alternative is to look for the latest patch
and download the same when no explicit patch is given.
This commit does the same by identifying the latest patch
using "git ls-remote".

Change-Id: Ia5fa7364415f53a3d9436df4643e38f3c90ded58
2018-03-17 16:29:23 +05:30
Samuel Holland
5f0e57d2ca init: Remove string concat in no-op os.path.join
This also fixes a line length warning.

Change-Id: I9c1ab65f83a35581dd657a707c7bc3c69db2b1dc
2018-01-22 11:00:24 -06:00
Samuel Holland
baa0009355 Support relative paths in --reference
Put the correctly-expanded relative paths in objects/info/alternates.
From gitrepository-layout(5), this path should be "relative to the
object database, not to the repository".

Change-Id: I7b2027ae23cf7d367b80f5a187603c4cbacdb2de
2018-01-22 10:57:29 -06:00
Masaya Suzuki
305a2d029f Support --push-option in upload subcommand
Change-Id: I44836f8c66ded5a96cbf5431912e027e681f6529
2017-11-13 15:48:49 -08:00
David Pursehouse
f46902a800 forall: Clarify expansion of REPO_ environment values with -c
If a user executes:

  repo forall -c echo $REPO_PROJECT

then $REPO_NAME is expanded by the user's shell first, and passed
as $1 to the shell that executes echo. This will either result in
no output, or output of whatever REPO_NAME is set to in the user's
shell. Either way, this is an unexpected result.

The correct way to do it is:

  repo forall -c 'echo $REPO_PROJECT'

such that $REPO_NAME is passed in to the shell literally, and then
expanded to the value set in the environment that was passed to
the shell.

Update the documentation to make this clearer.

Change-Id: I713caee914172ad8d8f0fafacd27026502436f0d
2017-10-31 13:07:55 +09:00
Renaud Paquay
010fed7711 Replace all os.remove calls
os.remove raises an exception when deleting read-only files on
Windows. Replace all calls with calls to platform_utils.remove,
which deals with deals with that issue.

Change-Id: I4dc9e0c9a36b4238880520c69f5075eca40f3e66
2017-08-31 13:49:36 -07:00
Renaud Paquay
227ad2ef42 Implement islink, readlink and realpath using Win32 api
Change-Id: I18452cbb32d24db73601ad10485dbe6bb278731c
2017-08-31 13:49:01 -07:00
David Pursehouse
2c57d619bc Merge "Add option '--no-cert-checks' for 'upload' sub command." 2017-08-30 15:11:10 +00:00
David Pursehouse
d1ebc89a08 Merge changes from topic "windows-support"
* changes:
  Port os.rename calls to work on Windows
  Workaround shutil.rmtree limitation on Windows
  Add support for creating symbolic links on Windows
  Make "git command" and "forall" work on Windows
2017-08-30 10:24:03 +00:00
Zac Livingston
9ead97bb51 When starting a branch, do not use a tag or change value for branch.merge
When starting a branch, branch.merge is set to project revision unless
the revision is a SHA1. In that case, branch.merge is set to dest_branch
if defined or manifest default revision otherwise. This special handling
allows repo upload to work when the project revision is a SHA1.

Extend the special handling to also happen when the project revision
is a tag value or a change value so that repo upload will work in those
case as well.

Change-Id: Iff81ece40e770cd02535e80dcb023564d42dcf47
2017-08-25 09:10:29 +09:00
Łukasz Gardoń
bed59cec5e Add option '--no-cert-checks' for 'upload' sub command.
This option allow to bypass verification ssl certification while
establishing connection with Gerrit to upload review.

Change-Id: If2e15f5a273c18a700eb5093ca8a4d5a4cbf80cd
2017-08-23 14:06:14 +02:00
Jonathan Nieder
c94d6eb902 Revert "Migrate git-repo to create private changes rather than drafts"
This reverts commit d88f53e2b9. I merged
it too hastily without paying enough attention to compatibility with
released Gerrit versions.

Change-Id: I4028d4737df1255f11e217da183a19a010597d5b
2017-08-08 18:34:53 +00:00
Changcheng Xiao
d88f53e2b9 Migrate git-repo to create private changes rather than drafts
Considering that some users might expect changes created with
'-d' option are not public. Private changes may be a better
choice here than work-in-progress changes.

Change-Id: I46a8fb9ae38beb41cf96d6abe82bea6db2439669
2017-08-07 15:08:18 +02:00
Changcheng Xiao
87984c6db4 Add options for git-repo to support private and wip changes
This change adds options for git-repo tool to support private
changes and work-in-progress changes.

Change-Id: I343491f5949f06f1580d53f9cc0dee2dca09130f
2017-08-07 15:02:39 +02:00
David Pursehouse
ffc1401327 Merge "download: try to choose . as default project if none" 2017-08-02 07:19:45 +00:00
David Pursehouse
8a6eeed7f5 Merge "Always print percentage when syncing quietly" 2017-08-02 07:01:08 +00:00
Tim Schumacher
7be072efa6 Always print percentage when syncing quietly
Change-Id: I574396e63520781067ed1e991c41caf7640e5731
2017-07-15 16:44:55 +00:00
Nicolas Cornu
7482a96443 download: try to choose . as default project if none
Change-Id: I28b5e3be5f3c9a4c077af87d6a3e0cc3b96a1b9d
2017-07-12 10:15:06 +02:00
David Pursehouse
3bcd30545e Fix "list comprehension redefines 'x'" warnings from pyflakes
$ git ls-files | grep py$ | xargs pyflakes
  subcmds/stage.py:101: list comprehension redefines 'p' from line 63
  subcmds/sync.py:784: list comprehension redefines 'p' from line 664
  subcmds/upload.py:467: list comprehension redefines 'avail' from line 454

Change-Id: Ia65d1a72ed185ab3357e1a91ed4450c719e75a7c
2017-07-10 23:26:04 +00:00
Martin Kelly
224a31a765 init: add missing submodule arg
The submodule argument to Sync_LocalHalf was missing in
MetaBranchSwitch, causing submodules not to get synced when the
-b/--manifest-branch argument to init is used.

Change-Id: Ie86d271abac2020725770be36ead83be3326e64b
Signed-off-by: Martin Kelly <mkelly@xevo.com>
2017-07-10 14:50:52 -07:00
Nicolas Cornu
b54343d9fd Tell the user if it will upload a draft
Change-Id: Ie004ec9d61603f3f618c47597947b82c59f2839c
2017-07-10 13:20:01 +00:00
David Pursehouse
259f16520a Merge "Add a newline after "Fetching projects" progress output" 2017-06-28 04:24:03 +00:00
Nicolas Cornu
8419ab22d6 sync: Continue job if some fetchs failed but force-broken is set
With --force-broken it continue to fetch other projects but nothing
is added in directory because it abort some lines later.

Change-Id: I32c4a4619b3028893dc4f98e8d4e5bc5c09adb27
2017-06-16 12:23:26 +02:00