Commit Graph

1687 Commits

Author SHA1 Message Date
David Pursehouse
d1ebc89a08 Merge changes from topic "windows-support"
* changes:
  Port os.rename calls to work on Windows
  Workaround shutil.rmtree limitation on Windows
  Add support for creating symbolic links on Windows
  Make "git command" and "forall" work on Windows
2017-08-30 10:24:03 +00:00
Christian Koestlin
2ec2a5d64c Fixed upload to remotes with the url ssh://hostname
Change-Id: I1d0dd4d3f90eac45205f6f4ca98a29b0babdbc3f
2017-08-29 20:16:08 +00:00
Zac Livingston
9ead97bb51 When starting a branch, do not use a tag or change value for branch.merge
When starting a branch, branch.merge is set to project revision unless
the revision is a SHA1. In that case, branch.merge is set to dest_branch
if defined or manifest default revision otherwise. This special handling
allows repo upload to work when the project revision is a SHA1.

Extend the special handling to also happen when the project revision
is a tag value or a change value so that repo upload will work in those
case as well.

Change-Id: Iff81ece40e770cd02535e80dcb023564d42dcf47
2017-08-25 09:10:29 +09:00
Zac Livingston
e43322625a Print a message when fetching is skipped for an immutable ref
The output indicates that fetching happens even when it is skipped.

To avoid confusion, print a message when fetching is skipped for
an immutable ref so that the user knows when and why a fetch is skipped.

Change-Id: Id6e4812cebc5e57d379feb76a9d034af0b93043b
2017-08-25 00:00:02 +00:00
Łukasz Gardoń
bed59cec5e Add option '--no-cert-checks' for 'upload' sub command.
This option allow to bypass verification ssl certification while
establishing connection with Gerrit to upload review.

Change-Id: If2e15f5a273c18a700eb5093ca8a4d5a4cbf80cd
2017-08-23 14:06:14 +02:00
Jonathan Nieder
c94d6eb902 Revert "Migrate git-repo to create private changes rather than drafts"
This reverts commit d88f53e2b9. I merged
it too hastily without paying enough attention to compatibility with
released Gerrit versions.

Change-Id: I4028d4737df1255f11e217da183a19a010597d5b
2017-08-08 18:34:53 +00:00
Changcheng Xiao
d88f53e2b9 Migrate git-repo to create private changes rather than drafts
Considering that some users might expect changes created with
'-d' option are not public. Private changes may be a better
choice here than work-in-progress changes.

Change-Id: I46a8fb9ae38beb41cf96d6abe82bea6db2439669
2017-08-07 15:08:18 +02:00
Changcheng Xiao
87984c6db4 Add options for git-repo to support private and wip changes
This change adds options for git-repo tool to support private
changes and work-in-progress changes.

Change-Id: I343491f5949f06f1580d53f9cc0dee2dca09130f
2017-08-07 15:02:39 +02:00
David Pursehouse
ffc1401327 Merge "download: try to choose . as default project if none" 2017-08-02 07:19:45 +00:00
David Pursehouse
8a6eeed7f5 Merge "Always print percentage when syncing quietly" 2017-08-02 07:01:08 +00:00
Tim Schumacher
7be072efa6 Always print percentage when syncing quietly
Change-Id: I574396e63520781067ed1e991c41caf7640e5731
2017-07-15 16:44:55 +00:00
Nicolas Cornu
7482a96443 download: try to choose . as default project if none
Change-Id: I28b5e3be5f3c9a4c077af87d6a3e0cc3b96a1b9d
2017-07-12 10:15:06 +02:00
David Pursehouse
3bcd30545e Fix "list comprehension redefines 'x'" warnings from pyflakes
$ git ls-files | grep py$ | xargs pyflakes
  subcmds/stage.py:101: list comprehension redefines 'p' from line 63
  subcmds/sync.py:784: list comprehension redefines 'p' from line 664
  subcmds/upload.py:467: list comprehension redefines 'avail' from line 454

Change-Id: Ia65d1a72ed185ab3357e1a91ed4450c719e75a7c
2017-07-10 23:26:04 +00:00
Martin Kelly
224a31a765 init: add missing submodule arg
The submodule argument to Sync_LocalHalf was missing in
MetaBranchSwitch, causing submodules not to get synced when the
-b/--manifest-branch argument to init is used.

Change-Id: Ie86d271abac2020725770be36ead83be3326e64b
Signed-off-by: Martin Kelly <mkelly@xevo.com>
2017-07-10 14:50:52 -07:00
Nicolas Cornu
b54343d9fd Tell the user if it will upload a draft
Change-Id: Ie004ec9d61603f3f618c47597947b82c59f2839c
2017-07-10 13:20:01 +00:00
David Pursehouse
259f16520a Merge "Add a newline after "Fetching projects" progress output" 2017-06-28 04:24:03 +00:00
Nicolas Cornu
8419ab22d6 sync: Continue job if some fetchs failed but force-broken is set
With --force-broken it continue to fetch other projects but nothing
is added in directory because it abort some lines later.

Change-Id: I32c4a4619b3028893dc4f98e8d4e5bc5c09adb27
2017-06-16 12:23:26 +02:00
Tim Schumacher
913327f10c
Add a newline after "Fetching projects" progress output
Output before change:

    Fetching project platform/packages/providers/UserDictionaryProvider
    Fetching projects:  66% (773/1171)  Fetching project platform/external/regex-re2
    Fetching project device/generic/mini-emulator-x86_64

Output after change:

    Fetching project platform/packages/providers/UserDictionaryProvider
    Fetching projects:  66% (773/1171)
    Fetching project platform/external/regex-re2
    Fetching project device/generic/mini-emulator-x86_64

Change-Id: I4da84da58316c69294e4da2792f83885dc942701
2017-06-13 13:03:39 +02:00
Renaud Paquay
ad1abcb556 Port os.rename calls to work on Windows
os.rename fails on Windows if the destination exists, so replace
os.rename to platform_utils.rename which handles the platform
differences.

Change-Id: I15a86f10f65eedee5b003b80f88a0c28a3e1aa48
2017-05-29 19:33:07 +09:00
Renaud Paquay
a65adf74f9 Workaround shutil.rmtree limitation on Windows
By default, shutil.rmtree raises an exception when deleting readonly
files on Windows.

Replace all shutil.rmtree with platform_utils.rmtree, which adds an
error handler to make files read-write when they can't be deleted.

Change-Id: I9cfea9a7b3703fb16a82cf69331540c2c179ed53
2017-05-29 19:32:31 +09:00
Renaud Paquay
d5cec5e752 Add support for creating symbolic links on Windows
Replace all calls to os.symlink with platform_utils.symlink.

The Windows implementation calls into the CreateSymbolicLinkW Win32
API, as os.symlink is not supported.

Separate the Win32 API definitions into a separate module
platform_utils_win32 for clarity.

Change-Id: I0714c598664c2df93383734e609d948692c17ec5
2017-05-29 19:30:34 +09:00
Renaud Paquay
2e70291162 Make "git command" and "forall" work on Windows
Python on Windows does not support non blocking file operations.
To workaround this issue, we instead use Threads and a Queue to
simulate non-blocking calls. This is happens only when running
with the native Windows version of Python, meaning Linux and Cygwin
are not affected by this change.

Change-Id: I4ce23827b096c5138f67a85c721f58a12279bb6f
2017-05-29 19:29:30 +09:00
Renaud Paquay
35d22217a5 Ensure repo waits for child process to terminate
See http://stackoverflow.com/questions/7004687/os-exec-on-windows:

execv on Windows does not behave as on Linux, i.e. a new process is
spawned and the parent process terminates right away, which makes the
shell prompt come back too soon.

Change-Id: I1f8d23208765988629f081e9b949c67cf71c08ae
2017-05-29 13:56:18 +09:00
David Pursehouse
a24671f661 Merge "init: allow relative path on --reference argument" 2017-05-29 04:54:12 +00:00
David Riley
e0684addee sync: Add support to dump a JSON event log of all sync events.
Change-Id: Id4852968ac1b2bf0093007cf2e5ca951ddab8b3b
2017-05-29 13:39:54 +09:00
Renaud Paquay
fef9f21b28 Fix misplaced file separator string.replace call
Project names are stored as path using the '/' file separator, and
stored in a dictionary as keys.

Change-Id: Ide40dfe840958ac0d46caae5f77f1a49d71c9d90
2017-05-28 21:18:13 +09:00
Renaud Paquay
6a470be220 Use OS file separator
Change-Id: I46b15bc1c1b4f2300a6fd98fe16c755da4910e7a
2017-05-28 21:16:15 +09:00
David Pursehouse
169d8ae93c Merge "pre-auto-gc: Add support for Windows" 2017-05-28 00:37:59 +00:00
YOUNG HO CHA
c79d3b8fd1 init: allow relative path on --reference argument
Change-Id: I41d6be6bc035fdddb5a27c072994439986d58d58
Signed-off-by: YOUNG HO CHA <ganadist@gmail.com>
2017-05-28 00:33:25 +00:00
David Riley
aa90021fbc Set result if sys.exit() is called by subcommand.
Allows the finally branch to make sure of the return code.

Change-Id: I7a796da5b60269cbd71aad953f1b9bb762b8eef8
2017-05-27 13:32:00 +09:00
Jiri tyr
fddfa6fbac Adding include element into the top-level element
The documentation of the XML file format contains DTD which contains
definition of all allowed elements and attributes. The "include" element
is defined but it's not referenced from the top-level "manifest"
element.

This patch is adding the "include" element into the list of elements of
the top-level "manifest" element.

Change-Id: I33beb8ef2846bbf42ffd42e6ae6888828566d604
2017-05-27 04:26:15 +00:00
David Pursehouse
997a92bd58 Merge "Add option REPO_IGNORE_SSH_INFO to ignore ssh_info" 2017-05-27 04:25:39 +00:00
David Pursehouse
fbcbcabe98 Merge "init: add --submodules to sync manifest submodules" 2017-05-27 04:24:58 +00:00
Timo Lotterbach
eec726c6d8 Add option REPO_IGNORE_SSH_INFO to ignore ssh_info
This is required for setups, where Gerrit access using ssh is only available
for some networks.
For network without ssh access, repo will get ssh_info from Gerrit and
use ssh for communications - which will fail. To support this setup
we need to have an option to ignore the ssh_info provided by Gerrit and
use http(s).

Using git insteadOf as alternative results in the inability to add
reviewers using "repo upload --re=...", since the syntax of adding
reviewers differs for ssh and https. repo is assuming an ssh
connection and uses "git push --receive-pack=...", which will fail
since git silently uses https for push operation. repo must be aware
that https is used so it uses "git push remote ...:refs/for/...%r=..."
for upload.

Change-Id: Idd83baef0fb26ffcc9ac65e204b68d323ce177a1
2017-05-26 15:11:11 +02:00
David Pursehouse
666debc518 gitc_delete: Remove unused imports
Change-Id: I672189ba99e18dca3956e2396c921d1ef0ca2ddd
2017-05-26 21:53:34 +09:00
David Pursehouse
c354a9b922 abandon: fix usage of undefined variable
As reported by pyflakes:

  subcmds/abandon.py:84: undefined name 'p'

The name of the variable should be 'proj'.

Change-Id: Ic09eb92e8db6b510e99efce010bd0bb094d7cbfe
2017-05-26 21:52:23 +09:00
David Pursehouse
06848b2415 Update .mailmap
Change-Id: Icabc7fb2161b661c2df9290a1ca6f75b9b1b8e1b
2017-05-26 21:44:57 +09:00
Martin Kelly
e4e94d26ae init: add --submodules to sync manifest submodules
repo sync can sync submodules via the --fetch-submodules option.
However, if the manifest repo has submodules, those will not be synced.
Having submodules in the manifest repo -- while not commonly done -- can
be useful for inheriting a manifest from another project using <include>
and layering changes on top of it.  In this way, you can avoid having to
deal with merge conflicts between your own manifests and the other
project's manifests (for example, if you're managing an Android fork).

Add a --submodule option to init that automatically syncs the submodules
in the manifest repo whenever the manifest repo changes.

Change-Id: I45d34f04517774c1462d7f233f482d1d81a332a8
Signed-off-by: Martin Kelly <mkelly@xevo.com>
2017-05-23 16:51:31 -07:00
Sebastian Schuberth
c9439facdd pre-auto-gc: Add support for Windows
Previously, this would always have exited with 1 on Windows, causing "git
gc --auto" to abort. Fix this by adding support for Windows.

Change-Id: Ie519b366a11b6b18b2d465e892e738de3f4bbc99
2017-05-03 11:26:21 +00:00
heping
3d7bbc9edf project.py: fix performance issue with --reference when the mirrored repository has many refs
Change-Id: Id0183903597f872eee80ca32a8050125b187a3d4
2017-04-12 20:04:47 +08:00
Marc Herbert
ffb4b89099 sync.py: report the remote URL on fatal git remote errors
repo can be configured to download from any number of remote git repos.
However when one fails repo doesn't report which one. Example:
Fatal: remote error: Daily ls-remote rate limit exceeded for IP xx.xx.xx.xx

TEST=repo init -q -u https://chromium.googlesource.com/chromiumos/manifest.git
  # Apply patch in ./.repo/repo/
  # Simulate a git remote error:
  sed -i -e 's#chromiumos/docs#chromiumos/XXdocs#' .repo/manifests/full.xml
  repo sync --quiet --force-sync docs
  # error message now shows the remote URL

Optional test tip: reduce the time.sleep(random(...)) in ./.repo/repo/project.py

Change-Id: I4509383b6a43a8e66064778e8ed612d8a735c8b6
2017-04-04 22:10:34 -07:00
Mike Frysinger
04071c1c72 manifest-format: fix EMPTY keyword usage
The keyword EMPTY doesn't use parens.

BUG=git-repo:140

Change-Id: I7cd28a09c401520a72e5c244a77d9d70385f1b61
2016-12-28 16:07:16 -05:00
Jonathan Nieder
7de8c5db78 Merge "init: Add no-tags and current branch options" 2016-12-09 02:33:42 +00:00
David Pursehouse
bb9c42cf1d Merge "Fix removing broken symlink in reference dir" 2016-12-06 07:51:01 +00:00
Naseer Ahmed
f4dda9a1be init: Add no-tags and current branch options
This avoids fetching tags and branches for huge manifests

Change-Id: I19c9724d75364440b881b297d42b906f541f73ff
2016-12-01 19:03:41 -05:00
David Pursehouse
b881d227f3 Merge "Add a check and more output to protect against invalid REPO_URLs" 2016-10-29 07:28:35 +00:00
Dan Willemsen
8e2d1d521e Merge "Fix checkout error when depth passed to repo init and revision is a sha1" 2016-10-28 19:30:45 +00:00
Sebastian Schuberth
27226e742d Add a check and more output to protect against invalid REPO_URLs
If you don't know that the url to git-repo itself can be overridden via
REPO_URL, it's hard to debug cases where REPO_URL is accidentally set to
another repository, e.g. inside a Jenkins CI job. What makes is even
harder is that the ".repo/repo" directory gets silently removed in such
cases as verifications fails, which makes it impossible to look at the
cloned files to understand the problem.

To better protect against such an issue, warn if the cloned git-repo
repository does not contain a top-level "repo" file, and state that the
".repo/repo" directory will be removed in case of a clone failure.

Change-Id: I697b4999205a5967910c0237772ccaada01e74d4
2016-10-28 14:43:02 +02:00
Aymen Bouaziz
6c5944606a Fix checkout error when depth passed to repo init and revision is a sha1
Currently, if direct fetch of a sha1 is not supported by git server and
depth option is used, we fallback on syncing the upstream branch by
ignoring depth option.

This fallback doesn't work in next 2 cases:
(1) upstream attribute is not specified in manifest
(2) depth option is passed to repo init command
    (not with clone-depth attribute in manifest)

This commit do the following:
- fixes (1) by updating condition used to apply fallback
  first we retry with depth set to None, then by syncing all branches
- fixes (2) by passing depth as argument of _RemoteFetch() method
  thus, its value is not set again to depth value passed to repo init
  command when applying fallback

Change-Id: Ifd6fffafc49ba229df624b0d7b64c83d47619d17
2016-10-28 14:29:57 +02:00
David Pursehouse
ae81c964b6 Merge "implement optional '--all' in the abandon command" 2016-10-28 07:52:31 +00:00