Commit Graph

370 Commits

Author SHA1 Message Date
Łukasz Gardoń
bed59cec5e Add option '--no-cert-checks' for 'upload' sub command.
This option allow to bypass verification ssl certification while
establishing connection with Gerrit to upload review.

Change-Id: If2e15f5a273c18a700eb5093ca8a4d5a4cbf80cd
2017-08-23 14:06:14 +02:00
Jonathan Nieder
c94d6eb902 Revert "Migrate git-repo to create private changes rather than drafts"
This reverts commit d88f53e2b9. I merged
it too hastily without paying enough attention to compatibility with
released Gerrit versions.

Change-Id: I4028d4737df1255f11e217da183a19a010597d5b
2017-08-08 18:34:53 +00:00
Changcheng Xiao
d88f53e2b9 Migrate git-repo to create private changes rather than drafts
Considering that some users might expect changes created with
'-d' option are not public. Private changes may be a better
choice here than work-in-progress changes.

Change-Id: I46a8fb9ae38beb41cf96d6abe82bea6db2439669
2017-08-07 15:08:18 +02:00
Changcheng Xiao
87984c6db4 Add options for git-repo to support private and wip changes
This change adds options for git-repo tool to support private
changes and work-in-progress changes.

Change-Id: I343491f5949f06f1580d53f9cc0dee2dca09130f
2017-08-07 15:02:39 +02:00
David Pursehouse
ffc1401327 Merge "download: try to choose . as default project if none" 2017-08-02 07:19:45 +00:00
David Pursehouse
8a6eeed7f5 Merge "Always print percentage when syncing quietly" 2017-08-02 07:01:08 +00:00
Tim Schumacher
7be072efa6 Always print percentage when syncing quietly
Change-Id: I574396e63520781067ed1e991c41caf7640e5731
2017-07-15 16:44:55 +00:00
Nicolas Cornu
7482a96443 download: try to choose . as default project if none
Change-Id: I28b5e3be5f3c9a4c077af87d6a3e0cc3b96a1b9d
2017-07-12 10:15:06 +02:00
David Pursehouse
3bcd30545e Fix "list comprehension redefines 'x'" warnings from pyflakes
$ git ls-files | grep py$ | xargs pyflakes
  subcmds/stage.py:101: list comprehension redefines 'p' from line 63
  subcmds/sync.py:784: list comprehension redefines 'p' from line 664
  subcmds/upload.py:467: list comprehension redefines 'avail' from line 454

Change-Id: Ia65d1a72ed185ab3357e1a91ed4450c719e75a7c
2017-07-10 23:26:04 +00:00
Martin Kelly
224a31a765 init: add missing submodule arg
The submodule argument to Sync_LocalHalf was missing in
MetaBranchSwitch, causing submodules not to get synced when the
-b/--manifest-branch argument to init is used.

Change-Id: Ie86d271abac2020725770be36ead83be3326e64b
Signed-off-by: Martin Kelly <mkelly@xevo.com>
2017-07-10 14:50:52 -07:00
Nicolas Cornu
b54343d9fd Tell the user if it will upload a draft
Change-Id: Ie004ec9d61603f3f618c47597947b82c59f2839c
2017-07-10 13:20:01 +00:00
David Pursehouse
259f16520a Merge "Add a newline after "Fetching projects" progress output" 2017-06-28 04:24:03 +00:00
Nicolas Cornu
8419ab22d6 sync: Continue job if some fetchs failed but force-broken is set
With --force-broken it continue to fetch other projects but nothing
is added in directory because it abort some lines later.

Change-Id: I32c4a4619b3028893dc4f98e8d4e5bc5c09adb27
2017-06-16 12:23:26 +02:00
Tim Schumacher
913327f10c
Add a newline after "Fetching projects" progress output
Output before change:

    Fetching project platform/packages/providers/UserDictionaryProvider
    Fetching projects:  66% (773/1171)  Fetching project platform/external/regex-re2
    Fetching project device/generic/mini-emulator-x86_64

Output after change:

    Fetching project platform/packages/providers/UserDictionaryProvider
    Fetching projects:  66% (773/1171)
    Fetching project platform/external/regex-re2
    Fetching project device/generic/mini-emulator-x86_64

Change-Id: I4da84da58316c69294e4da2792f83885dc942701
2017-06-13 13:03:39 +02:00
David Pursehouse
a24671f661 Merge "init: allow relative path on --reference argument" 2017-05-29 04:54:12 +00:00
David Riley
e0684addee sync: Add support to dump a JSON event log of all sync events.
Change-Id: Id4852968ac1b2bf0093007cf2e5ca951ddab8b3b
2017-05-29 13:39:54 +09:00
YOUNG HO CHA
c79d3b8fd1 init: allow relative path on --reference argument
Change-Id: I41d6be6bc035fdddb5a27c072994439986d58d58
Signed-off-by: YOUNG HO CHA <ganadist@gmail.com>
2017-05-28 00:33:25 +00:00
David Pursehouse
fbcbcabe98 Merge "init: add --submodules to sync manifest submodules" 2017-05-27 04:24:58 +00:00
David Pursehouse
666debc518 gitc_delete: Remove unused imports
Change-Id: I672189ba99e18dca3956e2396c921d1ef0ca2ddd
2017-05-26 21:53:34 +09:00
David Pursehouse
c354a9b922 abandon: fix usage of undefined variable
As reported by pyflakes:

  subcmds/abandon.py:84: undefined name 'p'

The name of the variable should be 'proj'.

Change-Id: Ic09eb92e8db6b510e99efce010bd0bb094d7cbfe
2017-05-26 21:52:23 +09:00
Martin Kelly
e4e94d26ae init: add --submodules to sync manifest submodules
repo sync can sync submodules via the --fetch-submodules option.
However, if the manifest repo has submodules, those will not be synced.
Having submodules in the manifest repo -- while not commonly done -- can
be useful for inheriting a manifest from another project using <include>
and layering changes on top of it.  In this way, you can avoid having to
deal with merge conflicts between your own manifests and the other
project's manifests (for example, if you're managing an Android fork).

Add a --submodule option to init that automatically syncs the submodules
in the manifest repo whenever the manifest repo changes.

Change-Id: I45d34f04517774c1462d7f233f482d1d81a332a8
Signed-off-by: Martin Kelly <mkelly@xevo.com>
2017-05-23 16:51:31 -07:00
Marc Herbert
ffb4b89099 sync.py: report the remote URL on fatal git remote errors
repo can be configured to download from any number of remote git repos.
However when one fails repo doesn't report which one. Example:
Fatal: remote error: Daily ls-remote rate limit exceeded for IP xx.xx.xx.xx

TEST=repo init -q -u https://chromium.googlesource.com/chromiumos/manifest.git
  # Apply patch in ./.repo/repo/
  # Simulate a git remote error:
  sed -i -e 's#chromiumos/docs#chromiumos/XXdocs#' .repo/manifests/full.xml
  repo sync --quiet --force-sync docs
  # error message now shows the remote URL

Optional test tip: reduce the time.sleep(random(...)) in ./.repo/repo/project.py

Change-Id: I4509383b6a43a8e66064778e8ed612d8a735c8b6
2017-04-04 22:10:34 -07:00
Naseer Ahmed
f4dda9a1be init: Add no-tags and current branch options
This avoids fetching tags and branches for huge manifests

Change-Id: I19c9724d75364440b881b297d42b906f541f73ff
2016-12-01 19:03:41 -05:00
David Pursehouse
ae81c964b6 Merge "implement optional '--all' in the abandon command" 2016-10-28 07:52:31 +00:00
Andrew Wheeler
4d5bb68d58 status: add -q/--quiet option
The --quiet option reduces the output to just
a list of projects with modified workspaces (and
orphans if -o is specified)

A common use case is when performing a full-workspace
merge.  The integrator will kick-off a merge via:

    repo forall -c git merge <some tag>

And then produce a short list of conflicted projects via:

    repo status -q

The integrator can then iteratively fix and clean up all conficted
components.  The merge is complete when:

    repo status -q

    returns no output.

Change-Id: Ibbba8713eac35befd8287c95948874e23fd5c7e2
2016-10-17 15:24:09 -05:00
Kyunam.jo
2e14792a94 implement optional '--all' in the abandon command
when you want to delete all local branches, you should be find
all branches' name, and type them behind 'repo abandon' command.

Usage:
    repo abandon --all [<project>...]

Change-Id: I4d391f37fb9d89b8095488c585468eafc1a35f31
2016-10-17 02:29:42 +00:00
David Pursehouse
82f67987a3 Merge "sync: Fix semaphore release bug that causes thread 'leaks'" 2016-10-17 01:10:16 +00:00
Kyunam.jo
699bcd40be Removed duplication code in abandon.py
code about getting argument is duplicated.
so this line is removed

Change-Id: Id321b999c7dacdb403cd986cbf35f8db62efc157
2016-10-12 10:02:30 +09:00
Andrew Wheeler
7f1ccfbb7b sync: Fix semaphore release bug that causes thread 'leaks'
When repo syncs a manifest that utilizes multiple branches
in the same project, then the sync will use an extra
thread for each "duplicate".  For example, if
the manifest includes the project "foo" and "bar"
twice, then "repo sync -jN" will fetch with N+2 threads.

This is caused by _FetchHelper() releasing the thread semaphore
object each time it's called, even though _FetchProjectList()
may call this function multiple times within the scope of a
single thread.

Fix by moving the thread semaphore release to
_FetchProjectList(), which is only called once per thread
instance.

Change-Id: I1da78b145e09524d40457db5ca5c37d315432bd8
2016-10-11 14:10:34 -05:00
Dan Willemsen
eceeb1b1f5 Support broken symlinks when cleaning obsolete paths
When there's a symlink to a directory, os.walk still lists the symlink
in dirs, even if it isn't configured to follow symlinks. This will fail
the listdirs check if the symlink is broken (either before or during the
cleanup). So instead, check for directory symlinks and remove them using
os.remove.

Bug: Issue 231
Change-Id: I0ec45a26be566613a4a39bf694a3d9c6328481c2
2016-09-27 03:05:11 +00:00
Dan Willemsen
4350791e0d On project cleanup, don't remove nested projects
When there are nested projects in a manifest, like on AOSP right now:

<project path="build" name="platform/build" />
<project path="build/blueprint" name="platform/build/blueprint" />
<project path="build/kati" name="platform/build/kati" />
<project path="build/soong" name="platform/build/soong" />

And the top "build" project is removed (or renamed to remove the
nesting), repo just wipes away everything under build/ and re-creates
the projects that are still there. But it only checks to see if the
build/ project is dirty, so if there are dirty files in a nested
project, they'll just be blown away, and a fresh worktree checked out.

Instead, behave similarly to how `git clean -dxf` behaves and preserve
any subdirectories that have git repositories in them. This isn't as
strict as git -- it does not check to see if the '.git' entry is a
readable gitdir, just whether an entry named '.git' exists.

If it encounters any errors removing files, we'll print them all out to
stderr and tell the user that we were unable to clean up the obsolete
project, that they should clean it up manually, then sync again.

Change-Id: I2f6a7dd205a8e0b7590ca5369e9b0ba21d5a6f77
2016-09-20 17:16:12 -07:00
David Pursehouse
01b7d758d5 Merge "When syncing a project with a shared object store, disable automatic pruning." 2016-09-14 20:48:24 +00:00
David Pursehouse
d9de945d8a Merge "upload: short circuit when nothing is pending" 2016-09-14 05:20:36 +00:00
Gabe Black
2ff302929c When syncing a project with a shared object store, disable automatic pruning.
The shared object stores confuse git and make it throw away objects which are
still in use. We'll avoid that problem by disabling automatic pruning on those
projects, but there's nothing preventing a user from changing the config back
or pruning a repository manually.

BUG=chromium:375945
TEST=Ran repo sync on fresh ChromeOS checkout, starting with a branch of repo
with this change. Verified that the kernel projects and no others were
identified as having shared object stores, and that repo successfully disabled
automatic pruning in their configs. Re-enabled pruning and ran repo sync just
on one of the kernel directories. Verified that pruning was re-disabled as a
result.

Change-Id: I728ed5b06f0087aeb5a23ba8f5410a7cd10af5b0
2016-09-14 00:19:44 -04:00
Vadim Bendebury
e5c0ea0a95 Consider local project to be default for 'repo start'
The requirement to explicitly specify the local project when starting
a new repo branch is somewhat counter intuitive.

This patch uses the current directory's git tree as the default
project.

Tested by running

  'repo start <name>'

observed that the result is the same as if running

  'repo start <name> .'

Change-Id: If106caa801b4cd5ba70dbe8354a227d59f100aa3
2016-09-14 00:17:45 -04:00
Mike Frysinger
163a3be18b upload: short circuit when nothing is pending
When nothing is pending, most of this code is already short-circuited.
Hoist the single check up to make this more obvious/slightly faster.

Change-Id: Iec3a7e08eacd23a7c5f964900d5776bf5252c804
2016-09-14 00:16:37 -04:00
David Pursehouse
6a0a3648f1 Merge "init: Add --no-clone-bundle option" 2016-08-17 04:57:28 +00:00
David Pursehouse
6118faa118 Merge "init: Respect --quiet option when synching manifest repository" 2016-08-17 04:04:14 +00:00
Mike Frysinger
40252c20f7 RepoHook: allow users to approve hooks via manifests
The constant prompting when registered hooks change can be tedious and
has a large multiplication factor when the project is large (e.g. the
AOSP).  It gets worse as people want to write more checks, hooks, docs,
and tests (or fix bugs), but every CL that goes in will trigger a new
prompt to approve.

Let's tweak our trust model when it comes to hooks.  Since people start
off by calling `repo init` with a URL to a manifest, and that manifest
defines all the hooks, anchor trust in that.  This requires that we get
the manifest over a trusted link (e.g. https or ssh) so that it can't
be MITM-ed.  If the user chooses to use an untrusted link (e.g. git or
http), then we'll fallback to the existing hash based approval.

Bug: Issue 226
Change-Id: I77be9e4397383f264fcdaefb582e345ea4069a13
2016-08-16 13:02:52 -04:00
Hu xiuyun
9711a98d6c init: Add --no-clone-bundle option
Bug: Issue 218
Change-Id: I42ba1f5fb9168875da0df6bdf4fe44c8d6498d54
2016-08-15 09:51:48 +09:00
David Pursehouse
438eade413 init: Respect --quiet option when synching manifest repository
Change-Id: Ib58b7dd971670e0888e6428333050700e776b0de
2016-08-15 09:51:48 +09:00
Sebastian Schuberth
7ecccf6225 diffmanifests: support custom git pretty format strings
Change-Id: I29f4f1351c421f393328514d145df1a96aed9ee2
2016-04-21 18:36:11 +00:00
David Pursehouse
79fba68e40 sync: Update help text for --smart-sync to be more specific
The --smart-sync option should return the manifest for *the latest*
known good build.

Change-Id: I2f3216b5b9e1af2ea5f9c3bf1c025813a3b77581
2016-04-13 18:03:00 +09:00
Takeshi Kanemoto
1f0564406b Add --inverse-regex option to forall subcommand
Make it possible to exclude projects using regex/wildcard.

The syntax is similar to that of the -r option, e.g.:

  repo forall -i ^platform/ ^device/ -c 'echo $REPO_PROJECT'

Change-Id: Id250de5665152228c044c79337d3ac15b5696484
2016-04-05 07:28:27 +00:00
Alexandre Garnier
4cfb6d7167 Better error display on forall
It was only displaying 'Project list error: GitError()'
without any useful info about the project nor the error

Change-Id: Iad66cbaa03cad1053b5ae9ecc90d7772aa42ac13
2016-02-18 01:29:54 +00:00
Dan Willemsen
ecf0a6c92b Merge "GITC: Fix 'repo start <branch> <repo>/<subdir>'" 2016-02-04 22:36:05 +00:00
Dan Willemsen
04197a5144 GITC: Fix 'repo start <branch> <repo>/<subdir>'
As soon as we wrote the gitc manifest, the folder for that repo became
empty, causing the next GetProjects lookup to fail. Reorder the
GetProjects calls so that they all happen while we still have the
repository contents available.

If you were already in a subdir, for cases like 'repo start <branch> .',
this would still fail, since the working directory would disappear out
from under you. That's fine most of the time, since we shouldn't be
doing operations based on the local directory, but git has a realpath
function that tries to restore CWD by chdir'ing back to it. So if the
working directory no longer exists, chdir to the topdir before
continuing.

Change-Id: Ibdf6cd37ff6e5a5f8338347c3919175491f7166f
2016-02-04 14:31:55 -08:00
Xiaohui Chen
0b4cb325c6 Add option to rebase onto project's manifest version
Some teams have a continuous build server that would mark certain
manifest green and safe to sync to.  Then team members could repo
sync to that particular manifest file and make sure they always
sync to a green build.  But if she/he has some local changes and
wants to rebase, currently it would be a manual process to find the
correct version to rebase onto.  This patch helps with that use
case by automating the process to rebase onto the currently synced
manifest version.

Change-Id: I847c9eb6addf7f84fd3f5594fbf8c0bcc103f9a5
2016-01-28 10:20:03 -08:00
Hu Xiuyun
e9becc079c Sync: Fix error exit code when both -n and -f are used
When repo sync is used with -f (--force-error) and a project fails to
sync, the sync will continue but then exit with an error status.

However if -n (--network-only) is also used, the exit code is 0, even
when a project failed.

Modify the logic to make sure the sync exits with the correct status.

Bug: Issue 214
Change-Id: I0b5d97a34642c5aa3743750ef14a42c9d5743c1d
2015-11-26 02:25:43 +00:00
David Pursehouse
74cfd2709b Sync: Add option to prune refs during sync
By passing --prune to the sync command, the --prune option is
given to the `git fetch`, causing refs that no longer exist on
the remote to be removed.

Change-Id: I3cedacce14276d96ac2d5aabf2d07fd05e92bc02
2015-10-27 03:04:17 +00:00