Commit Graph

1338 Commits

Author SHA1 Message Date
David Pursehouse
ae81c964b6 Merge "implement optional '--all' in the abandon command" 2016-10-28 07:52:31 +00:00
Dan Willemsen
e02c17c9ea Merge "_CheckDirReference: log actual error before suggesting --force-sync" 2016-10-28 06:49:07 +00:00
Joe Kilner
6e31079033 Add sso to list of known schemes for relative URLs
repo already special-cases sso:// URLs to behave similarly to https://
and rpc:// elsewhere in repo, but it forgot to do so here.

Noticed when trying to use relative URLs in a manifest obtained using
an sso:// URL.

Change-Id: Ia11469a09bbd6e444dbc4f22c82f9bbe9f5fd083
2016-10-27 15:56:38 -07:00
Marc Herbert
ec287902e6 _CheckDirReference: log actual error before suggesting --force-sync
A recent backward incompatible change created confusion and loss of
productivity and highlighted the very limited amount of information
provided when repo sync fails; merely recommending to --force-sync
and blow-up git repos without any hint as to why. The addition of
this basic _error(...) call would have provided a clue and will in
the future.

BUG=Issue 232
TEST=simulate a breakage similar to the ones reported at
  https://groups.google.com/a/chromium.org/forum/#!topic/chromium-os-dev/2-0oCy_CX5s
  cd .repo/projects/src/third_party/libapps.git/
  file info; rm info; ln -s wronglink info
  cd -
  repo sync src/third_party/libapps/
  # error message now shows the failure

Change-Id: Idd2f177a096f1ad686caa8c67cb361d594ccaa57
2016-10-27 12:58:26 -07:00
Andrew Wheeler
4d5bb68d58 status: add -q/--quiet option
The --quiet option reduces the output to just
a list of projects with modified workspaces (and
orphans if -o is specified)

A common use case is when performing a full-workspace
merge.  The integrator will kick-off a merge via:

    repo forall -c git merge <some tag>

And then produce a short list of conflicted projects via:

    repo status -q

The integrator can then iteratively fix and clean up all conficted
components.  The merge is complete when:

    repo status -q

    returns no output.

Change-Id: Ibbba8713eac35befd8287c95948874e23fd5c7e2
2016-10-17 15:24:09 -05:00
Kyunam.jo
2e14792a94 implement optional '--all' in the abandon command
when you want to delete all local branches, you should be find
all branches' name, and type them behind 'repo abandon' command.

Usage:
    repo abandon --all [<project>...]

Change-Id: I4d391f37fb9d89b8095488c585468eafc1a35f31
2016-10-17 02:29:42 +00:00
David Pursehouse
82f67987a3 Merge "sync: Fix semaphore release bug that causes thread 'leaks'" 2016-10-17 01:10:16 +00:00
Kyunam.jo
699bcd40be Removed duplication code in abandon.py
code about getting argument is duplicated.
so this line is removed

Change-Id: Id321b999c7dacdb403cd986cbf35f8db62efc157
2016-10-12 10:02:30 +09:00
Andrew Wheeler
7f1ccfbb7b sync: Fix semaphore release bug that causes thread 'leaks'
When repo syncs a manifest that utilizes multiple branches
in the same project, then the sync will use an extra
thread for each "duplicate".  For example, if
the manifest includes the project "foo" and "bar"
twice, then "repo sync -jN" will fetch with N+2 threads.

This is caused by _FetchHelper() releasing the thread semaphore
object each time it's called, even though _FetchProjectList()
may call this function multiple times within the scope of a
single thread.

Fix by moving the thread semaphore release to
_FetchProjectList(), which is only called once per thread
instance.

Change-Id: I1da78b145e09524d40457db5ca5c37d315432bd8
2016-10-11 14:10:34 -05:00
Dan Willemsen
eceeb1b1f5 Support broken symlinks when cleaning obsolete paths
When there's a symlink to a directory, os.walk still lists the symlink
in dirs, even if it isn't configured to follow symlinks. This will fail
the listdirs check if the symlink is broken (either before or during the
cleanup). So instead, check for directory symlinks and remove them using
os.remove.

Bug: Issue 231
Change-Id: I0ec45a26be566613a4a39bf694a3d9c6328481c2
2016-09-27 03:05:11 +00:00
Dan Willemsen
16889ba43d Revert "Repo: fall back to http, if ssh connection fails for http repos"
This reverts commit 488bf092d5.

Issue 230

Change-Id: I3a5725301f576e1a2ac499cb6daa631895115640
2016-09-22 16:40:27 +00:00
David Pursehouse
40d3952270 Merge "On project cleanup, don't remove nested projects" 2016-09-21 22:23:44 +00:00
Dan Willemsen
4350791e0d On project cleanup, don't remove nested projects
When there are nested projects in a manifest, like on AOSP right now:

<project path="build" name="platform/build" />
<project path="build/blueprint" name="platform/build/blueprint" />
<project path="build/kati" name="platform/build/kati" />
<project path="build/soong" name="platform/build/soong" />

And the top "build" project is removed (or renamed to remove the
nesting), repo just wipes away everything under build/ and re-creates
the projects that are still there. But it only checks to see if the
build/ project is dirty, so if there are dirty files in a nested
project, they'll just be blown away, and a fresh worktree checked out.

Instead, behave similarly to how `git clean -dxf` behaves and preserve
any subdirectories that have git repositories in them. This isn't as
strict as git -- it does not check to see if the '.git' entry is a
readable gitdir, just whether an entry named '.git' exists.

If it encounters any errors removing files, we'll print them all out to
stderr and tell the user that we were unable to clean up the obsolete
project, that they should clean it up manually, then sync again.

Change-Id: I2f6a7dd205a8e0b7590ca5369e9b0ba21d5a6f77
2016-09-20 17:16:12 -07:00
Steve Rae
d648045366 implement optional 'pushurl' in the manifest file
Allow the 'remote' element in the manifest file to define an optional
'pushurl' attribute which is passed into the .git/config file.

Change-Id: If342d299d371374aedc4440645798888869c9714
Signed-off-by: Steve Rae <steve.rae@raedomain.com>
2016-09-20 15:31:20 +00:00
David Pursehouse
628456833a Merge "Repo: improve error detection for new ssh connections" 2016-09-20 08:06:12 +00:00
David Pursehouse
2aa61d0bc8 Merge "Repo: fall back to http, if ssh connection fails for http repos" 2016-09-20 08:05:57 +00:00
David Pursehouse
4aed6f8c7d Merge "Replace pylint with pyflakes/flake8" 2016-09-20 06:57:16 +00:00
David Pursehouse
01b7d758d5 Merge "When syncing a project with a shared object store, disable automatic pruning." 2016-09-14 20:48:24 +00:00
David Pursehouse
267ac57361 Merge "repo: add comment for updating maintainer keys" 2016-09-14 14:06:42 +00:00
David Pursehouse
bb5b1a076b Replace pylint with pyflakes/flake8
pylint reports a lot of warnings, but many of them are false positive,
and it's difficult to configure it. It also seems that for some reason
the included config file is not working well with the latest version.

Update the documentation to recommend using pyflakes and flake8 instead
of pylint. Remove the pylint config and add a basic flake8 config with
minimum settings:

- Maximum line length 80 columns
- Ignore warnings about indentation (repo uses 2 rather than expected 4)
- Ignore warnings about import placement

In this commit no code cleanup is done, and it's expected that most of
the files will throw up quite a few warnings, at least for flake8. These
can be cleaned up in follow-up commits.

The existing pylint suppression comments are left as-is. These will be
helpful when cleaning up pyflakes warnings later.

Change-Id: I2f7cb4340266ed07cc973ca6483b8f09d66a765b
2016-09-14 09:49:02 +02:00
David Pursehouse
e01ee026e6 Merge "Fix submodule checkout error when using sync-s option" 2016-09-14 07:45:50 +00:00
Mike Frysinger
e4433653db repo: add comment for updating maintainer keys
Change-Id: Ic1e7557f9597234033561ab9fb3104b87e30015e
2016-09-14 01:28:30 -04:00
David Pursehouse
d9de945d8a Merge "upload: short circuit when nothing is pending" 2016-09-14 05:20:36 +00:00
Gabe Black
2ff302929c When syncing a project with a shared object store, disable automatic pruning.
The shared object stores confuse git and make it throw away objects which are
still in use. We'll avoid that problem by disabling automatic pruning on those
projects, but there's nothing preventing a user from changing the config back
or pruning a repository manually.

BUG=chromium:375945
TEST=Ran repo sync on fresh ChromeOS checkout, starting with a branch of repo
with this change. Verified that the kernel projects and no others were
identified as having shared object stores, and that repo successfully disabled
automatic pruning in their configs. Re-enabled pruning and ran repo sync just
on one of the kernel directories. Verified that pruning was re-disabled as a
result.

Change-Id: I728ed5b06f0087aeb5a23ba8f5410a7cd10af5b0
2016-09-14 00:19:44 -04:00
Vadim Bendebury
e5c0ea0a95 Consider local project to be default for 'repo start'
The requirement to explicitly specify the local project when starting
a new repo branch is somewhat counter intuitive.

This patch uses the current directory's git tree as the default
project.

Tested by running

  'repo start <name>'

observed that the result is the same as if running

  'repo start <name> .'

Change-Id: If106caa801b4cd5ba70dbe8354a227d59f100aa3
2016-09-14 00:17:45 -04:00
Mike Frysinger
163a3be18b upload: short circuit when nothing is pending
When nothing is pending, most of this code is already short-circuited.
Hoist the single check up to make this more obvious/slightly faster.

Change-Id: Iec3a7e08eacd23a7c5f964900d5776bf5252c804
2016-09-14 00:16:37 -04:00
David Pursehouse
7a77c16d37 Update mailmap
Order the entries alphabetically and add a couple more.

Change-Id: I8d98e8a5a1dd6868b566a42428030d2040b8af7a
2016-09-02 11:12:28 +09:00
Timo Lotterbach
488bf092d5 Repo: fall back to http, if ssh connection fails for http repos
if a gerrit server has ssh and https access enabled, but user access
(for some users) is limited to https, 'repo upload' command will fail
for them.

Gerrit returns a ssh configuration (gerrit/ssh_info), that does not work
for users limited to https.

With this patch repo will test, if the returned ssh configuration from
gerrit/ssh_info is working. if not, it will fall back to https for upload.

Change-Id: If98f472e994f350bf71f35610cd649b163f1ab33
2016-08-30 07:35:03 +00:00
Timo Lotterbach
05dc46b0e3 Repo: improve error detection for new ssh connections
this check can only detect errors that happen within 1 sec after launching
ssh. But this is typically enough to catch configuration issues like
'connection refused' or 'authentication failed'.

Change-Id: I00b6f62d4c2889b1faa6c820e49a198554c92795
2016-08-30 07:34:33 +00:00
Dan Willemsen
39252ba028 gitc: Lower concurrent ls-projects requests
Too many requests at the same time is causing 502 errors.

Change-Id: Ic8fbb2fbb7fb6014733fa5be018d2dc02472f704
2016-08-23 14:19:00 -07:00
Jonathan Nieder
71e4cea6de RepoHook: do not list options twice during hash based approval
Instead of

  Do you want to allow this script to run (yes/yes-never-ask-again/NO)? (yes/always/NO)?

ask

  Do you want to allow this script to run (yes/always/NO)?

Change-Id: I5f5a2d0e88086a8d85e54fb8623a62d74a20956a
Signed-off-by: Jonathan Nieder <jrn@google.com>
2016-08-18 17:06:26 -07:00
David Pursehouse
c4c2b066d1 Increment the wrapper version
There have been a number of changes in the repo wrapper since the last
increment that was done in fee390ee:

- 9711a98 init: Add --no-clone-bundle option
- 631d0ec Support non-ASCII GNUPGHOME environment variable
- 4088eb4 repo: Cleaned up pylint/pep8 violations
- 5553628 repo: Add check of REPO_URL env variable
- 745b4ad Fix gitc-init behavior
- d3ddcdb Ignore clone.bundle on HTTP 501, i.e. Not Implemented

Change-Id: I3f763ef0ec2df2d726dff429021b48ad474148f1
2016-08-17 13:58:17 +09:00
David Pursehouse
6a0a3648f1 Merge "init: Add --no-clone-bundle option" 2016-08-17 04:57:28 +00:00
David Pursehouse
6118faa118 Merge "init: Respect --quiet option when synching manifest repository" 2016-08-17 04:04:14 +00:00
David Pursehouse
183c52ab02 Merge "project: Set config option to skip lfs smudge filter" 2016-08-17 01:00:12 +00:00
Jonathan Nieder
58f85f9a30 Merge "RepoHook: allow users to approve hooks via manifests" 2016-08-16 18:05:23 +00:00
Mike Frysinger
40252c20f7 RepoHook: allow users to approve hooks via manifests
The constant prompting when registered hooks change can be tedious and
has a large multiplication factor when the project is large (e.g. the
AOSP).  It gets worse as people want to write more checks, hooks, docs,
and tests (or fix bugs), but every CL that goes in will trigger a new
prompt to approve.

Let's tweak our trust model when it comes to hooks.  Since people start
off by calling `repo init` with a URL to a manifest, and that manifest
defines all the hooks, anchor trust in that.  This requires that we get
the manifest over a trusted link (e.g. https or ssh) so that it can't
be MITM-ed.  If the user chooses to use an untrusted link (e.g. git or
http), then we'll fallback to the existing hash based approval.

Bug: Issue 226
Change-Id: I77be9e4397383f264fcdaefb582e345ea4069a13
2016-08-16 13:02:52 -04:00
David Pursehouse
76a4a9df86 project: Set config option to skip lfs smudge filter
During sync, repo runs `git read-tree --reset -u -v HEAD` which causes
git-lfs's smudge filter to run. However this fails because git-lfs does
not work with bare repositories.

Add lfs.filter configuration to the project config as suggested in the
comments on the upstream git-lfs client issue [1]. This prevents the
smudge filter from running, and the sync completes successfully.

For any projects that have LFS objects, `git lfs pull` must be executed.

[1] https://github.com/github/git-lfs/issues/1422

Bug: Issue 224
Change-Id: I091ff37998131e2e6bbc59aa37ee352fe12d7fcd
2016-08-16 21:55:36 +09:00
Mike Frysinger
befaec1e56 improve docs
Change-Id: Ide4008f09c2f17f8fb3d85dfffe94544abfdd6a6
2016-08-16 00:14:28 -04:00
Hu xiuyun
9711a98d6c init: Add --no-clone-bundle option
Bug: Issue 218
Change-Id: I42ba1f5fb9168875da0df6bdf4fe44c8d6498d54
2016-08-15 09:51:48 +09:00
David Pursehouse
438eade413 init: Respect --quiet option when synching manifest repository
Change-Id: Ib58b7dd971670e0888e6428333050700e776b0de
2016-08-15 09:51:48 +09:00
David Pursehouse
69297c1b77 Merge "Support non-ASCII GNUPGHOME environment variable" 2016-08-15 00:51:32 +00:00
David Pursehouse
8016f60a46 Merge "repo: Repo does not always handle '.' parameter correctly" 2016-08-14 08:50:28 +00:00
Dāvis Mosāns
631d0ec708 Support non-ASCII GNUPGHOME environment variable
Here we don't need to encode this gpg_dir string when using
Python 2.7 on Linux.

Change-Id: I56724e9511d3b1aea61535e654a45c212130630d
2016-07-16 22:10:06 +03:00
Xin Li
f97e72e5dd Bail out when manifest is referencing a bad SHA-1 revision.
BUG: Issue 222
Change-Id: Ie0a64b39922d6fdf1be2989eb514985be8490278
2016-06-29 11:01:43 -07:00
Cheuk Leung
8ac0c96537 Fix removing broken symlink in reference dir
Re-ordered to first create the symlink before checking the source
file and remove the destination if the source does not exists.

Change-Id: Iae923ba2ef0ba5a8dc1b8e42d8cc3f3708f773af
2016-06-29 05:43:01 +00:00
David Pursehouse
faaddc9b4e Merge "Adds additional crlf clobber avoidance." 2016-06-29 05:42:20 +00:00
David Pursehouse
a36af0767b Merge "Fix variable assignment" 2016-06-29 01:43:41 +00:00
Aymen Bouaziz
037040f73e Fix variable assignment
If upstream string is empty, current_branch_only variable will be assigned
to an empty string.

This is not what we expect here as this variable is a boolean.

Change-Id: Ibba935e25a74c2be1e50c88b4b403cf394ba365e
2016-06-28 12:31:25 +02:00
Aymen Bouaziz
2598ed06f1 Fix submodule checkout error when using sync-s option
When sync-s="true" option is used, the checkout of a submodule will try
to use the revision attribute of the parent project.

If this revision is a named reference, the checkout will fail if there
is no reference with this name in the submodule.

The proposed solution is to use the git commit id as revisionExpr for
submodules.

Change-Id: Ie8390a11957fd6a9c61289c6861d13cb3fa11678
2016-06-27 20:16:45 +02:00