Commit Graph

1434 Commits

Author SHA1 Message Date
Andrew Wheeler
4d5bb68d58 status: add -q/--quiet option
The --quiet option reduces the output to just
a list of projects with modified workspaces (and
orphans if -o is specified)

A common use case is when performing a full-workspace
merge.  The integrator will kick-off a merge via:

    repo forall -c git merge <some tag>

And then produce a short list of conflicted projects via:

    repo status -q

The integrator can then iteratively fix and clean up all conficted
components.  The merge is complete when:

    repo status -q

    returns no output.

Change-Id: Ibbba8713eac35befd8287c95948874e23fd5c7e2
2016-10-17 15:24:09 -05:00
Kyunam.jo
2e14792a94 implement optional '--all' in the abandon command
when you want to delete all local branches, you should be find
all branches' name, and type them behind 'repo abandon' command.

Usage:
    repo abandon --all [<project>...]

Change-Id: I4d391f37fb9d89b8095488c585468eafc1a35f31
2016-10-17 02:29:42 +00:00
David Pursehouse
82f67987a3 Merge "sync: Fix semaphore release bug that causes thread 'leaks'" 2016-10-17 01:10:16 +00:00
Kyunam.jo
699bcd40be Removed duplication code in abandon.py
code about getting argument is duplicated.
so this line is removed

Change-Id: Id321b999c7dacdb403cd986cbf35f8db62efc157
2016-10-12 10:02:30 +09:00
Andrew Wheeler
7f1ccfbb7b sync: Fix semaphore release bug that causes thread 'leaks'
When repo syncs a manifest that utilizes multiple branches
in the same project, then the sync will use an extra
thread for each "duplicate".  For example, if
the manifest includes the project "foo" and "bar"
twice, then "repo sync -jN" will fetch with N+2 threads.

This is caused by _FetchHelper() releasing the thread semaphore
object each time it's called, even though _FetchProjectList()
may call this function multiple times within the scope of a
single thread.

Fix by moving the thread semaphore release to
_FetchProjectList(), which is only called once per thread
instance.

Change-Id: I1da78b145e09524d40457db5ca5c37d315432bd8
2016-10-11 14:10:34 -05:00
Dan Willemsen
eceeb1b1f5 Support broken symlinks when cleaning obsolete paths
When there's a symlink to a directory, os.walk still lists the symlink
in dirs, even if it isn't configured to follow symlinks. This will fail
the listdirs check if the symlink is broken (either before or during the
cleanup). So instead, check for directory symlinks and remove them using
os.remove.

Bug: Issue 231
Change-Id: I0ec45a26be566613a4a39bf694a3d9c6328481c2
2016-09-27 03:05:11 +00:00
Dan Willemsen
16889ba43d Revert "Repo: fall back to http, if ssh connection fails for http repos"
This reverts commit 488bf092d5.

Issue 230

Change-Id: I3a5725301f576e1a2ac499cb6daa631895115640
2016-09-22 16:40:27 +00:00
David Pursehouse
40d3952270 Merge "On project cleanup, don't remove nested projects" 2016-09-21 22:23:44 +00:00
Dan Willemsen
4350791e0d On project cleanup, don't remove nested projects
When there are nested projects in a manifest, like on AOSP right now:

<project path="build" name="platform/build" />
<project path="build/blueprint" name="platform/build/blueprint" />
<project path="build/kati" name="platform/build/kati" />
<project path="build/soong" name="platform/build/soong" />

And the top "build" project is removed (or renamed to remove the
nesting), repo just wipes away everything under build/ and re-creates
the projects that are still there. But it only checks to see if the
build/ project is dirty, so if there are dirty files in a nested
project, they'll just be blown away, and a fresh worktree checked out.

Instead, behave similarly to how `git clean -dxf` behaves and preserve
any subdirectories that have git repositories in them. This isn't as
strict as git -- it does not check to see if the '.git' entry is a
readable gitdir, just whether an entry named '.git' exists.

If it encounters any errors removing files, we'll print them all out to
stderr and tell the user that we were unable to clean up the obsolete
project, that they should clean it up manually, then sync again.

Change-Id: I2f6a7dd205a8e0b7590ca5369e9b0ba21d5a6f77
2016-09-20 17:16:12 -07:00
Steve Rae
d648045366 implement optional 'pushurl' in the manifest file
Allow the 'remote' element in the manifest file to define an optional
'pushurl' attribute which is passed into the .git/config file.

Change-Id: If342d299d371374aedc4440645798888869c9714
Signed-off-by: Steve Rae <steve.rae@raedomain.com>
2016-09-20 15:31:20 +00:00
David Pursehouse
628456833a Merge "Repo: improve error detection for new ssh connections" 2016-09-20 08:06:12 +00:00
David Pursehouse
2aa61d0bc8 Merge "Repo: fall back to http, if ssh connection fails for http repos" 2016-09-20 08:05:57 +00:00
David Pursehouse
4aed6f8c7d Merge "Replace pylint with pyflakes/flake8" 2016-09-20 06:57:16 +00:00
David Pursehouse
01b7d758d5 Merge "When syncing a project with a shared object store, disable automatic pruning." 2016-09-14 20:48:24 +00:00
David Pursehouse
267ac57361 Merge "repo: add comment for updating maintainer keys" 2016-09-14 14:06:42 +00:00
David Pursehouse
bb5b1a076b Replace pylint with pyflakes/flake8
pylint reports a lot of warnings, but many of them are false positive,
and it's difficult to configure it. It also seems that for some reason
the included config file is not working well with the latest version.

Update the documentation to recommend using pyflakes and flake8 instead
of pylint. Remove the pylint config and add a basic flake8 config with
minimum settings:

- Maximum line length 80 columns
- Ignore warnings about indentation (repo uses 2 rather than expected 4)
- Ignore warnings about import placement

In this commit no code cleanup is done, and it's expected that most of
the files will throw up quite a few warnings, at least for flake8. These
can be cleaned up in follow-up commits.

The existing pylint suppression comments are left as-is. These will be
helpful when cleaning up pyflakes warnings later.

Change-Id: I2f7cb4340266ed07cc973ca6483b8f09d66a765b
2016-09-14 09:49:02 +02:00
David Pursehouse
e01ee026e6 Merge "Fix submodule checkout error when using sync-s option" 2016-09-14 07:45:50 +00:00
Mike Frysinger
e4433653db repo: add comment for updating maintainer keys
Change-Id: Ic1e7557f9597234033561ab9fb3104b87e30015e
2016-09-14 01:28:30 -04:00
David Pursehouse
d9de945d8a Merge "upload: short circuit when nothing is pending" 2016-09-14 05:20:36 +00:00
Gabe Black
2ff302929c When syncing a project with a shared object store, disable automatic pruning.
The shared object stores confuse git and make it throw away objects which are
still in use. We'll avoid that problem by disabling automatic pruning on those
projects, but there's nothing preventing a user from changing the config back
or pruning a repository manually.

BUG=chromium:375945
TEST=Ran repo sync on fresh ChromeOS checkout, starting with a branch of repo
with this change. Verified that the kernel projects and no others were
identified as having shared object stores, and that repo successfully disabled
automatic pruning in their configs. Re-enabled pruning and ran repo sync just
on one of the kernel directories. Verified that pruning was re-disabled as a
result.

Change-Id: I728ed5b06f0087aeb5a23ba8f5410a7cd10af5b0
2016-09-14 00:19:44 -04:00
Vadim Bendebury
e5c0ea0a95 Consider local project to be default for 'repo start'
The requirement to explicitly specify the local project when starting
a new repo branch is somewhat counter intuitive.

This patch uses the current directory's git tree as the default
project.

Tested by running

  'repo start <name>'

observed that the result is the same as if running

  'repo start <name> .'

Change-Id: If106caa801b4cd5ba70dbe8354a227d59f100aa3
2016-09-14 00:17:45 -04:00
Mike Frysinger
163a3be18b upload: short circuit when nothing is pending
When nothing is pending, most of this code is already short-circuited.
Hoist the single check up to make this more obvious/slightly faster.

Change-Id: Iec3a7e08eacd23a7c5f964900d5776bf5252c804
2016-09-14 00:16:37 -04:00
David Pursehouse
7a77c16d37 Update mailmap
Order the entries alphabetically and add a couple more.

Change-Id: I8d98e8a5a1dd6868b566a42428030d2040b8af7a
2016-09-02 11:12:28 +09:00
Timo Lotterbach
488bf092d5 Repo: fall back to http, if ssh connection fails for http repos
if a gerrit server has ssh and https access enabled, but user access
(for some users) is limited to https, 'repo upload' command will fail
for them.

Gerrit returns a ssh configuration (gerrit/ssh_info), that does not work
for users limited to https.

With this patch repo will test, if the returned ssh configuration from
gerrit/ssh_info is working. if not, it will fall back to https for upload.

Change-Id: If98f472e994f350bf71f35610cd649b163f1ab33
2016-08-30 07:35:03 +00:00
Timo Lotterbach
05dc46b0e3 Repo: improve error detection for new ssh connections
this check can only detect errors that happen within 1 sec after launching
ssh. But this is typically enough to catch configuration issues like
'connection refused' or 'authentication failed'.

Change-Id: I00b6f62d4c2889b1faa6c820e49a198554c92795
2016-08-30 07:34:33 +00:00
Dan Willemsen
39252ba028 gitc: Lower concurrent ls-projects requests
Too many requests at the same time is causing 502 errors.

Change-Id: Ic8fbb2fbb7fb6014733fa5be018d2dc02472f704
2016-08-23 14:19:00 -07:00
Jonathan Nieder
71e4cea6de RepoHook: do not list options twice during hash based approval
Instead of

  Do you want to allow this script to run (yes/yes-never-ask-again/NO)? (yes/always/NO)?

ask

  Do you want to allow this script to run (yes/always/NO)?

Change-Id: I5f5a2d0e88086a8d85e54fb8623a62d74a20956a
Signed-off-by: Jonathan Nieder <jrn@google.com>
2016-08-18 17:06:26 -07:00
David Pursehouse
c4c2b066d1 Increment the wrapper version
There have been a number of changes in the repo wrapper since the last
increment that was done in fee390ee:

- 9711a98 init: Add --no-clone-bundle option
- 631d0ec Support non-ASCII GNUPGHOME environment variable
- 4088eb4 repo: Cleaned up pylint/pep8 violations
- 5553628 repo: Add check of REPO_URL env variable
- 745b4ad Fix gitc-init behavior
- d3ddcdb Ignore clone.bundle on HTTP 501, i.e. Not Implemented

Change-Id: I3f763ef0ec2df2d726dff429021b48ad474148f1
2016-08-17 13:58:17 +09:00
David Pursehouse
6a0a3648f1 Merge "init: Add --no-clone-bundle option" 2016-08-17 04:57:28 +00:00
David Pursehouse
6118faa118 Merge "init: Respect --quiet option when synching manifest repository" 2016-08-17 04:04:14 +00:00
David Pursehouse
183c52ab02 Merge "project: Set config option to skip lfs smudge filter" 2016-08-17 01:00:12 +00:00
Jonathan Nieder
58f85f9a30 Merge "RepoHook: allow users to approve hooks via manifests" 2016-08-16 18:05:23 +00:00
Mike Frysinger
40252c20f7 RepoHook: allow users to approve hooks via manifests
The constant prompting when registered hooks change can be tedious and
has a large multiplication factor when the project is large (e.g. the
AOSP).  It gets worse as people want to write more checks, hooks, docs,
and tests (or fix bugs), but every CL that goes in will trigger a new
prompt to approve.

Let's tweak our trust model when it comes to hooks.  Since people start
off by calling `repo init` with a URL to a manifest, and that manifest
defines all the hooks, anchor trust in that.  This requires that we get
the manifest over a trusted link (e.g. https or ssh) so that it can't
be MITM-ed.  If the user chooses to use an untrusted link (e.g. git or
http), then we'll fallback to the existing hash based approval.

Bug: Issue 226
Change-Id: I77be9e4397383f264fcdaefb582e345ea4069a13
2016-08-16 13:02:52 -04:00
David Pursehouse
76a4a9df86 project: Set config option to skip lfs smudge filter
During sync, repo runs `git read-tree --reset -u -v HEAD` which causes
git-lfs's smudge filter to run. However this fails because git-lfs does
not work with bare repositories.

Add lfs.filter configuration to the project config as suggested in the
comments on the upstream git-lfs client issue [1]. This prevents the
smudge filter from running, and the sync completes successfully.

For any projects that have LFS objects, `git lfs pull` must be executed.

[1] https://github.com/github/git-lfs/issues/1422

Bug: Issue 224
Change-Id: I091ff37998131e2e6bbc59aa37ee352fe12d7fcd
2016-08-16 21:55:36 +09:00
Mike Frysinger
befaec1e56 improve docs
Change-Id: Ide4008f09c2f17f8fb3d85dfffe94544abfdd6a6
2016-08-16 00:14:28 -04:00
Hu xiuyun
9711a98d6c init: Add --no-clone-bundle option
Bug: Issue 218
Change-Id: I42ba1f5fb9168875da0df6bdf4fe44c8d6498d54
2016-08-15 09:51:48 +09:00
David Pursehouse
438eade413 init: Respect --quiet option when synching manifest repository
Change-Id: Ib58b7dd971670e0888e6428333050700e776b0de
2016-08-15 09:51:48 +09:00
David Pursehouse
69297c1b77 Merge "Support non-ASCII GNUPGHOME environment variable" 2016-08-15 00:51:32 +00:00
David Pursehouse
8016f60a46 Merge "repo: Repo does not always handle '.' parameter correctly" 2016-08-14 08:50:28 +00:00
Dāvis Mosāns
631d0ec708 Support non-ASCII GNUPGHOME environment variable
Here we don't need to encode this gpg_dir string when using
Python 2.7 on Linux.

Change-Id: I56724e9511d3b1aea61535e654a45c212130630d
2016-07-16 22:10:06 +03:00
Xin Li
f97e72e5dd Bail out when manifest is referencing a bad SHA-1 revision.
BUG: Issue 222
Change-Id: Ie0a64b39922d6fdf1be2989eb514985be8490278
2016-06-29 11:01:43 -07:00
Cheuk Leung
8ac0c96537 Fix removing broken symlink in reference dir
Re-ordered to first create the symlink before checking the source
file and remove the destination if the source does not exists.

Change-Id: Iae923ba2ef0ba5a8dc1b8e42d8cc3f3708f773af
2016-06-29 05:43:01 +00:00
David Pursehouse
faaddc9b4e Merge "Adds additional crlf clobber avoidance." 2016-06-29 05:42:20 +00:00
David Pursehouse
a36af0767b Merge "Fix variable assignment" 2016-06-29 01:43:41 +00:00
Aymen Bouaziz
037040f73e Fix variable assignment
If upstream string is empty, current_branch_only variable will be assigned
to an empty string.

This is not what we expect here as this variable is a boolean.

Change-Id: Ibba935e25a74c2be1e50c88b4b403cf394ba365e
2016-06-28 12:31:25 +02:00
Aymen Bouaziz
2598ed06f1 Fix submodule checkout error when using sync-s option
When sync-s="true" option is used, the checkout of a submodule will try
to use the revision attribute of the parent project.

If this revision is a named reference, the checkout will fail if there
is no reference with this name in the submodule.

The proposed solution is to use the git commit id as revisionExpr for
submodules.

Change-Id: Ie8390a11957fd6a9c61289c6861d13cb3fa11678
2016-06-27 20:16:45 +02:00
Fredrik Berggren
01952e6634 Adds additional crlf clobber avoidance.
Adds the hook-scripts to .gitattributes due to the shell-scripts not
liking CRLF which they will get if a user sets 'autocrlf = true'
in their global gitconfig.

Further, since the python interpreter can handle either CRLF or LF, 
python-scripts specific line-ending rules have been removed.

Change-Id: I2d6bfd491b2f626b9ca93c40a3a7f2cfba6c54f0
2016-06-22 08:36:45 +00:00
Stefan Beller
9d2b14d2ec pylint: Fix unused-{argument,variable} warning
This commit fixes 4 out of the remaining 5 pylint warnings:

$ pylint --rcfile=.pylintrc *.py
************* Module gitc_utils
W:146, 0: TODO(sbasi/jorg): Come up with a solution to remove the sleep below. (fixme)
W:130, 6: Unused variable 'name' (unused-variable)
************* Module main
W:382,32: Unused argument 'fp' (unused-argument)
W:382,36: Unused argument 'code' (unused-argument)
W:382,42: Unused argument 'msg' (unused-argument)

Change-Id: Ie3d77b9a65b7daefaa9aa4b80f4b682c1678fd58
Signed-off-by: Stefan Beller <sbeller@google.com>
2016-06-21 11:48:57 -07:00
Stefan Beller
6685106306 pylint: fix indentation in manifest_xml
This fixes pylint warning:

************* Module manifest_xml
W:975, 0: Bad indentation. Found 8 spaces, expected 6 (bad-indentation)

Change-Id: I967212f9439430351836ebdc27e442d7b77476e2
Signed-off-by: Stefan Beller <sbeller@google.com>
2016-06-17 16:45:48 -07:00
Stefan Beller
d64e8eee51 pylint: ignore bad-whitespace
This ignores whitespaces errors, which we have quite a few of in argument
lists, for example:

************* Module git_config
C:209, 0: No space allowed around keyword argument assignment
  def HasSection(self, section, subsection = ''):
                                           ^ (bad-whitespace)
C:320, 0: No space allowed around keyword argument assignment
                   capture_stdout = True,
                                  ^ (bad-whitespace)
C:321, 0: No space allowed around keyword argument assignment
                   capture_stderr = True)
                                  ^ (bad-whitespace)
C:427, 0: Exactly one space required after comma
                     '-o','ControlPath %s' % ssh_sock(),
                         ^ (bad-whitespace)
C:436, 0: Exactly one space required after comma
    check_command = command_base + ['-O','check']
                                        ^ (bad-whitespace)
C:464, 0: Exactly one space required after comma
             % (host,port, str(e)), file=sys.stderr)
                    ^ (bad-whitespace)
C:707, 0: No space allowed around keyword argument assignment
    return self._config.GetString(key, all_keys = all_keys)
                                                ^ (bad-whitespace)
C:759, 0: No space allowed around keyword argument assignment
    return self._config.GetString(key, all_keys = all_keys)
                                                ^ (bad-whitespace)

Change-Id: Ia8f154f6741ce609787551f65877d7584c457903
Signed-off-by: Stefan Beller <sbeller@google.com>
2016-06-17 16:37:24 -07:00