Commit Graph

1403 Commits

Author SHA1 Message Date
David Pursehouse
666debc518 gitc_delete: Remove unused imports
Change-Id: I672189ba99e18dca3956e2396c921d1ef0ca2ddd
2017-05-26 21:53:34 +09:00
David Pursehouse
c354a9b922 abandon: fix usage of undefined variable
As reported by pyflakes:

  subcmds/abandon.py:84: undefined name 'p'

The name of the variable should be 'proj'.

Change-Id: Ic09eb92e8db6b510e99efce010bd0bb094d7cbfe
2017-05-26 21:52:23 +09:00
David Pursehouse
06848b2415 Update .mailmap
Change-Id: Icabc7fb2161b661c2df9290a1ca6f75b9b1b8e1b
2017-05-26 21:44:57 +09:00
Martin Kelly
e4e94d26ae init: add --submodules to sync manifest submodules
repo sync can sync submodules via the --fetch-submodules option.
However, if the manifest repo has submodules, those will not be synced.
Having submodules in the manifest repo -- while not commonly done -- can
be useful for inheriting a manifest from another project using <include>
and layering changes on top of it.  In this way, you can avoid having to
deal with merge conflicts between your own manifests and the other
project's manifests (for example, if you're managing an Android fork).

Add a --submodule option to init that automatically syncs the submodules
in the manifest repo whenever the manifest repo changes.

Change-Id: I45d34f04517774c1462d7f233f482d1d81a332a8
Signed-off-by: Martin Kelly <mkelly@xevo.com>
2017-05-23 16:51:31 -07:00
Sebastian Schuberth
c9439facdd pre-auto-gc: Add support for Windows
Previously, this would always have exited with 1 on Windows, causing "git
gc --auto" to abort. Fix this by adding support for Windows.

Change-Id: Ie519b366a11b6b18b2d465e892e738de3f4bbc99
2017-05-03 11:26:21 +00:00
heping
3d7bbc9edf project.py: fix performance issue with --reference when the mirrored repository has many refs
Change-Id: Id0183903597f872eee80ca32a8050125b187a3d4
2017-04-12 20:04:47 +08:00
Marc Herbert
ffb4b89099 sync.py: report the remote URL on fatal git remote errors
repo can be configured to download from any number of remote git repos.
However when one fails repo doesn't report which one. Example:
Fatal: remote error: Daily ls-remote rate limit exceeded for IP xx.xx.xx.xx

TEST=repo init -q -u https://chromium.googlesource.com/chromiumos/manifest.git
  # Apply patch in ./.repo/repo/
  # Simulate a git remote error:
  sed -i -e 's#chromiumos/docs#chromiumos/XXdocs#' .repo/manifests/full.xml
  repo sync --quiet --force-sync docs
  # error message now shows the remote URL

Optional test tip: reduce the time.sleep(random(...)) in ./.repo/repo/project.py

Change-Id: I4509383b6a43a8e66064778e8ed612d8a735c8b6
2017-04-04 22:10:34 -07:00
Mike Frysinger
04071c1c72 manifest-format: fix EMPTY keyword usage
The keyword EMPTY doesn't use parens.

BUG=git-repo:140

Change-Id: I7cd28a09c401520a72e5c244a77d9d70385f1b61
2016-12-28 16:07:16 -05:00
Jonathan Nieder
7de8c5db78 Merge "init: Add no-tags and current branch options" 2016-12-09 02:33:42 +00:00
David Pursehouse
bb9c42cf1d Merge "Fix removing broken symlink in reference dir" 2016-12-06 07:51:01 +00:00
Naseer Ahmed
f4dda9a1be init: Add no-tags and current branch options
This avoids fetching tags and branches for huge manifests

Change-Id: I19c9724d75364440b881b297d42b906f541f73ff
2016-12-01 19:03:41 -05:00
David Pursehouse
b881d227f3 Merge "Add a check and more output to protect against invalid REPO_URLs" 2016-10-29 07:28:35 +00:00
Dan Willemsen
8e2d1d521e Merge "Fix checkout error when depth passed to repo init and revision is a sha1" 2016-10-28 19:30:45 +00:00
Sebastian Schuberth
27226e742d Add a check and more output to protect against invalid REPO_URLs
If you don't know that the url to git-repo itself can be overridden via
REPO_URL, it's hard to debug cases where REPO_URL is accidentally set to
another repository, e.g. inside a Jenkins CI job. What makes is even
harder is that the ".repo/repo" directory gets silently removed in such
cases as verifications fails, which makes it impossible to look at the
cloned files to understand the problem.

To better protect against such an issue, warn if the cloned git-repo
repository does not contain a top-level "repo" file, and state that the
".repo/repo" directory will be removed in case of a clone failure.

Change-Id: I697b4999205a5967910c0237772ccaada01e74d4
2016-10-28 14:43:02 +02:00
Aymen Bouaziz
6c5944606a Fix checkout error when depth passed to repo init and revision is a sha1
Currently, if direct fetch of a sha1 is not supported by git server and
depth option is used, we fallback on syncing the upstream branch by
ignoring depth option.

This fallback doesn't work in next 2 cases:
(1) upstream attribute is not specified in manifest
(2) depth option is passed to repo init command
    (not with clone-depth attribute in manifest)

This commit do the following:
- fixes (1) by updating condition used to apply fallback
  first we retry with depth set to None, then by syncing all branches
- fixes (2) by passing depth as argument of _RemoteFetch() method
  thus, its value is not set again to depth value passed to repo init
  command when applying fallback

Change-Id: Ifd6fffafc49ba229df624b0d7b64c83d47619d17
2016-10-28 14:29:57 +02:00
David Pursehouse
ae81c964b6 Merge "implement optional '--all' in the abandon command" 2016-10-28 07:52:31 +00:00
Dan Willemsen
e02c17c9ea Merge "_CheckDirReference: log actual error before suggesting --force-sync" 2016-10-28 06:49:07 +00:00
Joe Kilner
6e31079033 Add sso to list of known schemes for relative URLs
repo already special-cases sso:// URLs to behave similarly to https://
and rpc:// elsewhere in repo, but it forgot to do so here.

Noticed when trying to use relative URLs in a manifest obtained using
an sso:// URL.

Change-Id: Ia11469a09bbd6e444dbc4f22c82f9bbe9f5fd083
2016-10-27 15:56:38 -07:00
Marc Herbert
ec287902e6 _CheckDirReference: log actual error before suggesting --force-sync
A recent backward incompatible change created confusion and loss of
productivity and highlighted the very limited amount of information
provided when repo sync fails; merely recommending to --force-sync
and blow-up git repos without any hint as to why. The addition of
this basic _error(...) call would have provided a clue and will in
the future.

BUG=Issue 232
TEST=simulate a breakage similar to the ones reported at
  https://groups.google.com/a/chromium.org/forum/#!topic/chromium-os-dev/2-0oCy_CX5s
  cd .repo/projects/src/third_party/libapps.git/
  file info; rm info; ln -s wronglink info
  cd -
  repo sync src/third_party/libapps/
  # error message now shows the failure

Change-Id: Idd2f177a096f1ad686caa8c67cb361d594ccaa57
2016-10-27 12:58:26 -07:00
Andrew Wheeler
4d5bb68d58 status: add -q/--quiet option
The --quiet option reduces the output to just
a list of projects with modified workspaces (and
orphans if -o is specified)

A common use case is when performing a full-workspace
merge.  The integrator will kick-off a merge via:

    repo forall -c git merge <some tag>

And then produce a short list of conflicted projects via:

    repo status -q

The integrator can then iteratively fix and clean up all conficted
components.  The merge is complete when:

    repo status -q

    returns no output.

Change-Id: Ibbba8713eac35befd8287c95948874e23fd5c7e2
2016-10-17 15:24:09 -05:00
Kyunam.jo
2e14792a94 implement optional '--all' in the abandon command
when you want to delete all local branches, you should be find
all branches' name, and type them behind 'repo abandon' command.

Usage:
    repo abandon --all [<project>...]

Change-Id: I4d391f37fb9d89b8095488c585468eafc1a35f31
2016-10-17 02:29:42 +00:00
David Pursehouse
82f67987a3 Merge "sync: Fix semaphore release bug that causes thread 'leaks'" 2016-10-17 01:10:16 +00:00
Kyunam.jo
699bcd40be Removed duplication code in abandon.py
code about getting argument is duplicated.
so this line is removed

Change-Id: Id321b999c7dacdb403cd986cbf35f8db62efc157
2016-10-12 10:02:30 +09:00
Andrew Wheeler
7f1ccfbb7b sync: Fix semaphore release bug that causes thread 'leaks'
When repo syncs a manifest that utilizes multiple branches
in the same project, then the sync will use an extra
thread for each "duplicate".  For example, if
the manifest includes the project "foo" and "bar"
twice, then "repo sync -jN" will fetch with N+2 threads.

This is caused by _FetchHelper() releasing the thread semaphore
object each time it's called, even though _FetchProjectList()
may call this function multiple times within the scope of a
single thread.

Fix by moving the thread semaphore release to
_FetchProjectList(), which is only called once per thread
instance.

Change-Id: I1da78b145e09524d40457db5ca5c37d315432bd8
2016-10-11 14:10:34 -05:00
Dan Willemsen
eceeb1b1f5 Support broken symlinks when cleaning obsolete paths
When there's a symlink to a directory, os.walk still lists the symlink
in dirs, even if it isn't configured to follow symlinks. This will fail
the listdirs check if the symlink is broken (either before or during the
cleanup). So instead, check for directory symlinks and remove them using
os.remove.

Bug: Issue 231
Change-Id: I0ec45a26be566613a4a39bf694a3d9c6328481c2
2016-09-27 03:05:11 +00:00
Dan Willemsen
16889ba43d Revert "Repo: fall back to http, if ssh connection fails for http repos"
This reverts commit 488bf092d5.

Issue 230

Change-Id: I3a5725301f576e1a2ac499cb6daa631895115640
2016-09-22 16:40:27 +00:00
David Pursehouse
40d3952270 Merge "On project cleanup, don't remove nested projects" 2016-09-21 22:23:44 +00:00
Dan Willemsen
4350791e0d On project cleanup, don't remove nested projects
When there are nested projects in a manifest, like on AOSP right now:

<project path="build" name="platform/build" />
<project path="build/blueprint" name="platform/build/blueprint" />
<project path="build/kati" name="platform/build/kati" />
<project path="build/soong" name="platform/build/soong" />

And the top "build" project is removed (or renamed to remove the
nesting), repo just wipes away everything under build/ and re-creates
the projects that are still there. But it only checks to see if the
build/ project is dirty, so if there are dirty files in a nested
project, they'll just be blown away, and a fresh worktree checked out.

Instead, behave similarly to how `git clean -dxf` behaves and preserve
any subdirectories that have git repositories in them. This isn't as
strict as git -- it does not check to see if the '.git' entry is a
readable gitdir, just whether an entry named '.git' exists.

If it encounters any errors removing files, we'll print them all out to
stderr and tell the user that we were unable to clean up the obsolete
project, that they should clean it up manually, then sync again.

Change-Id: I2f6a7dd205a8e0b7590ca5369e9b0ba21d5a6f77
2016-09-20 17:16:12 -07:00
Steve Rae
d648045366 implement optional 'pushurl' in the manifest file
Allow the 'remote' element in the manifest file to define an optional
'pushurl' attribute which is passed into the .git/config file.

Change-Id: If342d299d371374aedc4440645798888869c9714
Signed-off-by: Steve Rae <steve.rae@raedomain.com>
2016-09-20 15:31:20 +00:00
David Pursehouse
628456833a Merge "Repo: improve error detection for new ssh connections" 2016-09-20 08:06:12 +00:00
David Pursehouse
2aa61d0bc8 Merge "Repo: fall back to http, if ssh connection fails for http repos" 2016-09-20 08:05:57 +00:00
David Pursehouse
4aed6f8c7d Merge "Replace pylint with pyflakes/flake8" 2016-09-20 06:57:16 +00:00
David Pursehouse
01b7d758d5 Merge "When syncing a project with a shared object store, disable automatic pruning." 2016-09-14 20:48:24 +00:00
David Pursehouse
267ac57361 Merge "repo: add comment for updating maintainer keys" 2016-09-14 14:06:42 +00:00
David Pursehouse
bb5b1a076b Replace pylint with pyflakes/flake8
pylint reports a lot of warnings, but many of them are false positive,
and it's difficult to configure it. It also seems that for some reason
the included config file is not working well with the latest version.

Update the documentation to recommend using pyflakes and flake8 instead
of pylint. Remove the pylint config and add a basic flake8 config with
minimum settings:

- Maximum line length 80 columns
- Ignore warnings about indentation (repo uses 2 rather than expected 4)
- Ignore warnings about import placement

In this commit no code cleanup is done, and it's expected that most of
the files will throw up quite a few warnings, at least for flake8. These
can be cleaned up in follow-up commits.

The existing pylint suppression comments are left as-is. These will be
helpful when cleaning up pyflakes warnings later.

Change-Id: I2f7cb4340266ed07cc973ca6483b8f09d66a765b
2016-09-14 09:49:02 +02:00
David Pursehouse
e01ee026e6 Merge "Fix submodule checkout error when using sync-s option" 2016-09-14 07:45:50 +00:00
Mike Frysinger
e4433653db repo: add comment for updating maintainer keys
Change-Id: Ic1e7557f9597234033561ab9fb3104b87e30015e
2016-09-14 01:28:30 -04:00
David Pursehouse
d9de945d8a Merge "upload: short circuit when nothing is pending" 2016-09-14 05:20:36 +00:00
Gabe Black
2ff302929c When syncing a project with a shared object store, disable automatic pruning.
The shared object stores confuse git and make it throw away objects which are
still in use. We'll avoid that problem by disabling automatic pruning on those
projects, but there's nothing preventing a user from changing the config back
or pruning a repository manually.

BUG=chromium:375945
TEST=Ran repo sync on fresh ChromeOS checkout, starting with a branch of repo
with this change. Verified that the kernel projects and no others were
identified as having shared object stores, and that repo successfully disabled
automatic pruning in their configs. Re-enabled pruning and ran repo sync just
on one of the kernel directories. Verified that pruning was re-disabled as a
result.

Change-Id: I728ed5b06f0087aeb5a23ba8f5410a7cd10af5b0
2016-09-14 00:19:44 -04:00
Vadim Bendebury
e5c0ea0a95 Consider local project to be default for 'repo start'
The requirement to explicitly specify the local project when starting
a new repo branch is somewhat counter intuitive.

This patch uses the current directory's git tree as the default
project.

Tested by running

  'repo start <name>'

observed that the result is the same as if running

  'repo start <name> .'

Change-Id: If106caa801b4cd5ba70dbe8354a227d59f100aa3
2016-09-14 00:17:45 -04:00
Mike Frysinger
163a3be18b upload: short circuit when nothing is pending
When nothing is pending, most of this code is already short-circuited.
Hoist the single check up to make this more obvious/slightly faster.

Change-Id: Iec3a7e08eacd23a7c5f964900d5776bf5252c804
2016-09-14 00:16:37 -04:00
David Pursehouse
7a77c16d37 Update mailmap
Order the entries alphabetically and add a couple more.

Change-Id: I8d98e8a5a1dd6868b566a42428030d2040b8af7a
2016-09-02 11:12:28 +09:00
Timo Lotterbach
488bf092d5 Repo: fall back to http, if ssh connection fails for http repos
if a gerrit server has ssh and https access enabled, but user access
(for some users) is limited to https, 'repo upload' command will fail
for them.

Gerrit returns a ssh configuration (gerrit/ssh_info), that does not work
for users limited to https.

With this patch repo will test, if the returned ssh configuration from
gerrit/ssh_info is working. if not, it will fall back to https for upload.

Change-Id: If98f472e994f350bf71f35610cd649b163f1ab33
2016-08-30 07:35:03 +00:00
Timo Lotterbach
05dc46b0e3 Repo: improve error detection for new ssh connections
this check can only detect errors that happen within 1 sec after launching
ssh. But this is typically enough to catch configuration issues like
'connection refused' or 'authentication failed'.

Change-Id: I00b6f62d4c2889b1faa6c820e49a198554c92795
2016-08-30 07:34:33 +00:00
Dan Willemsen
39252ba028 gitc: Lower concurrent ls-projects requests
Too many requests at the same time is causing 502 errors.

Change-Id: Ic8fbb2fbb7fb6014733fa5be018d2dc02472f704
2016-08-23 14:19:00 -07:00
Jonathan Nieder
71e4cea6de RepoHook: do not list options twice during hash based approval
Instead of

  Do you want to allow this script to run (yes/yes-never-ask-again/NO)? (yes/always/NO)?

ask

  Do you want to allow this script to run (yes/always/NO)?

Change-Id: I5f5a2d0e88086a8d85e54fb8623a62d74a20956a
Signed-off-by: Jonathan Nieder <jrn@google.com>
2016-08-18 17:06:26 -07:00
David Pursehouse
c4c2b066d1 Increment the wrapper version
There have been a number of changes in the repo wrapper since the last
increment that was done in fee390ee:

- 9711a98 init: Add --no-clone-bundle option
- 631d0ec Support non-ASCII GNUPGHOME environment variable
- 4088eb4 repo: Cleaned up pylint/pep8 violations
- 5553628 repo: Add check of REPO_URL env variable
- 745b4ad Fix gitc-init behavior
- d3ddcdb Ignore clone.bundle on HTTP 501, i.e. Not Implemented

Change-Id: I3f763ef0ec2df2d726dff429021b48ad474148f1
2016-08-17 13:58:17 +09:00
David Pursehouse
6a0a3648f1 Merge "init: Add --no-clone-bundle option" 2016-08-17 04:57:28 +00:00
David Pursehouse
6118faa118 Merge "init: Respect --quiet option when synching manifest repository" 2016-08-17 04:04:14 +00:00
David Pursehouse
183c52ab02 Merge "project: Set config option to skip lfs smudge filter" 2016-08-17 01:00:12 +00:00