Compare commits

..

19 Commits

Author SHA1 Message Date
4a478edb44 init, sync: fixed flake8 warnings.
Tested:
+ run_tests
+ flake8 subcmds/init.py
+ flake8 subcmds/sync.py

Change-Id: Ie337481d8a210bfc49b0745f75c05a308a0e74d3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/324155
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-11-18 16:22:40 +00:00
6bd89aa657 superproject: Inherit --no-use-superproject with --mirror option.
init.py
+ Similar to opt.archive, gave an error if --mirror option is
  used with --use-superproject.

sync.py
+ Defaulted to --no-use-superproject if manifest is a mirror or
  archive (similar to error at line# 1067).

Tested:
+ run_tests
+ flake8 (will fix known errors in another CL).

$ repo_dev init -u sso://googleplex-android.git.corp.google.com/platform/manifest --use-superproject --mirror
Usage: repo init [options] [manifest url]

main.py: error: --mirror and --use-superproject cannot be used together.

+ repo init and repo sync with --mirror and without --mirror
  options.
  $ repo_dev init -u https://android.googlesource.com/platform/manifest
  $ repo_dev sync
    ...superproject.git: Initial setup for superproject completed.

+ With --mirror option, verfied there are no exceptions in git_superproject.py

Bug: [google internal] b/206537893
Change-Id: I059f20e76f0ab36f0587f29779bb53ede4663bd4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323955
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-11-18 01:27:41 +00:00
9c1fc5bc5d sync: Handle tag ref in "upstream" field
repo sync only handles a git tag properly when it is in the "revision"
field. However, "revision locked manifests" (`repo manifest
--revision-as-HEAD`) specifies the tag in the "upstream" field. The
issue is that this tag is not fetched. Only the commit that the tag
points to is fetched. This cases issues as
self._CheckForImmutableRevision() runs and comes to the conclusion that
the tag was changed while in fact, it was just not fetched. This causes
a full sync.

File docs/manifest-format.md, section Element-project:
> Attribute upstream: Name of the Git ref in which a sha1 can be found.
Used when syncing a revision locked manifest in -c mode to avoid having
to sync the entire ref space. Project elements not setting their own
upstream will inherit this value.

Change-Id: I0507d3a5f30aee8920a9f820bafedb48dd5db554
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323620
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Robin Schneider <ypid@riseup.net>
2021-11-16 20:43:57 +00:00
333c0a499b project: init hooks in objdir only
objdir is the .repo/project-objects/ dir based on the remote path.
gitdir is the .repo/projects/ dir based on the local source checkout
path.  When we setup the gitdir, we symlink "hooks" to the one in the
objdir.  But when we go to initialize the hooks, we do it via gitdir.
There is a 1-to-many mapping from project-objects to projects, so
initializing via gitdir can be repetitive.  Collapse the hook init
logic to the objdir init path.

Change-Id: I828fca60ce6e125d6706c709cdb2797faa40aa50
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323815
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-15 19:50:18 +00:00
fdeb20f43f sync: link the internal-fs-layout doc into checkouts
This should make it easy to discover for people poking around .repo/.

Change-Id: Ie5051551f25127c0592df5e36efba7bb2263e5d4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323701
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-15 01:39:53 +00:00
bf40957b38 git-review: add config file
This is used by the `git review` tool that some people use.

Change-Id: I8dac4e1dad155109a05181deaec61e1a74857b1f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323698
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-15 01:39:36 +00:00
f9e81c922d SUBMITTING_PATCHES: link to commit message style docs
Change-Id: I2090ebc43fc1c816b941a53dd89dbedf7bc61289
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323696
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jack Neus <jackneus@google.com>
2021-11-15 01:39:16 +00:00
e6601067ed man: refresh pages
Change-Id: I3f2c3ad77c16a76276bba2954887ab9e7605661c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323516
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-12 17:30:45 +00:00
3001d6a426 help: fix grammar in help text
Bug: https://crbug.com/gerrit/14838
Change-Id: Ic5000921ba9a1baa086153630ebbb429e3d17642
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323515
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-12 17:30:32 +00:00
00c5ea3787 Fix typo for ValueError
which will cause error log like below:
NameError: name 'ValueErrorl' is not defined

Change-Id: I388886b7cf6d700e224c3847b7ba4ba4fe9c041d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323015
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: 彭杨益 <pyy101727@gmail.com>
2021-11-07 02:32:08 +00:00
0531a623e1 sync: make --prune the default
If a remote deletes a ref, and it points to an object that doesn't
exist locally, we can get into a bad state, and the only way for the
user to recover is to run `repo sync --prune` (and to know that is
the option they need).  The error message is not helpful:

fatal: bad object refs/remotes/cros/firmware-zork-13421.B-master
error: https://chromium.googlesource.com/chromiumos/platform/ec did not send all necessary objects

This situation can also come up when the remote renames refs in a
UNIX FS incompatible way.  For example, replacing refs/heads/foo
with refs/heads/foo/bar.

Also add a --no-prune option for users to disable the behavior.

Bug: https://issuetracker.google.com/203366450
Change-Id: Icf45d838a10938feb091d29800f7e49240830ec3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322956
Reviewed-by: Andrew Lamb <andrewlamb@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-05 20:13:30 +00:00
2273f46cb3 sync: fix --tags option
This has been broken since it was added where --tags was actually
the same as --no-tags.  Oddly, it was copied from init where the
logic is correct.

Bug: https://crbug.com/gerrit/12401
Change-Id: I15b89da1a655176a11bebc22573b25c728055328
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322955
Reviewed-by: Andrew Lamb <andrewlamb@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-05 20:13:02 +00:00
7b9b251a5e project: fix format string in error message
BUG=None

Change-Id: I0b195fd919c6db8cb3547e8d6f4c733f2bd4a535
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322735
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
2021-11-05 16:59:14 +00:00
6251729cb4 superproject: added 'implies -c' in the help of --use-superproject option.
sync.py: deleted unused import errno.

Tested:
$ ./run_tests
$ flake8 repo subcmds/sync.py

Bug: https://crbug.com/gerrit/15208
Change-Id: I2bb3098f5602ded3861e000100766041ad93b53d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322555
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-11-01 22:15:53 +00:00
11b30b91df Support more url schemes for getting standalone manifest
urllib.requests.urlopen also supports file, so call it unless the
scheme is 'gs'.  This adds http, https, and ftp support.

Change-Id: I3f215c3ebd8e6dee29ba14c7e79ed99d37287109
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322095
Reviewed-by: Michael Kelly <mkelly@arista.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Matt Story <mstory@arista.com>
2021-10-27 13:20:35 +00:00
198838599c fetch: Fix stderr handling for gsutil
Previously gsutil stderr was getting piped into stdout, which
yields bad results if there are non-fatal warnings in stderr.

Additionally, we should fail outright if gsutil fails (by adding
`check = True`) rather than fail later on when we try to sync to
a manifest that is in fact just a stderr dump.

BUG=none
TEST=manual runs with bad gs urls

Change-Id: Id71791d0c3f180bd0601ef2c783a8e8e4afa8f59
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/321935
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-26 22:18:28 +00:00
282d0cae89 ssh: handle FileNotFoundError errors
If ssh isn't installed, it throws a distinct error we have to catch.

Bug: https://crbug.com/gerrit/15196
Change-Id: I0660e842c304ce7575f5cb100894d05fd65f9454
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322055
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-10-26 16:18:45 +00:00
03ff276cd7 sync: properly handle standalone manifests for sync command
sync should not attempt to sync the manifest project if it was
created from a standalone manifest. The current work around is to
run sync with --nmu.

BUG=none
TEST=manual runs

Change-Id: I2e121af0badf9642143e77c7af89d1c2d993b0f3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/321195
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-15 17:20:00 +00:00
4ee4a45d03 subcmds/sync: Use pack-refs instead of gc for redundant gitdirs.
Previously `git gc` was being run on every gitdir even when they shared
the same objects. Instead only call it once and use pack-refs for the
gitdirs that were not gc'ed.

Bug: https://crbug.com/gerrit/15113
Test: repo sync -j # and check that git pack-refs is called
Change-Id: Icff37ab3ec78cfb44391d8cc7f2d875991532320
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/320275
Tested-by: Allen Webb <allenwebb@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-14 12:27:12 +00:00
16 changed files with 152 additions and 77 deletions

5
.gitreview Normal file
View File

@ -0,0 +1,5 @@
[gerrit]
host=gerrit-review.googlesource.com
scheme=https
project=git-repo.git
defaultbranch=main

View File

@ -3,7 +3,7 @@
# Short Version
- Make small logical changes.
- Provide a meaningful commit message.
- [Provide a meaningful commit message][commit-message-style].
- Check for coding errors and style nits with flake8.
- Make sure all code is under the Apache License, 2.0.
- Publish your changes for review.
@ -26,10 +26,11 @@ yourself with the following relevant bits.
## Make separate commits for logically separate changes.
Unless your patch is really trivial, you should not be sending
out a patch that was generated between your working tree and your
commit head. Instead, always make a commit with complete commit
message and generate a series of patches from your repository.
Unless your patch is really trivial, you should not be sending out a patch that
was generated between your working tree and your commit head.
Instead, always make a commit with a complete
[commit message][commit-message-style] and generate a series of patches from
your repository.
It is a good discipline.
Describe the technical detail of the change(s).
@ -171,3 +172,6 @@ After you receive a Code-Review+2 from the maintainer, select the Verified
button on the gerrit page for the change. This verifies that you have tested
your changes and notifies the maintainer that they are ready to be submitted.
The maintainer will then submit your changes to the repository.
[commit-message-style]: https://chris.beams.io/posts/git-commit/

View File

@ -17,8 +17,10 @@
import subprocess
import sys
from urllib.parse import urlparse
from urllib.request import urlopen
def fetch_file(url):
def fetch_file(url, verbose=False):
"""Fetch a file from the specified source using the appropriate protocol.
Returns:
@ -29,13 +31,15 @@ def fetch_file(url):
cmd = ['gsutil', 'cat', url]
try:
result = subprocess.run(
cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
check=True)
if result.stderr and verbose:
print('warning: non-fatal error running "gsutil": %s' % result.stderr,
file=sys.stderr)
return result.stdout
except subprocess.CalledProcessError as e:
print('fatal: error running "gsutil": %s' % e.output,
print('fatal: error running "gsutil": %s' % e.stderr,
file=sys.stderr)
sys.exit(1)
if scheme == 'file':
with open(url[len('file://'):], 'rb') as f:
return f.read()
raise ValueError('unsupported url %s' % url)
with urlopen(url) as f:
return f.read()

View File

@ -352,7 +352,7 @@ class GitConfig(object):
Trace(': parsing %s', self.file)
with open(self._json) as fd:
return json.load(fd)
except (IOError, ValueErrorl):
except (IOError, ValueError):
platform_utils.remove(self._json, missing_ok=True)
return None

View File

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "September 2021" "repo gitc-init" "Repo Manual"
.TH REPO "1" "November 2021" "repo gitc-init" "Repo Manual"
.SH NAME
repo \- repo gitc-init - manual page for repo gitc-init
.SH SYNOPSIS
@ -31,10 +31,6 @@ manifest branch or revision (use HEAD for default)
\fB\-m\fR NAME.xml, \fB\-\-manifest\-name\fR=\fI\,NAME\/\fR.xml
initial manifest file
.TP
\fB\-\-standalone\-manifest\fR
download the manifest as a static file rather then
create a git checkout of the manifest repo
.TP
\fB\-g\fR GROUP, \fB\-\-groups\fR=\fI\,GROUP\/\fR
restrict manifest projects to ones with specified
group(s) [default|all|G1,G2,G3|G4,\-G5,\-G6]
@ -45,6 +41,10 @@ platform group [auto|all|none|linux|darwin|...]
.TP
\fB\-\-submodules\fR
sync any submodules associated with the manifest repo
.TP
\fB\-\-standalone\-manifest\fR
download the manifest as a static file rather then
create a git checkout of the manifest repo
.SS Manifest (only) checkout options:
.TP
\fB\-\-current\-branch\fR
@ -96,7 +96,8 @@ filter for use with \fB\-\-partial\-clone\fR [default:
blob:none]
.TP
\fB\-\-use\-superproject\fR
use the manifest superproject to sync projects
use the manifest superproject to sync projects;
implies \fB\-c\fR
.TP
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects

View File

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "September 2021" "repo init" "Repo Manual"
.TH REPO "1" "November 2021" "repo init" "Repo Manual"
.SH NAME
repo \- repo init - manual page for repo init
.SH SYNOPSIS
@ -31,10 +31,6 @@ manifest branch or revision (use HEAD for default)
\fB\-m\fR NAME.xml, \fB\-\-manifest\-name\fR=\fI\,NAME\/\fR.xml
initial manifest file
.TP
\fB\-\-standalone\-manifest\fR
download the manifest as a static file rather then
create a git checkout of the manifest repo
.TP
\fB\-g\fR GROUP, \fB\-\-groups\fR=\fI\,GROUP\/\fR
restrict manifest projects to ones with specified
group(s) [default|all|G1,G2,G3|G4,\-G5,\-G6]
@ -45,6 +41,10 @@ platform group [auto|all|none|linux|darwin|...]
.TP
\fB\-\-submodules\fR
sync any submodules associated with the manifest repo
.TP
\fB\-\-standalone\-manifest\fR
download the manifest as a static file rather then
create a git checkout of the manifest repo
.SS Manifest (only) checkout options:
.TP
\fB\-c\fR, \fB\-\-current\-branch\fR
@ -96,7 +96,8 @@ filter for use with \fB\-\-partial\-clone\fR [default:
blob:none]
.TP
\fB\-\-use\-superproject\fR
use the manifest superproject to sync projects
use the manifest superproject to sync projects;
implies \fB\-c\fR
.TP
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects

View File

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2021" "repo manifest" "Repo Manual"
.TH REPO "1" "November 2021" "repo manifest" "Repo Manual"
.SH NAME
repo \- repo manifest - manual page for repo manifest
.SH SYNOPSIS
@ -161,6 +161,7 @@ CDATA #IMPLIED>
<!ELEMENT extend\-project EMPTY>
<!ATTLIST extend\-project name CDATA #REQUIRED>
<!ATTLIST extend\-project path CDATA #IMPLIED>
<!ATTLIST extend\-project dest\-path CDATA #IMPLIED>
<!ATTLIST extend\-project groups CDATA #IMPLIED>
<!ATTLIST extend\-project revision CDATA #IMPLIED>
<!ATTLIST extend\-project remote CDATA #IMPLIED>
@ -174,8 +175,9 @@ CDATA #IMPLIED>
<!ATTLIST repo\-hooks enabled\-list CDATA #REQUIRED>
.IP
<!ELEMENT superproject EMPTY>
<!ATTLIST superproject name CDATA #REQUIRED>
<!ATTLIST superproject remote IDREF #IMPLIED>
<!ATTLIST superproject name CDATA #REQUIRED>
<!ATTLIST superproject remote IDREF #IMPLIED>
<!ATTLIST superproject revision CDATA #IMPLIED>
.IP
<!ELEMENT contactinfo EMPTY>
<!ATTLIST contactinfo bugurl CDATA #REQUIRED>
@ -385,6 +387,11 @@ original manifest.
Attribute `path`: If specified, limit the change to projects checked out at the
specified path, rather than all projects with the given name.
.PP
Attribute `dest\-path`: If specified, a path relative to the top directory of the
repo client where the Git working directory for this project should be placed.
This is used to move a project in the checkout by overriding the existing `path`
setting.
.PP
Attribute `groups`: List of additional groups to which this project belongs.
Same syntax as the corresponding element of `project`.
.PP
@ -477,6 +484,10 @@ project](#element\-project) for more information.
Attribute `remote`: Name of a previously defined remote element. If not supplied
the remote given by the default element is used.
.PP
Attribute `revision`: Name of the Git branch the manifest wants to track for
this superproject. If not supplied the revision given by the remote element is
used if applicable, else the default element is used.
.PP
Element contactinfo
.PP
*** *Note*: This is currently a WIP. ***

View File

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2021" "repo smartsync" "Repo Manual"
.TH REPO "1" "November 2021" "repo smartsync" "Repo Manual"
.SH NAME
repo \- repo smartsync - manual page for repo smartsync
.SH SYNOPSIS
@ -80,7 +80,8 @@ password to authenticate with the manifest server
fetch submodules from server
.TP
\fB\-\-use\-superproject\fR
use the manifest superproject to sync projects
use the manifest superproject to sync projects;
implies \fB\-c\fR
.TP
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects
@ -89,7 +90,7 @@ disable use of manifest superprojects
fetch tags
.TP
\fB\-\-no\-tags\fR
don't fetch tags
don't fetch tags (default)
.TP
\fB\-\-optimized\-fetch\fR
only fetch projects fixed to sha1 if revision does not
@ -100,6 +101,10 @@ number of times to retry fetches on transient errors
.TP
\fB\-\-prune\fR
delete refs that no longer exist on the remote
(default)
.TP
\fB\-\-no\-prune\fR
do not delete refs that no longer exist on the remote
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR

View File

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2021" "repo sync" "Repo Manual"
.TH REPO "1" "November 2021" "repo sync" "Repo Manual"
.SH NAME
repo \- repo sync - manual page for repo sync
.SH SYNOPSIS
@ -80,7 +80,8 @@ password to authenticate with the manifest server
fetch submodules from server
.TP
\fB\-\-use\-superproject\fR
use the manifest superproject to sync projects
use the manifest superproject to sync projects;
implies \fB\-c\fR
.TP
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects
@ -89,7 +90,7 @@ disable use of manifest superprojects
fetch tags
.TP
\fB\-\-no\-tags\fR
don't fetch tags
don't fetch tags (default)
.TP
\fB\-\-optimized\-fetch\fR
only fetch projects fixed to sha1 if revision does not
@ -100,6 +101,10 @@ number of times to retry fetches on transient errors
.TP
\fB\-\-prune\fR
delete refs that no longer exist on the remote
(default)
.TP
\fB\-\-no\-prune\fR
do not delete refs that no longer exist on the remote
.TP
\fB\-s\fR, \fB\-\-smart\-sync\fR
smart sync using manifest from the latest known good

View File

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2021" "repo" "Repo Manual"
.TH REPO "1" "November 2021" "repo" "Repo Manual"
.SH NAME
repo \- repository management tool built on top of git
.SH SYNOPSIS
@ -43,7 +43,7 @@ filename of event log to append timeline to
.TP
\fB\-\-git\-trace2\-event\-log\fR=\fI\,GIT_TRACE2_EVENT_LOG\/\fR
directory to write git trace2 event log to
.SS "The complete list of recognized repo commands are:"
.SS "The complete list of recognized repo commands is:"
.TP
abandon
Permanently abandon a development branch

View File

@ -2044,8 +2044,11 @@ class Project(object):
if current_branch_only:
if self.revisionExpr.startswith(R_TAGS):
# this is a tag and its sha1 value should never change
# This is a tag and its commit id should never change.
tag_name = self.revisionExpr[len(R_TAGS):]
elif self.upstream and self.upstream.startswith(R_TAGS):
# This is a tag and its commit id should never change.
tag_name = self.upstream[len(R_TAGS):]
if is_sha1 or tag_name is not None:
if self._CheckForImmutableRevision():
@ -2440,7 +2443,7 @@ class Project(object):
if quiet:
cmd.append('-q')
if GitCommand(self, cmd).Wait() != 0:
raise GitError('%s submodule update --init --recursive %s ' % self.name)
raise GitError('%s submodule update --init --recursive ' % self.name)
def _Rebase(self, upstream, onto=None):
cmd = ['rebase']
@ -2466,6 +2469,8 @@ class Project(object):
os.makedirs(self.objdir)
self.bare_objdir.init()
self._UpdateHooks(quiet=quiet)
if self.use_git_worktrees:
# Enable per-worktree config file support if possible. This is more a
# nice-to-have feature for users rather than a hard requirement.
@ -2526,8 +2531,6 @@ class Project(object):
_lwrite(os.path.join(self.gitdir, 'objects/info/alternates'),
os.path.join(ref_dir, 'objects') + '\n')
self._UpdateHooks(quiet=quiet)
m = self.manifest.manifestProject.config
for key in ['user.name', 'user.email']:
if m.Has(key, include_defaults=False):
@ -2543,11 +2546,11 @@ class Project(object):
raise
def _UpdateHooks(self, quiet=False):
if os.path.exists(self.gitdir):
if os.path.exists(self.objdir):
self._InitHooks(quiet=quiet)
def _InitHooks(self, quiet=False):
hooks = platform_utils.realpath(self._gitdir_path('hooks'))
hooks = platform_utils.realpath(os.path.join(self.objdir, 'hooks'))
if not os.path.exists(hooks):
os.makedirs(hooks)
for stock_hook in _ProjectHooks():
@ -2831,9 +2834,6 @@ class Project(object):
'for other options.')
return 'filesystem must support symlinks'
def _gitdir_path(self, path):
return platform_utils.realpath(os.path.join(self.gitdir, path))
def _revlist(self, *args, **kw):
a = []
a.extend(args)

2
repo
View File

@ -372,7 +372,7 @@ def InitParser(parser, gitc_init=False):
help='filter for use with --partial-clone '
'[default: %default]')
group.add_option('--use-superproject', action='store_true', default=None,
help='use the manifest superproject to sync projects')
help='use the manifest superproject to sync projects; implies -c')
group.add_option('--no-use-superproject', action='store_false',
dest='use_superproject',
help='disable use of manifest superprojects')

3
ssh.py
View File

@ -52,6 +52,9 @@ def version():
"""return ssh version as a tuple"""
try:
return _parse_ssh_version()
except FileNotFoundError:
print('fatal: ssh not installed', file=sys.stderr)
sys.exit(1)
except subprocess.CalledProcessError:
print('fatal: unable to detect ssh version', file=sys.stderr)
sys.exit(1)

View File

@ -53,7 +53,7 @@ Displays detailed usage information about a command.
self.PrintAllCommandsBody()
def PrintAllCommandsBody(self):
print('The complete list of recognized repo commands are:')
print('The complete list of recognized repo commands is:')
commandNames = list(sorted(all_commands))
self._PrintCommands(commandNames)
print("See 'repo help <command>' for more information on a "

View File

@ -15,7 +15,6 @@
import os
import platform
import re
import subprocess
import sys
import urllib.parse
@ -132,8 +131,8 @@ to update the working directory files.
'cannot be re-initialized without --manifest-url/-u')
sys.exit(1)
if opt.standalone_manifest or (
was_standalone_manifest and opt.manifest_url):
if opt.standalone_manifest or (was_standalone_manifest and
opt.manifest_url):
m.config.ClearCache()
if m.gitdir and os.path.exists(m.gitdir):
platform_utils.rmtree(m.gitdir)
@ -298,7 +297,7 @@ to update the working directory files.
if standalone_manifest:
if is_new:
manifest_name = 'default.xml'
manifest_data = fetch.fetch_file(opt.manifest_url)
manifest_data = fetch.fetch_file(opt.manifest_url, verbose=opt.verbose)
dest = os.path.join(m.worktree, manifest_name)
os.makedirs(os.path.dirname(dest), exist_ok=True)
with open(dest, 'wb') as f:
@ -478,11 +477,16 @@ to update the working directory files.
# Check this here, else manifest will be tagged "not new" and init won't be
# possible anymore without removing the .repo/manifests directory.
if opt.archive and opt.mirror:
self.OptionParser.error('--mirror and --archive cannot be used together.')
if opt.mirror:
if opt.archive:
self.OptionParser.error('--mirror and --archive cannot be used '
'together.')
if opt.use_superproject is not None:
self.OptionParser.error('--mirror and --use-superproject cannot be '
'used together.')
if opt.standalone_manifest and (
opt.manifest_branch or opt.manifest_name != 'default.xml'):
if opt.standalone_manifest and (opt.manifest_branch or
opt.manifest_name != 'default.xml'):
self.OptionParser.error('--manifest-branch and --manifest-name cannot'
' be used with --standalone-manifest.')

View File

@ -12,7 +12,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import errno
import functools
import http.cookiejar as cookielib
import io
@ -235,24 +234,25 @@ later is required to fix a server side protocol bug.
dest='fetch_submodules', action='store_true',
help='fetch submodules from server')
p.add_option('--use-superproject', action='store_true',
help='use the manifest superproject to sync projects')
help='use the manifest superproject to sync projects; implies -c')
p.add_option('--no-use-superproject', action='store_false',
dest='use_superproject',
help='disable use of manifest superprojects')
p.add_option('--tags',
action='store_false',
p.add_option('--tags', action='store_true',
help='fetch tags')
p.add_option('--no-tags',
dest='tags', action='store_false',
help="don't fetch tags")
help="don't fetch tags (default)")
p.add_option('--optimized-fetch',
dest='optimized_fetch', action='store_true',
help='only fetch projects fixed to sha1 if revision does not exist locally')
p.add_option('--retry-fetches',
default=0, action='store', type='int',
help='number of times to retry fetches on transient errors')
p.add_option('--prune', dest='prune', action='store_true',
help='delete refs that no longer exist on the remote')
p.add_option('--prune', action='store_true',
help='delete refs that no longer exist on the remote (default)')
p.add_option('--no-prune', dest='prune', action='store_false',
help='do not delete refs that no longer exist on the remote')
if show_smart:
p.add_option('-s', '--smart-sync',
dest='smart_sync', action='store_true',
@ -448,8 +448,8 @@ later is required to fix a server side protocol bug.
else:
pm.update(inc=0, msg='warming up')
chunksize = 4
with multiprocessing.Pool(
jobs, initializer=self._FetchInitChild, initargs=(ssh_proxy,)) as pool:
with multiprocessing.Pool(jobs, initializer=self._FetchInitChild,
initargs=(ssh_proxy,)) as pool:
results = pool.imap_unordered(
functools.partial(self._FetchProjectList, opt),
projects_list,
@ -605,7 +605,7 @@ later is required to fix a server side protocol bug.
pm = Progress('Garbage collecting', len(projects), delay=False, quiet=opt.quiet)
pm.update(inc=0, msg='prescan')
gc_gitdirs = {}
tidy_dirs = {}
for project in projects:
# Make sure pruning never kicks in with shared projects.
if (not project.use_git_worktrees and
@ -623,17 +623,28 @@ later is required to fix a server side protocol bug.
file=sys.stderr)
project.config.SetString('gc.pruneExpire', 'never')
project.config.SetString('gc.autoDetach', 'false')
gc_gitdirs[project.gitdir] = project.bare_git
pm.update(inc=len(projects) - len(gc_gitdirs), msg='warming up')
# Only call git gc once per objdir, but call pack-refs for the remainder.
if project.objdir not in tidy_dirs:
tidy_dirs[project.objdir] = (
True, # Run a full gc.
project.bare_git,
)
elif project.gitdir not in tidy_dirs:
tidy_dirs[project.gitdir] = (
False, # Do not run a full gc; just run pack-refs.
project.bare_git,
)
cpu_count = os.cpu_count()
jobs = min(self.jobs, cpu_count)
if jobs < 2:
for bare_git in gc_gitdirs.values():
for (run_gc, bare_git) in tidy_dirs.values():
pm.update(msg=bare_git._project.name)
bare_git.gc('--auto')
if run_gc:
bare_git.gc('--auto')
else:
bare_git.pack_refs()
pm.end()
return
@ -642,11 +653,14 @@ later is required to fix a server side protocol bug.
threads = set()
sem = _threading.Semaphore(jobs)
def GC(bare_git):
def tidy_up(run_gc, bare_git):
pm.start(bare_git._project.name)
try:
try:
bare_git.gc('--auto', config=config)
if run_gc:
bare_git.gc('--auto', config=config)
else:
bare_git.pack_refs(config=config)
except GitError:
err_event.set()
except Exception:
@ -656,11 +670,11 @@ later is required to fix a server side protocol bug.
pm.finish(bare_git._project.name)
sem.release()
for bare_git in gc_gitdirs.values():
for (run_gc, bare_git) in tidy_dirs.values():
if err_event.is_set() and opt.fail_fast:
break
sem.acquire()
t = _threading.Thread(target=GC, args=(bare_git,))
t = _threading.Thread(target=tidy_up, args=(run_gc, bare_git,))
t.daemon = True
threads.add(t)
t.start()
@ -753,7 +767,7 @@ later is required to fix a server side protocol bug.
with open(copylinkfile_path, 'rb') as fp:
try:
old_copylinkfile_paths = json.load(fp)
except:
except Exception:
print('error: %s is not a json formatted file.' %
copylinkfile_path, file=sys.stderr)
platform_utils.remove(copylinkfile_path)
@ -915,6 +929,9 @@ later is required to fix a server side protocol bug.
if None in [opt.manifest_server_username, opt.manifest_server_password]:
self.OptionParser.error('both -u and -p must be given')
if opt.prune is None:
opt.prune = True
def Execute(self, opt, args):
if opt.jobs:
self.jobs = opt.jobs
@ -955,18 +972,24 @@ later is required to fix a server side protocol bug.
file=sys.stderr)
mp = self.manifest.manifestProject
mp.PreSync()
is_standalone_manifest = mp.config.GetString('manifest.standalone')
if not is_standalone_manifest:
mp.PreSync()
if opt.repo_upgraded:
_PostRepoUpgrade(self.manifest, quiet=opt.quiet)
if not opt.mp_update:
print('Skipping update of local manifest project.')
else:
elif not is_standalone_manifest:
self._UpdateManifestProject(opt, mp, manifest_name)
load_local_manifests = not self.manifest.HasLocalManifests
use_superproject = git_superproject.UseSuperproject(opt, self.manifest)
if self.manifest.IsMirror or self.manifest.IsArchive:
# Don't use superproject, because we have no working tree.
use_superproject = False
print('Defaulting to no-use-superproject because there is no working tree.')
superproject_logging_data = {
'superproject': use_superproject,
'haslocalmanifests': bool(self.manifest.HasLocalManifests),
@ -1102,6 +1125,15 @@ later is required to fix a server side protocol bug.
def _PostRepoUpgrade(manifest, quiet=False):
# Link the docs for the internal .repo/ layout for people
link = os.path.join(manifest.repodir, 'internal-fs-layout.md')
if not platform_utils.islink(link):
target = os.path.join('repo', 'docs', 'internal-fs-layout.md')
try:
platform_utils.symlink(target, link)
except Exception:
pass
wrapper = Wrapper()
if wrapper.NeedSetupGnuPG():
wrapper.SetupGnuPG(quiet)