Compare commits

...

24 Commits

Author SHA1 Message Date
c3d2f2b76f Ignore /clone.bundle on HTTP 401, 403 and 404
401: Unauthorized, authentication may be required. This is usually
     handled internally by the HTTP client in Python. If it reaches
     our code in repo, the Python HTTP client didn't find a password
     in ~/.netrc that it could use.

403: Authentication was supplied, but is incorrect. It might be
     that the CDN doesn't want to offer this clone.bundle file
     to the client, but the Git fetch operation would still be
     successful. This might arise if branch level read controls
     were used in Gerrit Code Review and the /clone.bundle file
     contained branches not visible to the client.

404: The server has no /clone.bundle file available.

In all of these cases, sliently ignore the /clone.bundle file HTTP
error and let the Git operation take over.

Change-Id: I1787f3cac86b017351952bbb793fe5874d83c72b
2012-03-22 14:18:40 -07:00
cd7c5deca0 Do not change branch.foo.merge in case of manifest sync
In case of manifest/smart sync repo changes ".merge" config
option from branch to SHA. Doing 'repo upload' fails as
repo tries to upload to a remote branch that looks like SHA
(e.g. refs/for/23423423423423423423423)

Do not update the .merge in case if revision is SHA.

Change-Id: I9139708fa17f21eec5a7e23c3255333626bf529e
2012-03-20 14:11:56 -07:00
e02ac0af2e sync: --no-clone-bundle disables the clone bundle support
Change-Id: Ia9ed7da8451b273c1be620c3dd0dcad777b29096
2012-03-14 15:38:28 -07:00
898e12a2d9 Permit - in URL schemes for special URLs
Clients might be using their own special git-remote-* helper that
has a hypen in its name. Permit - in the scheme part of the URL
when trying to decide if it is an SSH URL and assume it is *not*
SSH if the URL matches "foo-bar://" style.

Change-Id: I7ba2d810a614f6e605a441d5972902c4a14e73fd
2012-03-14 15:28:22 -07:00
ae0a36c9a5 Add support for Apache Digest authentication for repo init.
repo tool supports only Basic authentication for now. For those
who want to use this tool to manage their own projects, in case
the administrator has configured the Apache server with Digest
authentication method, users will fail to be authenticated when
they run the command 'repo init'.
Add the digest authentication password manager to the handler
list will fix this issue.

Since Git HTTP protocol will require the user be authenticated
for fetch operation first before pushing commits to the remote,
it is unlikely for the administrator to implement anonymous
read (aka pull) access and write access (aka push) for
authenticated user. Both read and write have to be authenticated.
Be aware that the user may have to add an extra line in his
~/.netrc file:
-------------------
account example.com
-------------------
where 'example.com' is the realm for Apache Digest authentication.

Change-Id: I76eb27b205554426d9ce1965deaaf727b87916cd
Signed-off-by: Xiaodong Xu <stid.smth@gmail.com>
2012-03-14 15:01:34 -07:00
76abcc1d1e repo status to print project name on clean gits
repo status just prints "# on branch oprofile" if you have branched
in clean status. This doesn't really tell which branch is meant.

Instead we can use the same syntax with modified gits which will
give us detailed information.

Change-Id: I55fe5154d278e10a814281dd2ba501ec6e956730
2012-03-12 12:25:40 -07:00
d315382572 Add 'rebase="false"' attribute to the <project/> XML.
This new attribute can prevent 'repo sync' from automatically rebasing.

I hit a situation in where one of the git repositories I was tracking
was actually an external repository that I wanted to pull commits
into and merge myself. (NOT rebase, since that would lose the merge
history.) In this case, I'm not using 'repo upload', I'm manually
managing the merges to and from this repository.

Everything was going great until I typed 'repo sync' and it rebased
my manually-merged tree. Hence the option to skip it.

Change-Id: I965e0dd1acb87f4a56752ebedc7e2de1c502dbf8
2012-03-12 12:24:22 -07:00
43bda84362 Avoid missing content-length header in project.py
Occassionally, the content-length may be missing when using urlib
in python 2.6 and 2.7.  This change assumes the value of the header is
0 if it doesn't exist

Change-Id: Iaf1c8a796bc667823d4d7c30f9b617644b271d00
2012-03-12 12:13:15 -07:00
9b017dab46 Update SUBMITTING_PATCHES
The review server is now at gerrit-review.googlesource.com.

Change-Id: I4be67fdb1876eb2e2af4420ac63557596b9e233b
2012-02-28 18:54:33 -08:00
e9dc3b3368 sync: Add manifest_name parameter
This parameter changes the manifest used by 'repo sync' for only
this execution. It should be useful for developers wishing to get
the repo temporarily into a known state, without clobbering their
existing manifest.

Tested by shifting Chrome OS between minilayout and full, and
between several release-builder-generated manifests.

Change-Id: I14194b665195b0e78f368d9ec8b8a83227af2627
2012-01-26 12:32:36 -05:00
c9571423f8 upload: Support uploading to Gerrit over https://
If SSH is not available, Gerrit returns NOT_AVAILABLE to the /ssh_info
query made by repo upload. In this case fallback to the /p/$PROJECT URL
that Gerrit also exports and use that for uploads.

Change-Id: I1e3e39ab709ecc0a692614a41a42446426f39c08
2012-01-11 16:18:40 -08:00
34fb20f67c Revert "Default repo manifest settings in git config"
This reverts commit ee1c2f5717.

This breaks a lot of buildbot systems. Rolling it back for now
until we can understand what the breakage was and how to fix it.
2011-11-30 13:41:02 -08:00
ecff4f17b0 Describe the repo launch version in repo version
repo version v1.7.8
         (from https://android.googlesource.com/tools/repo.git)
  repo launcher version 1.14
         (from /home/sop/bin/repo)
  git version 1.7.8.rc2.256.gcc761
  Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
  [GCC 4.4.3]

Change-Id: Ifcbe5b0e226a1a6ca85455eb62e4da5e9a0f0ca0
2011-11-29 15:02:15 -08:00
cc14fa9820 Improve error handling when reading loose refs
When repo is trying to figure out branches the repository has by
traversing refs/heads, add exception handling for readline.

Change-Id: If3b2a3720c6496f52f629aa9a2539f186d6ec882
2011-11-29 14:43:04 -08:00
3ce2a6b46b Propagate result codes from subcmds to sys.exit().
Allows scripts driving repo to know when git failures have
occurred, not just repo internal errors.

Change-Id: Id20fbbb405c35a148e72c87b822da3f3bf93839c
2011-11-29 14:38:19 -08:00
841be34968 Don't prompt the user for name/email unless necessary
If the user has already configured a workspace, use these values
when re-running 'repo init'.

Otherwise, if the user has global name and e-mail set, use these.

It's always possible to override this and be prompted by specifying
--config-name when running 'repo init'.

Change-Id: If45f0e4b14884071439fb02709dc5cb53f070f60
2011-11-29 14:31:56 -08:00
ee1c2f5717 Default repo manifest settings in git config
A default manifest URL can be specified using:
  git config --global repo-manifest.<id>.url <url>

A default manifest server can be specified using:
  git config --global repo-manifest.<id>.server <url>

A default git mirror reference can be specified using:
  git config --global repo-manifest.<id>.reference <path>

This will allow the user to use 'repo init -u <id>' as
a shorter alternative to specifying the full URL.

Also, manifest server will not have to be specified in the
manifest XML and the reference will not have to be specified
on the command line. If they are, they will override these
default values however.

Change-Id: Ifdbc160bd5909ec7df9efb0c5d7136f1d9351754
Signed-off-by: Victor Boivie <victor.boivie@sonyericsson.com>
2011-11-29 14:24:58 -08:00
6a1f737380 Added remote destination branch information when uploading.
Several times one have done an upload only to later notice in gerrit
that the upload was done to the wrong branch as the git has not yet
been branched for the current git. This change will make repo print
what the destination branch is when asking the user if she wants to
go through with the upload.

Change-Id: Ia9c3a92a6a04c022edfebf4f8d651ac062bb1f3b
2011-11-29 14:01:57 -08:00
e9311273dd repo: capitalize default prompt char
It is common in command line tools to indicate what the default answer
will be if the user simply hits enter.  In repo, the display is just
"y/n" with no indication as to which is the default.  So change the n
to N in the messages since that is how repo operates.

Change-Id: I81819ae630355072eb0365e59168b0921289498f
2011-11-29 12:38:52 -08:00
605a9a487b Fixed UnicodeDecodeError while uploading changes.
When commit with comment that has non-ASCII characters,
UnicodeDecodeError will be raised
while uploading multiple project/branch changes.
Because some strings in script are not str type, but unicode.
So all the strings are decoded to unicode,
and python use ascii to do this,
it can not decode non-ASCII characters,
so UnicodeDecodeError raised.

Signed-off-by: chenguodong <chenguodong@huawei.com>

Change-Id: I46447f489a4b9760a5899c7ba9d764b688594e46
2011-11-29 12:11:41 -08:00
2a32f6afa6 Fix typo
Change-Id: Idd68ad0a34fcf4bd4e18b0248f50187a539d610a
2011-11-29 12:09:35 -08:00
498fe90b45 Stabilize repo communication with subprocesses.
Make repo use the standard way in python to work with pipes.
Communication via pipes to sub processes is done by calling
communicate(). This will make repo not hang every now and
then.

Change-Id: Ibe2c4ecbdbcbe72f0b725ca50d54088e5646fc5d
2011-11-29 11:54:58 -08:00
53d6f4d17e Add a sync flag that fetches only current branch
There is also shortcuts in case if the "current branch" is
a persistent revision such as tag or sha1. We check if the
persistent revision is present locally and if it does - do
no fetch anything from the server.

This greately reduces sync time and size of the on-disk repo

Change-Id: I23c6d95185474ed6e1a03c836a47f489953b99be
2011-11-03 13:08:27 -07:00
9d8f914fe8 Remove extra '/' in RemoteSpec
urljoin appends a '/' if only the domain is in the url path.  This
change strips that off before creating a RemoteSpec
2011-11-03 13:05:14 -07:00
13 changed files with 260 additions and 167 deletions

View File

@ -5,7 +5,7 @@ Short Version:
- Make sure all code is under the Apache License, 2.0.
- Publish your changes for review:
git push ssh://review.source.android.com:29418/tools/repo.git HEAD:refs/for/master
git push https://gerrit-review.googlesource.com/git-repo HEAD:refs/for/maint
Long Version:
@ -55,24 +55,23 @@ Do not email your patches to anyone.
Instead, login to the Gerrit Code Review tool at:
https://review.source.android.com/
https://gerrit-review.googlesource.com/
Ensure you have completed one of the necessary contributor
agreements, providing documentation to the project maintainers that
they have right to redistribute your work under the Apache License:
https://review.source.android.com/#settings,agreements
https://gerrit-review.googlesource.com/#/settings/agreements
Ensure you have registered one or more SSH public keys, so you can
push your commits directly over SSH:
Ensure you have obtained an HTTP password to authenticate:
https://review.source.android.com/#settings,ssh-keys
https://gerrit-review.googlesource.com/new-password
Push your patches over SSH to the review server, possibly through
Push your patches over HTTPS to the review server, possibly through
a remembered remote to make this easier in the future:
git config remote.review.url ssh://review.source.android.com:29418/tools/repo.git
git config remote.review.push HEAD:refs/for/master
git config remote.review.url https://gerrit-review.googlesource.com/git-repo
git config remote.review.push HEAD:refs/for/maint
git push review

View File

@ -221,26 +221,10 @@ class GitCommand(object):
self.stdin = p.stdin
def Wait(self):
p = self.process
if p.stdin:
p.stdin.close()
self.stdin = None
if p.stdout:
self.stdout = p.stdout.read()
p.stdout.close()
else:
p.stdout = None
if p.stderr:
self.stderr = p.stderr.read()
p.stderr.close()
else:
p.stderr = None
try:
rc = p.wait()
p = self.process
(self.stdout, self.stderr) = p.communicate()
rc = p.returncode
finally:
_remove_ssh_client(p)
return rc

View File

@ -488,7 +488,7 @@ def close_ssh():
_master_keys_lock = None
URI_SCP = re.compile(r'^([^@:]*@?[^:/]{1,}):')
URI_ALL = re.compile(r'^([a-z][a-z+]*)://([^@/]*@?[^/]*)/')
URI_ALL = re.compile(r'^([a-z][a-z+-]*)://([^@/]*@?[^/]*)/')
def GetSchemeFromUrl(url):
m = URI_ALL.match(url)
@ -527,7 +527,7 @@ class Remote(object):
self.projectname = self._Get('projectname')
self.fetch = map(lambda x: RefSpec.FromString(x),
self._Get('fetch', all=True))
self._review_protocol = None
self._review_url = None
def _InsteadOf(self):
globCfg = GitConfig.ForUser()
@ -554,9 +554,8 @@ class Remote(object):
connectionUrl = self._InsteadOf()
return _preconnect(connectionUrl)
@property
def ReviewProtocol(self):
if self._review_protocol is None:
def ReviewUrl(self, userEmail):
if self._review_url is None:
if self.review is None:
return None
@ -565,67 +564,47 @@ class Remote(object):
u = 'http://%s' % u
if u.endswith('/Gerrit'):
u = u[:len(u) - len('/Gerrit')]
if not u.endswith('/ssh_info'):
if not u.endswith('/'):
u += '/'
u += 'ssh_info'
if u.endswith('/ssh_info'):
u = u[:len(u) - len('/ssh_info')]
if not u.endswith('/'):
u += '/'
http_url = u
if u in REVIEW_CACHE:
info = REVIEW_CACHE[u]
self._review_protocol = info[0]
self._review_host = info[1]
self._review_port = info[2]
self._review_url = REVIEW_CACHE[u]
elif 'REPO_HOST_PORT_INFO' in os.environ:
info = os.environ['REPO_HOST_PORT_INFO']
self._review_protocol = 'ssh'
self._review_host = info.split(" ")[0]
self._review_port = info.split(" ")[1]
REVIEW_CACHE[u] = (
self._review_protocol,
self._review_host,
self._review_port)
host, port = os.environ['REPO_HOST_PORT_INFO'].split()
self._review_url = self._SshReviewUrl(userEmail, host, port)
REVIEW_CACHE[u] = self._review_url
else:
try:
info = urllib2.urlopen(u).read()
if info == 'NOT_AVAILABLE':
raise UploadError('%s: SSH disabled' % self.review)
info_url = u + 'ssh_info'
info = urllib2.urlopen(info_url).read()
if '<' in info:
# Assume the server gave us some sort of HTML
# response back, like maybe a login page.
#
raise UploadError('%s: Cannot parse response' % u)
raise UploadError('%s: Cannot parse response' % info_url)
self._review_protocol = 'ssh'
self._review_host = info.split(" ")[0]
self._review_port = info.split(" ")[1]
except urllib2.HTTPError, e:
if e.code == 404:
self._review_protocol = 'http-post'
self._review_host = None
self._review_port = None
if info == 'NOT_AVAILABLE':
# Assume HTTP if SSH is not enabled.
self._review_url = http_url + 'p/'
else:
raise UploadError('Upload over SSH unavailable')
host, port = info.split()
self._review_url = self._SshReviewUrl(userEmail, host, port)
except urllib2.HTTPError, e:
raise UploadError('%s: %s' % (self.review, str(e)))
except urllib2.URLError, e:
raise UploadError('%s: %s' % (self.review, str(e)))
REVIEW_CACHE[u] = (
self._review_protocol,
self._review_host,
self._review_port)
return self._review_protocol
REVIEW_CACHE[u] = self._review_url
return self._review_url + self.projectname
def SshReviewUrl(self, userEmail):
if self.ReviewProtocol != 'ssh':
return None
def _SshReviewUrl(self, userEmail, host, port):
username = self._config.GetString('review.%s.username' % self.review)
if username is None:
username = userEmail.split("@")[0]
return 'ssh://%s@%s:%s/%s' % (
username,
self._review_host,
self._review_port,
self.projectname)
username = userEmail.split('@')[0]
return 'ssh://%s@%s:%s/' % (username, host, port)
def ToLocal(self, rev):
"""Convert a remote revision string to something we have locally.

View File

@ -139,13 +139,15 @@ class GitRefs(object):
def _ReadLoose1(self, path, name):
try:
fd = open(path, 'rb')
mtime = os.path.getmtime(path)
except OSError:
return
except IOError:
except:
return
try:
id = fd.readline()
try:
mtime = os.path.getmtime(path)
id = fd.readline()
except:
return
finally:
fd.close()

54
main.py
View File

@ -36,6 +36,7 @@ from git_config import init_ssh, close_ssh
from command import InteractiveCommand
from command import MirrorSafeCommand
from command import PagedCommand
from subcmds.version import Version
from editor import Editor
from error import DownloadError
from error import ManifestInvalidRevisionError
@ -73,6 +74,7 @@ class _Repo(object):
all_commands['branch'] = all_commands['branches']
def _Run(self, argv):
result = 0
name = None
glob = []
@ -96,7 +98,7 @@ class _Repo(object):
name = 'version'
else:
print >>sys.stderr, 'fatal: invalid usage of --version'
sys.exit(1)
return 1
try:
cmd = self.commands[name]
@ -104,7 +106,7 @@ class _Repo(object):
print >>sys.stderr,\
"repo: '%s' is not a repo command. See 'repo help'."\
% name
sys.exit(1)
return 1
cmd.repodir = self.repodir
cmd.manifest = XmlManifest(cmd.repodir)
@ -114,7 +116,7 @@ class _Repo(object):
print >>sys.stderr, \
"fatal: '%s' requires a working directory"\
% name
sys.exit(1)
return 1
copts, cargs = cmd.OptionParser.parse_args(argv)
@ -132,7 +134,7 @@ class _Repo(object):
try:
start = time.time()
try:
cmd.Execute(copts, cargs)
result = cmd.Execute(copts, cargs)
finally:
elapsed = time.time() - start
hours, remainder = divmod(elapsed, 3600)
@ -146,16 +148,18 @@ class _Repo(object):
% (hours, minutes, seconds)
except DownloadError, e:
print >>sys.stderr, 'error: %s' % str(e)
sys.exit(1)
return 1
except ManifestInvalidRevisionError, e:
print >>sys.stderr, 'error: %s' % str(e)
sys.exit(1)
return 1
except NoSuchProjectError, e:
if e.name:
print >>sys.stderr, 'error: project %s not found' % e.name
else:
print >>sys.stderr, 'error: no project in current directory'
sys.exit(1)
return 1
return result
def _MyRepoPath():
return os.path.dirname(__file__)
@ -291,6 +295,24 @@ class _BasicAuthHandler(urllib2.HTTPBasicAuthHandler):
self.retried = 0
raise
class _DigestAuthHandler(urllib2.HTTPDigestAuthHandler):
def http_error_auth_reqed(self, auth_header, host, req, headers):
try:
old_add_header = req.add_header
def _add_header(name, val):
val = val.replace('\n', '')
old_add_header(name, val)
req.add_header = _add_header
return urllib2.AbstractDigestAuthHandler.http_error_auth_reqed(
self, auth_header, host, req, headers)
except:
reset = getattr(self, 'reset_retry_count', None)
if reset is not None:
reset()
elif getattr(self, 'retried', None):
self.retried = 0
raise
def init_http():
handlers = [_UserAgentHandler()]
@ -299,13 +321,14 @@ def init_http():
n = netrc.netrc()
for host in n.hosts:
p = n.hosts[host]
mgr.add_password(None, 'http://%s/' % host, p[0], p[2])
mgr.add_password(None, 'https://%s/' % host, p[0], p[2])
mgr.add_password(p[1], 'http://%s/' % host, p[0], p[2])
mgr.add_password(p[1], 'https://%s/' % host, p[0], p[2])
except netrc.NetrcParseError:
pass
except IOError:
pass
handlers.append(_BasicAuthHandler(mgr))
handlers.append(_DigestAuthHandler(mgr))
if 'http_proxy' in os.environ:
url = os.environ['http_proxy']
@ -316,6 +339,8 @@ def init_http():
urllib2.install_opener(urllib2.build_opener(*handlers))
def _Main(argv):
result = 0
opt = optparse.OptionParser(usage="repo wrapperinfo -- ...")
opt.add_option("--repo-dir", dest="repodir",
help="path to .repo/")
@ -329,16 +354,19 @@ def _Main(argv):
_CheckWrapperVersion(opt.wrapper_version, opt.wrapper_path)
_CheckRepoDir(opt.repodir)
Version.wrapper_version = opt.wrapper_version
Version.wrapper_path = opt.wrapper_path
repo = _Repo(opt.repodir)
try:
try:
init_ssh()
init_http()
repo._Run(argv)
result = repo._Run(argv) or 0
finally:
close_ssh()
except KeyboardInterrupt:
sys.exit(1)
result = 1
except RepoChangedException, rce:
# If repo changed, re-exec ourselves.
#
@ -349,7 +377,9 @@ def _Main(argv):
except OSError, e:
print >>sys.stderr, 'fatal: cannot restart repo after upgrade'
print >>sys.stderr, 'fatal: %s' % e
sys.exit(128)
result = 128
sys.exit(result)
if __name__ == '__main__':
_Main(sys.argv[1:])

View File

@ -59,7 +59,7 @@ class _XmlRemote(object):
return re.sub(r'^gopher://', '', url)
def ToRemoteSpec(self, projectName):
url = self.resolvedFetchUrl + '/' + projectName
url = self.resolvedFetchUrl.rstrip('/') + '/' + projectName
return RemoteSpec(self.name, url, self.reviewUrl)
class XmlManifest(object):
@ -498,6 +498,12 @@ class XmlManifest(object):
"project %s path cannot be absolute in %s" % \
(name, self.manifestFile)
rebase = node.getAttribute('rebase')
if not rebase:
rebase = True
else:
rebase = rebase.lower() in ("yes", "true", "1")
if self.IsMirror:
relpath = None
worktree = None
@ -513,7 +519,8 @@ class XmlManifest(object):
worktree = worktree,
relpath = path,
revisionExpr = revisionExpr,
revisionId = None)
revisionId = None,
rebase = rebase)
for n in node.childNodes:
if n.nodeName == 'copyfile':

View File

@ -36,7 +36,7 @@ except ImportError:
from color import Coloring
from git_command import GitCommand
from git_config import GitConfig, IsId, GetSchemeFromUrl
from git_config import GitConfig, IsId, GetSchemeFromUrl, ID_RE
from error import DownloadError
from error import GitError, HookError, ImportError, UploadError
from error import ManifestInvalidRevisionError
@ -503,7 +503,8 @@ class Project(object):
worktree,
relpath,
revisionExpr,
revisionId):
revisionId,
rebase = True):
self.manifest = manifest
self.name = name
self.remote = remote
@ -522,6 +523,8 @@ class Project(object):
else:
self.revisionId = revisionId
self.rebase = rebase
self.snapshots = {}
self.copyfiles = []
self.config = GitConfig.ForRepository(
@ -688,7 +691,7 @@ class Project(object):
di = self.work_git.DiffZ('diff-index', '-M', '--cached', HEAD)
df = self.work_git.DiffZ('diff-files')
do = self.work_git.LsOthers()
if not rb and not di and not df and not do:
if not rb and not di and not df and not do and not self.CurrentBranch:
return 'CLEAN'
out = StatusColoring(self.config)
@ -866,31 +869,30 @@ class Project(object):
branch.remote.projectname = self.name
branch.remote.Save()
if branch.remote.ReviewProtocol == 'ssh':
if dest_branch.startswith(R_HEADS):
dest_branch = dest_branch[len(R_HEADS):]
url = branch.remote.ReviewUrl(self.UserEmail)
if url is None:
raise UploadError('review not configured')
cmd = ['push']
if url.startswith('ssh://'):
rp = ['gerrit receive-pack']
for e in people[0]:
rp.append('--reviewer=%s' % sq(e))
for e in people[1]:
rp.append('--cc=%s' % sq(e))
ref_spec = '%s:refs/for/%s' % (R_HEADS + branch.name, dest_branch)
if auto_topic:
ref_spec = ref_spec + '/' + branch.name
cmd = ['push']
cmd.append('--receive-pack=%s' % " ".join(rp))
cmd.append(branch.remote.SshReviewUrl(self.UserEmail))
cmd.append(ref_spec)
if GitCommand(self, cmd, bare = True).Wait() != 0:
raise UploadError('Upload failed')
cmd.append(url)
else:
raise UploadError('Unsupported protocol %s' \
% branch.remote.review)
if dest_branch.startswith(R_HEADS):
dest_branch = dest_branch[len(R_HEADS):]
ref_spec = '%s:refs/for/%s' % (R_HEADS + branch.name, dest_branch)
if auto_topic:
ref_spec = ref_spec + '/' + branch.name
cmd.append(ref_spec)
if GitCommand(self, cmd, bare = True).Wait() != 0:
raise UploadError('Upload failed')
msg = "posted to %s for %s" % (branch.remote.review, dest_branch)
self.bare_git.UpdateRef(R_PUB + branch.name,
@ -900,7 +902,11 @@ class Project(object):
## Sync ##
def Sync_NetworkHalf(self, quiet=False, is_new=None):
def Sync_NetworkHalf(self,
quiet=False,
is_new=None,
current_branch_only=False,
clone_bundle=True):
"""Perform only the network IO portion of the sync process.
Local working directory/branch state is not affected.
"""
@ -923,24 +929,15 @@ class Project(object):
else:
alt_dir = None
if alt_dir is None and self._ApplyCloneBundle(initial=is_new, quiet=quiet):
if clone_bundle \
and alt_dir is None \
and self._ApplyCloneBundle(initial=is_new, quiet=quiet):
is_new = False
if not self._RemoteFetch(initial=is_new, quiet=quiet, alt_dir=alt_dir):
if not self._RemoteFetch(initial=is_new, quiet=quiet, alt_dir=alt_dir,
current_branch_only=current_branch_only):
return False
#Check that the requested ref was found after fetch
#
try:
self.GetRevisionId()
except ManifestInvalidRevisionError:
# if the ref is a tag. We can try fetching
# the tag manually as a last resort
#
rev = self.revisionExpr
if rev.startswith(R_TAGS):
self._RemoteFetch(None, rev[len(R_TAGS):], quiet=quiet)
if self.worktree:
self._InitMRef()
else:
@ -1027,7 +1024,7 @@ class Project(object):
if not branch.LocalMerge:
# The current branch has no tracking configuration.
# Jump off it to a deatched HEAD.
# Jump off it to a detached HEAD.
#
syncbuf.info(self,
"leaving %s; does not track upstream",
@ -1105,10 +1102,12 @@ class Project(object):
len(local_changes) - cnt_mine)
branch.remote = self.GetRemote(self.remote.name)
branch.merge = self.revisionExpr
if not ID_RE.match(self.revisionExpr):
# in case of manifest sync the revisionExpr might be a SHA1
branch.merge = self.revisionExpr
branch.Save()
if cnt_mine > 0:
if cnt_mine > 0 and self.rebase:
def _dorebase():
self._Rebase(upstream = '%s^1' % last_mine, onto = revid)
self._CopyFiles()
@ -1335,10 +1334,30 @@ class Project(object):
## Direct Git Commands ##
def _RemoteFetch(self, name=None, tag=None,
def _RemoteFetch(self, name=None,
current_branch_only=False,
initial=False,
quiet=False,
alt_dir=None):
is_sha1 = False
tag_name = None
if current_branch_only:
if ID_RE.match(self.revisionExpr) is not None:
is_sha1 = True
elif self.revisionExpr.startswith(R_TAGS):
# this is a tag and its sha1 value should never change
tag_name = self.revisionExpr[len(R_TAGS):]
if is_sha1 or tag_name is not None:
try:
self.GetRevisionId()
return True
except ManifestInvalidRevisionError:
# There is no such persistent revision. We have to fetch it.
pass
if not name:
name = self.remote.name
@ -1401,9 +1420,19 @@ class Project(object):
if not self.worktree:
cmd.append('--update-head-ok')
cmd.append(name)
if tag is not None:
if not current_branch_only or is_sha1:
# Fetch whole repo
cmd.append('--tags')
cmd.append((u'+refs/heads/*:') + remote.ToLocal('refs/heads/*'))
elif tag_name is not None:
cmd.append('tag')
cmd.append(tag)
cmd.append(tag_name)
else:
branch = self.revisionExpr
if branch.startswith(R_HEADS):
branch = branch[len(R_HEADS):]
cmd.append((u'+refs/heads/%s:' % branch) + remote.ToLocal('refs/heads/%s' % branch))
ok = False
for i in range(2):
@ -1428,6 +1457,8 @@ class Project(object):
remote = self.GetRemote(self.remote.name)
bundle_url = remote.url + '/clone.bundle'
bundle_url = GitConfig.ForUser().UrlInsteadOf(bundle_url)
if GetSchemeFromUrl(bundle_url) in ('persistent-http', 'persistent-https'):
bundle_url = bundle_url[len('persistent-'):]
if GetSchemeFromUrl(bundle_url) not in ('http', 'https'):
return False
@ -1485,7 +1516,7 @@ class Project(object):
except:
return None
if e.code == 404:
if e.code in (401, 403, 404):
keep = False
return False
elif _content_type() == 'text/plain':
@ -1511,7 +1542,7 @@ class Project(object):
p = None
try:
size = r.headers['content-length']
size = r.headers.get('content-length', 0)
unit = 1 << 10
if size and not quiet:

14
repo
View File

@ -28,7 +28,7 @@ if __name__ == '__main__':
del magic
# increment this whenever we make important changes to this script
VERSION = (1, 13)
VERSION = (1, 14)
# increment this if the MAINTAINER_KEYS block is modified
KEYRING_VERSION = (1,0)
@ -139,6 +139,11 @@ group.add_option('--no-repo-verify',
dest='no_repo_verify', action='store_true',
help='do not verify repo source code')
# Other
group = init_optparse.add_option_group('Other options')
group.add_option('--config-name',
dest='config_name', action="store_true", default=False,
help='Always prompt for name/e-mail')
class CloneFailure(Exception):
"""Indicate the remote clone of repo itself failed.
@ -149,7 +154,7 @@ def _Init(args):
"""Installs repo by cloning it over the network.
"""
opt, args = init_optparse.parse_args(args)
if args or not opt.manifest_url:
if args:
init_optparse.print_usage()
sys.exit(1)
@ -306,11 +311,12 @@ def _InitHttp():
n = netrc.netrc()
for host in n.hosts:
p = n.hosts[host]
mgr.add_password(None, 'http://%s/' % host, p[0], p[2])
mgr.add_password(None, 'https://%s/' % host, p[0], p[2])
mgr.add_password(p[1], 'http://%s/' % host, p[0], p[2])
mgr.add_password(p[1], 'https://%s/' % host, p[0], p[2])
except:
pass
handlers.append(urllib2.HTTPBasicAuthHandler(mgr))
handlers.append(urllib2.HTTPDigestAuthHandler(mgr))
if 'http_proxy' in os.environ:
url = os.environ['http_proxy']

View File

@ -99,6 +99,12 @@ to update the working directory files.
dest='no_repo_verify', action='store_true',
help='do not verify repo source code')
# Other
g = p.add_option_group('Other options')
g.add_option('--config-name',
dest='config_name', action="store_true", default=False,
help='Always prompt for name/e-mail')
def _SyncManifest(self, opt):
m = self.manifest.manifestProject
is_new = not m.Exists
@ -179,6 +185,24 @@ to update the working directory files.
return value
return a
def _ShouldConfigureUser(self):
gc = self.manifest.globalConfig
mp = self.manifest.manifestProject
# If we don't have local settings, get from global.
if not mp.config.Has('user.name') or not mp.config.Has('user.email'):
if not gc.Has('user.name') or not gc.Has('user.email'):
return True
mp.config.SetString('user.name', gc.GetString('user.name'))
mp.config.SetString('user.email', gc.GetString('user.email'))
print ''
print 'Your identity is: %s <%s>' % (mp.config.GetString('user.name'),
mp.config.GetString('user.email'))
print 'If you want to change this, please re-run \'repo init\' with --config-name'
return False
def _ConfigureUser(self):
mp = self.manifest.manifestProject
@ -189,7 +213,7 @@ to update the working directory files.
print ''
print 'Your identity is: %s <%s>' % (name, email)
sys.stdout.write('is this correct [y/n]? ')
sys.stdout.write('is this correct [y/N]? ')
a = sys.stdin.readline().strip()
if a in ('yes', 'y', 't', 'true'):
break
@ -231,7 +255,7 @@ to update the working directory files.
out.printer(fg='black', attr=c)(' %-6s ', c)
out.nl()
sys.stdout.write('Enable color display in this user account (y/n)? ')
sys.stdout.write('Enable color display in this user account (y/N)? ')
a = sys.stdin.readline().strip().lower()
if a in ('y', 'yes', 't', 'true', 'on'):
gc.SetString('color.ui', 'auto')
@ -261,7 +285,8 @@ to update the working directory files.
self._LinkManifest(opt.manifest_name)
if os.isatty(0) and os.isatty(1) and not self.manifest.IsMirror:
self._ConfigureUser()
if opt.config_name or self._ShouldConfigureUser():
self._ConfigureUser()
self._ConfigureColor()
self._ConfigureDepth(opt)

View File

@ -101,19 +101,6 @@ the following meanings:
all = self.GetProjects(args)
counter = itertools.count()
on = {}
for project in all:
cb = project.CurrentBranch
if cb:
if cb not in on:
on[cb] = []
on[cb].append(project)
branch_names = list(on.keys())
branch_names.sort()
for cb in branch_names:
print '# on branch %s' % cb
if opt.jobs == 1:
for project in all:
state = project.PrintWorkTreeStatus()

View File

@ -86,6 +86,12 @@ specify a custom tag/label.
The -f/--force-broken option can be used to proceed with syncing
other projects if a project sync fails.
The --no-clone-bundle option disables any attempt to use
$URL/clone.bundle to bootstrap a new Git repository from a
resumeable bundle file on a content delivery network. This
may be necessary if there are problems with the local Python
HTTP client or proxy configuration, but the Git binary works.
SSH Connections
---------------
@ -131,12 +137,21 @@ later is required to fix a server side protocol bug.
p.add_option('-d','--detach',
dest='detach_head', action='store_true',
help='detach projects back to manifest revision')
p.add_option('-c','--current-branch',
dest='current_branch_only', action='store_true',
help='fetch only current branch from server')
p.add_option('-q','--quiet',
dest='quiet', action='store_true',
help='be more quiet')
p.add_option('-j','--jobs',
dest='jobs', action='store', type='int',
help="projects to fetch simultaneously (default %d)" % self.jobs)
p.add_option('-m', '--manifest-name',
dest='manifest_name',
help='temporary manifest to use for this sync', metavar='NAME.xml')
p.add_option('--no-clone-bundle',
dest='no_clone_bundle', action='store_true',
help='disable use of /clone.bundle on HTTP/HTTPS')
if show_smart:
p.add_option('-s', '--smart-sync',
dest='smart_sync', action='store_true',
@ -179,7 +194,10 @@ later is required to fix a server side protocol bug.
# - We always make sure we unlock the lock if we locked it.
try:
try:
success = project.Sync_NetworkHalf(quiet=opt.quiet)
success = project.Sync_NetworkHalf(
quiet=opt.quiet,
current_branch_only=opt.current_branch_only,
clone_bundle=not opt.no_clone_bundle)
# Lock around all the rest of the code, since printing, updating a set
# and Progress.update() are not thread safe.
@ -212,7 +230,8 @@ later is required to fix a server side protocol bug.
if self.jobs == 1:
for project in projects:
pm.update()
if project.Sync_NetworkHalf(quiet=opt.quiet):
if project.Sync_NetworkHalf(quiet=opt.quiet,
current_branch_only=opt.current_branch_only):
fetched.add(project.gitdir)
else:
print >>sys.stderr, 'error: Cannot fetch %s' % project.name
@ -328,6 +347,15 @@ uncommitted changes are present' % project.relpath
if opt.network_only and opt.local_only:
print >>sys.stderr, 'error: cannot combine -n and -l'
sys.exit(1)
if opt.manifest_name and opt.smart_sync:
print >>sys.stderr, 'error: cannot combine -m and -s'
sys.exit(1)
if opt.manifest_name and opt.smart_tag:
print >>sys.stderr, 'error: cannot combine -m and -t'
sys.exit(1)
if opt.manifest_name:
self.manifest.Override(opt.manifest_name)
if opt.smart_sync or opt.smart_tag:
if not self.manifest.manifest_server:
@ -388,7 +416,8 @@ uncommitted changes are present' % project.relpath
_PostRepoUpgrade(self.manifest)
if not opt.local_only:
mp.Sync_NetworkHalf(quiet=opt.quiet)
mp.Sync_NetworkHalf(quiet=opt.quiet,
current_branch_only=opt.current_branch_only)
if mp.HasChanges:
syncbuf = SyncBuffer(mp.config)

View File

@ -73,7 +73,7 @@ Configuration
review.URL.autoupload:
To disable the "Upload ... (y/n)?" prompt, you can set a per-project
To disable the "Upload ... (y/N)?" prompt, you can set a per-project
or global Git configuration option. If review.URL.autoupload is set
to "true" then repo will assume you always answer "y" at the prompt,
and will not prompt you further. If it is set to "false" then repo
@ -162,7 +162,7 @@ Gerrit Code Review: http://code.google.com/p/gerrit/
date = branch.date
list = branch.commits
print 'Upload project %s/:' % project.relpath
print 'Upload project %s/ to remote branch %s:' % (project.relpath, project.revisionExpr)
print ' branch %s (%2d commit%s, %s):' % (
name,
len(list),
@ -171,7 +171,7 @@ Gerrit Code Review: http://code.google.com/p/gerrit/
for commit in list:
print ' %s' % commit
sys.stdout.write('to %s (y/n)? ' % remote.review)
sys.stdout.write('to %s (y/N)? ' % remote.review)
answer = sys.stdin.readline().strip()
answer = answer in ('y', 'Y', 'yes', '1', 'true', 't')
@ -202,11 +202,12 @@ Gerrit Code Review: http://code.google.com/p/gerrit/
if b:
script.append('#')
script.append('# branch %s (%2d commit%s, %s):' % (
script.append('# branch %s (%2d commit%s, %s) to remote branch %s:' % (
name,
len(list),
len(list) != 1 and 's' or '',
date))
date,
project.revisionExpr))
for commit in list:
script.append('# %s' % commit)
b[name] = branch
@ -215,6 +216,11 @@ Gerrit Code Review: http://code.google.com/p/gerrit/
branches[project.name] = b
script.append('')
script = [ x.encode('utf-8')
if issubclass(type(x), unicode)
else x
for x in script ]
script = Editor.EditString("\n".join(script)).split("\n")
project_re = re.compile(r'^#?\s*project\s*([^\s]+)/:$')
@ -294,7 +300,7 @@ Gerrit Code Review: http://code.google.com/p/gerrit/
# if they want to auto upload, let's not ask because it could be automated
if answer is None:
sys.stdout.write('Uncommitted changes in ' + branch.project.name + ' (did you forget to amend?). Continue uploading? (y/n) ')
sys.stdout.write('Uncommitted changes in ' + branch.project.name + ' (did you forget to amend?). Continue uploading? (y/N) ')
a = sys.stdin.readline().strip().lower()
if a not in ('y', 'yes', 't', 'true', 'on'):
print >>sys.stderr, "skipping upload"

View File

@ -19,6 +19,9 @@ from git_command import git
from project import HEAD
class Version(Command, MirrorSafeCommand):
wrapper_version = None
wrapper_path = None
common = False
helpSummary = "Display the version of repo"
helpUsage = """
@ -31,5 +34,10 @@ class Version(Command, MirrorSafeCommand):
print 'repo version %s' % rp.work_git.describe(HEAD)
print ' (from %s)' % rem.url
if Version.wrapper_path is not None:
print 'repo launcher version %s' % Version.wrapper_version
print ' (from %s)' % Version.wrapper_path
print git.version().strip()
print 'Python %s' % sys.version