git-repo/subcmds/upload.py

423 lines
14 KiB
Python
Raw Normal View History

2008-10-21 14:00:00 +00:00
#
# Copyright (C) 2008 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import print_function
import copy
2008-10-21 14:00:00 +00:00
import re
import sys
from command import InteractiveCommand
from editor import Editor
Support repo-level pre-upload hook and prep for future hooks. All repo-level hooks are expected to live in a single project at the top level of that project. The name of the hooks project is provided in the manifest.xml. The manifest also lists which hooks are enabled to make it obvious if a file somehow failed to sync down (or got deleted). Before running any hook, we will prompt the user to make sure that it is OK. A user can deny running the hook, allow once, or allow "forever" (until hooks change). This tries to keep with the git spirit of not automatically running anything on the user's computer that got synced down. Note that individual repo commands can add always options to avoid these prompts as they see fit (see below for the 'upload' options). When hooks are run, they are loaded into the current interpreter (the one running repo) and their main() function is run. This mechanism is used (instead of using subprocess) to make it easier to expand to a richer hook interface in the future. During loading, the interpreter's sys.path is updated to contain the directory containing the hooks so that hooks can be split into multiple files. The upload command has two options that control hook behavior: - no-verify=False, verify=False (DEFAULT): If stdout is a tty, can prompt about running upload hooks if needed. If user denies running hooks, the upload is cancelled. If stdout is not a tty and we would need to prompt about upload hooks, upload is cancelled. - no-verify=False, verify=True: Always run upload hooks with no prompt. - no-verify=True, verify=False: Never run upload hooks, but upload anyway (AKA bypass hooks). - no-verify=True, verify=True: Invalid Sample bit of manifest.xml code for enabling hooks (assumes you have a project named 'hooks' where hooks are stored): <repo-hooks in-project="hooks" enabled-list="pre-upload" /> Sample main() function in pre-upload.py in hooks directory: def main(project_list, **kwargs): print ('These projects will be uploaded: %s' % ', '.join(project_list)) print ('I am being a good boy and ignoring anything in kwargs\n' 'that I don\'t understand.') print 'I fail 50% of the time. How flaky.' if random.random() <= .5: raise Exception('Pre-upload hook failed. Have a nice day.') Change-Id: I5cefa2cd5865c72589263cf8e2f152a43c122f70
2011-03-04 19:54:18 +00:00
from error import HookError, UploadError
from project import RepoHook
2008-10-21 14:00:00 +00:00
from pyversion import is_python3
if not is_python3():
# pylint:disable=W0622
input = raw_input
# pylint:enable=W0622
UNUSUAL_COMMIT_THRESHOLD = 5
def _ConfirmManyUploads(multiple_branches=False):
if multiple_branches:
print('ATTENTION: One or more branches has an unusually high number '
'of commits.')
else:
print('ATTENTION: You are uploading an unusually high number of commits.')
print('YOU PROBABLY DO NOT MEAN TO DO THIS. (Did you rebase across '
'branches?)')
answer = input("If you are sure you intend to do this, type 'yes': ").strip()
return answer == "yes"
2008-10-21 14:00:00 +00:00
def _die(fmt, *args):
msg = fmt % args
print('error: %s' % msg, file=sys.stderr)
2008-10-21 14:00:00 +00:00
sys.exit(1)
def _SplitEmails(values):
result = []
for value in values:
result.extend([s.strip() for s in value.split(',')])
return result
2008-10-21 14:00:00 +00:00
class Upload(InteractiveCommand):
common = True
helpSummary = "Upload changes for code review"
helpUsage = """
%prog [--re --cc] [<project>]...
2008-10-21 14:00:00 +00:00
"""
helpDescription = """
The '%prog' command is used to send changes to the Gerrit Code
Review system. It searches for topic branches in local projects
that have not yet been published for review. If multiple topic
branches are found, '%prog' opens an editor to allow the user to
select which branches to upload.
'%prog' searches for uploadable changes in all projects listed at
the command line. Projects can be specified either by name, or by
a relative or absolute path to the project's local directory. If no
projects are specified, '%prog' will search for uploadable changes
in all projects listed in the manifest.
If the --reviewers or --cc options are passed, those emails are
added to the respective list of users, and emails are sent to any
new users. Users passed as --reviewers must already be registered
with the code review system, or the upload will fail.
Configuration
-------------
review.URL.autoupload:
To disable the "Upload ... (y/N)?" prompt, you can set a per-project
or global Git configuration option. If review.URL.autoupload is set
to "true" then repo will assume you always answer "y" at the prompt,
and will not prompt you further. If it is set to "false" then repo
will assume you always answer "n", and will abort.
review.URL.autocopy:
To automatically copy a user or mailing list to all uploaded reviews,
you can set a per-project or global Git option to do so. Specifically,
review.URL.autocopy can be set to a comma separated list of reviewers
who you always want copied on all uploads with a non-empty --re
argument.
review.URL.username:
Override the username used to connect to Gerrit Code Review.
By default the local part of the email address is used.
The URL must match the review URL listed in the manifest XML file,
or in the .git/config within the project. For example:
[remote "origin"]
url = git://git.example.com/project.git
review = http://review.example.com/
[review "http://review.example.com/"]
autoupload = true
autocopy = johndoe@company.com,my-team-alias@company.com
review.URL.uploadtopic:
To add a topic branch whenever uploading a commit, you can set a
per-project or global Git option to do so. If review.URL.uploadtopic
is set to "true" then repo will assume you always want the equivalent
of the -t option to the repo command. If unset or set to "false" then
repo will make use of only the command line option.
References
----------
Gerrit Code Review: http://code.google.com/p/gerrit/
2008-10-21 14:00:00 +00:00
"""
def _Options(self, p):
p.add_option('-t',
dest='auto_topic', action='store_true',
help='Send local branch name to Gerrit Code Review')
p.add_option('--re', '--reviewers',
type='string', action='append', dest='reviewers',
help='Request reviews from these people.')
p.add_option('--cc',
type='string', action='append', dest='cc',
help='Also send email to these email addresses.')
p.add_option('--br',
type='string', action='store', dest='branch',
help='Branch to upload.')
p.add_option('--cbr', '--current-branch',
dest='current_branch', action='store_true',
help='Upload current git branch.')
p.add_option('-d', '--draft',
action='store_true', dest='draft', default=False,
help='If specified, upload as a draft.')
p.add_option('-D', '--destination', '--dest',
type='string', action='store', dest='dest_branch',
metavar='BRANCH',
help='Submit for review on this target branch.')
Support repo-level pre-upload hook and prep for future hooks. All repo-level hooks are expected to live in a single project at the top level of that project. The name of the hooks project is provided in the manifest.xml. The manifest also lists which hooks are enabled to make it obvious if a file somehow failed to sync down (or got deleted). Before running any hook, we will prompt the user to make sure that it is OK. A user can deny running the hook, allow once, or allow "forever" (until hooks change). This tries to keep with the git spirit of not automatically running anything on the user's computer that got synced down. Note that individual repo commands can add always options to avoid these prompts as they see fit (see below for the 'upload' options). When hooks are run, they are loaded into the current interpreter (the one running repo) and their main() function is run. This mechanism is used (instead of using subprocess) to make it easier to expand to a richer hook interface in the future. During loading, the interpreter's sys.path is updated to contain the directory containing the hooks so that hooks can be split into multiple files. The upload command has two options that control hook behavior: - no-verify=False, verify=False (DEFAULT): If stdout is a tty, can prompt about running upload hooks if needed. If user denies running hooks, the upload is cancelled. If stdout is not a tty and we would need to prompt about upload hooks, upload is cancelled. - no-verify=False, verify=True: Always run upload hooks with no prompt. - no-verify=True, verify=False: Never run upload hooks, but upload anyway (AKA bypass hooks). - no-verify=True, verify=True: Invalid Sample bit of manifest.xml code for enabling hooks (assumes you have a project named 'hooks' where hooks are stored): <repo-hooks in-project="hooks" enabled-list="pre-upload" /> Sample main() function in pre-upload.py in hooks directory: def main(project_list, **kwargs): print ('These projects will be uploaded: %s' % ', '.join(project_list)) print ('I am being a good boy and ignoring anything in kwargs\n' 'that I don\'t understand.') print 'I fail 50% of the time. How flaky.' if random.random() <= .5: raise Exception('Pre-upload hook failed. Have a nice day.') Change-Id: I5cefa2cd5865c72589263cf8e2f152a43c122f70
2011-03-04 19:54:18 +00:00
# Options relating to upload hook. Note that verify and no-verify are NOT
# opposites of each other, which is why they store to different locations.
# We are using them to match 'git commit' syntax.
#
# Combinations:
# - no-verify=False, verify=False (DEFAULT):
# If stdout is a tty, can prompt about running upload hooks if needed.
# If user denies running hooks, the upload is cancelled. If stdout is
# not a tty and we would need to prompt about upload hooks, upload is
# cancelled.
# - no-verify=False, verify=True:
# Always run upload hooks with no prompt.
# - no-verify=True, verify=False:
# Never run upload hooks, but upload anyway (AKA bypass hooks).
# - no-verify=True, verify=True:
# Invalid
p.add_option('--no-verify',
dest='bypass_hooks', action='store_true',
help='Do not run the upload hook.')
p.add_option('--verify',
dest='allow_all_hooks', action='store_true',
help='Run the upload hook without prompting.')
def _SingleBranch(self, opt, branch, people):
2008-10-21 14:00:00 +00:00
project = branch.project
name = branch.name
remote = project.GetBranch(name).remote
key = 'review.%s.autoupload' % remote.review
answer = project.config.GetBoolean(key)
if answer is False:
_die("upload blocked by %s = false" % key)
if answer is None:
date = branch.date
commit_list = branch.commits
destination = project.dest_branch or project.revisionExpr
print('Upload project %s/ to remote branch %s:' % (project.relpath, destination))
print(' branch %s (%2d commit%s, %s):' % (
name,
len(commit_list),
len(commit_list) != 1 and 's' or '',
date))
for commit in commit_list:
print(' %s' % commit)
sys.stdout.write('to %s (y/N)? ' % remote.review)
answer = sys.stdin.readline().strip().lower()
answer = answer in ('y', 'yes', '1', 'true', 't')
2008-10-21 14:00:00 +00:00
if answer:
if len(branch.commits) > UNUSUAL_COMMIT_THRESHOLD:
answer = _ConfirmManyUploads()
if answer:
self._UploadAndReport(opt, [branch], people)
2008-10-21 14:00:00 +00:00
else:
_die("upload aborted by user")
def _MultipleBranches(self, opt, pending, people):
2008-10-21 14:00:00 +00:00
projects = {}
branches = {}
script = []
script.append('# Uncomment the branches to upload:')
for project, avail in pending:
script.append('#')
script.append('# project %s/:' % project.relpath)
b = {}
for branch in avail:
if branch is None:
continue
2008-10-21 14:00:00 +00:00
name = branch.name
date = branch.date
commit_list = branch.commits
2008-10-21 14:00:00 +00:00
if b:
script.append('#')
destination = opt.dest_branch or project.dest_branch or project.revisionExpr
script.append('# branch %s (%2d commit%s, %s) to remote branch %s:' % (
2008-10-21 14:00:00 +00:00
name,
len(commit_list),
len(commit_list) != 1 and 's' or '',
date,
destination))
for commit in commit_list:
2008-10-21 14:00:00 +00:00
script.append('# %s' % commit)
b[name] = branch
projects[project.relpath] = project
branches[project.name] = b
script.append('')
script = [ x.encode('utf-8')
if issubclass(type(x), unicode)
else x
for x in script ]
2008-10-21 14:00:00 +00:00
script = Editor.EditString("\n".join(script)).split("\n")
project_re = re.compile(r'^#?\s*project\s*([^\s]+)/:$')
branch_re = re.compile(r'^\s*branch\s*([^\s(]+)\s*\(.*')
project = None
todo = []
for line in script:
m = project_re.match(line)
if m:
name = m.group(1)
project = projects.get(name)
if not project:
_die('project %s not available for upload', name)
continue
m = branch_re.match(line)
if m:
name = m.group(1)
if not project:
_die('project for branch %s not in script', name)
branch = branches[project.name].get(name)
if not branch:
_die('branch %s not in %s', name, project.relpath)
todo.append(branch)
if not todo:
_die("nothing uncommented for upload")
many_commits = False
for branch in todo:
if len(branch.commits) > UNUSUAL_COMMIT_THRESHOLD:
many_commits = True
break
if many_commits:
if not _ConfirmManyUploads(multiple_branches=True):
_die("upload aborted by user")
self._UploadAndReport(opt, todo, people)
2008-10-21 14:00:00 +00:00
def _AppendAutoCcList(self, branch, people):
"""
Appends the list of users in the CC list in the git project's config if a
non-empty reviewer list was found.
"""
name = branch.name
project = branch.project
key = 'review.%s.autocopy' % project.GetBranch(name).remote.review
raw_list = project.config.GetString(key)
if not raw_list is None and len(people[0]) > 0:
people[1].extend([entry.strip() for entry in raw_list.split(',')])
def _FindGerritChange(self, branch):
last_pub = branch.project.WasPublished(branch.name)
if last_pub is None:
return ""
refs = branch.GetPublishedRefs()
try:
# refs/changes/XYZ/N --> XYZ
return refs.get(last_pub).split('/')[-2]
except (AttributeError, IndexError):
return ""
def _UploadAndReport(self, opt, todo, original_people):
2008-10-21 14:00:00 +00:00
have_errors = False
for branch in todo:
try:
people = copy.deepcopy(original_people)
self._AppendAutoCcList(branch, people)
# Check if there are local changes that may have been forgotten
if branch.project.HasChanges():
key = 'review.%s.autoupload' % branch.project.remote.review
answer = branch.project.config.GetBoolean(key)
# if they want to auto upload, let's not ask because it could be automated
if answer is None:
sys.stdout.write('Uncommitted changes in ' + branch.project.name + ' (did you forget to amend?). Continue uploading? (y/N) ')
a = sys.stdin.readline().strip().lower()
if a not in ('y', 'yes', 't', 'true', 'on'):
print("skipping upload", file=sys.stderr)
branch.uploaded = False
branch.error = 'User aborted'
continue
# Check if topic branches should be sent to the server during upload
if opt.auto_topic is not True:
key = 'review.%s.uploadtopic' % branch.project.remote.review
opt.auto_topic = branch.project.config.GetBoolean(key)
destination = opt.dest_branch or branch.project.dest_branch or branch.project.revisionExpr
branch.UploadForReview(people, auto_topic=opt.auto_topic, draft=opt.draft, dest_branch=destination)
2008-10-21 14:00:00 +00:00
branch.uploaded = True
except UploadError as e:
2008-10-21 14:00:00 +00:00
branch.error = e
branch.uploaded = False
have_errors = True
print(file=sys.stderr)
print('----------------------------------------------------------------------', file=sys.stderr)
2008-10-21 14:00:00 +00:00
if have_errors:
for branch in todo:
if not branch.uploaded:
if len(str(branch.error)) <= 30:
fmt = ' (%s)'
else:
fmt = '\n (%s)'
print(('[FAILED] %-15s %-15s' + fmt) % (
2008-10-21 14:00:00 +00:00
branch.project.relpath + '/', \
branch.name, \
str(branch.error)),
file=sys.stderr)
print()
2008-10-21 14:00:00 +00:00
for branch in todo:
if branch.uploaded:
print('[OK ] %-15s %s' % (
branch.project.relpath + '/',
branch.name),
file=sys.stderr)
2008-10-21 14:00:00 +00:00
if have_errors:
sys.exit(1)
def Execute(self, opt, args):
project_list = self.GetProjects(args)
pending = []
reviewers = []
cc = []
branch = None
if opt.branch:
branch = opt.branch
Support repo-level pre-upload hook and prep for future hooks. All repo-level hooks are expected to live in a single project at the top level of that project. The name of the hooks project is provided in the manifest.xml. The manifest also lists which hooks are enabled to make it obvious if a file somehow failed to sync down (or got deleted). Before running any hook, we will prompt the user to make sure that it is OK. A user can deny running the hook, allow once, or allow "forever" (until hooks change). This tries to keep with the git spirit of not automatically running anything on the user's computer that got synced down. Note that individual repo commands can add always options to avoid these prompts as they see fit (see below for the 'upload' options). When hooks are run, they are loaded into the current interpreter (the one running repo) and their main() function is run. This mechanism is used (instead of using subprocess) to make it easier to expand to a richer hook interface in the future. During loading, the interpreter's sys.path is updated to contain the directory containing the hooks so that hooks can be split into multiple files. The upload command has two options that control hook behavior: - no-verify=False, verify=False (DEFAULT): If stdout is a tty, can prompt about running upload hooks if needed. If user denies running hooks, the upload is cancelled. If stdout is not a tty and we would need to prompt about upload hooks, upload is cancelled. - no-verify=False, verify=True: Always run upload hooks with no prompt. - no-verify=True, verify=False: Never run upload hooks, but upload anyway (AKA bypass hooks). - no-verify=True, verify=True: Invalid Sample bit of manifest.xml code for enabling hooks (assumes you have a project named 'hooks' where hooks are stored): <repo-hooks in-project="hooks" enabled-list="pre-upload" /> Sample main() function in pre-upload.py in hooks directory: def main(project_list, **kwargs): print ('These projects will be uploaded: %s' % ', '.join(project_list)) print ('I am being a good boy and ignoring anything in kwargs\n' 'that I don\'t understand.') print 'I fail 50% of the time. How flaky.' if random.random() <= .5: raise Exception('Pre-upload hook failed. Have a nice day.') Change-Id: I5cefa2cd5865c72589263cf8e2f152a43c122f70
2011-03-04 19:54:18 +00:00
for project in project_list:
if opt.current_branch:
cbr = project.CurrentBranch
avail = [project.GetUploadableBranch(cbr)] if cbr else None
else:
avail = project.GetUploadableBranches(branch)
Support repo-level pre-upload hook and prep for future hooks. All repo-level hooks are expected to live in a single project at the top level of that project. The name of the hooks project is provided in the manifest.xml. The manifest also lists which hooks are enabled to make it obvious if a file somehow failed to sync down (or got deleted). Before running any hook, we will prompt the user to make sure that it is OK. A user can deny running the hook, allow once, or allow "forever" (until hooks change). This tries to keep with the git spirit of not automatically running anything on the user's computer that got synced down. Note that individual repo commands can add always options to avoid these prompts as they see fit (see below for the 'upload' options). When hooks are run, they are loaded into the current interpreter (the one running repo) and their main() function is run. This mechanism is used (instead of using subprocess) to make it easier to expand to a richer hook interface in the future. During loading, the interpreter's sys.path is updated to contain the directory containing the hooks so that hooks can be split into multiple files. The upload command has two options that control hook behavior: - no-verify=False, verify=False (DEFAULT): If stdout is a tty, can prompt about running upload hooks if needed. If user denies running hooks, the upload is cancelled. If stdout is not a tty and we would need to prompt about upload hooks, upload is cancelled. - no-verify=False, verify=True: Always run upload hooks with no prompt. - no-verify=True, verify=False: Never run upload hooks, but upload anyway (AKA bypass hooks). - no-verify=True, verify=True: Invalid Sample bit of manifest.xml code for enabling hooks (assumes you have a project named 'hooks' where hooks are stored): <repo-hooks in-project="hooks" enabled-list="pre-upload" /> Sample main() function in pre-upload.py in hooks directory: def main(project_list, **kwargs): print ('These projects will be uploaded: %s' % ', '.join(project_list)) print ('I am being a good boy and ignoring anything in kwargs\n' 'that I don\'t understand.') print 'I fail 50% of the time. How flaky.' if random.random() <= .5: raise Exception('Pre-upload hook failed. Have a nice day.') Change-Id: I5cefa2cd5865c72589263cf8e2f152a43c122f70
2011-03-04 19:54:18 +00:00
if avail:
pending.append((project, avail))
if pending and (not opt.bypass_hooks):
hook = RepoHook('pre-upload', self.manifest.repo_hooks_project,
self.manifest.topdir, abort_if_user_denies=True)
pending_proj_names = [project.name for (project, avail) in pending]
try:
hook.Run(opt.allow_all_hooks, project_list=pending_proj_names)
except HookError as e:
print("ERROR: %s" % str(e), file=sys.stderr)
Support repo-level pre-upload hook and prep for future hooks. All repo-level hooks are expected to live in a single project at the top level of that project. The name of the hooks project is provided in the manifest.xml. The manifest also lists which hooks are enabled to make it obvious if a file somehow failed to sync down (or got deleted). Before running any hook, we will prompt the user to make sure that it is OK. A user can deny running the hook, allow once, or allow "forever" (until hooks change). This tries to keep with the git spirit of not automatically running anything on the user's computer that got synced down. Note that individual repo commands can add always options to avoid these prompts as they see fit (see below for the 'upload' options). When hooks are run, they are loaded into the current interpreter (the one running repo) and their main() function is run. This mechanism is used (instead of using subprocess) to make it easier to expand to a richer hook interface in the future. During loading, the interpreter's sys.path is updated to contain the directory containing the hooks so that hooks can be split into multiple files. The upload command has two options that control hook behavior: - no-verify=False, verify=False (DEFAULT): If stdout is a tty, can prompt about running upload hooks if needed. If user denies running hooks, the upload is cancelled. If stdout is not a tty and we would need to prompt about upload hooks, upload is cancelled. - no-verify=False, verify=True: Always run upload hooks with no prompt. - no-verify=True, verify=False: Never run upload hooks, but upload anyway (AKA bypass hooks). - no-verify=True, verify=True: Invalid Sample bit of manifest.xml code for enabling hooks (assumes you have a project named 'hooks' where hooks are stored): <repo-hooks in-project="hooks" enabled-list="pre-upload" /> Sample main() function in pre-upload.py in hooks directory: def main(project_list, **kwargs): print ('These projects will be uploaded: %s' % ', '.join(project_list)) print ('I am being a good boy and ignoring anything in kwargs\n' 'that I don\'t understand.') print 'I fail 50% of the time. How flaky.' if random.random() <= .5: raise Exception('Pre-upload hook failed. Have a nice day.') Change-Id: I5cefa2cd5865c72589263cf8e2f152a43c122f70
2011-03-04 19:54:18 +00:00
return
if opt.reviewers:
reviewers = _SplitEmails(opt.reviewers)
if opt.cc:
cc = _SplitEmails(opt.cc)
people = (reviewers, cc)
2008-10-21 14:00:00 +00:00
if not pending:
print("no branches ready for upload", file=sys.stderr)
2008-10-21 14:00:00 +00:00
elif len(pending) == 1 and len(pending[0][1]) == 1:
self._SingleBranch(opt, pending[0][1][0], people)
2008-10-21 14:00:00 +00:00
else:
self._MultipleBranches(opt, pending, people)