Added feature to print a <notice> from manifest at the end of a sync.

This feature is used to convey information on a when a branch has
ceased development or if it is an experimental branch with a few
gotchas, etc.

You add it to your manifest XML by doing something like this:
<manifest>
  <notice>
    NOTE TO DEVELOPERS:
      If you checkin code, you have to pinky-swear that it contains no bugs.
      Anyone who breaks their promise will have tomatoes thrown at them in the
      team meeting.  Be sure to bring an extra set of clothes.
  </notice>

  <remote ... />
  ...
</manifest>

Carriage returns and indentation are relevant for the text in this tag.

This feature was requested by Anush Elangovan on the ChromiumOS team.

.gitignore

@@ -1 +1,2 @@
 *.pyc
+.repopickle_*

.project

@@ -0,0 +1,17 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<projectDescription>
+	<name>repo</name>
+	<comment></comment>
+	<projects>
+	</projects>
+	<buildSpec>
+		<buildCommand>
+			<name>org.python.pydev.PyDevBuilder</name>
+			<arguments>
+			</arguments>
+		</buildCommand>
+	</buildSpec>
+	<natures>
+		<nature>org.python.pydev.pythonNature</nature>
+	</natures>
+</projectDescription>

.pydevproject

@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?eclipse-pydev version="1.0"?>
+
+<pydev_project>
+<pydev_pathproperty name="org.python.pydev.PROJECT_SOURCE_PATH">
+<path>/repo</path>
+</pydev_pathproperty>
+<pydev_property name="org.python.pydev.PYTHON_PROJECT_VERSION">python 2.4</pydev_property>
+<pydev_property name="org.python.pydev.PYTHON_PROJECT_INTERPRETER">Default</pydev_property>
+</pydev_project>

SUBMITTING_PATCHES

@@ -0,0 +1,80 @@
+Short Version:
+
+ - Make small logical changes.
+ - Provide a meaningful commit message.
+ - Make sure all code is under the Apache License, 2.0.
+ - Publish your changes for review:
+
+   git push ssh://review.source.android.com:29418/tools/repo.git HEAD:refs/for/master
+
+
+Long Version:
+
+I wanted a file describing how to submit patches for repo,
+so I started with the one found in the core Git distribution
+(Documentation/SubmittingPatches), which itself was based on the
+patch submission guidelines for the Linux kernel.
+
+However there are some differences, so please review and familiarize
+yourself with the following relevant bits:
+
+
+(1) Make separate commits for logically separate changes.
+
+Unless your patch is really trivial, you should not be sending
+out a patch that was generated between your working tree and your
+commit head.  Instead, always make a commit with complete commit
+message and generate a series of patches from your repository.
+It is a good discipline.
+
+Describe the technical detail of the change(s).
+
+If your description starts to get too long, that's a sign that you
+probably need to split up your commit to finer grained pieces.
+
+
+(2) Check the license
+
+repo is licensed under the Apache License, 2.0.
+
+Because of this licensing model *every* file within the project
+*must* list the license that covers it in the header of the file.
+Any new contributions to an existing file *must* be submitted under
+the current license of that file.  Any new files *must* clearly
+indicate which license they are provided under in the file header.
+
+Please verify that you are legally allowed and willing to submit your
+changes under the license covering each file *prior* to submitting
+your patch.  It is virtually impossible to remove a patch once it
+has been applied and pushed out.
+
+
+(3) Sending your patches.
+
+Do not email your patches to anyone.
+
+Instead, login to the Gerrit Code Review tool at:
+
+  https://review.source.android.com/
+
+Ensure you have completed one of the necessary contributor
+agreements, providing documentation to the project maintainers that
+they have right to redistribute your work under the Apache License:
+
+  https://review.source.android.com/#settings,agreements
+
+Ensure you have registered one or more SSH public keys, so you can
+push your commits directly over SSH:
+
+  https://review.source.android.com/#settings,ssh-keys
+
+Push your patches over SSH to the review server, possibly through
+a remembered remote to make this easier in the future:
+
+   git config remote.review.url ssh://review.source.android.com:29418/tools/repo.git
+   git config remote.review.push HEAD:refs/for/master
+
+   git push review
+
+You will be automatically emailed a copy of your commits, and any
+comments made by the project maintainers.

docs/manifest-format.txt

@@ -20,35 +20,33 @@ A manifest XML file (e.g. 'default.xml') roughly conforms to the
 following DTD:
 
   <!DOCTYPE manifest [
-    <!ELEMENT manifest (remote*,
+    <!ELEMENT manifest (notice?,
+                        remote*,
                         default?,
+                        manifest-server?,
                         remove-project*,
-                        project*,
-                        add-remote*)>
+                        project*)>
+  
+    <!ELEMENT notice (#PCDATA)>
   
     <!ELEMENT remote (EMPTY)>
     <!ATTLIST remote name         ID    #REQUIRED>
     <!ATTLIST remote fetch        CDATA #REQUIRED>
     <!ATTLIST remote review       CDATA #IMPLIED>
-    <!ATTLIST remote project-name CDATA #IMPLIED>
   
     <!ELEMENT default (EMPTY)>
     <!ATTLIST default remote   IDREF #IMPLIED>
     <!ATTLIST default revision CDATA #IMPLIED>
+
+    <!ELEMENT manifest-server (EMPTY)>
+    <!ATTLIST url              CDATA #REQUIRED>
   
-    <!ELEMENT project (remote*)>
+    <!ELEMENT project (EMPTY)>
     <!ATTLIST project name     CDATA #REQUIRED>
     <!ATTLIST project path     CDATA #IMPLIED>
     <!ATTLIST project remote   IDREF #IMPLIED>
     <!ATTLIST project revision CDATA #IMPLIED>
   
-    <!ELEMENT add-remote (EMPTY)>
-    <!ATTLIST add-remote to-project   ID    #REQUIRED>
-    <!ATTLIST add-remote name         ID    #REQUIRED>
-    <!ATTLIST add-remote fetch        CDATA #REQUIRED>
-    <!ATTLIST add-remote review       CDATA #IMPLIED>
-    <!ATTLIST add-remote project-name CDATA #IMPLIED>
-  
     <!ELEMENT remove-project (EMPTY)>
     <!ATTLIST remove-project name  CDATA #REQUIRED>
   ]>
@@ -82,25 +80,6 @@ Attribute `review`: Hostname of the Gerrit server where reviews
 are uploaded to by `repo upload`.  This attribute is optional;
 if not specified then `repo upload` will not function.
 
-Attribute `project-name`: Specifies the name of this project used
-by the review server given in the review attribute of this element.
-Only permitted when the remote element is nested inside of a project
-element (see below).  If not given, defaults to the name supplied
-in the project's name attribute.
-
-Element add-remote
-------------------
-
-Adds a remote to an existing project, whose name is given by the
-to-project attribute.  This is functionally equivalent to nesting
-a remote element under the project, but has the advantage that it
-can be specified in the uesr's `local_manifest.xml` to add a remote
-to a project declared by the normal manifest.
-
-The element can be used to add a fork of an existing project that
-the user needs to work with.
-
-
 Element default
 ---------------
 
@@ -117,6 +96,27 @@ Attribute `revision`: Name of a Git branch (e.g. `master` or
 revision attribute will use this revision.
 
 
+Element manifest-server
+-----------------------
+
+At most one manifest-server may be specified. The url attribute
+is used to specify the URL of a manifest server, which is an
+XML RPC service that will return a manifest in which each project
+is pegged to a known good revision for the current branch and
+target.
+
+The manifest server should implement:
+
+  GetApprovedManifest(branch, target)
+
+The target to use is defined by environment variables TARGET_PRODUCT
+and TARGET_BUILD_VARIANT. These variables are used to create a string
+of the form $TARGET_PRODUCT-$TARGET_BUILD_VARIANT, e.g. passion-userdebug.
+If one of those variables or both are not present, the program will call
+GetApprovedManifest without the target paramater and the manifest server
+should choose a reasonable default target.
+
+
 Element project
 ---------------
 
@@ -152,13 +152,6 @@ Tags and/or explicit SHA-1s should work in theory, but have not
 been extensively tested.  If not supplied the revision given by
 the default element is used.
 
-Child element `remote`: Described like the top-level remote element,
-but adds an additional remote to only this project.  These additional
-remotes are fetched from first on the initial `repo sync`, causing
-the majority of the project's object database to be obtained through
-these additional remotes.
-
-
 Element remove-project
 ----------------------
 

editor.py

@@ -14,6 +14,7 @@
 # limitations under the License.
 
 import os
+import re
 import sys
 import subprocess
 import tempfile
@@ -38,9 +39,10 @@ class Editor(object):
     if e:
       return e
 
-    e = cls.globalConfig.GetString('core.editor')
-    if e:
-      return e
+    if cls.globalConfig:
+      e = cls.globalConfig.GetString('core.editor')
+      if e:
+        return e
 
     e = os.getenv('VISUAL')
     if e:
@@ -69,15 +71,33 @@ least one of these before using this command."""
       Returns:
         new value of edited text; None if editing did not succeed
     """
-    editor = cls._GetEditor().split()
+    editor = cls._GetEditor()
+    if editor == ':':
+      return data
+
     fd, path = tempfile.mkstemp()
     try:
       os.write(fd, data)
       os.close(fd)
       fd = None
 
-      if subprocess.Popen(editor + [path]).wait() != 0:
-        raise EditorError()
+      if re.compile("^.*[$ \t'].*$").match(editor):
+        args = [editor + ' "$@"', 'sh']
+        shell = True
+      else:
+        args = [editor]
+        shell = False
+      args.append(path)
+
+      try:
+        rc = subprocess.Popen(args, shell=shell).wait()
+      except OSError, e:
+        raise EditorError('editor failed, %s: %s %s'
+          % (str(e), editor, path))
+      if rc != 0:
+        raise EditorError('editor failed with exit status %d: %s %s'
+          % (rc, editor, path))
+
       fd2 = open(path)
       try:
         return fd2.read()

error.py

@@ -24,6 +24,11 @@ class ManifestInvalidRevisionError(Exception):
 class EditorError(Exception):
   """Unspecified error from the user's text editor.
   """
+  def __init__(self, reason):
+    self.reason = reason
+
+  def __str__(self):
+    return self.reason
 
 class GitError(Exception):
   """Unspecified internal error from git.

git_command.py

@@ -17,6 +17,7 @@ import os
 import sys
 import subprocess
 import tempfile
+from signal import SIGTERM
 from error import GitError
 from trace import REPO_TRACE, IsTrace, Trace
 
@@ -29,8 +30,9 @@ LAST_CWD = None
 
 _ssh_proxy_path = None
 _ssh_sock_path = None
+_ssh_clients = []
 
-def _ssh_sock(create=True):
+def ssh_sock(create=True):
   global _ssh_sock_path
   if _ssh_sock_path is None:
     if not create:
@@ -51,6 +53,24 @@ def _ssh_proxy():
       'git_ssh')
   return _ssh_proxy_path
 
+def _add_ssh_client(p):
+  _ssh_clients.append(p)
+
+def _remove_ssh_client(p):
+  try:
+    _ssh_clients.remove(p)
+  except ValueError:
+    pass
+
+def terminate_ssh_clients():
+  global _ssh_clients
+  for p in _ssh_clients:
+    try:
+      os.kill(p.pid, SIGTERM)
+      p.wait()
+    except OSError:
+      pass
+  _ssh_clients = []
 
 class _GitCall(object):
   def version(self):
@@ -68,6 +88,30 @@ class _GitCall(object):
     return fun
 git = _GitCall()
 
+_git_version = None
+
+def git_require(min_version, fail=False):
+  global _git_version
+
+  if _git_version is None:
+    ver_str = git.version()
+    if ver_str.startswith('git version '):
+      _git_version = tuple(
+        map(lambda x: int(x),
+          ver_str[len('git version '):].strip().split('.')[0:3]
+        ))
+    else:
+      print >>sys.stderr, 'fatal: "%s" unsupported' % ver_str
+      sys.exit(1)
+
+  if min_version <= _git_version:
+    return True
+  if fail:
+    need = '.'.join(map(lambda x: str(x), min_version))
+    print >>sys.stderr, 'fatal: git %s or later required' % need
+    sys.exit(1)
+  return False
+
 class GitCommand(object):
   def __init__(self,
                project,
@@ -95,7 +139,7 @@ class GitCommand(object):
     if disable_editor:
       env['GIT_EDITOR'] = ':'
     if ssh_proxy:
-      env['REPO_SSH_SOCK'] = _ssh_sock()
+      env['REPO_SSH_SOCK'] = ssh_sock()
       env['GIT_SSH'] = _ssh_proxy()
 
     if project:
@@ -164,6 +208,9 @@ class GitCommand(object):
     except Exception, e:
       raise GitError('%s: %s' % (command[1], e))
 
+    if ssh_proxy:
+      _add_ssh_client(p)
+
     self.process = p
     self.stdin = p.stdin
 
@@ -186,4 +233,8 @@ class GitCommand(object):
     else:
       p.stderr = None
 
-    return self.process.wait()
+    try:
+      rc = p.wait()
+    finally:
+      _remove_ssh_client(p)
+    return rc

git_config.py

@@ -23,7 +23,10 @@ from signal import SIGTERM
 from urllib2 import urlopen, HTTPError
 from error import GitError, UploadError
 from trace import Trace
-from git_command import GitCommand, _ssh_sock
+
+from git_command import GitCommand
+from git_command import ssh_sock
+from git_command import terminate_ssh_clients
 
 R_HEADS = 'refs/heads/'
 R_TAGS  = 'refs/tags/'
@@ -56,16 +59,20 @@ class GitConfig(object):
     return cls(file = os.path.join(gitdir, 'config'),
                defaults = defaults)
 
-  def __init__(self, file, defaults=None):
+  def __init__(self, file, defaults=None, pickleFile=None):
     self.file = file
     self.defaults = defaults
     self._cache_dict = None
     self._section_dict = None
     self._remotes = {}
     self._branches = {}
-    self._pickle = os.path.join(
-      os.path.dirname(self.file),
-      '.repopickle_' + os.path.basename(self.file))
+
+    if pickleFile is None:
+      self._pickle = os.path.join(
+        os.path.dirname(self.file),
+        '.repopickle_' + os.path.basename(self.file))
+    else:
+      self._pickle = pickleFile
 
   def Has(self, name, include_defaults = True):
     """Return true if this configuration file has the key.
@@ -172,6 +179,11 @@ class GitConfig(object):
       self._branches[b.name] = b
     return b
 
+  def GetSubSections(self, section):
+    """List all subsection names matching $section.*.*
+    """
+    return self._sections.get(section, set())
+
   def HasSection(self, section, subsection = ''):
     """Does at least one key in section.subsection exist?
     """
@@ -227,6 +239,9 @@ class GitConfig(object):
         return cPickle.load(fd)
       finally:
         fd.close()
+    except EOFError:
+      os.remove(self._pickle)
+      return None
     except IOError:
       os.remove(self._pickle)
       return None
@@ -242,26 +257,35 @@ class GitConfig(object):
       finally:
         fd.close()
     except IOError:
-      os.remove(self._pickle)
+      if os.path.exists(self._pickle):
+        os.remove(self._pickle)
     except cPickle.PickleError:
-      os.remove(self._pickle)
+      if os.path.exists(self._pickle):
+        os.remove(self._pickle)
 
   def _ReadGit(self):
-    d = self._do('--null', '--list')
-    c = {}
-    while d:
-      lf = d.index('\n')
-      nul = d.index('\0', lf + 1)
+    """
+    Read configuration data from git.
 
-      key = _key(d[0:lf])
-      val = d[lf + 1:nul]
+    This internal method populates the GitConfig cache.
+
+    """
+    c = {}
+    d = self._do('--null', '--list')
+    if d is None:
+      return c
+    for line in d.rstrip('\0').split('\0'):
+      if '\n' in line:
+          key, val = line.split('\n', 1)
+      else:
+          key = line
+          val = None
 
       if key in c:
         c[key].append(val)
       else:
         c[key] = [val]
 
-      d = d[nul + 1:]
     return c
 
   def _do(self, *args):
@@ -334,32 +358,63 @@ class RefSpec(object):
     return s
 
 
-_ssh_cache = {}
+_master_processes = []
+_master_keys = set()
 _ssh_master = True
 
 def _open_ssh(host, port=None):
   global _ssh_master
 
-  if port is None:
-    port = 22
+  # Check to see whether we already think that the master is running; if we
+  # think it's already running, return right away.
+  if port is not None:
+    key = '%s:%s' % (host, port)
+  else:
+    key = host
 
-  key = '%s:%s' % (host, port)
-  if key in _ssh_cache:
+  if key in _master_keys:
     return True
 
   if not _ssh_master \
   or 'GIT_SSH' in os.environ \
-  or sys.platform == 'win32':
+  or sys.platform in ('win32', 'cygwin'):
     # failed earlier, or cygwin ssh can't do this
     #
     return False
 
-  command = ['ssh',
-             '-o','ControlPath %s' % _ssh_sock(),
-             '-p',str(port),
-             '-M',
-             '-N',
-             host]
+  # We will make two calls to ssh; this is the common part of both calls.
+  command_base = ['ssh',
+                   '-o','ControlPath %s' % ssh_sock(),
+                   host]
+  if port is not None:
+    command_base[1:1] = ['-p',str(port)]
+
+  # Since the key wasn't in _master_keys, we think that master isn't running.
+  # ...but before actually starting a master, we'll double-check.  This can
+  # be important because we can't tell that that 'git@myhost.com' is the same
+  # as 'myhost.com' where "User git" is setup in the user's ~/.ssh/config file.
+  check_command = command_base + ['-O','check']
+  try:
+    Trace(': %s', ' '.join(check_command))
+    check_process = subprocess.Popen(check_command,
+                                     stdout=subprocess.PIPE,
+                                     stderr=subprocess.PIPE)
+    check_process.communicate() # read output, but ignore it...
+    isnt_running = check_process.wait()
+
+    if not isnt_running:
+      # Our double-check found that the master _was_ infact running.  Add to
+      # the list of keys.
+      _master_keys.add(key)
+      return True
+  except Exception:
+    # Ignore excpetions.  We we will fall back to the normal command and print
+    # to the log there.
+    pass
+
+  command = command_base[:1] + \
+            ['-M', '-N'] + \
+            command_base[1:]
   try:
     Trace(': %s', ' '.join(command))
     p = subprocess.Popen(command)
@@ -370,17 +425,24 @@ def _open_ssh(host, port=None):
       % (host,port, str(e))
     return False
 
-  _ssh_cache[key] = p
+  _master_processes.append(p)
+  _master_keys.add(key)
   time.sleep(1)
   return True
 
 def close_ssh():
-  for key,p in _ssh_cache.iteritems():
-    os.kill(p.pid, SIGTERM)
-    p.wait()
-  _ssh_cache.clear()
+  terminate_ssh_clients()
 
-  d = _ssh_sock(create=False)
+  for p in _master_processes:
+    try:
+      os.kill(p.pid, SIGTERM)
+      p.wait()
+    except OSError:
+      pass
+  del _master_processes[:]
+  _master_keys.clear()
+
+  d = ssh_sock(create=False)
   if d:
     try:
       os.rmdir(os.path.dirname(d))
@@ -388,7 +450,7 @@ def close_ssh():
       pass
 
 URI_SCP = re.compile(r'^([^@:]*@?[^:/]{1,}):')
-URI_ALL = re.compile(r'^([a-z][a-z+]*)://([^@/]*@?[^/])/')
+URI_ALL = re.compile(r'^([a-z][a-z+]*)://([^@/]*@?[^/]*)/')
 
 def _preconnect(url):
   m = URI_ALL.match(url)
@@ -397,6 +459,8 @@ def _preconnect(url):
     host = m.group(2)
     if ':' in host:
       host, port = host.split(':')
+    else:
+      port = None
     if scheme in ('ssh', 'git+ssh', 'ssh+git'):
       return _open_ssh(host, port)
     return False
@@ -406,6 +470,7 @@ def _preconnect(url):
     host = m.group(1)
     return _open_ssh(host)
 
+  return False
 
 class Remote(object):
   """Configuration options related to a remote.
@@ -420,8 +485,30 @@ class Remote(object):
                      self._Get('fetch', all=True))
     self._review_protocol = None
 
+  def _InsteadOf(self):
+    globCfg = GitConfig.ForUser()
+    urlList = globCfg.GetSubSections('url')
+    longest = ""
+    longestUrl = ""
+
+    for url in urlList:
+      key = "url." + url + ".insteadOf"
+      insteadOfList = globCfg.GetString(key, all=True)
+
+      for insteadOf in insteadOfList:
+        if self.url.startswith(insteadOf) \
+        and len(insteadOf) > len(longest):
+          longest = insteadOf
+          longestUrl = url
+
+    if len(longest) == 0:
+      return self.url
+
+    return self.url.replace(longest, longestUrl, 1)
+
   def PreConnectFetch(self):
-    return _preconnect(self.url)
+    connectionUrl = self._InsteadOf()
+    return _preconnect(connectionUrl)
 
   @property
   def ReviewProtocol(self):
@@ -475,8 +562,11 @@ class Remote(object):
   def SshReviewUrl(self, userEmail):
     if self.ReviewProtocol != 'ssh':
       return None
+    username = self._config.GetString('review.%s.username' % self.review)
+    if username is None:
+      username = userEmail.split("@")[0]
     return 'ssh://%s@%s:%s/%s' % (
-      userEmail.split("@")[0],
+      username,
       self._review_host,
       self._review_port,
       self.projectname)

git_ssh

@@ -1,2 +1,2 @@
 #!/bin/sh
-exec ssh -o "ControlPath $REPO_SSH_SOCK" "$@"
+exec ssh -o "ControlMaster no" -o "ControlPath $REPO_SSH_SOCK" "$@"

hooks/commit-msg

@@ -0,0 +1,101 @@
+#!/bin/sh
+# From Gerrit Code Review 2.1.2-rc2-33-g7e30c72
+#
+# Part of Gerrit Code Review (http://code.google.com/p/gerrit/)
+#
+# Copyright (C) 2009 The Android Open Source Project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+CHANGE_ID_AFTER="Bug|Issue"
+MSG="$1"
+
+# Check for, and add if missing, a unique Change-Id
+#
+add_ChangeId() {
+	clean_message=$(sed -e '
+		/^diff --git a\/.*/{
+			s///
+			q
+		}
+		/^Signed-off-by:/d
+		/^#/d
+	' "$MSG" | git stripspace)
+	if test -z "$clean_message"
+	then
+		return
+	fi
+
+	if grep -i '^Change-Id:' "$MSG" >/dev/null
+	then
+		return
+	fi
+
+	id=$(_gen_ChangeId)
+	perl -e '
+		$MSG = shift;
+		$id = shift;
+		$CHANGE_ID_AFTER = shift;
+
+		undef $/;
+		open(I, $MSG); $_ = <I>; close I;
+		s|^diff --git a/.*||ms;
+		s|^#.*$||mg;
+		exit unless $_;
+
+		@message = split /\n/;
+		$haveFooter = 0;
+		$startFooter = @message;
+		for($line = @message - 1; $line >= 0; $line--) {
+			$_ = $message[$line];
+
+			($haveFooter++, next) if /^[a-zA-Z0-9-]+:/;
+			next if /^[ []/;
+			$startFooter = $line if ($haveFooter && /^\r?$/);
+			last;
+		}
+
+		@footer = @message[$startFooter+1..@message];
+		@message = @message[0..$startFooter];
+		push(@footer, "") unless @footer;
+
+		for ($line = 0; $line < @footer; $line++) {
+			$_ = $footer[$line];
+			next if /^($CHANGE_ID_AFTER):/i;
+			last;
+		}
+		splice(@footer, $line, 0, "Change-Id: I$id");
+
+		$_ = join("\n", @message, @footer);
+		open(O, ">$MSG"); print O; close O;
+	' "$MSG" "$id" "$CHANGE_ID_AFTER"
+}
+_gen_ChangeIdInput() {
+	echo "tree $(git write-tree)"
+	if parent=$(git rev-parse HEAD^0 2>/dev/null)
+	then
+		echo "parent $parent"
+	fi
+	echo "author $(git var GIT_AUTHOR_IDENT)"
+	echo "committer $(git var GIT_COMMITTER_IDENT)"
+	echo
+	printf '%s' "$clean_message"
+}
+_gen_ChangeId() {
+	_gen_ChangeIdInput |
+	git hash-object -t commit --stdin
+}
+
+
+add_ChangeId

main.py

@@ -36,7 +36,7 @@ from editor import Editor
 from error import ManifestInvalidRevisionError
 from error import NoSuchProjectError
 from error import RepoChangedException
-from manifest import Manifest
+from manifest_xml import XmlManifest
 from pager import RunPager
 
 from subcmds import all as all_commands
@@ -97,7 +97,7 @@ class _Repo(object):
       sys.exit(1)
 
     cmd.repodir = self.repodir
-    cmd.manifest = Manifest(cmd.repodir)
+    cmd.manifest = XmlManifest(cmd.repodir)
     Editor.globalConfig = cmd.manifest.globalConfig
 
     if not isinstance(cmd, MirrorSafeCommand) and cmd.manifest.IsMirror:

manifest_xml.py

@@ -18,8 +18,7 @@ import sys
 import xml.dom.minidom
 
 from git_config import GitConfig, IsId
-from project import Project, MetaProject, R_HEADS, HEAD
-from remote import Remote
+from project import RemoteSpec, Project, MetaProject, R_HEADS, HEAD
 from error import ManifestParseError
 
 MANIFEST_FILE_NAME = 'manifest.xml'
@@ -28,11 +27,26 @@ LOCAL_MANIFEST_NAME = 'local_manifest.xml'
 class _Default(object):
   """Project defaults within the manifest."""
 
-  revision = None
+  revisionExpr = None
   remote = None
 
+class _XmlRemote(object):
+  def __init__(self,
+               name,
+               fetch=None,
+               review=None):
+    self.name = name
+    self.fetchUrl = fetch
+    self.reviewUrl = review
 
-class Manifest(object):
+  def ToRemoteSpec(self, projectName):
+    url = self.fetchUrl
+    while url.endswith('/'):
+      url = url[:-1]
+    url += '/%s.git' % projectName
+    return RemoteSpec(self.name, url, self.reviewUrl)
+
+class XmlManifest(object):
   """manages the repo configuration file"""
 
   def __init__(self, repodir):
@@ -51,8 +65,8 @@ class Manifest(object):
 
     self._Unload()
 
-  def Link(self, name):
-    """Update the repo metadata to use a different manifest.
+  def Override(self, name):
+    """Use a different manifest, just for the current instantiation.
     """
     path = os.path.join(self.manifestProject.worktree, name)
     if not os.path.isfile(path):
@@ -66,6 +80,11 @@ class Manifest(object):
     finally:
       self.manifestFile = old
 
+  def Link(self, name):
+    """Update the repo metadata to use a different manifest.
+    """
+    self.Override(name)
+
     try:
       if os.path.exists(self.manifestFile):
         os.remove(self.manifestFile)
@@ -80,8 +99,6 @@ class Manifest(object):
     e.setAttribute('fetch', r.fetchUrl)
     if r.reviewUrl is not None:
       e.setAttribute('review', r.reviewUrl)
-    if r.projectName is not None:
-      e.setAttribute('project-name', r.projectName)
 
   def Save(self, fd, peg_rev=False):
     """Write the current manifest out to the given file descriptor.
@@ -90,6 +107,15 @@ class Manifest(object):
     root = doc.createElement('manifest')
     doc.appendChild(root)
 
+    # Save out the notice.  There's a little bit of work here to give it the
+    # right whitespace, which assumes that the notice is automatically indented
+    # by 4 by minidom.
+    if self.notice:
+      notice_element = root.appendChild(doc.createElement('notice'))
+      notice_lines = self.notice.splitlines()
+      indented_notice = ('\n'.join(" "*4 + line for line in notice_lines))[4:]
+      notice_element.appendChild(doc.createTextNode(indented_notice))
+
     d = self.default
     sort_remotes = list(self.remotes.keys())
     sort_remotes.sort()
@@ -104,13 +130,19 @@ class Manifest(object):
     if d.remote:
       have_default = True
       e.setAttribute('remote', d.remote.name)
-    if d.revision:
+    if d.revisionExpr:
       have_default = True
-      e.setAttribute('revision', d.revision)
+      e.setAttribute('revision', d.revisionExpr)
     if have_default:
       root.appendChild(e)
       root.appendChild(doc.createTextNode(''))
 
+    if self._manifest_server:
+      e = doc.createElement('manifest-server')
+      e.setAttribute('url', self._manifest_server)
+      root.appendChild(e)
+      root.appendChild(doc.createTextNode(''))
+
     sort_projects = list(self.projects.keys())
     sort_projects.sort()
 
@@ -126,15 +158,13 @@ class Manifest(object):
       if peg_rev:
         if self.IsMirror:
           e.setAttribute('revision',
-                         p.bare_git.rev_parse(p.revision + '^0'))
+                         p.bare_git.rev_parse(p.revisionExpr + '^0'))
         else:
           e.setAttribute('revision',
                          p.work_git.rev_parse(HEAD + '^0'))
-      elif not d.revision or p.revision != d.revision:
-        e.setAttribute('revision', p.revision)
+      elif not d.revisionExpr or p.revisionExpr != d.revisionExpr:
+        e.setAttribute('revision', p.revisionExpr)
 
-      for r in p.extraRemotes:
-        self._RemoteToXml(p.extraRemotes[r], doc, e)
       for c in p.copyfiles:
         ce = doc.createElement('copyfile')
         ce.setAttribute('src', c.src)
@@ -158,6 +188,16 @@ class Manifest(object):
     self._Load()
     return self._default
 
+  @property
+  def notice(self):
+    self._Load()
+    return self._notice
+
+  @property
+  def manifest_server(self):
+    self._Load()
+    return self._manifest_server
+
   @property
   def IsMirror(self):
     return self.manifestProject.config.GetBoolean('repo.mirror')
@@ -167,13 +207,15 @@ class Manifest(object):
     self._projects = {}
     self._remotes = {}
     self._default = None
+    self._notice = None
     self.branch = None
+    self._manifest_server = None
 
   def _Load(self):
     if not self._loaded:
       m = self.manifestProject
       b = m.GetBranch(m.CurrentBranch).merge
-      if b.startswith(R_HEADS):
+      if b is not None and b.startswith(R_HEADS):
         b = b[len(R_HEADS):]
       self.branch = b
 
@@ -236,6 +278,23 @@ class Manifest(object):
     if self._default is None:
       self._default = _Default()
 
+    for node in config.childNodes:
+      if node.nodeName == 'notice':
+        if self._notice is not None:
+          raise ManifestParseError, \
+                'duplicate notice in %s' % \
+                (self.manifestFile)
+        self._notice = self._ParseNotice(node)
+
+    for node in config.childNodes:
+      if node.nodeName == 'manifest-server':
+        url = self._reqatt(node, 'url')
+        if self._manifest_server is not None:
+            raise ManifestParseError, \
+                'duplicate manifest-server in %s' % \
+                (self.manifestFile)
+        self._manifest_server = url
+
     for node in config.childNodes:
       if node.nodeName == 'project':
         project = self._ParseProject(node)
@@ -245,16 +304,6 @@ class Manifest(object):
                 (project.name, self.manifestFile)
         self._projects[project.name] = project
 
-    for node in config.childNodes:
-      if node.nodeName == 'add-remote':
-        pn = self._reqatt(node, 'to-project')
-        project = self._projects.get(pn)
-        if not project:
-          raise ManifestParseError, \
-                'project %s not defined in %s' % \
-                (pn, self.manifestFile)
-        self._ParseProjectExtraRemote(project, node)
-
   def _AddMetaProjectMirror(self, m):
     name = None
     m_url = m.GetRemote(m.remote.name).url
@@ -271,7 +320,7 @@ class Manifest(object):
 
     if name is None:
       s = m_url.rindex('/') + 1
-      remote = Remote('origin', fetch = m_url[:s])
+      remote = _XmlRemote('origin', m_url[:s])
       name = m_url[s:]
 
     if name.endswith('.git'):
@@ -282,11 +331,12 @@ class Manifest(object):
       gitdir = os.path.join(self.topdir, '%s.git' % name)
       project = Project(manifest = self,
                         name = name,
-                        remote = remote,
+                        remote = remote.ToRemoteSpec(name),
                         gitdir = gitdir,
                         worktree = None,
                         relpath = None,
-                        revision = m.revision)
+                        revisionExpr = m.revisionExpr,
+                        revisionId = None)
       self._projects[project.name] = project
 
   def _ParseRemote(self, node):
@@ -298,21 +348,7 @@ class Manifest(object):
     review = node.getAttribute('review')
     if review == '':
       review = None
-
-    projectName = node.getAttribute('project-name')
-    if projectName == '':
-      projectName = None
-
-    r = Remote(name=name,
-               fetch=fetch,
-               review=review,
-               projectName=projectName)
-
-    for n in node.childNodes:
-      if n.nodeName == 'require':
-        r.requiredCommits.append(self._reqatt(n, 'commit'))
-
-    return r
+    return _XmlRemote(name, fetch, review)
 
   def _ParseDefault(self, node):
     """
@@ -320,15 +356,54 @@ class Manifest(object):
     """
     d = _Default()
     d.remote = self._get_remote(node)
-    d.revision = node.getAttribute('revision')
-    if d.revision == '':
-      d.revision = None
+    d.revisionExpr = node.getAttribute('revision')
+    if d.revisionExpr == '':
+      d.revisionExpr = None
     return d
 
+  def _ParseNotice(self, node):
+    """
+    reads a <notice> element from the manifest file
+
+    The <notice> element is distinct from other tags in the XML in that the
+    data is conveyed between the start and end tag (it's not an empty-element
+    tag).
+
+    The white space (carriage returns, indentation) for the notice element is
+    relevant and is parsed in a way that is based on how python docstrings work.
+    In fact, the code is remarkably similar to here:
+      http://www.python.org/dev/peps/pep-0257/
+    """
+    # Get the data out of the node...
+    notice = node.childNodes[0].data
+
+    # Figure out minimum indentation, skipping the first line (the same line
+    # as the <notice> tag)...
+    minIndent = sys.maxint
+    lines = notice.splitlines()
+    for line in lines[1:]:
+      lstrippedLine = line.lstrip()
+      if lstrippedLine:
+        indent = len(line) - len(lstrippedLine)
+        minIndent = min(indent, minIndent)
+
+    # Strip leading / trailing blank lines and also indentation.
+    cleanLines = [lines[0].strip()]
+    for line in lines[1:]:
+      cleanLines.append(line[minIndent:].rstrip())
+
+    # Clear completely blank lines from front and back...
+    while cleanLines and not cleanLines[0]:
+      del cleanLines[0]
+    while cleanLines and not cleanLines[-1]:
+      del cleanLines[-1]
+
+    return '\n'.join(cleanLines)
+
   def _ParseProject(self, node):
     """
     reads a <project> element from the manifest file
-    """ 
+    """
     name = self._reqatt(node, 'name')
 
     remote = self._get_remote(node)
@@ -339,10 +414,10 @@ class Manifest(object):
             "no remote for project %s within %s" % \
             (name, self.manifestFile)
 
-    revision = node.getAttribute('revision')
-    if not revision:
-      revision = self._default.revision
-    if not revision:
+    revisionExpr = node.getAttribute('revision')
+    if not revisionExpr:
+      revisionExpr = self._default.revisionExpr
+    if not revisionExpr:
       raise ManifestParseError, \
             "no revision for project %s within %s" % \
             (name, self.manifestFile)
@@ -365,29 +440,19 @@ class Manifest(object):
 
     project = Project(manifest = self,
                       name = name,
-                      remote = remote,
+                      remote = remote.ToRemoteSpec(name),
                       gitdir = gitdir,
                       worktree = worktree,
                       relpath = path,
-                      revision = revision)
+                      revisionExpr = revisionExpr,
+                      revisionId = None)
 
     for n in node.childNodes:
-      if n.nodeName == 'remote':
-        self._ParseProjectExtraRemote(project, n)
-      elif n.nodeName == 'copyfile':
+      if n.nodeName == 'copyfile':
         self._ParseCopyFile(project, n)
 
     return project
 
-  def _ParseProjectExtraRemote(self, project, n):
-    r = self._ParseRemote(n)
-    if project.extraRemotes.get(r.name) \
-       or project.remote.name == r.name:
-      raise ManifestParseError, \
-            'duplicate remote %s in project %s in %s' % \
-            (r.name, project.name, self.manifestFile)
-    project.extraRemotes[r.name] = r
-
   def _ParseCopyFile(self, project, node):
     src = self._reqatt(node, 'src')
     dest = self._reqatt(node, 'dest')

progress.py

@@ -13,10 +13,13 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import os
 import sys
 from time import time
 from trace import IsTrace
 
+_NOT_TTY = not os.isatty(2)
+
 class Progress(object):
   def __init__(self, title, total=0):
     self._title = title
@@ -29,7 +32,7 @@ class Progress(object):
   def update(self, inc=1):
     self._done += inc
 
-    if IsTrace():
+    if _NOT_TTY or IsTrace():
       return
 
     if not self._show:
@@ -56,7 +59,7 @@ class Progress(object):
         sys.stderr.flush()
 
   def end(self):
-    if IsTrace() or not self._show:
+    if _NOT_TTY or IsTrace() or not self._show:
       return
 
     if self._total <= 0:

project.py

@@ -26,7 +26,6 @@ from git_command import GitCommand
 from git_config import GitConfig, IsId
 from error import GitError, ImportError, UploadError
 from error import ManifestInvalidRevisionError
-from remote import Remote
 
 from git_refs import GitRefs, HEAD, R_HEADS, R_TAGS, R_PUB, R_M
 
@@ -112,7 +111,6 @@ class ReviewableBranch(object):
     self.project = project
     self.branch = branch
     self.base = base
-    self.replace_changes = None
 
   @property
   def name(self):
@@ -150,11 +148,24 @@ class ReviewableBranch(object):
       R_HEADS + self.name,
       '--')
 
-  def UploadForReview(self, people):
+  def UploadForReview(self, people, auto_topic=False):
     self.project.UploadForReview(self.name,
-                                 self.replace_changes,
-                                 people)
+                                 people,
+                                 auto_topic=auto_topic)
 
+  def GetPublishedRefs(self):
+    refs = {}
+    output = self.project.bare_git.ls_remote(
+      self.branch.remote.SshReviewUrl(self.project.UserEmail),
+      'refs/changes/*')
+    for line in output.split('\n'):
+      try:
+        (sha, ref) = line.split()
+        refs[sha] = ref
+      except ValueError:
+        pass
+
+    return refs
 
 class StatusColoring(Coloring):
   def __init__(self, config):
@@ -191,6 +202,10 @@ class _CopyFile:
         # remove existing file first, since it might be read-only
         if os.path.exists(dest):
           os.remove(dest)
+        else:
+          dir = os.path.dirname(dest)
+          if not os.path.isdir(dir):
+            os.makedirs(dir)
         shutil.copy(src, dest)
         # make the file read-only
         mode = os.stat(dest)[stat.ST_MODE]
@@ -199,6 +214,14 @@ class _CopyFile:
       except IOError:
         _error('Cannot copy file %s to %s', src, dest)
 
+class RemoteSpec(object):
+  def __init__(self,
+               name,
+               url = None,
+               review = None):
+    self.name = name
+    self.url = url
+    self.review = review
 
 class Project(object):
   def __init__(self,
@@ -208,16 +231,24 @@ class Project(object):
                gitdir,
                worktree,
                relpath,
-               revision):
+               revisionExpr,
+               revisionId):
     self.manifest = manifest
     self.name = name
     self.remote = remote
     self.gitdir = gitdir
     self.worktree = worktree
     self.relpath = relpath
-    self.revision = revision
+    self.revisionExpr = revisionExpr
+
+    if   revisionId is None \
+     and revisionExpr \
+     and IsId(revisionExpr):
+      self.revisionId = revisionExpr
+    else:
+      self.revisionId = revisionId
+
     self.snapshots = {}
-    self.extraRemotes = {}
     self.copyfiles = []
     self.config = GitConfig.ForRepository(
                     gitdir = self.gitdir,
@@ -251,7 +282,7 @@ class Project(object):
     return os.path.exists(os.path.join(g, 'rebase-apply')) \
         or os.path.exists(os.path.join(g, 'rebase-merge')) \
         or os.path.exists(os.path.join(w, '.dotest'))
-    
+
   def IsDirty(self, consider_untracked=True):
     """Is the working directory modified in some way?
     """
@@ -336,6 +367,27 @@ class Project(object):
 
 ## Status Display ##
 
+  def HasChanges(self):
+    """Returns true if there are uncommitted changes.
+    """
+    self.work_git.update_index('-q',
+                               '--unmerged',
+                               '--ignore-missing',
+                               '--refresh')
+    if self.IsRebaseInProgress():
+      return True
+
+    if self.work_git.DiffZ('diff-index', '--cached', HEAD):
+      return True
+
+    if self.work_git.DiffZ('diff-files'):
+      return True
+
+    if self.work_git.LsOthers():
+      return True
+
+    return False
+
   def PrintWorkTreeStatus(self):
     """Prints the status of the repository to stdout.
     """
@@ -384,7 +436,7 @@ class Project(object):
 
       try: f = df[p]
       except KeyError: f = None
- 
+
       if i: i_status = i.status.upper()
       else: i_status = '-'
 
@@ -502,7 +554,9 @@ class Project(object):
         return rb
     return None
 
-  def UploadForReview(self, branch=None, replace_changes=None, people=([],[])):
+  def UploadForReview(self, branch=None,
+                      people=([],[]),
+                      auto_topic=False):
     """Uploads the named branch for code review.
     """
     if branch is None:
@@ -534,13 +588,15 @@ class Project(object):
       for e in people[1]:
         rp.append('--cc=%s' % sq(e))
 
+      ref_spec = '%s:refs/for/%s' % (R_HEADS + branch.name, dest_branch)
+      if auto_topic:
+        ref_spec = ref_spec + '/' + branch.name
+
       cmd = ['push']
       cmd.append('--receive-pack=%s' % " ".join(rp))
       cmd.append(branch.remote.SshReviewUrl(self.UserEmail))
-      cmd.append('%s:refs/for/%s' % (R_HEADS + branch.name, dest_branch))
-      if replace_changes:
-        for change_id,commit_id in replace_changes.iteritems():
-          cmd.append('%s:refs/changes/%s/new' % (commit_id, change_id))
+      cmd.append(ref_spec)
+
       if GitCommand(self, cmd, bare = True).Wait() != 0:
         raise UploadError('Upload failed')
 
@@ -556,22 +612,33 @@ class Project(object):
 
 ## Sync ##
 
-  def Sync_NetworkHalf(self):
+  def Sync_NetworkHalf(self, quiet=False):
     """Perform only the network IO portion of the sync process.
        Local working directory/branch state is not affected.
     """
-    if not self.Exists:
-      print >>sys.stderr
-      print >>sys.stderr, 'Initializing project %s ...' % self.name
+    is_new = not self.Exists
+    if is_new:
+      if not quiet:
+        print >>sys.stderr
+        print >>sys.stderr, 'Initializing project %s ...' % self.name
       self._InitGitDir()
 
     self._InitRemote()
-    for r in self.extraRemotes.values():
-      if not self._RemoteFetch(r.name):
-        return False
-    if not self._RemoteFetch():
+    if not self._RemoteFetch(initial=is_new, quiet=quiet):
       return False
 
+    #Check that the requested ref was found after fetch
+    #
+    try:
+      self.GetRevisionId()
+    except ManifestInvalidRevisionError:
+      # if the ref is a tag. We can try fetching
+      # the tag manually as a last resort
+      #
+      rev = self.revisionExpr
+      if rev.startswith(R_TAGS):
+        self._RemoteFetch(None, rev[len(R_TAGS):], quiet=quiet)
+
     if self.worktree:
       self._InitMRef()
     else:
@@ -589,6 +656,23 @@ class Project(object):
     for file in self.copyfiles:
       file._Copy()
 
+  def GetRevisionId(self, all=None):
+    if self.revisionId:
+      return self.revisionId
+
+    rem = self.GetRemote(self.remote.name)
+    rev = rem.ToLocal(self.revisionExpr)
+
+    if all is not None and rev in all:
+      return all[rev]
+
+    try:
+      return self.bare_git.rev_parse('--verify', '%s^0' % rev)
+    except GitError:
+      raise ManifestInvalidRevisionError(
+        'revision %s in %s not found' % (self.revisionExpr,
+                                         self.name))
+
   def Sync_LocalHalf(self, syncbuf):
     """Perform only the local IO portion of the sync process.
        Network access is not required.
@@ -597,19 +681,7 @@ class Project(object):
     all = self.bare_ref.all
     self.CleanPublishedCache(all)
 
-    rem = self.GetRemote(self.remote.name)
-    rev = rem.ToLocal(self.revision)
-    if rev in all:
-      revid = all[rev]
-    elif IsId(rev):
-      revid = rev
-    else:
-      try:
-        revid = self.bare_git.rev_parse('--verify', '%s^0' % rev)
-      except GitError:
-        raise ManifestInvalidRevisionError(
-          'revision %s in %s not found' % (self.revision, self.name))
-
+    revid = self.GetRevisionId(all)
     head = self.work_git.GetHead()
     if head.startswith(R_HEADS):
       branch = head[len(R_HEADS):]
@@ -633,11 +705,11 @@ class Project(object):
         #
         return
 
-      lost = self._revlist(not_rev(rev), HEAD)
+      lost = self._revlist(not_rev(revid), HEAD)
       if lost:
         syncbuf.info(self, "discarding %d commits", len(lost))
       try:
-        self._Checkout(rev, quiet=True)
+        self._Checkout(revid, quiet=True)
       except GitError, e:
         syncbuf.fail(self, e)
         return
@@ -650,9 +722,8 @@ class Project(object):
       return
 
     branch = self.GetBranch(branch)
-    merge = branch.LocalMerge
 
-    if not merge:
+    if not branch.LocalMerge:
       # The current branch has no tracking configuration.
       # Jump off it to a deatched HEAD.
       #
@@ -660,96 +731,96 @@ class Project(object):
                    "leaving %s; does not track upstream",
                    branch.name)
       try:
-        self._Checkout(rev, quiet=True)
+        self._Checkout(revid, quiet=True)
       except GitError, e:
         syncbuf.fail(self, e)
         return
       self._CopyFiles()
       return
 
-    upstream_gain = self._revlist(not_rev(HEAD), rev)
+    upstream_gain = self._revlist(not_rev(HEAD), revid)
     pub = self.WasPublished(branch.name, all)
     if pub:
-      not_merged = self._revlist(not_rev(rev), pub)
+      not_merged = self._revlist(not_rev(revid), pub)
       if not_merged:
         if upstream_gain:
           # The user has published this branch and some of those
           # commits are not yet merged upstream.  We do not want
           # to rewrite the published commits so we punt.
           #
-          syncbuf.info(self,
-                       "branch %s is published but is now %d commits behind",
-                       branch.name,
-                       len(upstream_gain))
+          syncbuf.fail(self,
+                       "branch %s is published (but not merged) and is now %d commits behind"
+                       % (branch.name, len(upstream_gain)))
         return
       elif pub == head:
         # All published commits are merged, and thus we are a
         # strict subset.  We can fast-forward safely.
         #
         def _doff():
-          self._FastForward(rev)
+          self._FastForward(revid)
           self._CopyFiles()
         syncbuf.later1(self, _doff)
         return
 
-    if merge == rev:
-      try:
-        old_merge = self.bare_git.rev_parse('%s@{1}' % merge)
-      except GitError:
-        old_merge = merge
-      if old_merge == '0000000000000000000000000000000000000000' \
-         or old_merge == '':
-        old_merge = merge
-    else:
-      # The upstream switched on us.  Time to cross our fingers
-      # and pray that the old upstream also wasn't in the habit
-      # of rebasing itself.
-      #
-      syncbuf.info(self, "manifest switched %s...%s", merge, rev)
-      old_merge = merge
+    # Examine the local commits not in the remote.  Find the
+    # last one attributed to this user, if any.
+    #
+    local_changes = self._revlist(not_rev(revid), HEAD, format='%H %ce')
+    last_mine = None
+    cnt_mine = 0
+    for commit in local_changes:
+      commit_id, committer_email = commit.split(' ', 1)
+      if committer_email == self.UserEmail:
+        last_mine = commit_id
+        cnt_mine += 1
 
-    if rev == old_merge:
-      upstream_lost = []
-    else:
-      upstream_lost = self._revlist(not_rev(rev), old_merge)
-
-    if not upstream_lost and not upstream_gain:
-      # Trivially no changes caused by the upstream.
-      #
+    if not upstream_gain and cnt_mine == len(local_changes):
       return
 
     if self.IsDirty(consider_untracked=False):
       syncbuf.fail(self, _DirtyError())
       return
 
-    if upstream_lost:
+    # If the upstream switched on us, warn the user.
+    #
+    if branch.merge != self.revisionExpr:
+      if branch.merge and self.revisionExpr:
+        syncbuf.info(self,
+                     'manifest switched %s...%s',
+                     branch.merge,
+                     self.revisionExpr)
+      elif branch.merge:
+        syncbuf.info(self,
+                     'manifest no longer tracks %s',
+                     branch.merge)
+
+    if cnt_mine < len(local_changes):
       # Upstream rebased.  Not everything in HEAD
-      # may have been caused by the user.
+      # was created by this user.
       #
       syncbuf.info(self,
                    "discarding %d commits removed from upstream",
-                   len(upstream_lost))
+                   len(local_changes) - cnt_mine)
 
-    branch.remote = rem
-    branch.merge = self.revision
+    branch.remote = self.GetRemote(self.remote.name)
+    branch.merge = self.revisionExpr
     branch.Save()
 
-    my_changes = self._revlist(not_rev(old_merge), HEAD)
-    if my_changes:
+    if cnt_mine > 0:
       def _dorebase():
-        self._Rebase(upstream = old_merge, onto = rev)
+        self._Rebase(upstream = '%s^1' % last_mine, onto = revid)
         self._CopyFiles()
       syncbuf.later2(self, _dorebase)
-    elif upstream_lost:
+    elif local_changes:
       try:
-        self._ResetHard(rev)
+        self._ResetHard(revid)
         self._CopyFiles()
       except GitError, e:
         syncbuf.fail(self, e)
         return
     else:
       def _doff():
-        self._FastForward(rev)
+        self._FastForward(revid)
         self._CopyFiles()
       syncbuf.later1(self, _doff)
 
@@ -771,7 +842,7 @@ class Project(object):
     if GitCommand(self, cmd, bare=True).Wait() != 0:
       return None
     return DownloadedChange(self,
-                            remote.ToLocal(self.revision),
+                            self.GetRevisionId(),
                             change_id,
                             patch_id,
                             self.bare_git.rev_parse('FETCH_HEAD'))
@@ -795,15 +866,8 @@ class Project(object):
 
     branch = self.GetBranch(name)
     branch.remote = self.GetRemote(self.remote.name)
-    branch.merge = self.revision
-
-    rev = branch.LocalMerge
-    if rev in all:
-      revid = all[rev]
-    elif IsId(rev):
-      revid = rev
-    else:
-      revid = None
+    branch.merge = self.revisionExpr
+    revid = self.GetRevisionId(all)
 
     if head.startswith(R_HEADS):
       try:
@@ -824,7 +888,7 @@ class Project(object):
       return True
 
     if GitCommand(self,
-                  ['checkout', '-b', branch.name, rev],
+                  ['checkout', '-b', branch.name, revid],
                   capture_stdout = True,
                   capture_stderr = True).Wait() == 0:
       branch.Save()
@@ -885,19 +949,12 @@ class Project(object):
       #
       head = all[head]
 
-      rev = self.GetRemote(self.remote.name).ToLocal(self.revision)
-      if rev in all:
-        revid = all[rev]
-      elif IsId(rev):
-        revid = rev
-      else:
-        revid = None
-
-      if revid and head == revid:
+      revid = self.GetRevisionId(all)
+      if head == revid:
         _lwrite(os.path.join(self.worktree, '.git', HEAD),
                 '%s\n' % revid)
       else:
-        self._Checkout(rev, quiet=True)
+        self._Checkout(revid, quiet=True)
 
     return GitCommand(self,
                       ['branch', '-D', name],
@@ -916,7 +973,7 @@ class Project(object):
         if cb is None or name != cb:
           kill.append(name)
 
-    rev = self.GetRemote(self.remote.name).ToLocal(self.revision)
+    rev = self.GetRevisionId(left)
     if cb is not None \
        and not self._revlist(HEAD + '...' + rev) \
        and not self.IsDirty(consider_untracked = False):
@@ -963,7 +1020,9 @@ class Project(object):
 
 ## Direct Git Commands ##
 
-  def _RemoteFetch(self, name=None):
+  def _RemoteFetch(self, name=None, tag=None,
+                   initial=False,
+                   quiet=False):
     if not name:
       name = self.remote.name
 
@@ -971,14 +1030,84 @@ class Project(object):
     if self.GetRemote(name).PreConnectFetch():
       ssh_proxy = True
 
+    if initial:
+      alt = os.path.join(self.gitdir, 'objects/info/alternates')
+      try:
+        fd = open(alt, 'rb')
+        try:
+          ref_dir = fd.readline()
+          if ref_dir and ref_dir.endswith('\n'):
+            ref_dir = ref_dir[:-1]
+        finally:
+          fd.close()
+      except IOError, e:
+        ref_dir = None
+
+      if ref_dir and 'objects' == os.path.basename(ref_dir):
+        ref_dir = os.path.dirname(ref_dir)
+        packed_refs = os.path.join(self.gitdir, 'packed-refs')
+        remote = self.GetRemote(name)
+
+        all = self.bare_ref.all
+        ids = set(all.values())
+        tmp = set()
+
+        for r, id in GitRefs(ref_dir).all.iteritems():
+          if r not in all:
+            if r.startswith(R_TAGS) or remote.WritesTo(r):
+              all[r] = id
+              ids.add(id)
+              continue
+
+          if id in ids:
+            continue
+
+          r = 'refs/_alt/%s' % id
+          all[r] = id
+          ids.add(id)
+          tmp.add(r)
+
+        ref_names = list(all.keys())
+        ref_names.sort()
+
+        tmp_packed = ''
+        old_packed = ''
+
+        for r in ref_names:
+          line = '%s %s\n' % (all[r], r)
+          tmp_packed += line
+          if r not in tmp:
+            old_packed += line
+
+        _lwrite(packed_refs, tmp_packed)
+
+      else:
+        ref_dir = None
+
     cmd = ['fetch']
+    if quiet:
+      cmd.append('--quiet')
     if not self.worktree:
       cmd.append('--update-head-ok')
     cmd.append(name)
-    return GitCommand(self,
-                      cmd,
-                      bare = True,
-                      ssh_proxy = ssh_proxy).Wait() == 0
+    if tag is not None:
+      cmd.append('tag')
+      cmd.append(tag)
+
+    ok = GitCommand(self,
+                    cmd,
+                    bare = True,
+                    ssh_proxy = ssh_proxy).Wait() == 0
+
+    if initial:
+      if ref_dir:
+        if old_packed != '':
+          _lwrite(packed_refs, old_packed)
+        else:
+          os.remove(packed_refs)
+      self.bare_git.pack_refs('--all', '--prune')
+
+    return ok
 
   def _Checkout(self, rev, quiet=False):
     cmd = ['checkout']
@@ -1016,6 +1145,27 @@ class Project(object):
       os.makedirs(self.gitdir)
       self.bare_git.init()
 
+      mp = self.manifest.manifestProject
+      ref_dir = mp.config.GetString('repo.reference')
+
+      if ref_dir:
+        mirror_git = os.path.join(ref_dir, self.name + '.git')
+        repo_git = os.path.join(ref_dir, '.repo', 'projects',
+                                self.relpath + '.git')
+
+        if os.path.exists(mirror_git):
+          ref_dir = mirror_git
+
+        elif os.path.exists(repo_git):
+          ref_dir = repo_git
+
+        else:
+          ref_dir = None
+
+        if ref_dir:
+          _lwrite(os.path.join(self.gitdir, 'objects/info/alternates'),
+                  os.path.join(ref_dir, 'objects') + '\n')
+
       if self.manifest.IsMirror:
         self.config.SetString('core.bare', 'true')
       else:
@@ -1040,29 +1190,37 @@ class Project(object):
     if not os.path.exists(hooks):
       os.makedirs(hooks)
     for stock_hook in repo_hooks():
-      dst = os.path.join(hooks, os.path.basename(stock_hook))
+      name = os.path.basename(stock_hook)
+
+      if name in ('commit-msg') and not self.remote.review:
+        # Don't install a Gerrit Code Review hook if this
+        # project does not appear to use it for reviews.
+        #
+        continue
+
+      dst = os.path.join(hooks, name)
+      if os.path.islink(dst):
+        continue
+      if os.path.exists(dst):
+        if filecmp.cmp(stock_hook, dst, shallow=False):
+          os.remove(dst)
+        else:
+          _error("%s: Not replacing %s hook", self.relpath, name)
+          continue
       try:
         os.symlink(relpath(stock_hook, dst), dst)
       except OSError, e:
-        if e.errno == errno.EEXIST:
-          pass
-        elif e.errno == errno.EPERM:
+        if e.errno == errno.EPERM:
           raise GitError('filesystem must support symlinks')
         else:
           raise
 
   def _InitRemote(self):
-    if self.remote.fetchUrl:
+    if self.remote.url:
       remote = self.GetRemote(self.remote.name)
-
-      url = self.remote.fetchUrl
-      while url.endswith('/'):
-        url = url[:-1]
-      url += '/%s.git' % self.name
-      remote.url = url
-      remote.review = self.remote.reviewUrl
-      if remote.projectname is None:
-        remote.projectname = self.name
+      remote.url = self.remote.url
+      remote.review = self.remote.review
+      remote.projectname = self.name
 
       if self.worktree:
         remote.ResetFetch(mirror=False)
@@ -1070,37 +1228,27 @@ class Project(object):
         remote.ResetFetch(mirror=True)
       remote.Save()
 
-    for r in self.extraRemotes.values():
-      remote = self.GetRemote(r.name)
-      remote.url = r.fetchUrl
-      remote.review = r.reviewUrl
-      if r.projectName:
-        remote.projectname = r.projectName
-      elif remote.projectname is None:
-        remote.projectname = self.name
-      remote.ResetFetch()
-      remote.Save()
-
   def _InitMRef(self):
     if self.manifest.branch:
-      msg = 'manifest set to %s' % self.revision
-      ref = R_M + self.manifest.branch
-      cur = self.bare_ref.symref(ref)
-
-      if IsId(self.revision):
-        if cur != '' or self.bare_ref.get(ref) != self.revision:
-          dst = self.revision + '^0'
-          self.bare_git.UpdateRef(ref, dst, message = msg, detach = True)
-      else:
-        remote = self.GetRemote(self.remote.name)
-        dst = remote.ToLocal(self.revision)
-        if cur != dst:
-          self.bare_git.symbolic_ref('-m', msg, ref, dst)
+      self._InitAnyMRef(R_M + self.manifest.branch)
 
   def _InitMirrorHead(self):
-    dst = self.GetRemote(self.remote.name).ToLocal(self.revision)
-    msg = 'manifest set to %s' % self.revision
-    self.bare_git.SetHead(dst, message=msg)
+    self._InitAnyMRef(HEAD)
+
+  def _InitAnyMRef(self, ref):
+    cur = self.bare_ref.symref(ref)
+
+    if self.revisionId:
+      if cur != '' or self.bare_ref.get(ref) != self.revisionId:
+        msg = 'manifest set to %s' % self.revisionId
+        dst = self.revisionId + '^0'
+        self.bare_git.UpdateRef(ref, dst, message = msg, detach = True)
+    else:
+      remote = self.GetRemote(self.remote.name)
+      dst = remote.ToLocal(self.revisionExpr)
+      if cur != dst:
+        msg = 'manifest set to %s' % self.revisionExpr
+        self.bare_git.symbolic_ref('-m', msg, ref, dst)
 
   def _InitWorkTree(self):
     dotgit = os.path.join(self.worktree, '.git')
@@ -1120,21 +1268,21 @@ class Project(object):
         try:
           src = os.path.join(self.gitdir, name)
           dst = os.path.join(dotgit, name)
-          os.symlink(relpath(src, dst), dst)
+          if os.path.islink(dst) or not os.path.exists(dst):
+            os.symlink(relpath(src, dst), dst)
+          else:
+            raise GitError('cannot overwrite a local work tree')
         except OSError, e:
           if e.errno == errno.EPERM:
             raise GitError('filesystem must support symlinks')
           else:
             raise
 
-      rev = self.GetRemote(self.remote.name).ToLocal(self.revision)
-      rev = self.bare_git.rev_parse('%s^0' % rev)
-
-      _lwrite(os.path.join(dotgit, HEAD), '%s\n' % rev)
+      _lwrite(os.path.join(dotgit, HEAD), '%s\n' % self.GetRevisionId())
 
       cmd = ['read-tree', '--reset', '-u']
       cmd.append('-v')
-      cmd.append('HEAD')
+      cmd.append(HEAD)
       if GitCommand(self, cmd).Wait() != 0:
         raise GitError("cannot initialize work tree")
       self._CopyFiles()
@@ -1142,11 +1290,11 @@ class Project(object):
   def _gitdir_path(self, path):
     return os.path.join(self.gitdir, path)
 
-  def _revlist(self, *args):
-    cmd = []
-    cmd.extend(args)
-    cmd.append('--')
-    return self.work_git.rev_list(*args)
+  def _revlist(self, *args, **kw):
+    a = []
+    a.extend(args)
+    a.append('--')
+    return self.work_git.rev_list(*a, **kw)
 
   @property
   def _allrefs(self):
@@ -1271,8 +1419,11 @@ class Project(object):
       self.update_ref('-d', name, old)
       self._project.bare_ref.deleted(name)
 
-    def rev_list(self, *args):
-      cmdv = ['rev-list']
+    def rev_list(self, *args, **kw):
+      if 'format' in kw:
+        cmdv = ['log', '--pretty=format:%s' % kw['format']]
+      else:
+        cmdv = ['rev-list']
       cmdv.extend(args)
       p = GitCommand(self._project,
                      cmdv,
@@ -1281,7 +1432,9 @@ class Project(object):
                      capture_stderr = True)
       r = []
       for line in p.process.stdout:
-        r.append(line[:-1])
+        if line[-1] == '\n':
+          line = line[:-1]
+        r.append(line)
       if p.Wait() != 0:
         raise GitError('%s rev-list %s: %s' % (
                        self._project.name,
@@ -1428,9 +1581,10 @@ class MetaProject(Project):
                      name = name,
                      gitdir = gitdir,
                      worktree = worktree,
-                     remote = Remote('origin'),
+                     remote = RemoteSpec('origin'),
                      relpath = '.repo/%s' % name,
-                     revision = 'refs/heads/master')
+                     revisionExpr = 'refs/heads/master',
+                     revisionId = None)
 
   def PreSync(self):
     if self.Exists:
@@ -1438,7 +1592,8 @@ class MetaProject(Project):
       if cb:
         base = self.GetBranch(cb).merge
         if base:
-          self.revision = base
+          self.revisionExpr = base
+          self.revisionId = None
 
   @property
   def LastFetch(self):
@@ -1452,16 +1607,11 @@ class MetaProject(Project):
   def HasChanges(self):
     """Has the remote received new commits not yet checked out?
     """
-    if not self.remote or not self.revision:
+    if not self.remote or not self.revisionExpr:
       return False
 
     all = self.bare_ref.all
-    rev = self.GetRemote(self.remote.name).ToLocal(self.revision)
-    if rev in all:
-      revid = all[rev]
-    else:
-      revid = rev
-
+    revid = self.GetRevisionId(all)
     head = self.work_git.GetHead()
     if head.startswith(R_HEADS):
       try:
@@ -1471,6 +1621,6 @@ class MetaProject(Project):
 
     if revid == head:
       return False
-    elif self._revlist(not_rev(HEAD), rev):
+    elif self._revlist(not_rev(HEAD), revid):
       return True
     return False

repo

@@ -28,7 +28,7 @@ if __name__ == '__main__':
 del magic
 
 # increment this whenever we make important changes to this script
-VERSION = (1, 8)
+VERSION = (1, 9)
 
 # increment this if the MAINTAINER_KEYS block is modified
 KEYRING_VERSION = (1,0)
@@ -118,6 +118,9 @@ group.add_option('-m', '--manifest-name',
 group.add_option('--mirror',
                  dest='mirror', action='store_true',
                  help='mirror the forrest')
+group.add_option('--reference',
+                 dest='reference',
+                 help='location of mirror directory', metavar='DIR')
 
 # Tool
 group = init_optparse.add_option_group('repo Version options')
@@ -505,7 +508,7 @@ def _RunSelf(wrapper_path):
   my_git = os.path.join(my_dir, '.git')
 
   if os.path.isfile(my_main) and os.path.isdir(my_git):
-    for name in ['manifest.py',
+    for name in ['git_config.py',
                  'project.py',
                  'subcmds']:
       if not os.path.exists(os.path.join(my_dir, name)):

subcmds/branches.py

@@ -61,12 +61,34 @@ class Branches(Command):
 %prog [<project>...]
 
 Summarizes the currently available topic branches.
-"""
 
-  def _Options(self, p):
-    p.add_option('-a', '--all',
-                 dest='all', action='store_true',
-                 help='show all branches, not just the majority')
+Branch Display
+--------------
+
+The branch display output by this command is organized into four
+columns of information; for example:
+
+ *P nocolor                   | in repo
+    repo2                     |
+
+The first column contains a * if the branch is the currently
+checked out branch in any of the specified projects, or a blank
+if no project has the branch checked out.
+
+The second column contains either blank, p or P, depending upon
+the upload status of the branch.
+
+ (blank): branch not yet published by repo upload
+       P: all commits were published by repo upload
+       p: only some commits were published by repo upload
+
+The third column contains the branch name.
+
+The fourth column (after the | separator) lists the projects that
+the branch appears in, or does not appear in.  If no project list
+is shown, then the branch appears in all projects.
+
+"""
 
   def Execute(self, opt, args):
     projects = self.GetProjects(args)
@@ -84,18 +106,6 @@ Summarizes the currently available topic branches.
     names = all.keys()
     names.sort()
 
-    if not opt.all and not args:
-      # No -a and no specific projects listed; try to filter the
-      # results down to only the majority of projects.
-      #
-      n = []
-      for name in names:
-        i = all[name]
-        if i.IsCurrent \
-        or 80 <= (100 * len(i.projects)) / project_cnt:
-          n.append(name)
-      names = n
-
     if not names:
       print >>sys.stderr, '   (no branches)'
       return
@@ -126,7 +136,7 @@ Summarizes the currently available topic branches.
       hdr('%c%c %-*s' % (current, published, width, name))
       out.write(' |')
 
-      if in_cnt < project_cnt and (in_cnt == 1 or opt.all):
+      if in_cnt < project_cnt:
         fmt = out.write
         paths = []
         if in_cnt < project_cnt - in_cnt: 
@@ -140,15 +150,17 @@ Summarizes the currently available topic branches.
           for b in i.projects:
             have.add(b.project)
           for p in projects:
-            paths.append(p.relpath)
+            if not p in have:
+              paths.append(p.relpath)
 
         s = ' %s %s' % (type, ', '.join(paths))
         if width + 7 + len(s) < 80:
           fmt(s)
         else:
-          out.nl()
-          fmt('    %s:' % type)
+          fmt(' %s:' % type)
           for p in paths:
             out.nl()
-            fmt('      %s' % p)
+            fmt(width*' ' + '          %s' % p)
+      else:
+        out.write(' in all projects')
       out.nl()

subcmds/forall.py

@@ -160,10 +160,8 @@ terminal and are not redirected.
       setenv('REPO_PROJECT', project.name)
       setenv('REPO_PATH', project.relpath)
       setenv('REPO_REMOTE', project.remote.name)
-      setenv('REPO_LREV', project\
-        .GetRemote(project.remote.name)\
-        .ToLocal(project.revision))
-      setenv('REPO_RREV', project.revision)
+      setenv('REPO_LREV', project.GetRevisionId())
+      setenv('REPO_RREV', project.revisionExpr)
 
       if mirror:
         setenv('GIT_DIR', project.gitdir)

subcmds/grep.py

@@ -17,7 +17,7 @@ import sys
 from optparse import SUPPRESS_HELP
 from color import Coloring
 from command import PagedCommand
-from git_command import GitCommand
+from git_command import git_require, GitCommand
 
 class GrepColoring(Coloring):
   def __init__(self, config):
@@ -158,7 +158,7 @@ contain a line that matches both expressions:
     out = GrepColoring(self.manifest.manifestProject.config)
 
     cmd_argv = ['grep']
-    if out.is_on:
+    if out.is_on and git_require((1,6,3)):
       cmd_argv.append('--color')
     cmd_argv.extend(getattr(opt,'cmd_argv',[]))
 
@@ -204,7 +204,7 @@ contain a line that matches both expressions:
           else:
             out.project('--- project %s ---' % project.relpath)
             out.nl()
-            out.write(p.stderr)
+            out.write("%s", p.stderr)
             out.nl()
         continue
       have_match = True
@@ -217,17 +217,17 @@ contain a line that matches both expressions:
       if have_rev and full_name:
         for line in r:
           rev, line = line.split(':', 1)
-          out.write(rev)
+          out.write("%s", rev)
           out.write(':')
           out.project(project.relpath)
           out.write('/')
-          out.write(line)
+          out.write("%s", line)
           out.nl()
       elif full_name:
         for line in r:
           out.project(project.relpath)
           out.write('/')
-          out.write(line)
+          out.write("%s", line)
           out.nl()
       else:
         for line in r:

subcmds/init.py

@@ -19,9 +19,8 @@ import sys
 from color import Coloring
 from command import InteractiveCommand, MirrorSafeCommand
 from error import ManifestParseError
-from remote import Remote
 from project import SyncBuffer
-from git_command import git, MIN_GIT_VERSION
+from git_command import git_require, MIN_GIT_VERSION
 
 class Init(InteractiveCommand, MirrorSafeCommand):
   common = True
@@ -42,6 +41,13 @@ The optional -m argument can be used to specify an alternate manifest
 to be used. If no manifest is specified, the manifest default.xml
 will be used.
 
+The --reference option can be used to point to a directory that
+has the content of a --mirror sync. This will make the working
+directory use as much data as possible from the local reference
+directory when fetching from the server. This will make the sync
+go a lot faster by reducing data traffic on the network.
+
+
 Switching Manifest Branches
 ---------------------------
 
@@ -72,7 +78,9 @@ to update the working directory files.
     g.add_option('--mirror',
                  dest='mirror', action='store_true',
                  help='mirror the forrest')
-
+    g.add_option('--reference',
+                 dest='reference',
+                 help='location of mirror directory', metavar='DIR')
 
     # Tool
     g = p.add_option_group('repo Version options')
@@ -86,19 +94,6 @@ to update the working directory files.
                  dest='no_repo_verify', action='store_true',
                  help='do not verify repo source code')
 
-  def _CheckGitVersion(self):
-    ver_str = git.version()
-    if not ver_str.startswith('git version '):
-      print >>sys.stderr, 'error: "%s" unsupported' % ver_str
-      sys.exit(1)
-
-    ver_str = ver_str[len('git version '):].strip()
-    ver_act = tuple(map(lambda x: int(x), ver_str.split('.')[0:3]))
-    if ver_act < MIN_GIT_VERSION:
-      need = '.'.join(map(lambda x: str(x), MIN_GIT_VERSION))
-      print >>sys.stderr, 'fatal: git %s or later required' % need
-      sys.exit(1)
-
   def _SyncManifest(self, opt):
     m = self.manifest.manifestProject
     is_new = not m.Exists
@@ -114,12 +109,12 @@ to update the working directory files.
       m._InitGitDir()
 
       if opt.manifest_branch:
-        m.revision = opt.manifest_branch
+        m.revisionExpr = opt.manifest_branch
       else:
-        m.revision = 'refs/heads/master'
+        m.revisionExpr = 'refs/heads/master'
     else:
       if opt.manifest_branch:
-        m.revision = opt.manifest_branch
+        m.revisionExpr = opt.manifest_branch
       else:
         m.PreSync()
 
@@ -129,6 +124,9 @@ to update the working directory files.
       r.ResetFetch()
       r.Save()
 
+    if opt.reference:
+      m.config.SetString('repo.reference', opt.reference)
+
     if opt.mirror:
       if is_new:
         m.config.SetString('repo.mirror', 'true')
@@ -162,20 +160,34 @@ to update the working directory files.
       print >>sys.stderr, 'fatal: %s' % str(e)
       sys.exit(1)
 
-  def _PromptKey(self, prompt, key, value):
+  def _Prompt(self, prompt, value):
     mp = self.manifest.manifestProject
 
     sys.stdout.write('%-10s [%s]: ' % (prompt, value))
     a = sys.stdin.readline().strip()
-    if a != '' and a != value:
-      mp.config.SetString(key, a)
+    if a == '':
+      return value
+    return a
 
   def _ConfigureUser(self):
     mp = self.manifest.manifestProject
 
-    print ''
-    self._PromptKey('Your Name', 'user.name', mp.UserName)
-    self._PromptKey('Your Email', 'user.email', mp.UserEmail)
+    while True:
+      print ''
+      name  = self._Prompt('Your Name', mp.UserName)
+      email = self._Prompt('Your Email', mp.UserEmail)
+
+      print ''
+      print 'Your identity is: %s <%s>' % (name, email)
+      sys.stdout.write('is this correct [y/n]? ')
+      a = sys.stdin.readline().strip()
+      if a in ('yes', 'y', 't', 'true'):
+        break
+
+    if name != mp.UserName:
+      mp.config.SetString('user.name', name)
+    if email != mp.UserEmail:
+      mp.config.SetString('user.email', email)
 
   def _HasColorSet(self, gc):
     for n in ['ui', 'diff', 'status']:
@@ -215,7 +227,7 @@ to update the working directory files.
       gc.SetString('color.ui', 'auto')
 
   def Execute(self, opt, args):
-    self._CheckGitVersion()
+    git_require(MIN_GIT_VERSION, fail=True)
     self._SyncManifest(opt)
     self._LinkManifest(opt.manifest_name)
 

subcmds/rebase.py

@@ -0,0 +1,107 @@
+#
+# Copyright (C) 2010 The Android Open Source Project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import sys
+
+from command import Command
+from git_command import GitCommand
+from git_refs import GitRefs, HEAD, R_HEADS, R_TAGS, R_PUB, R_M
+from error import GitError
+
+class Rebase(Command):
+  common = True
+  helpSummary = "Rebase local branches on upstream branch"
+  helpUsage = """
+%prog {[<project>...] | -i <project>...}
+"""
+  helpDescription = """
+'%prog' uses git rebase to move local changes in the current topic branch to
+the HEAD of the upstream history, useful when you have made commits in a topic
+branch but need to incorporate new upstream changes "underneath" them.
+"""
+
+  def _Options(self, p):
+    p.add_option('-i', '--interactive',
+                dest="interactive", action="store_true",
+                help="interactive rebase (single project only)")
+
+    p.add_option('-f', '--force-rebase',
+                 dest='force_rebase', action='store_true',
+                 help='Pass --force-rebase to git rebase')
+    p.add_option('--no-ff',
+                 dest='no_ff', action='store_true',
+                 help='Pass --no-ff to git rebase')
+    p.add_option('-q', '--quiet',
+                 dest='quiet', action='store_true',
+                 help='Pass --quiet to git rebase')
+    p.add_option('--autosquash',
+                 dest='autosquash', action='store_true',
+                 help='Pass --autosquash to git rebase')
+    p.add_option('--whitespace',
+                 dest='whitespace', action='store', metavar='WS',
+                 help='Pass --whitespace to git rebase')
+
+  def Execute(self, opt, args):
+    all = self.GetProjects(args)
+    one_project = len(all) == 1
+
+    if opt.interactive and not one_project:
+      print >>sys.stderr, 'error: interactive rebase not supported with multiple projects'
+      return -1
+
+    for project in all:
+      cb = project.CurrentBranch
+      if not cb:
+        if one_project:
+          print >>sys.stderr, "error: project %s has a detatched HEAD" % project.relpath
+          return -1
+        # ignore branches with detatched HEADs
+        continue
+
+      upbranch = project.GetBranch(cb)
+      if not upbranch.LocalMerge:
+        if one_project:
+          print >>sys.stderr, "error: project %s does not track any remote branches" % project.relpath
+          return -1
+        # ignore branches without remotes
+        continue
+
+      args = ["rebase"]
+
+      if opt.whitespace:
+        args.append('--whitespace=%s' % opt.whitespace)
+
+      if opt.quiet:
+        args.append('--quiet')
+
+      if opt.force_rebase:
+        args.append('--force-rebase')
+
+      if opt.no_ff:
+        args.append('--no-ff')
+
+      if opt.autosquash:
+        args.append('--autosquash')
+
+      if opt.interactive:
+        args.append("-i")
+
+      args.append(upbranch.LocalMerge)
+
+      print >>sys.stderr, '# %s: rebasing %s -> %s' % \
+        (project.relpath, cb, upbranch.LocalMerge)
+
+      if GitCommand(project, args).Wait() != 0:
+        return -1

subcmds/smartsync.py

@@ -1,5 +1,5 @@
 #
-# Copyright (C) 2008 The Android Open Source Project
+# Copyright (C) 2010 The Android Open Source Project
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -13,13 +13,21 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-class Remote(object):
-  def __init__(self, name,
-               fetch=None,
-               review=None,
-               projectName=None):
-    self.name = name
-    self.fetchUrl = fetch
-    self.reviewUrl = review
-    self.projectName = projectName
-    self.requiredCommits = []
+from sync import Sync
+
+class Smartsync(Sync):
+  common = True
+  helpSummary = "Update working tree to the latest known good revision"
+  helpUsage = """
+%prog [<project>...]
+"""
+  helpDescription = """
+The '%prog' command is a shortcut for sync -s.
+"""
+
+  def _Options(self, p):
+    Sync._Options(self, p, show_smart=False)
+
+  def Execute(self, opt, args):
+    opt.smart_sync = True
+    Sync.Execute(self, opt, args)

subcmds/start.py

@@ -22,13 +22,18 @@ class Start(Command):
   common = True
   helpSummary = "Start a new branch for development"
   helpUsage = """
-%prog <newbranchname> [<project>...]
+%prog <newbranchname> [--all | <project>...]
 """
   helpDescription = """
 '%prog' begins a new branch of development, starting from the
 revision specified in the manifest.
 """
 
+  def _Options(self, p):
+    p.add_option('--all',
+                 dest='all', action='store_true',
+                 help='begin branch in all projects')
+
   def Execute(self, opt, args):
     if not args:
       self.Usage()
@@ -39,7 +44,14 @@ revision specified in the manifest.
       sys.exit(1)
 
     err = []
-    all = self.GetProjects(args[1:])
+    projects = []
+    if not opt.all:
+      projects = args[1:]
+      if len(projects) < 1:
+        print >>sys.stderr, "error: at least one project must be specified"
+        sys.exit(1)
+
+    all = self.GetProjects(projects)
 
     pm = Progress('Starting %s' % nb, len(all))
     for project in all:

subcmds/sync.py

@@ -16,12 +16,23 @@
 from optparse import SUPPRESS_HELP
 import os
 import re
+import shutil
+import socket
 import subprocess
 import sys
 import time
+import xmlrpclib
+
+try:
+  import threading as _threading
+except ImportError:
+  import dummy_threading as _threading
 
 from git_command import GIT
+from git_refs import R_HEADS
 from project import HEAD
+from project import Project
+from project import RemoteSpec
 from command import Command, MirrorSafeCommand
 from error import RepoChangedException, GitError
 from project import R_HEADS
@@ -29,6 +40,7 @@ from project import SyncBuffer
 from progress import Progress
 
 class Sync(Command, MirrorSafeCommand):
+  jobs = 1
   common = True
   helpSummary = "Update working tree to the latest revision"
   helpUsage = """
@@ -54,6 +66,13 @@ back to the manifest revision.  This option is especially helpful
 if the project is currently on a topic branch, but the manifest
 revision is temporarily needed.
 
+The -s/--smart-sync option can be used to sync to a known good
+build as specified by the manifest-server element in the current
+manifest.
+
+The -f/--force-broken option can be used to proceed with syncing
+other projects if a project sync fails.
+
 SSH Connections
 ---------------
 
@@ -84,7 +103,10 @@ later is required to fix a server side protocol bug.
 
 """
 
-  def _Options(self, p):
+  def _Options(self, p, show_smart=True):
+    p.add_option('-f', '--force-broken',
+                 dest='force_broken', action='store_true',
+                 help="continue sync even if a project fails to sync")
     p.add_option('-l','--local-only',
                  dest='local_only', action='store_true',
                  help="only update working tree, don't fetch")
@@ -94,6 +116,16 @@ later is required to fix a server side protocol bug.
     p.add_option('-d','--detach',
                  dest='detach_head', action='store_true',
                  help='detach projects back to manifest revision')
+    p.add_option('-q','--quiet',
+                 dest='quiet', action='store_true',
+                 help='be more quiet')
+    p.add_option('-j','--jobs',
+                 dest='jobs', action='store', type='int',
+                 help="number of projects to fetch simultaneously")
+    if show_smart:
+      p.add_option('-s', '--smart-sync',
+                   dest='smart_sync', action='store_true',
+                   help='smart sync using manifest from a known good build')
 
     g = p.add_option_group('repo Version options')
     g.add_option('--no-repo-verify',
@@ -103,21 +135,120 @@ later is required to fix a server side protocol bug.
                  dest='repo_upgraded', action='store_true',
                  help=SUPPRESS_HELP)
 
-  def _Fetch(self, projects):
+  def _FetchHelper(self, opt, project, lock, fetched, pm, sem):
+      if not project.Sync_NetworkHalf(quiet=opt.quiet):
+        print >>sys.stderr, 'error: Cannot fetch %s' % project.name
+        if opt.force_broken:
+          print >>sys.stderr, 'warn: --force-broken, continuing to sync'
+        else:
+          sem.release()
+          sys.exit(1)
+
+      lock.acquire()
+      fetched.add(project.gitdir)
+      pm.update()
+      lock.release()
+      sem.release()
+
+  def _Fetch(self, projects, opt):
     fetched = set()
     pm = Progress('Fetching projects', len(projects))
-    for project in projects:
-      pm.update()
 
-      if project.Sync_NetworkHalf():
-        fetched.add(project.gitdir)
-      else:
-        print >>sys.stderr, 'error: Cannot fetch %s' % project.name
-        sys.exit(1)
+    if self.jobs == 1:
+      for project in projects:
+        pm.update()
+        if project.Sync_NetworkHalf(quiet=opt.quiet):
+          fetched.add(project.gitdir)
+        else:
+          print >>sys.stderr, 'error: Cannot fetch %s' % project.name
+          if opt.force_broken:
+            print >>sys.stderr, 'warn: --force-broken, continuing to sync'
+          else:
+            sys.exit(1)
+    else:
+      threads = set()
+      lock = _threading.Lock()
+      sem = _threading.Semaphore(self.jobs)
+      for project in projects:
+        sem.acquire()
+        t = _threading.Thread(target = self._FetchHelper,
+                              args = (opt,
+                                      project,
+                                      lock,
+                                      fetched,
+                                      pm,
+                                      sem))
+        threads.add(t)
+        t.start()
+
+      for t in threads:
+        t.join()
+
     pm.end()
     return fetched
 
+  def UpdateProjectList(self):
+    new_project_paths = []
+    for project in self.manifest.projects.values():
+      if project.relpath:
+        new_project_paths.append(project.relpath)
+    file_name = 'project.list'
+    file_path = os.path.join(self.manifest.repodir, file_name)
+    old_project_paths = []
+
+    if os.path.exists(file_path):
+      fd = open(file_path, 'r')
+      try:
+        old_project_paths = fd.read().split('\n')
+      finally:
+        fd.close()
+      for path in old_project_paths:
+        if not path:
+          continue
+        if path not in new_project_paths:
+          """If the path has already been deleted, we don't need to do it
+          """
+          if os.path.exists(self.manifest.topdir + '/' + path):
+              project = Project(
+                             manifest = self.manifest,
+                             name = path,
+                             remote = RemoteSpec('origin'),
+                             gitdir = os.path.join(self.manifest.topdir,
+                                                   path, '.git'),
+                             worktree = os.path.join(self.manifest.topdir, path),
+                             relpath = path,
+                             revisionExpr = 'HEAD',
+                             revisionId = None)
+
+              if project.IsDirty():
+                print >>sys.stderr, 'error: Cannot remove project "%s": \
+uncommitted changes are present' % project.relpath
+                print >>sys.stderr, '       commit changes, then run sync again'
+                return -1
+              else:
+                print >>sys.stderr, 'Deleting obsolete path %s' % project.worktree
+                shutil.rmtree(project.worktree)
+                # Try deleting parent subdirs if they are empty
+                dir = os.path.dirname(project.worktree)
+                while dir != self.manifest.topdir:
+                  try:
+                    os.rmdir(dir)
+                  except OSError:
+                    break
+                  dir = os.path.dirname(dir)
+
+    new_project_paths.sort()
+    fd = open(file_path, 'w')
+    try:
+      fd.write('\n'.join(new_project_paths))
+      fd.write('\n')
+    finally:
+      fd.close()
+    return 0
+
   def Execute(self, opt, args):
+    if opt.jobs:
+      self.jobs = opt.jobs
     if opt.network_only and opt.detach_head:
       print >>sys.stderr, 'error: cannot combine -n and -d'
       sys.exit(1)
@@ -125,6 +256,51 @@ later is required to fix a server side protocol bug.
       print >>sys.stderr, 'error: cannot combine -n and -l'
       sys.exit(1)
 
+    if opt.smart_sync:
+      if not self.manifest.manifest_server:
+        print >>sys.stderr, \
+            'error: cannot smart sync: no manifest server defined in manifest'
+        sys.exit(1)
+      try:
+        server = xmlrpclib.Server(self.manifest.manifest_server)
+        p = self.manifest.manifestProject
+        b = p.GetBranch(p.CurrentBranch)
+        branch = b.merge
+        if branch.startswith(R_HEADS):
+          branch = branch[len(R_HEADS):]
+
+        env = dict(os.environ)
+        if (env.has_key('TARGET_PRODUCT') and
+            env.has_key('TARGET_BUILD_VARIANT')):
+          target = '%s-%s' % (env['TARGET_PRODUCT'],
+                              env['TARGET_BUILD_VARIANT'])
+          [success, manifest_str] = server.GetApprovedManifest(branch, target)
+        else:
+          [success, manifest_str] = server.GetApprovedManifest(branch)
+
+        if success:
+          manifest_name = "smart_sync_override.xml"
+          manifest_path = os.path.join(self.manifest.manifestProject.worktree,
+                                       manifest_name)
+          try:
+            f = open(manifest_path, 'w')
+            try:
+              f.write(manifest_str)
+            finally:
+              f.close()
+          except IOError:
+            print >>sys.stderr, 'error: cannot write manifest to %s' % \
+                manifest_path
+            sys.exit(1)
+          self.manifest.Override(manifest_name)
+        else:
+          print >>sys.stderr, 'error: %s' % manifest_str
+          sys.exit(1)
+      except socket.error:
+        print >>sys.stderr, 'error: cannot connect to manifest server %s' % (
+            self.manifest.manifest_server)
+        sys.exit(1)
+
     rp = self.manifest.repoProject
     rp.PreSync()
 
@@ -134,6 +310,15 @@ later is required to fix a server side protocol bug.
     if opt.repo_upgraded:
       _PostRepoUpgrade(self.manifest)
 
+    if not opt.local_only:
+      mp.Sync_NetworkHalf(quiet=opt.quiet)
+
+    if mp.HasChanges:
+      syncbuf = SyncBuffer(mp.config)
+      mp.Sync_LocalHalf(syncbuf)
+      if not syncbuf.Finish():
+        sys.exit(1)
+      self.manifest._Unload()
     all = self.GetProjects(args, missing_ok=True)
 
     if not opt.local_only:
@@ -141,28 +326,28 @@ later is required to fix a server side protocol bug.
       now = time.time()
       if (24 * 60 * 60) <= (now - rp.LastFetch):
         to_fetch.append(rp)
-      to_fetch.append(mp)
       to_fetch.extend(all)
 
-      fetched = self._Fetch(to_fetch)
+      fetched = self._Fetch(to_fetch, opt)
       _PostRepoFetch(rp, opt.no_repo_verify)
       if opt.network_only:
         # bail out now; the rest touches the working tree
         return
 
-      if mp.HasChanges:
-        syncbuf = SyncBuffer(mp.config)
-        mp.Sync_LocalHalf(syncbuf)
-        if not syncbuf.Finish():
-          sys.exit(1)
-
         self.manifest._Unload()
         all = self.GetProjects(args, missing_ok=True)
         missing = []
         for project in all:
           if project.gitdir not in fetched:
             missing.append(project)
-        self._Fetch(missing)
+        self._Fetch(missing, opt)
+
+    if self.manifest.IsMirror:
+      # bail out now, we have no working tree
+      return
+
+    if self.UpdateProjectList():
+      sys.exit(1)
 
     syncbuf = SyncBuffer(mp.config,
                          detach_head = opt.detach_head)
@@ -176,6 +361,10 @@ later is required to fix a server side protocol bug.
     if not syncbuf.Finish():
       sys.exit(1)
 
+    # If there's a notice that's supposed to print at the end of the sync, print
+    # it now...
+    if self.manifest.notice:
+      print self.manifest.notice
 
 def _PostRepoUpgrade(manifest):
   for project in manifest.projects.values():
@@ -207,17 +396,14 @@ def _VerifyTag(project):
 warning: Cannot automatically authenticate repo."""
     return True
 
-  remote = project.GetRemote(project.remote.name)
-  ref = remote.ToLocal(project.revision)
-
   try:
-    cur = project.bare_git.describe(ref)
+    cur = project.bare_git.describe(project.GetRevisionId())
   except GitError:
     cur = None
 
   if not cur \
      or re.compile(r'^.*-[0-9]{1,}-g[0-9a-f]{1,}$').match(cur):
-    rev = project.revision
+    rev = project.revisionExpr
     if rev.startswith(R_HEADS):
       rev = rev[len(R_HEADS):]
 

subcmds/upload.py

@@ -13,6 +13,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import copy
 import re
 import sys
 
@@ -20,6 +21,17 @@ from command import InteractiveCommand
 from editor import Editor
 from error import UploadError
 
+UNUSUAL_COMMIT_THRESHOLD = 5
+
+def _ConfirmManyUploads(multiple_branches=False):
+  if multiple_branches:
+    print "ATTENTION: One or more branches has an unusually high number of commits."
+  else:
+    print "ATTENTION: You are uploading an unusually high number of commits."
+  print "YOU PROBABLY DO NOT MEAN TO DO THIS. (Did you rebase across branches?)"
+  answer = raw_input("If you are sure you intend to do this, type 'yes': ").strip()
+  return answer == "yes"
+
 def _die(fmt, *args):
   msg = fmt % args
   print >>sys.stderr, 'error: %s' % msg
@@ -35,7 +47,7 @@ class Upload(InteractiveCommand):
   common = True
   helpSummary = "Upload changes for code review"
   helpUsage="""
-%prog [--re --cc] {[<project>]... | --replace <project>}
+%prog [--re --cc] [<project>]...
 """
   helpDescription = """
 The '%prog' command is used to send changes to the Gerrit Code
@@ -55,12 +67,6 @@ added to the respective list of users, and emails are sent to any
 new users.  Users passed as --reviewers must already be registered
 with the code review system, or the upload will fail.
 
-If the --replace option is passed the user can designate which
-existing change(s) in Gerrit match up to the commits in the branch
-being uploaded.  For each matched pair of change,commit the commit
-will be added as a new patch set, completely replacing the set of
-files and description associated with the change in Gerrit.
-
 Configuration
 -------------
 
@@ -72,6 +78,19 @@ to "true" then repo will assume you always answer "y" at the prompt,
 and will not prompt you further.  If it is set to "false" then repo
 will assume you always answer "n", and will abort.
 
+review.URL.autocopy:
+
+To automatically copy a user or mailing list to all uploaded reviews,
+you can set a per-project or global Git option to do so. Specifically,
+review.URL.autocopy can be set to a comma separated list of reviewers
+who you always want copied on all uploads with a non-empty --re
+argument.
+
+review.URL.username:
+
+Override the username used to connect to Gerrit Code Review.
+By default the local part of the email address is used.
+
 The URL must match the review URL listed in the manifest XML file,
 or in the .git/config within the project.  For example:
 
@@ -81,6 +100,7 @@ or in the .git/config within the project.  For example:
 
   [review "http://review.example.com/"]
     autoupload = true
+    autocopy = johndoe@company.com,my-team-alias@company.com
 
 References
 ----------
@@ -90,9 +110,9 @@ Gerrit Code Review:  http://code.google.com/p/gerrit/
 """
 
   def _Options(self, p):
-    p.add_option('--replace',
-                 dest='replace', action='store_true',
-                 help='Upload replacement patchesets from this branch')
+    p.add_option('-t',
+                 dest='auto_topic', action='store_true',
+                 help='Send local branch name to Gerrit Code Review')
     p.add_option('--re', '--reviewers',
                  type='string',  action='append', dest='reviewers',
                  help='Request reviews from these people.')
@@ -100,7 +120,7 @@ Gerrit Code Review:  http://code.google.com/p/gerrit/
                  type='string',  action='append', dest='cc',
                  help='Also send email to these email addresses.')
 
-  def _SingleBranch(self, branch, people):
+  def _SingleBranch(self, opt, branch, people):
     project = branch.project
     name = branch.name
     remote = project.GetBranch(name).remote
@@ -129,11 +149,15 @@ Gerrit Code Review:  http://code.google.com/p/gerrit/
       answer = answer in ('y', 'Y', 'yes', '1', 'true', 't')
 
     if answer:
-      self._UploadAndReport([branch], people)
+      if len(branch.commits) > UNUSUAL_COMMIT_THRESHOLD:
+        answer = _ConfirmManyUploads()
+
+    if answer:
+      self._UploadAndReport(opt, [branch], people)
     else:
       _die("upload aborted by user")
 
-  def _MultipleBranches(self, pending, people):
+  def _MultipleBranches(self, opt, pending, people):
     projects = {}
     branches = {}
 
@@ -192,62 +216,66 @@ Gerrit Code Review:  http://code.google.com/p/gerrit/
         todo.append(branch)
     if not todo:
       _die("nothing uncommented for upload")
-    self._UploadAndReport(todo, people)
 
-  def _ReplaceBranch(self, project, people):
-    branch = project.CurrentBranch
-    if not branch:
-      print >>sys.stdout, "no branches ready for upload"
-      return
-    branch = project.GetUploadableBranch(branch)
-    if not branch:
-      print >>sys.stdout, "no branches ready for upload"
-      return
+    many_commits = False
+    for branch in todo:
+      if len(branch.commits) > UNUSUAL_COMMIT_THRESHOLD:
+        many_commits = True
+        break
+    if many_commits:
+      if not _ConfirmManyUploads(multiple_branches=True):
+        _die("upload aborted by user")
 
-    script = []
-    script.append('# Replacing from branch %s' % branch.name)
-    for commit in branch.commits:
-      script.append('[      ] %s' % commit)
-    script.append('')
-    script.append('# Insert change numbers in the brackets to add a new patch set.')
-    script.append('# To create a new change record, leave the brackets empty.')
+    self._UploadAndReport(opt, todo, people)
 
-    script = Editor.EditString("\n".join(script)).split("\n")
+  def _AppendAutoCcList(self, branch, people):
+    """
+    Appends the list of users in the CC list in the git project's config if a
+    non-empty reviewer list was found.
+    """
 
-    change_re = re.compile(r'^\[\s*(\d{1,})\s*\]\s*([0-9a-f]{1,}) .*$')
-    to_replace = dict()
-    full_hashes = branch.unabbrev_commits
+    name = branch.name
+    project = branch.project
+    key = 'review.%s.autocopy' % project.GetBranch(name).remote.review
+    raw_list = project.config.GetString(key)
+    if not raw_list is None and len(people[0]) > 0:
+      people[1].extend([entry.strip() for entry in raw_list.split(',')])
 
-    for line in script:
-      m = change_re.match(line)
-      if m:
-        c = m.group(1)
-        f = m.group(2)
-        try:
-          f = full_hashes[f]
-        except KeyError:
-          print 'fh = %s' % full_hashes
-          print >>sys.stderr, "error: commit %s not found" % f
-          sys.exit(1)
-        if c in to_replace:
-          print >>sys.stderr,\
-            "error: change %s cannot accept multiple commits" % c
-          sys.exit(1)
-        to_replace[c] = f
+  def _FindGerritChange(self, branch):
+    last_pub = branch.project.WasPublished(branch.name)
+    if last_pub is None:
+      return ""
 
-    if not to_replace:
-      print >>sys.stderr, "error: no replacements specified"
-      print >>sys.stderr, "       use 'repo upload' without --replace"
-      sys.exit(1)
+    refs = branch.GetPublishedRefs()
+    try:
+      # refs/changes/XYZ/N --> XYZ
+      return refs.get(last_pub).split('/')[-2]
+    except:
+      return ""
 
-    branch.replace_changes = to_replace
-    self._UploadAndReport([branch], people)
-
-  def _UploadAndReport(self, todo, people):
+  def _UploadAndReport(self, opt, todo, original_people):
     have_errors = False
     for branch in todo:
       try:
-        branch.UploadForReview(people)
+        people = copy.deepcopy(original_people)
+        self._AppendAutoCcList(branch, people)
+
+        # Check if there are local changes that may have been forgotten
+        if branch.project.HasChanges():
+            key = 'review.%s.autoupload' % branch.project.remote.review
+            answer = branch.project.config.GetBoolean(key)
+
+            # if they want to auto upload, let's not ask because it could be automated
+            if answer is None:
+                sys.stdout.write('Uncommitted changes in ' + branch.project.name + ' (did you forget to amend?). Continue uploading? (y/n) ')
+                a = sys.stdin.readline().strip().lower()
+                if a not in ('y', 'yes', 't', 'true', 'on'):
+                    print >>sys.stderr, "skipping upload"
+                    branch.uploaded = False
+                    branch.error = 'User aborted'
+                    continue
+
+        branch.UploadForReview(people, auto_topic=opt.auto_topic)
         branch.uploaded = True
       except UploadError, e:
         branch.error = e
@@ -287,14 +315,6 @@ Gerrit Code Review:  http://code.google.com/p/gerrit/
       cc = _SplitEmails(opt.cc)
     people = (reviewers,cc)
 
-    if opt.replace:
-      if len(project_list) != 1:
-        print >>sys.stderr, \
-              'error: --replace requires exactly one project'
-        sys.exit(1)
-      self._ReplaceBranch(project_list[0], people)
-      return
-
     for project in project_list:
       avail = project.GetUploadableBranches()
       if avail:
@@ -303,6 +323,6 @@ Gerrit Code Review:  http://code.google.com/p/gerrit/
     if not pending:
       print >>sys.stdout, "no branches ready for upload"
     elif len(pending) == 1 and len(pending[0][1]) == 1:
-      self._SingleBranch(pending[0][1][0], people)
+      self._SingleBranch(opt, pending[0][1][0], people)
     else:
-      self._MultipleBranches(pending, people)
+      self._MultipleBranches(opt, pending, people)

tests/fixtures/test.gitconfig

@@ -0,0 +1,3 @@
+[section]
+	empty
+	nonempty = true

tests/test_git_config.py

@@ -0,0 +1,52 @@
+import os
+import unittest
+
+import git_config
+
+def fixture(*paths):
+    """Return a path relative to test/fixtures.
+    """
+    return os.path.join(os.path.dirname(__file__), 'fixtures', *paths)
+
+class GitConfigUnitTest(unittest.TestCase):
+    """Tests the GitConfig class.
+    """
+    def setUp(self):
+        """Create a GitConfig object using the test.gitconfig fixture.
+        """
+        config_fixture = fixture('test.gitconfig')
+        self.config = git_config.GitConfig(config_fixture)
+
+    def test_GetString_with_empty_config_values(self):
+        """
+        Test config entries with no value.
+
+        [section]
+            empty
+
+        """
+        val = self.config.GetString('section.empty')
+        self.assertEqual(val, None)
+
+    def test_GetString_with_true_value(self):
+        """
+        Test config entries with a string value.
+
+        [section]
+            nonempty = true
+
+        """
+        val = self.config.GetString('section.nonempty')
+        self.assertEqual(val, 'true')
+
+    def test_GetString_from_missing_file(self):
+        """
+        Test missing config file
+        """
+        config_fixture = fixture('not.present.gitconfig')
+        config = git_config.GitConfig(config_fixture)
+        val = config.GetString('empty')
+        self.assertEqual(val, None)
+
+if __name__ == '__main__':
+    unittest.main()