* stable:
Automatically install Gerrit Code Review's commit-msg hook
Fail sync when encountering "N commits behind."
Check that we are not overwriting a local repository when syncing.
Honor url.insteadOf when setting up SSH control master connection
sync: Fix split call on malformed email addresses
Fixing project renaming bug.
Conflicts:
hooks/commit-msg
project.py
subcmds/sync.py
Change-Id: I5eaf8fef8cbe4a95d124368112293a9ca64325bf
Most users of repo are also using Gerrit Code Review, and will want
the commit-msg hook to be automatically installed into their local
projects so that Change-Ids are assigned when commits are created,
not when they are first uploaded.
(cherry picked from commit a949fa5d20
but squashed with latest hook script from version 2.1.2)
Change-Id: Ie68b2d60ac85d8c2285d2e1e6a4536eb76695547
Signed-off-by: Shawn O. Pearce <sop@google.com>
This is almost always something the user needs to address
before continuing work, so promoting it to a failure (rather
than simply an informational message) seems the right way to
go. As a side-effect, repo will now exit with a non-zero
status code in this situation, so pipelines of the form
`repo sync && make` will fail if there are branches that
are stalled due to uploaded but unmerged patches.
If a local git repository exists within the same folder as a new project that
is added, when the user syncs the repo, the sync will overwrite the local
files under the project's .git repository with its own symlinks. Make sure
that we do not overwrite 'normal' files in repo and throw an error when
that happens.
If an email address in a commit object contains a space, like a few
malformed ones on the Linux kernel, we still want to split only on
the first space.
Unfortunately my brain was too damaged by Perl and originally wrote
the split asking for 2 results; in Python split's argument is how
many splits to perform. Here we want only 1 split, to break apart
the commit identity from the email address on the same line.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Most users of repo are also using Gerrit Code Review, and will want
the commit-msg hook to be automatically installed into their local
projects so that Change-Ids are assigned when commits are created,
not when they are first uploaded.
Change-Id: Ide42e93b068832f099d68a79c2863d22145d05ad
Signed-off-by: Shawn O. Pearce <sop@google.com>
Some types of manifests might prefer to put their meta project work
tree under topdir, rather than inside of the .repo/ directory. We
can support that by allowing relpath to be optionally passed in.
Signed-off-by: Shawn O. Pearce <sop@google.com>
The -o option permits the user to control the name of the manifest's
remote, which normally is hardcoded to be 'origin', but can differ
because we derive it at runtime from the configuration file.
Signed-off-by: Shawn O. Pearce <sop@google.com>
I plan to have the new submodule manifest format use a different
layout for the m refs than the XML manifest format has used in
the past. Thus we need to move the behavior management into the
manifest object, and out of the project, so we can change it.
Signed-off-by: Shawn O. Pearce <sop@google.com>
The _LinkWorkTree method can now be used to relink the work tree,
such as if the real repository was moved to a different location
on disk.
Signed-off-by: Shawn O. Pearce <sop@google.com>
We accidentally introduced this message during 1.6.8 by always
invoking `git rebase` when there were no new commits from the
upstream, but the user had local commits.
Signed-off-by: Shawn O. Pearce <sop@google.com>
The revisionExpr field now holds an expression from the manifest,
such as "refs/heads/master", while revisionId holds the current
commit-ish SHA-1 of the revisionExpr. Currently that is only
filled in if the manifest points directly to a SHA-1.
Signed-off-by: Shawn O. Pearce <sop@google.com>
The trick of looking at the reflog for the remote tracking branch
and only going back one commit works some of the time, but not all of
the time. Its sort of relying on the fact that the user didn't use
`repo sync -n` or `git fetch` to only update the tracking branches
and skip the working directory update.
Doing this right requires looking through the history of the SHA-1
source (what the upstream used to be) and finding a spot where the
DAG diveraged away suddenly, and consider that to be the rewind
point. That's really difficult to do, as we don't have a clear
picture of what that old point was.
A close approximation is to list all of the commits that are in
HEAD, but not the new upstream, and rebase all of those where the
committer email address is this user's email address. In most cases,
this will effectively rebase only the user's new original work.
If the user is the project maintainer and rewound the branch
themselves, and they don't want all of the commits they have created
to be rebased onto the new upstream, they should handle the rebase
on their own, after the sync is complete.
Signed-off-by: Shawn O. Pearce <sop@google.com>
We now feed Project a RemoteSpec, instead of the Remote directly
from the XmlManifest. This way the RemoteSpec already has the
full project URL, rather than just the base, permitting other
types of manifests to produce the URL in their own style.
Signed-off-by: Shawn O. Pearce <sop@google.com>
These aren't that widely used, and actually make it difficult for
users to fully mirror a forest of repositories, and then permit
someone else to clone off that forest, rather then the original
upstream servers.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Performance improvements in repo sync caused us to skip out of the
initial Sync_LocalHalf without ever running CopyFiles, so we didn't
create the top level Makefile in new clients whose manifest request
one with a <copyfile> element.
Now we run CopyFiles after the initial read-tree that populates
the project working directory.
Signed-off-by: Shawn O. Pearce <sop@google.com>
If the current branch is published, but all published commits are
merged into the manifest revision, but there is also at least one
unpublished commit on the current branch, we should rebase the
unpublished commit, rather than creating a merge commit.
Signed-off-by: Shawn O. Pearce <sop@google.com>
By creating a background ssh "control master" process which lives
for the duration of our sync cycle we can easily cut the time for
a no-op sync of 132 projects from 60s to 18s.
Bug: REPO-11
Signed-off-by: Shawn O. Pearce <sop@google.com>
Most projects will have their branch heads matching in all branches,
so switching between them should be just a matter of updating the
work tree's HEAD symref. This can be done in pure Python, saving
quite a bit of time over forking 'git checkout'.
Signed-off-by: Shawn O. Pearce <sop@google.com>
This is mostly useful if the number of projects to switch is many
(e.g. all of Android) and a large number of them are behind the
current manifest revision. We wind up needing to run git just to
make the working tree match, and that often makes the command take
a couple of seconds longer than we'd like.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Its quite common for most projects to be matching the current
manifest revision, as most developers only modify one or two projects
at any one time. We can speed up `repo start foo` (that impacts
the entire client) by performing most of the branch creation and
switch operations in pure Python, and thus avoid 4 forks per project.
Signed-off-by: Shawn O. Pearce <sop@google.com>
These used to be used back when we had Gerrit 1.x support and used
HTTP based uploads to transmit changes for review. Since we moved
entirely to Gerrit 2.x, these are no longer called.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Its unlikely that a new version of repo will be delivered in any
given day, so we now check only once every 24 hours to see if repo
has been updated. This reduces the sync cost, as we no longer need
to contact the repo distribution servers every time we do a sync.
repo selfupdate can still be used to force a check.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Usually repo is upgraded only once a week, if that often. Most of
the time we invoke HasChanges on the repo project (or even on the
manifest project) the current HEAD will resolve to the same SHA-1
as the remote tracking ref, and there are therefore no changes.
Signed-off-by: Shawn O. Pearce <sop@google.com>
If the m/BRANCH ref is already pointing at the value set in the
manifest there is no reason to set it again. Leave it alone,
thus saving a full fork+exec call.
Signed-off-by: Shawn O. Pearce <sop@google.com>
By resolving the current HEAD and the manifest revision using pure
Python, we can in the common case of "no changes" avoid a lot of
git operations and directly jump out of the local sync method.
This reduces the no-op `repo sync -l` time for Android's 114 projects
from more than 6s to under 0.8s.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Its much faster to read the refs from 114 projects when the reader
is pure Python and just doing file IO than forking 114 git commands
and parsing their output.
The reader caches refs based upon file mtimes. If any single ref
file has been modified since the last read, we re-read the entire
repository's ref namespace. This simplifies the code as we don't
need to worry about shooting down symbolic-refs, but it may cause
more IO than is necessary if only one ref gets updated.
This change drops `repo branches` in Android from 1.658s to 0.206s.
Likewise, `repo sync` improves dramatically as well.
Signed-off-by: Shawn O. Pearce <sop@google.com>
This is invoked once per project in `repo sync`. Taking it out
saves about 1/114 of a second, so on a large set of projects like
Android it can save up to a full second of sync time.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Someone pointed out this message isn't always the truth; so we
shouldn't print it. The code path is executed when there are
published commits, yet our output talks about unpublished ones.
Signed-off-by: Shawn O. Pearce <sop@google.com>
We now try to sync all projects that can be done safely first, before
we start rebasing user commits over the upstream. This has the nice
effect of making the local tree as close to the upstream as possible
before the user has to start resolving merge conflicts, as that extra
information in other projects may aid in the conflict resolution.
Informational output is buffered and delayed until calculation for
all projects has been done, so that the user gets one concise list
of notice messages, rather than it interrupting the progress meter.
Fast-forward output is now prefixed with the project header, so the
user can see which project that update is taking place in, and make
some relation of the diffstat back to the project name.
Rebase output is now prefixed with the project header, so that if
the rebase fails, the user can see which project we were operating
on and can try to address the failure themselves.
Since rebase sits on a detached HEAD, we now look for an in-progress
rebase during sync, so we can alert the user that the given project
is in a state we cannot handle.
Signed-off-by: Shawn O. Pearce <sop@google.com>
rebase interactive (aka rebase -i) has changed in newer versions
of git, and doesn't always generate the sequence of commits the
same way it used to. It also doesn't handle having a previously
applied commit try to be applied again.
The default rebase algorithm is better suited to our needs.
It uses --ignore-if-in-upstream when generating the patch series
for git-am, and git-am with its 3-way fallback is able to handle
a rename case just as well as the cherry-pick variant used by -m.
Its also a generally faster implementation.
Signed-off-by: Shawn O. Pearce <sop@google.com>
If there is nothing output at all, tell the user the working tree is
completely clean. It just gives them a bit more of a warm-fuzzy
feeling knowing repo and until the end. It also more closely
matches with the output of git status.
Signed-off-by: Shawn O. Pearce <sop@google.com>
The -d flag moves the project back to a detached HEAD state,
matching what is listed in the manifest. This can be useful to
set a client to something stable (or at least well-known), such as
before a sequence of 'repo download' commands are used to get some
changes for testing.
Signed-off-by: Shawn O. Pearce <sop@google.com>
If `repo start foo` fails due to uncommitted and unmergeable changes
in a single project, we have switched half of the projects over to
the new target branches, but didn't on the one that failed to move.
This change improves the situation by doing three things differently:
- We keep going when we encounter an error, so other projects
that can successfully switch still switch.
- We ignore projects whose current branch is already on the
requested name; they are logically already setup.
- We checkout the branch if it already exists, rather than
trying to recreate the branch.
Bug: REPO-22
Signed-off-by: Shawn O. Pearce <sop@google.com>
There isn't any great value in buffering stdout into memory
coming from git checkout. So don't bother doing it.
Signed-off-by: Shawn O. Pearce <sop@google.com>
We now display a summary of the available topic branches in this
client, based upon a sorted union of all existing projects.
Bug: REPO-21
Signed-off-by: Shawn O. Pearce <sop@google.com>
This can be useful to create a new manifest from an existing client,
especially if the client wants to use the "-r" option to set each
project's revision to the current commit SHA-1, making a sort of a
tag file that can be used to recreate this exact state elsewhere.
Signed-off-by: Shawn O. Pearce <sop@google.com>
When creating a mirror repository we will always be using a bare
repository. Setting $GIT_DIR/config to have core.bare = true is
reasonable and helps Git to recognize the environment it is in.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Months ago when the Android Open Source Project launched we had some
import errors that had to be fixed and worked over. These hacks
were here to help users update their clients to newer versions of
the imported code.
Its very likely all clients have either been deleted, or have been
updated and have the fixed imports. So we don't need this hack in
repo anymore.
If a very ancient client still existed, it would need to be created
from scratch anyway, due to the Android cupcake branch merging
into master and the manifest changes not being able to be handled
correctly by repo. A new client wouldn't have the incorrectly
imported code in it, and thus wouldn't need this hack.
Signed-off-by: Shawn O. Pearce <sop@google.com>
I missed a parameter in the format string, but still provided the
value in the parameter list, so the format failed to produce an
output message.
Bug: REPO-15
Signed-off-by: Shawn O. Pearce <sop@google.com>
If a manifest specifies an invalid revision property, give the
user a better error message detaling the problem, instead of an
ugly Python traceback with a strange Git error message.
Bug: REPO-2
Signed-off-by: Shawn O. Pearce <sop@google.com>
