All repo-level hooks are expected to live in a single project at the
top level of that project. The name of the hooks project is provided
in the manifest.xml. The manifest also lists which hooks are enabled
to make it obvious if a file somehow failed to sync down (or got
deleted).
Before running any hook, we will prompt the user to make sure that it
is OK. A user can deny running the hook, allow once, or allow
"forever" (until hooks change). This tries to keep with the git
spirit of not automatically running anything on the user's computer
that got synced down. Note that individual repo commands can add
always options to avoid these prompts as they see fit (see below for
the 'upload' options).
When hooks are run, they are loaded into the current interpreter (the
one running repo) and their main() function is run. This mechanism is
used (instead of using subprocess) to make it easier to expand to a
richer hook interface in the future. During loading, the
interpreter's sys.path is updated to contain the directory containing
the hooks so that hooks can be split into multiple files.
The upload command has two options that control hook behavior:
- no-verify=False, verify=False (DEFAULT):
If stdout is a tty, can prompt about running upload hooks if needed.
If user denies running hooks, the upload is cancelled. If stdout is
not a tty and we would need to prompt about upload hooks, upload is
cancelled.
- no-verify=False, verify=True:
Always run upload hooks with no prompt.
- no-verify=True, verify=False:
Never run upload hooks, but upload anyway (AKA bypass hooks).
- no-verify=True, verify=True:
Invalid
Sample bit of manifest.xml code for enabling hooks (assumes you have a
project named 'hooks' where hooks are stored):
<repo-hooks in-project="hooks" enabled-list="pre-upload" />
Sample main() function in pre-upload.py in hooks directory:
def main(project_list, **kwargs):
print ('These projects will be uploaded: %s' %
', '.join(project_list))
print ('I am being a good boy and ignoring anything in kwargs\n'
'that I don\'t understand.')
print 'I fail 50% of the time. How flaky.'
if random.random() <= .5:
raise Exception('Pre-upload hook failed. Have a nice day.')
Change-Id: I5cefa2cd5865c72589263cf8e2f152a43c122f70
Users may wind up with a lot of loose object content in projects they
don't frequently make changes in, but that are modified by others.
Since we bypass many git code paths that would have otherwise called
out to `git gc --auto`, its possible for these projects to have
their loose object database grow out of control. To help prevent
that, we now invoke it ourselves during the network half of sync.
Signed-off-by: Shawn O. Pearce <sop@google.com>
(cherry picked from commit 1875ddd47c)
Instead of giving a Python backtrace when there is a connectivity
problem during repo upload, report that we cannot access the host,
and why, with a halfway decent error message.
Bug: REPO-45
Change-Id: I9a45b387e86e48073a2d99bd6d594c1a7d6d99d4
Signed-off-by: Shawn O. Pearce <sop@google.com>
(cherry picked from commit d2dfac81ad)
If a project is missing locally, it might be OK to skip over it
and continue running the same command in other projects.
Bug: REPO-43
Change-Id: I64f97eb315f379ab2c51fc53d24ed340b3d09250
Signed-off-by: Shawn O. Pearce <sop@google.com>
(cherry picked from commit d4cd69bdef)
Windows allows the environment to have unicode values.
This will cause Python to fail to execute the command.
Change-Id: I37d922c3d7ced0d5b4883f0220346ac42defc5e9
Signed-off-by: Shawn O. Pearce <sop@google.com>
This feature is used to convey information on a when a branch has
ceased development or if it is an experimental branch with a few
gotchas, etc.
You add it to your manifest XML by doing something like this:
<manifest>
<notice>
NOTE TO DEVELOPERS:
If you checkin code, you have to pinky-swear that it contains no bugs.
Anyone who breaks their promise will have tomatoes thrown at them in the
team meeting. Be sure to bring an extra set of clothes.
</notice>
<remote ... />
...
</manifest>
Carriage returns and indentation are relevant for the text in this tag.
This feature was requested by Anush Elangovan on the ChromiumOS team.
This adds a new flag -f/--force-broken that will allow the rest of
the sync process to continue instead of bailing when a particular
project fails to sync.
Change-Id: I23680f2ee7927410f7ed930b1d469424c9aa246e
Signed-off-by: Andrei Warkentin <andreiw@motorola.com>
Signed-off-by: Shawn O. Pearce <sop@google.com>
It hasn't been necessary for a long time, and its
functionality can be accomplished with 'git push'.
Change-Id: Ic00d3adbe4cee7be3955117489c69d6e90106559
Use git clone to initialize a new repository, and when possible
allow callers to use --reference to reuse an existing checkout as
the initial object storage area for the new checkout.
Change-Id: Ie27f760247f311ce484c6d3e85a90d94da2febfc
Signed-off-by: Shawn O. Pearce <sop@google.com>
--replace started to fail due to a Python error, I forgot to pass
through the opt structure to the replace function.
Change-Id: Ifcd7a0c715c3fd9070a4c58208612a626382de35
Signed-off-by: Shawn O. Pearce <sop@google.com>
Passing through --whitespace=fix to rebase can be useful
to clean up a branch prior to uploading it for review.
Change-Id: Id85f1912e5e11ff9602e3b342c2fd7441abe67d7
Signed-off-by: Shawn O. Pearce <sop@google.com>
Some users might need to use a different login name than the local
part of their email address for their Gerrit Code Review user
account. Allow it to be overridden with the review.HOST.username
configuration variable.
Change-Id: I714469142ac7feadf09fee9c26680c0e09076b75
Signed-off-by: Shawn O. Pearce <sop@google.com>
If the -t flag is given to upload, the local branch name is
automatically sent to Gerrit Code Review as the topic branch name
for the change(s). This requires the server to be Gerrit Code
Review v2.1.3-53-gd50c94e or later, which isn't widely deployed
right now, so the default is opt-out.
Change-Id: I034fcacb405b7cb909147152db427fe69dd7bcbf
Signed-off-by: Shawn O. Pearce <sop@google.com>
Usage: repo rebase [[-i] <project>...]
Rebases the current topic branch of the specified (or all)
projects against the appropriate upstream.
Note: Interactive rebase is currently only supported when
exactly one project is specified on the command line.
Change-Id: I7376e35f27a6585149def82938c1ca99f36db2c4
Signed-off-by: Shawn O. Pearce <sop@google.com>
The upload command will read review.URL.autocopy from the project's
configuration and append the list of e-mails specified to the
--cc argument of the upload command if a non-empty --re argument
was provided.
Change-Id: I2424517d17dd3444b20f0e6a003be6e70b8904f6
Signed-off-by: Shawn O. Pearce <sop@google.com>
This fixes some format string bugs in grep which cause repo to with
"TypeError: not enough arguments for format string" when grepping and
the output contains a valid Python format string.
Change-Id: Ice8968ea106148d409490e4f71a2833b0cc80816
This patch does two things for being compatibile with
those Python which are built without threading support:
1. As the Python document and Shawn suggested, import dummy_threading
when the threading is not available.
2. Reserve the single threaded code and make it default.
In cases the --jobs does not work properly with dummy_threading,
we still have a safe fallback.
Change-Id: I40909ef8e9b5c22f315c0a1da9be38eed8b0a2dc
Add a sentinel check to require a second explicit confirmation if the
user is attempting to upload (or upload --replace) an unusually large
number of commits. This may help the user to catch an accidentally
incorrect rebase they had done previously.
Change-Id: I12c4d102f90a631d6ad193486a70ffd520ef6ae0
The manifest server doesn't want to have refs/heads passed to it, so
we need to strip that when the branch contains it.
Change-Id: I044f8a9629220e886fd5e02e3c1ac4b4bb6020ba
Do not error if a project is missing on the filesystem, is deleted
from manifest.xml, but still exists in project.list.
Change-Id: I1d13e435473c83091e27e4df571504ef493282dd
This option allows the user to specify a manifest server to use when
syncing. This manifest server will provide a manifest pegging each
project to a known green build. This allows developers to work on a
known good tree that is known to build and pass tests, preventing
failed builds to hamper productivity.
The manifest used is not "sticky" so as to allow subsequent
'repo sync' calls to sync to the tip of the tree.
Change-Id: Id0a24ece20f5a88034ad364b416a1dd2e394226d
This bug happens when a project gets added to the manifest, and
then is renamed. Users who happened to have run "repo sync" after
the project was added but before the rename happened will try to
read the data from the old project, as the manifest was only updated
after all projects were updated successfully.
When someone copies and pastes a setup line from a web page,
they might actually copy 'repo sync' onto the clipboard and wind
up pasting it into the "Your Name" prompt. This means they will
initialize their client with the user name of "repo sync", creating
some rather funny looking commits later on. For example:
To setup your source tree:
mkdir ~/code
cd ~/code
repo init -u git://....
repo sync
If this entire block was just blindly copy and pasted into the
terminal, the shell won't read "repo sync" but "repo init" will.
By showing the user their full identity string, and asking them
to confirm it before we continue, we can give the hapless user a
chance to recover from this mistake, without unfairly harming those
who were actually named 'repo' by their parents.
Signed-off-by: Shawn O. Pearce <sop@google.com>
The --color flag wasn't introduced until git 1.6.3. Prior to that
version, `git grep --color` just produces a fatal error, as it is
an unsupported option. Since this is just pretty output and is not
critical to execution, we can simply omit the option if the version
of git we are running on doesn't support it.
Signed-off-by: Shawn O. Pearce <sop@google.com>
This way we can use it to detect feature support in the underlying
git, such as new options or commands that have been added in more
recent versions.
Signed-off-by: Shawn O. Pearce <sop@google.com>
If a line is blank in project.list, its not a relevant project path,
so skip over it. Existing project.list files may have blank lines if
sync was run with no projects at all, and the file was created empty.
Signed-off-by: Shawn O. Pearce <sop@google.com>
We have no working tree, so we cannot update the project.list
state file, nor should we try to delete a directory if a project is
removed from the manifest. Clients would still need the repository
for historical records.
Signed-off-by: Shawn O. Pearce <sop@google.com>
After a repo sync, some of the project paths might need
to be removed. This changes maintains a list of project
paths from the previous sync operation and compares.
The revisionExpr field now holds an expression from the manifest,
such as "refs/heads/master", while revisionId holds the current
commit-ish SHA-1 of the revisionExpr. Currently that is only
filled in if the manifest points directly to a SHA-1.
Signed-off-by: Shawn O. Pearce <sop@google.com>
We now feed Project a RemoteSpec, instead of the Remote directly
from the XmlManifest. This way the RemoteSpec already has the
full project URL, rather than just the base, permitting other
types of manifests to produce the URL in their own style.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Extensive discussion with users lead to the fact that needing to
supply -a to view what they really wanted to see was just wrong.
Signed-off-by: Shawn O. Pearce <sop@google.com>
The level 2 headings (denoted by ~) indent the heading two spaces,
but continue to use the bold formatter to offset them from the
other surrounding text.
Signed-off-by: Shawn O. Pearce <sop@google.com>
This way its clear the command did something, and reported
that it had nothing to show you, because you have no active
branches in this client.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Most projects will have their branch heads matching in all branches,
so switching between them should be just a matter of updating the
work tree's HEAD symref. This can be done in pure Python, saving
quite a bit of time over forking 'git checkout'.
Signed-off-by: Shawn O. Pearce <sop@google.com>
This is mostly useful if the number of projects to switch is many
(e.g. all of Android) and a large number of them are behind the
current manifest revision. We wind up needing to run git just to
make the working tree match, and that often makes the command take
a couple of seconds longer than we'd like.
Signed-off-by: Shawn O. Pearce <sop@google.com>
When trying to read log output from many projects at once it can
be difficult to make sense of which messages came from where.
For many professional developers it is common to want to view the
last week's worth of your work, so you can write a weekly summary
of your activity for your status report.
This is easier with the new -p option:
repo forall -pc git log --reverse --since=1.week.ago --author=sop
produces a report of all commits written by me in the last week,
formatted in a paged output display, with headers inserted in
front of each project's output.
Where this can be even more useful is with git log's pickaxe,
e.g. now we can use:
repo forall -pc git log -Sbar v1.0..v1.1
to locate all additions or removals of the symbol 'bar' since v1.0,
up to and including v1.1. Before displaying the matching commits in
a project, a project header is shown, giving the user some context
information for the matching results.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Generally we only show the project path, relative from the top of the
client. Showing the project name may be confusing for the end-user.
Signed-off-by: Shawn O. Pearce <sop@google.com>
This gives the user the last chance to confirm where the change is
going to be sent to. Knowing the review server URL will help the
user decide if continuing with the upload makes sense.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Its unlikely that a new version of repo will be delivered in any
given day, so we now check only once every 24 hours to see if repo
has been updated. This reduces the sync cost, as we no longer need
to contact the repo distribution servers every time we do a sync.
repo selfupdate can still be used to force a check.
Signed-off-by: Shawn O. Pearce <sop@google.com>
If the user has disabled a prompt, skip the two commands we use to
obtain the list of commits and the date of the branch. These will
never be displayed and just waste the end-user's time.
Signed-off-by: Shawn O. Pearce <sop@google.com>
If review.URL.autoupload is set to true in a project's .git/config
or in ~/.gitconfig then `repo upload` will automatically upload,
and skip prompting the end-user.
Conversely, if review.URL.autoupload is set to false, then repo
will refuse to upload to that project.
Bug: REPO-25
Signed-off-by: Shawn O. Pearce <sop@google.com>
We now try to sync all projects that can be done safely first, before
we start rebasing user commits over the upstream. This has the nice
effect of making the local tree as close to the upstream as possible
before the user has to start resolving merge conflicts, as that extra
information in other projects may aid in the conflict resolution.
Informational output is buffered and delayed until calculation for
all projects has been done, so that the user gets one concise list
of notice messages, rather than it interrupting the progress meter.
Fast-forward output is now prefixed with the project header, so the
user can see which project that update is taking place in, and make
some relation of the diffstat back to the project name.
Rebase output is now prefixed with the project header, so that if
the rebase fails, the user can see which project we were operating
on and can try to address the failure themselves.
Since rebase sits on a detached HEAD, we now look for an in-progress
rebase during sync, so we can alert the user that the given project
is in a state we cannot handle.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Users may want to upgrade only repo to the latest release, but
leave their working tree state alone and avoid 'repo sync'.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Users can now use 'repo grep' to search all projects, rather than
'repo forall -c git grep'. Its not only shorter to type, but it
also filters results better by highlighting which projects matched
in the client workspace.
Signed-off-by: Shawn O. Pearce <sop@google.com>
If there is nothing output at all, tell the user the working tree is
completely clean. It just gives them a bit more of a warm-fuzzy
feeling knowing repo and until the end. It also more closely
matches with the output of git status.
Signed-off-by: Shawn O. Pearce <sop@google.com>
This way users can see how much is left during fetch. Its
especially useful when most syncs are no-ops but there are
hundreds of repositories to poll.
Signed-off-by: Shawn O. Pearce <sop@google.com>
This permits usage of 'repo sync' while offline, as we bypass the
network based portions of the code and do only the local sync.
An example use case might be:
repo sync -n ; # while we have network
... some time later ...
repo sync -l ; # while without network, come up to date
Signed-off-by: Shawn O. Pearce <sop@google.com>
The -d flag moves the project back to a detached HEAD state,
matching what is listed in the manifest. This can be useful to
set a client to something stable (or at least well-known), such as
before a sequence of 'repo download' commands are used to get some
changes for testing.
Signed-off-by: Shawn O. Pearce <sop@google.com>
This makes it easier to update all repositories, without actually
impacting the working directory, or learning about how to use
`repo forall -c 'git fetch $REPO_REMOTE' `.
Signed-off-by: Shawn O. Pearce <sop@google.com>
This is only meant to be passed through while repo upgrades itself
during a sync. It should never be something a user invokes on
their own.
Signed-off-by: Shawn O. Pearce <sop@google.com>
If `repo start foo` fails due to uncommitted and unmergeable changes
in a single project, we have switched half of the projects over to
the new target branches, but didn't on the one that failed to move.
This change improves the situation by doing three things differently:
- We keep going when we encounter an error, so other projects
that can successfully switch still switch.
- We ignore projects whose current branch is already on the
requested name; they are logically already setup.
- We checkout the branch if it already exists, rather than
trying to recreate the branch.
Bug: REPO-22
Signed-off-by: Shawn O. Pearce <sop@google.com>
We now display a summary of the available topic branches in this
client, based upon a sorted union of all existing projects.
Bug: REPO-21
Signed-off-by: Shawn O. Pearce <sop@google.com>
Modern Gerrit2 automatically outputs the URL for each commit to
stderr as it creates the records. Dumping the URL ourselves is
unnecessary additional output, and worse is just an approximate
guess for the correct web URL. Gerrit might not live at the top
level directory for the server, or might even prefer a different
hostname for web connections than what is listed in the manifest.
Signed-off-by: Shawn O. Pearce <sop@google.com>
On a mirror client we don't prompt for user.name,user.email as the
data is only necessary if you will make new commits. On a re-init
we were testing the command line option, not the existing IsMirror
property from the manifest configuration file.
Signed-off-by: Shawn O. Pearce <sop@google.com>
This allows the user to run "repo init -u" again after an
initial attempt failed due to an invalid URL.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Instead of a stack trace ending in origin/master not existing we
now tell the user the manifest url is invalid if 'git fetch' has
failed out early.
Signed-off-by: Shawn O. Pearce <sop@google.com>
If the value obtained is None we now set the variable to
'' instead, in an attempt to make execve() happier about
our 3rd argument, the env dictionary.
Signed-off-by: Shawn O. Pearce <sop@google.com>
We now correctly support re-initializing an existing client to point
to a different branch of the same manifest repository, effectively
allowing the client to switch the baseline it is operating on.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Simply setting repo.mirror true doesn't make a client into a mirror.
The on-disk layout is completely wrong for a mirror repository,
and until we fix our layout for a non-mirror client to more closely
resemble the upstream we can't do anything to easily turn on or
turn off the mirror status flag.
Signed-off-by: Shawn O. Pearce <sop@google.com>
This can be useful to create a new manifest from an existing client,
especially if the client wants to use the "-r" option to set each
project's revision to the current commit SHA-1, making a sort of a
tag file that can be used to recreate this exact state elsewhere.
Signed-off-by: Shawn O. Pearce <sop@google.com>
If a client was created with "repo init --mirror" then there are
no working directories present, and no files checked out. Using
a command like "repo status" in this context makes no sense, and
actually throws back a Pytyon traceback at the console when the
underlying commands fail out.
We now tag commands with the MirrorSafeCommand type if they are
able to be executed within a mirror directory safely. Using a
command in a mirror which lacks this base class results in a
useful error letting you know the command isn't supported.
Bug: REPO-14
Signed-off-by: Shawn O. Pearce <sop@google.com>
REPO_PATH is the path relative the the root of the client.
REPO_REMOTE is the name of the remote system from the manifest.
REPO_LREV is the name of the revision from the manifest, but
translated to something the local repository knows.
REPO_RREV is the name of the revision from the manifest.
This allows us to do commands like:
repo forall -c 'echo "(cd $REPO_PATH && git checkout `git rev-parse HEAD`)"'
Gerrit won't permit more than one commit using the same change
number during a replacement request, so we should error out if
the user has asked for this in their upload edit script.
Signed-off-by: Shawn O. Pearce <sop@google.com>
Users are prompted with the list of known changes we are about
to upload, and they can fill out the current change numbers for
any changes which already exist in the data store. For each of
those changes the change number and commit id is sent as part of
the upload request, so Gerrit can insert the new commit as a new
patch set of the existing change, rather than make a new change.
This facility permits developers to replace a patch so they can
address comments made on a prior version of the same change.
Signed-off-by: Shawn O. Pearce <sop@google.com>
The mirror option downloads a complete forrest (as described by the
manifest) and creates a replica of the remote repositories rather
than a client working directory. This permits other clients to
sync off the mirror site.
A mirror can be positioned in a "DMZ", where the mirror executes
"repo sync" to obtain changes from the external upstream and
clients inside the protected zone operate off the mirror only,
and therefore do not require direct git:// access to the external
upstream repositories.
Signed-off-by: Shawn O. Pearce <sop@google.com>
