When using the --smart-sync or --smart-tag option, and the specified
manifest server is hosted on a server that requires authentication,
repo sync fails with the error: HTTP 401 Unauthorized.
Add support for getting the credentials from the .netrc file.
If a .netrc file exists in the user's home directory, and it contains
credentials for the hostname of the manifest server specified in the
manifest, use the credentials to authenticate with the manifest server
using the URL syntax extension for Basic Authentication:
http://user:password@host:port/path
Credentials from the .netrc file are only used if the manifest server
URL specified in the manifest does not already include credentials.
Change-Id: I06e6586e8849d0cd12fa9746789e8d45d5b1f848
`R_HEADS` is imported twice, from both the git_refs and project
modules.
It is actually defined in git_refs, and in project it is imported
from there, so the import of `R_HEADS` from project in the sync
module is redundant. Remove it.
`HEAD` is imported from project, but like `R_HEADS` it is actually
defined in git_refs. Import it from git_refs instead.
Change-Id: I8e2b0217d0d9f9f4ee5ef5b8cd0b026174ac52f4
When connecting to the manifest server, exceptions can occur but
are not caught, resulting in the repo sync exiting with a python
traceback.
Add handling of the following exceptions:
- IOError, which can be raised for example if the manifest server
URL is malformed.
- xmlrpclib.ProtocolError, which can be raised if the connection
to the manifest server fails with HTTP error.
- xmlrpclib.Fault, which can be raised if the RPC call fails for
some other reason.
Change-Id: I3a4830aef0941debadd515aac776a3932e28a943
Instead of every group being in the group "default", every project
is now in the group "all". A group that should not be downloaded
by default may be added to the group "notdefault".
This allows all group names to be positive (instead of removing groups
directly in the manifest with -default) and offers a clear way of
selecting every project (--groups all).
Change-Id: I99cd70309adb1f8460db3bbc6eff46bdcd22256f
The threaded 'repo sync' implementation would very often freeze the
process when interrupted by the user with Ctrl-C. The only solution
being to kill -9 the process explicitly from another terminal.
The reason for this is best explained here:
http://snakesthatbite.blogspot.fr/2010/09/cpython-threading-interrupting.html
This patch makes all helper sync threads 'daemon', which allows the
process to terminate immediately on Ctrl-C.
Note that this will forcefully kill all threads in case of interruption; this
is generally a bad thing, but:
1/ This is equivalent to calling kill -9 in another terminal, which
is the _only_ thing that can currently stop the process.
2/ There doesn't seem to be a way to tell the worker threads to
gently stop when they are in a blocking operation anyway (even
in the non-threaded case).
+ Do the same for "repo status -j<count>".
Change-Id: Ieaf45b0eacee36f35427f8edafd87415c2aa7be4
The overview command shows an overview of each branch in all (or the
specified) projects. The overview lists any local commits that have
not yet been merged into the project.
The report output is inspired by the report displayed following a
"repo prune" event, with the addition of listing the one-line log
messages for each commit that is not yet merged.
The report can also be filtered to show only active branches; by
default all branches that have commits beyond the upstream HEAD will
be listed.
Change-Id: Ibe67793991ad1aa38de3bc9747de4ba64e5591aa
Currently repo-rebase requires that all modifications be committed
locally before it will allow the rebase. In high-velocity environments,
you may want to just pull in newer code without explicitly creating
local commits, which is typically achieved using git-stash.
If called with the --auto-stash command line argument, and it is
determined that the current index is dirty, the local modifications
are stashed, and the rebase continues. If a stash was performed, that
stash is popped once the rebase completes.
Note that there is still a possibility that the git-stash pop will
result in a merge conflict.
Change-Id: Ibe3da96f0b4486cb7ce8d040639187e26501f6af
See repo issue #46 :
https://code.google.com/p/git-repo/issues/detail?id=46
When using repo init -b on an already existing repository,
the next sync will try to rebase changes coming from the old manifest
branch onto the new, leading in the best case scenario to conflicts
and in the worst case scenario to an incorrect "mixed up" manifest.
This patch fixes this by deleting the "default" branch in the local
manifest repository when the -d init switch is used, thus forcing
repo to perform a fresh checkout of the new manifest branch
Change-Id: I379e4875ec5357d8614d1197b6afbe58f9606751
This patch adds the option to include topic branches by adding the
following to a .gitconfig file:
uploadtopic = true
This option is only read in when the -t option is not already
specified at the command line.
Change-Id: I0e0eea49438bb4e4a21c2ac5bd498b68b5a9a845
Allows to ff-only a gerrit patch
This patch is necessary to automatically ensure that the patch will
be correctly submitted on ff-only gerrit projects
You can now use:
repo download (--ff-only|-f) project changeid/patchnumber
This is useful to automate verification of fast forward status of a patch
in the context of build automation, and commit gating (e.g. buildbot)
Change-Id: I403a667557a105411a633e62c8eec23d93724b43
Signed-off-by: Erwan Mahe <erwan.mahe@intel.com>
Signed-off-by: Pierre Tardy <pierre.tardy@intel.com>
BZ: 4779
Allows to revert a gerrit patch
This patch is necessary for the on-demand creation of
engineering builds using buildbot
You can now use:
repo download [--revert|-r project changeid/patchnumber
This is useful to automate reverting of a patch
in the context of build automation, and regression bisection
Change-Id: I3985e80e4b2a230f83526191ea1379765a54bdcf
Signed-off-by: Erwan Mahe <erwan.mahe@intel.com>
Signed-off-by: Pierre Tardy <pierre.tardy@intel.com>
default option uses git checkout, and thus overwrite the previous
checkouts. this is a problem for automated builds of several
changesets in the same project for daily builds of pending submission
You can now use:
repo download [--cherry-pick|-c] project changeid/patchnumber
This will parse the manifest, cd to the corresponding project
download the changes to FETCH_HEAD and cherry-pick the result.
This is useful to automate cherry-picking of a patch
in the context of build automation, and commit gating (e.g. buildbot)
Change-Id: Ib638afd87677f1be197afb7b0f73c70fb98909fe
Signed-off-by: Pierre Tardy <pierre.tardy@intel.com>
repo status should output filenames one by one instead of trying to
build a string from incompatible encodings (like utf-8 and sjis
filenames)
Change-Id: I52282236ececa562f109f9ea4b2e971d2b4bc045
Fixes three errors:
Python doesn't like the line wrap after 'and'.
platform.system is a function, needs to be platform.system().
Typo all_platfroms instead of all_platforms.
Change-Id: Ia875e521bc01ae2eb321ec62d839173c00f86c2d
Projects may optionally specify their platform
(eg, groups="platform-linux" in the manifest).
By default, repo will automatically detect the platform. However,
users may specify --platform=[auto|all|linux|darwin].
Change-Id: Ie678851fb2fec5b0938aede01f16c53138a16537
Every project is in group "default". "-default" does not remove
it from this project. All group names specified in the manifest
are positive names as opposed to a mix of negative and positive.
Specified groups are resolved in order. If init is supplied with
--groups="group1,-group2", the following describes the project
selection when syncing:
* all projects in "group1" will be added, and
* all projects in "group2" will be removed.
Change-Id: I1df3dcdb64bbd4cd80d675f9b2d3becbf721f661
Allow the optional addition of "annotation" nodes nested under
projects. Each annotation node must have "name" and "value"
attributes. These name/value pairs will be exported into the
environment during any forall command, prefixed with "REPO__"
In addition, an optional "keep" attribute with case insensitive "true"
or "false" values can be included to determine whether the annotation
will be exported with 'repo manifest'
Change-Id: Icd7540afaae02c958f769ce3d25661aa721a9de8
Signed-off-by: James W. Mills <jameswmills@gmail.com>
Allows specifying a list of groups with a -g argument to repo init.
The groups act on a group= attribute specified on projects in the
manifest.
All projects are implicitly labelled with "default" unless they are
explicitly labelled "-default".
Prefixing a group with "-" removes matching projects from the list
of projects to sync.
If any non-inverted manifest groups are specified, the default label
is ignored.
Change-Id: I3a0dd7a93a8a1756205de1d03eee8c00906af0e5
Reviewed-on: https://gerrit-review.googlesource.com/34570
Reviewed-by: Shawn Pearce <sop@google.com>
Tested-by: Shawn Pearce <sop@google.com>
The -u option causes 'repo diff' to generate diff output
with file paths relative to the repository root,
so the output can be applied to the Unix 'patch' command.
The name '-u' was selected for convenience, because
both 'diff' and 'git diff' accept the option with the same name
to generate an 'unified diff' output suitable for 'patch' command.
Change-Id: I79c8356db4ed20ecaccc258b3ba139db76666fe0
Reviewed-on: https://gerrit-review.googlesource.com/34380
Reviewed-by: Shawn Pearce <sop@google.com>
Tested-by: Shawn Pearce <sop@google.com>
A convenient equivalent to `repo upload --br=<current git branch>`.
Note that the head branch will be selected for each project
uploaded by repo, so different branches may be uploaded for
different projects.
Change-Id: I10ad8ceaa63f055105c2d847c6e329fa4226dbaf
repo status just prints "# on branch oprofile" if you have branched
in clean status. This doesn't really tell which branch is meant.
Instead we can use the same syntax with modified gits which will
give us detailed information.
Change-Id: I55fe5154d278e10a814281dd2ba501ec6e956730
This parameter changes the manifest used by 'repo sync' for only
this execution. It should be useful for developers wishing to get
the repo temporarily into a known state, without clobbering their
existing manifest.
Tested by shifting Chrome OS between minilayout and full, and
between several release-builder-generated manifests.
Change-Id: I14194b665195b0e78f368d9ec8b8a83227af2627
This reverts commit ee1c2f5717.
This breaks a lot of buildbot systems. Rolling it back for now
until we can understand what the breakage was and how to fix it.
If the user has already configured a workspace, use these values
when re-running 'repo init'.
Otherwise, if the user has global name and e-mail set, use these.
It's always possible to override this and be prompted by specifying
--config-name when running 'repo init'.
Change-Id: If45f0e4b14884071439fb02709dc5cb53f070f60
A default manifest URL can be specified using:
git config --global repo-manifest.<id>.url <url>
A default manifest server can be specified using:
git config --global repo-manifest.<id>.server <url>
A default git mirror reference can be specified using:
git config --global repo-manifest.<id>.reference <path>
This will allow the user to use 'repo init -u <id>' as
a shorter alternative to specifying the full URL.
Also, manifest server will not have to be specified in the
manifest XML and the reference will not have to be specified
on the command line. If they are, they will override these
default values however.
Change-Id: Ifdbc160bd5909ec7df9efb0c5d7136f1d9351754
Signed-off-by: Victor Boivie <victor.boivie@sonyericsson.com>
Several times one have done an upload only to later notice in gerrit
that the upload was done to the wrong branch as the git has not yet
been branched for the current git. This change will make repo print
what the destination branch is when asking the user if she wants to
go through with the upload.
Change-Id: Ia9c3a92a6a04c022edfebf4f8d651ac062bb1f3b
It is common in command line tools to indicate what the default answer
will be if the user simply hits enter. In repo, the display is just
"y/n" with no indication as to which is the default. So change the n
to N in the messages since that is how repo operates.
Change-Id: I81819ae630355072eb0365e59168b0921289498f
When commit with comment that has non-ASCII characters,
UnicodeDecodeError will be raised
while uploading multiple project/branch changes.
Because some strings in script are not str type, but unicode.
So all the strings are decoded to unicode,
and python use ascii to do this,
it can not decode non-ASCII characters,
so UnicodeDecodeError raised.
Signed-off-by: chenguodong <chenguodong@huawei.com>
Change-Id: I46447f489a4b9760a5899c7ba9d764b688594e46
There is also shortcuts in case if the "current branch" is
a persistent revision such as tag or sha1. We check if the
persistent revision is present locally and if it does - do
no fetch anything from the server.
This greately reduces sync time and size of the on-disk repo
Change-Id: I23c6d95185474ed6e1a03c836a47f489953b99be
help sync crashed as sync required the manifest to be configured to
create the option parser, as the default number of jobs is required.
Change-Id: Ie75e8d75ac0e38313e4aab451cbb24430e84def5
Signed-off-by: Shawn O. Pearce <sop@google.com>
An HTTP (or HTTPS) based remote server may now offer a 'clone.bundle'
file in each repository's Git directory. Over an http:// or https://
remote repo will first ask for '$URL/clone.bundle', and if present
download this to bootstrap the local client, rather than relying
on the native Git transport to initialize the new repository.
Bundles may be hosted elsewhere. The client automatically follows a
HTTP 302 redirect to acquire the bundle file. This allows servers
to direct clients to cached copies residing on content delivery
networks, where the bundle may be closer to the end-user.
Bundle downloads are resumeable from where they last left off,
allowing clients to initialize large repositories even when the
connection gets interrupted.
If a bundle does not exist for a repository (a HTTP 404 response
code is returned for '$URL/clone.bundle'), the native Git transport
is used instead. If the client is performing a shallow sync, the
bundle transport is not used, as there is no way to embed shallow
data into the bundle.
Change-Id: I05dad17792fd6fd20635a0f71589566e557cc743
Signed-off-by: Shawn O. Pearce <sop@google.com>
If the manifest is updated and the default sync-j attribute
was modified, honor it during this sync session if the user
has not supplied a -j flag on the command line.
Change-Id: I127ee5c779e2bbbb40b30bddc10ec1fa704b3bf3
Signed-off-by: Shawn O. Pearce <sop@google.com>
This permits manifest authors to suggest a number of parallel
fetch operations against a remote server. For example, Gerrit
Code Review servers support queuing of requests and processes
them in first-in, first-out order. Running concurrent fetches
can utilize multiple CPUs on the Gerrit server, but will also
decrease overall operation latency by having the request put
into the queue ready to execute as soon as a CPU is free.
Change-Id: I3d3904acb6f63516bae4b071c510ad57a2afab18
Signed-off-by: Shawn O. Pearce <sop@google.com>
Each worker thread requires at least 3 file descriptors to run the
forked 'git fetch' child to operate against the local repository.
Mac OS X has the RLIMIT_NOFILE set to 256 by default, which means
a sync -j128 often fails when the workers run out of pipes within
the Python parent process.
Change-Id: I2cdb14621b899424b079daf7969bc8c16b85b903
Signed-off-by: Shawn O. Pearce <sop@google.com>
This is an evolution of 'smart-sync' that adds a new option, -t,
that allows you to specify a tag/label to use instead of the
"latest good build" on the current manifest branch which -s does.
Signed-off-by: Victor Boivie <victor.boivie@sonyericsson.com>
Change-Id: I8c20fd91104a6aafa0271d4d33f6c4850aade17e
This commit adds a --br=<branch> option to repo upload.
repo currently examines every non-published branch. This is problematic
for my workflow. I have many branches in my kernel tree. Many of these
branches are based off of upstream remotes (I have many remotes) and
will never be uploaded (they'll get sent upstream as a patch).
Having repo scan these branches adds to my upload processing time
and clutters the branch selection buffer. I've also seen repo get
confused when one of my branches is 1000s of commits different from
m/master.
Change-Id: I68fa18951ea59ba373277b57ffcaf8cddd7e7a40
It is undesired to have the same Change-Id:-line for two separate
commits, and when cherry-picking, the user must manually change it.
If this is not done, bad things may happen (such as when the user
is uploading the cherry-picked commit to Gerrit, it will instead
see it as a new patch-set for the original change, or worse).
repo cherry-pick works the same was as git cherry-pick, except that
it replaces the Change-Id with a new one and adds a reference
back to the commit from where it was picked.
On failures (when git can not successfully apply the cherry-picked
commit), instructions will be written to the user.
Change-Id: I5a38b89839f91848fad43386d43cae2f6cdabf83
In the current version of repo checkout, we often get the error:
error: no project has branch xyzzy
...even when the actual error was something else. This fixes it
to only report the 'no project has branch' when that is actually true.
This fix is very similar to one made for 'repo abandon':
https://review.source.android.com/#change,22207
The repo checkout error is filed as: <http://crosbug.com/6514>
TEST=manual
A sample creating a case where 'git checkout' will fail:
$ repo start branch1 .
$ repo start branch2 .
$ touch bogusfile
$ git add bogusfile
$ git commit -m "create bogus file"
[branch2 f8b6b08] create bogus file
0 files changed, 0 insertions(+), 0 deletions(-)
create mode 100644 bogusfile
$ echo "More" >> bogusfile
$ repo checkout branch1 .
error: chromite/: cannot checkout branch1
A sample case showing that we still fail if no project has a branch:
$ repo checkout xyzzy .
error: no project has branch xyzzy
Change-Id: I48a8e258fa7a9c1f2800dafc683787204bbfcc63
This is the simplest fix: if we had problems syncing the
manifest.git directory and we were the ones that created it,
we should delete it. This doesn't try to do anything complex
like try to recover from a .repo directory that got broken in
some other way.
This is filed as: <http://crosbug.com/13403>
TEST=manual
Init once with a bad URL:
$ repo init -u http://foobar.example.com
Getting manifest ...
from http://foobar.example.com
Connection closed by 172.22.121.77
error: Couldn't resolve host 'foobar.example.com' while accessing http://foobar.example.com/info/refs
fatal: HTTP request failed
fatal: cannot obtain manifest http://foobar.example.com
Init again: identical to the first. Good:
$ repo init -u http://foobar.example.com
Getting manifest ...
from http://foobar.example.com
Connection closed by 172.22.121.77
error: Couldn't resolve host 'foobar.example.com' while accessing http://foobar.example.com/info/refs
fatal: HTTP request failed
fatal: cannot obtain manifest http://foobar.example.com
Init with correct URL:
$ repo init -u http://git.chromium.org/git/manifest -m minilayout.xml
Getting manifest ...
from http://git.chromium.org/git/manifest
[ ... cut ... ]
repo initialized in /.../repoiniterr
Try a bad URL after a good one; it doesn't get saved (good):
$ repo init -u http://foobar.example.com
Connection closed by 172.22.121.77
error: Couldn't resolve host 'foobar.example.com' while accessing http://foobar.example.com/info/refs
fatal: HTTP request failed
fatal: cannot obtain manifest http://foobar.example.com
Just to confirm, I can still do a good one after a bad...
$ repo init -u http://git.chromium.org/git/manifest -m minilayout.xml
Your Name [George Washington]:
Your Email [george@washington.example.com]:
Your identity is: George Washington <george@washington.example.com>
is this correct [y/n]? y
repo initialized in /.../repoiniterr
Change-Id: I1692821a330d97b1d218b2e191a93245b33f2362
The main fix is to give an error message if nothing was actually
abandoned. See <http://crosbug.com/6041>.
The secondary fix is to list projects where the abandon happened.
This could be done in a separate CL or dropped altogether if requested.
TEST=manual
$ repo abandon dougabc; echo $?
Abandon dougabc: 100% (127/127), done.
Abandoned in 2 project(s):
chromite
src/platform/init
0
$ repo abandon dougabc; echo $?
Abandon dougabc: 100% (127/127), done.
error: no project has branch dougabc
1
$ repo abandon dougabc; echo $?
Abandon dougabc: 100% (127/127), done.
error: chromite/: cannot abandon dougabc
1
Change-Id: I79520cc3279291acadc1a24ca34a761e9de04ed4
