Add two new command line options, -u/--manifest-server-username and
-p/--manifest-server-password, which can be used to specify a username
and password to authenticate to the manifest server when using the
-s/--smart-sync or -t/--smart-tag option.
If -u and -p are not specified when using the -s or -t option, use
authentication credentials from the .netrc file (if there are any).
Authentication credentials from -u/-p or .netrc are not used if the
manifest server specified in the manifest file already includes
credentials.
Change-Id: I6cf9540d28f6cef64c5694e8928cfe367a71d28d
manifest_xml: import `HEAD` and `R_HEADS` from correct module
version: import `HEAD` from correct module
`HEAD` and `R_HEADS` should be imported from the git_refs module,
where they are originally defined, rather than from the project
module.
repo: remove unused import of readline
cherry_pick: import standard modules on separate lines
smartsync: import subcmd modules explicitly from subcmd
Use:
`import re
import sys`
and
`from subcmds.sync import Sync`
Instead of:
`import sys, re`
and
`from sync import Sync`
Change-Id: Ie10dd6832710939634c4f5c86b9ba5a9cd6fc92e
When using the --smart-sync or --smart-tag option, and the specified
manifest server is hosted on a server that requires authentication,
repo sync fails with the error: HTTP 401 Unauthorized.
Add support for getting the credentials from the .netrc file.
If a .netrc file exists in the user's home directory, and it contains
credentials for the hostname of the manifest server specified in the
manifest, use the credentials to authenticate with the manifest server
using the URL syntax extension for Basic Authentication:
http://user:password@host:port/path
Credentials from the .netrc file are only used if the manifest server
URL specified in the manifest does not already include credentials.
Change-Id: I06e6586e8849d0cd12fa9746789e8d45d5b1f848
If the first line of manifest.xml is a XML comment, root.childNodes[0]
is not a 'manifest' element node. The python minidom module will makes
a 'Comment' node as root.childNodes[0]. Since the original code only
checks whether the first child node is 'manifest', it couldn't do any
command including 'sync' due to the 'ManifestParseError' exception. This
patch could allow the comments between '<?xml ...?>' and '<manifest>' in
the manifest.xml file.
Change-Id: I0b81dea4f806965eca90f704c8aa7df49c579402
`R_HEADS` is imported twice, from both the git_refs and project
modules.
It is actually defined in git_refs, and in project it is imported
from there, so the import of `R_HEADS` from project in the sync
module is redundant. Remove it.
`HEAD` is imported from project, but like `R_HEADS` it is actually
defined in git_refs. Import it from git_refs instead.
Change-Id: I8e2b0217d0d9f9f4ee5ef5b8cd0b026174ac52f4
When connecting to the manifest server, exceptions can occur but
are not caught, resulting in the repo sync exiting with a python
traceback.
Add handling of the following exceptions:
- IOError, which can be raised for example if the manifest server
URL is malformed.
- xmlrpclib.ProtocolError, which can be raised if the connection
to the manifest server fails with HTTP error.
- xmlrpclib.Fault, which can be raised if the RPC call fails for
some other reason.
Change-Id: I3a4830aef0941debadd515aac776a3932e28a943
Catch curl failures to download clone.bundle; don't let git try to parse
the 404 page as a bundle file (was causing much user confusion).
This should eliminate false error messages from init and sync such as:
error: '.repo/manifests.git/clone.bundle' does not look like a v2 bundle file
fatal: Could not read bundle '.repo/manifests.git/clone.bundle'.
error: RPC failed; result=22, HTTP code = 400
Signed-off-by: Matt Gumbel <matthew.k.gumbel@intel.com>
Change-Id: I7994f7c0baecfb45bb5a5850c48bd2a0ffabe773
Instead of every group being in the group "default", every project
is now in the group "all". A group that should not be downloaded
by default may be added to the group "notdefault".
This allows all group names to be positive (instead of removing groups
directly in the manifest with -default) and offers a clear way of
selecting every project (--groups all).
Change-Id: I99cd70309adb1f8460db3bbc6eff46bdcd22256f
The threaded 'repo sync' implementation would very often freeze the
process when interrupted by the user with Ctrl-C. The only solution
being to kill -9 the process explicitly from another terminal.
The reason for this is best explained here:
http://snakesthatbite.blogspot.fr/2010/09/cpython-threading-interrupting.html
This patch makes all helper sync threads 'daemon', which allows the
process to terminate immediately on Ctrl-C.
Note that this will forcefully kill all threads in case of interruption; this
is generally a bad thing, but:
1/ This is equivalent to calling kill -9 in another terminal, which
is the _only_ thing that can currently stop the process.
2/ There doesn't seem to be a way to tell the worker threads to
gently stop when they are in a blocking operation anyway (even
in the non-threaded case).
+ Do the same for "repo status -j<count>".
Change-Id: Ieaf45b0eacee36f35427f8edafd87415c2aa7be4
master's original purpose was to forge ahead on using git submodules,
but this route has been abandoned.
Change-Id: I164a9efc7821bcd1b941ad76649764722046081b
One of the recent changes introduced implicit path:xxx and name:xxx groups
to every project, however they are not being stripped when generating
a manifest using "repo manifest" command resulting in clutter
Change-Id: Iec8610ba794b2fe4a6cdf0f59ca561595b66f9b5
urllib2 is not thread safe and may be causing sync to lock up or
not work correctly on various platforms. Instead use the command
line curl program.
Change-Id: I36eaf18bb4df089d26ea99d533cb015e7c616eb0
The Content-Length when resuming is the number of bytes that
remain in the file. To compute the total size as expected by
the progress meter, we must add the bytes already stored.
While we are in this method fix uses of % operator to ensure
a tuple is always supplied.
Change-Id: Ic899231b5bc0ab43b3ddb1d29845f6390e820115
The `alias` is an optional attribute in element `remote`. It can be
used to override attibute `name` to be set as the remote name in each
project's .git/config. Its value can be duplicated while attribute
`name` has to be unique across the manifest file. This helps each
project to be able to have same remote name which actually points
to different remote url.
It eases some automation scripts to be able to checkout/push to same
remote name but actually different remote url, like:
repo forall -c "git checkout -b work same_remote/work"
repo forall -c "git push same_remote work:work"
for example:
The manifest with 'alias' will look like:
<?xml version='1.0' encoding='UTF-8'?>
<manifest>
<remote alias="same_alias" fetch="git://git.external1.org/" name="ext1"
review="http://review.external1.org"/>
<remote alias="same_alias" fetch="git://git.external2.org/" name="ext2"
review="http://review.external2.org"/>
<remote alias="same_alias" fetch="ssh://git.internal.com:29418" name="int"
review="http://review.internal.com"/>
<default remote="int" revision="int-branch" sync-j="2"/>
<project name="path/to/project1" path="project1" remote="ext1"/>
<project name="path/to/project2" path="project2" remote="ext2"/>
<project name="path/to/project3" path="project3"/>
...
</manifest>
In each project, use command "git remote -v"
project1:
same_alias git://git.external1.org/project1 (fetch)
same_alias git://git.external1.org/project1 (push)
project2:
same_alias git://git.external2.org/project2 (fetch)
same_alias git://git.external2.org/project2 (push)
project3:
same_alias ssh://git.internal.com:29418/project3 (fetch)
same_alias ssh://git.internal.com:29418/project3 (push)
Change-Id: I2c48263097ff107f0c978f3e83966ae71d06cb90
The overview command shows an overview of each branch in all (or the
specified) projects. The overview lists any local commits that have
not yet been merged into the project.
The report output is inspired by the report displayed following a
"repo prune" event, with the addition of listing the one-line log
messages for each commit that is not yet merged.
The report can also be filtered to show only active branches; by
default all branches that have commits beyond the upstream HEAD will
be listed.
Change-Id: Ibe67793991ad1aa38de3bc9747de4ba64e5591aa
For CrOS, we have scenarios were people checkout a smaller version
of our manifest via groups, and enable individual repositories as
needed for their work. Previously this was via local_manifest
manipulation, which breaks via manifest-groups would require a
remove-project tag.
Via injecting the projects name into the projects groups, this
allows us to instead manipulate the configured groups allowing
the user to turn on/off projects as necessary.
Change-Id: I07b7918e16cc9dc28eb47e19a46a04dc4fd0be74
The fix for issue #46 in 5d016502eb appears to break syncing in some
situations: the branch is deleted after the point where it's been
configured, which deletes part of its configuration and causes the
config to change each time you call `repo init`, alternating between a
configuration that works and one that doesn't.
Instead of deleting the branch with git branch -D, use git update-ref -d
which just deletes the ref (to avoid the rebase) without touching the
configuration for the branch that was set up during the first repo init.
This appears to ensure the config is left in a valid state all the time
no matter what combination of repo init commands you run, without
reintroducing the rebasing issue.
Change-Id: Iaadaa6e56a46840bbc593fa5b35cb5b34cd3ce69
Git requires the values in this environment variable to be
single quoted. repo must wrap the expression into '' before
adding it to the environment.
Change-Id: I20a1fb8772f9aa6e9fd5a0516c981c2ca020ef05
This patch fixes repo behaviour when running sync -d with unmodified
topic branches.
Prior to this patch sync -d would see the latest revision is already
checked out, thus staying on the branch. Since "-d" means detach we
should follow git's behaviour and actually detach from the branch in
that case.
Basic test case - after a fresh repo init + sync -
* repo start --all testdetach
* repo sync -d
* repo status
-> status shows active topic branch "testdetach",
should show :
nothing to commit (working directory clean)
Change-Id: Ic1351e6b5721b76557a51ab09f9dd42c38a4b415
Currently repo-rebase requires that all modifications be committed
locally before it will allow the rebase. In high-velocity environments,
you may want to just pull in newer code without explicitly creating
local commits, which is typically achieved using git-stash.
If called with the --auto-stash command line argument, and it is
determined that the current index is dirty, the local modifications
are stashed, and the rebase continues. If a stash was performed, that
stash is popped once the rebase completes.
Note that there is still a possibility that the git-stash pop will
result in a merge conflict.
Change-Id: Ibe3da96f0b4486cb7ce8d040639187e26501f6af
See repo issue #46 :
https://code.google.com/p/git-repo/issues/detail?id=46
When using repo init -b on an already existing repository,
the next sync will try to rebase changes coming from the old manifest
branch onto the new, leading in the best case scenario to conflicts
and in the worst case scenario to an incorrect "mixed up" manifest.
This patch fixes this by deleting the "default" branch in the local
manifest repository when the -d init switch is used, thus forcing
repo to perform a fresh checkout of the new manifest branch
Change-Id: I379e4875ec5357d8614d1197b6afbe58f9606751
Calculation of where the include file lives was broken by 23acdd3f14
since it resulted in looking for the first include in .repo, rather
than .repo/manifests.
While people can work around it via setting their includes to
manifests/<include-target>, that breaks down since each layer of
includes would then have to be relative.
As such, restore the behaviour back to 2644874d; manifests includes
are calculated relative to the manifest root (ie, .repo/manifests);
local manifests includes are calculated relative to .repo/ .
Change-Id: I74c19ba614c41d2f08cd3e9fd094f3c510e3bfd1
Ancient versions of Git don't understand the -c command line flag
that we tried to use to pass http_proxy down into Git on Darwin.
Use the environment variable instead, to more gracefully degrade
with these old versions.
Change-Id: Iffffa32088c1fd803895b990b3377ecfec6a1b14
This patch adds the option to include topic branches by adding the
following to a .gitconfig file:
uploadtopic = true
This option is only read in when the -t option is not already
specified at the command line.
Change-Id: I0e0eea49438bb4e4a21c2ac5bd498b68b5a9a845
This is basically the same repository, but may be slightly more
up-to-date than the one on code.google.com/p/git-repo.
Change-Id: I5c99539f53231958eefb6993f00997c9adf0a3c9
Fix detection for Git not being in $PATH during the initial
run of `repo init` in a new directory.
Change-Id: I2b1fcce1fb8afc47271f5c3bd2a28369009b2fb7
The system libcurl library seems to ignore http_proxy on Mac OS
X systems. Copy the http_proxy environment variable (if set) as
`git -c http.proxy` whenever running a Git command.
Change-Id: I0ab29336897178f70b85092601f9fcc306dd17e1
Barring any kernel bugs, if this directory exists and there is
a symlink in there (which will point to the battery object),
that means there is a battery known to the kernel.
No symlink should mean no battery as far as the kernel is concerned.
Change-Id: Ib12819a5bbb816f0ae5ca080e5812a2db08441e9
Mirror manifest and repo projects are outside the manifest and
have no groups. Allow project groups to be None for these
projects.
Change-Id: I3e1c4add894fe1c43aa4e77a1fc1558aa10dd191
Combine manifest and local_manifest into a single list of elements
before parsing. This will allow elements in the local_manifest to
affect elements in the main manifest.
Change-Id: I4d34c9260b299a76be2960b07c0c3fe1af35f33c
Having the ability to include other manifests is a very practical feature
to ease the managment of manifest. It allows to divide a manifest into separate
files, and create different environment depending on what we want to release
You can have unlimited recursion of include, the manifest configs will simply be concatenated
as if it was in a single file.
command "repo manifest" will create a single manifest, and not recreate the manifest hierarchy
for example:
Our developement manifest will look like:
<?xml version='1.0' encoding='UTF-8'?>
<manifest>
<default revision="platform/android/main" remote="intel"/>
<include name="server.xml"/> <!-- The Server configuration -->
<include name="aosp.xml" /> <!-- All the AOSP projects -->
<include name="bsp.xml" /> <!-- The BSP projects that we release in source form -->
<include name="bsp-priv.xml" /> <!-- The source of the BSP projects we release in binary form -->
</manifest>
Our release manifest will look like:
<?xml version='1.0' encoding='UTF-8'?>
<manifest>
<default revision="platform/android/release-ext" remote="intel"/>
<include name="server.xml"/> <!-- The Server configuration -->
<include name="aosp.xml" /> <!-- All the AOSP projects -->
<include name="bsp.xml" /> <!-- The BSP projects that we release in source form -->
<include name="bsp-ext.xml" /> <!-- The PREBUILT version of the BSP projects we release in binary form -->
</manifest>
And it is also easy to create and maintain feature branch with a manifest that looks like:
<?xml version='1.0' encoding='UTF-8'?>
<manifest>
<default revision="feature_branch_foobar" remote="intel"/>
<include name="server.xml"/> <!-- The Server configuration -->
<include name="aosp.xml" /> <!-- All the AOSP projects -->
<include name="bsp.xml" /> <!-- The BSP projects that we release in source form -->
<include name="bsp-priv.xml" /> <!-- The source of the BSP projects we release in binary form -->
</manifest>
Signed-off-by: Brian Harring <brian.harring@intel.com>
Signed-off-by: Pierre Tardy <pierre.tardy@intel.com>
Change-Id: I833a30d303039e485888768e6b81561b7665e89d
Allows to ff-only a gerrit patch
This patch is necessary to automatically ensure that the patch will
be correctly submitted on ff-only gerrit projects
You can now use:
repo download (--ff-only|-f) project changeid/patchnumber
This is useful to automate verification of fast forward status of a patch
in the context of build automation, and commit gating (e.g. buildbot)
Change-Id: I403a667557a105411a633e62c8eec23d93724b43
Signed-off-by: Erwan Mahe <erwan.mahe@intel.com>
Signed-off-by: Pierre Tardy <pierre.tardy@intel.com>
BZ: 4779
Allows to revert a gerrit patch
This patch is necessary for the on-demand creation of
engineering builds using buildbot
You can now use:
repo download [--revert|-r project changeid/patchnumber
This is useful to automate reverting of a patch
in the context of build automation, and regression bisection
Change-Id: I3985e80e4b2a230f83526191ea1379765a54bdcf
Signed-off-by: Erwan Mahe <erwan.mahe@intel.com>
Signed-off-by: Pierre Tardy <pierre.tardy@intel.com>
default option uses git checkout, and thus overwrite the previous
checkouts. this is a problem for automated builds of several
changesets in the same project for daily builds of pending submission
You can now use:
repo download [--cherry-pick|-c] project changeid/patchnumber
This will parse the manifest, cd to the corresponding project
download the changes to FETCH_HEAD and cherry-pick the result.
This is useful to automate cherry-picking of a patch
in the context of build automation, and commit gating (e.g. buildbot)
Change-Id: Ib638afd87677f1be197afb7b0f73c70fb98909fe
Signed-off-by: Pierre Tardy <pierre.tardy@intel.com>
repo status should output filenames one by one instead of trying to
build a string from incompatible encodings (like utf-8 and sjis
filenames)
Change-Id: I52282236ececa562f109f9ea4b2e971d2b4bc045
There are use-cases when fetching all branch is impractical and
we really need to fetch only one branch/tag.
e.g. there is a large project with binaries and every update of a
binary file is put to a separate branch.
The whole project history might be too large to allow users fetch it.
Add 'sync-c' option to 'project' and 'default' tags to make it possible
to configure 'sync-c' behavior at per-project and per-manifest level.
Note that currently there is no possibility to revert boolean flag from
command line. If 'sync-c' is set in manifest then you cannot make
full fetch by providing a repo tool argument.
Change-Id: Ie36fe5737304930493740370239403986590f593
