Commit Graph

484 Commits

Author SHA1 Message Date
Conley Owens
84f7e137c2 Merge "Check for a cookie file when fetching clone.bundle." 2013-01-17 20:02:03 +00:00
Joe Onorato
26e2475a0f If parsing the manifests fails, reset the XmlManifest object
so that if it's called again, we see the correct errors.

Change-Id: I909488feeac04aecfc92a9b5d6fb17827ef2f213
2013-01-17 09:10:00 -08:00
Conley Owens
c59bafffb9 Merge "Pass full path of the XML in local_manifests to the parser" 2013-01-17 17:06:49 +00:00
Conley Owens
0290cad5db Merge "Make -notdefault a default manifest group" 2013-01-14 22:34:13 +00:00
Torne (Richard Coles)
ed68d0e852 Check for a cookie file when fetching clone.bundle.
If the user's git config specifies a cookie file for HTTP, use it when
fetching clone.bundle, as it may contain the required login credentials
to get access (e.g. when used with Compute Engine service accounts).

Change-Id: I12ee9e38694822ef1de4ed62138c3876c43f431b
2013-01-11 16:22:54 +00:00
Tobias Droste
1a5c774cbf Pass full path of the XML in local_manifests to the parser
This fixes the following python error message if the
current working directory is not .repo/local_manifests:
IOError: [Errno 2] No such file or directory: 'local_manifest.xml'

Signed-off-by: Tobias Droste <tdroste87@gmail.com>
CC: David Pursehouse <david.pursehouse@sonymobile.com>
Change-Id: I4668dc04219b6233c7ff6ca3b4138bec9ee3529f
2013-01-07 07:46:18 +01:00
Shawn Pearce
a9f11b3cb2 Support resolving relative fetch URLs on persistent-https://
Some versions of Python will only attempt to resolve a relative
URL if they understand the URL scheme. Convert persistent-http://
and persistent-https:// schemes to the more typical http:// and
https:// versions for the resolve call.

Change-Id: I99072d5a69be8cfaa429a3ab177ba644d928ffba
2013-01-02 15:41:57 -08:00
Matt Gumbel
0c635bb427 Make -notdefault a default manifest group
When trying to render manifest with SHAs, projects in group notdefault
caused the following crash:

    Traceback (most recent call last):
      File ".repo/repo/main.py", line 385, in <module>
        _Main(sys.argv[1:])
      File ".repo/repo/main.py", line 365, in _Main
        result = repo._Run(argv) or 0
      File ".repo/repo/main.py", line 137, in _Run
        result = cmd.Execute(copts, cargs)
      File ".repo/repo/subcmds/manifest.py", line 129, in Execute
        self._Output(opt, manifest)
      File ".repo/repo/subcmds/manifest.py", line 79, in _Output
        peg_rev = opt.peg_rev)
      File ".repo/repo/manifest_xml.py", line 199, in Save
        p.work_git.rev_parse(HEAD + '^0'))
      File ".repo/repo/project.py", line 2035, in runner
        capture_stderr = True)
      File ".repo/repo/git_command.py", line 215, in __init__
        raise GitError('%s: %s' % (command[1], e))
    error.GitError: rev-parse: [Errno 2] No such file or directory: 'prebuilts/eclipse-build-deps'

This patch resolves the issue by making sure that -notdefault is always
used as a default manifest group so that notdefault projects are not
rendered out by the manifest subcmd.

Change-Id: I4a8bd18ea7600309f39ceff1b1ab6e1ff3adf21d
Signed-off-by: Matt Gumbel <matthew.k.gumbel@intel.com>
2012-12-21 10:14:53 -08:00
Torne (Richard Coles)
7bdbde7af8 Allow sync to run even when the manifest is broken.
If the current manifest is broken then "repo sync" fails because it
can't retrieve the default value for --jobs. Use 1 in this case, in
order that you can "repo sync" to get a fixed manifest (assuming someone
fixed it upstream).

Change-Id: I4262abb59311f1e851ca2a663438a7e9f796b9f6
2012-12-05 11:01:36 +00:00
Che-Liang Chiou
b2bd91c99b Represent git-submodule as nested projects, take 2
(Previous submission of this change broke Android buildbot due to
 incorrect regular expression for parsing git-config output.  During
 investigation, we also found that Android, which pulls Chromium, has a
 workaround for Chromium's submodules; its manifest includes Chromium's
 submodules.  This new change, in addition to fixing the regex, also
 take this type of workarounds into consideration; it adds a new
 attribute that makes repo not fetch submodules unless submodules have a
 project element defined in the manifest, or this attribute is
 overridden by a parent project element or by the default element.)

We need a representation of git-submodule in repo; otherwise repo will
not sync submodules, and leave workspace in a broken state.  Of course
this will not be a problem if all projects are owned by the owner of the
manifest file, who may simply choose not to use git-submodule in all
projects.  However, this is not possible in practice because manifest
file owner is unlikely to own all upstream projects.

As git submodules are simply git repositories, it is natural to treat
them as plain repo projects that live inside a repo project.  That is,
we could use recursively declared projects to denote the is-submodule
relation of git repositories.

The behavior of repo remains the same to projects that do not have a
sub-project within.  As for parent projects, repo fetches them and their
sub-projects as normal projects, and then checks out subprojects at the
commit specified in parent's commit object.  The sub-project is fetched
at a path relative to parent project's working directory; so the path
specified in manifest file should match that of .gitmodules file.

If a submodule is not registered in repo manifest, repo will derive its
properties from itself and its parent project, which might not always be
correct.  In such cases, the subproject is called a derived subproject.

To a user, a sub-project is merely a git-submodule; so all tips of
working with a git-submodule apply here, too.  For example, you should
not run `repo sync` in a parent repository if its submodule is dirty.

Change-Id: I4b8344c1b9ccad2f58ad304573133e5d52e1faef
2012-11-19 10:45:21 -08:00
David Pursehouse
3f5ea0b182 Allow init command to set options from environment variables
The manifest URL and mirror location can be specified in environment
variables which will be used if the options are not passed on the
command line

Change-Id: Ida87968b4a91189822c3738f835e2631e10b847e
2012-11-17 12:40:42 +09:00
David Pursehouse
b148ac9d9a Allow command options to be set from environment variables
Extend the Command base class to allow options to be set from values
in environment variables, if the user has not given the option on the
command line and the environment variable is set.

Derived classes of Command can override the implementation of the method
_GetEnvironmentOptions to configure which of its options may be set from
environment variables.

Change-Id: I7c780bcf9644d6567893d9930984c054bce7351e
2012-11-17 12:40:42 +09:00
Conley Owens
a67df63ef1 Merge "Raise a NoManifestException when the manifest DNE" 2012-11-16 10:39:24 -08:00
David Pursehouse
f91074881f Better error message if 'remove-project' refers to non-existent project
If a local manifest includes a 'remove-project' element that refers to
a project that does not exist in the manifest, the error message is a
bit cryptic.

Change the error message to make it clearer what is wrong.

Change-Id: I0b1043aaec87893c3128211d3a9ab2db6d600755
2012-11-16 19:12:55 +09:00
Conley Owens
75ee0570da Raise a NoManifestException when the manifest DNE
When a command (eg, `repo forall`) expects the manifest project to
exist, but there is no manifest, an IOException gets raised.  This
change defines a new Exception type to be raised in these cases and
raises it when project.py fails to read the manifest.

Change-Id: Iac576c293a37f7d8f60cd4f6aa95b2c97f9e7957
2012-11-15 18:50:11 -08:00
Victor Boivie
88b86728a4 Add option to abort on error in forall
Add a new option (-e, --abort-on-errors) which will cause forall to
abort without iterating through remaining projects if a command
exits unsuccessfully.

Bug: Issue 17
Change-Id: Ibea405e0d98b575ad3bda719d511f6982511c19c
Signed-off-by: Victor Boivie <victor.boivie@sonyericsson.com>
2012-11-16 04:22:10 +09:00
Conley Owens
e66291f6d0 Merge "Simplify error handling in subcommand execution" 2012-11-14 16:22:41 -08:00
Conley Owens
7ba25bedf9 Simplify error handling in subcommand execution
Instead of using a nested try (which repo is plagued with), use a single
try when executing the appropriate subcommand.

Change-Id: I32dbfc010c740c0cc42ef8fb6a83dfe87f87e54a
2012-11-14 14:18:06 -08:00
David Pursehouse
3794a78b80 Sync help text in repo from init.py
Change Ia6032865f9296b29524c2c25b72bd8e175b30489 improved the
help text for the init command, but the same improvement was not made
in repo.

Change-Id: Idc34e479b5237137b90e8b040824776e4f7883b0
2012-11-15 06:21:24 +09:00
Olof Johansson
33949c34d2 Add repo info command
The info command will print information regarding the current manifest
and local git branch. It will also show the difference of commits
between the local branch and the remote branch.

It also incorporates an overview command into info which shows commits
over all branches.

Change-Id: Iafedd978f44c84d240c010897eff58bbfbd7de71
2012-11-15 03:29:01 +09:00
David Pursehouse
8f62fb7bd3 Tidy up code formatting a bit more
Enable the following Pylint warnings:

  C0322: Operator not preceded by a space
  C0323: Operator not followed by a space
  C0324: Comma not followed by a space

And make the necessary fixes.

Change-Id: I74d74283ad5138cbaf28d492b18614eb355ff9fe
2012-11-14 12:09:38 +09:00
David Pursehouse
98ffba1401 Fix: "Statement seems to have no effect"
Pylint raises an error on the call:

  print

Change it to:

 print()

Change-Id: I507e1b3dd928fa6c32ea7e86260fb3d7b1428e6f
2012-11-14 11:38:57 +09:00
David Pursehouse
c1b86a2323 Fix inconsistent indentation
The repo coding style is to indent at 2 characters, but there are
many places where this is not followed.

Enable pylint warning "W0311: Bad indentation" and make sure all
indentation is at multiples of 2 characters.

Change-Id: I68f0f64470789ce2429ab11104d15d380a63e6a8
2012-11-14 11:38:57 +09:00
Sarah Owens
cecd1d864f Change print statements to work in python3
This is part of a series of changes to introduce Python3 support.

Change-Id: I373be5de7141aa127d7debdbce1df39148dbec32
2012-11-13 17:33:56 -08:00
David Pursehouse
fc241240d8 Convert prompt answers to lower case before checking
When prompting for yes/no answers, convert the answer to lower
case before comparing.  This makes it easier to catch answers
like "Yes", "yes", and "YES" with a comparison only for "yes".

Change-Id: I06da8281cec81a7438ebb46ddaf3344d12abe1eb
2012-11-14 09:19:39 +09:00
David Pursehouse
9f3406ea46 Minor documentation formatting and grammatical fixes
Change-Id: Iaac6377c787b3bb42242780e9d1116e718e0188d
2012-11-14 08:54:43 +09:00
David Pursehouse
b1525bffae Fix documentation reference to local_manifest.xml
Documentation of the remove-project element still refers explicitly
to local_manifest.xml.

Change it to the more generic "a local manifest".

Change-Id: I6278beab99a582fae26a4e053adc110362c714c2
2012-11-14 08:54:04 +09:00
David Pursehouse
685f080d62 More code style cleanup
Clean up a few more unnecessary usages of lambda in `repo` that were missed
in the previous sweep that only considered files ending in .py.

Remove a duplicate import.

Change-Id: I03cf467a5630cbe4eee6649520c52e94a7db76be
2012-11-14 08:34:39 +09:00
David Pursehouse
8898e2f26d Remove magic hack
It should be assumed that on modern development environments, python
is accessible to /usr/bin/env

Change the shebang as necessary and remove the magic hack.

This also means losing the -E option on the call to python, so that
PYTHONPATH and PYTHONHOME will be respected and local configuration
problems in those vars would be noticed

Change-Id: I6f0708ca7693f05a4c3621c338f03619563ba630
2012-11-14 08:17:11 +09:00
David Pursehouse
52f1e5d911 Make load order of local manifests deterministic
Local manifest files stored in the local_manifests folder are loaded
in alphabetical order, so it's easier to know in which order project
removals/additions/modifications will be applied.

If local_manifests.xml exists, it will be loaded before the files in
local_manifests.

Change-Id: Ia5c0349608f1823b4662cd6b340b99915bd973d5
2012-11-14 05:05:32 +09:00
Conley Owens
8e3d355d44 Merge "Print an error message when aborted by user" 2012-11-12 17:35:47 -08:00
Conley Owens
4a4776e9ab Merge "Handle manifest parse errors in main" 2012-11-12 17:35:40 -08:00
Conley Owens
2fa715f8b5 Merge "Better handling of duplicate remotes" 2012-11-12 17:35:30 -08:00
Conley Owens
6287543e35 Merge "Change usages of xrange() to range()" 2012-11-12 17:30:55 -08:00
David Pursehouse
b0936b0e20 Print an error message when aborted by user
Change-Id: If7378c5deaace0ac6ab2be961e38644d9373557d
2012-11-13 09:56:16 +09:00
David Pursehouse
0b8df7be79 Handle manifest parse errors in main
Add handling of manifest parse errors in the main method, and
print an error.  This will prevent python tracebacks being
dumped in many cases.

Change-Id: I75e73539afa34049f73c993dbfda203f1ad33b45
2012-11-13 09:54:47 +09:00
David Pursehouse
717ece9d81 Better handling of duplicate remotes
In the current implementation, an error is raised if a remote with the
same name is defined more than once.  The check is only that the remote
has the same name as an existing remote.

With the support for multiple local manifests, it is more likely than
before that the same remote is defined in more than one manifest.

Change the check so that it only raises an error if a remote is defined
more than once with the same name, but different attributes.

Change-Id: Ic3608646cf9f40aa2bea7015d3ecd099c5f5f835
2012-11-13 09:35:37 +09:00
David Pursehouse
5566ae5dde Print deprecation warning when local_manifest.xml is used
The preferred way to specify local manifests is to drop the file(s)
in the local_manifests folder.  Print a deprecation warning when
the legacy local_manifest.xml file is used.

Change-Id: Ice85bd06fb612d6fcceeaa0755efd130556c4464
2012-11-13 08:19:51 +09:00
David Pursehouse
2d5a0df798 Add support for multiple local manifests
Add support for multiple local manifests stored in the local_manifests
folder under the .repo home directory.

Local manifests will be processed in addition to local_manifest.xml.

Change-Id: Ia0569cea7e9ae0fe3208a8ffef5d9679e14db03b
2012-11-13 08:19:51 +09:00
David Pursehouse
f7fc8a95be Handle XML errors when parsing the manifest
Catch ExpatError and exit gracefully with an error message, rather
than exiting with a python traceback.

Change-Id: Ifd0a7762aab4e8de63dab8a66117170a05586866
2012-11-13 05:53:41 +09:00
Conley Owens
1ad7b555df Merge "Always show --manifest-server-* options" 2012-11-07 12:39:25 -08:00
David Pursehouse
7e6dd2dff0 Fix pylint warning W0108: Lambda may not be necessary
Remove unnecessary usage of lambda.

Change-Id: I06d41933057d60d15d307ee800cca052a44754c6
2012-11-07 08:39:57 +09:00
Conley Owens
8d070cfb25 Always show --manifest-server-* options
The --manifest-server-* flags broke the smartsync subcmd since
the corresponding variables weren't getting set.  This change
ensures that they will always be set, regardless of whether we are
using sync -s or smartsync.

Change-Id: I1b642038787f2114fa812ecbc15c64e431bbb829
2012-11-06 13:14:31 -08:00
Sarah Owens
a6053d54f1 Change usages of xrange() to range()
In Python3, range() creates a generator rather than a list.

None of the parameters in the ranges changed looked large enough
to create an impact in memory in Python2.  Note: the only use of
range() was for iteration and did not need to be changed.

This is part of a series of changes to introduce Python3 support.

Change-Id: I50b665f9296ea160a5076c71f36a65f76e47029f
2012-11-01 13:36:50 -07:00
Conley Owens
e072a92a9b Merge "Use python3 urllib when urllib2 not available" 2012-11-01 10:13:34 -07:00
Conley Owens
7601ee2608 Merge "Use 'stat' package instead of literals for mkdir()" 2012-11-01 10:01:18 -07:00
Sarah Owens
1f7627fd3c Use python3 urllib when urllib2 not available
This is part of a series of changes to introduce Python3 support.

Change-Id: I605b145791053c1f2d7bf3c907c5a68649b21d12
2012-10-31 14:26:48 -07:00
Dave Borowitz
b42b4746af project: Require git >= 1.7.2 for setting config on command line
This option causes the git call to fail, which probably indicates a
programming error; callers should check the git version and change the
call appropriately if -c is not available. Failing loudly is preferable
to failing silently in the general case.

For an example of correctly checking at the call site, see I8fd313dd.
If callers prefer to fail silently, they may set GIT_CONFIG_PARAMETERS
in the environment rather than using the config kwarg to pass
configuration.

Change-Id: I0de18153d44d3225cd3031e6ead54461430ed334
2012-10-31 12:27:27 -07:00
Dave Borowitz
e21526754b sync: Only parallelize gc for git >= 1.7.2
This minimum version is required for the -c argument to set config on
the command line. Without this option, git by default uses as many
threads per invocation as there are CPUs, so we cannot safely
parallelize without hosing a system.

Change-Id: I8fd313dd84917658162b5134b2d9aa34a96f2772
2012-10-31 12:27:17 -07:00
Sarah Owens
60798a32f6 Use 'stat' package instead of literals for mkdir()
This is part of a series of changes to introduce Python3 support.

Change-Id: Ic988ad181d32357d82dfa554e70d8525118334c0
2012-10-31 09:11:16 -07:00