Commit Graph

122 Commits

Author SHA1 Message Date
Conley Owens
1e7ab2a63f Respect remote aliases
Previously, change I7150e449341ed8655d398956a095261978d95870
had broken alias support in order to fix the manifest command to keep
it from spitting projects that point to an alias that wasn't recorded.
This commit reverts that commit and instead solves the issue more
correctly, outputting the alias in the remote node of the manifest and
respecting that alias when outputting the list of projects.

Change-Id: I941fc4adb7121d2e61cedc5838e80d3918c977c3
2013-10-08 17:26:57 -07:00
Conley Owens
b6a16e6390 Give the node _Default class a destBranchExpr
This is to avoid the following AttributeError:

line 681, in _ParseProject
AttributeError: '_Default' object has no attribute 'destBranchExpr'

Change-Id: Ia9f7e2cce1409d22d71bc8a74b33edf2b27702ca
2013-09-25 15:07:22 -07:00
Chirayu Desai
db2ad9dfce Fix urllib.parse (urlparse) handling
Revert "Fix "'module' object is not callable" error", and fix it properly.

* The urlparse module is renamed to urllib.parse in Python 3.
* This commit fixes the code to use "urllib.parse.urlparse"
  instead of creating a new module urlib and setting
  urlib.parse to urlparse.urlparse.
* Fixes an AttributeError:
  'function' object has no attribute 'uses_relative'

This reverts commit cd51f17c64.

Change-Id: I48490b20ecd19cf5a6edd835506ea5a467d556ac
2013-06-11 08:21:10 +00:00
David Pursehouse
cd51f17c64 Fix "'module' object is not callable" error
In a couple of files the urlparse method was not being set up
correctly for python < 3 and this resulted in an error being
thrown when trying to call it.

Change-Id: I4d2040ac77101e4e228ee225862f365ae3d96cec
2013-06-08 14:50:41 +09:00
Bryan Jacobs
f609f91b72 Send reviews to a different branch from fetch
This adds the ability to have reviews pushed to a different branch
from the one on which changes are based. This is useful for "gateway"
systems without smartsync.

Change-Id: I3a8a0fabcaf6055e62d3fb55f89c944e2f81569f
2013-05-24 12:17:22 -04:00
David Pursehouse
59bbb580e3 Move Python version checking to a separate module
Add a new module with methods for checking the Python version.

Instead of handling Python3 imports with try...except blocks, first
check the python version and then import the relevant modules.  This
makes the code a bit cleaner and will result in less diff when/if we
remove support for Python < 3 later.

Use the same mechanism to handle `input` vs. `raw_input` and add
suppression of pylint warnings caused by redefinition of the built-in
method `input`.

Change-Id: Ia403e525b88d77640a741ac50382146e7d635924
Also-by: Chirayu Desai <cdesai@cyanogenmod.org>
Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
2013-05-23 07:28:53 +00:00
Chirayu Desai
217ea7d274 Some fixes for supporting python3
* Fix imports.
* Use python3 syntax.
* Wrap map() calls with list().
* Use list() only wherever needed.
  (Thanks Conley!)
* Fix dictionary iteration methods
  (s/iteritems/items/).
* Make use of sorted() in appropriate places
* Use iterators directly in the loop.
* Don't use .keys() wherever it isn't needed.
* Use sys.maxsize instead of sys.maxint

TODO:
* Make repo work fully with python3. :)

Some of this was done by the '2to3' tool [1], by
applying the needed fixes in a way that doesn't
break compatibility with python2.

Links:
[1]: http://docs.python.org/2/library/2to3.html

Change-Id: Ibdf3bf9a530d716db905733cb9bfef83a48820f7
Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
2013-04-18 21:35:49 +05:30
Scott Fan
db83b1b5ab Allow mirror to be created in directories specified by 'path' attribute
In some cases, especially when local manifest files exist, users may want
to force the mirrored repositories to be created in folders according to
their 'path' attribute in the manifest, rather than according to the name
of the repositories.

To enable this functionality for specified mirror, add a new attribute
'force-path' for that project in the manifest, set its value to 'true'.

Change-Id: I61df8c987a23d84309b113e7d886ec90c838a6cc
Signed-off-by: Scott Fan <fancp2007@gmail.com>
2013-04-11 08:59:09 +08:00
David Pursehouse
ede7f12d4a Allow clone depth to be specified per project
If the clone-depth attribute is set on a project, its value will
be used to set the depth when fetching the git.  The value, if
given, must be a positive integer.

The value in the clone-depth attribute overrides any value given to
repo init via the --depth command line option.

Change-Id: I273015b3724213600b63e40cca4cafaa9f782ddf
2013-04-10 09:17:50 +09:00
Brian Harring
33e0456737 Fix repo manifest support of remote aliases.
Long story short, w/out this modification the manifest dump points
at the alias, rather than the actual remote for the project.  This
breaks sync'ing for scenarios where the alias doesn't have the same
repos available as the remote, plus just plain is wrong.

Change-Id: I7150e449341ed8655d398956a095261978d95870
2013-04-03 20:54:49 +00:00
David Pursehouse
4eb285cf90 Fix: local manifest deprecation warning appears more than once
When running repo sync, the local_manifest.xml deprecation warning
is shown twice.

Add a flag to ensure that it is only displayed once.

Change-Id: Icfa2b0b6249c037c29771f9860252e6eda3ae651
2013-02-17 21:23:33 +09:00
David Pursehouse
5f434ed723 Exit with fatal error if local manifest file cannot be parsed
If the .repo/local_manifests folder includes a local manifest file
that cannot be parsed, the current behaviour is to catch the parse
exception, print a warning, and continue to process remaining files.

This can cause any errors to go unnoticed.

Remove the exception handling, so that the exception is instead
caught in main._Main, and repo exits with a fatal error.

Change-Id: I75a70b7b850d2eb3e4ac99d435a4568ff598b7f4
2013-02-17 21:20:20 +09:00
David Pursehouse
606eab8043 Show full path of local_manifests folder in deprecation warning
When a local_manifest.xml file is present, a deprecation warning
is printed telling the user to put local manifest files in the
`local_manifests` directory.

Include the full path to the `local_manifests` directory in the
warning, to reduce confusion about where the directory should be
located.  Also enclose the directory name in backticks.

Change-Id: I85710cfbd6e77fb2fa6b7b0ce66d77693ccd649f
2013-02-17 21:19:58 +09:00
Shawn Pearce
4baf87f92c Merge "should use os.path.lexist instead of os.path.exist" 2013-01-28 16:25:55 +00:00
Joe Onorato
26e2475a0f If parsing the manifests fails, reset the XmlManifest object
so that if it's called again, we see the correct errors.

Change-Id: I909488feeac04aecfc92a9b5d6fb17827ef2f213
2013-01-17 09:10:00 -08:00
Conley Owens
c59bafffb9 Merge "Pass full path of the XML in local_manifests to the parser" 2013-01-17 17:06:49 +00:00
Conley Owens
0290cad5db Merge "Make -notdefault a default manifest group" 2013-01-14 22:34:13 +00:00
Tobias Droste
1a5c774cbf Pass full path of the XML in local_manifests to the parser
This fixes the following python error message if the
current working directory is not .repo/local_manifests:
IOError: [Errno 2] No such file or directory: 'local_manifest.xml'

Signed-off-by: Tobias Droste <tdroste87@gmail.com>
CC: David Pursehouse <david.pursehouse@sonymobile.com>
Change-Id: I4668dc04219b6233c7ff6ca3b4138bec9ee3529f
2013-01-07 07:46:18 +01:00
Shawn Pearce
a9f11b3cb2 Support resolving relative fetch URLs on persistent-https://
Some versions of Python will only attempt to resolve a relative
URL if they understand the URL scheme. Convert persistent-http://
and persistent-https:// schemes to the more typical http:// and
https:// versions for the resolve call.

Change-Id: I99072d5a69be8cfaa429a3ab177ba644d928ffba
2013-01-02 15:41:57 -08:00
Matt Gumbel
0c635bb427 Make -notdefault a default manifest group
When trying to render manifest with SHAs, projects in group notdefault
caused the following crash:

    Traceback (most recent call last):
      File ".repo/repo/main.py", line 385, in <module>
        _Main(sys.argv[1:])
      File ".repo/repo/main.py", line 365, in _Main
        result = repo._Run(argv) or 0
      File ".repo/repo/main.py", line 137, in _Run
        result = cmd.Execute(copts, cargs)
      File ".repo/repo/subcmds/manifest.py", line 129, in Execute
        self._Output(opt, manifest)
      File ".repo/repo/subcmds/manifest.py", line 79, in _Output
        peg_rev = opt.peg_rev)
      File ".repo/repo/manifest_xml.py", line 199, in Save
        p.work_git.rev_parse(HEAD + '^0'))
      File ".repo/repo/project.py", line 2035, in runner
        capture_stderr = True)
      File ".repo/repo/git_command.py", line 215, in __init__
        raise GitError('%s: %s' % (command[1], e))
    error.GitError: rev-parse: [Errno 2] No such file or directory: 'prebuilts/eclipse-build-deps'

This patch resolves the issue by making sure that -notdefault is always
used as a default manifest group so that notdefault projects are not
rendered out by the manifest subcmd.

Change-Id: I4a8bd18ea7600309f39ceff1b1ab6e1ff3adf21d
Signed-off-by: Matt Gumbel <matthew.k.gumbel@intel.com>
2012-12-21 10:14:53 -08:00
Sebastian Frias
223bf963f0 should use os.path.lexist instead of os.path.exist
The logic of the program requires a check on the existence of the
link itself

See repo  issue #125  :
        https://code.google.com/p/git-repo/issues/detail?id=125

Change-Id: Ia7300d22d6d656259f47c539febf1597f0c35538
2012-11-23 10:53:12 +01:00
Che-Liang Chiou
b2bd91c99b Represent git-submodule as nested projects, take 2
(Previous submission of this change broke Android buildbot due to
 incorrect regular expression for parsing git-config output.  During
 investigation, we also found that Android, which pulls Chromium, has a
 workaround for Chromium's submodules; its manifest includes Chromium's
 submodules.  This new change, in addition to fixing the regex, also
 take this type of workarounds into consideration; it adds a new
 attribute that makes repo not fetch submodules unless submodules have a
 project element defined in the manifest, or this attribute is
 overridden by a parent project element or by the default element.)

We need a representation of git-submodule in repo; otherwise repo will
not sync submodules, and leave workspace in a broken state.  Of course
this will not be a problem if all projects are owned by the owner of the
manifest file, who may simply choose not to use git-submodule in all
projects.  However, this is not possible in practice because manifest
file owner is unlikely to own all upstream projects.

As git submodules are simply git repositories, it is natural to treat
them as plain repo projects that live inside a repo project.  That is,
we could use recursively declared projects to denote the is-submodule
relation of git repositories.

The behavior of repo remains the same to projects that do not have a
sub-project within.  As for parent projects, repo fetches them and their
sub-projects as normal projects, and then checks out subprojects at the
commit specified in parent's commit object.  The sub-project is fetched
at a path relative to parent project's working directory; so the path
specified in manifest file should match that of .gitmodules file.

If a submodule is not registered in repo manifest, repo will derive its
properties from itself and its parent project, which might not always be
correct.  In such cases, the subproject is called a derived subproject.

To a user, a sub-project is merely a git-submodule; so all tips of
working with a git-submodule apply here, too.  For example, you should
not run `repo sync` in a parent repository if its submodule is dirty.

Change-Id: I4b8344c1b9ccad2f58ad304573133e5d52e1faef
2012-11-19 10:45:21 -08:00
David Pursehouse
f91074881f Better error message if 'remove-project' refers to non-existent project
If a local manifest includes a 'remove-project' element that refers to
a project that does not exist in the manifest, the error message is a
bit cryptic.

Change the error message to make it clearer what is wrong.

Change-Id: I0b1043aaec87893c3128211d3a9ab2db6d600755
2012-11-16 19:12:55 +09:00
David Pursehouse
c1b86a2323 Fix inconsistent indentation
The repo coding style is to indent at 2 characters, but there are
many places where this is not followed.

Enable pylint warning "W0311: Bad indentation" and make sure all
indentation is at multiples of 2 characters.

Change-Id: I68f0f64470789ce2429ab11104d15d380a63e6a8
2012-11-14 11:38:57 +09:00
Sarah Owens
cecd1d864f Change print statements to work in python3
This is part of a series of changes to introduce Python3 support.

Change-Id: I373be5de7141aa127d7debdbce1df39148dbec32
2012-11-13 17:33:56 -08:00
David Pursehouse
52f1e5d911 Make load order of local manifests deterministic
Local manifest files stored in the local_manifests folder are loaded
in alphabetical order, so it's easier to know in which order project
removals/additions/modifications will be applied.

If local_manifests.xml exists, it will be loaded before the files in
local_manifests.

Change-Id: Ia5c0349608f1823b4662cd6b340b99915bd973d5
2012-11-14 05:05:32 +09:00
David Pursehouse
717ece9d81 Better handling of duplicate remotes
In the current implementation, an error is raised if a remote with the
same name is defined more than once.  The check is only that the remote
has the same name as an existing remote.

With the support for multiple local manifests, it is more likely than
before that the same remote is defined in more than one manifest.

Change the check so that it only raises an error if a remote is defined
more than once with the same name, but different attributes.

Change-Id: Ic3608646cf9f40aa2bea7015d3ecd099c5f5f835
2012-11-13 09:35:37 +09:00
David Pursehouse
5566ae5dde Print deprecation warning when local_manifest.xml is used
The preferred way to specify local manifests is to drop the file(s)
in the local_manifests folder.  Print a deprecation warning when
the legacy local_manifest.xml file is used.

Change-Id: Ice85bd06fb612d6fcceeaa0755efd130556c4464
2012-11-13 08:19:51 +09:00
David Pursehouse
2d5a0df798 Add support for multiple local manifests
Add support for multiple local manifests stored in the local_manifests
folder under the .repo home directory.

Local manifests will be processed in addition to local_manifest.xml.

Change-Id: Ia0569cea7e9ae0fe3208a8ffef5d9679e14db03b
2012-11-13 08:19:51 +09:00
David Pursehouse
f7fc8a95be Handle XML errors when parsing the manifest
Catch ExpatError and exit gracefully with an error message, rather
than exiting with a python traceback.

Change-Id: Ifd0a7762aab4e8de63dab8a66117170a05586866
2012-11-13 05:53:41 +09:00
David Pursehouse
1d947b3034 Even more coding style cleanup
Fixing some more pylint warnings:

W1401: Anomalous backslash in string
W0623: Redefining name 'name' from outer scope
W0702: No exception type(s) specified
E0102: name: function already defined line n

Change-Id: I5afcdb4771ce210390a79981937806e30900a93c
2012-10-30 10:28:20 +09:00
Shawn O. Pearce
cd81dd6403 Revert "Represent git-submodule as nested projects"
This reverts commit 69998b0c6f.
Broke Android's non-gitmodule use case.

Conflicts:
	project.py
	subcmds/sync.py

Change-Id: I68ceeb63d8ee3b939f85a64736bdc81dfa352aed
2012-10-26 12:24:57 -07:00
Conley Owens
608aff7f62 Merge "Use modern Python exception syntax" 2012-10-25 10:03:37 -07:00
Sarah Owens
a5be53f9c8 Use modern Python exception syntax
"except Exception as e" instead of "except Exception, e"

This is part of a transition to supporting Python 3.  Python >= 2.6
support "as" syntax.

Note: this removes Python 2.5 support.

Change-Id: I309599f3981bba2b46111c43102bee38ff132803
2012-10-23 21:35:59 -07:00
David Pursehouse
4f7bdea9d2 Add pylint configuration and instructions
pylint configuration file (.pylintrc) is added, and submission
instructions are updated to include pylint usage steps.

Deprecated pylint suppression (`disable-msg`) is updated in a few
modules to make it work properly with the latest version (0.26).

Change-Id: I4ec2ef318e23557a374ecdbf40fe12645766830c
2012-10-24 10:18:13 +09:00
Che-Liang Chiou
69998b0c6f Represent git-submodule as nested projects
We need a representation of git-submodule in repo; otherwise repo will
not sync submodules, and leave workspace in a broken state.  Of course
this will not be a problem if all projects are owned by the owner of the
manifest file, who may simply choose not to use git-submodule in all
projects.  However, this is not possible in practice because manifest
file owner is unlikely to own all upstream projects.

As git submodules are simply git repositories, it is natural to treat
them as plain repo projects that live inside a repo project.  That is,
we could use recursively declared projects to denote the is-submodule
relation of git repositories.

The behavior of repo remains the same to projects that do not have a
sub-project within.  As for parent projects, repo fetches them and their
sub-projects as normal projects, and then checks out subprojects at the
commit specified in parent's commit object.  The sub-project is fetched
at a path relative to parent project's working directory; so the path
specified in manifest file should match that of .gitmodules file.

If a submodule is not registered in repo manifest, repo will derive its
properties from itself and its parent project, which might not always be
correct.  In such cases, the subproject is called a derived subproject.

To a user, a sub-project is merely a git-submodule; so all tips of
working with a git-submodule apply here, too.  For example, you should
not run `repo sync` in a parent repository if its submodule is dirty.

Change-Id: I541e9e2ac1a70304272dbe09724572aa1004eb5c
2012-10-23 16:08:58 -07:00
David Pursehouse
5c6eeac8f0 More coding style cleanup
Fixing more issues found with pylint.  Some that were supposed to
have been fixed in the previous sweep (Ie0db839e) but were missed:

C0321: More than one statement on a single line
W0622: Redefining built-in 'name'

And some more:

W0631: Using possibly undefined loop variable 'name'
W0223: Method 'name' is abstract in class 'name' but is not overridden
W0231: __init__ method from base class 'name' is not called

Change-Id: Ie119183708609d6279e973057a385fde864230c3
2012-10-22 12:30:14 +09:00
David Pursehouse
8a68ff9605 Coding style cleanup
Fix the following issues reported by pylint:

C0321: More than one statement on a single line
W0622: Redefining built-in 'name'
W0612: Unused variable 'name'
W0613: Unused argument 'name'
W0102: Dangerous default value 'value' as argument
W0105: String statement has no effect

Also fixed a few cases of inconsistent indentation.

Change-Id: Ie0db839e7c57d576cff12d8c055fe87030d00744
2012-10-09 12:45:30 +02:00
Conley Owens
3ff9decfd4 Merge "manifest: record the original revision when in -r mode." 2012-10-03 16:49:12 -07:00
Brian Harring
14a6674e32 manifest: record the original revision when in -r mode.
Currently when doing a sync against a revision locked manifest,
sync has no option but to fall back to sync'ing the entire refs space;
it doesn't know which ref to ask for that contains the sha1 it wants.

This sucks if we're in -c mode; thus when we generate a revision
locked manifest, record the originating branch- and try syncing that
branch first.  If the sha1 is found within that branch, this saves
us having to pull down the rest of the repo- a potentially heavy
saving.

If that branch doesn't have the desired sha1, we fallback to sync'ing
everything.

Change-Id: I99a5e44fa1d792dfcada76956a2363187df94cf1
2012-09-28 22:31:27 -07:00
David Holmer
9779565abf Fix incorrect default_groups when parsing projects from XML manifest
Change Details:
* Switch first default group to 'all' instead of 'default'

Change Benefits:
* More consistent with default_groups in the counterpart Save() function
* Fixes bug where command 'repo manifest' added an extra 'default'
  group to every output project element groups attribute. This bug was
  particularly confusing for projects which had 'groups="notdefault"'
  as they were output as 'groups="notdefault,default"' by 'repo manifest'

Change-Id: I5611c027a982d3394899466248b971910bec8c6b
2012-09-26 01:58:48 -04:00
David Pursehouse
e00aa6b923 Clean up imports
manifest_xml: import `HEAD` and `R_HEADS` from correct module
version: import `HEAD` from correct module

`HEAD` and `R_HEADS` should be imported from the git_refs module,
where they are originally defined, rather than from the project
module.

repo: remove unused import of readline

cherry_pick: import standard modules on separate lines
smartsync: import subcmd modules explicitly from subcmd

Use:
  `import re
  import sys`
and
  `from subcmds.sync import Sync`

Instead of:
  `import sys, re`
and
  `from sync import Sync`

Change-Id: Ie10dd6832710939634c4f5c86b9ba5a9cd6fc92e
2012-09-18 09:54:57 +02:00
Jooncheol Park
34acdd2534 Fix ManifestParseError when first child node is comment
If the first line of manifest.xml is a XML comment, root.childNodes[0]
is not a 'manifest' element node. The python minidom module will makes
a 'Comment' node as root.childNodes[0]. Since the original code only
checks whether the first child node is 'manifest', it couldn't do any
command including 'sync' due to the 'ManifestParseError' exception. This
patch could allow the comments between '<?xml ...?>' and '<manifest>' in
the manifest.xml file.

Change-Id: I0b81dea4f806965eca90f704c8aa7df49c579402
2012-09-07 08:38:08 -07:00
Conley Owens
bb1b5f5f86 Allow projects to be specified as notdefault
Instead of every group being in the group "default", every project
is now in the group "all".   A group that should not be downloaded
by default may be added to the group "notdefault".

This allows all group names to be positive (instead of removing groups
directly in the manifest with -default) and offers a clear way of
selecting every project (--groups all).

Change-Id: I99cd70309adb1f8460db3bbc6eff46bdcd22256f
2012-09-05 11:46:48 -07:00
David Pursehouse
e15c65abc2 Remove unused imports
There are several imports that are not used.  Remove them.

Change-Id: I2ac3be66827bd68d3faedcef7d6bbf30ea01d3f2
2012-08-23 12:15:26 +02:00
Dmitry Fink
17f85eab24 Omit all default groups when generating a manifest
One of the recent changes introduced implicit path:xxx and name:xxx groups
to every project, however they are not being stripped when generating
a manifest using "repo manifest" command resulting in clutter

Change-Id: Iec8610ba794b2fe4a6cdf0f59ca561595b66f9b5
2012-08-07 11:42:54 -07:00
Shawn O. Pearce
f35b2d9c31 Fix mirror mode
Change-Id: Ica0e8392562a7ae5aad7e45441c1540e5e2b0238
2012-08-02 11:46:22 -07:00
Yestin Sun
b292b98c3e Add remote alias support in manifest
The `alias` is an optional attribute in element `remote`. It can be
used to override attibute `name` to be set as the remote name in each
project's .git/config. Its value can be duplicated while attribute
`name` has to be unique across the manifest file. This helps each
project to be able to have same remote name which actually points
to different remote url.

It eases some automation scripts to be able to checkout/push to same
remote name but actually different remote url, like:

repo forall -c "git checkout -b work same_remote/work"
repo forall -c "git push same_remote work:work"

for example:
The manifest with 'alias' will look like:

<?xml version='1.0' encoding='UTF-8'?>
<manifest>
  <remote alias="same_alias" fetch="git://git.external1.org/" name="ext1"
      review="http://review.external1.org"/>
  <remote alias="same_alias" fetch="git://git.external2.org/" name="ext2"
      review="http://review.external2.org"/>
  <remote alias="same_alias" fetch="ssh://git.internal.com:29418" name="int"
      review="http://review.internal.com"/>
  <default remote="int" revision="int-branch" sync-j="2"/>
  <project name="path/to/project1" path="project1" remote="ext1"/>
  <project name="path/to/project2" path="project2" remote="ext2"/>
  <project name="path/to/project3" path="project3"/>
  ...
</manifest>

In each project, use command "git remote -v"

project1:
same_alias  git://git.external1.org/project1 (fetch)
same_alias  git://git.external1.org/project1 (push)

project2:
same_alias  git://git.external2.org/project2 (fetch)
same_alias  git://git.external2.org/project2 (push)

project3:
same_alias  ssh://git.internal.com:29418/project3 (fetch)
same_alias  ssh://git.internal.com:29418/project3 (push)

Change-Id: I2c48263097ff107f0c978f3e83966ae71d06cb90
2012-07-31 22:13:13 -07:00
Brian Harring
7da1314e38 Inject the project name into each projects groups.
For CrOS, we have scenarios were people checkout a smaller version
of our manifest via groups, and enable individual repositories as
needed for their work.  Previously this was via local_manifest
manipulation, which breaks via manifest-groups would require a
remove-project tag.

Via injecting the projects name into the projects groups, this
allows us to instead manipulate the configured groups allowing
the user to turn on/off projects as necessary.

Change-Id: I07b7918e16cc9dc28eb47e19a46a04dc4fd0be74
2012-07-31 22:05:44 -07:00
Brian Harring
475a47d531 Restore include support.
Calculation of where the include file lives was broken by 23acdd3f14
since it resulted in looking for the first include in .repo, rather
than .repo/manifests.

While people can work around it via setting their includes to
manifests/<include-target>, that breaks down since each layer of
includes would then have to be relative.

As such, restore the behaviour back to 2644874d; manifests includes
are calculated relative to the manifest root (ie, .repo/manifests);
local manifests includes are calculated relative to .repo/ .

Change-Id: I74c19ba614c41d2f08cd3e9fd094f3c510e3bfd1
2012-06-07 20:19:04 -07:00
Colin Cross
23acdd3f14 Parse manifest and local_manifest together
Combine manifest and local_manifest into a single list of elements
before parsing.  This will allow elements in the local_manifest to
affect elements in the main manifest.

Change-Id: I4d34c9260b299a76be2960b07c0c3fe1af35f33c
2012-05-24 09:32:15 -07:00
Brian Harring
2644874d9d ManifestXml: add include support
Having the ability to include other manifests is a very practical feature
to ease the managment of manifest. It allows to divide a manifest into separate
files, and create different environment depending  on what we want to release

You can have unlimited recursion of include, the manifest configs will simply be concatenated
as if it was in a single file.

command "repo manifest" will create a single manifest, and not recreate the manifest hierarchy

for example:
Our developement manifest will look like:

<?xml version='1.0' encoding='UTF-8'?>
<manifest>
  <default revision="platform/android/main" remote="intel"/>
  <include name="server.xml"/> <!-- The Server configuration -->
  <include name="aosp.xml" />  <!-- All the AOSP projects -->
  <include name="bsp.xml" />   <!-- The BSP projects that we release in source form -->
  <include name="bsp-priv.xml" /> <!-- The source of the BSP projects we release in binary form -->
</manifest>

Our release manifest will look like:

<?xml version='1.0' encoding='UTF-8'?>
<manifest>
  <default revision="platform/android/release-ext" remote="intel"/>
  <include name="server.xml"/> <!-- The Server configuration -->
  <include name="aosp.xml" />  <!-- All the AOSP projects -->
  <include name="bsp.xml" />   <!-- The BSP projects that we release in source form -->
  <include name="bsp-ext.xml" /> <!-- The PREBUILT version of the BSP projects we release in binary form -->
</manifest>

And it is also easy to create and maintain feature branch with a manifest that looks like:

<?xml version='1.0' encoding='UTF-8'?>
<manifest>
  <default revision="feature_branch_foobar" remote="intel"/>
  <include name="server.xml"/> <!-- The Server configuration -->
  <include name="aosp.xml" />  <!-- All the AOSP projects -->
  <include name="bsp.xml" />   <!-- The BSP projects that we release in source form -->
  <include name="bsp-priv.xml" /> <!-- The source of the BSP projects we release in binary form -->
</manifest>

Signed-off-by: Brian Harring <brian.harring@intel.com>
Signed-off-by: Pierre Tardy <pierre.tardy@intel.com>
Change-Id: I833a30d303039e485888768e6b81561b7665e89d
2012-05-24 09:07:24 -07:00
Anatol Pomazau
79770d269e Add sync-c option to manifest
There are use-cases when fetching all branch is impractical and
we really need to fetch only one branch/tag.
e.g. there is a large project with binaries and every update of a
binary file is put to a separate branch.
The whole project history might be too large to allow users fetch it.

Add 'sync-c' option to 'project' and 'default' tags to make it possible
to configure 'sync-c' behavior at per-project and per-manifest level.

Note that currently there is no possibility to revert boolean flag from
command line. If 'sync-c' is set in manifest then you cannot make
full fetch by providing a repo tool argument.

Change-Id: Ie36fe5737304930493740370239403986590f593
2012-04-23 14:10:52 -07:00
Colin Cross
c39864f5e1 Treat groups= as default
Previous incarnations of groups support left "groups=" in the
repo .config, which is now treated as "delete all the projects".
Treat empty groups configuration the same as no groups
configuration.

Change-Id: I57dab8dac55bdbf4cc181e2748cd2e4e510764f5
2012-04-23 13:43:41 -07:00
Conley Owens
971de8ea7b Refine groups functionality
Every project is in group "default".  "-default" does not remove
it from this project.  All group names specified in the manifest
are positive names as opposed to a mix of negative and positive.

Specified groups are resolved in order.  If init is supplied with
--groups="group1,-group2", the following describes the project
selection when syncing:

  * all projects in "group1" will be added, and
  * all projects in "group2" will be removed.

Change-Id: I1df3dcdb64bbd4cd80d675f9b2d3becbf721f661
2012-04-23 12:39:05 -07:00
James W. Mills
24c1308840 Add project annotation handling to repo
Allow the optional addition of "annotation" nodes nested under
projects.  Each annotation node must have "name" and "value"
attributes.  These name/value pairs will be exported into the
environment during any forall command, prefixed with "REPO__"

In addition, an optional "keep" attribute with case insensitive "true"
or "false" values can be included to determine whether the annotation
will be exported with 'repo manifest'

Change-Id: Icd7540afaae02c958f769ce3d25661aa721a9de8
Signed-off-by: James W. Mills <jameswmills@gmail.com>
2012-04-23 12:35:08 -07:00
Colin Cross
5acde75e5d Add manifest groups
Allows specifying a list of groups with a -g argument to repo init.
The groups act on a group= attribute specified on projects in the
manifest.
All projects are implicitly labelled with "default" unless they are
explicitly labelled "-default".
Prefixing a group with "-" removes matching projects from the list
of projects to sync.
If any non-inverted manifest groups are specified, the default label
is ignored.

Change-Id: I3a0dd7a93a8a1756205de1d03eee8c00906af0e5
Reviewed-on: https://gerrit-review.googlesource.com/34570
Reviewed-by: Shawn Pearce <sop@google.com>
Tested-by: Shawn Pearce <sop@google.com>
2012-04-13 09:46:00 -07:00
Mike Pontillo
d315382572 Add 'rebase="false"' attribute to the <project/> XML.
This new attribute can prevent 'repo sync' from automatically rebasing.

I hit a situation in where one of the git repositories I was tracking
was actually an external repository that I wanted to pull commits
into and merge myself. (NOT rebase, since that would lose the merge
history.) In this case, I'm not using 'repo upload', I'm manually
managing the merges to and from this repository.

Everything was going great until I typed 'repo sync' and it rebased
my manually-merged tree. Hence the option to skip it.

Change-Id: I965e0dd1acb87f4a56752ebedc7e2de1c502dbf8
2012-03-12 12:24:22 -07:00
Shawn O. Pearce
34fb20f67c Revert "Default repo manifest settings in git config"
This reverts commit ee1c2f5717.

This breaks a lot of buildbot systems. Rolling it back for now
until we can understand what the breakage was and how to fix it.
2011-11-30 13:41:02 -08:00
Victor Boivie
ee1c2f5717 Default repo manifest settings in git config
A default manifest URL can be specified using:
  git config --global repo-manifest.<id>.url <url>

A default manifest server can be specified using:
  git config --global repo-manifest.<id>.server <url>

A default git mirror reference can be specified using:
  git config --global repo-manifest.<id>.reference <path>

This will allow the user to use 'repo init -u <id>' as
a shorter alternative to specifying the full URL.

Also, manifest server will not have to be specified in the
manifest XML and the reference will not have to be specified
on the command line. If they are, they will override these
default values however.

Change-Id: Ifdbc160bd5909ec7df9efb0c5d7136f1d9351754
Signed-off-by: Victor Boivie <victor.boivie@sonyericsson.com>
2011-11-29 14:24:58 -08:00
Conley Owens
9d8f914fe8 Remove extra '/' in RemoteSpec
urljoin appends a '/' if only the domain is in the url path.  This
change strips that off before creating a RemoteSpec
2011-11-03 13:05:14 -07:00
Conley Owens
ceea368e88 Correctly name projects when mirroring
A bug introduced by relative urls caused projects such as manifest.git
to be placed in the root directory instead of the directory they should
by in.

This fix creates and refers to a resolvedFetchUrl in the _XmlRemote
class in order to get a fetchUrl that is never relative.
2011-10-20 11:01:38 -07:00
Conley Owens
db728cd866 Allow remote url to be relative to manifst url 2011-09-28 10:07:01 -07:00
Shawn O. Pearce
6392c87945 sync: Allow -j to have a default in manifest
This permits manifest authors to suggest a number of parallel
fetch operations against a remote server. For example, Gerrit
Code Review servers support queuing of requests and processes
them in first-in, first-out order. Running concurrent fetches
can utilize multiple CPUs on the Gerrit server, but will also
decrease overall operation latency by having the request put
into the queue ready to execute as soon as a CPU is free.

Change-Id: I3d3904acb6f63516bae4b071c510ad57a2afab18
Signed-off-by: Shawn O. Pearce <sop@google.com>
2011-09-22 18:08:27 -07:00
Doug Anderson
37282b4b9c Support repo-level pre-upload hook and prep for future hooks.
All repo-level hooks are expected to live in a single project at the
top level of that project.  The name of the hooks project is provided
in the manifest.xml.  The manifest also lists which hooks are enabled
to make it obvious if a file somehow failed to sync down (or got
deleted).

Before running any hook, we will prompt the user to make sure that it
is OK.  A user can deny running the hook, allow once, or allow
"forever" (until hooks change).  This tries to keep with the git
spirit of not automatically running anything on the user's computer
that got synced down.  Note that individual repo commands can add
always options to avoid these prompts as they see fit (see below for
the 'upload' options).

When hooks are run, they are loaded into the current interpreter (the
one running repo) and their main() function is run.  This mechanism is
used (instead of using subprocess) to make it easier to expand to a
richer hook interface in the future.  During loading, the
interpreter's sys.path is updated to contain the directory containing
the hooks so that hooks can be split into multiple files.

The upload command has two options that control hook behavior:
  - no-verify=False, verify=False (DEFAULT):
    If stdout is a tty, can prompt about running upload hooks if needed.
    If user denies running hooks, the upload is cancelled.  If stdout is
    not a tty and we would need to prompt about upload hooks, upload is
    cancelled.
  - no-verify=False, verify=True:
    Always run upload hooks with no prompt.
  - no-verify=True, verify=False:
    Never run upload hooks, but upload anyway (AKA bypass hooks).
  - no-verify=True, verify=True:
    Invalid

Sample bit of manifest.xml code for enabling hooks (assumes you have a
project named 'hooks' where hooks are stored):
  <repo-hooks in-project="hooks" enabled-list="pre-upload" />

Sample main() function in pre-upload.py in hooks directory:
  def main(project_list, **kwargs):
    print ('These projects will be uploaded: %s' %
           ', '.join(project_list))
    print ('I am being a good boy and ignoring anything in kwargs\n'
           'that I don\'t understand.')
    print 'I fail 50% of the time.  How flaky.'
    if random.random() <= .5:
      raise Exception('Pre-upload hook failed.  Have a nice day.')

Change-Id: I5cefa2cd5865c72589263cf8e2f152a43c122f70
2011-03-11 11:53:23 -08:00
Anthony Newnam
df14a70c45 Make path references OS independent
Change-Id: I5573995adfd52fd54bddc62d1d1ea78fb1328130
(cherry picked from commit b0f9a02394)

Conflicts:

	command.py
2011-01-09 17:39:19 -08:00
Doug Anderson
2b8db3ce3e Added feature to print a <notice> from manifest at the end of a sync.
This feature is used to convey information on a when a branch has
ceased development or if it is an experimental branch with a few
gotchas, etc.

You add it to your manifest XML by doing something like this:
<manifest>
  <notice>
    NOTE TO DEVELOPERS:
      If you checkin code, you have to pinky-swear that it contains no bugs.
      Anyone who breaks their promise will have tomatoes thrown at them in the
      team meeting.  Be sure to bring an extra set of clothes.
  </notice>

  <remote ... />
  ...
</manifest>

Carriage returns and indentation are relevant for the text in this tag.

This feature was requested by Anush Elangovan on the ChromiumOS team.
2010-11-01 15:08:06 -07:00
Nico Sallembien
a1bfd2cd72 Add a 'smart sync' option to repo sync
This option allows the user to specify a manifest server to use when
syncing. This manifest server will provide a manifest pegging each
project to a known green build. This allows developers to work on a
known good tree that is known to build and pass tests, preventing
failed builds to hamper productivity.

The manifest used is not "sticky" so as to allow subsequent
'repo sync' calls to sync to the tip of the tree.

Change-Id: Id0a24ece20f5a88034ad364b416a1dd2e394226d
2010-04-13 10:20:37 -07:00
Shawn O. Pearce
21c5c34ee2 Support detached HEAD in manifest repository
If the manifest repository is on a detached HEAD and we are parsing
an XML formatted manifest we should simply set the branch property
to None, rather than crash with an AttributeError.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-06-25 16:47:30 -07:00
Shawn O. Pearce
3c8dea1f8d Change project.revision to revisionExpr and revisionId
The revisionExpr field now holds an expression from the manifest,
such as "refs/heads/master", while revisionId holds the current
commit-ish SHA-1 of the revisionExpr.  Currently that is only
filled in if the manifest points directly to a SHA-1.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-05-29 18:45:20 -07:00
Shawn O. Pearce
d1f70d9929 Refactor how projects parse remotes so it can be replaced
We now feed Project a RemoteSpec, instead of the Remote directly
from the XmlManifest.  This way the RemoteSpec already has the
full project URL, rather than just the base, permitting other
types of manifests to produce the URL in their own style.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-05-29 09:31:28 -07:00
Shawn O. Pearce
c8a300f639 Refactor Manifest to be XmlManifest
We'll soon be supporting two different manifest formats, but we
can't immediately remove support for the current XML one that is
in wide spread use within Android.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-05-29 09:31:28 -07:00