Commit Graph

72 Commits

Author SHA1 Message Date
Conley Owens
bb1b5f5f86 Allow projects to be specified as notdefault
Instead of every group being in the group "default", every project
is now in the group "all".   A group that should not be downloaded
by default may be added to the group "notdefault".

This allows all group names to be positive (instead of removing groups
directly in the manifest with -default) and offers a clear way of
selecting every project (--groups all).

Change-Id: I99cd70309adb1f8460db3bbc6eff46bdcd22256f
2012-09-05 11:46:48 -07:00
David Pursehouse
9a27d0111d manifest-format.txt: Add documentation for GetManifest RPC method
Add documentation of the GetManifest RPC method in the
manifest-server section.

Change-Id: I5cda5929bc8a0ca9d3f2b9da63216427041d2823
2012-09-05 06:00:47 -07:00
David Pursehouse
daa851f6cd manifest-format.txt: Fix a couple of minor spelling mistakes
Change-Id: Ic2d266c8cf08827a71846db9d3711feb02885f01
2012-08-22 09:39:41 -07:00
Yestin Sun
b292b98c3e Add remote alias support in manifest
The `alias` is an optional attribute in element `remote`. It can be
used to override attibute `name` to be set as the remote name in each
project's .git/config. Its value can be duplicated while attribute
`name` has to be unique across the manifest file. This helps each
project to be able to have same remote name which actually points
to different remote url.

It eases some automation scripts to be able to checkout/push to same
remote name but actually different remote url, like:

repo forall -c "git checkout -b work same_remote/work"
repo forall -c "git push same_remote work:work"

for example:
The manifest with 'alias' will look like:

<?xml version='1.0' encoding='UTF-8'?>
<manifest>
  <remote alias="same_alias" fetch="git://git.external1.org/" name="ext1"
      review="http://review.external1.org"/>
  <remote alias="same_alias" fetch="git://git.external2.org/" name="ext2"
      review="http://review.external2.org"/>
  <remote alias="same_alias" fetch="ssh://git.internal.com:29418" name="int"
      review="http://review.internal.com"/>
  <default remote="int" revision="int-branch" sync-j="2"/>
  <project name="path/to/project1" path="project1" remote="ext1"/>
  <project name="path/to/project2" path="project2" remote="ext2"/>
  <project name="path/to/project3" path="project3"/>
  ...
</manifest>

In each project, use command "git remote -v"

project1:
same_alias  git://git.external1.org/project1 (fetch)
same_alias  git://git.external1.org/project1 (push)

project2:
same_alias  git://git.external2.org/project2 (fetch)
same_alias  git://git.external2.org/project2 (push)

project3:
same_alias  ssh://git.internal.com:29418/project3 (fetch)
same_alias  ssh://git.internal.com:29418/project3 (push)

Change-Id: I2c48263097ff107f0c978f3e83966ae71d06cb90
2012-07-31 22:13:13 -07:00
Brian Harring
7da1314e38 Inject the project name into each projects groups.
For CrOS, we have scenarios were people checkout a smaller version
of our manifest via groups, and enable individual repositories as
needed for their work.  Previously this was via local_manifest
manipulation, which breaks via manifest-groups would require a
remove-project tag.

Via injecting the projects name into the projects groups, this
allows us to instead manipulate the configured groups allowing
the user to turn on/off projects as necessary.

Change-Id: I07b7918e16cc9dc28eb47e19a46a04dc4fd0be74
2012-07-31 22:05:44 -07:00
Brian Harring
2644874d9d ManifestXml: add include support
Having the ability to include other manifests is a very practical feature
to ease the managment of manifest. It allows to divide a manifest into separate
files, and create different environment depending  on what we want to release

You can have unlimited recursion of include, the manifest configs will simply be concatenated
as if it was in a single file.

command "repo manifest" will create a single manifest, and not recreate the manifest hierarchy

for example:
Our developement manifest will look like:

<?xml version='1.0' encoding='UTF-8'?>
<manifest>
  <default revision="platform/android/main" remote="intel"/>
  <include name="server.xml"/> <!-- The Server configuration -->
  <include name="aosp.xml" />  <!-- All the AOSP projects -->
  <include name="bsp.xml" />   <!-- The BSP projects that we release in source form -->
  <include name="bsp-priv.xml" /> <!-- The source of the BSP projects we release in binary form -->
</manifest>

Our release manifest will look like:

<?xml version='1.0' encoding='UTF-8'?>
<manifest>
  <default revision="platform/android/release-ext" remote="intel"/>
  <include name="server.xml"/> <!-- The Server configuration -->
  <include name="aosp.xml" />  <!-- All the AOSP projects -->
  <include name="bsp.xml" />   <!-- The BSP projects that we release in source form -->
  <include name="bsp-ext.xml" /> <!-- The PREBUILT version of the BSP projects we release in binary form -->
</manifest>

And it is also easy to create and maintain feature branch with a manifest that looks like:

<?xml version='1.0' encoding='UTF-8'?>
<manifest>
  <default revision="feature_branch_foobar" remote="intel"/>
  <include name="server.xml"/> <!-- The Server configuration -->
  <include name="aosp.xml" />  <!-- All the AOSP projects -->
  <include name="bsp.xml" />   <!-- The BSP projects that we release in source form -->
  <include name="bsp-priv.xml" /> <!-- The source of the BSP projects we release in binary form -->
</manifest>

Signed-off-by: Brian Harring <brian.harring@intel.com>
Signed-off-by: Pierre Tardy <pierre.tardy@intel.com>
Change-Id: I833a30d303039e485888768e6b81561b7665e89d
2012-05-24 09:07:24 -07:00
Anatol Pomazau
79770d269e Add sync-c option to manifest
There are use-cases when fetching all branch is impractical and
we really need to fetch only one branch/tag.
e.g. there is a large project with binaries and every update of a
binary file is put to a separate branch.
The whole project history might be too large to allow users fetch it.

Add 'sync-c' option to 'project' and 'default' tags to make it possible
to configure 'sync-c' behavior at per-project and per-manifest level.

Note that currently there is no possibility to revert boolean flag from
command line. If 'sync-c' is set in manifest then you cannot make
full fetch by providing a repo tool argument.

Change-Id: Ie36fe5737304930493740370239403986590f593
2012-04-23 14:10:52 -07:00
Conley Owens
971de8ea7b Refine groups functionality
Every project is in group "default".  "-default" does not remove
it from this project.  All group names specified in the manifest
are positive names as opposed to a mix of negative and positive.

Specified groups are resolved in order.  If init is supplied with
--groups="group1,-group2", the following describes the project
selection when syncing:

  * all projects in "group1" will be added, and
  * all projects in "group2" will be removed.

Change-Id: I1df3dcdb64bbd4cd80d675f9b2d3becbf721f661
2012-04-23 12:39:05 -07:00
James W. Mills
24c1308840 Add project annotation handling to repo
Allow the optional addition of "annotation" nodes nested under
projects.  Each annotation node must have "name" and "value"
attributes.  These name/value pairs will be exported into the
environment during any forall command, prefixed with "REPO__"

In addition, an optional "keep" attribute with case insensitive "true"
or "false" values can be included to determine whether the annotation
will be exported with 'repo manifest'

Change-Id: Icd7540afaae02c958f769ce3d25661aa721a9de8
Signed-off-by: James W. Mills <jameswmills@gmail.com>
2012-04-23 12:35:08 -07:00
Colin Cross
5acde75e5d Add manifest groups
Allows specifying a list of groups with a -g argument to repo init.
The groups act on a group= attribute specified on projects in the
manifest.
All projects are implicitly labelled with "default" unless they are
explicitly labelled "-default".
Prefixing a group with "-" removes matching projects from the list
of projects to sync.
If any non-inverted manifest groups are specified, the default label
is ignored.

Change-Id: I3a0dd7a93a8a1756205de1d03eee8c00906af0e5
Reviewed-on: https://gerrit-review.googlesource.com/34570
Reviewed-by: Shawn Pearce <sop@google.com>
Tested-by: Shawn Pearce <sop@google.com>
2012-04-13 09:46:00 -07:00
Shawn O. Pearce
6392c87945 sync: Allow -j to have a default in manifest
This permits manifest authors to suggest a number of parallel
fetch operations against a remote server. For example, Gerrit
Code Review servers support queuing of requests and processes
them in first-in, first-out order. Running concurrent fetches
can utilize multiple CPUs on the Gerrit server, but will also
decrease overall operation latency by having the request put
into the queue ready to execute as soon as a CPU is free.

Change-Id: I3d3904acb6f63516bae4b071c510ad57a2afab18
Signed-off-by: Shawn O. Pearce <sop@google.com>
2011-09-22 18:08:27 -07:00
Doug Anderson
37282b4b9c Support repo-level pre-upload hook and prep for future hooks.
All repo-level hooks are expected to live in a single project at the
top level of that project.  The name of the hooks project is provided
in the manifest.xml.  The manifest also lists which hooks are enabled
to make it obvious if a file somehow failed to sync down (or got
deleted).

Before running any hook, we will prompt the user to make sure that it
is OK.  A user can deny running the hook, allow once, or allow
"forever" (until hooks change).  This tries to keep with the git
spirit of not automatically running anything on the user's computer
that got synced down.  Note that individual repo commands can add
always options to avoid these prompts as they see fit (see below for
the 'upload' options).

When hooks are run, they are loaded into the current interpreter (the
one running repo) and their main() function is run.  This mechanism is
used (instead of using subprocess) to make it easier to expand to a
richer hook interface in the future.  During loading, the
interpreter's sys.path is updated to contain the directory containing
the hooks so that hooks can be split into multiple files.

The upload command has two options that control hook behavior:
  - no-verify=False, verify=False (DEFAULT):
    If stdout is a tty, can prompt about running upload hooks if needed.
    If user denies running hooks, the upload is cancelled.  If stdout is
    not a tty and we would need to prompt about upload hooks, upload is
    cancelled.
  - no-verify=False, verify=True:
    Always run upload hooks with no prompt.
  - no-verify=True, verify=False:
    Never run upload hooks, but upload anyway (AKA bypass hooks).
  - no-verify=True, verify=True:
    Invalid

Sample bit of manifest.xml code for enabling hooks (assumes you have a
project named 'hooks' where hooks are stored):
  <repo-hooks in-project="hooks" enabled-list="pre-upload" />

Sample main() function in pre-upload.py in hooks directory:
  def main(project_list, **kwargs):
    print ('These projects will be uploaded: %s' %
           ', '.join(project_list))
    print ('I am being a good boy and ignoring anything in kwargs\n'
           'that I don\'t understand.')
    print 'I fail 50% of the time.  How flaky.'
    if random.random() <= .5:
      raise Exception('Pre-upload hook failed.  Have a nice day.')

Change-Id: I5cefa2cd5865c72589263cf8e2f152a43c122f70
2011-03-11 11:53:23 -08:00
Doug Anderson
2b8db3ce3e Added feature to print a <notice> from manifest at the end of a sync.
This feature is used to convey information on a when a branch has
ceased development or if it is an experimental branch with a few
gotchas, etc.

You add it to your manifest XML by doing something like this:
<manifest>
  <notice>
    NOTE TO DEVELOPERS:
      If you checkin code, you have to pinky-swear that it contains no bugs.
      Anyone who breaks their promise will have tomatoes thrown at them in the
      team meeting.  Be sure to bring an extra set of clothes.
  </notice>

  <remote ... />
  ...
</manifest>

Carriage returns and indentation are relevant for the text in this tag.

This feature was requested by Anush Elangovan on the ChromiumOS team.
2010-11-01 15:08:06 -07:00
Nico Sallembien
a1bfd2cd72 Add a 'smart sync' option to repo sync
This option allows the user to specify a manifest server to use when
syncing. This manifest server will provide a manifest pegging each
project to a known green build. This allows developers to work on a
known good tree that is known to build and pass tests, preventing
failed builds to hamper productivity.

The manifest used is not "sticky" so as to allow subsequent
'repo sync' calls to sync to the tip of the tree.

Change-Id: Id0a24ece20f5a88034ad364b416a1dd2e394226d
2010-04-13 10:20:37 -07:00
Shawn O. Pearce
242b52690d Remove support for the extra <remote> definitions in manifests
These aren't that widely used, and actually make it difficult for
users to fully mirror a forest of repositories, and then permit
someone else to clone off that forest, rather then the original
upstream servers.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-05-19 13:01:52 -07:00
Shawn O. Pearce
a490f03dc2 Correct note about local_manifest.xml capabilities
With the <remove-project> element we can remove projects, and
fully replace them with a different definition.  So this note
is out of date.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-18 11:25:58 -07:00
Shawn O. Pearce
43c3d9ea17 Add a 'repo manifest' command whose help is the manifest file format
This should make it easier for users to discover the file format
on their own, and read about it.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-03-04 14:26:50 -08:00
Shawn O. Pearce
03eaf07ec6 Support <remove-project name="X"> in manifest to remove/replace X
The manifest files now permit removing a project so the user can
either keep it out of their client, or replace it with a different
project using an entirely different configuration.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2008-11-20 11:54:46 -08:00
Shawn O. Pearce
70939e2f73 Add <add-remote to-project="..."> to inject additional remotes
This way users can add forks they know about to an existing project
that was already declared in the primary manifest.  This is mostly
useful with the Linux kernel project, where multiple forks is quite
common for the main upstream tree (e.g. Linus' tree), a platform
architecture tree (e.g. ARM) and a device specific tree (e.g. the
msm7k tree used by Android).

Signed-off-by: Shawn O. Pearce <sop@google.com>
2008-11-06 11:23:08 -08:00
Shawn O. Pearce
ae6e0949d1 Add <remote project-name="..."> attribute within projects
By setting a project-name on a remote nested within a project forks
of a project like the Linux kernel can be easily handled by fetching
all relevant forks into the same client side project under different
remote names.  Developers can create branches off different remotes
using `git checkout --track -b $myname $remote/$branch` and later
`repo upload` automatically redirects to the proper fork project
in the code review server.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2008-11-06 11:23:06 -08:00
Shawn O. Pearce
70cd4ab270 Add some short documentation about the local manifest
Signed-off-by: Shawn O. Pearce <sop@google.com>
2008-11-06 08:48:44 -08:00
Shawn O. Pearce
3e5481999d Add a basic outline of the repo manifest file format
Signed-off-by: Shawn O. Pearce <sop@google.com>
2008-11-04 11:19:36 -08:00